Add generate_audio file/format params; bring Piper TTS up with es_MX default
generate_audio gains output_path + format params (wav, mp3, ogg, flac, m4a). Non-wav formats route through ffmpeg (added to the Dockerfile). Files land under /output/ inside the container, bind-mounted from the host's TTS_OUTPUT_HOST_DIR (defaults to ~/mcspeak-out). Subdirectories are auto-created. Path validation rejects writes outside /output/ — both absolute escapes like /etc/passwd and traversal forms like ../../etc/passwd are canonicalized and checked against the mount root. Extension auto-corrects to match the requested format. piper-tts service added to compose using rhasspy/wyoming-piper. Default voice es_MX-ald-medium is pre-warmed at container start and persists in ./piper-data/ across recreates (~60 MB download on first run). Port 10200 published to localhost so host-side scripts can also reach Wyoming directly; mcspeak inside the stack uses container DNS. Voice configurable via TTS_PIPER_VOICE — the same env var threads through to both wyoming-piper's --voice flag and mcspeak's settings.piper_voice, so they stay in sync. PiperEngine.__init__ accepts a default_voice override; settings.piper_voice threads it from the env. list_engines now reports the configured default rather than the hardcoded en_US-lessac-medium. speak() / generate_audio() calls with engine=piper and no explicit voice use the configured one. mcspeak-internal network changed from internal:true to default-bridge. internal:true was overkill — the goal was per-stack DNS scoping (which compose provides automatically via the project-prefixed network name), not internet isolation. The latter broke piper-tts's HuggingFace voice download with "Name resolution failure" on first start. .gitignore: add piper-data/ so the downloaded voice models stay out of the repo.
This commit is contained in:
parent
ec4e1b6093
commit
27b4c4ae29
@ -1,6 +1,15 @@
|
|||||||
# Compose project namespace -- prevents collisions with other stacks
|
# Compose project namespace -- prevents collisions with other stacks
|
||||||
COMPOSE_PROJECT=mcspeak
|
COMPOSE_PROJECT=mcspeak
|
||||||
|
|
||||||
|
# Host directory for generate_audio output (output_path / format params).
|
||||||
|
# Mounted as /output inside the container. Defaults to ~/mcspeak-out when unset.
|
||||||
|
# TTS_OUTPUT_HOST_DIR=/home/you/audio-out
|
||||||
|
|
||||||
|
# Default Piper voice to pre-warm at piper-tts container start. Other voices
|
||||||
|
# requested via the speak() voice= param are downloaded on demand. Full list:
|
||||||
|
# https://github.com/rhasspy/piper/blob/master/VOICES.md
|
||||||
|
# TTS_PIPER_VOICE=es_MX-ald-medium
|
||||||
|
|
||||||
# Path to Orpheus GGUF model file (required by llama-server service)
|
# Path to Orpheus GGUF model file (required by llama-server service)
|
||||||
# Tip: if you've already pulled it via Ollama, find the blob with:
|
# Tip: if you've already pulled it via Ollama, find the blob with:
|
||||||
# ollama show --modelfile orpheus | grep FROM
|
# ollama show --modelfile orpheus | grep FROM
|
||||||
|
|||||||
1
.gitignore
vendored
1
.gitignore
vendored
@ -1,5 +1,6 @@
|
|||||||
# Models (large binary files)
|
# Models (large binary files)
|
||||||
models/
|
models/
|
||||||
|
piper-data/
|
||||||
*.onnx
|
*.onnx
|
||||||
*.bin
|
*.bin
|
||||||
|
|
||||||
|
|||||||
@ -2,9 +2,11 @@ FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim
|
|||||||
|
|
||||||
# PipeWire client tools — pw-play connects to the host's PipeWire socket
|
# PipeWire client tools — pw-play connects to the host's PipeWire socket
|
||||||
# pulseaudio-utils provides pactl for media ducking (volume control via PulseAudio compat)
|
# pulseaudio-utils provides pactl for media ducking (volume control via PulseAudio compat)
|
||||||
|
# ffmpeg handles WAV→{mp3,ogg,flac,m4a} conversion for generate_audio's format param
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
pipewire-bin \
|
pipewire-bin \
|
||||||
pulseaudio-utils \
|
pulseaudio-utils \
|
||||||
|
ffmpeg \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|||||||
2
Makefile
2
Makefile
@ -9,11 +9,13 @@ build:
|
|||||||
docker compose build
|
docker compose build
|
||||||
|
|
||||||
up: build
|
up: build
|
||||||
|
@mkdir -p $${TTS_OUTPUT_HOST_DIR:-$$HOME/mcspeak-out}
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
@sleep 2
|
@sleep 2
|
||||||
docker compose logs --tail 20
|
docker compose logs --tail 20
|
||||||
|
|
||||||
up-with-orpheus: build
|
up-with-orpheus: build
|
||||||
|
@mkdir -p $${TTS_OUTPUT_HOST_DIR:-$$HOME/mcspeak-out}
|
||||||
docker compose --profile with-orpheus up -d
|
docker compose --profile with-orpheus up -d
|
||||||
@sleep 2
|
@sleep 2
|
||||||
docker compose logs --tail 20
|
docker compose logs --tail 20
|
||||||
|
|||||||
@ -33,6 +33,9 @@ services:
|
|||||||
- /run/user/1000/pipewire-0:/run/user/1000/pipewire-0
|
- /run/user/1000/pipewire-0:/run/user/1000/pipewire-0
|
||||||
# PulseAudio compat socket for media ducking (volume control)
|
# PulseAudio compat socket for media ducking (volume control)
|
||||||
- /run/user/1000/pulse:/run/user/1000/pulse
|
- /run/user/1000/pulse:/run/user/1000/pulse
|
||||||
|
# Host-visible output dir for generate_audio's output_path/format params.
|
||||||
|
# Override TTS_OUTPUT_HOST_DIR in .env to use a different host directory.
|
||||||
|
- ${TTS_OUTPUT_HOST_DIR:-${HOME}/mcspeak-out}:/output
|
||||||
depends_on:
|
depends_on:
|
||||||
llama-server:
|
llama-server:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
@ -46,6 +49,27 @@ services:
|
|||||||
caddy: mctalkbox.l.supported.systems
|
caddy: mctalkbox.l.supported.systems
|
||||||
caddy.reverse_proxy: "{{upstreams 8371}}"
|
caddy.reverse_proxy: "{{upstreams 8371}}"
|
||||||
|
|
||||||
|
piper-tts:
|
||||||
|
# Default-on. Wyoming protocol TTS, ~500 MB image + ~60 MB voice download
|
||||||
|
# on first run (cached in ./piper-data thereafter). Voice is configurable
|
||||||
|
# via TTS_PIPER_VOICE; the wyoming-piper server will auto-download any
|
||||||
|
# voice mcspeak requests at speak() time even if it's not the default.
|
||||||
|
image: rhasspy/wyoming-piper:latest
|
||||||
|
container_name: piper-tts
|
||||||
|
restart: unless-stopped
|
||||||
|
command: --voice ${TTS_PIPER_VOICE:-es_MX-ald-medium}
|
||||||
|
# Published to localhost so host-side scripts can hit Wyoming directly
|
||||||
|
# at 127.0.0.1:10200 if needed. mcspeak inside the stack reaches it via
|
||||||
|
# the container DNS name (TTS_PIPER_HOST=piper-tts above).
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:10200:10200"
|
||||||
|
volumes:
|
||||||
|
# Host-visible voice cache so models survive container recreates and
|
||||||
|
# are inspectable from the host (~/claude/mctalkbox/piper-data/).
|
||||||
|
- ./piper-data:/data
|
||||||
|
networks:
|
||||||
|
- mcspeak-internal
|
||||||
|
|
||||||
llama-server:
|
llama-server:
|
||||||
# Opt-in: only starts when `docker compose --profile with-orpheus up`.
|
# Opt-in: only starts when `docker compose --profile with-orpheus up`.
|
||||||
# Default `make up` runs kokoro-only without GPU dependencies.
|
# Default `make up` runs kokoro-only without GPU dependencies.
|
||||||
@ -90,7 +114,9 @@ volumes:
|
|||||||
networks:
|
networks:
|
||||||
caddy:
|
caddy:
|
||||||
external: true
|
external: true
|
||||||
# Private per-stack network for mcspeak ↔ llama-server. Auto-created by
|
# Private per-stack network for mcspeak ↔ piper-tts ↔ llama-server.
|
||||||
# compose, isolated from other stacks (no DNS leak via shared caddy).
|
# Auto-created by compose, scoped to this project so other stacks can't
|
||||||
mcspeak-internal:
|
# resolve our service names. NOT marked internal:true because piper-tts
|
||||||
internal: true
|
# needs HuggingFace access to download voice models on first run; the
|
||||||
|
# per-project scoping is what we actually wanted, not internet isolation.
|
||||||
|
mcspeak-internal: {}
|
||||||
|
|||||||
@ -77,6 +77,57 @@ def wav_duration(path: Path) -> float:
|
|||||||
return wf.getnframes() / wf.getframerate()
|
return wf.getnframes() / wf.getframerate()
|
||||||
|
|
||||||
|
|
||||||
|
# Supported output formats for convert_audio. WAV is a passthrough copy;
|
||||||
|
# everything else routes through ffmpeg with format-appropriate codec args.
|
||||||
|
SUPPORTED_FORMATS = ("wav", "mp3", "ogg", "flac", "m4a")
|
||||||
|
|
||||||
|
_FFMPEG_CODEC_ARGS: dict[str, list[str]] = {
|
||||||
|
"mp3": ["-codec:a", "libmp3lame", "-qscale:a", "2"], # VBR ~190 kbps
|
||||||
|
"ogg": ["-codec:a", "libvorbis", "-qscale:a", "5"], # VBR ~160 kbps
|
||||||
|
"flac": ["-codec:a", "flac"], # lossless
|
||||||
|
"m4a": ["-codec:a", "aac", "-b:a", "192k"], # CBR 192 kbps AAC
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ConversionError(RuntimeError):
|
||||||
|
"""Raised when format conversion fails."""
|
||||||
|
|
||||||
|
|
||||||
|
async def convert_audio(src_wav: Path, dest: Path, fmt: str) -> Path:
|
||||||
|
"""Convert a source WAV to the requested format at dest.
|
||||||
|
|
||||||
|
`fmt == "wav"` shells out to a plain copy (no ffmpeg invocation).
|
||||||
|
Other formats use ffmpeg with codec args from _FFMPEG_CODEC_ARGS.
|
||||||
|
|
||||||
|
Raises ConversionError on ffmpeg non-zero exit or unsupported format.
|
||||||
|
"""
|
||||||
|
if fmt not in SUPPORTED_FORMATS:
|
||||||
|
raise ConversionError(
|
||||||
|
f"Unsupported format {fmt!r}. Supported: {list(SUPPORTED_FORMATS)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
if fmt == "wav":
|
||||||
|
shutil.copyfile(src_wav, dest)
|
||||||
|
return dest
|
||||||
|
|
||||||
|
proc = await asyncio.create_subprocess_exec(
|
||||||
|
"ffmpeg", "-y", "-loglevel", "error", "-i", str(src_wav),
|
||||||
|
*_FFMPEG_CODEC_ARGS[fmt],
|
||||||
|
str(dest),
|
||||||
|
stdout=asyncio.subprocess.DEVNULL,
|
||||||
|
stderr=asyncio.subprocess.PIPE,
|
||||||
|
)
|
||||||
|
_, stderr = await proc.communicate()
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise ConversionError(
|
||||||
|
f"ffmpeg failed ({proc.returncode}) converting to {fmt}: "
|
||||||
|
f"{stderr.decode(errors='replace').strip()[:500]}"
|
||||||
|
)
|
||||||
|
return dest
|
||||||
|
|
||||||
|
|
||||||
class PlaybackError(RuntimeError):
|
class PlaybackError(RuntimeError):
|
||||||
"""Raised when audio playback fails or times out."""
|
"""Raised when audio playback fails or times out."""
|
||||||
|
|
||||||
|
|||||||
@ -18,9 +18,14 @@ class PiperEngine(TTSEngine):
|
|||||||
name = "piper"
|
name = "piper"
|
||||||
default_voice = "en_US-lessac-medium"
|
default_voice = "en_US-lessac-medium"
|
||||||
|
|
||||||
def __init__(self, host: str, port: int) -> None:
|
def __init__(self, host: str, port: int, default_voice: str | None = None) -> None:
|
||||||
self._host = host
|
self._host = host
|
||||||
self._port = port
|
self._port = port
|
||||||
|
# Override the class-level default so list_engines reports the
|
||||||
|
# configured voice (synthesize() uses self.default_voice as the
|
||||||
|
# voice= fallback, see below).
|
||||||
|
if default_voice:
|
||||||
|
self.default_voice = default_voice
|
||||||
|
|
||||||
async def synthesize(self, text: str, voice: str | None = None) -> TTSResult:
|
async def synthesize(self, text: str, voice: str | None = None) -> TTSResult:
|
||||||
voice = voice or self.default_voice
|
voice = voice or self.default_voice
|
||||||
|
|||||||
@ -12,7 +12,7 @@ from fastmcp import Context, FastMCP
|
|||||||
from fastmcp.server.dependencies import CurrentContext
|
from fastmcp.server.dependencies import CurrentContext
|
||||||
from mcp.types import ToolAnnotations
|
from mcp.types import ToolAnnotations
|
||||||
|
|
||||||
from .audio import play_audio
|
from .audio import ConversionError, SUPPORTED_FORMATS, convert_audio, play_audio
|
||||||
from .engines.base import TTSEngine
|
from .engines.base import TTSEngine
|
||||||
from .engines.kokoro import KokoroEngine
|
from .engines.kokoro import KokoroEngine
|
||||||
from .engines.orpheus import OrpheusEngine
|
from .engines.orpheus import OrpheusEngine
|
||||||
@ -24,6 +24,48 @@ from .tones import generate_tones, resolve_tone
|
|||||||
from .voice_identity import VoiceIdentityCache, get_project_name, resolve_voice
|
from .voice_identity import VoiceIdentityCache, get_project_name, resolve_voice
|
||||||
|
|
||||||
ENGINE_NAMES = Literal["piper", "kokoro", "orpheus"]
|
ENGINE_NAMES = Literal["piper", "kokoro", "orpheus"]
|
||||||
|
FORMAT_NAMES = Literal["wav", "mp3", "ogg", "flac", "m4a"]
|
||||||
|
|
||||||
|
# Container-side bind mount for host-visible saved audio.
|
||||||
|
# Host path is configured via TTS_OUTPUT_HOST_DIR (default ~/mcspeak-out).
|
||||||
|
OUTPUT_DIR_CONTAINER = Path("/output")
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_output_path(output_path: str | None, fmt: str) -> Path:
|
||||||
|
"""Resolve a user-supplied output_path to an absolute path under /output/.
|
||||||
|
|
||||||
|
None → auto-named under /output/.
|
||||||
|
Relative → joined under /output/.
|
||||||
|
Absolute → must canonicalize under /output/ (rejects writes elsewhere
|
||||||
|
and path-traversal escapes via "..").
|
||||||
|
Extension auto-corrected to match `fmt` if mismatched.
|
||||||
|
"""
|
||||||
|
base = OUTPUT_DIR_CONTAINER
|
||||||
|
|
||||||
|
if output_path is None:
|
||||||
|
ts = time.strftime("%Y%m%d-%H%M%S")
|
||||||
|
return base / f"mcspeak-{ts}.{fmt}"
|
||||||
|
|
||||||
|
p = Path(output_path)
|
||||||
|
full = p if p.is_absolute() else base / p
|
||||||
|
|
||||||
|
# Canonicalize without requiring existence (resolve(strict=False) is default).
|
||||||
|
# Anchor against base.resolve() to also reject e.g. "/output/../etc/passwd".
|
||||||
|
base_resolved = base.resolve()
|
||||||
|
full_resolved = full.resolve()
|
||||||
|
try:
|
||||||
|
full_resolved.relative_to(base_resolved)
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError(
|
||||||
|
f"output_path must resolve under {base}/ (got {output_path!r}). "
|
||||||
|
f"Inside the container, {base}/ is bind-mounted from the host's "
|
||||||
|
f"TTS_OUTPUT_HOST_DIR. Use a relative path or one starting with {base}/."
|
||||||
|
)
|
||||||
|
|
||||||
|
expected_ext = f".{fmt}"
|
||||||
|
if full_resolved.suffix.lower() != expected_ext:
|
||||||
|
full_resolved = full_resolved.with_suffix(expected_ext)
|
||||||
|
return full_resolved
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@ -49,7 +91,10 @@ async def app_lifespan(server: FastMCP):
|
|||||||
|
|
||||||
# --- Build engines ---
|
# --- Build engines ---
|
||||||
engines: dict[str, TTSEngine] = {
|
engines: dict[str, TTSEngine] = {
|
||||||
"piper": PiperEngine(settings.piper_host, settings.piper_port),
|
"piper": PiperEngine(
|
||||||
|
settings.piper_host, settings.piper_port,
|
||||||
|
default_voice=settings.piper_voice,
|
||||||
|
),
|
||||||
"kokoro": KokoroEngine(kokoro_model),
|
"kokoro": KokoroEngine(kokoro_model),
|
||||||
"orpheus": OrpheusEngine(settings.orpheus_url),
|
"orpheus": OrpheusEngine(settings.orpheus_url),
|
||||||
}
|
}
|
||||||
@ -574,18 +619,30 @@ async def generate_audio(
|
|||||||
engine: ENGINE_NAMES = "kokoro",
|
engine: ENGINE_NAMES = "kokoro",
|
||||||
voice: str | None = None,
|
voice: str | None = None,
|
||||||
project: str | None = None,
|
project: str | None = None,
|
||||||
|
output_path: str | None = None,
|
||||||
|
format: FORMAT_NAMES = "wav",
|
||||||
ctx: Context = CurrentContext(),
|
ctx: Context = CurrentContext(),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
"""Synthesize text to a WAV file without playing it.
|
"""Synthesize text to an audio file without playing it.
|
||||||
|
|
||||||
Bypasses the speech queue — multiple agents can generate simultaneously.
|
Bypasses the speech queue — multiple agents can generate simultaneously.
|
||||||
Returns the file path and metadata.
|
|
||||||
|
Default behavior (no output_path, format=wav) writes a WAV to the
|
||||||
|
container's /tmp/mcspeak/ — fast and stable for in-container reads, but
|
||||||
|
NOT visible from the host. For host-visible output, either pass an
|
||||||
|
output_path under /output/ or pick a non-wav format; both paths route the
|
||||||
|
file to /output/ which is bind-mounted from TTS_OUTPUT_HOST_DIR (default
|
||||||
|
~/mcspeak-out on the host).
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
text: Text to synthesize.
|
text: Text to synthesize.
|
||||||
engine: TTS engine to use.
|
engine: TTS engine to use.
|
||||||
voice: Voice name (use list_voices to see options). None = auto-assigned by project.
|
voice: Voice name (use list_voices to see options). None = auto-assigned by project.
|
||||||
project: Project name for voice identity (auto-detected from MCP roots if omitted).
|
project: Project name for voice identity (auto-detected from MCP roots if omitted).
|
||||||
|
output_path: Where to write the file. None = today's behavior for wav, auto-named
|
||||||
|
under /output/ for non-wav formats. Relative paths join /output/; absolute
|
||||||
|
paths must resolve under /output/. Extension auto-corrected to match format.
|
||||||
|
format: Output format — wav, mp3, ogg, flac, or m4a. Non-wav formats use ffmpeg.
|
||||||
"""
|
"""
|
||||||
engines, _, voice_cache, _, _ = _get_state(ctx)
|
engines, _, voice_cache, _, _ = _get_state(ctx)
|
||||||
|
|
||||||
@ -600,12 +657,30 @@ async def generate_audio(
|
|||||||
await ctx.info(f"Generating audio with {engine}...")
|
await ctx.info(f"Generating audio with {engine}...")
|
||||||
result = await eng.synthesize(text, voice)
|
result = await eng.synthesize(text, voice)
|
||||||
|
|
||||||
|
# Custom path or non-wav format → convert + write to /output/. Otherwise
|
||||||
|
# keep the synthesized WAV at its in-container path (backward compat).
|
||||||
|
if output_path is not None or format != "wav":
|
||||||
|
try:
|
||||||
|
dest = _resolve_output_path(output_path, format)
|
||||||
|
await ctx.info(f"Writing {format} to {dest}")
|
||||||
|
await convert_audio(result.audio_path, dest, format)
|
||||||
|
except (ValueError, ConversionError) as e:
|
||||||
|
return {
|
||||||
|
"error": str(e),
|
||||||
|
"engine": result.engine,
|
||||||
|
"voice": result.voice,
|
||||||
|
}
|
||||||
|
final_path = dest
|
||||||
|
else:
|
||||||
|
final_path = result.audio_path
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"file": str(result.audio_path),
|
"file": str(final_path),
|
||||||
"duration_seconds": result.duration_seconds,
|
"duration_seconds": result.duration_seconds,
|
||||||
"sample_rate": result.sample_rate,
|
"sample_rate": result.sample_rate,
|
||||||
"engine": result.engine,
|
"engine": result.engine,
|
||||||
"voice": result.voice,
|
"voice": result.voice,
|
||||||
|
"format": format,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@ -15,6 +15,10 @@ class Settings(BaseSettings):
|
|||||||
# Piper (Wyoming protocol)
|
# Piper (Wyoming protocol)
|
||||||
piper_host: str = "172.26.0.3"
|
piper_host: str = "172.26.0.3"
|
||||||
piper_port: int = 10200
|
piper_port: int = 10200
|
||||||
|
# Default voice used when speak()/generate_audio() picks engine=piper
|
||||||
|
# without an explicit voice. Mirrors the --voice flag passed to the
|
||||||
|
# wyoming-piper container in compose so the pre-warmed model matches.
|
||||||
|
piper_voice: str = "es_MX-ald-medium"
|
||||||
|
|
||||||
# Kokoro (ONNX)
|
# Kokoro (ONNX)
|
||||||
kokoro_model: Path = Path("models/kokoro/kokoro-v1.0.onnx")
|
kokoro_model: Path = Path("models/kokoro/kokoro-v1.0.onnx")
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user