Fix stream-restore mute + entry tone overlap, make Orpheus opt-in

Stream-restore mute: pw-play streams were being silently restored to
0% volume by PulseAudio's stream-restore module matching the music
role key, producing audible-but-clean-exit playback. audio.py now
bumps each new pw-play sink-input to 100% as a background task; the
new value re-stamps stream-restore on stream end.

Entry tone overlap: when speak() arrived during prior playback, its
entry tone played immediately from the handler and overlapped the
ongoing audio. queue.is_idle() now gates this — tone plays now when
idle (preserves latency-hiding intent), else defers to the consumer
via _WorkItem.entry_tone, which plays it right before the item's
audio. Chunked path defers only on chunk 0.

Docker: llama-server gated behind a `with-orpheus` compose profile;
mcspeak.depends_on uses required:false. `make up` defaults to
kokoro-only with no GPU dependency; `make up-with-orpheus` runs the
full stack. Rename dootie-internal network → mcspeak-internal
(auto-created per-stack via internal:true). Add 127.0.0.1:8371 port
mapping for local MCP clients.

README: fix claude mcp add command (was stdio, must be --transport
http); document kokoro-only default and the with-orpheus path; add
Kokoro model download step. .env.example added so `cp .env.example
.env` matches the documented setup.
This commit is contained in:
Ryan Malloy 2026-05-28 14:14:38 -06:00
parent 44fec2ec32
commit ec4e1b6093
7 changed files with 210 additions and 31 deletions

15
.env.example Normal file
View File

@ -0,0 +1,15 @@
# Compose project namespace -- prevents collisions with other stacks
COMPOSE_PROJECT=mcspeak
# Path to Orpheus GGUF model file (required by llama-server service)
# Tip: if you've already pulled it via Ollama, find the blob with:
# ollama show --modelfile orpheus | grep FROM
ORPHEUS_GGUF_PATH=/path/to/orpheus.gguf
# Optional overrides -- see README "Configuration" for the full list
# TTS_ENTRY_TONE=chirp # chirp | apollo | none | /path/to.wav
# TTS_EXIT_TONE=roger # roger | quindar-out | none | /path/to.wav
# TTS_CANCEL_TONE=scratch # scratch | reverse-roger | none | /path/to.wav
# TTS_DUCK_MEDIA=true # fade host audio while speaking
# TTS_VOICE_IDENTITY=true # auto-assign distinct voices per project
# TTS_SHUTDOWN_TIMEOUT=30 # max seconds to wait for current speech on stop

View File

@ -1,7 +1,9 @@
include .env
# .env is optional in kokoro-only mode (only needed for ORPHEUS_GGUF_PATH).
# The leading dash makes Make silently tolerate a missing file.
-include .env
export
.PHONY: build up down logs restart status bench
.PHONY: build up up-with-orpheus down logs restart status bench
build:
docker compose build
@ -11,6 +13,11 @@ up: build
@sleep 2
docker compose logs --tail 20
up-with-orpheus: build
docker compose --profile with-orpheus up -d
@sleep 2
docker compose logs --tail 20
down:
docker compose down

View File

@ -8,6 +8,21 @@ Multi-engine text-to-speech server exposed as MCP tools via [FastMCP 3.0](https:
## Quick Start
McSpeak runs a Streamable HTTP MCP server on `:8371`. Start the server, then point any MCP client at it.
**1. Get the Kokoro model files** (~340 MB, one-time):
```bash
mkdir -p models/kokoro && cd models/kokoro
curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin
cd -
```
The defaults look for `models/kokoro/kokoro-v1.0.onnx` and `models/kokoro/voices-v1.0.bin` relative to the working directory. Override with `TTS_KOKORO_MODEL` / `TTS_KOKORO_VOICES` if you put them elsewhere.
**2. Start the server:**
```bash
# Run directly from PyPI (no install needed)
uvx mcspeak
@ -17,15 +32,17 @@ pip install mcspeak
mcspeak
```
The server starts on `http://0.0.0.0:8371` and exposes MCP tools over Streamable HTTP. Point any MCP client at it.
Wait for `McSpeak ready on 0.0.0.0:8371` in the logs. Piper and Orpheus will report `unhealthy` unless their backends are running -- that's expected; Kokoro alone is enough to start.
**Add to Claude Code:**
**3. Wire it into Claude Code** (HTTP transport, server must be running):
```bash
claude mcp add mcspeak -- uvx mcspeak
claude mcp add --transport http mcspeak http://127.0.0.1:8371/mcp
```
For Docker deployment (recommended -- handles PipeWire, Piper, and Orpheus GPU inference), see [Docker Setup](#docker-setup) below.
`mcspeak` ships as a Streamable HTTP server, not a stdio one, so `--transport http` is required. The older form `claude mcp add mcspeak -- uvx mcspeak` registers stdio and will fail to connect.
For multi-engine Docker deployment with Piper and GPU-accelerated Orpheus, see [Docker Setup](#docker-setup) below.
## MCP Tools
@ -102,26 +119,38 @@ All settings use the `TTS_` prefix and can be set via environment variables or `
## Docker Setup
Docker is the recommended deployment. The compose file runs McSpeak alongside a GPU-accelerated llama-server for Orpheus.
The compose file defaults to a **kokoro-only stack** — no GPU required. Orpheus (GPU-accelerated, llama-server backed) is opt-in via the `with-orpheus` profile. Piper integrates via its host/port settings; you supply your own Wyoming server.
**Prerequisites:**
- PipeWire running on the host (for `pw-play` audio output)
- NVIDIA GPU + nvidia-container-toolkit (for Orpheus only)
- Kokoro ONNX model files in `./models/kokoro/`
**Prerequisites (base, kokoro-only):**
- PipeWire on the host (the container plays audio via the host's `pw-play` socket)
- An external Docker network named `caddy` (create with `docker network create caddy` if it doesn't exist). The private `mcspeak-internal` network is auto-created per-stack.
- Kokoro ONNX model files in `./models/kokoro/` (see [Quick Start](#quick-start) for the download)
**Additional prerequisites for `with-orpheus`:**
- NVIDIA GPU + nvidia-container-toolkit
- Orpheus GGUF model on disk — path goes in `.env` as `ORPHEUS_GGUF_PATH`
**Bring it up (kokoro-only):**
```bash
make up # build + start kokoro-only
make logs # follow logs
make status # health check
```
**Bring it up with Orpheus:**
```bash
# Copy and edit .env
cp .env.example .env
# Set ORPHEUS_GGUF_PATH to your Orpheus GGUF model location
# Edit .env: set ORPHEUS_GGUF_PATH to your Orpheus GGUF file
make up-with-orpheus
```
# Build and start
make up
Once `make status` shows the container as healthy, register it with Claude Code. The compose file applies a `caddy-docker-proxy` label exposing the server at `mctalkbox.l.supported.systems` -- if you're not using that hostname, point at the container's mapped port directly:
# Follow logs
make logs
# Check health
make status
```bash
# Direct connection (replace with your hostname/port if different)
claude mcp add --transport http mcspeak http://127.0.0.1:8371/mcp
```
The container mounts the host PipeWire socket (`/run/user/1000/pipewire-0`) for audio playback and the PulseAudio compat socket (`/run/user/1000/pulse`) for media ducking.

View File

@ -7,6 +7,11 @@ services:
# Default: 3 + 30 + 5 = 38s. Increase if TTS_SHUTDOWN_TIMEOUT > 30.
stop_grace_period: 38s
env_file: .env
# Publish to localhost only — Claude Code (and any local MCP client) talks
# to 127.0.0.1:8371. External HTTPS access still flows through the caddy
# label below if caddy-docker-proxy is running.
ports:
- "127.0.0.1:8371:8371"
environment:
# Override for Docker networking (container DNS instead of IPs)
TTS_PIPER_HOST: piper-tts
@ -31,14 +36,20 @@ services:
depends_on:
llama-server:
condition: service_healthy
# required: false lets mcspeak start without llama-server when the
# with-orpheus profile isn't active (kokoro-only mode is the default).
required: false
networks:
- caddy
- dootie-internal
- mcspeak-internal
labels:
caddy: mctalkbox.l.supported.systems
caddy.reverse_proxy: "{{upstreams 8371}}"
llama-server:
# Opt-in: only starts when `docker compose --profile with-orpheus up`.
# Default `make up` runs kokoro-only without GPU dependencies.
profiles: ["with-orpheus"]
build:
context: .
dockerfile: llama-server.Dockerfile
@ -52,15 +63,19 @@ services:
count: 1
capabilities: [gpu]
volumes:
# GGUF model file (set ORPHEUS_GGUF_PATH in .env, e.g. from Ollama blob storage)
- ${ORPHEUS_GGUF_PATH}:/models/orpheus.gguf:ro
# GGUF model file (set ORPHEUS_GGUF_PATH in .env, e.g. from Ollama blob
# storage). The /dev/null fallback lets compose parse cleanly when the
# with-orpheus profile is inactive; activating the profile without
# setting ORPHEUS_GGUF_PATH will produce a clear "model load failed" at
# llama-server startup rather than a confusing compose parse error.
- ${ORPHEUS_GGUF_PATH:-/dev/null}:/models/orpheus.gguf:ro
command: >-
--host 0.0.0.0 --port 8081
--model /models/orpheus.gguf
--n-gpu-layers 999 --ctx-size 4096
--flash-attn --cont-batching
networks:
- dootie-internal
- mcspeak-internal
healthcheck:
test: ["CMD", "curl", "-sf", "http://127.0.0.1:8081/health"]
interval: 15s
@ -75,5 +90,7 @@ volumes:
networks:
caddy:
external: true
dootie-internal:
external: true
# Private per-stack network for mcspeak ↔ llama-server. Auto-created by
# compose, isolated from other stacks (no DNS leak via shared caddy).
mcspeak-internal:
internal: true

View File

@ -2,6 +2,8 @@
import asyncio
import itertools
import re
import shutil
import time
import wave
from pathlib import Path
@ -11,6 +13,7 @@ import numpy as np
from .settings import settings
_counter = itertools.count(1)
_HAS_PACTL = shutil.which("pactl") is not None
def write_wav(
@ -78,6 +81,60 @@ class PlaybackError(RuntimeError):
"""Raised when audio playback fails or times out."""
async def _find_recent_pwplay_sink_input() -> str | None:
"""Return the sink-input ID of the most recent pw-play stream, or None."""
if not _HAS_PACTL:
return None
proc = await asyncio.create_subprocess_exec(
"pactl", "list", "sink-inputs",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.DEVNULL,
)
stdout, _ = await proc.communicate()
if proc.returncode != 0:
return None
text = stdout.decode(errors="replace")
target_id: str | None = None
for block in re.split(r"^Sink Input #", text, flags=re.MULTILINE)[1:]:
if 'application.name = "pw-play"' in block:
m = re.match(r"(\d+)", block)
if m:
target_id = m.group(1) # last match wins — highest ID is newest
return target_id
async def _force_pwplay_volume_100() -> None:
"""Override stream-restore by forcing the latest pw-play sink-input to 100%.
PulseAudio's module-stream-restore saves per-app/per-role volumes and
restores them on each new stream. If a music-role stream was ever set
to 0% (e.g. an accidental pavucontrol slip), every subsequent pw-play
call inherits 0% audio plays cleanly but silently with no error.
Polls every 50ms for up to 500ms. The first successful bump re-stamps
stream-restore's saved value when the stream ends, so subsequent plays
inherit 100% naturally. Silent no-op if pactl is missing.
"""
if not _HAS_PACTL:
return
loop = asyncio.get_event_loop()
deadline = loop.time() + 0.5
while loop.time() < deadline:
await asyncio.sleep(0.05)
sink_input_id = await _find_recent_pwplay_sink_input()
if sink_input_id:
try:
proc = await asyncio.create_subprocess_exec(
"pactl", "set-sink-input-volume", sink_input_id, "100%",
stdout=asyncio.subprocess.DEVNULL,
stderr=asyncio.subprocess.DEVNULL,
)
await proc.wait()
except Exception:
pass # Non-fatal — playback continues regardless
return
async def play_audio(path: Path, expected_seconds: float = 0) -> None:
"""Play a WAV file through PipeWire (pw-play). Async wrapper.
@ -96,11 +153,14 @@ async def play_audio(path: Path, expected_seconds: float = 0) -> None:
stdout=asyncio.subprocess.DEVNULL,
stderr=asyncio.subprocess.PIPE,
)
# Defensive volume override — guards against module-stream-restore drift.
vol_task = asyncio.create_task(_force_pwplay_volume_100())
try:
_, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
vol_task.cancel()
raise PlaybackError(
f"pw-play timed out after {timeout:.0f}s "
f"(expected {expected_seconds:.1f}s audio)"
@ -109,7 +169,10 @@ async def play_audio(path: Path, expected_seconds: float = 0) -> None:
# Consumer was cancelled (shutdown or cancel) — don't orphan the subprocess
proc.kill()
await proc.wait()
vol_task.cancel()
raise
if not vol_task.done():
vol_task.cancel()
if proc.returncode != 0:
raise PlaybackError(
f"pw-play failed ({proc.returncode}): {stderr.decode().strip()}"

View File

@ -43,6 +43,11 @@ class _WorkItem:
speech_id: str = field(default="", compare=False)
enqueued_at: float = field(default_factory=time.time, compare=False)
suppress_exit_tone: bool = field(default=False, compare=False)
# Entry tone deferred from speak() handler when the queue was busy at
# call time. Consumer plays this right before the item's audio so the
# tone doesn't overlap a previously-playing item. None = no entry tone
# (either no tone configured, or already played immediately by the handler).
entry_tone: Path | None = field(default=None, compare=False)
class SpeechQueue:
@ -95,6 +100,15 @@ class SpeechQueue:
def current_speaker(self) -> str | None:
return self._current.speech_id if self._current else None
def is_idle(self) -> bool:
"""True when nothing is playing and nothing is queued.
Used by the speak() handler to decide whether to play the entry tone
immediately (idle latency-hiding) or defer it to the consumer
(busy tone plays right before this item, no overlap).
"""
return self._current is None and self._queue.empty()
def status(self) -> dict:
return {
"current_speaker": self.current_speaker,
@ -217,6 +231,15 @@ class SpeechQueue:
self._item_cancelled = False
try:
# Deferred entry tone — speak() handler deferred this when the
# queue was busy at call time. Play it now so it lands right
# before this item's audio (no overlap with prior playback).
if item.entry_tone:
try:
await play_audio(item.entry_tone, expected_seconds=0.3)
except PlaybackError:
pass # Non-fatal
await play_audio(
item.result.audio_path,
expected_seconds=item.result.duration_seconds,
@ -302,12 +325,17 @@ class SpeechQueue:
result: TTSResult,
priority: Priority = Priority.NORMAL,
suppress_exit_tone: bool = False,
entry_tone: Path | None = None,
) -> dict:
"""Enqueue a TTSResult for playback. Returns immediately with enqueue metadata.
Does NOT block until playback finishes use get_status(speech_id) to
check the outcome later.
If entry_tone is given, the consumer plays it immediately before this
item's audio (used when speak() handler deferred the tone because the
queue was busy at call time).
Raises QueueFull if the queue is at max capacity (backpressure).
"""
if self._stopped:
@ -329,6 +357,7 @@ class SpeechQueue:
future=future,
speech_id=speech_id,
suppress_exit_tone=suppress_exit_tone,
entry_tone=entry_tone,
)
try:

View File

@ -253,15 +253,23 @@ async def _speak_single(
Used for short texts (< 20 words) and as the fallback when text has
no sentence boundaries.
"""
# Entry tone gating — idle queue plays tone now (hides synth latency);
# busy queue defers tone to the consumer so it doesn't overlap whatever's
# already playing. Capture idle state before duck/synth changes it.
queue_was_idle = queue.is_idle()
deferred_entry_tone: Path | None = (
None if queue_was_idle or entry_tone is None else entry_tone
)
try:
# Duck external media with crossfade into entry tone
if ducker:
duck_task = asyncio.create_task(ducker.duck())
await asyncio.sleep(0.15) # Let fade start — media dips to ~70%
if entry_tone:
if entry_tone and queue_was_idle:
await _play_tone(entry_tone) # Tone crossfades over fading media
await duck_task # Ensure duck completes
elif entry_tone:
elif entry_tone and queue_was_idle:
await _play_tone(entry_tone)
await ctx.report_progress(progress=5, total=100, message="Entry tone")
@ -270,7 +278,9 @@ async def _speak_single(
await ctx.report_progress(progress=30, total=100, message="Synthesizing...")
await ctx.info(f"Synthesized {result.duration_seconds:.1f}s audio, enqueueing...")
enqueue_result = await queue.enqueue(result, priority=priority)
enqueue_result = await queue.enqueue(
result, priority=priority, entry_tone=deferred_entry_tone,
)
if not enqueue_result.get("queued"):
return enqueue_result
@ -319,15 +329,23 @@ async def _speak_chunked(
all_speech_ids: list[str] = []
first_enqueue_time: float | None = None
# Entry tone gating (see _speak_single for full rationale). Only the FIRST
# chunk carries the deferred tone — subsequent chunks are continuations of
# the same message and don't get their own tone.
queue_was_idle = queue.is_idle()
deferred_entry_tone: Path | None = (
None if queue_was_idle or entry_tone is None else entry_tone
)
try:
# Duck external media with crossfade into entry tone
if ducker:
duck_task = asyncio.create_task(ducker.duck())
await asyncio.sleep(0.15) # Let fade start — media dips to ~70%
if entry_tone:
if entry_tone and queue_was_idle:
await _play_tone(entry_tone) # Tone crossfades over fading media
await duck_task # Ensure duck completes
elif entry_tone:
elif entry_tone and queue_was_idle:
await _play_tone(entry_tone)
await ctx.report_progress(progress=5, total=100, message="Entry tone")
@ -347,6 +365,7 @@ async def _speak_chunked(
result,
priority=priority,
suppress_exit_tone=not is_last,
entry_tone=deferred_entry_tone if i == 0 else None,
)
if not enqueue_result.get("queued"):