Fix stream-restore mute + entry tone overlap, make Orpheus opt-in
Stream-restore mute: pw-play streams were being silently restored to 0% volume by PulseAudio's stream-restore module matching the music role key, producing audible-but-clean-exit playback. audio.py now bumps each new pw-play sink-input to 100% as a background task; the new value re-stamps stream-restore on stream end. Entry tone overlap: when speak() arrived during prior playback, its entry tone played immediately from the handler and overlapped the ongoing audio. queue.is_idle() now gates this — tone plays now when idle (preserves latency-hiding intent), else defers to the consumer via _WorkItem.entry_tone, which plays it right before the item's audio. Chunked path defers only on chunk 0. Docker: llama-server gated behind a `with-orpheus` compose profile; mcspeak.depends_on uses required:false. `make up` defaults to kokoro-only with no GPU dependency; `make up-with-orpheus` runs the full stack. Rename dootie-internal network → mcspeak-internal (auto-created per-stack via internal:true). Add 127.0.0.1:8371 port mapping for local MCP clients. README: fix claude mcp add command (was stdio, must be --transport http); document kokoro-only default and the with-orpheus path; add Kokoro model download step. .env.example added so `cp .env.example .env` matches the documented setup.
This commit is contained in:
parent
44fec2ec32
commit
ec4e1b6093
15
.env.example
Normal file
15
.env.example
Normal file
@ -0,0 +1,15 @@
|
||||
# Compose project namespace -- prevents collisions with other stacks
|
||||
COMPOSE_PROJECT=mcspeak
|
||||
|
||||
# Path to Orpheus GGUF model file (required by llama-server service)
|
||||
# Tip: if you've already pulled it via Ollama, find the blob with:
|
||||
# ollama show --modelfile orpheus | grep FROM
|
||||
ORPHEUS_GGUF_PATH=/path/to/orpheus.gguf
|
||||
|
||||
# Optional overrides -- see README "Configuration" for the full list
|
||||
# TTS_ENTRY_TONE=chirp # chirp | apollo | none | /path/to.wav
|
||||
# TTS_EXIT_TONE=roger # roger | quindar-out | none | /path/to.wav
|
||||
# TTS_CANCEL_TONE=scratch # scratch | reverse-roger | none | /path/to.wav
|
||||
# TTS_DUCK_MEDIA=true # fade host audio while speaking
|
||||
# TTS_VOICE_IDENTITY=true # auto-assign distinct voices per project
|
||||
# TTS_SHUTDOWN_TIMEOUT=30 # max seconds to wait for current speech on stop
|
||||
11
Makefile
11
Makefile
@ -1,7 +1,9 @@
|
||||
include .env
|
||||
# .env is optional in kokoro-only mode (only needed for ORPHEUS_GGUF_PATH).
|
||||
# The leading dash makes Make silently tolerate a missing file.
|
||||
-include .env
|
||||
export
|
||||
|
||||
.PHONY: build up down logs restart status bench
|
||||
.PHONY: build up up-with-orpheus down logs restart status bench
|
||||
|
||||
build:
|
||||
docker compose build
|
||||
@ -11,6 +13,11 @@ up: build
|
||||
@sleep 2
|
||||
docker compose logs --tail 20
|
||||
|
||||
up-with-orpheus: build
|
||||
docker compose --profile with-orpheus up -d
|
||||
@sleep 2
|
||||
docker compose logs --tail 20
|
||||
|
||||
down:
|
||||
docker compose down
|
||||
|
||||
|
||||
65
README.md
65
README.md
@ -8,6 +8,21 @@ Multi-engine text-to-speech server exposed as MCP tools via [FastMCP 3.0](https:
|
||||
|
||||
## Quick Start
|
||||
|
||||
McSpeak runs a Streamable HTTP MCP server on `:8371`. Start the server, then point any MCP client at it.
|
||||
|
||||
**1. Get the Kokoro model files** (~340 MB, one-time):
|
||||
|
||||
```bash
|
||||
mkdir -p models/kokoro && cd models/kokoro
|
||||
curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
|
||||
curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin
|
||||
cd -
|
||||
```
|
||||
|
||||
The defaults look for `models/kokoro/kokoro-v1.0.onnx` and `models/kokoro/voices-v1.0.bin` relative to the working directory. Override with `TTS_KOKORO_MODEL` / `TTS_KOKORO_VOICES` if you put them elsewhere.
|
||||
|
||||
**2. Start the server:**
|
||||
|
||||
```bash
|
||||
# Run directly from PyPI (no install needed)
|
||||
uvx mcspeak
|
||||
@ -17,15 +32,17 @@ pip install mcspeak
|
||||
mcspeak
|
||||
```
|
||||
|
||||
The server starts on `http://0.0.0.0:8371` and exposes MCP tools over Streamable HTTP. Point any MCP client at it.
|
||||
Wait for `McSpeak ready on 0.0.0.0:8371` in the logs. Piper and Orpheus will report `unhealthy` unless their backends are running -- that's expected; Kokoro alone is enough to start.
|
||||
|
||||
**Add to Claude Code:**
|
||||
**3. Wire it into Claude Code** (HTTP transport, server must be running):
|
||||
|
||||
```bash
|
||||
claude mcp add mcspeak -- uvx mcspeak
|
||||
claude mcp add --transport http mcspeak http://127.0.0.1:8371/mcp
|
||||
```
|
||||
|
||||
For Docker deployment (recommended -- handles PipeWire, Piper, and Orpheus GPU inference), see [Docker Setup](#docker-setup) below.
|
||||
`mcspeak` ships as a Streamable HTTP server, not a stdio one, so `--transport http` is required. The older form `claude mcp add mcspeak -- uvx mcspeak` registers stdio and will fail to connect.
|
||||
|
||||
For multi-engine Docker deployment with Piper and GPU-accelerated Orpheus, see [Docker Setup](#docker-setup) below.
|
||||
|
||||
## MCP Tools
|
||||
|
||||
@ -102,26 +119,38 @@ All settings use the `TTS_` prefix and can be set via environment variables or `
|
||||
|
||||
## Docker Setup
|
||||
|
||||
Docker is the recommended deployment. The compose file runs McSpeak alongside a GPU-accelerated llama-server for Orpheus.
|
||||
The compose file defaults to a **kokoro-only stack** — no GPU required. Orpheus (GPU-accelerated, llama-server backed) is opt-in via the `with-orpheus` profile. Piper integrates via its host/port settings; you supply your own Wyoming server.
|
||||
|
||||
**Prerequisites:**
|
||||
- PipeWire running on the host (for `pw-play` audio output)
|
||||
- NVIDIA GPU + nvidia-container-toolkit (for Orpheus only)
|
||||
- Kokoro ONNX model files in `./models/kokoro/`
|
||||
**Prerequisites (base, kokoro-only):**
|
||||
- PipeWire on the host (the container plays audio via the host's `pw-play` socket)
|
||||
- An external Docker network named `caddy` (create with `docker network create caddy` if it doesn't exist). The private `mcspeak-internal` network is auto-created per-stack.
|
||||
- Kokoro ONNX model files in `./models/kokoro/` (see [Quick Start](#quick-start) for the download)
|
||||
|
||||
**Additional prerequisites for `with-orpheus`:**
|
||||
- NVIDIA GPU + nvidia-container-toolkit
|
||||
- Orpheus GGUF model on disk — path goes in `.env` as `ORPHEUS_GGUF_PATH`
|
||||
|
||||
**Bring it up (kokoro-only):**
|
||||
|
||||
```bash
|
||||
make up # build + start kokoro-only
|
||||
make logs # follow logs
|
||||
make status # health check
|
||||
```
|
||||
|
||||
**Bring it up with Orpheus:**
|
||||
|
||||
```bash
|
||||
# Copy and edit .env
|
||||
cp .env.example .env
|
||||
# Set ORPHEUS_GGUF_PATH to your Orpheus GGUF model location
|
||||
# Edit .env: set ORPHEUS_GGUF_PATH to your Orpheus GGUF file
|
||||
make up-with-orpheus
|
||||
```
|
||||
|
||||
# Build and start
|
||||
make up
|
||||
Once `make status` shows the container as healthy, register it with Claude Code. The compose file applies a `caddy-docker-proxy` label exposing the server at `mctalkbox.l.supported.systems` -- if you're not using that hostname, point at the container's mapped port directly:
|
||||
|
||||
# Follow logs
|
||||
make logs
|
||||
|
||||
# Check health
|
||||
make status
|
||||
```bash
|
||||
# Direct connection (replace with your hostname/port if different)
|
||||
claude mcp add --transport http mcspeak http://127.0.0.1:8371/mcp
|
||||
```
|
||||
|
||||
The container mounts the host PipeWire socket (`/run/user/1000/pipewire-0`) for audio playback and the PulseAudio compat socket (`/run/user/1000/pulse`) for media ducking.
|
||||
|
||||
@ -7,6 +7,11 @@ services:
|
||||
# Default: 3 + 30 + 5 = 38s. Increase if TTS_SHUTDOWN_TIMEOUT > 30.
|
||||
stop_grace_period: 38s
|
||||
env_file: .env
|
||||
# Publish to localhost only — Claude Code (and any local MCP client) talks
|
||||
# to 127.0.0.1:8371. External HTTPS access still flows through the caddy
|
||||
# label below if caddy-docker-proxy is running.
|
||||
ports:
|
||||
- "127.0.0.1:8371:8371"
|
||||
environment:
|
||||
# Override for Docker networking (container DNS instead of IPs)
|
||||
TTS_PIPER_HOST: piper-tts
|
||||
@ -31,14 +36,20 @@ services:
|
||||
depends_on:
|
||||
llama-server:
|
||||
condition: service_healthy
|
||||
# required: false lets mcspeak start without llama-server when the
|
||||
# with-orpheus profile isn't active (kokoro-only mode is the default).
|
||||
required: false
|
||||
networks:
|
||||
- caddy
|
||||
- dootie-internal
|
||||
- mcspeak-internal
|
||||
labels:
|
||||
caddy: mctalkbox.l.supported.systems
|
||||
caddy.reverse_proxy: "{{upstreams 8371}}"
|
||||
|
||||
llama-server:
|
||||
# Opt-in: only starts when `docker compose --profile with-orpheus up`.
|
||||
# Default `make up` runs kokoro-only without GPU dependencies.
|
||||
profiles: ["with-orpheus"]
|
||||
build:
|
||||
context: .
|
||||
dockerfile: llama-server.Dockerfile
|
||||
@ -52,15 +63,19 @@ services:
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
volumes:
|
||||
# GGUF model file (set ORPHEUS_GGUF_PATH in .env, e.g. from Ollama blob storage)
|
||||
- ${ORPHEUS_GGUF_PATH}:/models/orpheus.gguf:ro
|
||||
# GGUF model file (set ORPHEUS_GGUF_PATH in .env, e.g. from Ollama blob
|
||||
# storage). The /dev/null fallback lets compose parse cleanly when the
|
||||
# with-orpheus profile is inactive; activating the profile without
|
||||
# setting ORPHEUS_GGUF_PATH will produce a clear "model load failed" at
|
||||
# llama-server startup rather than a confusing compose parse error.
|
||||
- ${ORPHEUS_GGUF_PATH:-/dev/null}:/models/orpheus.gguf:ro
|
||||
command: >-
|
||||
--host 0.0.0.0 --port 8081
|
||||
--model /models/orpheus.gguf
|
||||
--n-gpu-layers 999 --ctx-size 4096
|
||||
--flash-attn --cont-batching
|
||||
networks:
|
||||
- dootie-internal
|
||||
- mcspeak-internal
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://127.0.0.1:8081/health"]
|
||||
interval: 15s
|
||||
@ -75,5 +90,7 @@ volumes:
|
||||
networks:
|
||||
caddy:
|
||||
external: true
|
||||
dootie-internal:
|
||||
external: true
|
||||
# Private per-stack network for mcspeak ↔ llama-server. Auto-created by
|
||||
# compose, isolated from other stacks (no DNS leak via shared caddy).
|
||||
mcspeak-internal:
|
||||
internal: true
|
||||
|
||||
@ -2,6 +2,8 @@
|
||||
|
||||
import asyncio
|
||||
import itertools
|
||||
import re
|
||||
import shutil
|
||||
import time
|
||||
import wave
|
||||
from pathlib import Path
|
||||
@ -11,6 +13,7 @@ import numpy as np
|
||||
from .settings import settings
|
||||
|
||||
_counter = itertools.count(1)
|
||||
_HAS_PACTL = shutil.which("pactl") is not None
|
||||
|
||||
|
||||
def write_wav(
|
||||
@ -78,6 +81,60 @@ class PlaybackError(RuntimeError):
|
||||
"""Raised when audio playback fails or times out."""
|
||||
|
||||
|
||||
async def _find_recent_pwplay_sink_input() -> str | None:
|
||||
"""Return the sink-input ID of the most recent pw-play stream, or None."""
|
||||
if not _HAS_PACTL:
|
||||
return None
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"pactl", "list", "sink-inputs",
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.DEVNULL,
|
||||
)
|
||||
stdout, _ = await proc.communicate()
|
||||
if proc.returncode != 0:
|
||||
return None
|
||||
text = stdout.decode(errors="replace")
|
||||
target_id: str | None = None
|
||||
for block in re.split(r"^Sink Input #", text, flags=re.MULTILINE)[1:]:
|
||||
if 'application.name = "pw-play"' in block:
|
||||
m = re.match(r"(\d+)", block)
|
||||
if m:
|
||||
target_id = m.group(1) # last match wins — highest ID is newest
|
||||
return target_id
|
||||
|
||||
|
||||
async def _force_pwplay_volume_100() -> None:
|
||||
"""Override stream-restore by forcing the latest pw-play sink-input to 100%.
|
||||
|
||||
PulseAudio's module-stream-restore saves per-app/per-role volumes and
|
||||
restores them on each new stream. If a music-role stream was ever set
|
||||
to 0% (e.g. an accidental pavucontrol slip), every subsequent pw-play
|
||||
call inherits 0% — audio plays cleanly but silently with no error.
|
||||
|
||||
Polls every 50ms for up to 500ms. The first successful bump re-stamps
|
||||
stream-restore's saved value when the stream ends, so subsequent plays
|
||||
inherit 100% naturally. Silent no-op if pactl is missing.
|
||||
"""
|
||||
if not _HAS_PACTL:
|
||||
return
|
||||
loop = asyncio.get_event_loop()
|
||||
deadline = loop.time() + 0.5
|
||||
while loop.time() < deadline:
|
||||
await asyncio.sleep(0.05)
|
||||
sink_input_id = await _find_recent_pwplay_sink_input()
|
||||
if sink_input_id:
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"pactl", "set-sink-input-volume", sink_input_id, "100%",
|
||||
stdout=asyncio.subprocess.DEVNULL,
|
||||
stderr=asyncio.subprocess.DEVNULL,
|
||||
)
|
||||
await proc.wait()
|
||||
except Exception:
|
||||
pass # Non-fatal — playback continues regardless
|
||||
return
|
||||
|
||||
|
||||
async def play_audio(path: Path, expected_seconds: float = 0) -> None:
|
||||
"""Play a WAV file through PipeWire (pw-play). Async wrapper.
|
||||
|
||||
@ -96,11 +153,14 @@ async def play_audio(path: Path, expected_seconds: float = 0) -> None:
|
||||
stdout=asyncio.subprocess.DEVNULL,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
# Defensive volume override — guards against module-stream-restore drift.
|
||||
vol_task = asyncio.create_task(_force_pwplay_volume_100())
|
||||
try:
|
||||
_, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
|
||||
except asyncio.TimeoutError:
|
||||
proc.kill()
|
||||
await proc.wait()
|
||||
vol_task.cancel()
|
||||
raise PlaybackError(
|
||||
f"pw-play timed out after {timeout:.0f}s "
|
||||
f"(expected {expected_seconds:.1f}s audio)"
|
||||
@ -109,7 +169,10 @@ async def play_audio(path: Path, expected_seconds: float = 0) -> None:
|
||||
# Consumer was cancelled (shutdown or cancel) — don't orphan the subprocess
|
||||
proc.kill()
|
||||
await proc.wait()
|
||||
vol_task.cancel()
|
||||
raise
|
||||
if not vol_task.done():
|
||||
vol_task.cancel()
|
||||
if proc.returncode != 0:
|
||||
raise PlaybackError(
|
||||
f"pw-play failed ({proc.returncode}): {stderr.decode().strip()}"
|
||||
|
||||
@ -43,6 +43,11 @@ class _WorkItem:
|
||||
speech_id: str = field(default="", compare=False)
|
||||
enqueued_at: float = field(default_factory=time.time, compare=False)
|
||||
suppress_exit_tone: bool = field(default=False, compare=False)
|
||||
# Entry tone deferred from speak() handler when the queue was busy at
|
||||
# call time. Consumer plays this right before the item's audio so the
|
||||
# tone doesn't overlap a previously-playing item. None = no entry tone
|
||||
# (either no tone configured, or already played immediately by the handler).
|
||||
entry_tone: Path | None = field(default=None, compare=False)
|
||||
|
||||
|
||||
class SpeechQueue:
|
||||
@ -95,6 +100,15 @@ class SpeechQueue:
|
||||
def current_speaker(self) -> str | None:
|
||||
return self._current.speech_id if self._current else None
|
||||
|
||||
def is_idle(self) -> bool:
|
||||
"""True when nothing is playing and nothing is queued.
|
||||
|
||||
Used by the speak() handler to decide whether to play the entry tone
|
||||
immediately (idle → latency-hiding) or defer it to the consumer
|
||||
(busy → tone plays right before this item, no overlap).
|
||||
"""
|
||||
return self._current is None and self._queue.empty()
|
||||
|
||||
def status(self) -> dict:
|
||||
return {
|
||||
"current_speaker": self.current_speaker,
|
||||
@ -217,6 +231,15 @@ class SpeechQueue:
|
||||
self._item_cancelled = False
|
||||
|
||||
try:
|
||||
# Deferred entry tone — speak() handler deferred this when the
|
||||
# queue was busy at call time. Play it now so it lands right
|
||||
# before this item's audio (no overlap with prior playback).
|
||||
if item.entry_tone:
|
||||
try:
|
||||
await play_audio(item.entry_tone, expected_seconds=0.3)
|
||||
except PlaybackError:
|
||||
pass # Non-fatal
|
||||
|
||||
await play_audio(
|
||||
item.result.audio_path,
|
||||
expected_seconds=item.result.duration_seconds,
|
||||
@ -302,12 +325,17 @@ class SpeechQueue:
|
||||
result: TTSResult,
|
||||
priority: Priority = Priority.NORMAL,
|
||||
suppress_exit_tone: bool = False,
|
||||
entry_tone: Path | None = None,
|
||||
) -> dict:
|
||||
"""Enqueue a TTSResult for playback. Returns immediately with enqueue metadata.
|
||||
|
||||
Does NOT block until playback finishes — use get_status(speech_id) to
|
||||
check the outcome later.
|
||||
|
||||
If entry_tone is given, the consumer plays it immediately before this
|
||||
item's audio (used when speak() handler deferred the tone because the
|
||||
queue was busy at call time).
|
||||
|
||||
Raises QueueFull if the queue is at max capacity (backpressure).
|
||||
"""
|
||||
if self._stopped:
|
||||
@ -329,6 +357,7 @@ class SpeechQueue:
|
||||
future=future,
|
||||
speech_id=speech_id,
|
||||
suppress_exit_tone=suppress_exit_tone,
|
||||
entry_tone=entry_tone,
|
||||
)
|
||||
|
||||
try:
|
||||
|
||||
@ -253,15 +253,23 @@ async def _speak_single(
|
||||
Used for short texts (< 20 words) and as the fallback when text has
|
||||
no sentence boundaries.
|
||||
"""
|
||||
# Entry tone gating — idle queue plays tone now (hides synth latency);
|
||||
# busy queue defers tone to the consumer so it doesn't overlap whatever's
|
||||
# already playing. Capture idle state before duck/synth changes it.
|
||||
queue_was_idle = queue.is_idle()
|
||||
deferred_entry_tone: Path | None = (
|
||||
None if queue_was_idle or entry_tone is None else entry_tone
|
||||
)
|
||||
|
||||
try:
|
||||
# Duck external media with crossfade into entry tone
|
||||
if ducker:
|
||||
duck_task = asyncio.create_task(ducker.duck())
|
||||
await asyncio.sleep(0.15) # Let fade start — media dips to ~70%
|
||||
if entry_tone:
|
||||
if entry_tone and queue_was_idle:
|
||||
await _play_tone(entry_tone) # Tone crossfades over fading media
|
||||
await duck_task # Ensure duck completes
|
||||
elif entry_tone:
|
||||
elif entry_tone and queue_was_idle:
|
||||
await _play_tone(entry_tone)
|
||||
await ctx.report_progress(progress=5, total=100, message="Entry tone")
|
||||
|
||||
@ -270,7 +278,9 @@ async def _speak_single(
|
||||
await ctx.report_progress(progress=30, total=100, message="Synthesizing...")
|
||||
|
||||
await ctx.info(f"Synthesized {result.duration_seconds:.1f}s audio, enqueueing...")
|
||||
enqueue_result = await queue.enqueue(result, priority=priority)
|
||||
enqueue_result = await queue.enqueue(
|
||||
result, priority=priority, entry_tone=deferred_entry_tone,
|
||||
)
|
||||
|
||||
if not enqueue_result.get("queued"):
|
||||
return enqueue_result
|
||||
@ -319,15 +329,23 @@ async def _speak_chunked(
|
||||
all_speech_ids: list[str] = []
|
||||
first_enqueue_time: float | None = None
|
||||
|
||||
# Entry tone gating (see _speak_single for full rationale). Only the FIRST
|
||||
# chunk carries the deferred tone — subsequent chunks are continuations of
|
||||
# the same message and don't get their own tone.
|
||||
queue_was_idle = queue.is_idle()
|
||||
deferred_entry_tone: Path | None = (
|
||||
None if queue_was_idle or entry_tone is None else entry_tone
|
||||
)
|
||||
|
||||
try:
|
||||
# Duck external media with crossfade into entry tone
|
||||
if ducker:
|
||||
duck_task = asyncio.create_task(ducker.duck())
|
||||
await asyncio.sleep(0.15) # Let fade start — media dips to ~70%
|
||||
if entry_tone:
|
||||
if entry_tone and queue_was_idle:
|
||||
await _play_tone(entry_tone) # Tone crossfades over fading media
|
||||
await duck_task # Ensure duck completes
|
||||
elif entry_tone:
|
||||
elif entry_tone and queue_was_idle:
|
||||
await _play_tone(entry_tone)
|
||||
await ctx.report_progress(progress=5, total=100, message="Entry tone")
|
||||
|
||||
@ -347,6 +365,7 @@ async def _speak_chunked(
|
||||
result,
|
||||
priority=priority,
|
||||
suppress_exit_tone=not is_last,
|
||||
entry_tone=deferred_entry_tone if i == 0 else None,
|
||||
)
|
||||
|
||||
if not enqueue_result.get("queued"):
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user