# McSpeak

Multi-engine text-to-speech MCP server exposed via FastMCP 3.0 Streamable HTTP. Engines: Kokoro (ONNX), Piper (Wyoming/Docker), Orpheus (llama-server + SNAC).

## Build & Run

```bash
make up        # build + start (docker compose)
make logs      # follow logs
make restart   # restart containers
make status    # show running containers + health
```

## Entry & Exit Tones (Beep System)

Queued speech playback (`speak()`) is bookended by short alert tones. `generate_audio()` is unaffected (file-only, no playback).

### Tone Positions

| Position | When | Purpose |
|----------|------|---------|
| **Entry tone** | Immediately when `speak()` is called (before synthesis) | "I heard you" acknowledgement — covers synthesis latency |
| **Exit tone** | After speech, queue empty | "Over and out" — channel clear |
| **Standby tone** | After speech, more queued | "Standby" — more messages coming |
| **Cancel tone** | After `cancel_speech()` or MCP cancellation kills playback | "Nevermind" — speech was aborted |

### Available Tones

| Name | Frequency | Duration | Inspired by |
|------|-----------|----------|-------------|
| `chirp` | 1800 Hz | ~144 ms | Nextel iDEN Talk Permit Tone (TPT) — the 24/24/24/24/48 ms on/off pattern |
| `apollo` | 2525 Hz | 250 ms | NASA quindar intro (key-up) tone used during Apollo missions |
| `roger` | 1400-1000 Hz | ~100 ms | Classic CB radio descending two-tone roger beep |
| `quindar-out` | 2475 Hz | 250 ms | NASA quindar unkey tone (distinct frequency from intro) |
| `standby` | 1000-1400 Hz | ~60 ms | Ascending blip — inverse of roger, signals "more coming" |
| `scratch` | 2000-300 Hz sweep + noise | ~120 ms | Vinyl record scratch — needle yanked off the platter |
| `reverse-roger` | 1000-1400 Hz | ~100 ms | Ascending two-tone — mathematical inverse of roger beep |

### Configuration

```env
TTS_ENTRY_TONE=chirp        # before speech (chirp, apollo, none, or /path/to/custom.wav)
TTS_EXIT_TONE=roger          # after speech, queue empty (roger, quindar-out, none, or path)
TTS_CANCEL_TONE=scratch      # on cancel (scratch, reverse-roger, none, or /path/to/custom.wav)
TTS_SHUTDOWN_TIMEOUT=30      # max seconds to wait for current speech on container stop
```

The standby tone is always the built-in ascending blip. It plays instead of the exit tone when more items are queued.

Tones are generated programmatically at startup (48kHz, 16-bit PCM, -3 dB headroom) in `tones.py` using numpy. No bundled audio assets.

## Voice Identity (Project-Aware Voices)

When multiple Claude Code sessions connect simultaneously, voice identity gives each project a distinct voice via round-robin assignment from a curated English voice pool.

### How It Works

1. Client calls `speak()` or `generate_audio()` without specifying `voice=`
2. Project is identified via the `project` tool parameter, or falls back to MCP Roots (`list_roots()` with 2s timeout)
3. The next unused voice is assigned from the interleaved pool (alternating gender and accent for maximum contrast)
4. Assignment is persisted to `/data/voice-assignments.json` — survives server restarts

Explicit `voice=` parameter always overrides auto-assignment. Voice pools are cached for 5 minutes (picks up blacklist/engine changes).

**Note:** MCP Roots require stateful Streamable HTTP. With `stateless_http=True` (current default), roots will timeout — the `project` parameter is the primary identification method.

### Configuration

```env
TTS_VOICE_IDENTITY=true                          # Enable project-aware voice assignment
TTS_VOICE_IDENTITY_PREFIXES=af_,am_,bf_,bm_,ef_,em_  # English voice prefixes
TTS_VOICE_IDENTITY_EXCLUDE=af_nicole              # Available explicitly, excluded from auto-assign (whispery)
TTS_VOICE_IDENTITY_FILE=/data/voice-assignments.json  # Persist across restarts
TTS_ANNOUNCE_PROJECT=false                        # Prefix speech with project name
```

### Pool Interleave Order

Voices are interleaved for perceptual diversity: American female → British male → European female → American male → British female → European male. First 6 projects get maximally distinct voices.

### Files

- `voice_identity.py` — Pool filtering, interleaving, round-robin assignment, JSON persistence

## Media Ducking

When `speak()` is called, external audio streams (Firefox, Spotify, etc.) are automatically faded down via PulseAudio before speech begins, then faded back up after the exit tone. This creates a radio-broadcast-interruption effect where the entry tone crossfades over the fading media.

### Audio Timeline

```
0ms     ─ Media starts fading (vol 100%)
150ms   ─ Entry tone starts! Media at ~70%    ← crossfade overlap
294ms   ─ Entry tone ends, media at ~40%
500ms   ─ Media at 0% (ducked)
        ─ [synthesis + playback — media stays silent]
        ─ Exit tone plays (roger/standby)
        ─ Media fades back in over 1000ms
```

The duck fires in `speak()` (before synthesis), the unduck fires in the consumer (after exit/cancel tone). Media stays ducked across multiple queued items — only unducks when the queue is empty or on cancel/shutdown.

### Configuration

```env
TTS_DUCK_MEDIA=true             # Enable/disable (on by default)
TTS_DUCK_FADE_OUT_MS=500        # Fade-out duration (ms)
TTS_DUCK_FADE_IN_MS=1000        # Fade-in duration (slower = natural)
```

### Docker Requirements

The PulseAudio compatibility socket must be mounted in the container:

```yaml
volumes:
  - /run/user/1000/pulse:/run/user/1000/pulse
```

If the socket is missing or pactl fails, ducking silently no-ops — TTS still works normally.

**Why pactl, not pulsectl-asyncio?** PipeWire's PulseAudio compat layer silently drops `sink_input_volume_set` operations from pulsectl's native protocol connection, while `pactl` (using libpulse C library) works reliably. The `pulseaudio-utils` package is installed in the container for this reason.

### Non-fatal Design

All pactl errors are caught and logged. If PulseAudio is unavailable (no socket, wrong permissions, pactl not installed), duck/unduck become no-ops. This ensures TTS never breaks due to ducking failures.

### Files

- `media_duck.py` — `MediaDucker` class (async pactl subprocess volume control with stepped fades)

## Architecture

- `server.py` — FastMCP lifespan, tool definitions, engine setup
- `queue.py` — Producer-consumer speech queue with priority tiers and outcome tracking
- `tones.py` — Tone WAV generator (entry/exit/standby)
- `media_duck.py` — Async PulseAudio volume control for media ducking
- `audio.py` — WAV writing and `pw-play` async wrapper
- `settings.py` — Pydantic settings from env vars (prefix: `TTS_`)
- `engines/` — TTSEngine implementations (kokoro, piper, orpheus)

## `speak()` Progress Lifecycle

`speak()` blocks until playback finishes, reporting progress throughout. **Clients should call it in parallel with other tools** rather than blocking on it alone — the return value is informational (speech_id, duration) and never needed for subsequent reasoning.

### Progress notifications

A background ticker emits MCP progress notifications every ~0.5 seconds during playback. Each notification includes `progress` (0-100), `total` (100), and a `message` string. MCP-aware clients render this as a live progress bar.

| Progress | Phase | Message |
|----------|-------|---------|
| 5% | Entry tone played — audible "I heard you" | `Entry tone` |
| 30% | Synthesis complete | `Synthesizing...` |
| 35% | Enqueued for playback | `Playing audio — you can continue working` |
| 35-99% | Playing — progress tracks elapsed time vs expected duration | `Playing audio — you can continue working` |
| 100% | Playback finished | `Playback complete` |

The entry tone fires before synthesis, covering the 1-3s latency gap. The progress message intentionally says "you can continue working" to signal to LLM clients that they don't need to wait.

### Concurrency guidance

`speak()` blocks for 10-60s for typical text, but audio plays through physical speakers — the return value doesn't feed into subsequent reasoning. The server instructions and tool docstring explicitly tell clients to **call speak in parallel with other tools** in the same message. Tool annotations provide structured signals:

- `openWorldHint=True` — this tool interacts with the physical world (speakers)
- `idempotentHint=True` — safe to retry without side effects beyond replaying audio

`speech_status(speech_id)` is available for checking outcomes after the fact. Possible statuses: `completed`, `playing`, `queued`, `unknown`.

Outcomes are stored in a bounded ring (last 100 items) — old entries are evicted automatically.

## Chunked Synthesis (Pipelined Playback)

Long texts (>= 20 words with sentence boundaries) are automatically split into sentence-sized chunks and pipelined — synthesis of chunk N+1 overlaps playback of chunk N.

```
Single-shot:  [---12s synthesis---][----------48s playback----------]  First audio at T+12s
Chunked:      [~1s synth][play 1][~1s synth][play 2][~1s synth][play 3]...  First audio at T+1s
```

### Why no gaps

Kokoro synthesizes ~4x realtime on CPU, so synthesis always outpaces playback. `pw-play` adds ~10ms startup per chunk — imperceptible. Slower engines (Orpheus) may have brief gaps but still beat the full-synthesis-first approach.

### Short text bypass

Texts under 20 words or without sentence boundaries (`.!?` followed by whitespace) take the single-shot path — zero overhead, identical to pre-chunking behavior.

### Tone behavior

- Entry tone plays once at the start (before first chunk synthesis)
- Exit/standby tones are suppressed between chunks (`suppress_exit_tone` flag on `_WorkItem`)
- Final chunk plays the normal exit tone (roger) or standby tone

### Cancellation in chunked mode

- **Explicit `cancel_speech(speech_id)`** — cancels the returned speech_id (the final chunk). Already-playing earlier chunks finish naturally.
- **MCP disconnect** — the synthesis loop stops (remaining chunks aren't synthesized). Already-enqueued chunks play through.

### Progress lifecycle (chunked)

| Progress | Phase | Message |
|----------|-------|---------|
| 5% | Entry tone played | `Entry tone` |
| 5-30% | Synthesis progress across all chunks | `Synthesizing chunk N/M...` |
| 35% | All chunks enqueued | `Playing audio — you can continue working` |
| 35-99% | Playing — tracks elapsed time vs total duration | `Playing audio — you can continue working` |
| 100% | Final chunk playback finished | `Playback complete` |

### Files

- `server.py` — `split_text()`, `_speak_single()`, `_speak_chunked()`, `_await_with_progress()`
- `queue.py` — `suppress_exit_tone` field on `_WorkItem`

## Cancellation

Speech can be cancelled two ways:

1. **Explicit `cancel_speech(speech_id)`** — kills pw-play immediately, plays the cancel tone, and the consumer moves to the next item.
2. **MCP cancellation** — if a client disconnects or sends `notifications/cancelled`, the `speak()` handler is cancelled but playback continues. The consumer finishes the current audio naturally. This is intentional — audio is already synthesized and playing, so cutting it mid-sentence would be jarring.

When a currently-playing item is explicitly cancelled, pw-play is killed immediately and the cancel tone plays. When a queued item is cancelled, it's removed from the queue without ever playing. The consumer continues to the next item in both cases.

## Graceful Shutdown

On `docker compose down` or `make restart`, the server lets the currently-playing speech finish before stopping — no more mid-sentence cutoffs.

**How it works:** SIGTERM triggers a two-phase shutdown that shares Docker's wall-clock budget:

1. **Phase 1 — Handler drain (3s fixed):** Uvicorn cancels in-flight `speak()` handlers. The handlers re-raise `CancelledError` without touching the consumer — audio keeps playing.
2. **Phase 2 — Queue drain (`shutdown_timeout`, default 30s):** Lifespan finalizer calls `queue.stop()`, which waits for the consumer to finish the current audio. If the timeout expires, it force-cancels.

Docker's `stop_grace_period` must exceed the total: `3s + shutdown_timeout + 5s safety margin` (default 38s). If the max expected audio length exceeds `shutdown_timeout`, increase both `TTS_SHUTDOWN_TIMEOUT` and `stop_grace_period`.

**Entry tone timing:** The Nextel chirp fires immediately when `speak()` is called (before synthesis), acting as an audible "I heard you" that covers the 1-3s synthesis latency. Exit/standby tones still play from the consumer (they depend on queue state after playback).

## Key Design Decisions

- Speech queue is serialized (one playback at a time) but synthesis is parallel
- `speak()` blocks until playback finishes with live progress (5% → 30% → 35-99% → 100%)
- Progress uses a background ticker task, NOT `asyncio.wait_for` polling (see below)
- Entry tone is awaited in `speak()` before synthesis — covers latency gap
- Explicit `cancel_speech()` kills pw-play + plays cancel tone; MCP disconnect lets playback finish
- Consumer directly awaits `play_audio()`; `cancel()` targets the consumer task with `_item_cancelled` flag
- `speak()`'s CancelledError handler does NOT cancel the consumer — only explicit cancel does
- Graceful shutdown is two-phase: 3s handler drain (fixed) + `shutdown_timeout` queue drain, both sequential within Docker's `stop_grace_period`
- Tones are non-fatal: if `pw-play` fails on a tone, speech still plays
- Orpheus uses llama-server (not Ollama) for 15x throughput via continuous batching
- SNAC decoder is lazy-loaded on first Orpheus call to reduce idle memory

### Python 3.13 asyncio.wait_for pitfall

**Do NOT use `asyncio.wait_for(future, timeout)` in a polling loop to track progress.** In Python 3.13, `wait_for` cancels its inner task on timeout. When the inner task is awaiting the same `asyncio.Future` that the consumer will resolve, repeated cancel/re-await cycles cause a stale `CancelledError` to propagate to the consumer task — killing pw-play mid-playback. Instead, use a background `asyncio.create_task` ticker for progress and directly `await` the future for completion.