From d10eb9ab57d709a3569b2b53867c890931198686 Mon Sep 17 00:00:00 2001
From: Ryan Malloy <ryan@supported.systems>
Date: Mon, 2 Mar 2026 18:12:31 -0700
Subject: [PATCH] Fix graceful shutdown: let consumer finish audio on SIGTERM

speak()'s CancelledError handler was calling queue.cancel() which killed
pw-play mid-sentence during container restarts. Now it just re-raises,
letting the consumer finish naturally. Explicit cancel_speech() still
kills playback immediately.

Also override FastMCP's timeout_graceful_shutdown=0 in uvicorn config
so request handlers get shutdown_timeout seconds before cancellation.
---
 CLAUDE.md               | 13 +++++++------
 src/tts_mcp/__main__.py |  4 ++++
 src/tts_mcp/server.py   |  7 ++++---
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index e56233d..b1e42aa 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -113,10 +113,10 @@ Outcomes are stored in a bounded ring (last 100 items) — old entries are evict
 
 Speech can be cancelled two ways:
 
-1. **MCP cancellation** — the client sends `notifications/cancelled` for an in-flight `speak()` call. FastMCP throws `CancelledError` into the tool, which triggers `queue.cancel(speech_id)`.
-2. **Explicit `cancel_speech(speech_id)`** — a separate tool that cancels any queued or playing item.
+1. **Explicit `cancel_speech(speech_id)`** — kills pw-play immediately, plays the cancel tone, and the consumer moves to the next item.
+2. **MCP cancellation** — if a client disconnects or sends `notifications/cancelled`, the `speak()` handler is cancelled but playback continues. The consumer finishes the current audio naturally. This is intentional — audio is already synthesized and playing, so cutting it mid-sentence would be jarring.
 
-When a currently-playing item is cancelled, pw-play is killed immediately and the cancel tone plays. When a queued item is cancelled, it's removed from the queue without ever playing. The consumer continues to the next item in both cases.
+When a currently-playing item is explicitly cancelled, pw-play is killed immediately and the cancel tone plays. When a queued item is cancelled, it's removed from the queue without ever playing. The consumer continues to the next item in both cases.
 
 ## Graceful Shutdown
 
@@ -132,9 +132,10 @@ On `docker compose down` or `make restart`, the server lets the currently-playin
 - `speak()` blocks until playback finishes with live progress (5% → 30% → 35-99% → 100%)
 - Progress uses a background ticker task, NOT `asyncio.wait_for` polling (see below)
 - Entry tone is awaited in `speak()` before synthesis — covers latency gap
-- Cancellation via `cancel_speech()` or MCP `notifications/cancelled` kills pw-play + plays cancel tone
-- Consumer directly awaits `play_audio()`; cancel() targets the consumer task with `_item_cancelled` flag
-- Graceful shutdown waits for current speech, then drains pending items with error outcomes
+- Explicit `cancel_speech()` kills pw-play + plays cancel tone; MCP disconnect lets playback finish
+- Consumer directly awaits `play_audio()`; `cancel()` targets the consumer task with `_item_cancelled` flag
+- `speak()`'s CancelledError handler does NOT cancel the consumer — only explicit cancel does
+- Graceful shutdown: uvicorn waits `shutdown_timeout` for handlers, consumer finishes audio naturally, then lifespan runs `queue.stop()`
 - Tones are non-fatal: if `pw-play` fails on a tone, speech still plays
 - Orpheus uses llama-server (not Ollama) for 15x throughput via continuous batching
 - SNAC decoder is lazy-loaded on first Orpheus call to reduce idle memory
diff --git a/src/tts_mcp/__main__.py b/src/tts_mcp/__main__.py
index 27ec9e1..a9dab94 100644
--- a/src/tts_mcp/__main__.py
+++ b/src/tts_mcp/__main__.py
@@ -10,6 +10,10 @@ def main():
         host=settings.host,
         port=settings.port,
         stateless_http=True,
+        # Override FastMCP's default timeout_graceful_shutdown=0 so uvicorn
+        # lets in-flight speak() calls finish before cancelling them. Must
+        # be shorter than Docker's stop_grace_period (35s) to avoid SIGKILL.
+        uvicorn_config={"timeout_graceful_shutdown": settings.shutdown_timeout},
     )
 
 
diff --git a/src/tts_mcp/server.py b/src/tts_mcp/server.py
index dc81bed..8121d8b 100644
--- a/src/tts_mcp/server.py
+++ b/src/tts_mcp/server.py
@@ -257,9 +257,10 @@ async def speak(
         return outcome
 
     except asyncio.CancelledError:
-        # MCP client cancelled the tool call — cancel in-flight playback
-        if speech_id:
-            queue.cancel(speech_id)
+        # speak() was cancelled (server shutdown or MCP client disconnect).
+        # Do NOT cancel the consumer — let it finish the current audio.
+        # Shutdown: queue.stop() in the lifespan finalizer handles graceful drain.
+        # Client cancel: use cancel_speech(speech_id) to explicitly stop playback.
         raise