2 Commits

Author SHA1 Message Date
538b8a513e Replace Ollama with llama-server for 15x Orpheus throughput
Build llama.cpp from source with SM 120 CUDA kernels and FORCE_CUBLAS
for RTX 5070 Blackwell. Rewrite OrpheusEngine to stream tokens via SSE
and decode SNAC in overlapping 28-token batches (4 frames), replacing
the blocking requests+stream:false approach.

Performance: 13.5 → 170-213 tok/s. 100s audio generates in ~48s (2x
faster than realtime). Replaces requests with httpx async client.

Also switch MCP transport to stateless_http mode so container restarts
don't invalidate client sessions.
2026-02-21 21:33:23 -07:00
2fd84f0df7 Initial TTS MCP server with 3 engines
FastMCP 3.0 Streamable HTTP server exposing Piper (Wyoming/Docker),
Kokoro (ONNX), and Orpheus (Ollama+SNAC) as MCP tools. Includes a
FIFO speech queue so concurrent agents don't talk over each other —
waiting callers get queue position updates via ctx.info().

Tools: speak, generate_audio, list_voices, list_engines
Resource: audio://recent
2026-02-20 18:10:15 -07:00