mcspeak

Go to file

Ryan Malloy 538b8a513e Replace Ollama with llama-server for 15x Orpheus throughput

Build llama.cpp from source with SM 120 CUDA kernels and FORCE_CUBLAS
for RTX 5070 Blackwell. Rewrite OrpheusEngine to stream tokens via SSE
and decode SNAC in overlapping 28-token batches (4 frames), replacing
the blocking requests+stream:false approach.

Performance: 13.5 → 170-213 tok/s. 100s audio generates in ~48s (2x
faster than realtime). Replaces requests with httpx async client.

Also switch MCP transport to stateless_http mode so container restarts
don't invalidate client sessions.

2026-02-21 21:33:23 -07:00

src/tts_mcp

Replace Ollama with llama-server for 15x Orpheus throughput

2026-02-21 21:33:23 -07:00

.gitignore

Initial TTS MCP server with 3 engines

2026-02-20 18:10:15 -07:00

docker-compose.yml

Replace Ollama with llama-server for 15x Orpheus throughput

2026-02-21 21:33:23 -07:00

Dockerfile

Add Docker deployment with Caddy HTTPS reverse proxy