Build llama.cpp from source with SM 120 CUDA kernels and FORCE_CUBLAS
for RTX 5070 Blackwell. Rewrite OrpheusEngine to stream tokens via SSE
and decode SNAC in overlapping 28-token batches (4 frames), replacing
the blocking requests+stream:false approach.
Performance: 13.5 → 170-213 tok/s. 100s audio generates in ~48s (2x
faster than realtime). Replaces requests with httpx async client.
Also switch MCP transport to stateless_http mode so container restarts
don't invalidate client sessions.
SNAC + torch no longer load at startup — deferred to first Orpheus
call via double-checked locking. Startup drops from ~13s to 0.5s,
idle RAM reduced by ~200MB. OrpheusEngine constructor no longer
takes snac_model; it self-loads on demand.
CPU-only torch image (~180MB vs 873MB CUDA), PipeWire socket
passthrough for audio playback, SNAC HuggingFace cache volume.
Served at voice.l.supported.systems via caddy-docker-proxy.
