warehack.ing/mcspeak
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
mcspeak/PLATFORMS.md
Ryan Malloy 7910f36395 Pre-compile macOS recorder at startup; add PLATFORMS.md 
- server lifespan compiles the Swift recorder on macOS at startup so the first
  listen() isn't slowed by swiftc and build errors surface early (Linux/Windows
  skip this).
- PLATFORMS.md documents the four-touch-point audio backend, the macOS setup
  (built-ins + uv, no Docker), and the Windows plan (winsound + sounddevice).
2026-07-03 22:26:40 -06:00

3.2 KiB
Raw Blame History

Platform support

McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the audio I/O is OS-specific. Everything the mic/speaker touches goes through four subprocess touch-points in platform_audio.py, so porting to a new OS is adding a branch there — the queue, VAD, secretary, tones, and progress logic never change. Every backend's recorder emits the same wire format (raw s16, mono, 16 kHz, streamed to stdout), so the VAD frame loop in audio.py is byte-for-byte identical everywhere.

Touch-point Linux (default) macOS Windows (planned)
Play WAV pw-play afplay winsound.PlaySound (stdlib, in-process)
Record → s16 PCM/stdout pw-record - Swift/AVFoundation binary sounddevice (PortAudio) or ffmpeg -f dshow
Transcode / resample ffmpeg afconvert (m4a) / ffmpeg ffmpeg or in-process soundfile
Duck other apps pactl (per-app) none — deliberate no-op¹ pycaw (per-app)

¹ macOS ducking is intentionally a no-op: the only built-in volume control (osascript) is system-wide and would dim our own afplay voice. Per-app ducking needs CoreAudio, a future native-helper task.

Linux (current production)

Runs in Docker with the host PipeWire socket bind-mounted (/run/user/1000/pulse). make up builds + starts. This is the tested, shipping path — unchanged by the cross-platform work.

macOS (audio backend done; native run TBD)

Status: the four audio touch-points are implemented. Headless mic capture is verified on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet done: running the server natively + a live speech test.

Requirements (all present on the target Mac, all built-in except the venv):

  • Xcode command-line tools (xcode-select --install) for swiftc — the recorder is compiled once, cached in ~/.cache/mcspeak/.
  • afplay, afconvert, osascript ship with macOS.
  • uv for the venv (no Docker — Docker Desktop containers can't reach CoreAudio, which is why the Mac already runs its ML services natively).

Deployment sketch (native, as the console user who owns the audio session):

uv sync                       # or a minimal env for listen-only testing:
                              # numpy webrtcvad-wheels httpx pydantic-settings
uv run mcspeak                # server picks the macOS backend automatically

generate_audio in mp3/ogg/flac needs brew install ffmpeg; wav and m4a work with built-ins alone.

Windows (designed, not implemented)

The abstraction has the slots; the work is filling the four touch-points:

  • Play: winsound.PlaySound — Python stdlib, zero deps, WAV only.
  • Record: the real gap (same as macOS had). sounddevice (PortAudio pip wheel) is the cleanest — it streams numpy frames that feed webrtcvad directly; ffmpeg -f dshow is the subprocess alternative.
  • Transcode: ffmpeg, or do it in-process with soundfile/numpy.
  • Duck: pycaw (per-app WASAPI), or degrade to the no-op.

sounddevice could cover both Windows and macOS recording with one library if we ever want to drop the Swift compile step — macOS stays built-ins-only for now to keep it zero-dependency.