Pre-compile macOS recorder at startup; add PLATFORMS.md

- server lifespan compiles the Swift recorder on macOS at startup so the first
  listen() isn't slowed by swiftc and build errors surface early (Linux/Windows
  skip this).
- PLATFORMS.md documents the four-touch-point audio backend, the macOS setup
  (built-ins + uv, no Docker), and the Windows plan (winsound + sounddevice).
This commit is contained in:
Ryan Malloy 2026-07-03 22:26:40 -06:00
parent 3d5d8dad36
commit 7910f36395
2 changed files with 72 additions and 0 deletions

62
PLATFORMS.md Normal file
View File

@ -0,0 +1,62 @@
# Platform support
McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the
**audio I/O** is OS-specific. Everything the mic/speaker touches goes through
four subprocess touch-points in `platform_audio.py`, so porting to a new OS is
adding a branch there — the queue, VAD, secretary, tones, and progress logic
never change. Every backend's recorder emits the **same wire format** (raw
`s16`, mono, 16 kHz, streamed to stdout), so the VAD frame loop in `audio.py`
is byte-for-byte identical everywhere.
| Touch-point | Linux (default) | macOS | Windows (planned) |
|---|---|---|---|
| Play WAV | `pw-play` | `afplay` | `winsound.PlaySound` (stdlib, in-process) |
| Record → s16 PCM/stdout | `pw-record -` | Swift/AVFoundation binary | `sounddevice` (PortAudio) or `ffmpeg -f dshow` |
| Transcode / resample | `ffmpeg` | `afconvert` (m4a) / `ffmpeg` | `ffmpeg` or in-process `soundfile` |
| Duck other apps | `pactl` (per-app) | none — deliberate no-op¹ | `pycaw` (per-app) |
¹ macOS ducking is intentionally a no-op: the only built-in volume control
(`osascript`) is system-wide and would dim our own `afplay` voice. Per-app
ducking needs CoreAudio, a future native-helper task.
## Linux (current production)
Runs in Docker with the host PipeWire socket bind-mounted
(`/run/user/1000/pulse`). `make up` builds + starts. This is the tested,
shipping path — unchanged by the cross-platform work.
## macOS (audio backend done; native run TBD)
Status: the four audio touch-points are implemented. Headless mic capture is
**verified** on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet
done: running the server natively + a live speech test.
Requirements (all present on the target Mac, all built-in except the venv):
- **Xcode command-line tools** (`xcode-select --install`) for `swiftc` — the
recorder is compiled once, cached in `~/.cache/mcspeak/`.
- `afplay`, `afconvert`, `osascript` ship with macOS.
- `uv` for the venv (no Docker — Docker Desktop containers can't reach
CoreAudio, which is why the Mac already runs its ML services natively).
Deployment sketch (native, as the console user who owns the audio session):
```bash
uv sync # or a minimal env for listen-only testing:
# numpy webrtcvad-wheels httpx pydantic-settings
uv run mcspeak # server picks the macOS backend automatically
```
`generate_audio` in mp3/ogg/flac needs `brew install ffmpeg`; wav and m4a work
with built-ins alone.
## Windows (designed, not implemented)
The abstraction has the slots; the work is filling the four touch-points:
- **Play**: `winsound.PlaySound` — Python stdlib, zero deps, WAV only.
- **Record**: the real gap (same as macOS had). `sounddevice` (PortAudio pip
wheel) is the cleanest — it streams numpy frames that feed `webrtcvad`
directly; `ffmpeg -f dshow` is the subprocess alternative.
- **Transcode**: `ffmpeg`, or do it in-process with `soundfile`/`numpy`.
- **Duck**: `pycaw` (per-app WASAPI), or degrade to the no-op.
`sounddevice` could cover both Windows *and* macOS recording with one library
if we ever want to drop the Swift compile step — macOS stays built-ins-only
for now to keep it zero-dependency.

View File

@ -147,6 +147,16 @@ async def app_lifespan(server: FastMCP):
health = await eng.check_health()
print(f" {name}: {health['status']}", file=sys.stderr)
# On macOS, pre-compile the Swift mic recorder now so the first listen()
# isn't slowed by swiftc and any build error surfaces here, not mid-call.
from . import platform_audio
if platform_audio.IS_MACOS:
try:
rec = platform_audio.ensure_macos_recorder()
print(f" macOS mic recorder ready: {rec}", file=sys.stderr)
except platform_audio.AudioBackendError as e:
print(f" macOS mic recorder unavailable (listen will fail): {e}", file=sys.stderr)
# Generate tones and resolve which ones to use
tone_paths = generate_tones(settings.output_dir)
entry_tone = resolve_tone(settings.entry_tone, tone_paths, "entry_tone", default="heartbeat")