diff --git a/PLATFORMS.md b/PLATFORMS.md new file mode 100644 index 0000000..1af3130 --- /dev/null +++ b/PLATFORMS.md @@ -0,0 +1,62 @@ +# Platform support + +McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the +**audio I/O** is OS-specific. Everything the mic/speaker touches goes through +four subprocess touch-points in `platform_audio.py`, so porting to a new OS is +adding a branch there — the queue, VAD, secretary, tones, and progress logic +never change. Every backend's recorder emits the **same wire format** (raw +`s16`, mono, 16 kHz, streamed to stdout), so the VAD frame loop in `audio.py` +is byte-for-byte identical everywhere. + +| Touch-point | Linux (default) | macOS | Windows (planned) | +|---|---|---|---| +| Play WAV | `pw-play` | `afplay` | `winsound.PlaySound` (stdlib, in-process) | +| Record → s16 PCM/stdout | `pw-record -` | Swift/AVFoundation binary | `sounddevice` (PortAudio) or `ffmpeg -f dshow` | +| Transcode / resample | `ffmpeg` | `afconvert` (m4a) / `ffmpeg` | `ffmpeg` or in-process `soundfile` | +| Duck other apps | `pactl` (per-app) | none — deliberate no-op¹ | `pycaw` (per-app) | + +¹ macOS ducking is intentionally a no-op: the only built-in volume control +(`osascript`) is system-wide and would dim our own `afplay` voice. Per-app +ducking needs CoreAudio, a future native-helper task. + +## Linux (current production) + +Runs in Docker with the host PipeWire socket bind-mounted +(`/run/user/1000/pulse`). `make up` builds + starts. This is the tested, +shipping path — unchanged by the cross-platform work. + +## macOS (audio backend done; native run TBD) + +Status: the four audio touch-points are implemented. Headless mic capture is +**verified** on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet +done: running the server natively + a live speech test. + +Requirements (all present on the target Mac, all built-in except the venv): +- **Xcode command-line tools** (`xcode-select --install`) for `swiftc` — the + recorder is compiled once, cached in `~/.cache/mcspeak/`. +- `afplay`, `afconvert`, `osascript` ship with macOS. +- `uv` for the venv (no Docker — Docker Desktop containers can't reach + CoreAudio, which is why the Mac already runs its ML services natively). + +Deployment sketch (native, as the console user who owns the audio session): +```bash +uv sync # or a minimal env for listen-only testing: + # numpy webrtcvad-wheels httpx pydantic-settings +uv run mcspeak # server picks the macOS backend automatically +``` +`generate_audio` in mp3/ogg/flac needs `brew install ffmpeg`; wav and m4a work +with built-ins alone. + +## Windows (designed, not implemented) + +The abstraction has the slots; the work is filling the four touch-points: +- **Play**: `winsound.PlaySound` — Python stdlib, zero deps, WAV only. +- **Record**: the real gap (same as macOS had). `sounddevice` (PortAudio pip + wheel) is the cleanest — it streams numpy frames that feed `webrtcvad` + directly; `ffmpeg -f dshow` is the subprocess alternative. +- **Transcode**: `ffmpeg`, or do it in-process with `soundfile`/`numpy`. +- **Duck**: `pycaw` (per-app WASAPI), or degrade to the no-op. + +`sounddevice` could cover both Windows *and* macOS recording with one library +if we ever want to drop the Swift compile step — macOS stays built-ins-only +for now to keep it zero-dependency. diff --git a/src/mcspeak/server.py b/src/mcspeak/server.py index 3342177..a1fc7c0 100644 --- a/src/mcspeak/server.py +++ b/src/mcspeak/server.py @@ -147,6 +147,16 @@ async def app_lifespan(server: FastMCP): health = await eng.check_health() print(f" {name}: {health['status']}", file=sys.stderr) + # On macOS, pre-compile the Swift mic recorder now so the first listen() + # isn't slowed by swiftc and any build error surfaces here, not mid-call. + from . import platform_audio + if platform_audio.IS_MACOS: + try: + rec = platform_audio.ensure_macos_recorder() + print(f" macOS mic recorder ready: {rec}", file=sys.stderr) + except platform_audio.AudioBackendError as e: + print(f" macOS mic recorder unavailable (listen will fail): {e}", file=sys.stderr) + # Generate tones and resolve which ones to use tone_paths = generate_tones(settings.output_dir) entry_tone = resolve_tone(settings.entry_tone, tone_paths, "entry_tone", default="heartbeat")