Ryan Malloy
7910f36395
- server lifespan compiles the Swift recorder on macOS at startup so the first listen() isn't slowed by swiftc and build errors surface early (Linux/Windows skip this). - PLATFORMS.md documents the four-touch-point audio backend, the macOS setup (built-ins + uv, no Docker), and the Windows plan (winsound + sounddevice).
63 lines
3.2 KiB
Markdown
63 lines
3.2 KiB
Markdown
# Platform support
|
|
|
|
McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the
|
|
**audio I/O** is OS-specific. Everything the mic/speaker touches goes through
|
|
four subprocess touch-points in `platform_audio.py`, so porting to a new OS is
|
|
adding a branch there — the queue, VAD, secretary, tones, and progress logic
|
|
never change. Every backend's recorder emits the **same wire format** (raw
|
|
`s16`, mono, 16 kHz, streamed to stdout), so the VAD frame loop in `audio.py`
|
|
is byte-for-byte identical everywhere.
|
|
|
|
| Touch-point | Linux (default) | macOS | Windows (planned) |
|
|
|---|---|---|---|
|
|
| Play WAV | `pw-play` | `afplay` | `winsound.PlaySound` (stdlib, in-process) |
|
|
| Record → s16 PCM/stdout | `pw-record -` | Swift/AVFoundation binary | `sounddevice` (PortAudio) or `ffmpeg -f dshow` |
|
|
| Transcode / resample | `ffmpeg` | `afconvert` (m4a) / `ffmpeg` | `ffmpeg` or in-process `soundfile` |
|
|
| Duck other apps | `pactl` (per-app) | none — deliberate no-op¹ | `pycaw` (per-app) |
|
|
|
|
¹ macOS ducking is intentionally a no-op: the only built-in volume control
|
|
(`osascript`) is system-wide and would dim our own `afplay` voice. Per-app
|
|
ducking needs CoreAudio, a future native-helper task.
|
|
|
|
## Linux (current production)
|
|
|
|
Runs in Docker with the host PipeWire socket bind-mounted
|
|
(`/run/user/1000/pulse`). `make up` builds + starts. This is the tested,
|
|
shipping path — unchanged by the cross-platform work.
|
|
|
|
## macOS (audio backend done; native run TBD)
|
|
|
|
Status: the four audio touch-points are implemented. Headless mic capture is
|
|
**verified** on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet
|
|
done: running the server natively + a live speech test.
|
|
|
|
Requirements (all present on the target Mac, all built-in except the venv):
|
|
- **Xcode command-line tools** (`xcode-select --install`) for `swiftc` — the
|
|
recorder is compiled once, cached in `~/.cache/mcspeak/`.
|
|
- `afplay`, `afconvert`, `osascript` ship with macOS.
|
|
- `uv` for the venv (no Docker — Docker Desktop containers can't reach
|
|
CoreAudio, which is why the Mac already runs its ML services natively).
|
|
|
|
Deployment sketch (native, as the console user who owns the audio session):
|
|
```bash
|
|
uv sync # or a minimal env for listen-only testing:
|
|
# numpy webrtcvad-wheels httpx pydantic-settings
|
|
uv run mcspeak # server picks the macOS backend automatically
|
|
```
|
|
`generate_audio` in mp3/ogg/flac needs `brew install ffmpeg`; wav and m4a work
|
|
with built-ins alone.
|
|
|
|
## Windows (designed, not implemented)
|
|
|
|
The abstraction has the slots; the work is filling the four touch-points:
|
|
- **Play**: `winsound.PlaySound` — Python stdlib, zero deps, WAV only.
|
|
- **Record**: the real gap (same as macOS had). `sounddevice` (PortAudio pip
|
|
wheel) is the cleanest — it streams numpy frames that feed `webrtcvad`
|
|
directly; `ffmpeg -f dshow` is the subprocess alternative.
|
|
- **Transcode**: `ffmpeg`, or do it in-process with `soundfile`/`numpy`.
|
|
- **Duck**: `pycaw` (per-app WASAPI), or degrade to the no-op.
|
|
|
|
`sounddevice` could cover both Windows *and* macOS recording with one library
|
|
if we ever want to drop the Swift compile step — macOS stays built-ins-only
|
|
for now to keep it zero-dependency.
|