Pre-compile macOS recorder at startup; add PLATFORMS.md
- server lifespan compiles the Swift recorder on macOS at startup so the first listen() isn't slowed by swiftc and build errors surface early (Linux/Windows skip this). - PLATFORMS.md documents the four-touch-point audio backend, the macOS setup (built-ins + uv, no Docker), and the Windows plan (winsound + sounddevice).
This commit is contained in:
parent
3d5d8dad36
commit
7910f36395
62
PLATFORMS.md
Normal file
62
PLATFORMS.md
Normal file
@ -0,0 +1,62 @@
|
||||
# Platform support
|
||||
|
||||
McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the
|
||||
**audio I/O** is OS-specific. Everything the mic/speaker touches goes through
|
||||
four subprocess touch-points in `platform_audio.py`, so porting to a new OS is
|
||||
adding a branch there — the queue, VAD, secretary, tones, and progress logic
|
||||
never change. Every backend's recorder emits the **same wire format** (raw
|
||||
`s16`, mono, 16 kHz, streamed to stdout), so the VAD frame loop in `audio.py`
|
||||
is byte-for-byte identical everywhere.
|
||||
|
||||
| Touch-point | Linux (default) | macOS | Windows (planned) |
|
||||
|---|---|---|---|
|
||||
| Play WAV | `pw-play` | `afplay` | `winsound.PlaySound` (stdlib, in-process) |
|
||||
| Record → s16 PCM/stdout | `pw-record -` | Swift/AVFoundation binary | `sounddevice` (PortAudio) or `ffmpeg -f dshow` |
|
||||
| Transcode / resample | `ffmpeg` | `afconvert` (m4a) / `ffmpeg` | `ffmpeg` or in-process `soundfile` |
|
||||
| Duck other apps | `pactl` (per-app) | none — deliberate no-op¹ | `pycaw` (per-app) |
|
||||
|
||||
¹ macOS ducking is intentionally a no-op: the only built-in volume control
|
||||
(`osascript`) is system-wide and would dim our own `afplay` voice. Per-app
|
||||
ducking needs CoreAudio, a future native-helper task.
|
||||
|
||||
## Linux (current production)
|
||||
|
||||
Runs in Docker with the host PipeWire socket bind-mounted
|
||||
(`/run/user/1000/pulse`). `make up` builds + starts. This is the tested,
|
||||
shipping path — unchanged by the cross-platform work.
|
||||
|
||||
## macOS (audio backend done; native run TBD)
|
||||
|
||||
Status: the four audio touch-points are implemented. Headless mic capture is
|
||||
**verified** on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet
|
||||
done: running the server natively + a live speech test.
|
||||
|
||||
Requirements (all present on the target Mac, all built-in except the venv):
|
||||
- **Xcode command-line tools** (`xcode-select --install`) for `swiftc` — the
|
||||
recorder is compiled once, cached in `~/.cache/mcspeak/`.
|
||||
- `afplay`, `afconvert`, `osascript` ship with macOS.
|
||||
- `uv` for the venv (no Docker — Docker Desktop containers can't reach
|
||||
CoreAudio, which is why the Mac already runs its ML services natively).
|
||||
|
||||
Deployment sketch (native, as the console user who owns the audio session):
|
||||
```bash
|
||||
uv sync # or a minimal env for listen-only testing:
|
||||
# numpy webrtcvad-wheels httpx pydantic-settings
|
||||
uv run mcspeak # server picks the macOS backend automatically
|
||||
```
|
||||
`generate_audio` in mp3/ogg/flac needs `brew install ffmpeg`; wav and m4a work
|
||||
with built-ins alone.
|
||||
|
||||
## Windows (designed, not implemented)
|
||||
|
||||
The abstraction has the slots; the work is filling the four touch-points:
|
||||
- **Play**: `winsound.PlaySound` — Python stdlib, zero deps, WAV only.
|
||||
- **Record**: the real gap (same as macOS had). `sounddevice` (PortAudio pip
|
||||
wheel) is the cleanest — it streams numpy frames that feed `webrtcvad`
|
||||
directly; `ffmpeg -f dshow` is the subprocess alternative.
|
||||
- **Transcode**: `ffmpeg`, or do it in-process with `soundfile`/`numpy`.
|
||||
- **Duck**: `pycaw` (per-app WASAPI), or degrade to the no-op.
|
||||
|
||||
`sounddevice` could cover both Windows *and* macOS recording with one library
|
||||
if we ever want to drop the Swift compile step — macOS stays built-ins-only
|
||||
for now to keep it zero-dependency.
|
||||
@ -147,6 +147,16 @@ async def app_lifespan(server: FastMCP):
|
||||
health = await eng.check_health()
|
||||
print(f" {name}: {health['status']}", file=sys.stderr)
|
||||
|
||||
# On macOS, pre-compile the Swift mic recorder now so the first listen()
|
||||
# isn't slowed by swiftc and any build error surfaces here, not mid-call.
|
||||
from . import platform_audio
|
||||
if platform_audio.IS_MACOS:
|
||||
try:
|
||||
rec = platform_audio.ensure_macos_recorder()
|
||||
print(f" macOS mic recorder ready: {rec}", file=sys.stderr)
|
||||
except platform_audio.AudioBackendError as e:
|
||||
print(f" macOS mic recorder unavailable (listen will fail): {e}", file=sys.stderr)
|
||||
|
||||
# Generate tones and resolve which ones to use
|
||||
tone_paths = generate_tones(settings.output_dir)
|
||||
entry_tone = resolve_tone(settings.entry_tone, tone_paths, "entry_tone", default="heartbeat")
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user