Pre-compile macOS recorder at startup; add PLATFORMS.md
- server lifespan compiles the Swift recorder on macOS at startup so the first listen() isn't slowed by swiftc and build errors surface early (Linux/Windows skip this). - PLATFORMS.md documents the four-touch-point audio backend, the macOS setup (built-ins + uv, no Docker), and the Windows plan (winsound + sounddevice).
This commit is contained in:
parent
3d5d8dad36
commit
7910f36395
62
PLATFORMS.md
Normal file
62
PLATFORMS.md
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
# Platform support
|
||||||
|
|
||||||
|
McSpeak's engines (Kokoro/Piper/Orpheus) are cross-platform; only the
|
||||||
|
**audio I/O** is OS-specific. Everything the mic/speaker touches goes through
|
||||||
|
four subprocess touch-points in `platform_audio.py`, so porting to a new OS is
|
||||||
|
adding a branch there — the queue, VAD, secretary, tones, and progress logic
|
||||||
|
never change. Every backend's recorder emits the **same wire format** (raw
|
||||||
|
`s16`, mono, 16 kHz, streamed to stdout), so the VAD frame loop in `audio.py`
|
||||||
|
is byte-for-byte identical everywhere.
|
||||||
|
|
||||||
|
| Touch-point | Linux (default) | macOS | Windows (planned) |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Play WAV | `pw-play` | `afplay` | `winsound.PlaySound` (stdlib, in-process) |
|
||||||
|
| Record → s16 PCM/stdout | `pw-record -` | Swift/AVFoundation binary | `sounddevice` (PortAudio) or `ffmpeg -f dshow` |
|
||||||
|
| Transcode / resample | `ffmpeg` | `afconvert` (m4a) / `ffmpeg` | `ffmpeg` or in-process `soundfile` |
|
||||||
|
| Duck other apps | `pactl` (per-app) | none — deliberate no-op¹ | `pycaw` (per-app) |
|
||||||
|
|
||||||
|
¹ macOS ducking is intentionally a no-op: the only built-in volume control
|
||||||
|
(`osascript`) is system-wide and would dim our own `afplay` voice. Per-app
|
||||||
|
ducking needs CoreAudio, a future native-helper task.
|
||||||
|
|
||||||
|
## Linux (current production)
|
||||||
|
|
||||||
|
Runs in Docker with the host PipeWire socket bind-mounted
|
||||||
|
(`/run/user/1000/pulse`). `make up` builds + starts. This is the tested,
|
||||||
|
shipping path — unchanged by the cross-platform work.
|
||||||
|
|
||||||
|
## macOS (audio backend done; native run TBD)
|
||||||
|
|
||||||
|
Status: the four audio touch-points are implemented. Headless mic capture is
|
||||||
|
**verified** on an Apple-Silicon Mac over SSH (TCC does not block it). Not yet
|
||||||
|
done: running the server natively + a live speech test.
|
||||||
|
|
||||||
|
Requirements (all present on the target Mac, all built-in except the venv):
|
||||||
|
- **Xcode command-line tools** (`xcode-select --install`) for `swiftc` — the
|
||||||
|
recorder is compiled once, cached in `~/.cache/mcspeak/`.
|
||||||
|
- `afplay`, `afconvert`, `osascript` ship with macOS.
|
||||||
|
- `uv` for the venv (no Docker — Docker Desktop containers can't reach
|
||||||
|
CoreAudio, which is why the Mac already runs its ML services natively).
|
||||||
|
|
||||||
|
Deployment sketch (native, as the console user who owns the audio session):
|
||||||
|
```bash
|
||||||
|
uv sync # or a minimal env for listen-only testing:
|
||||||
|
# numpy webrtcvad-wheels httpx pydantic-settings
|
||||||
|
uv run mcspeak # server picks the macOS backend automatically
|
||||||
|
```
|
||||||
|
`generate_audio` in mp3/ogg/flac needs `brew install ffmpeg`; wav and m4a work
|
||||||
|
with built-ins alone.
|
||||||
|
|
||||||
|
## Windows (designed, not implemented)
|
||||||
|
|
||||||
|
The abstraction has the slots; the work is filling the four touch-points:
|
||||||
|
- **Play**: `winsound.PlaySound` — Python stdlib, zero deps, WAV only.
|
||||||
|
- **Record**: the real gap (same as macOS had). `sounddevice` (PortAudio pip
|
||||||
|
wheel) is the cleanest — it streams numpy frames that feed `webrtcvad`
|
||||||
|
directly; `ffmpeg -f dshow` is the subprocess alternative.
|
||||||
|
- **Transcode**: `ffmpeg`, or do it in-process with `soundfile`/`numpy`.
|
||||||
|
- **Duck**: `pycaw` (per-app WASAPI), or degrade to the no-op.
|
||||||
|
|
||||||
|
`sounddevice` could cover both Windows *and* macOS recording with one library
|
||||||
|
if we ever want to drop the Swift compile step — macOS stays built-ins-only
|
||||||
|
for now to keep it zero-dependency.
|
||||||
@ -147,6 +147,16 @@ async def app_lifespan(server: FastMCP):
|
|||||||
health = await eng.check_health()
|
health = await eng.check_health()
|
||||||
print(f" {name}: {health['status']}", file=sys.stderr)
|
print(f" {name}: {health['status']}", file=sys.stderr)
|
||||||
|
|
||||||
|
# On macOS, pre-compile the Swift mic recorder now so the first listen()
|
||||||
|
# isn't slowed by swiftc and any build error surfaces here, not mid-call.
|
||||||
|
from . import platform_audio
|
||||||
|
if platform_audio.IS_MACOS:
|
||||||
|
try:
|
||||||
|
rec = platform_audio.ensure_macos_recorder()
|
||||||
|
print(f" macOS mic recorder ready: {rec}", file=sys.stderr)
|
||||||
|
except platform_audio.AudioBackendError as e:
|
||||||
|
print(f" macOS mic recorder unavailable (listen will fail): {e}", file=sys.stderr)
|
||||||
|
|
||||||
# Generate tones and resolve which ones to use
|
# Generate tones and resolve which ones to use
|
||||||
tone_paths = generate_tones(settings.output_dir)
|
tone_paths = generate_tones(settings.output_dir)
|
||||||
entry_tone = resolve_tone(settings.entry_tone, tone_paths, "entry_tone", default="heartbeat")
|
entry_tone = resolve_tone(settings.entry_tone, tone_paths, "entry_tone", default="heartbeat")
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user