Lays the groundwork for the macOS (AppKit) and iOS (SwiftUI) clients with a shared
Swift core wrapping the C ABI, mirroring the proven Windows VoiceCat.Interop layer.
Architecture decision: macOS UI = AppKit (not SwiftUI) for the most mature VoiceOver
accessibility story — same rationale as the Windows client's WinForms-over-WinUI-3
decision. iOS stays SwiftUI. Recorded in docs/roadmap.md §2.
Build infrastructure (Phase 0):
- clients/apple/scripts/build-xcframework.sh: runs cmake --preset apple-dev, merges
libvoicecat.a + 107 vcpkg static deps into a single ~30 MB fat static library
(libvoicecat-fat.a) via libtool -static (SPM binary targets link one .a per slice),
stages voicecat.h + a generated module.modulemap (module VoiceCatC) into the headers,
runs xcodebuild -create-xcframework -> clients/apple/VoiceCatCore.xcframework.
VoiceCatCore Swift Package (Phase 1):
- Package.swift: binary target (VoiceCatCoreXCF) + library (VoiceCatCore) + test target.
- Sources/VoiceCatCore/: 7 files mirroring the C# VoiceCat.Interop patterns adapted to
Swift native C interop — Enums (9 Swift mirrors of C enums, UInt32-backed), Config,
Event (copies ev.text to String inside the callback — the #1 lifetime rule), Models
(10 Swift value types), Marshaling (C arrays -> Swift + immediate vc_free_*),
Callbacks (@convention(c) + Unmanaged.passUnretained, the Swift analog of C#'s
[UnmanagedCallersOnly] + GCHandle), VoiceCatClient (owns vc_client* as OpaquePointer,
all 38 C ABI functions, deinit -> vc_client_destroy then frees config CStrings, event
delivery on main queue via coalesced DispatchQueue.main drain).
Tests — 6/6 green (swift test against a real voicecat-server):
- testConnectTofuAuthListChannelsRoundTrips, testAdminChannelCrudAccountCrudRoundTrips,
testScreenAudioStreamStartsAndStops, testPerStreamRecvControlsRoundTrip, plus two
static smoke tests. Catches Swift-specific interop bugs (@convention(c) callback
lifetime, Unmanaged pointer resolution, CString memory management, enum raw-value
bridging, struct layout) that C++ ctest cannot. C++ suite still 21/21 green.
Docs updated (house rule): tech-stack.md §2, architecture.md §4, roadmap.md M4 + §2,
clients/apple/README.md (full rewrite), PROGRESS.md, .gitignore.
Three reported bugs traced to one root cause plus two missing designed features:
1. Stale users + eternal PLC hiss (root cause): ConnSession::close() silently
erased dropped users without broadcasting UserEvent::LEFT, so peers never
learned the user left and their audio engines never called remove_stream —
Opus PLC synthesized comfort noise forever. Fix: broadcast_left() helper
+ close() broadcasts LEFT before erasing.
2. PLC cap (defense-in-depth): on_playback now caps pure PLC at ~2s, then
emits digital silence so a stale stream can never hiss forever even if
remove_stream is skipped. Resets automatically on fresh packets.
3. No timeout / no ping: client never sent Ping, server had no last_seen /
reaper, so half-open connections (NAT timeout, wifi loss, sleep) left
ghost users forever. Fix: client Ping every 15s with RTT measurement,
ConnSession::last_seen bumped on every inbound TCP/UDP frame, steady_timer
reaper sweeps every 15s and drops sessions older than 45s (configurable
via server::Config).
4. UDP KEEPALIVE: client sends plaintext kFrameKeepalive every 5s; server
bumps last_seen + echoes back. Keeps NAT bindings alive and lets media
activity defer the reaper independently of TCP.
5. Graceful client disconnect: vc_disconnect() sends Disconnect{code=0} via
a flag-based io-thread exit (no double-close race); server handles
client-sent Disconnect with immediate close() + LEFT broadcast.
3 new tests: disconnect_left, plc_cap, reaper_timeout. 21/21 ctest green.
Docs: protocol.md §6/§7, voice.md §6, architecture.md §5, PROGRESS.md.
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
