Mirrors the Windows client's UI overhaul (commit 97fa659 + 540ec13) adapted to
Mac-native conventions. The main window is now just toolbar + channels + users
+ chat; audio device settings moved to a modeless Settings window.
- NSToolbar: Join Voice, Share Screen Audio, Mute, Deafen (SF Symbol toggle
buttons) + Output Volume slider (NSSlider 0-100, default 80). Voice actions,
mute/deafen, and output volume moved out of the bottom panel into the toolbar
- Audio device settings (input mode, VAD sensitivity, PTT key, device picker,
level meter) moved to a new SettingsWindowController -- a modeless window
opened via the app menu's "Settings..." (Cmd+,) item. Source-of-truth for
audio state lives in MainWindowController so voice start applies settings even
before the window has been opened; SettingsWindowController reads from /
writes back to those properties and applies changes live when voice is active.
Level meter forwarded from handleLevel -> updateLevel(rms:)
- Unified log: chat NSTextView + activity NSTableView collapsed into a single
NSTextView -- activity events in secondaryLabelColor (gray), chat in default
- Private messaging: scope dropdown removed; compose always sends to the
current channel. Each PM conversation opens in its own modeless
PrivateMessageWindowController. Incoming .textMessage with .private scope
routed to the right window; outgoing PMs echoed by server arrive through the
same path. "Send Private Message..." added to user context menu.
- Messages menu: "New Private Message..." (Cmd+Shift+N) opens a UserPickerSheet
listing all server users so you can PM anyone on the server
- Channel tree now shows live user counts, e.g. "General (3)"; refreshChannelTree
called on .userJoined/.userLeft (was missing)
- Voice menu: Join Voice (Cmd+Shift+V), Share Screen Audio (Cmd+Shift+S),
Mute (Cmd+Shift+M), Deafen (Cmd+Shift+D) -- NSMenuItem key equivalents with
[.command, .shift] mask, dispatched by the responder chain
- setOutputVolume(_:) wrapper added to VoiceCatClient.swift (was missing -- the
C ABI + C# wrapper shipped in commit 97fa659 but the Swift wrapper was never
added); wired end-to-end: toolbar slider -> client.setOutputVolume(gain)
Part A -- fixed and verified the previously-uncompiled Swift from the external
PCM feed/tap commit (615d2a8):
- Rebuilt the macOS xcframework slice (regenerated the module map from current
voicecat.h, exposing vc_pcm_sink_cb / vc_stream_feed_pcm / vc_set_pcm_sink)
- Fixed feedPcm type bug: size_t imports as Int in Swift not UInt; the original
UInt(samplesPerChannel) was wrong
- Added VoiceCatPcmSinkCallback typealias -- a Swift-idiomatic public alias for
the C vc_pcm_sink_cb so consumers (tests, the macOS app) can declare a sink
callback without directly importing the VoiceCatC C module. Mirrors the C#
VcPcmSinkCallback delegate
- keyCodeName helper deduplicated (was in PttKeyCaptureSheet.swift +
MainWindowController.swift -- now shared)
Platform-specific adaptations (vs. Windows): NSToolbar instead of ToolStrip;
global menu bar + NSMenuItem key equivalents (Cmd not Ctrl, responder-chain
dispatched, no custom key monitor needed); PM windows as modeless NSWindows;
picker as Mac sheet; gray = secondaryLabelColor; SF Symbols for toolbar icons.
swift test 10/10 (4 ExternalPcmTests + 6 VoiceCatClientSmokeTests against a
live server); xcodebuild Debug + Release BUILD SUCCEEDED with 0 Swift warnings.
Ctrl+Shift+V — join/leave voice
Ctrl+Shift+S — screen share toggle
Ctrl+Shift+M — mute/unmute mic
Ctrl+Shift+D — deafen/undeafen
Focus-scoped (same as PTT). Shortcuts display in the Voice menu for
the two menu-backed actions. Hotkeys are suppressed when a text box
has focus.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Voice actions (Join Voice, Share Screen Audio) moved to a ToolStrip toolbar and
a new Voice menu in the menu bar; removed from the bottom voice panel
- Activity log and chat log collapsed into a single RichTextBox (rtbLog); activity
events appear in gray, chat messages in default color
- Private messaging reworked: each conversation opens in its own modeless
PrivateMessageForm instead of sharing the main chat log via a scope dropdown;
cboScope removed; main compose bar always sends to the current channel
- New "Messages -> New Private Message..." menu item (Ctrl+P) opens a UserPickerDialog
listing all connected server users (not just the current channel) so you can PM
anyone on the server
- Channel tree now shows live user counts, e.g. "General (3)" -- counts sourced from
the existing _users dictionary which already tracks all server users with channel IDs
- Global output volume slider (TrackBar, 0-100, default 80) added to the right panel;
wired to new vc_set_output_volume C ABI function that applies a master gain multiplier
in the audio engine playback callback after mixing all streams
- vc_set_output_volume added end-to-end: voicecat.h, audio_engine.h/.cpp,
client.h/.cpp, voicecat.cpp, NativeMethods.cs, VoiceCatClient.cs
- Documented Windows PowerShell ctest requirement in AGENTS.md and CLAUDE.md:
MinGW binaries exit 0xc0000139 in Git Bash; always run ctest/.exe via PowerShell
22/22 ctest green (PowerShell); dotnet build 0 warnings.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Opus 1.6 DRED support end-to-end: encoder embeds 20 ms of ML
redundancy in every packet when enabled; decoder recovers lost frames
from the next buffered packet's DRED extension rather than falling back
to PLC comfort noise.
Protocol: bool dred = 11 added to AudioConfig (backward-compatible,
defaults false). C ABI: int dred added to vc_audio_config. Encoder:
OPUS_SET_DRED_DURATION(2) when dred=true. Decoder: OpusDREDDecoder +
per-stream OpusDRED scratch pre-allocated off the RT thread;
JitterBuffer::try_copy_front_payload peeks at the next packet without
popping on every PLC step; opus_decoder_dred_decode reconstructs the
lost frame if DRED data is present, otherwise falls back to PLC.
New test: test_dred_toggle (22/22 ctest green).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1398 → ~340 lines. All "Done" journal entries replaced by a compact
"Recent completed work" summary; M0–M4 sections condensed to scan-able
paragraphs. Pending items (iOS on-device verification, external PCM
feed/tap API plan) kept at full length. M5 task list updated to mark
macOS/iOS UI done and list remaining open items.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After the stereo-mic/A2DP debugging settled, the iOS audio code carried
leftover TeamTalk5 comparison notes, source-line citations, TEMP DIAGNOSTIC
markers, and "this was the bug" narratives that no longer help. Reworded
those to state the current rules; kept the comments that document real
constraints (the setPreferredInputNumberOfChannels(2) trap, the re-entrancy
guard, the ma_context no-session-management config).
Comment-only — no behavior change. core builds, ctest --preset dev 21/21.
The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.
Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.
Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.
Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three coordinated fixes for the bug where enabling stereo mic capture
caused all audio output (A2DP, speaker, wired) to go silent:
1. audio_engine.cpp — open playback before capture
On iOS, starting the stereo capture AudioUnit can trigger an audio
route reconfiguration that drops A2DP before the playback device has
a chance to claim the route. Opening and starting the playback device
first commits the output route (A2DP), so iOS is less likely to drop
it when stereo capture activates afterward.
2. client.cpp — decouple set_capture_channels from engine restart
Previously vc_set_capture_channels() stopped and restarted the engine
immediately, which opened capture first (old ordering) and raced
against the settling AVAudioSession route. Now it only stores the
channel count; the caller (Swift via vc_audio_restart) controls when
the engine restarts, after the route has settled.
3. IOSAudioRouter.swift — call audioRestart() after channel config
selectCaptureChannels() and applyPreset() now call audioRestart()
after applyConfiguration() + setCaptureChannels(). This is the
vc_audio_restart() path that was added to the ABI in fdcc84f but
never wired up in the Swift layer. The restart sees the stored
channel count and reopens devices in the correct order (playback
first, capture second).
The doStartMicStream path is unaffected: setCaptureChannels is called
before the server acknowledges the stream (engine not yet running), so
ensure_audio_running() picks up capture_channels=2 directly when the
stream is confirmed and opens with the right count from the start.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:
1. configureStereoCapture now calls setPreferredInput +
setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
Previously omitted based on incorrect diagnosis that
setPreferredInput collapsed A2DP — the real culprit was
setPreferredInputNumberOfChannels(2), which neither project uses.
2. New C ABI: vc_audio_restart (full stop + re-init, unlike
suspend/resume which only stop/start). Swift wrapper added.
The withAudioSuspend wrapper that used it was removed after
on-device testing showed it killed all audio (including
VoiceOver) when switching presets — the core's
set_capture_channels handles engine restart internally.
3. Bluetooth options: Voice Chat preset now includes BOTH
.allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.
4. Capture channels now reset when switching stereo→mono via
selectCaptureChannels/applyPreset. AudioSessionManager tracks
activeMicStreamId (set by SessionState on join/leave voice).
5. Docs synced: voice.md, tech-stack.md, architecture.md,
PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
references.
Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
Presets reorganized to give users choice at every level:
Always available (work with any output route):
- Voice Chat: AEC/AGC/HPF on, mono, system picks best route (BT HFP,
wired, or speaker). The standard iOS VoIP experience.
- Stereo Mic: Stereo built-in mic (front+back capsules), A2DP output
if BT connected else speaker/wired. Standard processing.
- Studio (No Processing): Stereo mic, no AEC/AGC/HPF (raw mode).
Maximum fidelity. Echo risk on speaker.
When Bluetooth connected:
- Bluetooth Headset (HFP): BT mic + BT output, AEC on, mono.
- BT Headphones + Mono Mic: A2DP output + built-in mic, mono, no AEC.
- BT Headphones + Stereo Mic: A2DP output + stereo built-in mic.
When wired headset/earpods connected:
- Wired Headset: Wired output + wired/built-in mic, AEC on, mono.
Always:
- Custom: shown when advanced settings don't match any preset.
Key changes from previous version:
- Stereo Mic is no longer gated behind Bluetooth — it's always available
and uses A2DP output if BT is connected, else speaker/wired.
- Wired headset detection (headphones/headsetMic/usbAudio port types)
with a dedicated preset.
- Voice Chat preset always available with AEC — the safe default.
- activePreset checks device-specific presets first so e.g. when BT is
connected and settings match 'Bluetooth Headset', it returns that
instead of the equivalent 'Voice Chat'.
- detectAudioDevices() replaces detectBluetooth(), detects both BT and
wired devices from currentRoute + availableInputs.
Replaces the flat list of audio settings with a preset picker that shows
context-appropriate options based on whether a Bluetooth device is connected.
Presets:
- Default (Phone Speaker): built-in mic + speaker, standard, mono
- Bluetooth Headset (HFP): BT mic + BT output, standard, mono — only shown
when a BT device is connected
- BT Headphones + Phone Mic: A2DP stereo output + built-in mic, standard,
mono — only shown when BT connected
- BT Headphones + Stereo Mic: A2DP stereo output + built-in mic stereo
(front+back capsules), standard, stereo — only shown when BT connected
- Custom: shown when advanced settings don't match any preset
When no Bluetooth device is connected, only 'Default' and 'Custom' appear,
with a hint to connect Bluetooth headphones for more options.
All granular controls (input port, orientation, polar pattern, mic mode,
channels, bluetooth mode, output route, AirPlay) are now under an
'Advanced Audio' disclosure group, collapsed by default.
IOSAudioRouter gains:
- AudioPreset enum with bluetoothMode/captureChannels/micMode/usesBuiltInMic
- hasBluetoothDevice detection (checks currentRoute + availableInputs for
bluetoothA2DP/bluetoothHFP port types)
- availablePresets (filtered by BT connection state)
- activePreset (computed from current settings)
- applyPreset() (sets all individual settings + finds built-in mic port UID)
Three bugs causing no audio output and no mic input:
1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware
AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS
mutes the output because it can't set up the voice processing pipeline on
an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC,
but audio routes correctly). .voiceChat kept for HFP and speaker modes.
Added info warning in Settings UI for A2DP no-AEC.
2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming()
which deactivated the AVAudioSession — but the AudioEngine keeps running for
remote audio playback, so leaving voice killed all remote audio. And the
session was never activated when a remote user started talking (only on
Join Voice), so you couldn't hear anyone before joining voice. Fix:
- ensureSessionActive() replaces activateForStreaming() — idempotent, called
on Join Voice AND on .streamStarted (remote user starts talking).
- stopMicStream() no longer deactivates the session.
- deactivateSession() called only on disconnect from server.
- isSessionActive flag tracks state, updated by interruption handler.
3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is
the default) and may put the session in a bad state on some devices. Fix:
only call it when stereo is explicitly selected. Also handle empty input
port ID (selecting 'Default' in the picker) correctly.
Added comprehensive route logging — after activation, logs the current output
and input route names so issues can be diagnosed from Console.app.
handleRouteChange called applyConfiguration() unconditionally, which called
setCategory/setPreferredInput/etc., which triggered another route-change
notification, which called applyConfiguration() again — an infinite loop that
burned CPU (phone slowdown) and repeatedly tore down/rebuilt the audio session
(audio cycling on/off, VoiceOver glitching).
Two fixes:
1. handleRouteChange now only re-applies config on external device changes
(.oldDeviceUnavailable / .newDeviceAvailable), not on .categoryChange /
.routeConfigurationChange which are triggered by our own setCategory calls.
2. IOSAudioRouter.applyConfiguration() gained a re-entrancy guard
(isApplyingConfiguration) for synchronous route-change notifications.
Also added os.Logger logging to both files (subsystem cat.voice.VoiceCatiOS)
so future issues can be debugged from Console.app on the Mac.
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
When no simulator is booted, grep -oE finds no UUID and exits 1. With
set -euo pipefail, that non-zero exit propagates through the command
substitution and kills the script at the UDID= assignment before it can
fall through to the boot-one branch.
Fix: add || true to both find_sim and sim_name pipelines so they always
return 0 regardless of whether a match was found.
Also replace sleep 2 with xcrun simctl bootstatus <UDID> -b, which blocks
until the simulator is fully booted — more reliable than a fixed delay
and faster when the simulator starts quickly.
Update docs/building.md §9 to show the bootstatus command.
scripts/build-ios-client.sh: builds VoiceCatiOS.app for iOS simulator
- calls build-xcframework.sh --all (all 3 slices required for iOS sim slice)
- xcodebuild -target VoiceCatiOS with SYMROOT=OBJROOT to co-locate SPM
and app build products (fixes "unable to resolve module dependency" error)
- stages result to dist/ios-client/
scripts/run-ios-simulator.sh: installs and launches on an iPhone simulator
- finds a booted simulator or boots the first available iPhone sim
- opens Simulator.app, installs via simctl install, launches via simctl launch
- --build flag to build first; --log to stream app logs; --device / --udid to
pin a specific simulator
scripts/build-all.sh: add --ios-client opt-in flag (macOS only)
docs/building.md:
- add iOS entry to quick-nav table
- update §6 (apple platform): remove "scaffolding" caveat now that iOS slices
are validated; trim to essentials + pointer to build-xcframework.sh
- add §9 (iOS client SwiftUI): XCFramework, xcodebuild command with rationale
for -target/-SYMROOT flags, run script usage, full simctl commands, Xcode UI
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
AudioSessionManager (AVAudioSession owner + interruption/route handling),
ServerListStore/SavedServer (App Group container + Keychain sharing),
and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)
C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.
XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.
Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
Add §7 (Windows client: dotnet build) and §8 (macOS client: XCFramework
+ xcodebuild) with the full build/run commands for each client platform.
Also add a quick-navigation table at the top of the doc.
The main window's NSSplitView panels (channel list, user list, chat,
activity log) were collapsing to zero size because:
1. The scroll views inside the split views were missing
translatesAutoresizingMaskIntoConstraints = false, so Auto Layout
couldn't manage their sizes.
2. The inner split views were also missing it.
3. The voice panel had no height constraint, so it expanded to fill
all available space (539px of the 600px window), starving the
outer split view down to 1px tall.
4. The split views had no initial divider positions, so panels
collapsed to zero even when the split view had space.
Add translatesAutoresizingMaskIntoConstraints = false to all four
scroll views and both inner split views, give the voice panel a
96px height constraint, and set initial divider positions after the
window is on screen. The panels now get reasonable space, are
visible on screen, and are reachable by VoiceOver.
Setting accessibilityLabel on the NSScrollView wrappers turned them into
leaf elements, so VoiceOver never descended into the document views
(NSOutlineView/NSTableView/NSTextView) inside. Tab still worked because
the key-view loop is independent of the accessibility tree.
Also removed the redundant setAccessibilityRole calls on the document
views — they already default to those roles, and re-setting the same
role can interfere with the view's custom a11y implementation.
Matches the pattern already used in ConnectWindowController, whose
server table VoiceOver reaches correctly.
The main window controller was created as a local variable in
ConnectWindowController.authSucceeded and never retained — ARC
deallocated it immediately, which destroyed the VoiceCatClient
(connection silently dropped), nil'd every button's weak target
(clicks did nothing), and killed event delivery (channel list,
messages, voice never worked). Symptom: TOFU (in the retained
connect controller) worked, but everything in the main window
was a zombie shell.
Fix: store the MainWindowController in a new field on
ConnectWindowController (which AppDelegate retains for the app's
lifetime). Also add NSLog diagnostics in deinit/bootstrap/handleEvent
so lifecycle and event delivery are observable from the terminal
or Console.app.
The previous "shipped" claim was false — xcodebuild had never been run and
the macOS app source was never committed. This commit adds the 17 Swift
source files + xcodeproj and fixes three real defect classes so Debug and
Release both build clean:
1. MainWindowController.swift compile errors:
- NSAccessibility.post arg order (element:notification:userInfo:)
- NSAccessibilityPriorityMedium -> NSAccessibilityPriorityLevel.medium
- StreamSummary.streamId -> .id (Identifiable conformance)
- drop redundant VoiceCatResult.description extension
2. Linker: add -lc++ to OTHER_LDFLAGS (libvoicecat-fat.a is C++20; pure-Swift
app target has no .cpp sources so libc++ wasn't pulled in — swift test
passed because Package.swift testTarget has linkerSettings: c++).
3. Release config: add ONLY_ACTIVE_ARCH=YES (XCFramework only has arm64).
Verified: clean Debug + Release builds, otool -L shows libc++.1.dylib,
nm shows _vc_client_create/_vc_version_string, app launches and runs.
Lays the groundwork for the macOS (AppKit) and iOS (SwiftUI) clients with a shared
Swift core wrapping the C ABI, mirroring the proven Windows VoiceCat.Interop layer.
Architecture decision: macOS UI = AppKit (not SwiftUI) for the most mature VoiceOver
accessibility story — same rationale as the Windows client's WinForms-over-WinUI-3
decision. iOS stays SwiftUI. Recorded in docs/roadmap.md §2.
Build infrastructure (Phase 0):
- clients/apple/scripts/build-xcframework.sh: runs cmake --preset apple-dev, merges
libvoicecat.a + 107 vcpkg static deps into a single ~30 MB fat static library
(libvoicecat-fat.a) via libtool -static (SPM binary targets link one .a per slice),
stages voicecat.h + a generated module.modulemap (module VoiceCatC) into the headers,
runs xcodebuild -create-xcframework -> clients/apple/VoiceCatCore.xcframework.
VoiceCatCore Swift Package (Phase 1):
- Package.swift: binary target (VoiceCatCoreXCF) + library (VoiceCatCore) + test target.
- Sources/VoiceCatCore/: 7 files mirroring the C# VoiceCat.Interop patterns adapted to
Swift native C interop — Enums (9 Swift mirrors of C enums, UInt32-backed), Config,
Event (copies ev.text to String inside the callback — the #1 lifetime rule), Models
(10 Swift value types), Marshaling (C arrays -> Swift + immediate vc_free_*),
Callbacks (@convention(c) + Unmanaged.passUnretained, the Swift analog of C#'s
[UnmanagedCallersOnly] + GCHandle), VoiceCatClient (owns vc_client* as OpaquePointer,
all 38 C ABI functions, deinit -> vc_client_destroy then frees config CStrings, event
delivery on main queue via coalesced DispatchQueue.main drain).
Tests — 6/6 green (swift test against a real voicecat-server):
- testConnectTofuAuthListChannelsRoundTrips, testAdminChannelCrudAccountCrudRoundTrips,
testScreenAudioStreamStartsAndStops, testPerStreamRecvControlsRoundTrip, plus two
static smoke tests. Catches Swift-specific interop bugs (@convention(c) callback
lifetime, Unmanaged pointer resolution, CString memory management, enum raw-value
bridging, struct layout) that C++ ctest cannot. C++ suite still 21/21 green.
Docs updated (house rule): tech-stack.md §2, architecture.md §4, roadmap.md M4 + §2,
clients/apple/README.md (full rewrite), PROGRESS.md, .gitignore.
macOS port groundwork — core, server, tools, and tests now build and run on
macOS 26.5 / Apple Silicon. ctest --preset dev green 21/21 (2 consecutive runs).
apple-dev produces valid arm64 libvoicecat.a + XCFramework for the Swift Package.
Three real cross-platform bugs found and fixed (all latent on Windows/Linux):
1. test_m2_voice.cpp POSIX branch missing <netdb.h> — Linux glibc transitively
includes it, macOS doesn't. Would fail on any strict POSIX system.
2. SIGPIPE killing processes on macOS — writing to a closed TCP socket raises
SIGPIPE by default (doesn't exist on Windows, benign on Linux). Fixed by
ignoring SIGPIPE in both core client init and server startup (POSIX-only,
#ifndef _WIN32). Production fix, not just tests.
3. Use-after-free of Asio's kqueue reactor on server shutdown — the
deterministic test_tofu_flow segfault. TcpServerConn's tls_read_loop runs on
a blocking-I/O thread; when Server::run() returned, io_context was destroyed
while those threads were still running. On macOS kqueue the reactor pointer
is null'd immediately -> segfault in socket.close(). Latent on Windows IOCP
and Linux epoll. Fix: TcpAcceptor now tracks connections; new shutdown()
closes all + joins threads before io is destroyed; Server::stop() now closes
acceptor + media_relay too (was just io.stop()).
Verified: dev + apple-dev presets build green, 21/21 tests pass, server starts
+ two vccli text chat over TLS (M1 on Mac), vccli --voice starts MIC stream via
CoreAudio (M2 protocol-level), vccli --list-devices enumerates CoreAudio
devices, xcodebuild -create-xcframework produces valid VoiceCatCore.xcframework.
No ABI or proto changes. Docs updated: building.md, clients/apple/README.md,
PROGRESS.md, CLAUDE.md status line.
Rationalize the preset set to match the project's actual state (past M5):
- Rename dev->skeleton (no-deps stub smoke), m1-dev->dev (default dev preset)
- Drop m2-dev (cache-identical to m1-dev)
- Add release preset (optimized + tests on, symbols kept)
- Strip server-release binaries (-s linker flag)
- Add apple-dev/apple-ios/apple-ios-sim scaffolding presets for XCFramework
Add cmake/voicecat-toolchain.cmake wrapper that auto-resolves the vcpkg
triplet from the host platform (x64-mingw-static/x64-linux/arm64-osx) so
the main presets work on Windows/Linux/macOS without per-OS variants.
Update all docs (building.md, CLAUDE.md, README.md, AGENTS.md, deployment.md,
tech-stack.md, client READMEs) and stale preset-name references in code
comments. No C++ behavior changes — the core was already portable.
PerUserTuningDialog previously broadcast one gain/mute/NR set to *all* of a
user's streams, even though the core mixer (AudioEngine::RemoteStream) and
the C ABI (vc_set_remote_stream) were already per-stream. The UI had no
per-mix controls anywhere.
Reworks the dialog to enumerate ListUserStreams on open and render one row
per stream (kind + label + Gain + Mute + NR), each wiring only to its own
stream_id. Adds a read-back ABI counterpart, vc_get_remote_stream, so the
dialog opens at the listener's actual current per-stream settings (defaults
1.0/unmuted/NR-off) rather than always 100%. Additive ABI change only; no
existing symbols touched.
Tests: test_m3_multistream extended with getter round-trip assertions; new
C# smoke test exercises the full P/Invoke marshaling path with two clients.
Docs: voice.md §10 notes the getter. NR checkbox keeps its honest
'passthrough' label (NS DSP still unbuilt per §8).
The TOFU pin store is written by vc_client during headless tests (the
relative fallback path in core/src/core/client.cpp when tofu_store_path
is unset). Ignore it so it doesn't show up as untracked after test runs.
Three reported bugs traced to one root cause plus two missing designed features:
1. Stale users + eternal PLC hiss (root cause): ConnSession::close() silently
erased dropped users without broadcasting UserEvent::LEFT, so peers never
learned the user left and their audio engines never called remove_stream —
Opus PLC synthesized comfort noise forever. Fix: broadcast_left() helper
+ close() broadcasts LEFT before erasing.
2. PLC cap (defense-in-depth): on_playback now caps pure PLC at ~2s, then
emits digital silence so a stale stream can never hiss forever even if
remove_stream is skipped. Resets automatically on fresh packets.
3. No timeout / no ping: client never sent Ping, server had no last_seen /
reaper, so half-open connections (NAT timeout, wifi loss, sleep) left
ghost users forever. Fix: client Ping every 15s with RTT measurement,
ConnSession::last_seen bumped on every inbound TCP/UDP frame, steady_timer
reaper sweeps every 15s and drops sessions older than 45s (configurable
via server::Config).
4. UDP KEEPALIVE: client sends plaintext kFrameKeepalive every 5s; server
bumps last_seen + echoes back. Keeps NAT bindings alive and lets media
activity defer the reaper independently of TCP.
5. Graceful client disconnect: vc_disconnect() sends Disconnect{code=0} via
a flag-based io-thread exit (no double-close race); server handles
client-sent Disconnect with immediate close() + LEFT broadcast.
3 new tests: disconnect_left, plc_cap, reaper_timeout. 21/21 ctest green.
Docs: protocol.md §6/§7, voice.md §6, architecture.md §5, PROGRESS.md.
start_loopback_capture hardcoded channels=1, forcing miniaudio to downmix the system's stereo mix to mono before the encoder saw it -- on_capture_frame then upmixed L=R to produce fake stereo. Now the loopback device opens in the channel's mode (stereo when the channel is stereo), CaptureCallback carries an explicit channels param so the encoder gets real interleaved L/R, and a mono fallback covers unusual render endpoints. New test_loopback_stereo_capture asserts L!=R end-to-end; 18/18 ctest green.
The core already supported SCREEN_AUDIO capture on Windows (post-M3 WASAPI
loopback via VOICECAT_HAS_LOOPBACK) and the C# Interop layer was complete
(VcStreamKind.ScreenAudio, StartStream/StopStream/SetRemoteStream). Only the
UI was missing -- no core, proto, or C ABI changes needed.
Adds a 'Share Screen Audio' toggle to the voice panel, independent of mic
voice (can share without joining voice). Disconnect/teardown now stops the
screen stream cleanly. New smoke test exercises the full StartStream ->
StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
The media AEAD nonce is an implicit per-direction monotonic counter;
open() reconstructs it from the 14-byte header seq field (the AAD), so
the contract is header.seq == the counter seal() used. The SFU relay
decrypted inbound frames with the sender key, re-sealed with the
recipient send_crypto (its own counter), but forwarded the sender
header verbatim -- so seq carried the wrong counter and the recipient
rebuilt the wrong nonce, silently dropping every relayed frame. It only
worked for a single first-ever sender into a fresh recipient, which is
why reverse/3rd-party audio failed.
Rewrite the outgoing header seq to the recipient peek_send_counter()
before re-sealing so each server->client direction is one contiguous
monotonic counter and the nonce always matches. Safe: the jitter buffer
orders by timestamp, not seq. No wire-format/proto/ABI change.
Adds test_relay_interleaved_reseal regression coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A connected Windows client would randomly snap from its joined channel
back to Lobby. Root cause was a state-sync inconsistency, not a drop:
the server delivered self-initiated state changes (channel join/leave,
stream announce/stop) only as a private *Result to the actor and
broadcast the authoritative UserEvent::UPDATED to everyone else. The
core never applied the result to its SessionModel, so vc_list_users()
kept self in the old channel; the Windows HandleUserUpdated rebuilds
_currentChannelId from vc_list_users() on any user's UPDATED event, so
the next unrelated event surfaced the stale self-channel.
Fix, per the response-vs-broadcast contract now documented in
docs/protocol.md §6: the *Result is pure ack/correlation/actor-private
payload; the resulting state change is broadcast to every client
INCLUDING the actor, and clients apply it to their local model rather
than re-deriving own state from a *Result.
- server: join/leave/stream announce+stop broadcast with exclude=0
- server: text fan-out includes the sender (channel + private echo)
- core: response handlers no longer mutate session_model_
- windows: drop optimistic text echo; render own message via the relay
- docs/protocol.md §6: document the response-vs-broadcast contract
Registry-level admin broadcasts (move/mute/kick/channel CRUD) already
used exclude=0 and were correct. ctest build/m1-dev 18/18 green;
VoiceCat.App builds 0 warnings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RemoteStream::playout_ts was seeded to 0 and only advanced inside the
decode loop (including on every PLC iteration), so it free-ran at ~1x
wall-clock regardless of whether the sender was transmitting. The
sender's frame timestamps only advance while it actually sends (the
VAD/PTT gate returns before ls.timestamp += samples). Across a late join
or any VAD/PTT silence gap the two clocks diverged without bound; once
past the jitter buffer's 500 ms late-drop window every real frame was
dropped-as-late (clock ahead) or never-due (clock behind) -> permanent
silence, while the talk indicator (driven by push_recv_frame, independent
of the jitter buffer) stayed lit.
Add JitterBuffer::peek_front_ts() (try-lock, RT-safe) and seed/re-sync
playout_ts to the earliest buffered frame on the first frame and whenever
it has drifted past +/-200/500 ms. This seeds startup and recovers after
every silence gap.
New regression test test_playout_resync free-runs the clock ~2 s past the
drop window, pushes a ts=0 frame, and asserts audible output: fails
(energy=0) without the fix, passes with it. ctest --preset m1-dev: 14/14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AudioEngine::on_capture() was passing miniaudio's hardware callback period
(commonly 480 samples / 10 ms on WASAPI shared mode) directly to opus_encode(),
which requires exactly frame_samples_ (960 for 20 ms @ 48 kHz). The mismatch
returned OPUS_BAD_ARG and silently dropped every real mic frame, while screen
share and injected test frames happened to be correctly sized and worked fine.
Fix: accumulate PCM in a pre-allocated CaptureAccum buffer (mirroring the
existing RemoteStream::ring fix on the playback side) and only call capture_cb_
when a full frame_samples_ chunk is ready. Same pattern applied to on_loopback().
Add test_capture_frame_accumulation() to verify the accumulator fires exactly the
right number of callbacks for misaligned chunk sizes (480, 240+720, 1920 samples).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the M4-placeholder stub with a real build/run guide covering prerequisites,
build order (server → DLL → dotnet build), DLL dependency verification, manual test
runbook, and the known limitations (focus-scoped PTT, NR passthrough, no admin UI,
TOFU-pins-leaf-cert vs Ed25519).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
on_playback() was passing miniaudio's hardware playback-callback frame
count to opus_decode()'s max_samples, instead of the decoder's fixed
frame size (960 samples @ 20ms/48kHz). Since real packets decode to
more samples than the (often smaller, e.g. ~480 on default low-latency
WASAPI) hardware period, opus_decode returned OPUS_BUFFER_TOO_SMALL on
nearly every callback -- packets were received/decrypted/jitter-buffered
correctly but never decoded into audible PCM. Result: control-plane
events and VAD worked, but zero audio in headphones.
mix_for_test()'s white-box test masked this since it always called
on_playback with frames == frame_samples, the one case where the bug
is invisible.
Fix: RemoteStream gained a small ring buffer (init_ring/push_ring/
pop_ring) that decouples decode cadence from playback-callback cadence.
on_playback now tops the ring up by decoding whole Opus frames (always
decoder.frame_samples(), never the hardware frame count) and drains
exactly what the callback asks for, silence-padding (PLC) on underrun.
Side effect: also fixes playout_ts, which was advancing by the wrong
unit (hardware frames instead of decoded samples) -- it now tracks
correctly against jitter-buffer timestamps.
ctest --test-dir build/m1-dev: 12/12 green.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docs/building.md: explains what each CMake preset (dev, m1-dev, m2-dev,
server-release) is actually for, and how to build voicecat-server + vccli
for manual testing. Linked from CLAUDE.md's doc index.
- core/include/voicecat.h, core/src/voicecat.cpp, core/src/protocol/protocol.h,
server/src/main.cpp: doc-header comments still claimed M0-skeleton/stub
behavior (VC_ERR_NOT_IMPLEMENTED everywhere, "prints what it would do",
protobuf codegen "commented") that M1-M3 made real. Updated to describe
current behavior, with the dev-preset stub fallback noted explicitly where
it still applies.
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_m2_voice passed against raw BSD sockets, but vc_client::stream_start/stop,
UDP binding, and capture/recv were still VC_ERR_NOT_IMPLEMENTED stubs -- meaning
vccli and any GUI client still couldn't actually talk. Implements the real
client-side UDP-binding handshake, media key derivation, capture->encode->seal->
send and recv->open->decode->playback paths, plus server-side StreamInfo
broadcast so peers learn about each other's streams via sync_remote_streams().
Adds test_voice_client_abi (two real vc_client instances, not raw sockets) and
vccli --voice/--mute/--text flags, manually verified live between two instances.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305
AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX,
an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an
APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain
wired through ConnSession/SessionRegistry into a new server-side SFU
(MediaRelay) that decrypts and re-encrypts frames per channel member.
Exit criterion verified: test_m2_voice — two headless clients relay 50
encrypted Opus frames through the server; ctest --preset m1-dev is 9/9
green. Also corrects protocol.md's UdpBinding diagram, which described the
UDP-side binding packet as AEAD-sealed when it is in fact a plaintext
bootstrap frame (separate from the TCP/TLS UdpBinding ack).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
