Recovery from the 99937c9 audio-device-change commit broadened the
route-change recovery set to "everything except categoryChange /
routeConfigurationChange", which added .override. But .override is
fired by our own applyA2dpSpeakerFallback() -> overrideOutputAudioPort,
which recoverAudio() calls on every recovery. On an A2DP preset
(Stereo Mic / Mono Mic), disconnecting AirPods ping-ponged:
oldDeviceUnavailable -> recoverAudio -> applyA2dpSpeakerFallback
-> overrideOutputAudioPort(.speaker) -> .override routeChange
-> recoverAudio -> applyConfiguration (setCategory resets override)
-> applyA2dpSpeakerFallback -> overrideOutputAudioPort -> .override -> ...
Each iteration also rebuilt the AVAudioEngine via reconfigure() ->
rebuild() -- the audible reinitialize loop + CPU spin. Voice Chat and
Built-in Mic + Speaker were unaffected (applyA2dpSpeakerFallback
early-returns for non-A2DP modes, so no overrideOutputAudioPort call).
Two-part fix (pure Swift iOS-app target; no C ABI / proto / docs changes):
1. AudioSessionManager.handleRouteChange: added .override to the skip
list alongside .categoryChange / .routeConfigurationChange. .override
is only ever fired by our own overrideOutputAudioPort call, so
treating it as a recovery reason is the loop by definition. The
AVAudioEngineConfigurationChange observer in IOSVoiceProcessingEngine
remains as the backstop if an override ever actually stops the engine.
2. IOSAudioRouter.applyA2dpSpeakerFallback: made idempotent via a
lastAppliedOutputOverride tracker that skips the redundant
overrideOutputAudioPort call when the desired state (.none for
external output present, .speaker otherwise) already matches. Reset
to nil at the top of applyConfiguration() (setCategory can reset the
override) and on a failed call. Defense-in-depth on top of fix 1.
Build: xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj
-scheme VoiceCatiOS -destination 'generic/platform=iOS' build green
(Xcode 26.5 / iOS 18.0).
Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired
headphones, AirPods) used to leave the iOS client in a dead/zombie state:
the engine went silent, no reconnect was attempted, and a live-session
disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout.
Reconnect (AppState.swift, SessionState.swift):
- Two-layer reconcile. Once SessionState overwrites client.onEvent at auth
success, AppState.handleConnectEvent no longer sees live-session events.
Added a weak SessionState.appState; SessionState.handleEvent .disconnected
calls appState.onLiveSessionDisconnected after the cue -- the single path
AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect
snapshots lastSession, stops audio, releases session/VoiceCatClient (io-
thread join via vc_client_destroy), resets the backoff, and arms
scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth
success via existing TOFU_MATCHED auto-confirm + idempotent join_channel).
- NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi
<-> cellular interface change or path .unsatisfied it calls
proactiveReconnect: tearing the session down BEFORE the C core notices the
dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff
tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature.
While mid-reconnect a .satisfied path resets the backoff for a fast retry.
- User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect
and cancel all reconnect state (task + monitor + lastSession + connectedServer).
Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift):
- Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/
.newDeviceAvailable route-change guard; fires on every externally-initiated
route change reason except the ones we cause ourselves (.categoryChange/
.routeConfigurationChange) to avoid a notification loop. Interruption-end
now always recovers instead of only when .shouldResume is set.
- Added AVAudioEngineConfigurationChange observer on the engine so a system
self-stop after our route-change handler wins the race is caught.
- IOSAudioEngine.rebuild() does a one-shot reactivation-retry on
engine.start() failure (iOS sometimes refuses until the session is
re-reactivated -- the silent-death case).
No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green
via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
Mic input gain (was 3x) and per-user receive gain (was 2x) had
asymmetric boost ceilings. Raise both, plus the desktop aux input
gain (was 3x), to a uniform 4x (400%) on macOS, iOS, and Windows.
The master Output volume slider is unchanged (still 1x). No core
changes needed: the C ABI only clamps negatives, so the ceilings
live entirely in the client UI sliders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On iOS the remote end heard the mic twice and crackly (BT headset + internal
mic in Voice Chat; mono + stereo copies of the internal mic in Stereo Mic).
Root cause is an io-thread ordering race. `ensure_audio_running()` only set
`external_capture` when a MIC stream already existed, but it also runs from
`sync_remote_streams` on the post-auth `ServerStateSnapshot` — before the user
joins voice. With `external_playback_` still false and no MIC stream,
`AudioEngine::start()` opened a real miniaudio capture device that stayed open
all session (later calls early-return on running()), racing the AVAudioEngine
input tap fed via `vc_stream_feed_pcm`. `on_capture_frame` then encoded+sent
both paths — the mic transmitted twice, the two unsynchronized capture clocks
producing the crackle.
- core (ensure_audio_running): force `external_capture = true` whenever
`external_playback_` is set, so iOS unified mode never opens a hardware
capture device. No-op on desktop.
- ios (AppState): move `setExternalPlayback(true)` before `connect()`, so the
flag is set before the io thread processes any message — closing the race.
Verified on device: remote end hears the iOS mic once and clean in both Voice
Chat (+ BT) and Stereo Mic.
The iOS mic was unusable — a consistent ~40-60ms flutter + volume fade
('slow fan') on every preset. The core sends each captured frame
synchronously (no send pacer), so packet cadence == capture cadence, and
the receiver's playout keeps near-zero buffering by design (its jitter
estimate keys off the regular sender timestamp, so it's blind to arrival
jitter). That's smooth only for a steady sender (desktop miniaudio =
steady 20ms); the iOS AVAudioEngine tap delivers ~2 frames per ~40ms
callback -> bursty -> receiver underruns -> PLC fade.
Fix (iOS-only): the mic tap writes converted 48kHz int16 to an SPSC ring;
a 20ms feed pump drains it and calls feedPcm at a steady cadence. The pump
primes a small prebuffer cushion (3 frames ~60ms, self-healing up to
~120ms on underrun) before releasing, so the tap's bursts can't drain it
to empty. Never reads a partial frame (read consumes what it returns ->
partials were the crackle), and rebuilds with the current channel count
each rebuild() (a frozen count fed mono-as-stereo = octave-up on a
Stereo->Voice Chat switch).
Trade-off: ~60-120ms added mic-send latency, unavoidable when de-bursting
for a near-zero-buffer receiver. PROGRESS.md notes the proper follow-up:
make the jitter buffer measure real RFC-3550 arrival jitter so the
receiver absorbs bursts itself.
Verified: xcodebuild Debug BUILD SUCCEEDED (iOS Simulator, arm64).
Expose the existing send-side vc_set_input_noise_reduction C ABI (MIC-only,
mono, LOCAL — denoises captured mic PCM before input gain and VAD/PTT gate)
as a persisted global toggle in each client's audio settings, applied live
and re-applied on Join Voice. Mirrors the existing mic-gain wiring pattern.
- Shared Swift (VoiceCatCore): add setInputNoiseReduction(_:) wrapper
- Windows: P/Invoke + SetInputNoiseReduction wrapper, MicNoiseReduction in
VoiceSettings, new checkbox in AudioSettingsForm (layout shifted +28px),
apply on Join Voice; also fix stale 'planned - currently passthrough'
label on the receive-side per-user NR checkbox (RNNoise now backs it)
- macOS: inputNoiseReduction state + UserDefaults in MainWindowController,
NR checkbox + nrChanged action in SettingsWindowController
- iOS: inputNoiseReduction in VoiceState + setter + restore in SessionState,
NR Toggle in SettingsView Voice section
Aux/screen are out of scope by design (core's NR guards kind == MIC). Apple
builds require a rebuilt VoiceCatCore.xcframework with VOICECAT_HAS_NS.
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two on-device bugs in the native iOS Voice-Processing path (Swift-only;
no core/ABI change).
1. Voice Chat (VPIO) silent playback: doStartMicStream() called
audioRestart() BEFORE startStream, so when the engine was already
running (a remote stream had started it) it reopened with
external_capture=false and opened a hardware miniaudio capture device.
The announce-result restart then early-returns (engine already running)
so that device was never dropped and fought the AVAudioEngine VPIO unit,
silencing playback. Now: setExternalPlayback first, then startStream
(stores external_feed synchronously), THEN audioRestart() — the core
reopens in full external mode (no hardware devices). Added VPIO
diagnostics: graph/route formats at start, ring written/read totals at
teardown.
2. Stereo Mic / Studio quiet earpiece: the .builtInMicBtA2dp presets omit
.defaultToSpeaker (it breaks A2DP) and skip forceSpeaker, so with no
Bluetooth connected output pinned to the quiet receiver. New
IOSAudioRouter.applyA2dpSpeakerFallback() overrides to the built-in
speaker when no external (A2DP/wired/AirPlay) output is present and
clears the override when one is — called after activation and on
device-change route changes.
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.
Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
AudioEngine opens no hardware playback device; a mixer-timer thread drives
on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
sink. start() also skips the hardware capture device when the MIC stream is
external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).
iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
presets; SessionState join/leave + reconcileVoicePath() switch paths;
Voice Chat defaults to speaker; Settings surfaces AEC/NS state.
Known: pending on-device verification; a few bugs to fix afterward.
Add a "speaker output" override so users can route audio to the built-in
speaker instead of the earpiece when no headphones/Bluetooth are connected.
Previously the receiver was the only fallback on the default Voice Chat preset.
The toggle inserts .defaultToSpeaker into the AVAudioSession category options
(skipped for the A2DP mode, where it would break Bluetooth routing). It yields
to connected BT/wired output and is orthogonal to preset matching. Exposed both
as a call-bar button in VoiceControlsView and a persisted Settings toggle.
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.
- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
(backward-compatible Codable), shown in Guest mode, wired into the guest auth
path -- guests could not set a display name on either before (only Windows)
Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
- Channels tab is now a drill-down on iPhone: ChannelBrowserView lists top-level
channels; ChannelDetailView shows the people in a channel, its sub-channels, and
an explicit Join button (with password prompt). iPad split view unchanged.
- Extract self-contained UserRow (context menu + sheets) from UserListView so admin
actions are reused in the drill-down.
- Fix off-screen chat compose box: pin VoiceControlsView via per-tab
.safeAreaInset(edge: .bottom) instead of a floating overlay, so it reserves layout
space above the tab bar (keeping the compose box visible, cooperating with keyboard
avoidance) without covering the tab bar buttons.
- Collapse Activity into Chat like macOS/Windows: ChatView renders a merged,
time-sorted timeline of messages + activity (activity rows in gray); remove the
Activity tab and ActivityLogView.
- Label the RPSystemBroadcastPickerView inner UIButton for VoiceOver
("Share/Stop sharing screen audio") instead of relying on an outer SwiftUI label.
The broadcast upload extension never appeared in the iOS broadcast picker
(or Control Center screen-recording list) because its Info.plist set
RPBroadcastProcessMode to the invalid string "BroadcastUpload". iOS only
recognises "RPBroadcastProcessModeSampleBuffer" for a sample-buffer upload
extension and silently rejects any other value, hiding the extension.
Also sign VoiceCatCore.xcframework in build-xcframework.sh: Xcode 15+ rejects
unsigned binary XCFrameworks consumed as SwiftPM binary targets. Signs with
CODESIGN_IDENTITY, else the first Apple Development identity, else ad-hoc.
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
After the stereo-mic/A2DP debugging settled, the iOS audio code carried
leftover TeamTalk5 comparison notes, source-line citations, TEMP DIAGNOSTIC
markers, and "this was the bug" narratives that no longer help. Reworded
those to state the current rules; kept the comments that document real
constraints (the setPreferredInputNumberOfChannels(2) trap, the re-entrancy
guard, the ma_context no-session-management config).
Comment-only — no behavior change. core builds, ctest --preset dev 21/21.
The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.
Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.
Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.
Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three coordinated fixes for the bug where enabling stereo mic capture
caused all audio output (A2DP, speaker, wired) to go silent:
1. audio_engine.cpp — open playback before capture
On iOS, starting the stereo capture AudioUnit can trigger an audio
route reconfiguration that drops A2DP before the playback device has
a chance to claim the route. Opening and starting the playback device
first commits the output route (A2DP), so iOS is less likely to drop
it when stereo capture activates afterward.
2. client.cpp — decouple set_capture_channels from engine restart
Previously vc_set_capture_channels() stopped and restarted the engine
immediately, which opened capture first (old ordering) and raced
against the settling AVAudioSession route. Now it only stores the
channel count; the caller (Swift via vc_audio_restart) controls when
the engine restarts, after the route has settled.
3. IOSAudioRouter.swift — call audioRestart() after channel config
selectCaptureChannels() and applyPreset() now call audioRestart()
after applyConfiguration() + setCaptureChannels(). This is the
vc_audio_restart() path that was added to the ABI in fdcc84f but
never wired up in the Swift layer. The restart sees the stored
channel count and reopens devices in the correct order (playback
first, capture second).
The doStartMicStream path is unaffected: setCaptureChannels is called
before the server acknowledges the stream (engine not yet running), so
ensure_audio_running() picks up capture_channels=2 directly when the
stream is confirmed and opens with the right count from the start.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:
1. configureStereoCapture now calls setPreferredInput +
setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
Previously omitted based on incorrect diagnosis that
setPreferredInput collapsed A2DP — the real culprit was
setPreferredInputNumberOfChannels(2), which neither project uses.
2. New C ABI: vc_audio_restart (full stop + re-init, unlike
suspend/resume which only stop/start). Swift wrapper added.
The withAudioSuspend wrapper that used it was removed after
on-device testing showed it killed all audio (including
VoiceOver) when switching presets — the core's
set_capture_channels handles engine restart internally.
3. Bluetooth options: Voice Chat preset now includes BOTH
.allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.
4. Capture channels now reset when switching stereo→mono via
selectCaptureChannels/applyPreset. AudioSessionManager tracks
activeMicStreamId (set by SessionState on join/leave voice).
5. Docs synced: voice.md, tech-stack.md, architecture.md,
PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
references.
Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
Presets reorganized to give users choice at every level:
Always available (work with any output route):
- Voice Chat: AEC/AGC/HPF on, mono, system picks best route (BT HFP,
wired, or speaker). The standard iOS VoIP experience.
- Stereo Mic: Stereo built-in mic (front+back capsules), A2DP output
if BT connected else speaker/wired. Standard processing.
- Studio (No Processing): Stereo mic, no AEC/AGC/HPF (raw mode).
Maximum fidelity. Echo risk on speaker.
When Bluetooth connected:
- Bluetooth Headset (HFP): BT mic + BT output, AEC on, mono.
- BT Headphones + Mono Mic: A2DP output + built-in mic, mono, no AEC.
- BT Headphones + Stereo Mic: A2DP output + stereo built-in mic.
When wired headset/earpods connected:
- Wired Headset: Wired output + wired/built-in mic, AEC on, mono.
Always:
- Custom: shown when advanced settings don't match any preset.
Key changes from previous version:
- Stereo Mic is no longer gated behind Bluetooth — it's always available
and uses A2DP output if BT is connected, else speaker/wired.
- Wired headset detection (headphones/headsetMic/usbAudio port types)
with a dedicated preset.
- Voice Chat preset always available with AEC — the safe default.
- activePreset checks device-specific presets first so e.g. when BT is
connected and settings match 'Bluetooth Headset', it returns that
instead of the equivalent 'Voice Chat'.
- detectAudioDevices() replaces detectBluetooth(), detects both BT and
wired devices from currentRoute + availableInputs.
Replaces the flat list of audio settings with a preset picker that shows
context-appropriate options based on whether a Bluetooth device is connected.
Presets:
- Default (Phone Speaker): built-in mic + speaker, standard, mono
- Bluetooth Headset (HFP): BT mic + BT output, standard, mono — only shown
when a BT device is connected
- BT Headphones + Phone Mic: A2DP stereo output + built-in mic, standard,
mono — only shown when BT connected
- BT Headphones + Stereo Mic: A2DP stereo output + built-in mic stereo
(front+back capsules), standard, stereo — only shown when BT connected
- Custom: shown when advanced settings don't match any preset
When no Bluetooth device is connected, only 'Default' and 'Custom' appear,
with a hint to connect Bluetooth headphones for more options.
All granular controls (input port, orientation, polar pattern, mic mode,
channels, bluetooth mode, output route, AirPlay) are now under an
'Advanced Audio' disclosure group, collapsed by default.
IOSAudioRouter gains:
- AudioPreset enum with bluetoothMode/captureChannels/micMode/usesBuiltInMic
- hasBluetoothDevice detection (checks currentRoute + availableInputs for
bluetoothA2DP/bluetoothHFP port types)
- availablePresets (filtered by BT connection state)
- activePreset (computed from current settings)
- applyPreset() (sets all individual settings + finds built-in mic port UID)
Three bugs causing no audio output and no mic input:
1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware
AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS
mutes the output because it can't set up the voice processing pipeline on
an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC,
but audio routes correctly). .voiceChat kept for HFP and speaker modes.
Added info warning in Settings UI for A2DP no-AEC.
2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming()
which deactivated the AVAudioSession — but the AudioEngine keeps running for
remote audio playback, so leaving voice killed all remote audio. And the
session was never activated when a remote user started talking (only on
Join Voice), so you couldn't hear anyone before joining voice. Fix:
- ensureSessionActive() replaces activateForStreaming() — idempotent, called
on Join Voice AND on .streamStarted (remote user starts talking).
- stopMicStream() no longer deactivates the session.
- deactivateSession() called only on disconnect from server.
- isSessionActive flag tracks state, updated by interruption handler.
3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is
the default) and may put the session in a bad state on some devices. Fix:
only call it when stereo is explicitly selected. Also handle empty input
port ID (selecting 'Default' in the picker) correctly.
Added comprehensive route logging — after activation, logs the current output
and input route names so issues can be diagnosed from Console.app.
handleRouteChange called applyConfiguration() unconditionally, which called
setCategory/setPreferredInput/etc., which triggered another route-change
notification, which called applyConfiguration() again — an infinite loop that
burned CPU (phone slowdown) and repeatedly tore down/rebuilt the audio session
(audio cycling on/off, VoiceOver glitching).
Two fixes:
1. handleRouteChange now only re-applies config on external device changes
(.oldDeviceUnavailable / .newDeviceAvailable), not on .categoryChange /
.routeConfigurationChange which are triggered by our own setCategory calls.
2. IOSAudioRouter.applyConfiguration() gained a re-entrancy guard
(isApplyingConfiguration) for synchronous route-change notifications.
Also added os.Logger logging to both files (subsystem cat.voice.VoiceCatiOS)
so future issues can be debugged from Console.app on the Mac.
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
AudioSessionManager (AVAudioSession owner + interruption/route handling),
ServerListStore/SavedServer (App Group container + Keychain sharing),
and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)
C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.
XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.
Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
