iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.
Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
AudioEngine opens no hardware playback device; a mixer-timer thread drives
on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
sink. start() also skips the hardware capture device when the MIC stream is
external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).
iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
presets; SessionState join/leave + reconcileVoicePath() switch paths;
Voice Chat defaults to speaker; Settings surfaces AEC/NS state.
Known: pending on-device verification; a few bugs to fix afterward.
After the stereo-mic/A2DP debugging settled, the iOS audio code carried
leftover TeamTalk5 comparison notes, source-line citations, TEMP DIAGNOSTIC
markers, and "this was the bug" narratives that no longer help. Reworded
those to state the current rules; kept the comments that document real
constraints (the setPreferredInputNumberOfChannels(2) trap, the re-entrancy
guard, the ma_context no-session-management config).
Comment-only — no behavior change. core builds, ctest --preset dev 21/21.
The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.
Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.
Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.
Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:
1. configureStereoCapture now calls setPreferredInput +
setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
Previously omitted based on incorrect diagnosis that
setPreferredInput collapsed A2DP — the real culprit was
setPreferredInputNumberOfChannels(2), which neither project uses.
2. New C ABI: vc_audio_restart (full stop + re-init, unlike
suspend/resume which only stop/start). Swift wrapper added.
The withAudioSuspend wrapper that used it was removed after
on-device testing showed it killed all audio (including
VoiceOver) when switching presets — the core's
set_capture_channels handles engine restart internally.
3. Bluetooth options: Voice Chat preset now includes BOTH
.allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.
4. Capture channels now reset when switching stereo→mono via
selectCaptureChannels/applyPreset. AudioSessionManager tracks
activeMicStreamId (set by SessionState on join/leave voice).
5. Docs synced: voice.md, tech-stack.md, architecture.md,
PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
references.
Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
Three bugs causing no audio output and no mic input:
1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware
AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS
mutes the output because it can't set up the voice processing pipeline on
an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC,
but audio routes correctly). .voiceChat kept for HFP and speaker modes.
Added info warning in Settings UI for A2DP no-AEC.
2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming()
which deactivated the AVAudioSession — but the AudioEngine keeps running for
remote audio playback, so leaving voice killed all remote audio. And the
session was never activated when a remote user started talking (only on
Join Voice), so you couldn't hear anyone before joining voice. Fix:
- ensureSessionActive() replaces activateForStreaming() — idempotent, called
on Join Voice AND on .streamStarted (remote user starts talking).
- stopMicStream() no longer deactivates the session.
- deactivateSession() called only on disconnect from server.
- isSessionActive flag tracks state, updated by interruption handler.
3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is
the default) and may put the session in a bad state on some devices. Fix:
only call it when stereo is explicitly selected. Also handle empty input
port ID (selecting 'Default' in the picker) correctly.
Added comprehensive route logging — after activation, logs the current output
and input route names so issues can be diagnosed from Console.app.
handleRouteChange called applyConfiguration() unconditionally, which called
setCategory/setPreferredInput/etc., which triggered another route-change
notification, which called applyConfiguration() again — an infinite loop that
burned CPU (phone slowdown) and repeatedly tore down/rebuilt the audio session
(audio cycling on/off, VoiceOver glitching).
Two fixes:
1. handleRouteChange now only re-applies config on external device changes
(.oldDeviceUnavailable / .newDeviceAvailable), not on .categoryChange /
.routeConfigurationChange which are triggered by our own setCategory calls.
2. IOSAudioRouter.applyConfiguration() gained a re-entrancy guard
(isApplyingConfiguration) for synchronous route-change notifications.
Also added os.Logger logging to both files (subsystem cat.voice.VoiceCatiOS)
so future issues can be debugged from Console.app on the Mac.
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
AudioSessionManager (AVAudioSession owner + interruption/route handling),
ServerListStore/SavedServer (App Group container + Keychain sharing),
and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)
C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.
XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.
Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
