Files
voice-cat/clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift

181 lines
8.8 KiB
Swift
Raw Normal View History

import AVFoundation
import os
import VoiceCatCore
private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "AudioSessionManager")
@MainActor
final class AudioSessionManager {
static let shared = AudioSessionManager()
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
/// Tracks whether WE activated the session. The session must be active whenever the
/// AudioEngine is running (for capture OR playback), so it is activated when any audio
/// needs to play (a remote stream started OR the user joins voice) and only deactivated
/// when disconnecting from the server not when leaving voice, since the user may still
/// want to hear remote audio.
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
private var isSessionActive = false
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine The iOS audio path was a hybrid: Voice-Chat-class presets ran a native VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every "no input / no output / both" bug lived in the seam between the two paths — the lingering miniaudio capture unit fighting VPIO, the audioRestart ordering dance, the route-change "glitching" loop, stereo<->mono stickiness, and "can't hear anyone". Switching presets/routes mid-call routinely dropped a direction. Drive ALL iOS audio through one AVAudioEngine with the core fully external at all times: setExternalPlayback(1) once at connect, every MIC stream external_feed=1, mic via vc_stream_feed_pcm, playback via vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so remote audio plays before joining voice). VPIO + AGC toggle per preset. Every preset/route/interruption change funnels through one deterministic Swift-only reconfigure (stop -> apply session config -> rebuild -> start) — no second path to hand off to, so a change can't drop a direction. - IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node playback, conditional mic tap, VPIO/AGC; one rebuild() backing startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels. - IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic / Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath. - AudioSessionManager slimmed; SessionState mic lifecycle collapsed; AppState wires external playback + listening at connect, stop at disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles. No core/ABI/test changes — relies on the already-shipped external API (test_external_pcm, test_external_playback). xcodebuild iOS device Debug BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
/// Whether the AVAudioSession is currently active (we activated it). Read by `IOSAudioRouter`
/// to decide whether the post-activation A2DP speaker fallback can be applied.
var isActive: Bool { isSessionActive }
func configure() {
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
// Load stored audio routing preferences and apply them before any audio session
// activation. IOSAudioRouter drives all iOS audio route selection via AVAudioSession;
// miniaudio (the core) does NOT touch AVAudioSession on iOS.
IOSAudioRouter.shared.loadStoredPreferences()
IOSAudioRouter.shared.applyConfiguration()
IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.addObserver(
self, selector: #selector(handleInterruption),
name: AVAudioSession.interruptionNotification, object: nil)
NotificationCenter.default.addObserver(
self, selector: #selector(handleRouteChange),
name: AVAudioSession.routeChangeNotification, object: nil)
}
2026-07-23 13:37:05 +02:00
/// Idempotently restores audio after an interruption or external route change.
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
func recoverAudio() {
guard IOSAudioEngine.shared.isConnected else { return }
do {
try ensureSessionActive()
} catch {
logger.error("recoverAudio — session activate failed: \(error.localizedDescription)")
}
IOSAudioRouter.shared.applyConfiguration()
if isSessionActive { IOSAudioRouter.shared.applyA2dpSpeakerFallback() }
IOSAudioEngine.shared.reconfigure()
logSessionState("after recoverAudio")
}
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
/// Activate the AVAudioSession if not already active. Call before any audio I/O:
/// when the user joins voice, or when a remote stream starts (so playback works even
/// before the user has joined voice). Idempotent safe to call multiple times.
func ensureSessionActive() throws {
guard !isSessionActive else {
logger.debug("ensureSessionActive — already active, skipping")
return
}
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
IOSAudioRouter.shared.applyConfiguration()
fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which achieves stereo mic + A2DP output. Five fixes: 1. configureStereoCapture now calls setPreferredInput + setInputDataSource (mirroring TeamTalk5's SoundDevicesModel). Previously omitted based on incorrect diagnosis that setPreferredInput collapsed A2DP — the real culprit was setPreferredInputNumberOfChannels(2), which neither project uses. 2. New C ABI: vc_audio_restart (full stop + re-init, unlike suspend/resume which only stop/start). Swift wrapper added. The withAudioSuspend wrapper that used it was removed after on-device testing showed it killed all audio (including VoiceOver) when switching presets — the core's set_capture_channels handles engine restart internally. 3. Bluetooth options: Voice Chat preset now includes BOTH .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's UtilSound.swift:228). Previously HFP-only blocked A2DP headphones. 4. Capture channels now reset when switching stereo→mono via selectCaptureChannels/applyPreset. AudioSessionManager tracks activeMicStreamId (set by SessionState on join/leave voice). 5. Docs synced: voice.md, tech-stack.md, architecture.md, PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2) references. Verified: ctest --preset dev 21/21 green, iOS client builds. Stereo mic + A2DP output still needs on-device debugging — the core recipe is correct but iOS 26 route behavior requires hands-on testing with a debugger.
2026-06-19 16:58:21 +02:00
let session = AVAudioSession.sharedInstance()
try session.setActive(true, options: [])
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
isSessionActive = true
// For the A2DP output presets, pick the right output once the session is live: defer to
// a connected A2DP/wired/AirPlay route, but fall back to the loud built-in speaker (not
// the quiet earpiece) when nothing external is connected. See applyA2dpSpeakerFallback().
IOSAudioRouter.shared.applyA2dpSpeakerFallback()
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
let route = AVAudioSession.sharedInstance().currentRoute
let outputNames = route.outputs.map { $0.portName }.joined(separator: ", ")
let inputNames = route.inputs.map { $0.portName }.joined(separator: ", ")
logger.info("session activated — outputs: [\(outputNames)], inputs: [\(inputNames)]")
fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death) The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25 runs an iOS "hack" that sets the session category by device type, then ma_context_init__coreaudio calls setCategory()+setActive() on every device open -- capture -> AVAudioSessionCategoryRecord with zero options. That wipes the .playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers / .allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and even wired) output. Stereo presets break worst because they rely on the A2DP output route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits directly and leaving the session entirely to the app. Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls (playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already activated on connect in AppState before any device opens). Context is lazily inited in start(), reused across restarts, uninited in ~AudioEngine. Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route change, on .streamStarted) to verify on-device that the category stays PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed. Windows: cmake --build --preset dev clean; ctest --preset dev 21/21. iOS build + on-device verification pending on Mac. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:34:07 +02:00
logSessionState("after activate")
}
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
/// Deactivate the AVAudioSession. Call ONLY when disconnecting from the server not
/// when leaving voice (the user may still want to hear remote audio).
func deactivateSession() {
guard isSessionActive else {
logger.debug("deactivateSession — not active, skipping")
return
}
try? AVAudioSession.sharedInstance().setActive(false,
options: .notifyOthersOnDeactivation)
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
isSessionActive = false
logger.info("session deactivated")
}
/// Log the full AVAudioSession state category, mode, options, and active route.
/// Useful for diagnosing routing issues, e.g. confirming the session stays
/// `PlayAndRecord` with `allowBluetoothA2DP` and keeps the A2DP output route even
/// after the mic engine starts.
fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death) The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25 runs an iOS "hack" that sets the session category by device type, then ma_context_init__coreaudio calls setCategory()+setActive() on every device open -- capture -> AVAudioSessionCategoryRecord with zero options. That wipes the .playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers / .allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and even wired) output. Stereo presets break worst because they rely on the A2DP output route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits directly and leaving the session entirely to the app. Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls (playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already activated on connect in AppState before any device opens). Context is lazily inited in start(), reused across restarts, uninited in ~AudioEngine. Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route change, on .streamStarted) to verify on-device that the category stays PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed. Windows: cmake --build --preset dev clean; ctest --preset dev 21/21. iOS build + on-device verification pending on Mac. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:34:07 +02:00
func logSessionState(_ when: String) {
let s = AVAudioSession.sharedInstance()
var opts: [String] = []
let o = s.categoryOptions
if o.contains(.mixWithOthers) { opts.append("mixWithOthers") }
if o.contains(.duckOthers) { opts.append("duckOthers") }
if o.contains(.allowBluetoothHFP) { opts.append("allowBluetoothHFP") }
if o.contains(.allowBluetoothA2DP) { opts.append("allowBluetoothA2DP") }
if o.contains(.allowAirPlay) { opts.append("allowAirPlay") }
if o.contains(.defaultToSpeaker) { opts.append("defaultToSpeaker") }
let outs = s.currentRoute.outputs.map { "\($0.portName)[\($0.portType.rawValue)]" }
.joined(separator: ", ")
let ins = s.currentRoute.inputs.map { "\($0.portName)[\($0.portType.rawValue)]" }
.joined(separator: ", ")
logger.info("""
[SESSION @ \(when, privacy: .public)] category=\(s.category.rawValue, privacy: .public) \
mode=\(s.mode.rawValue, privacy: .public) options=[\(opts.joined(separator: ","), privacy: .public)] \
inputs=[\(ins, privacy: .public)] outputs=[\(outs, privacy: .public)] \
inputCh=\(s.inputNumberOfChannels) outputCh=\(s.outputNumberOfChannels)
""")
}
@objc private func handleInterruption(_ notification: Notification) {
guard let info = notification.userInfo,
let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
let type = AVAudioSession.InterruptionType(rawValue: typeValue)
else { return }
switch type {
case .began:
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine The iOS audio path was a hybrid: Voice-Chat-class presets ran a native VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every "no input / no output / both" bug lived in the seam between the two paths — the lingering miniaudio capture unit fighting VPIO, the audioRestart ordering dance, the route-change "glitching" loop, stereo<->mono stickiness, and "can't hear anyone". Switching presets/routes mid-call routinely dropped a direction. Drive ALL iOS audio through one AVAudioEngine with the core fully external at all times: setExternalPlayback(1) once at connect, every MIC stream external_feed=1, mic via vc_stream_feed_pcm, playback via vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so remote audio plays before joining voice). VPIO + AGC toggle per preset. Every preset/route/interruption change funnels through one deterministic Swift-only reconfigure (stop -> apply session config -> rebuild -> start) — no second path to hand off to, so a change can't drop a direction. - IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node playback, conditional mic tap, VPIO/AGC; one rebuild() backing startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels. - IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic / Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath. - AudioSessionManager slimmed; SessionState mic lifecycle collapsed; AppState wires external playback + listening at connect, stop at disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles. No core/ABI/test changes — relies on the already-shipped external API (test_external_pcm, test_external_playback). xcodebuild iOS device Debug BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
// The system stops our AVAudioEngine and deactivates the session. Nothing to tear
// down `IOSAudioEngine` rebuilds on resume.
fix(ios): output muted with A2DP, session lifecycle, mic input issues Three bugs causing no audio output and no mic input: 1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS mutes the output because it can't set up the voice processing pipeline on an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC, but audio routes correctly). .voiceChat kept for HFP and speaker modes. Added info warning in Settings UI for A2DP no-AEC. 2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming() which deactivated the AVAudioSession — but the AudioEngine keeps running for remote audio playback, so leaving voice killed all remote audio. And the session was never activated when a remote user started talking (only on Join Voice), so you couldn't hear anyone before joining voice. Fix: - ensureSessionActive() replaces activateForStreaming() — idempotent, called on Join Voice AND on .streamStarted (remote user starts talking). - stopMicStream() no longer deactivates the session. - deactivateSession() called only on disconnect from server. - isSessionActive flag tracks state, updated by interruption handler. 3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is the default) and may put the session in a bad state on some devices. Fix: only call it when stereo is explicitly selected. Also handle empty input port ID (selecting 'Default' in the picker) correctly. Added comprehensive route logging — after activation, logs the current output and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
logger.info("interruption began — session suspended by system")
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine The iOS audio path was a hybrid: Voice-Chat-class presets ran a native VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every "no input / no output / both" bug lived in the seam between the two paths — the lingering miniaudio capture unit fighting VPIO, the audioRestart ordering dance, the route-change "glitching" loop, stereo<->mono stickiness, and "can't hear anyone". Switching presets/routes mid-call routinely dropped a direction. Drive ALL iOS audio through one AVAudioEngine with the core fully external at all times: setExternalPlayback(1) once at connect, every MIC stream external_feed=1, mic via vc_stream_feed_pcm, playback via vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so remote audio plays before joining voice). VPIO + AGC toggle per preset. Every preset/route/interruption change funnels through one deterministic Swift-only reconfigure (stop -> apply session config -> rebuild -> start) — no second path to hand off to, so a change can't drop a direction. - IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node playback, conditional mic tap, VPIO/AGC; one rebuild() backing startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels. - IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic / Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath. - AudioSessionManager slimmed; SessionState mic lifecycle collapsed; AppState wires external playback + listening at connect, stop at disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles. No core/ABI/test changes — relies on the already-shipped external API (test_external_pcm, test_external_playback). xcodebuild iOS device Debug BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
isSessionActive = false
case .ended:
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
// Always attempt recovery when we have a live session. iOS sometimes ends an
// interruption without the `.shouldResume` hint (e.g. Siri), and the previous
// behavior of only reactivating when `.shouldResume` was set left the session
// permanently dead audio never came back. `recoverAudio()` is intent-gated on
// `IOSAudioEngine.isConnected` and idempotent, so speculatively calling it is safe.
logger.info("interruption ended — recovery requested")
recoverAudio()
@unknown default: break
}
}
@objc private func handleRouteChange(_ notification: Notification) {
guard let info = notification.userInfo,
let reasonValue = info[AVAudioSessionRouteChangeReasonKey] as? UInt,
let reason = AVAudioSession.RouteChangeReason(rawValue: reasonValue)
else {
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
logger.warning("routeChange — unknown reason, refreshing + recovery")
IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
recoverAudio()
return
}
logger.info("routeChange reason=\(self.reasonLabel(reason))")
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
2026-07-23 13:37:05 +02:00
// Ignore notifications caused by our own configuration calls; rebuilding for them
// recursively emits more route changes. Engine-configuration notifications remain
// the recovery path if a self-initiated change actually stops AVAudioEngine.
fix(ios): break AirPods-disconnect reinitialize loop on A2DP presets Recovery from the 99937c9 audio-device-change commit broadened the route-change recovery set to "everything except categoryChange / routeConfigurationChange", which added .override. But .override is fired by our own applyA2dpSpeakerFallback() -> overrideOutputAudioPort, which recoverAudio() calls on every recovery. On an A2DP preset (Stereo Mic / Mono Mic), disconnecting AirPods ping-ponged: oldDeviceUnavailable -> recoverAudio -> applyA2dpSpeakerFallback -> overrideOutputAudioPort(.speaker) -> .override routeChange -> recoverAudio -> applyConfiguration (setCategory resets override) -> applyA2dpSpeakerFallback -> overrideOutputAudioPort -> .override -> ... Each iteration also rebuilt the AVAudioEngine via reconfigure() -> rebuild() -- the audible reinitialize loop + CPU spin. Voice Chat and Built-in Mic + Speaker were unaffected (applyA2dpSpeakerFallback early-returns for non-A2DP modes, so no overrideOutputAudioPort call). Two-part fix (pure Swift iOS-app target; no C ABI / proto / docs changes): 1. AudioSessionManager.handleRouteChange: added .override to the skip list alongside .categoryChange / .routeConfigurationChange. .override is only ever fired by our own overrideOutputAudioPort call, so treating it as a recovery reason is the loop by definition. The AVAudioEngineConfigurationChange observer in IOSVoiceProcessingEngine remains as the backstop if an override ever actually stops the engine. 2. IOSAudioRouter.applyA2dpSpeakerFallback: made idempotent via a lastAppliedOutputOverride tracker that skips the redundant overrideOutputAudioPort call when the desired state (.none for external output present, .speaker otherwise) already matches. Reset to nil at the top of applyConfiguration() (setCategory can reset the override) and on a failed call. Defense-in-depth on top of fix 1. Build: xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj -scheme VoiceCatiOS -destination 'generic/platform=iOS' build green (Xcode 26.5 / iOS 18.0).
2026-06-25 16:18:21 +02:00
if reason != .categoryChange && reason != .routeConfigurationChange && reason != .override {
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
recoverAudio()
}
fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death) The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25 runs an iOS "hack" that sets the session category by device type, then ma_context_init__coreaudio calls setCategory()+setActive() on every device open -- capture -> AVAudioSessionCategoryRecord with zero options. That wipes the .playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers / .allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and even wired) output. Stereo presets break worst because they rely on the A2DP output route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits directly and leaving the session entirely to the app. Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls (playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already activated on connect in AppState before any device opens). Context is lazily inited in start(), reused across restarts, uninited in ~AudioEngine. Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route change, on .streamStarted) to verify on-device that the category stays PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed. Windows: cmake --build --preset dev clean; ctest --preset dev 21/21. iOS build + on-device verification pending on Mac. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:34:07 +02:00
logSessionState("route change (\(reasonLabel(reason)))")
}
private func reasonLabel(_ reason: AVAudioSession.RouteChangeReason) -> String {
switch reason {
case .oldDeviceUnavailable: return "oldDeviceUnavailable"
case .newDeviceAvailable: return "newDeviceAvailable"
case .categoryChange: return "categoryChange"
case .override: return "override"
case .wakeFromSleep: return "wakeFromSleep"
case .noSuitableRouteForCategory: return "noSuitableRouteForCategory"
case .routeConfigurationChange: return "routeConfigurationChange"
feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
case .unknown: return "unknown"
@unknown default: return "unknown"
}
}
}
extension Notification.Name {
static let voiceCatDeviceListChanged = Notification.Name("cat.voice.deviceListChanged")
}