fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine

The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.

Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.

- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
  playback, conditional mic tap, VPIO/AGC; one rebuild() backing
  startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
  Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
  IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
  AppState wires external playback + listening at connect, stop at
  disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.

No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
This commit is contained in:
2026-06-23 02:45:53 +02:00
parent 7547b8e140
commit d30c4ee2f5
8 changed files with 438 additions and 414 deletions

View File

@@ -4,17 +4,18 @@ import VoiceCatCore
private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAudioRouter")
/// iOS audio routing layer drives all iOS audio route selection via `AVAudioSession`
/// *before* the core (miniaudio) opens its device. This class is the sole owner of the
/// session: miniaudio does NOT touch `AVAudioSession` on iOS, because the core opens its
/// devices through a `ma_context` configured with `sessionCategory = none` +
/// `noAudioSessionActivate/Deactivate` (see `AudioEngine::make_context_config` in
/// `core/src/audio/audio_engine.cpp`). Without that, miniaudio's default path resets the
/// category to `Record`/`Playback` with no options on every device open, wiping
/// `.allowBluetoothA2DP`/`.playAndRecord` and killing headphone/A2DP output so that
/// config must stay in place. All iOS audio routing (input port selection, mic
/// orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture) must be
/// driven from here.
/// iOS audio routing layer the sole owner of `AVAudioSession` on iOS. On iOS the core never
/// opens a hardware (miniaudio) device: a single `AVAudioEngine` (`IOSAudioEngine`) drives both
/// capture and playback and the core runs fully external (see docs/voice.md §8). This class just
/// configures the *route* category / mode / options, preferred input, data source, polar
/// pattern, stereo capsule and `IOSAudioEngine` binds to whatever route is established. After
/// any change here the engine is rebuilt via `IOSAudioEngine.reconfigure()` (a deterministic
/// Swift-only stop reconfigure start); there is no second (miniaudio) audio path to hand off
/// to, so a change cannot leave one direction dropped.
///
/// (The core's iOS `ma_context` is still configured with `sessionCategory = none` +
/// `noAudioSessionActivate/Deactivate` in `AudioEngine::make_context_config` so that, should the
/// core ever open a device, miniaudio would not reset the category but on iOS it does not.)
///
/// The three user-facing choices:
/// 1. **Input port** which physical input (built-in mic, Bluetooth HFP, headset,
@@ -64,77 +65,59 @@ final class IOSAudioRouter: ObservableObject {
@Published var selectedInputPortId: String?
@Published var selectedDataSourceId: String?
@Published var selectedPolarPattern: String?
/// Master voice-processing switch (Apple VPIO: AEC + noise suppression bundled together).
/// iOS exposes no per-stage toggle, so this is the finest "echo cancellation / noise
/// reduction" control available. Only takes effect on a VPIO-capable config (mono + standard
/// + not A2DP); stereo / A2DP configs can't use VPIO regardless. Default on.
@Published var voiceProcessingEnabled: Bool = true
/// VPIO automatic gain control the one VPIO sub-stage iOS lets us toggle independently.
/// Only meaningful when voice processing is active. Default on.
@Published var agcEnabled: Bool = true
@Published var showsRawModeSpeakerWarning: Bool = false
@Published var showsA2dpNoAecWarning: Bool = false
@Published var hasBluetoothDevice: Bool = false
@Published var hasWiredHeadset: Bool = false
/// Audio presets sensible combinations of settings for common scenarios.
/// The app is about choice: users can pick a preset for a quick start, then
/// fine-tune individual settings under "Advanced Audio".
/// Audio presets the four scenarios from the product spec. Pick a preset for a quick start,
/// then fine-tune individual settings under "Advanced". HFP / wired headsets are not separate
/// presets: Voice Chat lets the system route to them, and Advanced exposes manual selection.
enum AudioPreset: String, CaseIterable, Identifiable {
/// Standard iOS VoIP experience: AEC/AGC/HPF on, mono, system picks best route
/// (BT HFP if connected, wired if connected, speaker if nothing). Always available.
/// Voice chat: Apple VPIO does real AEC + noise suppression + AGC. Mono. The system picks
/// the best route (Bluetooth HFP / wired / speaker / earpiece). Always available.
case voiceChat = "Voice Chat"
/// Stereo built-in mic capture (front+back capsules). A2DP output if BT is connected,
/// else built-in speaker / wired. Standard processing (no AEC stereo needs a non-VPIO
/// mode). Always available.
/// Internal **stereo** built-in mic regardless of the output route. A2DP output when a
/// Bluetooth headset is connected, else built-in speaker / wired. No VPIO (stereo can't
/// use it). Always available.
case stereoMic = "Stereo Mic"
/// Maximum fidelity: stereo mic, no AEC/AGC/HPF (raw mode). A2DP output if BT connected,
/// else speaker/wired. Always available. Echo risk on speaker.
case studio = "Studio (No Processing)"
/// Bluetooth HFP: BT mic + BT output, AEC on, mono. Only when BT is connected.
case bluetoothHeadset = "Bluetooth Headset (HFP)"
/// A2DP stereo output + built-in mono mic, AEC off. Only when BT is connected. (For
/// A2DP output + stereo mic, use the Stereo Mic preset while BT is connected.)
case btHeadphonesMonoMic = "BT Headphones + Mono Mic"
/// Wired headset/earpods: wired output + wired mic (or built-in), AEC on, mono.
/// Only when a wired audio device is connected.
case wiredHeadset = "Wired Headset"
/// Settings don't match any preset user has tweaked advanced controls.
case custom = "Custom"
/// Internal **mono** built-in mic regardless of the output route. A2DP output when a
/// Bluetooth headset is connected, else built-in speaker / wired. No VPIO. Always available.
case monoMic = "Mono Mic"
/// Everything manual input port, mic orientation / polar pattern, mono/stereo, Bluetooth
/// mode, raw vs standard, and the VPIO / AGC toggles. Also the display state when the
/// individual settings don't match a named preset.
case advanced = "Advanced"
var id: String { rawValue }
var requiresBluetooth: Bool {
switch self {
case .bluetoothHeadset, .btHeadphonesMonoMic: return true
default: return false
}
}
var requiresWired: Bool {
self == .wiredHeadset
}
var bluetoothMode: BluetoothMode {
switch self {
case .voiceChat, .bluetoothHeadset: return .btHfpVoice
// A2DP output when BT is connected; falls back to speaker/wired when it isn't.
case .stereoMic, .studio, .btHeadphonesMonoMic: return .builtInMicBtA2dp
case .wiredHeadset: return .builtInMicSpeaker
case .custom: return .builtInMicSpeaker // placeholder
case .voiceChat: return .btHfpVoice
// Internal-mic presets: A2DP output when BT is connected; speaker/wired when not.
case .stereoMic, .monoMic: return .builtInMicBtA2dp
case .advanced: return .builtInMicSpeaker // placeholder; Advanced sets it manually
}
}
var captureChannels: CaptureChannels {
switch self {
case .stereoMic, .studio: return .stereo
default: return .mono
}
self == .stereoMic ? .stereo : .mono
}
var micMode: MicMode {
switch self {
case .studio: return .raw
default: return .standard
}
}
var micMode: MicMode { .standard }
/// Whether this preset explicitly selects the built-in mic port.
/// Whether this preset explicitly pins the built-in mic port (the internal-mic presets).
var usesBuiltInMic: Bool {
switch self {
case .stereoMic, .studio, .btHeadphonesMonoMic: return true
case .stereoMic, .monoMic: return true
default: return false
}
}
@@ -170,6 +153,8 @@ final class IOSAudioRouter: ObservableObject {
private let kPolarPattern = "cat.voice.audio.polarPattern"
private let kPreset = "cat.voice.audio.preset"
private let kForceSpeaker = "cat.voice.audio.forceSpeaker"
private let kVoiceProcessing = "cat.voice.audio.voiceProcessing"
private let kAgc = "cat.voice.audio.agc"
/// Re-entrancy guard: setCategory/setPreferredInput/etc. trigger route-change
/// notifications synchronously on the same thread. Without this guard,
@@ -271,55 +256,41 @@ final class IOSAudioRouter: ObservableObject {
}
}
/// The presets available given the current device connection state.
/// Always includes Voice Chat, Stereo Mic, Studio, and Custom. BT presets only when
/// a Bluetooth device is connected. Wired preset only when a wired device is connected.
var availablePresets: [AudioPreset] {
AudioPreset.allCases.filter { preset in
if preset == .custom { return true }
if preset.requiresBluetooth && !hasBluetoothDevice { return false }
if preset.requiresWired && !hasWiredHeadset { return false }
return true
}
}
/// The presets the user can pick. All four are always available the named presets simply
/// describe what to do "regardless of the output route", and Advanced is always offered.
var availablePresets: [AudioPreset] { AudioPreset.allCases }
/// Which preset matches the current settings, or .custom if nothing matches.
/// Checks device-specific presets first (BT, wired) so that e.g. when BT is connected
/// and settings match "Bluetooth Headset", it returns that instead of the equivalent
/// "Voice Chat" (which has the same bluetoothMode/micMode/channels but is more general).
/// Which named preset matches the current settings, or `.advanced` if nothing matches.
var activePreset: AudioPreset {
// Check device-specific presets first (most specific least specific)
let order: [AudioPreset] = [
.bluetoothHeadset, .btHeadphonesMonoMic,
.wiredHeadset,
.voiceChat, .stereoMic, .studio,
]
for preset in order {
for preset in [AudioPreset.voiceChat, .stereoMic, .monoMic] {
if bluetoothMode == preset.bluetoothMode
&& captureChannels == preset.captureChannels
&& micMode == preset.micMode {
// Don't match a BT preset if no BT is connected fall through to Voice Chat
if preset.requiresBluetooth && !hasBluetoothDevice { continue }
if preset.requiresWired && !hasWiredHeadset { continue }
return preset
}
}
return .custom
return .advanced
}
/// Whether the current configuration should use the native iOS Voice-Processing path (VPIO:
/// real AEC/NS/AGC via `IOSVoiceProcessingEngine`). True exactly when `applyConfiguration`
/// selects the `.voiceChat` AVAudioSession mode mono + standard processing + not A2DP
/// (A2DP / stereo / raw modes can't use VPIO, so they keep the core's miniaudio path).
/// Whether the current configuration should engage Apple's Voice-Processing I/O unit (VPIO:
/// real AEC + noise suppression + AGC, driven by `IOSAudioEngine`). VPIO forces mono and
/// can't run on an A2DP route, so it is available only for a mono + standard + non-A2DP
/// config, and then only when the user hasn't disabled it via the Advanced master toggle.
var currentConfigUsesVoiceProcessing: Bool {
voiceProcessingEnabled && voiceProcessingAvailable
}
/// Whether the current config *could* use VPIO (mono + standard + non-A2DP), independent of
/// the user's master toggle. Drives whether the Advanced "Voice Processing" switch is shown.
var voiceProcessingAvailable: Bool {
captureChannels == .mono && micMode == .standard && bluetoothMode != .builtInMicBtA2dp
}
// MARK: - Apply configuration
/// Apply the full audio configuration to AVAudioSession. Call this before the core
/// opens its capture device (i.e. before `startMicStream` `activateForStreaming`).
/// Re-entrant-safe: if a route-change notification fires synchronously during a
/// Apply the full audio configuration to AVAudioSession. Call this before (re)building the
/// `IOSAudioEngine` graph so the engine binds to the intended route (`applyAndReconfigure`
/// does both). Re-entrant-safe: if a route-change notification fires synchronously during a
/// `setCategory`/`setPreferredInput` call, the guard prevents re-entry.
func applyConfiguration() {
guard !isApplyingConfiguration else {
@@ -401,8 +372,8 @@ final class IOSAudioRouter: ObservableObject {
// route explicitly via setPreferredInput + setInputDataSource. With HFP disabled
// the system routes input to the built-in mic, but without the explicit
// preferred-input anchor the route can collapse during the mode switch
// (.voiceChat .default) and the output dies. The channel count is requested by
// miniaudio at the audio-unit level (vc_set_capture_channels), NOT via
// (.voiceChat .default) and the output dies. The channel count is carried by the
// engine's mic tap + vc_set_capture_channels, NOT via
// setPreferredInputNumberOfChannels(2) that call collapses the A2DP output route.
configureStereoCapture(session: session)
} else if let portId = selectedInputPortId, !portId.isEmpty,
@@ -431,9 +402,9 @@ final class IOSAudioRouter: ObservableObject {
/// 3. `setPreferredInput(builtIn)` anchor the input route explicitly. Without this
/// anchor the route can collapse during the mode switch (.voiceChat .default).
/// 4. `setInputDataSource(stereoSource)` commit the data source at the session level
/// The channel count itself is requested by miniaudio at the audio-unit level via
/// `vc_set_capture_channels(2)`. We must NOT call `setPreferredInputNumberOfChannels(2)`
/// that session-level call collapses the A2DP output route.
/// The channel count itself is carried by the engine's mic tap (which captures 2 channels)
/// plus `vc_set_capture_channels(2)` so the core encodes stereo. We must NOT call
/// `setPreferredInputNumberOfChannels(2)` that session-level call collapses the A2DP route.
private func configureStereoCapture(session: AVAudioSession) {
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
else {
@@ -538,6 +509,9 @@ final class IOSAudioRouter: ObservableObject {
selectedDataSourceId = UserDefaults.standard.string(forKey: kDataSourceId)
selectedPolarPattern = UserDefaults.standard.string(forKey: kPolarPattern)
forceSpeaker = UserDefaults.standard.bool(forKey: kForceSpeaker)
// VPIO toggles default ON when never set (object(forKey:) is nil use true).
voiceProcessingEnabled = (UserDefaults.standard.object(forKey: kVoiceProcessing) as? Bool) ?? true
agcEnabled = (UserDefaults.standard.object(forKey: kAgc) as? Bool) ?? true
}
/// Persist current selections to UserDefaults.
func savePreferences() {
@@ -548,86 +522,88 @@ final class IOSAudioRouter: ObservableObject {
UserDefaults.standard.set(selectedDataSourceId, forKey: kDataSourceId)
UserDefaults.standard.set(selectedPolarPattern, forKey: kPolarPattern)
UserDefaults.standard.set(forceSpeaker, forKey: kForceSpeaker)
UserDefaults.standard.set(voiceProcessingEnabled, forKey: kVoiceProcessing)
UserDefaults.standard.set(agcEnabled, forKey: kAgc)
}
// MARK: - Selection setters (called from SettingsView pickers)
/// Shared tail for every setting change: persist, re-apply the AVAudioSession config, refresh
/// the route lists, re-evaluate the A2DP speaker fallback, and rebind the live engine to the
/// new route. `IOSAudioEngine.reconfigure()` is a no-op when not connected, so this is safe to
/// call from Settings whether or not a session is in progress. There is no longer a second
/// (miniaudio) audio path to hand off to, so one engine rebuild is the whole story.
private func applyAndReconfigure() {
savePreferences()
applyConfiguration()
if AudioSessionManager.shared.isActive { applyA2dpSpeakerFallback() }
refreshRoutes()
IOSAudioEngine.shared.reconfigure()
}
func selectInputPort(_ portId: String) {
selectedInputPortId = portId
selectedDataSourceId = nil
selectedPolarPattern = nil
savePreferences()
applyConfiguration()
refreshRoutes()
applyAndReconfigure()
}
func selectDataSource(_ dataSourceId: String) {
selectedDataSourceId = dataSourceId
selectedPolarPattern = nil
savePreferences()
applyConfiguration()
refreshRoutes()
applyAndReconfigure()
}
func selectPolarPattern(_ pattern: String) {
selectedPolarPattern = pattern
savePreferences()
applyConfiguration()
refreshRoutes()
applyAndReconfigure()
}
func selectBluetoothMode(_ mode: BluetoothMode) {
bluetoothMode = mode
savePreferences()
applyConfiguration()
refreshRoutes()
// VPIO class or route may have changed restart the voice path if mic is active.
AudioSessionManager.shared.reconcileVoicePath?()
applyAndReconfigure()
}
func setForceSpeaker(_ on: Bool) {
forceSpeaker = on
savePreferences()
applyConfiguration()
refreshRoutes()
// Route changed under a possibly-running VPIO engine reconcile if mic is active.
AudioSessionManager.shared.reconcileVoicePath?()
applyAndReconfigure()
}
func selectMicMode(_ mode: MicMode) {
micMode = mode
applyAndReconfigure()
}
func setVoiceProcessingEnabled(_ on: Bool) {
voiceProcessingEnabled = on
applyAndReconfigure()
}
func setAgcEnabled(_ on: Bool) {
agcEnabled = on
// No session reconfigure needed just rebuild the engine so VPIO picks up the AGC flag.
savePreferences()
applyConfiguration()
updateWarnings()
// StandardRaw flips the VPIO class reconcile if mic is active.
AudioSessionManager.shared.reconcileVoicePath?()
IOSAudioEngine.shared.reconfigure()
}
func selectCaptureChannels(_ channels: CaptureChannels) {
captureChannels = channels
savePreferences()
applyConfiguration()
// Update the core's stored capture channel count (does not restart the engine).
if let streamId = AudioSessionManager.shared.activeMicStreamId {
_ = AudioSessionManager.shared.client?.setCaptureChannels(
streamId: streamId, channels: channels.channelCount)
}
// Restart the engine AFTER AVAudioSession routing has settled and the channel
// count is stored. The engine reopens playback first (committing the A2DP/output
// route), then capture avoiding the race where stereo capture activation drops
// A2DP before the playback device has a chance to claim the route.
_ = AudioSessionManager.shared.client?.audioRestart()
// Monostereo flips the VPIO class (stereo can't use VPIO) reconcile if mic is active.
AudioSessionManager.shared.reconcileVoicePath?()
if AudioSessionManager.shared.isActive { applyA2dpSpeakerFallback() }
refreshRoutes()
// Push the channel count into the core's MIC stream, then rebuild the engine graph so the
// mic tap captures the right number of channels. The engine owns the route now, so there's
// no stereo-vs-A2DP race to sequence around.
IOSAudioEngine.shared.setCaptureChannels(channels.channelCount)
}
// MARK: - Presets
/// Apply a preset sets all individual audio settings to the preset's values, then
/// applies the configuration. For presets that use the built-in mic (A2DP presets),
/// finds the built-in mic port UID from availableInputs.
/// Apply a named preset set all individual settings to the preset's values, then re-apply
/// the configuration and rebind the engine. The internal-mic presets pin the built-in mic.
func applyPreset(_ preset: AudioPreset) {
guard preset != .custom else { return } // can't "apply" custom it's a display state
guard preset != .advanced else { return } // Advanced is a display state, not "applied"
bluetoothMode = preset.bluetoothMode
micMode = preset.micMode
@@ -638,19 +614,17 @@ final class IOSAudioRouter: ObservableObject {
if preset == .voiceChat { forceSpeaker = true }
if preset.usesBuiltInMic {
// Find the built-in mic port from available inputs and select it.
let session = AVAudioSession.sharedInstance()
if let builtInMic = (session.availableInputs ?? []).first(where: {
// Pin the built-in mic. In stereo, iOS uses multiple capsules automatically; in mono
// the default orientation is fine so don't force a specific data source / pattern.
if let builtInMic = (AVAudioSession.sharedInstance().availableInputs ?? []).first(where: {
$0.portType == .builtInMic
}) {
selectedInputPortId = builtInMic.uid
}
// Don't set a specific data source in stereo mode, iOS uses multiple mic
// capsules automatically. In mono, the default orientation is fine.
selectedDataSourceId = nil
selectedPolarPattern = nil
} else {
// For Default and Bluetooth Headset presets, let the system pick the input.
// Voice Chat: let the system pick the input (Bluetooth HFP / wired / built-in).
selectedInputPortId = nil
selectedDataSourceId = nil
selectedPolarPattern = nil
@@ -659,18 +633,11 @@ final class IOSAudioRouter: ObservableObject {
UserDefaults.standard.set(preset.rawValue, forKey: kPreset)
savePreferences()
applyConfiguration()
// Update the core's stored capture channel count (does not restart the engine).
if let streamId = AudioSessionManager.shared.activeMicStreamId {
_ = AudioSessionManager.shared.client?.setCaptureChannels(
streamId: streamId, channels: preset.captureChannels.channelCount)
}
// Restart the engine AFTER AVAudioSession routing has settled and the channel
// count is stored. Playback opens first (commits A2DP route), then capture.
_ = AudioSessionManager.shared.client?.audioRestart()
if AudioSessionManager.shared.isActive { applyA2dpSpeakerFallback() }
refreshRoutes()
// The preset may have flipped the VPIO class (and/or the route) restart the voice path
// if the mic is active so AEC/NS engage (or disengage) to match the new preset.
AudioSessionManager.shared.reconcileVoicePath?()
// Push the channel count to the core, then rebuild the engine graph (VPIO on/off + tap).
IOSAudioEngine.shared.setCaptureChannels(preset.captureChannels.channelCount)
IOSAudioEngine.shared.reconfigure()
logger.info("applyPreset — \(preset.rawValue)")
}