fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI

Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:

1. configureStereoCapture now calls setPreferredInput +
   setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
   Previously omitted based on incorrect diagnosis that
   setPreferredInput collapsed A2DP — the real culprit was
   setPreferredInputNumberOfChannels(2), which neither project uses.

2. New C ABI: vc_audio_restart (full stop + re-init, unlike
   suspend/resume which only stop/start). Swift wrapper added.
   The withAudioSuspend wrapper that used it was removed after
   on-device testing showed it killed all audio (including
   VoiceOver) when switching presets — the core's
   set_capture_channels handles engine restart internally.

3. Bluetooth options: Voice Chat preset now includes BOTH
   .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
   UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.

4. Capture channels now reset when switching stereo→mono via
   selectCaptureChannels/applyPreset. AudioSessionManager tracks
   activeMicStreamId (set by SessionState on join/leave voice).

5. Docs synced: voice.md, tech-stack.md, architecture.md,
   PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
   references.

Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
This commit is contained in:
2026-06-19 16:58:21 +02:00
parent 1a1c8a1dfe
commit fdcc84fb42
14 changed files with 399 additions and 100 deletions

View File

@@ -16,7 +16,8 @@ private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAu
/// (orientation: front/back/top/bottom) and **polar pattern**
/// (omni/cardioid/subcardioid/bidirectional).
/// 2. **Bluetooth mode** how Bluetooth headsets are handled:
/// - "BT HFP voice" (`.allowBluetooth`): mono 8/16 kHz + heavy processing, BT mic.
/// - "BT HFP voice" (`.allowBluetoothHFP` + `.allowBluetoothA2DP`): both profiles
/// allowed, iOS picks HFP for two-way mic or A2DP for output-only. Mono, AEC on.
/// - "Built-in Mic + BT A2DP stereo" (`.allowBluetoothA2DP` only): stereo output,
/// built-in mic, no HFP processing.
/// - "Built-in Mic + Speaker" (neither): no Bluetooth at all.
@@ -24,9 +25,17 @@ private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAu
/// Raw/Studio (`.measurement`: all processing off). Raw mode is allowed always
/// but shows a warning when the output route is the speaker (echo risk, no AEC).
///
/// Additionally, **stereo capture** (2-channel built-in mic) can be enabled via
/// `setPreferredInputNumberOfChannels(2)` the core is then told via
/// `vc_set_capture_channels(streamId, 2)`.
/// Additionally, **stereo capture** (2-channel built-in mic) is enabled by switching the
/// built-in mic's data source to the `.stereo` polar pattern. The recipe (mirroring
/// TeamTalk5's `SoundDevicesModel.selectDataSource` + `UtilSound.setupSoundDevices`, which
/// achieves stereo mic + A2DP output simultaneously) is: `setPreferredDataSource(.stereo
/// source)` + `setPreferredPolarPattern(.stereo)` + `setPreferredInput(built-in mic)` +
/// `setInputDataSource(stereo source)`. The channel count itself is NOT requested via
/// `setPreferredInputNumberOfChannels(2)` that session-level call is what collapses the
/// A2DP output route (the original "stereo kills output" bug). Instead the core is told to
/// open the device with 2 channels via `vc_set_capture_channels(streamId, 2)`, and the
/// AVAudioSession input anchor (`setPreferredInput` + `setInputDataSource`) keeps the route
/// stable during the HFPA2DP and monostereo reconfigurations.
///
/// Voice Isolation / Wide Spectrum (iOS 17+/18+) are user-toggleable in Control Center
/// for `.voiceChat` apps surfaced as a hint, not a programmatic toggle.
@@ -59,20 +68,18 @@ final class IOSAudioRouter: ObservableObject {
/// Standard iOS VoIP experience: AEC/AGC/HPF on, mono, system picks best route
/// (BT HFP if connected, wired if connected, speaker if nothing). Always available.
case voiceChat = "Voice Chat"
/// Stereo built-in mic capture (front+back capsules). A2DP output if BT is
/// connected, else speaker/wired. Standard processing. Always available.
/// Stereo built-in mic capture (front+back capsules). A2DP output if BT is connected,
/// else built-in speaker / wired. Standard processing (no AEC stereo needs a non-VPIO
/// mode). Always available.
case stereoMic = "Stereo Mic"
/// Maximum fidelity: stereo mic, no AEC/AGC/HPF (raw mode). A2DP output if BT
/// connected, else speaker/wired. Always available. Echo risk on speaker.
/// Maximum fidelity: stereo mic, no AEC/AGC/HPF (raw mode). A2DP output if BT connected,
/// else speaker/wired. Always available. Echo risk on speaker.
case studio = "Studio (No Processing)"
/// Bluetooth HFP: BT mic + BT output, AEC on, mono. Only when BT is connected.
case bluetoothHeadset = "Bluetooth Headset (HFP)"
/// A2DP stereo output + built-in mono mic. No hardware AEC (A2DP incompatible).
/// Only when BT is connected.
/// A2DP stereo output + built-in mono mic, AEC off. Only when BT is connected. (For
/// A2DP output + stereo mic, use the Stereo Mic preset while BT is connected.)
case btHeadphonesMonoMic = "BT Headphones + Mono Mic"
/// A2DP stereo output + stereo built-in mic (front+back). No hardware AEC.
/// Only when BT is connected.
case btHeadphonesStereoMic = "BT Headphones + Stereo Mic"
/// Wired headset/earpods: wired output + wired mic (or built-in), AEC on, mono.
/// Only when a wired audio device is connected.
case wiredHeadset = "Wired Headset"
@@ -83,7 +90,7 @@ final class IOSAudioRouter: ObservableObject {
var requiresBluetooth: Bool {
switch self {
case .bluetoothHeadset, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return true
case .bluetoothHeadset, .btHeadphonesMonoMic: return true
default: return false
}
}
@@ -95,7 +102,8 @@ final class IOSAudioRouter: ObservableObject {
var bluetoothMode: BluetoothMode {
switch self {
case .voiceChat, .bluetoothHeadset: return .btHfpVoice
case .stereoMic, .studio, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return .builtInMicBtA2dp
// A2DP output when BT is connected; falls back to speaker/wired when it isn't.
case .stereoMic, .studio, .btHeadphonesMonoMic: return .builtInMicBtA2dp
case .wiredHeadset: return .builtInMicSpeaker
case .custom: return .builtInMicSpeaker // placeholder
}
@@ -103,7 +111,7 @@ final class IOSAudioRouter: ObservableObject {
var captureChannels: CaptureChannels {
switch self {
case .stereoMic, .studio, .btHeadphonesStereoMic: return .stereo
case .stereoMic, .studio: return .stereo
default: return .mono
}
}
@@ -118,7 +126,7 @@ final class IOSAudioRouter: ObservableObject {
/// Whether this preset explicitly selects the built-in mic port.
var usesBuiltInMic: Bool {
switch self {
case .stereoMic, .studio, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return true
case .stereoMic, .studio, .btHeadphonesMonoMic: return true
default: return false
}
}
@@ -273,7 +281,7 @@ final class IOSAudioRouter: ObservableObject {
var activePreset: AudioPreset {
// Check device-specific presets first (most specific least specific)
let order: [AudioPreset] = [
.bluetoothHeadset, .btHeadphonesMonoMic, .btHeadphonesStereoMic,
.bluetoothHeadset, .btHeadphonesMonoMic,
.wiredHeadset,
.voiceChat, .stereoMic, .studio,
]
@@ -307,96 +315,181 @@ final class IOSAudioRouter: ObservableObject {
let session = AVAudioSession.sharedInstance()
// 1. Build category options from bluetooth mode.
var options: AVAudioSession.CategoryOptions = [.defaultToSpeaker, .mixWithOthers]
// .mixWithOthers is ALWAYS set it keeps other audio (notably VoiceOver, which a
// blind user needs to operate the phone) audible while our session is active. Never
// drop it.
// .defaultToSpeaker is set ONLY for the speaker preset. It forces output to the
// built-in speaker instead of the receiver but it also actively breaks A2DP
// routing in .playAndRecord, so it must NOT be set for the A2DP or HFP presets.
// .allowAirPlay is added to the Bluetooth presets so AirPlay output also works.
var options: AVAudioSession.CategoryOptions = [.mixWithOthers]
switch bluetoothMode {
case .btHfpVoice:
options.insert(.allowBluetooth)
// Note: .allowBluetoothA2DP is NOT inserted forces HFP for the mic path.
case .builtInMicBtA2dp:
// Voice Chat: allow BOTH HFP and A2DP, let iOS pick the right profile for the
// connected device. This matches TeamTalk5's default (UtilSound.swift:228):
// [.allowBluetoothHFP, .allowAirPlay, .allowBluetoothA2DP]
// Making HFP and A2DP mutually exclusive (HFP-only here) blocks A2DP headphones
// from receiving audio the "Voice Chat kills Bluetooth output" regression.
// HFP is *preferred* (the system uses HFP when a two-way mic path is needed),
// but A2DP is still available for output-only scenarios.
options.insert(.allowBluetoothHFP)
options.insert(.allowBluetoothA2DP)
// Note: .allowBluetooth is NOT inserted no HFP, stereo A2DP output only.
options.insert(.allowAirPlay)
case .builtInMicBtA2dp:
// A2DP output only (no HFP). With HFP disabled the Bluetooth device can only be
// an OUTPUT (A2DP), so the system routes the mic to the built-in mic exactly
// what we want for "built-in mic + A2DP output", in either mono OR stereo.
// This matches TeamTalk5's A2DP mode (UtilSound.swift:232-233): remove HFP from
// the default set, leaving only A2DP.
options.insert(.allowBluetoothA2DP)
options.insert(.allowAirPlay)
case .builtInMicSpeaker:
// Neither Bluetooth option built-in mic + speaker/wired output only.
break
// Built-in mic + speaker/wired output only. Prefer speaker over the receiver.
options.insert(.defaultToSpeaker)
}
// 2. Set category + mode based on mic processing mode AND bluetooth mode.
// .voiceChat mode uses hardware AEC/AGC/HPF, but requires HFP-compatible routes.
// A2DP output is NOT HFP using .voiceChat with A2DP causes iOS to mute the output
// because it can't set up the voice processing pipeline on an A2DP route. So:
// - Standard + HFP or Speaker: .voiceChat (hardware AEC works)
// - Standard + A2DP: .default (no hardware AEC, but audio routes correctly A2DP
// headphones are in-ear/over-ear so echo from built-in mic is minimal)
// - Raw + any: .measurement (all processing off, regardless of bluetooth mode)
// 2. Set category + mode. Recipe validated against TeamTalk5 / Ferrite, which both do
// built-in stereo mic + A2DP Bluetooth output simultaneously:
// - Stereo capture: .default .voiceChat (the AEC/VPIO path) forces MONO, so stereo
// is only possible in a non-VPIO mode. .default supports multi-capsule stereo AND
// keeps the A2DP output route alive. (Earlier .videoRecording + a session-level
// channel-count request collapsed A2DP output the "stereo kills output" bug.)
// - Mono raw/studio: .measurement all system processing off.
// - Mono + A2DP output: .videoRecording keeps A2DP output without VPIO (no AEC).
// - Mono standard (HFP or speaker): .voiceChat hardware AEC/AGC/HPF.
let mode: AVAudioSession.Mode
switch (micMode, bluetoothMode) {
case (.standard, .builtInMicBtA2dp):
mode = .default // A2DP + hardware AEC = incompatible
case (.standard, _):
mode = .voiceChat // HFP or speaker: hardware AEC works
case (.raw, _):
mode = .measurement // all processing off
if captureChannels == .stereo {
mode = .default
} else if micMode == .raw {
mode = .measurement
} else if bluetoothMode == .builtInMicBtA2dp {
mode = .videoRecording
} else {
mode = .voiceChat
}
do {
try session.setCategory(.playAndRecord, mode: mode, options: options)
logger.info("setCategory ok — mode=\(self.modeLabel(mode)), bt=\(self.bluetoothMode.rawValue), options=\(self.optionsLabel(options))")
logger.info("setCategory ok — mode=\(self.modeLabel(mode)), bt=\(self.bluetoothMode.rawValue), ch=\(self.captureChannels.rawValue), options=\(self.optionsLabel(options))")
} catch {
logger.error("setCategory failed: \(error.localizedDescription)")
}
// 3. Set preferred input port (skip if "Default" empty/nil ID means use system default).
if let portId = selectedInputPortId, !portId.isEmpty,
let port = session.availableInputs?.first(where: { $0.uid == portId }) {
// 3. Input & mic-capsule configuration.
if captureChannels == .stereo {
// Stereo: enable the built-in mic's .stereo polar pattern AND anchor the input
// route explicitly via setPreferredInput + setInputDataSource. The session-level
// channel-count call (setPreferredInputNumberOfChannels(2)) is what collapses the
// A2DP output route NOT setPreferredInput (TeamTalk5 uses setPreferredInput and
// gets stereo + A2DP). With HFP disabled the system routes input to the built-in
// mic, but without the explicit preferred-input anchor the route can collapse
// during the mode switch (.voiceChat .default) and the output dies. The channel
// count is requested by miniaudio at the audio-unit level (vc_set_capture_channels).
configureStereoCapture(session: session)
} else if let portId = selectedInputPortId, !portId.isEmpty,
let port = session.availableInputs?.first(where: { $0.uid == portId }) {
// Mono with an explicit input-port selection (advanced settings).
do {
try session.setPreferredInput(port)
logger.info("setPreferredInput ok — \(port.portName)")
} catch {
logger.error("setPreferredInput failed: \(error.localizedDescription)")
}
// 4. Set preferred data source (orientation) on the selected input port.
if let dataSourceId = selectedDataSourceId, !dataSourceId.isEmpty,
let dataSource = port.dataSources?.first(where: { String(describing: $0.dataSourceID) == dataSourceId }) {
do {
try port.setPreferredDataSource(dataSource)
logger.info("setPreferredDataSource ok — \(dataSource.dataSourceName)")
} catch {
logger.error("setPreferredDataSource failed: \(error.localizedDescription)")
}
// 5. Set preferred polar pattern on the data source.
if let polarPattern = selectedPolarPattern, !polarPattern.isEmpty {
let pattern = AVAudioSession.PolarPattern(rawValue: polarPattern)
do {
try dataSource.setPreferredPolarPattern(pattern)
logger.info("setPreferredPolarPattern ok — \(polarPattern)")
} catch {
logger.error("setPreferredPolarPattern failed: \(error.localizedDescription)")
}
}
}
}
// 6. Set preferred input number of channels ONLY for stereo (non-default).
// Calling setPreferredInputNumberOfChannels(1) for mono is unnecessary (1 is the
// default) and may put the session in a bad state on some devices.
if captureChannels == .stereo {
do {
try session.setPreferredInputNumberOfChannels(2)
logger.info("setPreferredInputNumberOfChannels ok — 2 (stereo)")
} catch {
logger.error("setPreferredInputNumberOfChannels failed: \(error.localizedDescription)")
}
configureMonoCapture(session: session, port: port)
} else {
// Mono, system-default input. Still clear any leftover .stereo capsule from a
// prior stereo session so we actually return to mono.
clearStereoPolarPattern(session: session)
}
updateWarnings()
}
/// Enable 2-channel capture on the built-in mic. Mirrors TeamTalk5's recipe
/// (`SoundDevicesModel.selectDataSource` + `UtilSound.setupSoundDevices`), which
/// achieves stereo mic + A2DP Bluetooth output simultaneously:
/// 1. `setPreferredDataSource(stereoSource)` on the built-in mic port
/// 2. `setPreferredPolarPattern(.stereo)` on that data source
/// 3. `setPreferredInput(builtIn)` anchor the input route explicitly (this is NOT
/// what collapses A2DP the session-level `setPreferredInputNumberOfChannels(2)`
/// is. Without this anchor the route can collapse during the mode switch.)
/// 4. `setInputDataSource(stereoSource)` commit the data source at the session level
/// The channel count itself is requested by miniaudio at the audio-unit level via
/// `vc_set_capture_channels(2)`. We do NOT call `setPreferredInputNumberOfChannels(2)`
/// that session-level call is the one that collapses the A2DP output route.
private func configureStereoCapture(session: AVAudioSession) {
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
else {
logger.warning("stereo requested but no built-in mic available — staying mono")
return
}
guard let stereoSource = builtIn.dataSources?.first(where: {
$0.supportedPolarPatterns?.contains(.stereo) == true
}) else {
logger.warning("stereo requested but built-in mic has no .stereo data source — staying mono")
return
}
do {
try builtIn.setPreferredDataSource(stereoSource)
try stereoSource.setPreferredPolarPattern(.stereo)
// Anchor the input route explicitly. TeamTalk5 does this (SoundDevicesModel
// .selectDataSource:147); without it the route can collapse during the mode
// switch (.voiceChat .default) and the A2DP output dies.
try session.setPreferredInput(builtIn)
// Commit the data source at the session level (TeamTalk does this at
// SoundDevicesModel.selectDataSource:148). setPreferredDataSource alone only
// sets the port-level preference; setInputDataSource makes it the active source.
try session.setInputDataSource(stereoSource)
logger.info("stereo capsule enabled — source=\(stereoSource.dataSourceName), pattern=.stereo, input anchored")
} catch {
logger.error("stereo capsule setup failed: \(error.localizedDescription)")
}
}
/// Configure mono capture on an explicitly selected port: apply the user's chosen data source
/// (orientation) and polar pattern, resetting any prior `.stereo` pattern back to default.
private func configureMonoCapture(session: AVAudioSession, port: AVAudioSessionPortDescription) {
guard let dataSourceId = selectedDataSourceId, !dataSourceId.isEmpty,
let dataSource = port.dataSources?.first(where: {
String(describing: $0.dataSourceID) == dataSourceId
}) else {
// No explicit capsule choice make sure we're not stuck on a prior .stereo pattern.
clearStereoPolarPattern(session: session)
return
}
do {
try port.setPreferredDataSource(dataSource)
logger.info("setPreferredDataSource ok — \(dataSource.dataSourceName)")
} catch {
logger.error("setPreferredDataSource failed: \(error.localizedDescription)")
}
if let polarPattern = selectedPolarPattern, !polarPattern.isEmpty {
let pattern = AVAudioSession.PolarPattern(rawValue: polarPattern)
try? dataSource.setPreferredPolarPattern(pattern)
logger.info("setPreferredPolarPattern ok — \(polarPattern)")
} else {
// Clear any prior .stereo selection so mono capture returns to a mono capsule.
try? dataSource.setPreferredPolarPattern(nil)
}
}
/// Reset any built-in-mic data source that's currently on the `.stereo` polar pattern back to
/// the default (mono) pattern. Used when switching from a stereo session back to mono with no
/// explicit capsule selection, so the prior stereo capsule doesn't linger.
private func clearStereoPolarPattern(session: AVAudioSession) {
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
else { return }
for ds in builtIn.dataSources ?? [] where ds.selectedPolarPattern == .stereo {
try? ds.setPreferredPolarPattern(nil)
}
}
private func modeLabel(_ mode: AVAudioSession.Mode) -> String {
switch mode {
case .voiceChat: return "voiceChat"
case .measurement: return "measurement"
case .videoRecording: return "videoRecording"
case .default: return "default"
default: return "other"
}
@@ -406,7 +499,7 @@ final class IOSAudioRouter: ObservableObject {
var parts: [String] = []
if opts.contains(.defaultToSpeaker) { parts.append("defaultToSpeaker") }
if opts.contains(.mixWithOthers) { parts.append("mixWithOthers") }
if opts.contains(.allowBluetooth) { parts.append("allowBluetooth") }
if opts.contains(.allowBluetoothHFP) { parts.append("allowBluetoothHFP") }
if opts.contains(.allowBluetoothA2DP) { parts.append("allowBluetoothA2DP") }
return parts.joined(separator: ",")
}
@@ -484,6 +577,13 @@ final class IOSAudioRouter: ObservableObject {
captureChannels = channels
savePreferences()
applyConfiguration()
// Sync the core's capture channel count. The core's set_capture_channels handles
// the engine restart internally (stop + ensure_audio_running) no need for the
// Swift layer to suspend/restart separately.
if let streamId = AudioSessionManager.shared.activeMicStreamId {
_ = AudioSessionManager.shared.client?.setCaptureChannels(
streamId: streamId, channels: channels.channelCount)
}
}
// MARK: - Presets
@@ -520,6 +620,12 @@ final class IOSAudioRouter: ObservableObject {
UserDefaults.standard.set(preset.rawValue, forKey: kPreset)
savePreferences()
applyConfiguration()
// Sync the core's capture channel count. The core's set_capture_channels handles
// the engine restart internally.
if let streamId = AudioSessionManager.shared.activeMicStreamId {
_ = AudioSessionManager.shared.client?.setCaptureChannels(
streamId: streamId, channels: preset.captureChannels.channelCount)
}
refreshRoutes()
logger.info("applyPreset — \(preset.rawValue)")
}
@@ -532,10 +638,17 @@ final class IOSAudioRouter: ObservableObject {
let outputIsSpeaker = session.currentRoute.outputs.contains { $0.portType == .builtInSpeaker }
// Raw/Studio mode + speaker = echo risk (no AEC in .measurement mode)
showsRawModeSpeakerWarning = (micMode == .raw && outputIsSpeaker)
// Standard mode + A2DP = no hardware AEC (A2DP incompatible with .voiceChat mode)
showsA2dpNoAecWarning = (micMode == .standard && bluetoothMode == .builtInMicBtA2dp)
// A2DP output runs without hardware AEC (the .voiceChat AEC path isn't available on an
// A2DP route). Applies to both mono and stereo A2DP. Stereo also has no AEC (it can't
// use .voiceChat at all), but the message is the same and the warning already shows when
// the bluetooth mode is A2DP.
showsA2dpNoAecWarning = (bluetoothMode == .builtInMicBtA2dp)
}
/// Whether the current configuration wants Bluetooth A2DP output. Used after session
/// activation to clear any lingering speaker override that would pin output to the speaker.
var wantsA2dpOutput: Bool { bluetoothMode == .builtInMicBtA2dp }
/// The selected input port object, if any.
var selectedPort: IOSAudioInputPort? {
inputPorts.first(where: { $0.id == selectedInputPortId })