fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death)

The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.

Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.

Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.

Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-20 02:34:07 +02:00
parent dcb7e6eeca
commit f1e1ef59ed
6 changed files with 142 additions and 9 deletions

View File

@@ -10,6 +10,38 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action
- **Awaiting on-device verification:** **iOS stereo mic kills headphone/A2DP output — REAL
root cause found & fixed** (2026-06-20, on Windows; verify on Mac). All prior "fixes" (the
2026-06-19 entries below) targeted the Swift `IOSAudioRouter` on the false premise that
"miniaudio does NOT touch AVAudioSession on iOS." **It does.** The core opened its miniaudio
devices with `ma_device_init(nullptr, ...)`; with a NULL context, miniaudio (0.11.25) runs an
iOS "hack" (`miniaudio.h` ~44057) that picks a session category by device type, then
`ma_context_init__coreaudio` (~36552) calls `setCategory()` + `setActive()` on **every device
open** — capture → `AVAudioSessionCategoryRecord` with **zero options**. That wiped the
`.playAndRecord` category, the mode, and `.allowBluetoothA2DP`/`.mixWithOthers`/`.allowAirPlay`
that `IOSAudioRouter` had just configured → headphone/A2DP (and even wired) output died. The
stereo presets broke worst because they depend on the A2DP output route the wipe removed.
TeamTalk never hits this: its SDK opens RemoteIO/VPIO AudioUnits directly and leaves the
session entirely to the app (`UtilSound.swift`); miniaudio insists on managing it.
- **Fix (core, cross-platform safe):** `AudioEngine` now owns a `ma_context` built by
`make_context_config()` with `coreaudio.sessionCategory = ma_ios_session_category_none` +
`noAudioSessionActivate`/`noAudioSessionDeactivate = MA_TRUE`, and passes it to **all**
`ma_device_init` calls (playback, capture, loopback) and to `enumerate_devices`'s context.
miniaudio now never touches AVAudioSession; the Swift `IOSAudioRouter` is the sole owner
(session is already activated on connect in `AppState.swift:authResult`, before any device
opens, so removing miniaudio's self-activation is safe). Context is lazily inited in
`start()`, reused across restarts, uninited in `~AudioEngine`.
Files: `core/src/audio/audio_engine.{h,cpp}`.
- **TEMP diagnostics (remove after verification):** `AudioSessionManager.logSessionState(_:)`
logs category/mode/options/route; called after `ensureSessionActive`, on every route change,
and on `.streamStarted` (right after the core opens its devices). On Mac, watch the log when
joining voice with the Stereo Mic preset: category must stay `…PlayAndRecord` with
`allowBluetoothA2DP` and the output route must remain the headphones/A2DP device — NOT flip
to `…Record`. If confirmed, delete the `logSessionState` calls + method and the prior
band-aid comments in `IOSAudioRouter`/`audio_engine.cpp` can be trimmed.
- **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 21/21.
iOS build & on-device run still to be done by the user on the Mac.
- **Planned (not started):** **External PCM feed/tap API (`vc_stream_feed_pcm` +
`vc_set_pcm_sink`)** (2026-06-19, plan written on Windows; implement on Mac). A public,
documented API for driving audio streams with externally-provided PCM instead of (or in

View File

@@ -62,6 +62,7 @@ final class AudioSessionManager {
let outputNames = route.outputs.map { $0.portName }.joined(separator: ", ")
let inputNames = route.inputs.map { $0.portName }.joined(separator: ", ")
logger.info("session activated — outputs: [\(outputNames)], inputs: [\(inputNames)]")
logSessionState("after activate")
}
/// Deactivate the AVAudioSession. Call ONLY when disconnecting from the server not
@@ -77,6 +78,34 @@ final class AudioSessionManager {
logger.info("session deactivated")
}
/// TEMP DIAGNOSTIC (stereo-A2DP fix verification): log the full AVAudioSession state
/// category, mode, options, and active route. The bug was miniaudio resetting the category
/// to `Record` (no output) on device open; with the core's ma_context now configured to
/// leave the session alone, this should report `AVAudioSessionCategoryPlayAndRecord` with
/// `allowBluetoothA2DP` set, and the output route should be the headphones/A2DP device
/// even after the mic engine starts. Remove once verified on-device.
func logSessionState(_ when: String) {
let s = AVAudioSession.sharedInstance()
var opts: [String] = []
let o = s.categoryOptions
if o.contains(.mixWithOthers) { opts.append("mixWithOthers") }
if o.contains(.duckOthers) { opts.append("duckOthers") }
if o.contains(.allowBluetoothHFP) { opts.append("allowBluetoothHFP") }
if o.contains(.allowBluetoothA2DP) { opts.append("allowBluetoothA2DP") }
if o.contains(.allowAirPlay) { opts.append("allowAirPlay") }
if o.contains(.defaultToSpeaker) { opts.append("defaultToSpeaker") }
let outs = s.currentRoute.outputs.map { "\($0.portName)[\($0.portType.rawValue)]" }
.joined(separator: ", ")
let ins = s.currentRoute.inputs.map { "\($0.portName)[\($0.portType.rawValue)]" }
.joined(separator: ", ")
logger.info("""
[SESSION @ \(when, privacy: .public)] category=\(s.category.rawValue, privacy: .public) \
mode=\(s.mode.rawValue, privacy: .public) options=[\(opts.joined(separator: ","), privacy: .public)] \
inputs=[\(ins, privacy: .public)] outputs=[\(outs, privacy: .public)] \
inputCh=\(s.inputNumberOfChannels) outputCh=\(s.outputNumberOfChannels)
""")
}
@objc private func handleInterruption(_ notification: Notification) {
guard let info = notification.userInfo,
let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
@@ -133,6 +162,7 @@ final class AudioSessionManager {
IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
logSessionState("route change (\(reasonLabel(reason)))")
}
private func reasonLabel(_ reason: AVAudioSession.RouteChangeReason) -> String {

View File

@@ -6,8 +6,14 @@ private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAu
/// iOS audio routing layer drives all iOS audio route selection via `AVAudioSession`
/// *before* the core (miniaudio) opens its device. miniaudio does NOT touch
/// `AVAudioSession` on iOS; it opens the current default route via CoreAudio and that's
/// it. All iOS audio routing (input port selection, mic orientation/polar patterns,
/// `AVAudioSession` on iOS but ONLY because the core deliberately opens its devices
/// through a `ma_context` configured with `sessionCategory = none` +
/// `noAudioSessionActivate/Deactivate` (see `AudioEngine::make_context_config` in
/// `core/src/audio/audio_engine.cpp`). With miniaudio's default NULL-context path it WOULD
/// reset the category to `Record`/`Playback` with no options on every device open, wiping
/// `.allowBluetoothA2DP`/`.playAndRecord` and killing headphone/A2DP output (the long-standing
/// "stereo mic kills output" bug). With that disabled, this class is the sole owner of the
/// session. All iOS audio routing (input port selection, mic orientation/polar patterns,
/// HFP vs A2DP, measurement/raw mode, stereo capture) must be driven from here.
///
/// The three user-facing choices:

View File

@@ -110,6 +110,9 @@ final class SessionState {
addActivity("Audio session activate failed: \(error)")
}
}
// TEMP DIAGNOSTIC: the core opens its miniaudio devices around now log the
// session state to confirm miniaudio is no longer resetting the category to Record.
AudioSessionManager.shared.logSessionState("stream started (user \(ev.userId))")
addActivity("Stream started (user \(ev.userId))")
case .streamStopped:
addActivity("Stream stopped (user \(ev.userId))")

View File

@@ -138,7 +138,28 @@ void JitterBuffer::reset() {
AudioEngine::AudioEngine() = default;
AudioEngine::~AudioEngine() { stop(); }
AudioEngine::~AudioEngine() {
stop();
#ifdef VOICECAT_HAS_AUDIO
stop_loopback_capture(); // loopback has an independent lifecycle — close it before the context
if (context_inited_) {
ma_context_uninit(&context_);
context_inited_ = false;
}
#endif
}
#ifdef VOICECAT_HAS_AUDIO
ma_context_config AudioEngine::make_context_config() {
ma_context_config cfg = ma_context_config_init();
// iOS: leave AVAudioSession entirely to the Swift layer (IOSAudioRouter). See the
// make_context_config() declaration in audio_engine.h for the full rationale.
cfg.coreaudio.sessionCategory = ma_ios_session_category_none; // don't call setCategory
cfg.coreaudio.noAudioSessionActivate = MA_TRUE; // don't setActive(true) on device init
cfg.coreaudio.noAudioSessionDeactivate = MA_TRUE; // don't setActive(false) on device uninit
return cfg;
}
#endif
bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
if (running_.load()) return false;
@@ -163,6 +184,18 @@ bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
#endif
#ifdef VOICECAT_HAS_AUDIO
// Own the ma_context (lazily, reused across restarts) so miniaudio does not touch
// AVAudioSession on iOS — the Swift IOSAudioRouter is the sole session owner. Without
// this, ma_device_init(nullptr, ...) below would reset the session category to Record/
// Playback with no options on every open, killing headphone/A2DP output. See
// make_context_config() in audio_engine.h. If context init fails we fall back to a NULL
// context (degraded: miniaudio manages the session) rather than leaving audio dead.
if (!context_inited_) {
ma_context_config ctx_cfg = make_context_config();
context_inited_ = (ma_context_init(nullptr, 0, &ctx_cfg, &context_) == MA_SUCCESS);
}
ma_context* ctx = context_inited_ ? &context_ : nullptr;
// ── Playback device (opened first on iOS: commits the output route — e.g. A2DP —
// before the capture device starts. Starting stereo capture can trigger an iOS audio
// route reconfiguration; opening playback first ensures A2DP is already committed
@@ -179,7 +212,7 @@ bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
pb_cfg.pUserData = this;
pb_cfg.playback.pDeviceID = have_pb_id ? &pb_id : nullptr;
if (ma_device_init(nullptr, &pb_cfg, &playback_device_) == MA_SUCCESS) {
if (ma_device_init(ctx, &pb_cfg, &playback_device_) == MA_SUCCESS) {
if (ma_device_start(&playback_device_) == MA_SUCCESS) {
playback_started_ = true;
} else {
@@ -190,7 +223,7 @@ bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
// though WASAPI shared mode normally remixes transparently. Retry once at mono rather
// than leaving playback dead.
pb_cfg.playback.channels = 1;
if (ma_device_init(nullptr, &pb_cfg, &playback_device_) == MA_SUCCESS) {
if (ma_device_init(ctx, &pb_cfg, &playback_device_) == MA_SUCCESS) {
if (ma_device_start(&playback_device_) == MA_SUCCESS) {
playback_started_ = true;
params_.playback_channels = 1;
@@ -217,7 +250,7 @@ bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
// below becomes a zero-copy passthrough rather than a copy every two callbacks.
cap_cfg.periodSizeInFrames = static_cast<ma_uint32>(frame_samples_);
if (ma_device_init(nullptr, &cap_cfg, &capture_device_) == MA_SUCCESS) {
if (ma_device_init(ctx, &cap_cfg, &capture_device_) == MA_SUCCESS) {
if (ma_device_start(&capture_device_) == MA_SUCCESS) {
capture_started_ = true;
} else {
@@ -232,8 +265,12 @@ bool AudioEngine::start(const AudioParams& p, CaptureCallback capture_cb) {
std::vector<DeviceInfo> AudioEngine::enumerate_devices(bool capture) {
std::vector<DeviceInfo> result;
#ifdef VOICECAT_HAS_AUDIO
// Use the no-AVAudioSession-management config here too: device enumeration runs at
// SessionState init (refreshDevices) and on settings views, and a default-config context
// would call setCategory()/setActive() on iOS, disrupting the session the Swift layer owns.
ma_context ctx;
if (ma_context_init(nullptr, 0, nullptr, &ctx) != MA_SUCCESS) return result;
ma_context_config ctx_cfg = make_context_config();
if (ma_context_init(nullptr, 0, &ctx_cfg, &ctx) != MA_SUCCESS) return result;
ma_device_info* playback_infos = nullptr;
ma_uint32 playback_count = 0;
@@ -619,14 +656,19 @@ bool AudioEngine::start_loopback_capture(int kind, int channels) {
// along with everything else playing — an accepted self-echo-loop characteristic of
// desktop-audio capture, not a bug.
if (ma_device_init(nullptr, &cfg, &loopback_device_) != MA_SUCCESS) {
// Use the engine's owned context (shared with playback/capture) so loopback honors the
// same no-AVAudioSession-management config. Loopback is Windows/WASAPI-only today, where
// the coreaudio fields are irrelevant, but keep it consistent. start() inits the context
// before any loopback can be requested; fall back to NULL if it somehow isn't.
ma_context* ctx = context_inited_ ? &context_ : nullptr;
if (ma_device_init(ctx, &cfg, &loopback_device_) != MA_SUCCESS) {
// Fallback: some unusual render endpoints may reject channels=2 even though WASAPI
// shared mode normally remixes transparently. Retry once at mono (mirror the playback
// device's fallback in start()) rather than leaving screen-audio capture dead.
if (ch != 1) {
ch = 1;
cfg.capture.channels = 1;
if (ma_device_init(nullptr, &cfg, &loopback_device_) != MA_SUCCESS) return false;
if (ma_device_init(ctx, &cfg, &loopback_device_) != MA_SUCCESS) return false;
} else {
return false;
}

View File

@@ -292,6 +292,26 @@ class AudioEngine {
void on_capture(const int16_t* pcm, ma_uint32 frames);
void on_playback(int16_t* out, ma_uint32 frames);
// Build the ma_context config that keeps miniaudio from managing AVAudioSession on iOS.
// With a NULL context, ma_device_init runs miniaudio's iOS "hack" (miniaudio.h ~44057)
// that calls setCategory()/setActive() on EVERY device open — capture →
// AVAudioSessionCategoryRecord with zero options. That obliterates the category/mode/
// options the Swift IOSAudioRouter configured (notably .playAndRecord and
// .allowBluetoothA2DP), which is what killed headphone/A2DP output when stereo was
// selected. The iOS Swift layer is the SOLE owner of the audio session (activated in
// AppState on connect, configured by IOSAudioRouter); miniaudio must only open the
// AudioUnit against the already-configured, already-active route. These coreaudio fields
// are no-ops on non-Apple backends. See PROGRESS.md / the "stereo mic kills output"
// investigation.
static ma_context_config make_context_config();
// Owned context, shared by the playback, capture, and loopback devices so they all honor
// the no-session-management config above. Lives for the engine's lifetime (init lazily in
// start(), reused across stop()/start() restarts, uninit in the destructor) — loopback has
// an independent start/stop lifecycle, so the context must outlive a single stop().
ma_context context_{};
bool context_inited_ = false;
ma_device capture_device_{};
ma_device playback_device_{};
bool capture_started_ = false;