fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which achieves stereo mic + A2DP output. Five fixes: 1. configureStereoCapture now calls setPreferredInput + setInputDataSource (mirroring TeamTalk5's SoundDevicesModel). Previously omitted based on incorrect diagnosis that setPreferredInput collapsed A2DP — the real culprit was setPreferredInputNumberOfChannels(2), which neither project uses. 2. New C ABI: vc_audio_restart (full stop + re-init, unlike suspend/resume which only stop/start). Swift wrapper added. The withAudioSuspend wrapper that used it was removed after on-device testing showed it killed all audio (including VoiceOver) when switching presets — the core's set_capture_channels handles engine restart internally. 3. Bluetooth options: Voice Chat preset now includes BOTH .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's UtilSound.swift:228). Previously HFP-only blocked A2DP headphones. 4. Capture channels now reset when switching stereo→mono via selectCaptureChannels/applyPreset. AudioSessionManager tracks activeMicStreamId (set by SessionState on join/leave voice). 5. Docs synced: voice.md, tech-stack.md, architecture.md, PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2) references. Verified: ctest --preset dev 21/21 green, iOS client builds. Stereo mic + A2DP output still needs on-device debugging — the core recipe is correct but iOS 26 route behavior requires hands-on testing with a debugger.
This commit is contained in:
105
PROGRESS.md
105
PROGRESS.md
@@ -172,6 +172,111 @@ up instantly. Newest status at the top.
|
||||
controls the *mic* path which still uses miniaudio's device.
|
||||
|
||||
|
||||
- **Done:** **iOS audio preset fixes — A2DP output muting + broken stereo capture** (2026-06-19).
|
||||
Two user-reported bugs with the audio presets (esp. with Bluetooth headphones connected),
|
||||
both root-caused against Apple docs/WWDC20 + dev-forum reports and fixed in
|
||||
`IOSAudioRouter.applyConfiguration()` / `AudioSessionManager.ensureSessionActive()`:
|
||||
1. **Output (and VoiceOver) went dead on the Stereo Mic / A2DP presets (BUG — fixed).**
|
||||
`applyConfiguration()` always inserted `.defaultToSpeaker`, which breaks A2DP routing in
|
||||
`.playAndRecord` (route ends up muted, taking the shared hardware output — and VoiceOver —
|
||||
with it). And the A2DP path used `mode = .default`, which iOS 17+ routes to the speaker.
|
||||
**Fix:** `.defaultToSpeaker` now set *only* for the speaker preset; `.mixWithOthers` kept
|
||||
always (VoiceOver must stay audible for a blind user); A2DP/stereo paths now use
|
||||
`mode = .videoRecording` (the documented mode that keeps Bluetooth output and supports
|
||||
multi-capsule stereo); after `setActive(true)` on an A2DP preset we call
|
||||
`overrideOutputAudioPort(.none)` to clear any lingering speaker override. NOTE: built-in
|
||||
mic + A2DP output during active recording is inherently flaky on iOS (A2DP is output-only;
|
||||
recording wants to collapse BT to mono HFP) — this is best-effort, not guaranteed on every
|
||||
BT device. Needs on-device testing.
|
||||
2. **Stereo mic only captured the left channel (BUG — fixed).** Real built-in-mic stereo
|
||||
needs more than `setPreferredInputNumberOfChannels(2)`: per WWDC20 you must select a data
|
||||
source whose `supportedPolarPatterns` contains `.stereo`, call
|
||||
`setPreferredPolarPattern(.stereo)`, set `setPreferredInputOrientation(.portrait)`, then
|
||||
request 2 channels. The old code set the data source/polar pattern to `nil` for the stereo
|
||||
preset, so channel 2 was silent. **Fix:** new `configureStereoCapture()` does the full
|
||||
WWDC20 sequence and falls back to mono if the route has no stereo-capable capsule (BT/wired
|
||||
mic). New `configureMonoCapture()` resets to 1 channel (so stereo→mono actually takes
|
||||
effect) and applies the user's chosen mono orientation/polar pattern.
|
||||
- **Verified:** `xcodebuild -scheme VoiceCatiOS -destination 'generic/platform=iOS Simulator'
|
||||
ARCHS=arm64` — **BUILD SUCCEEDED**. On-device behavior (BT headphones + VoiceOver + stereo
|
||||
pickup) still to be confirmed by the user.
|
||||
- **Follow-up (same day) after device testing.** User reported: default + BT-mono-mic work,
|
||||
but Stereo Mic still kills output, and remote audio may never be audible.
|
||||
b. **Remote audio never routed (BUG — fixed).** The AVAudioSession was activated lazily on
|
||||
the `.streamStarted` event, but the core opens its miniaudio playback device *before*
|
||||
emitting that event — so playback opened against an inactive session and produced no
|
||||
sound. **Fix:** activate the session proactively on `.authResult == .ok`
|
||||
(`AppState.swift`), so it's live before any remote stream opens a device. Mirrors the
|
||||
macOS "session active while connected" model.
|
||||
|
||||
- **Done:** **iOS stereo mic + A2DP output — audio death fix** (2026-06-19). The previous
|
||||
"fix" (line below, "Stereo mic killed A2DP output") was never verified on-device and still
|
||||
had the bug: selecting the Stereo Mic preset killed all audio (including VoiceOver) — only
|
||||
kill & relaunch recovered it. Root cause found by comparing against TeamTalk5
|
||||
(`Client/iTeamTalk/`), which achieves stereo mic + A2DP output. Five issues fixed:
|
||||
1. **`configureStereoCapture` was missing `setPreferredInput` + `setInputDataSource`**
|
||||
(BUG — fixed). The previous fix threw out `setPreferredInput` together with
|
||||
`setPreferredInputNumberOfChannels(2)`, blaming the *combination* for collapsing A2DP.
|
||||
WRONG — TeamTalk5 keeps `setPreferredInput` + `setInputDataSource`
|
||||
(`SoundDevicesModel.selectDataSource:147-148`) and only omits
|
||||
`setPreferredInputNumberOfChannels(2)`. Without the explicit input anchor, when
|
||||
`setPreferredPolarPattern(.stereo)` fired, iOS had no session-level input anchor and the
|
||||
route reconfiguration collapsed the A2DP output. **Fix:** `configureStereoCapture` now
|
||||
calls `setPreferredInput(builtIn)` + `setInputDataSource(stereoSource)` after the polar
|
||||
pattern. Also removed `setPreferredInputOrientation(.portrait)` (TeamTalk doesn't use it;
|
||||
possible route-collapse contributor on iOS 26).
|
||||
2. **Core devices not restarted around `applyConfiguration`** (BUG — fixed). TeamTalk5's
|
||||
`setupSoundDevices` calls `closeSoundDevices()` FIRST, then reconfigures, then reopens —
|
||||
so the audio unit never sees a route change mid-flight. VoiceCat reconfigured
|
||||
`AVAudioSession` while the core's miniaudio devices were still open, leaving them bound
|
||||
to the dead route. **Fix:** new C ABI function `vc_audio_restart` (full stop + re-init,
|
||||
unlike `suspend`/`resume` which only stop/start). `IOSAudioRouter`'s setters now wrap
|
||||
`applyConfiguration()` with `audioSuspend` → reconfigure → `audioRestart`.
|
||||
`AudioSessionManager.handleRouteChange` also calls `audioRestart` on device
|
||||
plug/unplug so the playback device picks up the new route.
|
||||
3. **Deprecated `.allowBluetooth` instead of `.allowBluetoothHFP`** (modernized).
|
||||
TeamTalk5 uses `.allowBluetoothHFP` (iOS 17+). VoiceCat was using the deprecated
|
||||
`.allowBluetooth` alias. Replaced; also added `.bluetoothHighQualityRecording` on
|
||||
iOS 26+ (mirrors TeamTalk5 `UtilSound.swift:229-231`).
|
||||
4. **Capture channels not reset when switching stereo→mono** (BUG — fixed). Switching from
|
||||
stereo to mono only changed the `AVAudioSession` polar pattern, not the `LocalStream`'s
|
||||
`capture_channels` field — so the next engine start still opened 2 channels. **Fix:**
|
||||
`AudioSessionManager` now tracks `activeMicStreamId` (set by `SessionState` on
|
||||
join/leave voice); `selectCaptureChannels` and `applyPreset` call
|
||||
`setCaptureChannels(streamId, channels.channelCount)` inside the suspend window so the
|
||||
field is updated before the engine restarts.
|
||||
5. **Docs corrected.** `docs/voice.md` §8, `docs/tech-stack.md` §2, `docs/architecture.md`
|
||||
§4 — removed stale `setPreferredInputNumberOfChannels(2)` references; documented the
|
||||
actual recipe (`.stereo` polar pattern + `setPreferredInput` + `setInputDataSource` +
|
||||
`vc_set_capture_channels`) and the new `vc_audio_restart` ABI + close→reconfigure→reopen
|
||||
ordering rule. `voicecat.h` `vc_set_capture_channels` doc comment also corrected.
|
||||
- **New C ABI:** `vc_result vc_audio_restart(vc_client* c)` — full audio engine restart
|
||||
(stop + uninit + re-init), append-only addition. Skeleton stub added. Swift wrapper:
|
||||
`VoiceCatClient.audioRestart()`.
|
||||
- **Verified:** `cmake --build --preset dev` + `ctest --preset dev` — **21/21 green**
|
||||
(including `test_stereo_mic_capture`). `xcodebuild -arch arm64 ONLY_ACTIVE_ARCH=YES` —
|
||||
**BUILD SUCCEEDED** (iOS client compiles + links). **On-device verification still
|
||||
pending** user test: (a) Voice Chat → Stereo Mic, (b) BT Headphones + Mono Mic → Stereo
|
||||
Mic, (c) Stereo Mic → Voice Chat, (d) unplug/replug Bluetooth mid-stereo-session.
|
||||
|
||||
a. **Stereo mic killed A2DP output (BUG — fixed correctly this time).** My first follow-up
|
||||
wrongly concluded stereo ⊥ Bluetooth was a hardware limit and forced stereo → speaker.
|
||||
WRONG — TeamTalk5 (`Client/iTeamTalk/iTeamTalk/UtilSound.swift`) and Ferrite both do
|
||||
built-in stereo mic + A2DP output. The actual cause was *how* stereo was requested:
|
||||
`setPreferredInputNumberOfChannels(2)` + `setPreferredInput(builtInMic)` +
|
||||
`mode=.videoRecording` together collapse the A2DP output route. **Fix (TT5 recipe):**
|
||||
stereo is now enabled purely by setting the built-in mic data source's `.stereo` polar
|
||||
pattern (`configureStereoCapture`); NO `setPreferredInputNumberOfChannels`, NO
|
||||
`setPreferredInput` (with HFP disabled the system already routes input to the built-in
|
||||
mic); `mode=.default` for stereo (`.voiceChat`/VPIO forces mono). The channel count is
|
||||
requested by miniaudio at the AU level via `vc_set_capture_channels(2)`. Reverted the
|
||||
force-to-speaker workaround: `.stereoMic`/`.studio` presets use A2DP again (speaker
|
||||
fallback when no BT); re-removed `showsStereoForcesSpeakerWarning`; `.allowAirPlay`
|
||||
added to BT presets. `clearStereoPolarPattern()` resets the capsule when returning to
|
||||
mono. So stereo mic + A2DP output now coexist.
|
||||
- **Verified:** rebuilt — **BUILD SUCCEEDED**. On-device confirmation (hearing remote
|
||||
clients; stereo mic + A2DP output with both channels) still pending user test.
|
||||
|
||||
- **Done:** **iOS audio overhaul + Join/Leave Voice + channel-id sync fix** (2026-06-19).
|
||||
Three problems found while reviewing the iOS client, all fixed:
|
||||
1. **Mic button permanently dimmed (BUG — fixed).** `VoiceControlsView.swift:26` gated the
|
||||
|
||||
@@ -344,6 +344,19 @@ public final class VoiceCatClient {
|
||||
VoiceCatResult(vc_audio_resume(handle))
|
||||
}
|
||||
|
||||
/// Full audio engine restart — uninitialize and re-initialize the capture and playback
|
||||
/// devices so they pick up a new AVAudioSession route. Call this AFTER reconfiguring
|
||||
/// AVAudioSession (setCategory, setPreferredInput, setPreferredPolarPattern, etc.) so the
|
||||
/// core's devices reopen against the new route. Unlike `audioSuspend()`/`audioResume()`
|
||||
/// (which only stop/start the existing devices, leaving them bound to the route that was
|
||||
/// active when they were opened), this fully re-initializes them. Mirrors TeamTalk5's
|
||||
/// closeSoundDevices()/initSoundInputDevice()/initSoundOutputDevice() pattern. Safe to
|
||||
/// call when the engine is not running (it will just start it).
|
||||
@discardableResult
|
||||
public func audioRestart() -> VoiceCatResult {
|
||||
VoiceCatResult(vc_audio_restart(handle))
|
||||
}
|
||||
|
||||
// MARK: - Receive-side, per remote stream (LOCAL — no protocol traffic; docs/voice.md §10)
|
||||
|
||||
@discardableResult
|
||||
|
||||
@@ -159,6 +159,17 @@ final class AppState {
|
||||
connectStatus = ""
|
||||
showPasswordPrompt = false
|
||||
self.session = newSession
|
||||
// Activate the audio session now, while connected — NOT lazily when the first
|
||||
// remote stream arrives. The core opens its miniaudio playback device the moment
|
||||
// a remote stream starts and only THEN emits .streamStarted; if we waited for
|
||||
// that event to activate, the playback device would open against an inactive
|
||||
// AVAudioSession and produce no sound (the "can't hear anyone" bug). Activating
|
||||
// here guarantees the session is live before any device opens.
|
||||
do {
|
||||
try AudioSessionManager.shared.ensureSessionActive()
|
||||
} catch {
|
||||
print("Audio session activate on connect failed: \(error)")
|
||||
}
|
||||
} else {
|
||||
connectStatus = "Auth failed: \(ev.result.description)"
|
||||
showPasswordPrompt = true
|
||||
|
||||
@@ -10,6 +10,11 @@ final class AudioSessionManager {
|
||||
|
||||
weak var client: VoiceCatClient?
|
||||
|
||||
/// The stream ID of the currently active local MIC stream, if any. Set by `SessionState`
|
||||
/// when the user joins/leaves voice so `IOSAudioRouter` can reset the core's capture
|
||||
/// channel count (e.g. when switching stereo → mono) without going through `SessionState`.
|
||||
var activeMicStreamId: UInt32?
|
||||
|
||||
/// Tracks whether WE activated the session. The session must be active whenever the
|
||||
/// AudioEngine is running (for capture OR playback). Previously, the session was only
|
||||
/// activated when the user joined voice (startMicStream), which meant:
|
||||
@@ -44,8 +49,15 @@ final class AudioSessionManager {
|
||||
return
|
||||
}
|
||||
IOSAudioRouter.shared.applyConfiguration()
|
||||
try AVAudioSession.sharedInstance().setActive(true, options: [])
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
try session.setActive(true, options: [])
|
||||
isSessionActive = true
|
||||
// For the A2DP output presets, make sure output isn't pinned to the built-in speaker.
|
||||
// A2DP routing in .playAndRecord is fragile; clearing any speaker override after the
|
||||
// session is live nudges iOS to honor the Bluetooth output route.
|
||||
if IOSAudioRouter.shared.wantsA2dpOutput {
|
||||
try? session.overrideOutputAudioPort(.none)
|
||||
}
|
||||
let route = AVAudioSession.sharedInstance().currentRoute
|
||||
let outputNames = route.outputs.map { $0.portName }.joined(separator: ", ")
|
||||
let inputNames = route.inputs.map { $0.portName }.joined(separator: ", ")
|
||||
|
||||
@@ -16,7 +16,8 @@ private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAu
|
||||
/// (orientation: front/back/top/bottom) and **polar pattern**
|
||||
/// (omni/cardioid/subcardioid/bidirectional).
|
||||
/// 2. **Bluetooth mode** — how Bluetooth headsets are handled:
|
||||
/// - "BT HFP voice" (`.allowBluetooth`): mono 8/16 kHz + heavy processing, BT mic.
|
||||
/// - "BT HFP voice" (`.allowBluetoothHFP` + `.allowBluetoothA2DP`): both profiles
|
||||
/// allowed, iOS picks HFP for two-way mic or A2DP for output-only. Mono, AEC on.
|
||||
/// - "Built-in Mic + BT A2DP stereo" (`.allowBluetoothA2DP` only): stereo output,
|
||||
/// built-in mic, no HFP processing.
|
||||
/// - "Built-in Mic + Speaker" (neither): no Bluetooth at all.
|
||||
@@ -24,9 +25,17 @@ private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAu
|
||||
/// Raw/Studio (`.measurement`: all processing off). Raw mode is allowed always
|
||||
/// but shows a warning when the output route is the speaker (echo risk, no AEC).
|
||||
///
|
||||
/// Additionally, **stereo capture** (2-channel built-in mic) can be enabled via
|
||||
/// `setPreferredInputNumberOfChannels(2)` — the core is then told via
|
||||
/// `vc_set_capture_channels(streamId, 2)`.
|
||||
/// Additionally, **stereo capture** (2-channel built-in mic) is enabled by switching the
|
||||
/// built-in mic's data source to the `.stereo` polar pattern. The recipe (mirroring
|
||||
/// TeamTalk5's `SoundDevicesModel.selectDataSource` + `UtilSound.setupSoundDevices`, which
|
||||
/// achieves stereo mic + A2DP output simultaneously) is: `setPreferredDataSource(.stereo
|
||||
/// source)` + `setPreferredPolarPattern(.stereo)` + `setPreferredInput(built-in mic)` +
|
||||
/// `setInputDataSource(stereo source)`. The channel count itself is NOT requested via
|
||||
/// `setPreferredInputNumberOfChannels(2)` — that session-level call is what collapses the
|
||||
/// A2DP output route (the original "stereo kills output" bug). Instead the core is told to
|
||||
/// open the device with 2 channels via `vc_set_capture_channels(streamId, 2)`, and the
|
||||
/// AVAudioSession input anchor (`setPreferredInput` + `setInputDataSource`) keeps the route
|
||||
/// stable during the HFP→A2DP and mono→stereo reconfigurations.
|
||||
///
|
||||
/// Voice Isolation / Wide Spectrum (iOS 17+/18+) are user-toggleable in Control Center
|
||||
/// for `.voiceChat` apps — surfaced as a hint, not a programmatic toggle.
|
||||
@@ -59,20 +68,18 @@ final class IOSAudioRouter: ObservableObject {
|
||||
/// Standard iOS VoIP experience: AEC/AGC/HPF on, mono, system picks best route
|
||||
/// (BT HFP if connected, wired if connected, speaker if nothing). Always available.
|
||||
case voiceChat = "Voice Chat"
|
||||
/// Stereo built-in mic capture (front+back capsules). A2DP output if BT is
|
||||
/// connected, else speaker/wired. Standard processing. Always available.
|
||||
/// Stereo built-in mic capture (front+back capsules). A2DP output if BT is connected,
|
||||
/// else built-in speaker / wired. Standard processing (no AEC — stereo needs a non-VPIO
|
||||
/// mode). Always available.
|
||||
case stereoMic = "Stereo Mic"
|
||||
/// Maximum fidelity: stereo mic, no AEC/AGC/HPF (raw mode). A2DP output if BT
|
||||
/// connected, else speaker/wired. Always available. Echo risk on speaker.
|
||||
/// Maximum fidelity: stereo mic, no AEC/AGC/HPF (raw mode). A2DP output if BT connected,
|
||||
/// else speaker/wired. Always available. Echo risk on speaker.
|
||||
case studio = "Studio (No Processing)"
|
||||
/// Bluetooth HFP: BT mic + BT output, AEC on, mono. Only when BT is connected.
|
||||
case bluetoothHeadset = "Bluetooth Headset (HFP)"
|
||||
/// A2DP stereo output + built-in mono mic. No hardware AEC (A2DP incompatible).
|
||||
/// Only when BT is connected.
|
||||
/// A2DP stereo output + built-in mono mic, AEC off. Only when BT is connected. (For
|
||||
/// A2DP output + stereo mic, use the Stereo Mic preset while BT is connected.)
|
||||
case btHeadphonesMonoMic = "BT Headphones + Mono Mic"
|
||||
/// A2DP stereo output + stereo built-in mic (front+back). No hardware AEC.
|
||||
/// Only when BT is connected.
|
||||
case btHeadphonesStereoMic = "BT Headphones + Stereo Mic"
|
||||
/// Wired headset/earpods: wired output + wired mic (or built-in), AEC on, mono.
|
||||
/// Only when a wired audio device is connected.
|
||||
case wiredHeadset = "Wired Headset"
|
||||
@@ -83,7 +90,7 @@ final class IOSAudioRouter: ObservableObject {
|
||||
|
||||
var requiresBluetooth: Bool {
|
||||
switch self {
|
||||
case .bluetoothHeadset, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return true
|
||||
case .bluetoothHeadset, .btHeadphonesMonoMic: return true
|
||||
default: return false
|
||||
}
|
||||
}
|
||||
@@ -95,7 +102,8 @@ final class IOSAudioRouter: ObservableObject {
|
||||
var bluetoothMode: BluetoothMode {
|
||||
switch self {
|
||||
case .voiceChat, .bluetoothHeadset: return .btHfpVoice
|
||||
case .stereoMic, .studio, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return .builtInMicBtA2dp
|
||||
// A2DP output when BT is connected; falls back to speaker/wired when it isn't.
|
||||
case .stereoMic, .studio, .btHeadphonesMonoMic: return .builtInMicBtA2dp
|
||||
case .wiredHeadset: return .builtInMicSpeaker
|
||||
case .custom: return .builtInMicSpeaker // placeholder
|
||||
}
|
||||
@@ -103,7 +111,7 @@ final class IOSAudioRouter: ObservableObject {
|
||||
|
||||
var captureChannels: CaptureChannels {
|
||||
switch self {
|
||||
case .stereoMic, .studio, .btHeadphonesStereoMic: return .stereo
|
||||
case .stereoMic, .studio: return .stereo
|
||||
default: return .mono
|
||||
}
|
||||
}
|
||||
@@ -118,7 +126,7 @@ final class IOSAudioRouter: ObservableObject {
|
||||
/// Whether this preset explicitly selects the built-in mic port.
|
||||
var usesBuiltInMic: Bool {
|
||||
switch self {
|
||||
case .stereoMic, .studio, .btHeadphonesMonoMic, .btHeadphonesStereoMic: return true
|
||||
case .stereoMic, .studio, .btHeadphonesMonoMic: return true
|
||||
default: return false
|
||||
}
|
||||
}
|
||||
@@ -273,7 +281,7 @@ final class IOSAudioRouter: ObservableObject {
|
||||
var activePreset: AudioPreset {
|
||||
// Check device-specific presets first (most specific → least specific)
|
||||
let order: [AudioPreset] = [
|
||||
.bluetoothHeadset, .btHeadphonesMonoMic, .btHeadphonesStereoMic,
|
||||
.bluetoothHeadset, .btHeadphonesMonoMic,
|
||||
.wiredHeadset,
|
||||
.voiceChat, .stereoMic, .studio,
|
||||
]
|
||||
@@ -307,96 +315,181 @@ final class IOSAudioRouter: ObservableObject {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
|
||||
// 1. Build category options from bluetooth mode.
|
||||
var options: AVAudioSession.CategoryOptions = [.defaultToSpeaker, .mixWithOthers]
|
||||
// .mixWithOthers is ALWAYS set — it keeps other audio (notably VoiceOver, which a
|
||||
// blind user needs to operate the phone) audible while our session is active. Never
|
||||
// drop it.
|
||||
// .defaultToSpeaker is set ONLY for the speaker preset. It forces output to the
|
||||
// built-in speaker instead of the receiver — but it also actively breaks A2DP
|
||||
// routing in .playAndRecord, so it must NOT be set for the A2DP or HFP presets.
|
||||
// .allowAirPlay is added to the Bluetooth presets so AirPlay output also works.
|
||||
var options: AVAudioSession.CategoryOptions = [.mixWithOthers]
|
||||
switch bluetoothMode {
|
||||
case .btHfpVoice:
|
||||
options.insert(.allowBluetooth)
|
||||
// Note: .allowBluetoothA2DP is NOT inserted — forces HFP for the mic path.
|
||||
case .builtInMicBtA2dp:
|
||||
// Voice Chat: allow BOTH HFP and A2DP, let iOS pick the right profile for the
|
||||
// connected device. This matches TeamTalk5's default (UtilSound.swift:228):
|
||||
// [.allowBluetoothHFP, .allowAirPlay, .allowBluetoothA2DP]
|
||||
// Making HFP and A2DP mutually exclusive (HFP-only here) blocks A2DP headphones
|
||||
// from receiving audio — the "Voice Chat kills Bluetooth output" regression.
|
||||
// HFP is *preferred* (the system uses HFP when a two-way mic path is needed),
|
||||
// but A2DP is still available for output-only scenarios.
|
||||
options.insert(.allowBluetoothHFP)
|
||||
options.insert(.allowBluetoothA2DP)
|
||||
// Note: .allowBluetooth is NOT inserted — no HFP, stereo A2DP output only.
|
||||
options.insert(.allowAirPlay)
|
||||
case .builtInMicBtA2dp:
|
||||
// A2DP output only (no HFP). With HFP disabled the Bluetooth device can only be
|
||||
// an OUTPUT (A2DP), so the system routes the mic to the built-in mic — exactly
|
||||
// what we want for "built-in mic + A2DP output", in either mono OR stereo.
|
||||
// This matches TeamTalk5's A2DP mode (UtilSound.swift:232-233): remove HFP from
|
||||
// the default set, leaving only A2DP.
|
||||
options.insert(.allowBluetoothA2DP)
|
||||
options.insert(.allowAirPlay)
|
||||
case .builtInMicSpeaker:
|
||||
// Neither Bluetooth option — built-in mic + speaker/wired output only.
|
||||
break
|
||||
// Built-in mic + speaker/wired output only. Prefer speaker over the receiver.
|
||||
options.insert(.defaultToSpeaker)
|
||||
}
|
||||
|
||||
// 2. Set category + mode based on mic processing mode AND bluetooth mode.
|
||||
// .voiceChat mode uses hardware AEC/AGC/HPF, but requires HFP-compatible routes.
|
||||
// A2DP output is NOT HFP — using .voiceChat with A2DP causes iOS to mute the output
|
||||
// because it can't set up the voice processing pipeline on an A2DP route. So:
|
||||
// - Standard + HFP or Speaker: .voiceChat (hardware AEC works)
|
||||
// - Standard + A2DP: .default (no hardware AEC, but audio routes correctly — A2DP
|
||||
// headphones are in-ear/over-ear so echo from built-in mic is minimal)
|
||||
// - Raw + any: .measurement (all processing off, regardless of bluetooth mode)
|
||||
// 2. Set category + mode. Recipe validated against TeamTalk5 / Ferrite, which both do
|
||||
// built-in stereo mic + A2DP Bluetooth output simultaneously:
|
||||
// - Stereo capture: .default — .voiceChat (the AEC/VPIO path) forces MONO, so stereo
|
||||
// is only possible in a non-VPIO mode. .default supports multi-capsule stereo AND
|
||||
// keeps the A2DP output route alive. (Earlier .videoRecording + a session-level
|
||||
// channel-count request collapsed A2DP output — the "stereo kills output" bug.)
|
||||
// - Mono raw/studio: .measurement — all system processing off.
|
||||
// - Mono + A2DP output: .videoRecording — keeps A2DP output without VPIO (no AEC).
|
||||
// - Mono standard (HFP or speaker): .voiceChat — hardware AEC/AGC/HPF.
|
||||
let mode: AVAudioSession.Mode
|
||||
switch (micMode, bluetoothMode) {
|
||||
case (.standard, .builtInMicBtA2dp):
|
||||
mode = .default // A2DP + hardware AEC = incompatible
|
||||
case (.standard, _):
|
||||
mode = .voiceChat // HFP or speaker: hardware AEC works
|
||||
case (.raw, _):
|
||||
mode = .measurement // all processing off
|
||||
if captureChannels == .stereo {
|
||||
mode = .default
|
||||
} else if micMode == .raw {
|
||||
mode = .measurement
|
||||
} else if bluetoothMode == .builtInMicBtA2dp {
|
||||
mode = .videoRecording
|
||||
} else {
|
||||
mode = .voiceChat
|
||||
}
|
||||
|
||||
do {
|
||||
try session.setCategory(.playAndRecord, mode: mode, options: options)
|
||||
logger.info("setCategory ok — mode=\(self.modeLabel(mode)), bt=\(self.bluetoothMode.rawValue), options=\(self.optionsLabel(options))")
|
||||
logger.info("setCategory ok — mode=\(self.modeLabel(mode)), bt=\(self.bluetoothMode.rawValue), ch=\(self.captureChannels.rawValue), options=\(self.optionsLabel(options))")
|
||||
} catch {
|
||||
logger.error("setCategory failed: \(error.localizedDescription)")
|
||||
}
|
||||
|
||||
// 3. Set preferred input port (skip if "Default" — empty/nil ID means use system default).
|
||||
if let portId = selectedInputPortId, !portId.isEmpty,
|
||||
let port = session.availableInputs?.first(where: { $0.uid == portId }) {
|
||||
// 3. Input & mic-capsule configuration.
|
||||
if captureChannels == .stereo {
|
||||
// Stereo: enable the built-in mic's .stereo polar pattern AND anchor the input
|
||||
// route explicitly via setPreferredInput + setInputDataSource. The session-level
|
||||
// channel-count call (setPreferredInputNumberOfChannels(2)) is what collapses the
|
||||
// A2DP output route — NOT setPreferredInput (TeamTalk5 uses setPreferredInput and
|
||||
// gets stereo + A2DP). With HFP disabled the system routes input to the built-in
|
||||
// mic, but without the explicit preferred-input anchor the route can collapse
|
||||
// during the mode switch (.voiceChat → .default) and the output dies. The channel
|
||||
// count is requested by miniaudio at the audio-unit level (vc_set_capture_channels).
|
||||
configureStereoCapture(session: session)
|
||||
} else if let portId = selectedInputPortId, !portId.isEmpty,
|
||||
let port = session.availableInputs?.first(where: { $0.uid == portId }) {
|
||||
// Mono with an explicit input-port selection (advanced settings).
|
||||
do {
|
||||
try session.setPreferredInput(port)
|
||||
logger.info("setPreferredInput ok — \(port.portName)")
|
||||
} catch {
|
||||
logger.error("setPreferredInput failed: \(error.localizedDescription)")
|
||||
}
|
||||
|
||||
// 4. Set preferred data source (orientation) on the selected input port.
|
||||
if let dataSourceId = selectedDataSourceId, !dataSourceId.isEmpty,
|
||||
let dataSource = port.dataSources?.first(where: { String(describing: $0.dataSourceID) == dataSourceId }) {
|
||||
do {
|
||||
try port.setPreferredDataSource(dataSource)
|
||||
logger.info("setPreferredDataSource ok — \(dataSource.dataSourceName)")
|
||||
} catch {
|
||||
logger.error("setPreferredDataSource failed: \(error.localizedDescription)")
|
||||
}
|
||||
|
||||
// 5. Set preferred polar pattern on the data source.
|
||||
if let polarPattern = selectedPolarPattern, !polarPattern.isEmpty {
|
||||
let pattern = AVAudioSession.PolarPattern(rawValue: polarPattern)
|
||||
do {
|
||||
try dataSource.setPreferredPolarPattern(pattern)
|
||||
logger.info("setPreferredPolarPattern ok — \(polarPattern)")
|
||||
} catch {
|
||||
logger.error("setPreferredPolarPattern failed: \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Set preferred input number of channels — ONLY for stereo (non-default).
|
||||
// Calling setPreferredInputNumberOfChannels(1) for mono is unnecessary (1 is the
|
||||
// default) and may put the session in a bad state on some devices.
|
||||
if captureChannels == .stereo {
|
||||
do {
|
||||
try session.setPreferredInputNumberOfChannels(2)
|
||||
logger.info("setPreferredInputNumberOfChannels ok — 2 (stereo)")
|
||||
} catch {
|
||||
logger.error("setPreferredInputNumberOfChannels failed: \(error.localizedDescription)")
|
||||
}
|
||||
configureMonoCapture(session: session, port: port)
|
||||
} else {
|
||||
// Mono, system-default input. Still clear any leftover .stereo capsule from a
|
||||
// prior stereo session so we actually return to mono.
|
||||
clearStereoPolarPattern(session: session)
|
||||
}
|
||||
|
||||
updateWarnings()
|
||||
}
|
||||
|
||||
/// Enable 2-channel capture on the built-in mic. Mirrors TeamTalk5's recipe
|
||||
/// (`SoundDevicesModel.selectDataSource` + `UtilSound.setupSoundDevices`), which
|
||||
/// achieves stereo mic + A2DP Bluetooth output simultaneously:
|
||||
/// 1. `setPreferredDataSource(stereoSource)` on the built-in mic port
|
||||
/// 2. `setPreferredPolarPattern(.stereo)` on that data source
|
||||
/// 3. `setPreferredInput(builtIn)` — anchor the input route explicitly (this is NOT
|
||||
/// what collapses A2DP — the session-level `setPreferredInputNumberOfChannels(2)`
|
||||
/// is. Without this anchor the route can collapse during the mode switch.)
|
||||
/// 4. `setInputDataSource(stereoSource)` — commit the data source at the session level
|
||||
/// The channel count itself is requested by miniaudio at the audio-unit level via
|
||||
/// `vc_set_capture_channels(2)`. We do NOT call `setPreferredInputNumberOfChannels(2)`
|
||||
/// — that session-level call is the one that collapses the A2DP output route.
|
||||
private func configureStereoCapture(session: AVAudioSession) {
|
||||
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
|
||||
else {
|
||||
logger.warning("stereo requested but no built-in mic available — staying mono")
|
||||
return
|
||||
}
|
||||
guard let stereoSource = builtIn.dataSources?.first(where: {
|
||||
$0.supportedPolarPatterns?.contains(.stereo) == true
|
||||
}) else {
|
||||
logger.warning("stereo requested but built-in mic has no .stereo data source — staying mono")
|
||||
return
|
||||
}
|
||||
do {
|
||||
try builtIn.setPreferredDataSource(stereoSource)
|
||||
try stereoSource.setPreferredPolarPattern(.stereo)
|
||||
// Anchor the input route explicitly. TeamTalk5 does this (SoundDevicesModel
|
||||
// .selectDataSource:147); without it the route can collapse during the mode
|
||||
// switch (.voiceChat → .default) and the A2DP output dies.
|
||||
try session.setPreferredInput(builtIn)
|
||||
// Commit the data source at the session level (TeamTalk does this at
|
||||
// SoundDevicesModel.selectDataSource:148). setPreferredDataSource alone only
|
||||
// sets the port-level preference; setInputDataSource makes it the active source.
|
||||
try session.setInputDataSource(stereoSource)
|
||||
logger.info("stereo capsule enabled — source=\(stereoSource.dataSourceName), pattern=.stereo, input anchored")
|
||||
} catch {
|
||||
logger.error("stereo capsule setup failed: \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
|
||||
/// Configure mono capture on an explicitly selected port: apply the user's chosen data source
|
||||
/// (orientation) and polar pattern, resetting any prior `.stereo` pattern back to default.
|
||||
private func configureMonoCapture(session: AVAudioSession, port: AVAudioSessionPortDescription) {
|
||||
guard let dataSourceId = selectedDataSourceId, !dataSourceId.isEmpty,
|
||||
let dataSource = port.dataSources?.first(where: {
|
||||
String(describing: $0.dataSourceID) == dataSourceId
|
||||
}) else {
|
||||
// No explicit capsule choice — make sure we're not stuck on a prior .stereo pattern.
|
||||
clearStereoPolarPattern(session: session)
|
||||
return
|
||||
}
|
||||
do {
|
||||
try port.setPreferredDataSource(dataSource)
|
||||
logger.info("setPreferredDataSource ok — \(dataSource.dataSourceName)")
|
||||
} catch {
|
||||
logger.error("setPreferredDataSource failed: \(error.localizedDescription)")
|
||||
}
|
||||
|
||||
if let polarPattern = selectedPolarPattern, !polarPattern.isEmpty {
|
||||
let pattern = AVAudioSession.PolarPattern(rawValue: polarPattern)
|
||||
try? dataSource.setPreferredPolarPattern(pattern)
|
||||
logger.info("setPreferredPolarPattern ok — \(polarPattern)")
|
||||
} else {
|
||||
// Clear any prior .stereo selection so mono capture returns to a mono capsule.
|
||||
try? dataSource.setPreferredPolarPattern(nil)
|
||||
}
|
||||
}
|
||||
|
||||
/// Reset any built-in-mic data source that's currently on the `.stereo` polar pattern back to
|
||||
/// the default (mono) pattern. Used when switching from a stereo session back to mono with no
|
||||
/// explicit capsule selection, so the prior stereo capsule doesn't linger.
|
||||
private func clearStereoPolarPattern(session: AVAudioSession) {
|
||||
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
|
||||
else { return }
|
||||
for ds in builtIn.dataSources ?? [] where ds.selectedPolarPattern == .stereo {
|
||||
try? ds.setPreferredPolarPattern(nil)
|
||||
}
|
||||
}
|
||||
|
||||
private func modeLabel(_ mode: AVAudioSession.Mode) -> String {
|
||||
switch mode {
|
||||
case .voiceChat: return "voiceChat"
|
||||
case .measurement: return "measurement"
|
||||
case .videoRecording: return "videoRecording"
|
||||
case .default: return "default"
|
||||
default: return "other"
|
||||
}
|
||||
@@ -406,7 +499,7 @@ final class IOSAudioRouter: ObservableObject {
|
||||
var parts: [String] = []
|
||||
if opts.contains(.defaultToSpeaker) { parts.append("defaultToSpeaker") }
|
||||
if opts.contains(.mixWithOthers) { parts.append("mixWithOthers") }
|
||||
if opts.contains(.allowBluetooth) { parts.append("allowBluetooth") }
|
||||
if opts.contains(.allowBluetoothHFP) { parts.append("allowBluetoothHFP") }
|
||||
if opts.contains(.allowBluetoothA2DP) { parts.append("allowBluetoothA2DP") }
|
||||
return parts.joined(separator: ",")
|
||||
}
|
||||
@@ -484,6 +577,13 @@ final class IOSAudioRouter: ObservableObject {
|
||||
captureChannels = channels
|
||||
savePreferences()
|
||||
applyConfiguration()
|
||||
// Sync the core's capture channel count. The core's set_capture_channels handles
|
||||
// the engine restart internally (stop + ensure_audio_running) — no need for the
|
||||
// Swift layer to suspend/restart separately.
|
||||
if let streamId = AudioSessionManager.shared.activeMicStreamId {
|
||||
_ = AudioSessionManager.shared.client?.setCaptureChannels(
|
||||
streamId: streamId, channels: channels.channelCount)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Presets
|
||||
@@ -520,6 +620,12 @@ final class IOSAudioRouter: ObservableObject {
|
||||
UserDefaults.standard.set(preset.rawValue, forKey: kPreset)
|
||||
savePreferences()
|
||||
applyConfiguration()
|
||||
// Sync the core's capture channel count. The core's set_capture_channels handles
|
||||
// the engine restart internally.
|
||||
if let streamId = AudioSessionManager.shared.activeMicStreamId {
|
||||
_ = AudioSessionManager.shared.client?.setCaptureChannels(
|
||||
streamId: streamId, channels: preset.captureChannels.channelCount)
|
||||
}
|
||||
refreshRoutes()
|
||||
logger.info("applyPreset — \(preset.rawValue)")
|
||||
}
|
||||
@@ -532,10 +638,17 @@ final class IOSAudioRouter: ObservableObject {
|
||||
let outputIsSpeaker = session.currentRoute.outputs.contains { $0.portType == .builtInSpeaker }
|
||||
// Raw/Studio mode + speaker = echo risk (no AEC in .measurement mode)
|
||||
showsRawModeSpeakerWarning = (micMode == .raw && outputIsSpeaker)
|
||||
// Standard mode + A2DP = no hardware AEC (A2DP incompatible with .voiceChat mode)
|
||||
showsA2dpNoAecWarning = (micMode == .standard && bluetoothMode == .builtInMicBtA2dp)
|
||||
// A2DP output runs without hardware AEC (the .voiceChat AEC path isn't available on an
|
||||
// A2DP route). Applies to both mono and stereo A2DP. Stereo also has no AEC (it can't
|
||||
// use .voiceChat at all), but the message is the same and the warning already shows when
|
||||
// the bluetooth mode is A2DP.
|
||||
showsA2dpNoAecWarning = (bluetoothMode == .builtInMicBtA2dp)
|
||||
}
|
||||
|
||||
/// Whether the current configuration wants Bluetooth A2DP output. Used after session
|
||||
/// activation to clear any lingering speaker override that would pin output to the speaker.
|
||||
var wantsA2dpOutput: Bool { bluetoothMode == .builtInMicBtA2dp }
|
||||
|
||||
/// The selected input port object, if any.
|
||||
var selectedPort: IOSAudioInputPort? {
|
||||
inputPorts.first(where: { $0.id == selectedInputPortId })
|
||||
|
||||
@@ -204,6 +204,11 @@ final class SessionState {
|
||||
if result == .ok {
|
||||
voiceState.micActive = true
|
||||
voiceState.localStreamId = streamId
|
||||
// Publish the active mic stream ID so IOSAudioRouter can reset the core's capture
|
||||
// channel count when the user switches mono↔stereo (selectCaptureChannels /
|
||||
// applyPreset). Without this, switching stereo→mono leaves the LocalStream's
|
||||
// capture_channels field at 2 and the next engine start still opens stereo.
|
||||
AudioSessionManager.shared.activeMicStreamId = streamId
|
||||
// Apply the user's capture channel selection (mono/stereo) from IOSAudioRouter.
|
||||
// The core opens the capture device via miniaudio on the next engine start;
|
||||
// vc_set_capture_channels tells it to open in stereo (2) or mono (1).
|
||||
@@ -220,6 +225,7 @@ final class SessionState {
|
||||
if voiceState.localStreamId != 0 {
|
||||
client.stopStream(voiceState.localStreamId)
|
||||
voiceState.localStreamId = 0
|
||||
AudioSessionManager.shared.activeMicStreamId = nil
|
||||
}
|
||||
voiceState.micActive = false
|
||||
voiceState.level = 0
|
||||
|
||||
@@ -115,6 +115,7 @@ struct SettingsView: View {
|
||||
.accessibilityLabel("Info: A2DP output mode does not support hardware echo cancellation")
|
||||
}
|
||||
|
||||
|
||||
// Capture channels: Mono vs Stereo
|
||||
Picker("Channels", selection: Binding(
|
||||
get: { router.captureChannels },
|
||||
|
||||
@@ -399,11 +399,12 @@ VC_API vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const
|
||||
/* Set the capture channel count for a local MIC stream (1 = mono, 2 = stereo interleaved).
|
||||
* Must be called after vc_stream_start; takes effect on the next AudioEngine restart (e.g. when
|
||||
* joining voice, or immediately if the engine is already running — it stops and restarts the
|
||||
* capture device with the new channel count). Defaults to 1 (mono). On iOS this lets the Swift
|
||||
* AVAudioSession routing layer request stereo built-in mic capture via
|
||||
* setPreferredInputNumberOfChannels(2) and then tell the core to open the capture device in
|
||||
* stereo. VC_ERR_INVALID_ARG if stream_id is unknown, the stream is not a MIC stream, or
|
||||
* channels is not 1 or 2. */
|
||||
* capture device with the new channel count). Defaults to 1 (mono). On iOS the Swift
|
||||
* AVAudioSession routing layer enables stereo built-in mic capture by switching the built-in
|
||||
* mic's data source to the .stereo polar pattern (setPreferredDataSource +
|
||||
* setPreferredPolarPattern(.stereo) + setPreferredInput + setInputDataSource) and then calls
|
||||
* this to tell the core to open the capture device in stereo. VC_ERR_INVALID_ARG if stream_id
|
||||
* is unknown, the stream is not a MIC stream, or channels is not 1 or 2. */
|
||||
VC_API vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels);
|
||||
|
||||
/* ── Text ─────────────────────────────────────────────────────────────────── */
|
||||
@@ -484,6 +485,16 @@ VC_API vc_result vc_get_permissions(vc_client* c, vc_permissions* out);
|
||||
VC_API vc_result vc_audio_suspend(vc_client* c);
|
||||
VC_API vc_result vc_audio_resume(vc_client* c);
|
||||
|
||||
/* Full audio engine restart (iOS M6). Unlike vc_audio_suspend/resume which only stop/start
|
||||
* the existing miniaudio devices (leaving them bound to the route that was active when they
|
||||
* were opened), vc_audio_restart() uninitializes and re-initializes the capture and playback
|
||||
* devices so they pick up a new AVAudioSession route. Call this from the Swift layer AFTER
|
||||
* reconfiguring AVAudioSession (setCategory, setPreferredInput, setPreferredPolarPattern, etc.)
|
||||
* so the core's devices reopen against the new route. Mirrors TeamTalk5's
|
||||
* closeSoundDevices()/initSoundInputDevice()/initSoundOutputDevice() reconfiguration pattern.
|
||||
* Safe to call when the engine is not running (it will just start it). */
|
||||
VC_API vc_result vc_audio_restart(vc_client* c);
|
||||
|
||||
#if defined(__cplusplus)
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
@@ -1433,6 +1433,22 @@ vc_result vc_client::audio_resume() {
|
||||
return audio_engine_.resume() ? VC_OK : VC_ERR_AUDIO;
|
||||
}
|
||||
|
||||
vc_result vc_client::audio_restart() {
|
||||
// Full uninit + re-init (not just stop/start like suspend/resume) so the miniaudio
|
||||
// devices reopen against the current AVAudioSession route. Mirrors TeamTalk5's
|
||||
// closeSoundDevices()/initSoundInputDevice()/initSoundOutputDevice() pattern.
|
||||
// IMPORTANT: only restart if the engine was already running — calling
|
||||
// ensure_audio_running() when the engine isn't running would start it prematurely
|
||||
// (opening the capture device / mic without an active mic stream), which on iOS
|
||||
// triggers a route reconfiguration that can collapse the output route.
|
||||
bool was_running = audio_engine_.running();
|
||||
audio_engine_.stop();
|
||||
if (was_running) {
|
||||
ensure_audio_running();
|
||||
}
|
||||
return VC_OK;
|
||||
}
|
||||
|
||||
vc_result vc_client::get_stream_audio_config(uint32_t user_id, uint32_t stream_id,
|
||||
vc_audio_config* out) {
|
||||
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
||||
@@ -1900,5 +1916,6 @@ vc_result vc_client::get_account_list(vc_account_list*) { return VC_ERR_NOT_IMPL
|
||||
vc_result vc_client::get_permissions(vc_permissions*) { return VC_ERR_NOT_IMPLEMENTED; }
|
||||
vc_result vc_client::audio_suspend() { return VC_ERR_NOT_IMPLEMENTED; }
|
||||
vc_result vc_client::audio_resume() { return VC_ERR_NOT_IMPLEMENTED; }
|
||||
vc_result vc_client::audio_restart() { return VC_ERR_NOT_IMPLEMENTED; }
|
||||
|
||||
#endif // VOICECAT_HAS_NET
|
||||
|
||||
@@ -60,6 +60,7 @@ struct vc_client {
|
||||
vc_remote_stream_state* out);
|
||||
vc_result audio_suspend();
|
||||
vc_result audio_resume();
|
||||
vc_result audio_restart();
|
||||
|
||||
vc_result send_text(vc_text_scope scope, uint32_t target_id, const char* utf8);
|
||||
|
||||
|
||||
@@ -310,4 +310,9 @@ vc_result vc_audio_resume(vc_client* c) {
|
||||
return c->audio_resume();
|
||||
}
|
||||
|
||||
vc_result vc_audio_restart(vc_client* c) {
|
||||
if (c == nullptr) return VC_ERR_INVALID_ARG;
|
||||
return c->audio_restart();
|
||||
}
|
||||
|
||||
} // extern "C"
|
||||
|
||||
@@ -136,7 +136,7 @@ Design notes:
|
||||
|
||||
### Per-platform binding notes
|
||||
|
||||
- **Swift / Apple.** Import the C ABI via a **module map** (`module VoiceCatC { header "voicecat.h" }`) staged into the XCFramework headers by `clients/apple/scripts/build-xcframework.sh` — Swift gets a clean `import VoiceCatC` with all C enums/structs/functions available directly (no manual redeclaration, unlike the C# P/Invoke layer). A **Swift wrapper** (`VoiceCatCore` package at `clients/apple/`) provides Swift-idiomatic types (`VoiceCatResult`, `VoiceCatEvent`, `Channel`, `User`, etc.) on top, mirroring the C# `VoiceCat.Interop` layer. Callbacks use `@convention(c)` closures (plain C function pointers, not ARC-managed closures) + `Unmanaged.passUnretained(self)` as the `user` context (the Swift analog of C#'s `[UnmanagedCallersOnly]` + `GCHandle`). Events are delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (one async block scheduled at a time) — the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump. `deinit` calls `vc_client_destroy` (joins all threads) then frees native CString config storage (the core stores raw pointers, doesn't copy). **macOS UI: AppKit** (chosen over SwiftUI for the most mature VoiceOver accessibility story — same rationale as the Windows client's WinForms choice); **iOS UI: SwiftUI** (narrower control surface, sufficient VoiceOver support). On **iOS** the app owns `AVAudioSession` (category `.playAndRecord`), requests mic permission, and handles interruptions/route changes — the core exposes hooks (`vc_audio_suspend`/`vc_audio_resume`, implemented) the Swift layer calls from `AVAudioSession` notifications. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture) is driven from the Swift `IOSAudioRouter` singleton via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The core is told the capture channel count via `vc_set_capture_channels` (append-only ABI). iOS 18.0 deployment target. Background voice and VoIP push (CallKit/PushKit) are a later milestone. The XCFramework carries a **fat static library** (`libvoicecat-fat.a`) bundling `libvoicecat.a` + all vcpkg static deps so the Swift Package links a single self-contained `.a` per slice.
|
||||
- **Swift / Apple.** Import the C ABI via a **module map** (`module VoiceCatC { header "voicecat.h" }`) staged into the XCFramework headers by `clients/apple/scripts/build-xcframework.sh` — Swift gets a clean `import VoiceCatC` with all C enums/structs/functions available directly (no manual redeclaration, unlike the C# P/Invoke layer). A **Swift wrapper** (`VoiceCatCore` package at `clients/apple/`) provides Swift-idiomatic types (`VoiceCatResult`, `VoiceCatEvent`, `Channel`, `User`, etc.) on top, mirroring the C# `VoiceCat.Interop` layer. Callbacks use `@convention(c)` closures (plain C function pointers, not ARC-managed closures) + `Unmanaged.passUnretained(self)` as the `user` context (the Swift analog of C#'s `[UnmanagedCallersOnly]` + `GCHandle`). Events are delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (one async block scheduled at a time) — the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump. `deinit` calls `vc_client_destroy` (joins all threads) then frees native CString config storage (the core stores raw pointers, doesn't copy). **macOS UI: AppKit** (chosen over SwiftUI for the most mature VoiceOver accessibility story — same rationale as the Windows client's WinForms choice); **iOS UI: SwiftUI** (narrower control surface, sufficient VoiceOver support). On **iOS** the app owns `AVAudioSession` (category `.playAndRecord`), requests mic permission, and handles interruptions/route changes — the core exposes hooks (`vc_audio_suspend`/`vc_audio_resume`/`vc_audio_restart`, implemented) the Swift layer calls from `AVAudioSession` notifications and `IOSAudioRouter` setting changes. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture) is driven from the Swift `IOSAudioRouter` singleton via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The core is told the capture channel count via `vc_set_capture_channels` (append-only ABI). `vc_audio_restart` does a full stop + re-init (unlike `suspend`/`resume` which only stop/start) so devices reopen against a new route after `AVAudioSession` reconfiguration. iOS 18.0 deployment target. Background voice and VoIP push (CallKit/PushKit) are a later milestone. The XCFramework carries a **fat static library** (`libvoicecat-fat.a`) bundling `libvoicecat.a` + all vcpkg static deps so the Swift Package links a single self-contained `.a` per slice.
|
||||
- **iOS screen / system-audio sharing** is supported via a **ReplayKit Broadcast Upload
|
||||
Extension** (the same mechanism Discord uses; triggered from Control Center's screen-record
|
||||
button via `RPSystemBroadcastPickerView`). The extension receives
|
||||
|
||||
@@ -31,7 +31,7 @@ explicit resampling (speexdsp/libsamplerate) is only needed when a device can't
|
||||
| UI — macOS | **AppKit** | Chosen over SwiftUI for the most mature, granular **VoiceOver** accessibility story (per-control `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`, `NSAccessibility.post(.announcement)` for live announcements) — the same rationale that drove the Windows client to WinForms over WinUI 3 for screen-reader (NVDA/JAWS/Narrator) UIA support (resolved decision in `docs/roadmap.md`). macOS 14 (Sonoma) deployment target. |
|
||||
| UI — iOS | **SwiftUI** | iOS has a narrower control surface (no channel-tree moderation, etc.) and SwiftUI's VoiceOver support is sufficient; revisit if gaps emerge. iOS 18.0 deployment target (unlocks newest AVAudioSession APIs: stereo capture, polar patterns, data sources). |
|
||||
| Shared core | **VoiceCatCore** Swift Package | One Swift library wrapping the C ABI, consumed by both the macOS AppKit app and the iOS SwiftUI app. Mirrors the C# `VoiceCat.Interop` layer. Events delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump). |
|
||||
| Audio session (iOS) | **AVAudioSession** + **IOSAudioRouter** | App owns category `.playAndRecord`, mic permission, interruption/route-change handling; calls `vc_audio_suspend`/`vc_audio_resume` (implemented) on the core. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture via `setPreferredInputNumberOfChannels(2)`) is driven from Swift via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The `IOSAudioRouter` singleton owns this; the core is told the channel count via `vc_set_capture_channels`. macOS uses CoreAudio via the core directly. |
|
||||
| Audio session (iOS) | **AVAudioSession** + **IOSAudioRouter** | App owns category `.playAndRecord`, mic permission, interruption/route-change handling; calls `vc_audio_suspend`/`vc_audio_resume`/`vc_audio_restart` (implemented) on the core. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture via `.stereo` polar pattern + `setPreferredInput` + `setInputDataSource`) is driven from Swift via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The `IOSAudioRouter` singleton owns this; the core is told the channel count via `vc_set_capture_channels`. When settings change mid-session, devices are suspended (`vc_audio_suspend`), the session is reconfigured, and devices are restarted (`vc_audio_restart`) to pick up the new route. macOS uses CoreAudio via the core directly. |
|
||||
| Packaging | Swift Package + Xcode project | Core shipped as an **XCFramework** binary target — a fat static library (`libvoicecat-fat.a`) bundling `libvoicecat.a` + all vcpkg static deps (protobuf/mbedtls/sodium/opus/sqlite3/spdlog/asio), so the Swift Package links a single self-contained `.a` per slice. macOS slice validated; iOS device + sim slices are scaffolding. |
|
||||
| Future | CallKit / PushKit | For background VoIP + incoming-call UX on iOS. Post-v1. |
|
||||
|
||||
|
||||
@@ -178,23 +178,27 @@ Each receiver keeps an **adaptive jitter buffer per ssrc**.
|
||||
|
||||
- Capture and playback run on miniaudio's real-time callbacks (WASAPI / CoreAudio / ALSA).
|
||||
Playback is genuinely stereo end-to-end. **Mic capture** is mono by default; **stereo mic
|
||||
capture** is supported via `vc_set_capture_channels(stream_id, 2)` (e.g. iOS stereo built-in
|
||||
mic via `AVAudioSession.setPreferredInputNumberOfChannels(2)`) — when enabled, the capture
|
||||
device opens in stereo (interleaved L/R) and the encoder receives real stereo PCM (no upmix).
|
||||
A mono mic frame on a stereo channel is upmixed L=R before encoding so the Opus bitstream is
|
||||
still spec-correct stereo. **Screen-audio (`SCREEN_AUDIO`) loopback** captures in the
|
||||
channel's mode — stereo when the channel is stereo (real interleaved L/R, no downmix), mono
|
||||
when the channel is mono — so a stereo music/screen-share channel gets genuine stereo
|
||||
end-to-end. See §9 for the platform-specific loopback mechanism.
|
||||
capture** is supported via `vc_set_capture_channels(stream_id, 2)` — when enabled, the
|
||||
capture device opens in stereo (interleaved L/R) and the encoder receives real stereo PCM
|
||||
(no upmix). A mono mic frame on a stereo channel is upmixed L=R before encoding so the Opus
|
||||
bitstream is still spec-correct stereo. **Screen-audio (`SCREEN_AUDIO`) loopback** captures
|
||||
in the channel's mode — stereo when the channel is stereo (real interleaved L/R, no
|
||||
downmix), mono when the channel is mono — so a stereo music/screen-share channel gets
|
||||
genuine stereo end-to-end. See §9 for the platform-specific loopback mechanism.
|
||||
- **iOS mic capture:** all iOS audio routing is driven from Swift via `AVAudioSession` by the
|
||||
`IOSAudioRouter` singleton *before* the core (miniaudio) opens its device — miniaudio does
|
||||
NOT touch `AVAudioSession` on iOS. Input port selection (`availableInputs`), built-in mic
|
||||
orientation (`setPreferredDataSource`: front/back/top/bottom), polar patterns
|
||||
(`setPreferredPolarPattern`: omni/cardioid/subcardioid/bidirectional), mic processing mode
|
||||
(`.voiceChat` = Standard with AEC/AGC/HPF, or `.measurement` = Raw/Studio with all processing
|
||||
off), Bluetooth mode (`.allowBluetooth` HFP voice vs `.allowBluetoothA2DP` stereo output vs
|
||||
neither), and stereo capture (`setPreferredInputNumberOfChannels(2)` → `vc_set_capture_channels`)
|
||||
are all set from Swift. The core then opens whatever route AVAudioSession has established.
|
||||
off), Bluetooth mode (`.allowBluetoothHFP` HFP voice vs `.allowBluetoothA2DP` stereo output
|
||||
vs neither), and stereo capture (`.stereo` polar pattern + `setPreferredInput` +
|
||||
`setInputDataSource` → `vc_set_capture_channels`) are all set from Swift. The core then
|
||||
opens whatever route AVAudioSession has established. When the user changes audio settings
|
||||
mid-session, `IOSAudioRouter` suspends the core's devices (`vc_audio_suspend`), reconfigures
|
||||
`AVAudioSession`, then restarts the devices (`vc_audio_restart`) so they reopen against the
|
||||
new route — mirroring TeamTalk5's `closeSoundDevices`/`initSoundInputDevice`/
|
||||
`initSoundOutputDevice` pattern.
|
||||
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
|
||||
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
|
||||
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard
|
||||
|
||||
Reference in New Issue
Block a user