feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture

Three iOS client problems fixed plus a new core stereo-mic capture ABI:

1. Channel-id sync bug (mic button permanently dimmed): SessionState never
   synced currentChannelId from the self user's channelId on connect, so the
   mic button (gated on currentChannelId == 0) stayed dimmed. Added
   syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
   called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
   Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.

2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
   text button (parity with macOS). Mute/deafen disable when not in voice.

3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
   port selection, built-in mic orientation/polar patterns, Bluetooth
   HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
   UserDefaults persistence. AudioSessionManager delegates to it.

4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
   lets the core open the mic device in stereo (2-ch interleaved). LocalStream
   gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
   capture_accum_ + on_capture updated to channel-aware accumulation. Test
   test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
   VoiceCatClient.setCaptureChannels.

5. Settings UI rework: AVAudioSession-derived input/output tree replaces
   miniaudio device picker.

6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
   swift-tools-version 6.0 with swiftLanguageModes .v5.

Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.

Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
This commit is contained in:
2026-06-19 13:17:52 +02:00
parent a10a18aebe
commit 9fc51cffc4
23 changed files with 861 additions and 83 deletions

View File

@@ -177,12 +177,24 @@ Each receiver keeps an **adaptive jitter buffer per ssrc**.
```
- Capture and playback run on miniaudio's real-time callbacks (WASAPI / CoreAudio / ALSA).
Playback is genuinely stereo end-to-end. **Mic capture stays mono** (no stereo mic in v1);
a mono mic frame on a stereo channel is upmixed L=R before encoding so the Opus bitstream
is still spec-correct stereo. **Screen-audio (`SCREEN_AUDIO`) loopback** captures in the
Playback is genuinely stereo end-to-end. **Mic capture** is mono by default; **stereo mic
capture** is supported via `vc_set_capture_channels(stream_id, 2)` (e.g. iOS stereo built-in
mic via `AVAudioSession.setPreferredInputNumberOfChannels(2)`) — when enabled, the capture
device opens in stereo (interleaved L/R) and the encoder receives real stereo PCM (no upmix).
A mono mic frame on a stereo channel is upmixed L=R before encoding so the Opus bitstream is
still spec-correct stereo. **Screen-audio (`SCREEN_AUDIO`) loopback** captures in the
channel's mode — stereo when the channel is stereo (real interleaved L/R, no downmix), mono
when the channel is mono — so a stereo music/screen-share channel gets genuine stereo
end-to-end. See §9 for the platform-specific loopback mechanism.
- **iOS mic capture:** all iOS audio routing is driven from Swift via `AVAudioSession` by the
`IOSAudioRouter` singleton *before* the core (miniaudio) opens its device — miniaudio does
NOT touch `AVAudioSession` on iOS. Input port selection (`availableInputs`), built-in mic
orientation (`setPreferredDataSource`: front/back/top/bottom), polar patterns
(`setPreferredPolarPattern`: omni/cardioid/subcardioid/bidirectional), mic processing mode
(`.voiceChat` = Standard with AEC/AGC/HPF, or `.measurement` = Raw/Studio with all processing
off), Bluetooth mode (`.allowBluetooth` HFP voice vs `.allowBluetoothA2DP` stereo output vs
neither), and stereo capture (`setPreferredInputNumberOfChannels(2)``vc_set_capture_channels`)
are all set from Swift. The core then opens whatever route AVAudioSession has established.
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard