docs(progress): plan iOS audio overhaul + Join/Leave Voice + channel-id sync fix

PROGRESS.md

@@ -10,6 +10,109 @@ up instantly. Newest status at the top.
 
 ## ▶ Where we left off / next action
 
+- **Planned (not started):** **iOS audio overhaul + Join/Leave Voice + channel-id sync fix**
+  (2026-06-19, plan written on Windows; implement on Mac). Three problems found while reviewing
+  the iOS client:
+  1. **Mic button permanently dimmed (BUG — root cause).** `VoiceControlsView.swift:26` gates the
+     mic button on `session.currentChannelId == 0`, but `SessionState` never syncs
+     `currentChannelId` from the self user's `channelId` on connect. The server auto-places every
+     newly-authed user into the Lobby (channel 1, `server/src/session_registry.cpp:111`), but the
+     iOS client ignores it — `currentChannelId` is set in only two places: `.joinResult`
+     (`SessionState.swift:97`) and reset to 0 by `leaveChannel()` (`SessionState.swift:135`). The
+     `init` (`SessionState.swift:55-57`) calls `refreshUsers()` but throws away self's channelId;
+     the `.channelList`/`.userUpdated` handlers (`SessionState.swift:76-79`) likewise only re-fetch
+     the list. macOS does this sync correctly (`MainWindowController.swift:461,491,522`). Because
+     the button stays dimmed, `startMicStream()` (and thus the mic permission prompt) can never be
+     triggered — a chicken-and-egg that *looks* like a permission issue. **Not** a permission bug:
+     `Info.plist:23` declares `NSMicrophoneUsageDescription` and `SessionState.swift:139` calls
+     `AVAudioApplication.requestRecordPermission`. Fix = mirror macOS: add `syncSelfChannel()`,
+     call it from `init`/`.channelList`/`.userJoined`/`.userLeft`/`.userUpdated`/`.joinResult`;
+     also add `applyServerMuteState(muted:deafened:)` (iOS currently ignores server mute/deafen).
+  2. **No Join/Leave Voice button (parity gap).** macOS (`MainWindowController.swift:303,767-797`)
+     and Windows have an explicit "Join Voice"/"Leave Voice" button; iOS has only an icon mic
+     toggle. Add an explicit button. **Screen-audio button deferred** to the ReplayKit Broadcast
+     Upload Extension milestone (PROGRESS.md line below) — needs the extension to actually function.
+  3. **Limited audio input/output options (feature gap).** Current `SettingsView.swift:41-60` uses
+     miniaudio's `vc_list_devices` which on iOS returns ~2 entries ("iPhone mic"/"Default").
+     miniaudio does NOT touch `AVAudioSession` on iOS — it opens the current default route via
+     CoreAudio (AudioUnit/AudioQueue) and that's it. All iOS audio routing (input port selection,
+     mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture) must be
+     driven manually from Swift via `AVAudioSession` *before* the core opens its device. The
+     current `AudioSessionManager.swift:13-15` uses `.voiceChat` + `.allowBluetooth`, which forces
+     HFP (mono 8/16 kHz + heavy output processing) whenever a Bluetooth headset is connected —
+     this is why stereo output is degraded with BT headsets.
+
+  **Plan (decisions locked with user 2026-06-19):**
+  - **Part A — Channel-id sync (bug fix, smallest, unblocks mic button):** `SessionState` gains
+    `syncSelfChannel()` (finds self in `users`, sets `currentChannelId = user.channelId`); called
+    from `init`, `.channelList`, `.userJoined`, `.userLeft`, `.userUpdated`, `.joinResult`. Add
+    `applyServerMuteState(muted:deafened:)` (mirror macOS lines 693-700); call from `.userUpdated`.
+  - **Part B — Join/Leave Voice button:** Replace the icon-only mic toggle in `VoiceControlsView`
+    with an explicit "Join Voice"/"Leave Voice" button (mirror macOS `micToggleButton`). Keep PTT
+    path, mute/deafen, disconnect, level meter. No screen-audio button (deferred).
+  - **Part C — iOS audio routing layer (Swift, new `IOSAudioRouter.swift`):** Drive selection via
+    `AVAudioSession` before the core opens its device. Input enumeration via
+    `availableInputs` → ports (builtInMic/bluetoothHFP/headsetMic/usbAudio/airPlay); for builtInMic
+    walk `port.dataSources` → expose **orientation** (front/back/top/bottom) + **polar patterns**
+    (`supportedPolarPatterns`: omni/cardioid/subcardioid/bidirectional). Apply via
+    `setPreferredInput`/`setPreferredDataSource`/`setPreferredPolarPattern`/
+    `setPreferredInputNumberOfChannels(2)` for stereo. Bluetooth mode as category options: "BT HFP
+    voice" (`.allowBluetooth`, mono voice, BT mic) / "Built-in Mic + BT A2DP stereo"
+    (`.allowBluetoothA2DP` only — stereo output, built-in mic, **no HFP processing**) / "Built-in
+    Mic + Speaker" (neither). Mic processing mode: "Standard" (`.voiceChat`, AEC/AGC/HPF on) /
+    "Raw / Studio" (`.measurement`, all processing off — **allowed always with a warning** when
+    output route is the speaker, echo risk since no AEC). Output: read-only `currentRoute.outputs`
+    display + AirPlay via `AVRoutePickerView`. Persist choices in `UserDefaults`; re-apply on route
+    changes. Note: Voice Isolation / Wide Spectrum (iOS 17+/18+) are user-toggleable in Control
+    Center for `.voiceChat` apps — surface as a hint, not a programmatic toggle.
+  - **Part D — Core stereo-mic capture (C ABI change, append-only):** `AudioParams.capture_channels`
+    is hardcoded to 1 (`audio_engine.h:85`); encode path already stereo-capable (proven by WASAPI
+    loopback stereo work, 2026-06-17 entry below). New C ABI: `vc_result
+    vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels);` (1 or 2) — new
+    setter, not a struct change, keeps `vc_stream_desc` stable. `client.{h,cpp}`: `set_capture_channels`
+    → per-stream, flows into `AudioParams.capture_channels` before `AudioEngine::start` for MIC kind.
+    `client.cpp` `on_capture_frame`: mic path with `channels==2` reuses the existing stereo encode
+    branch. `audio_engine.cpp`: capture device already opens with `p.capture_channels` — just needs
+    the value to propagate. Swift `VoiceCatCore`: add `setCaptureChannels(streamId:channels:)`;
+    `IOSAudioRouter` calls it when user picks stereo built-in mic.
+  - **Part E — Settings UI rework:** Audio Input section (input port picker → if builtInMic, show
+    orientation + polar pattern sub-pickers + mic mode Standard/Raw + channels Mono/Stereo); Audio
+    Output section (bluetooth mode HFP/A2DP/Off + current route read-only + AirPlay button); Voice
+    section (existing: input mode, VAD threshold, PTT). Replace the current miniaudio-based device
+    picker (`SettingsView.swift:41-60`) with the AVAudioSession-derived tree.
+  - **Part F — Docs + deployment target:** Raise iOS deployment target to **18.0** (user decision —
+    unlocks newest audio APIs; `project.pbxproj` + `Package.swift` `.iOS(.v18)`). Update
+    `docs/tech-stack.md` §2 (iOS audio routing via AVAudioSession, stereo mic, deployment 18.0),
+    `docs/voice.md` (new iOS mic capture subsection), `docs/roadmap.md` (iOS pending list),
+    `docs/architecture.md` §4 (Swift binding notes — iOS audio routing in Swift layer, core
+    stereo-mic via new ABI setter), `PROGRESS.md` (this entry + next-action pointer).
+  - **Implementation order:** A (channel-id sync, testable immediately) → B (Join/Leave Voice) →
+    D (core stereo-mic + ABI, verify with `ctest --preset dev`) → C (Swift routing layer, depends
+    on D) → E (Settings UI, depends on C) → F (docs, same commit as work).
+  - **Verification:** `xcodebuild -target VoiceCatiOS -sdk iphonesimulator -configuration Debug
+    build` → BUILD SUCCEEDED; `swift test` green + stereo-mic capture round-trip test; `ctest
+    --preset dev` green (core changed in D); manual (simulator/device): join channel → Join Voice
+    works → input picker shows built-in mic sub-options (orientation/polar pattern) → toggle Raw
+    mode (warning appears on speaker) → toggle Bluetooth A2DP stereo output → toggle stereo capture.
+  - **Files to touch:**
+    - iOS Swift: `SessionState.swift`, `AudioSessionManager.swift` (+ new `IOSAudioRouter.swift`),
+      `Views/VoiceControlsView.swift`, `Views/SettingsView.swift`, `VoiceCatiOSApp.swift`,
+      `VoiceCatiOS.xcodeproj/project.pbxproj` (iOS 18.0).
+    - Swift core: `Sources/VoiceCatCore/VoiceCatClient.swift` (`setCaptureChannels`),
+      `Sources/VoiceCatCore/Models.swift` (input-port model), `Package.swift` (`.iOS(.v18)`).
+    - Core C++: `core/include/voicecat.h` (`vc_set_capture_channels`), `core/src/voicecat.cpp`,
+      `core/src/core/client.{h,cpp}`, `core/src/audio/audio_engine.{h,cpp}`.
+    - Docs: `docs/tech-stack.md`, `docs/voice.md`, `docs/roadmap.md`, `docs/architecture.md`,
+      `PROGRESS.md`.
+    - Tests: `clients/apple/Tests/VoiceCatCoreTests/`, `tests/` (stereo-mic encode test).
+  - **Why miniaudio can't do this alone:** On iOS, miniaudio uses CoreAudio (AudioUnit/AudioQueue)
+    to open the *current default route* for PCM I/O — it never touches `AVAudioSession`.
+    `ma_context_get_devices` on iOS returns a near-empty list (iOS doesn't expose a real CoreAudio
+    device list like macOS does). All iOS audio routing (input ports, data sources, polar patterns,
+    HFP/A2DP, measurement mode, `preferredInputNumberOfChannels`) must be driven from Swift via
+    `AVAudioSession` *before* miniaudio opens its device. miniaudio just reads whatever route
+    AVAudioSession has established.
+
 - **Done:** **iOS SwiftUI client — `VoiceCatiOS`** (2026-06-19). Full SwiftUI app at
   `clients/apple/iOS/VoiceCatiOS.xcodeproj`. Mirrors the macOS AppKit and Windows WinForms
   clients feature-for-feature: saved server list (JSON + Keychain passwords via App Group