voice-cat/PROGRESS.md

# PROGRESS — VoiceCat

Living status. **Update this file in the same commit as your work** so the next agent picks
up instantly. Newest status at the top.

- **Date convention:** ISO (YYYY-MM-DD).
- Statuses: `[ ]` not started · `[~]` in progress · `[x]` done.

---

## ▶ Where we left off / next action

- **Done (2026-06-25):** **iOS robustness — auto-reconnect after a network change + audio
  recovery when audio devices plug/unplug.** Two layers of bugs the iOS client had:
  (a) a `VC_EVENT_DISCONNECTED` from the C core on a Wi-Fi→cellular flip / DNS outage /
  server restart used to leave the session dead with no retry; (b) unplugging wired
  headphones or AirPods left the engine stopped forever — mic stopped transmitting and
  remote audio stayed silent (the server connection itself survived, but the audio graph
  did not recover).

  The first attempt wired reconnect into `AppState.handleConnectEvent`, but that handler
  never runs for a live-session disconnect: once `SessionState.init` overwrites
  `client.onEvent` (`SessionState.swift:87`), the `.disconnected` event is delivered to
  `SessionState.handleEvent`, which used to play a cue and do nothing else. So the live
  session would sit as a zombie for ~30-60 s (the C core's TCP keepalive/reaper timeout)
  and then play the "connection lost" sound with no reconnect armed — exactly what the
  user saw. The fix below has two parts addressing both the missing reconnect AND the
  long wait.

  1. **Event-driven reconnect** (`AppState.swift`, `SessionState.swift`): added a
     `weak var appState: AppState?` to `SessionState`, set by AppState on auth success.
     `SessionState.handleEvent` `.disconnected` now plays the cue and calls
     `appState?.onLiveSessionDisconnected()` — the SINGLE path by which AppState learns a
     live session dropped (since its own `handleConnectEvent` is bypassed for live-session
     events). `onLiveSessionDisconnected` calls a shared `teardownLiveSessionAndReconnect`
     that snapshots the live session into `LastSession`, stops the audio engine,
     deactivates the AVAudioSession, nil's `session` (which releases `VoiceCatClient` →
     `vc_client_destroy` joins the io thread), resets the backoff counter, and arms
     `scheduleReconnect`.
  2. **Path-driven proactive reconnect** (`AppState.swift`): an `NWPathMonitor`
     (`Network.framework`) now runs the whole time we're CONNECTED (started on auth
     success, not only when armed for reconnect) and stays armed across reconnects. Its
     `pathUpdateHandler` (dispatched to @MainActor) does two things:
       - While connected: a primary-interface change (Wi-Fi↔cellular) OR the path becoming
         `.unsatisfied` triggers `proactiveReconnect()` — tearing the live session down
         BEFORE the C core notices the dead TCP read. This is what collapses the 30-60 s
         reaper wait into ~1 s + the first backoff tick. Same-interface refreshes (Wi-Fi
         BSSID roams, signal-strength changes) are intentionally ignored (signature
         comparison via `pathSignature`); those usually don't break the TCP connection.
       - While mid-reconnect (no session): a path becoming `.satisfied` resets the backoff
         counter and arms `scheduleReconnect` for a fast-fresh retry.
     `userInitiatedDisconnect` distinguishes manual `disconnect()`/`cancelConnect()` (which
     set it true → cancel all reconnect state) from a network drop (which leaves it false).
     On a successful reconnect, `reconnectAttempt` resets and `lastSession` clears; the
     path monitor keeps watching for the next change. On user-initiated disconnect, all
     reconnect state (task + path monitor + `lastSession` + `connectedServer`) is cancelled.
  3. **Backoff + restore**: exponential backoff 1s → 2s → 4s → 8s → 16s → 30s cap,
     indefinite. TOFU pins match on the second connect (`VC_TOFU_MATCHED`) so the identity
     gate auto-confirms; on auth success `SessionState.requestRestore` issues a
     `joinChannel` and re-arms voice + restores the local mute/deafen state on the
     resulting `.joinResult`.
  4. **Audio recovery** (`AudioSessionManager.swift`, `IOSVoiceProcessingEngine.swift`):
     replaced the route-change handler's narrow `.oldDeviceUnavailable`/
     `.newDeviceAvailable` guard with a single intent-gated `recoverAudio()` path that
     re-activates the AVAudioSession, re-applies the route config, and rebuilds the
     engine; it runs on every externally-initiated route change reason except
     `.categoryChange`/`.routeConfigurationChange` (those we cause ourselves and would
     loop). Interruption-end now always calls `recoverAudio()` instead of only when
     `.shouldResume` is set (which left the session permanently dead after Siri). Added
     an `AVAudioEngineConfigurationChange` observer on the engine in `IOSAudioEngine`
     that catches the case where iOS stops the engine itself AFTER our route-change
     handler already rebuilt it (the previous rebuilds raced the engine's own self-stop
     and lost). And `IOSAudioEngine.rebuild()` now does a one-shot reactivation-retry on
     `engine.start()` failure — iOS sometimes refuses to start until the AVAudioSession is
     re-activated, which is the silent-death case.
  **Build:** `scripts/build-ios-client.sh --no-configure` green (Xcode 26.5 / iOS 18.0 sim
  SDK, Swift 5 mode). No C ABI / `voicecat.h` / `voicecat.proto` / C core changes; the
  existing TOFU auto-confirm (`VC_TOFU_MATCHED`) and idempotent `vc_join_channel` make
  reconnect+restore possible without new C ABI. macOS and Windows clients unchanged.
  **Next (manual, on-device):** verify unplugging AirPods/wired headphones mid-call keeps
  audio going through the loudspeaker; verify Wi-Fi→cellular flip mid-call now triggers a
  FAST reconnect (within a couple seconds, not 30-60 s) and lands in the same channel with
  voice re-armed; verify tapping Disconnect mid-reconnect-abort cancels cleanly.

- **Done (2026-06-24):** **Three bug fixes — voice join/leave, channel edit defaults, channel-update stream restart.**
  1. **Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.** Previously
     "Join Voice" only started the local mic — receiving was always on (gated by channel
     membership alone). Added a protocol-level voice subscription concept: new
     `SubscribeVoiceRequest`/`UnsubscribeVoiceRequest`/`VoiceSubscriptionResult` proto messages
     (`core/proto/voicecat.proto`), `User.voice_subscribed` field, `vc_join_voice`/`vc_leave_voice`
     C ABI functions (`core/include/voicecat.h`), `VC_EVENT_VOICE_STATE` event, server-side
     `voice_subscribed_` flag on `ConnSession` checked by the SFU relay's recipient filter
     (`SessionRegistry::find_channel_sessions` excludes non-subscribers; `MediaRelay::on_udp_frame`
     also skips non-subscribed senders). The core client gates `sync_remote_streams` on
     `voice_subscribed_`, tears down all remote decoders + stops local streams on leave, and
     re-syncs from the session model on join. All three clients (Windows/macOS/iOS) rewired
     their Join/Leave Voice button to call `joinVoice`+start mic / `leaveVoice`+core stops mic.
     The configured input mode (PTT/VAD/AlwaysOn) takes effect on join — no extra mic button.
     Text chat works regardless of voice subscription. **Apple clients not yet compile-verified
     (Windows environment).**
  2. **Channel edit dialog now shows the channel's actual current settings.** The read struct
     `vc_channel` (`voicecat.h`) was missing `sort_order` and `audio` fields — only the write
     struct `vc_channel_info` had them. Extended `vc_channel` with both (additive, no ABI break),
     updated the session model (`session::Channel`) and `apply_snapshot`/`apply_channel_event`
     to populate them, and updated `vc_list_channels` marshaling. All three clients now build
     the edit descriptor from the actual channel info instead of hardcoded defaults.
  3. **Channel parameter updates now automatically restart everyone's streams.** Previously
     editing a channel's audio config (codec/bitrate/sample-rate/FEC/DTX/etc.) persisted and
     broadcast a `ChannelEvent::UPDATED`, but no layer restarted streams — encoders/decoders
     are frozen at announce time. `handle_channel_event` (`core/src/core/client.cpp`) now
     detects audio-config changes on the user's current channel and calls
     `restart_active_streams_for_channel`, which stop→starts each active local stream. The
     server reads the updated channel config on re-announce, and peers' `sync_remote_streams`
     wire up fresh decoders at the new ssrc. The `LocalStream` struct now retains the stream
     label across restarts. No server or protocol change needed.

- **[ ] Soon — jitter buffer should measure REAL arrival jitter (RFC 3550), not sender
  timestamps.** `JitterBuffer::push` (`core/src/audio/audio_engine.cpp:84-108`) estimates
  jitter from `gap = ts - last_push_ts_`, where `ts` is the **sender's timestamp** — which is
  perfectly regular (`ls.timestamp += samples` every frame, independent of when the packet is
  actually sent). So `diff` is always ~0, `jitter_est_` stays 0, and `target_depth_ms_` is
  pinned at its ~20 ms floor. The buffer is therefore **blind to real network/arrival jitter
  and to bursty senders** — it never deepens. Combined with the playout deliberately seeding
  to near-zero depth (`on_playback`, ~line 715), the receiver tolerates only a *steady*
  sender. This is exactly why the iOS mic needed a send-side pacing cushion (below) and why
  genuine network jitter would also cause underruns. **Fix:** measure inter-arrival jitter
  the RFC 3550 way — `D = (arrival_j - arrival_i) - (ts_j - ts_i)` using a wall-clock arrival
  stamp captured in `push()` — and drive `target_depth_ms_` off that EWMA (keep the existing
  marker/silence-gap outlier rejection). Then the receiver absorbs bursts itself and the iOS
  send cushion could be reduced or removed. Shared-core change → add a test and re-verify
  desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).

- **Done (2026-06-24):** **Windows PTT can now work system-wide (in the background).** Previously
  the PTT key was focus-scoped (WinForms `KeyDown`/`KeyUp`, dead the moment the window lost
  focus). Added an AV-safe global path using the **Raw Input API** (`RegisterRawInputDevices` +
  `WM_INPUT` with `RIDEV_INPUTSINK`) — *not* a `WH_KEYBOARD_LL` low-level hook, which is the
  keylogger pattern AV heuristics flag (worse for our unsigned MinGW binary). New
  `clients/windows/VoiceCat.App/Native/RawInput.cs` (P/Invoke + structs); `MainForm` overrides
  `OnHandleCreated`/`OnHandleDestroyed`/`WndProc` to register the keyboard sink and handle
  `WM_INPUT`, gates the focus-scoped `KeyDown`/`KeyUp` handlers off when system-wide is on, makes
  the `Deactivate` force-release conditional, and adds a `GetAsyncKeyState` watchdog on the pump
  timer so a missed key-up (RDP/lock-screen focus switch) can't leave PTT stuck. New
  `VoiceSettings.SystemWidePtt` (default ON) with a "Works in the background (system-wide)"
  checkbox in the Audio settings PTT section. Build green (`dotnet build`, 0 warnings). **Next
  (manual):** verify background PTT against a live server, and confirm the binary trips no AV
  keyboard-hook detection.

- **Done (2026-06-23):** **Fixed: receive-side noise reduction silently skipped on stereo mic
  streams (regression from stereo-mic capture below).** The per-listener NR toggle
  (`vc_set_remote_stream(... noise_reduction)`) did nothing on Windows/macOS/iOS — the UI and
  the whole C-ABI→core path were correctly wired, but the decode loop gated the RNNoise pass on
  `dec_channels == 1` (`core/src/audio/audio_engine.cpp`), an old proxy for "this stream is
  voice" that assumed *stereo ⇒ screen-share*. The stereo-mic commit broke it: a stereo mic with
  **send-side NR off** transmits stereo Opus, so the receiver decoded `dec_channels == 2` and
  skipped NR entirely (gain/mute have no channel guard, which is why only NR looked broken).
  **Fix:** thread the stream *kind* through `init_recv_stream` into `RemoteStream::is_voice`
  (set from `si.kind() == STREAM_MIC` in `client.cpp`), gate receive NR on `is_voice` instead of
  channel count, and fold a stereo voice frame to mono → denoise → duplicate back across both
  channels in place (symmetric with the send-side downmix; RNNoise is mono-only). A stereo voice
  stream now plays mono while NR is on; a screen-audio share is never touched. New test
  `tests/test_recv_noise_reduction.cpp` drives `AudioEngine` and asserts a stereo voice stream's
  noise floor collapses with NR on (RMS 1046 → 0.1) while a screen-audio share stays unchanged
  (RMS ≈ 1015). Full `ctest --preset dev` green — **29/29**. Docs: voice.md §10. Clients need no
  change (shared-core fix). Not yet re-verified two-client E2E on real hardware.

- **Done (2026-06-23):** **Stereo mic capture on Windows & macOS desktop clients.** Both
  desktop mics were hard-mono: `ensure_audio_running()` defaults `capture_channels = 1` and
  neither client ever called `vc_set_capture_channels` (only iOS did). Added a **"Stereo
  microphone" toggle** to each client's Audio settings (off by default, persisted —
  `VoiceSettings.StereoMic` on Windows, `MainWindowController.stereoMic` /
  `voice.stereoMic` UserDefaults on macOS). It's applied to the core when the mic stream
  starts (stored on the stream before the announce round-trip, so the first device open picks
  it up) and live in settings via `vc_set_capture_channels` + `vc_audio_restart`. Exposed both
  ABI calls in the Windows interop (`NativeMethods`/`VoiceCatClient`); the macOS wrapper already
  had them. **Core fix:** `encode_and_send_frame` (`core/src/core/client.cpp`) now folds a
  stereo mic frame to mono when the channel is mono — previously the `channels == 2` branch
  encoded interleaved L/R directly even on a mono channel, feeding a mono `opus_encode` 2× its
  samples (wrong pitch / garbage). Real stereo still only reaches the wire on a **stereo
  channel** (encoder channel count = channel's Opus mode); on a mono channel the mic is cleanly
  downmixed. Test: `test_stereo_mic_mono_channel` in `tests/test_vad_ptt_devices.cpp`. Full
  `ctest --preset dev` green — 28/28. macOS Xcode build not compiled here (Windows host); the
  Swift changes follow existing `nrChanged`/`setInputDevice` patterns. Docs: voice.md §8.

- **Done (2026-06-23):** **Fixed iOS dual-stream / crackly mic — core opened a second
  (miniaudio) capture device alongside the AVAudioEngine tap.** Symptom: with two clients in
  a channel, the remote end heard the iOS mic **twice** and crackly. With Voice Chat + a BT
  headset, both the BT mic and the internal mic were captured; with Stereo Mic, both a mono
  and a stereo copy of the internal mic were sent simultaneously. Root cause is a timing gap
  in `vc_client::ensure_audio_running()` (`core/src/core/client.cpp`): `external_capture` was
  only set when a MIC stream already existed, but `ensure_audio_running` is also called from
  `sync_remote_streams` (triggered by the post-auth `ServerStateSnapshot`) **before** the user
  joins voice — so with no MIC stream, `external_capture` stayed `false` and
  `AudioEngine::start()` opened a real miniaudio capture device. Later the user joined voice →
  `IOSAudioEngine.startMic` installed the AVAudioEngine input tap → `feedPcm` →
  `inject_capture` → `on_capture_frame`. The miniaudio device was still open (the engine was
  already `running()`, so the later `ensure_audio_running` early-returned and never applied
  `external_feed`), and `on_capture_frame` encodes+sends every frame with **no deduplication**
  → the mic was sent twice. The two unsynchronized capture clocks interleaving in the encoder
  is the crackle; the mono miniaudio device + stereo AVAudioEngine tap is the "mono and stereo
  at the same time" on Stereo Mic.
  - **Fix 1 (core, `core/src/core/client.cpp:ensure_audio_running`):** force
    `p.external_capture = true` whenever `external_playback_` is set. In iOS unified mode the
    core must never open a hardware capture device — the AVAudioEngine owns the only mic path.
    No-op on desktop (`external_playback_` is never set there).
  - **Fix 2 (iOS, `clients/apple/iOS/VoiceCatiOS/AppState.swift`):** move
    `client.setExternalPlayback(true)` from the `authResult` handler to **before**
    `client.connect(...)`. The server sends `AuthResult` immediately followed by
    `ServerStateSnapshot`; `handle_server_state` runs `ensure_audio_running` on the io thread
    before the main thread drains `authResult`, so setting the flag post-auth raced. Setting it
    pre-connect guarantees `external_playback_` is true before any message is processed —
    eliminating the playback-device race too (the mixer timer + AVAudioEngine playback path +
    VPIO AEC reference are correct from the first frame).
  - **Verify:** `cmake --build --preset dev` clean; `ctest --preset dev` = 24/28 — the 4
    failures (`vad_ptt_devices`, `external_pcm`, `frame_ms_reframe`, `channel_samplerate`) are
    a **pre-existing** teardown `mutex lock failed` race, reproduced identically with the
    changes stashed. `external_playback` (the one test exercising this code path) **passes**.
    No xcframework rebuild needed (no new symbols). **Next (manual, on device):** two clients
    in a channel — Voice Chat + BT, and Stereo Mic — confirm the remote end hears the iOS mic
    once, clean (no duplicate, no crackle); confirm the iOS user hears the remote user cleanly
    with AEC working in Voice Chat.

- **Done (2026-06-23):** **Fixed iOS mic flutter / crackle / octave-up.** The iOS mic was
  unusable: a consistent ~40–60 ms flutter with volume fade ("talking through a slow fan") on
  every preset. Root cause: the core sends each captured frame **synchronously**
  (`on_capture_frame` → `encode_and_send_frame`, no send pacer), so packet cadence == capture
  cadence; and the receiver's playout keeps **near-zero buffering** by design and its jitter
  estimate is blind to arrival timing (see RFC-3550 item above). That's smooth only for a
  *steady* sender (desktop miniaudio = steady 20 ms), but the iOS `AVAudioEngine` input tap
  delivers ~2 frames per ~40 ms callback (more under VPIO) → bursty → receiver underruns → PLC
  fade.
  - **Fix (iOS-only, `clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift`):** the
    mic tap converts to 48 kHz int16 and writes a lock-free SPSC ring; a 20 ms feed pump
    drains it and calls `feedPcm` at a **steady** cadence so packets leave the core every
    20 ms (what the receiver expects). The pump **primes a small prebuffer cushion**
    (`PumpState.targetFrames`, 3 frames ≈ 60 ms, self-healing up to ~120 ms on underrun)
    before releasing, so the tap's bursts can't drain it to empty. Two correctness rules
    (each had bit us): never read a partial frame (`read` consumes what it returns →
    discarding partials caused crackle), and rebuild the pump with the current channel count
    every `rebuild()` (a frozen channel count fed mono-as-stereo = octave-up on a Stereo→Voice
    Chat switch). Trade-off: ~60–120 ms added mic-send latency — unavoidable when de-bursting
    for a near-zero-buffer receiver; the RFC-3550 fix above would let us shrink it.
  - **Verify:** `xcodebuild` Debug **BUILD SUCCEEDED** (iOS Simulator, arm64). Audible test
    requires a real device (simulator has no real mic route): mic should be smooth on Voice
    Chat / Mono Mic / Stereo Mic, including switching presets while live (no octave).

- **Done (2026-06-23):** **Fixed Apple client link failure (stale xcframework missing
  RNNoise).** Both `VoiceCatMac` and `VoiceCatiOS` failed to link with `Undefined symbols for
  architecture arm64: _rnnoise_create / _rnnoise_destroy / _rnnoise_process_frame`. Root
  cause: `clients/apple/scripts/build-xcframework.sh` merged vcpkg deps into the fat static
  lib but NOT the locally-built vendored `librnnoise.a` (a CMake target from
  `third_party/rnnoise/`, linked privately into `voicecat` via `VOICECAT_HAS_NS` — not a
  vcpkg dep). The xcframework had been rebuilt at 14:17 after the RNNoise commit but still
  omitted the symbols, so every slice's `libvoicecat-fat.a` referenced `_rnnoise_*` with no
  defining object. The iOS slices were also stale (pre-rnnoise) and absent from the
  xcframework entirely.
  - **Fix:** `build-xcframework.sh` now collects `.a` files from `build/<preset>/lib/`
    (excluding `libvoicecat*`) in addition to `vcpkg_installed/<triplet>/lib/`, so vendored
    CMake-target static libs like `librnnoise.a` are merged into the fat lib. Future-proof:
    any new vendored static-lib target landing in `build/<preset>/lib/` is picked up
    automatically. README "Fat static library" section updated.
  - **Verify:** rebuilt `VoiceCatCore.xcframework --all` → all 3 slices (macos-arm64,
    ios-arm64, ios-arm64-simulator) now carry 10 `_rnnoise_*` symbols each; fat lib
    ~30 MB → ~33 MB. `xcodebuild` Debug **BUILD SUCCEEDED** for `VoiceCatMac`,
    `VoiceCatiOS` (iphonesimulator arm64), and `VoiceCatiOS` (iphoneos arm64,
    `CODE_SIGNING_ALLOWED=NO`). No core/ABI/proto changes — xcframework artifact only.

- **Done (2026-06-23):** **Remote-stream noise suppression — real backend (RNNoise).** The
  two-sided NR plumbing (`RemoteStream::recv_ns` + `vc_set_remote_stream(... noise_reduction)`)
  was wired but **inert** — `ApmProcessor::create()` returned a no-op passthrough, because the
  originally-planned `webrtc-audio-processing` won't build on Windows/macOS. Replaced with
  **RNNoise** (BSD-3 + CC0), vendored at `third_party/rnnoise/` (the vcpkg port is `!windows
  !arm`), built as a standalone C static lib + `VOICECAT_HAS_NS`. One `RnnoiseProcessor`
  (`core/src/audio/apm_processor.cpp`) now backs **both** NR paths:
  - **Receive-side** (per-listener, per-`ssrc`): lit up automatically via the factory; gated to
    mono streams (`audio_engine.cpp` ~L791).
  - **Send-side** (mic, new): `vc_set_input_noise_reduction(client, enable)` ABI +
    `vc_client::mic_ns_`, run before input gain/VAD in `on_capture_frame`. A stereo mic is
    downmixed to mono **only when NR is on**; with NR off a stereo mic keeps full stereo.
  - RNNoise is mono/48 kHz/480-sample; our clock is fixed 48 kHz and Opus frame sizes are all
    multiples of 480, so no resampling. RT-safe: alloc at construction, lock-free in the callback.
  - **Verify status:** `ctest --preset dev` green — **28/28** (new `noise_suppression` test:
    feeds white noise through `ApmProcessor::create()`, measures **99.9%** RMS reduction). Build
    clean on the `dev` MinGW preset. **Next (manual):** add the on/off toggles to the client UIs
    (Windows Audio Settings dialog, macOS/iOS settings) calling the two ABIs; build `windows-client`
    + `apple-dev` presets to confirm RNNoise compiles under MinGW-DLL and arm64; two-client E2E.

- **Done (2026-06-23):** **Aux outgoing stream (mic + a second input device) — Windows + macOS.**
  Users can now transmit a second hardware input device (e.g. line-in / aux) alongside the mic, with
  its own device picker and volume, from Audio Settings. **No core/ABI/proto changes** — the aux is
  a `VC_STREAM_AUX_DEVICE` stream started with `external_feed=1`, captured client-side, and fed via
  `vc_stream_feed_pcm` (the same external-feed pipeline screen-audio uses). Per-kind `local_streams_`
  already allows mic + screen + one aux to coexist; volume is a client-side gain multiply (the core's
  `vc_set_input_gain` is mic-only/global). The aux is always-on (the core never gates `AUX_DEVICE` on
  VAD/PTT) and is tied to the voice session (started on Join Voice when enabled, stopped on Leave).
  - **Windows:** new `Audio/InputDeviceCapture.cs` (WASAPI shared-mode capture from a real input
    endpoint via `IMMDevice.Activate(IAudioClient)`, 48 kHz/s16, 20 ms frames) + `InputDeviceEnumerator`
    (WASAPI capture-endpoint list — separate from the core's miniaudio ids). Aux section in
    `AudioSettingsForm.cs` (enable checkbox, device combo, refresh, volume slider, accessible names,
    live-apply + Cancel revert via callbacks). Lifecycle in `MainForm.cs` (`_auxStreamId` +
    `InputDeviceCapture`). Persisted in `VoiceSettings.cs` (`AuxEnabled/AuxDeviceId/AuxGain`).
  - **macOS:** new `Audio/InputDeviceCapture.swift` (AVAudioEngine input-node tap pinned to the chosen
    Core Audio device via `kAudioOutputUnitProperty_CurrentDevice`; AVAudioConverter → 48 kHz int16;
    20 ms framing modelled on `ScreenAudioCapture`) + `InputDeviceEnumerator` (Core Audio device list
    by stable UID). Aux section in `SettingsWindowController.swift`; lifecycle + UserDefaults
    persistence (`voice.aux*`) in `MainWindowController.swift`. New file added to `project.pbxproj`.
  - **Verify status:** Windows C# solution builds clean (0 warn/0 err); `ctest` core suite unchanged
    (no core edits). **Next (manual):** on a Mac, build `VoiceCatMac.xcodeproj`; then two-client E2E —
    enable aux on a second input device, confirm two distinct streams for the sender and that the aux
    volume slider moves the aux level independently of the mic; confirm persistence across relaunch.

- **Done (2026-06-23):** **Input-settings persistence, mic input gain, + two iOS bugs (all 3
  clients).** Four fixes:
  1. **Input settings now persist.** Transmission mode (VAD/PTT/Always-On), VAD threshold, and the
     new mic gain were applied to the core + UI but never saved, so every relaunch reset to VAD
     defaults. Each client now persists them and re-applies on connect: iOS via `UserDefaults`
     (`SessionState.loadAndApplyVoiceSettings` + setter writes, keys `voice.*`); macOS via
     `UserDefaults` (`MainWindowController` `didSet` + `loadPersistedAudioSettings`, also restores
     the VAD slider from the stored threshold); Windows via new
     `VoiceCat.App/Models/VoiceSettings.cs` (JSON at `%AppData%\VoiceCat\voice.json`, mirrors
     `FeedbackSettings`) loaded/applied in `MainForm`.
  2. **Microphone input gain.** New global send-side API `vc_set_input_gain` (voicecat.h →
     `client.cpp::on_capture_frame`, applied to MIC PCM before the VAD gate, clamped to int16) plus
     Swift (`setInputGain`) and C# (`SetInputGain`) bindings. Mic-volume slider (0–300 %, default
     100 %) added to all three clients' input settings, persisted with the rest.
  3. **iOS chat send fixed.** `ChatView` called `sendText(scope:.channel)` with no `targetId` (→ 0),
     so channel messages went nowhere; now passes `session.currentChannelId`.
  4. **iOS per-user tuning reachable via VoiceOver.** The tuning sheet was long-press
     `.contextMenu` only (invisible to VoiceOver); `UserRow` now also exposes the same buttons as
     `.accessibilityActions` (no visual change), so the actions rotor reaches tuning + admin actions.
  - **Verified:** core `cmake --build --preset dev` clean; `ctest --preset dev` = 24/27 (the 3
    failures — `external_pcm`, `frame_ms_reframe`, `channel_samplerate` — are a pre-existing
    teardown crash on this machine, reproduced identically with the changes stashed). xcframework
    rebuilt (`--all`); **VoiceCatMac** and **VoiceCatiOS** (arm64 sim) → BUILD SUCCEEDED;
    `VoiceCat.Interop` (`dotnet build`) succeeded. **Windows App not built** (WinForms
    net10.0-windows can't build on macOS) — changes follow existing patterns; needs a Windows
    build + manual check.
  - **Next (manual):** on each client, set PTT + non-default VAD/mic-gain, relaunch → settings
    restored; boost a quiet mic and confirm others hear it louder; iOS send a channel message;
    iOS VoiceOver → focus a user → actions rotor opens tuning.

- **Done (2026-06-22):** **Fixed growing voice latency (jitter-buffer depth ratchet).** Symptom:
  end-to-end latency grew to multiple seconds and "drifted backward," reset only by leaving/
  rejoining voice (DTX/FEC/DRED on, 10% loss). Root cause was **not** the codec settings (10% loss
  is just an `OPUS_SET_PACKET_LOSS_PERC` encoder hint; FEC/DRED add no standing latency) but the
  receiver playout logic in `core/src/audio/audio_engine.cpp`: the playout clock free-ran in real
  time while the sender omitted silence from its timestamps and set **no header flags at all**, and
  the only correction snapped the clock to the *oldest* buffered frame (could only *add* latency) —
  with `target_depth_ms_` computed but never enforced, so latency could only grow or be reset.
  **Fix:** bounded-depth playout — (re)seed to the *leading edge* (newest frame) on start/marker/
  starve, and **frame-skip catch-up** that trims a backlog beyond `target + hysteresis` (the missing
  downward force). Plus hardening: adaptive late-drop window, talkspurt `kFlagMarker`/`kFlagDtx`
  now actually stamped by the sender (`client.cpp` send path) and consumed on recv, EWMA outlier
  rejection (silence gaps/stragglers no longer poison the estimate), duplicate counting, ring-
  underrun diagnostics (`stream_underruns`/`stream_duplicates`). New regression test
  `tests/test_jitter_depth.cpp` asserts depth stays bounded (<200 ms) while arrivals outrun playout
  for ~4 s. `ctest --preset dev` green — **27/27**. Docs: `docs/voice.md` §5 rewritten.
  - **Next (manual E2E):** two clients in a channel, DTX/FEC/DRED on — talk in alternating bursts
    for several minutes and confirm latency stays low/stable (no backward drift, no rejoin needed).

- **Windows done / Apple awaiting Mac build (2026-06-22):** **Event sound effects + optional
  text-to-speech for all clients.** Clients now play a cue per session event and can optionally
  speak it (TTS off by default; when on it announces joins/leaves and reads message/PM bodies).
  One canonical event→sound mapping (defined off the shared C ABI `vc_event` stream) is mirrored
  across all three clients; `self` vs others is `user_id == self_user_id`, and outgoing messages
  echo back as events so sent/recv cues need no separate send-path hook. Conservative defaults
  (join/leave, channel/PM sent+recv, login, logout, connection-lost, mic on/off ON; per-utterance
  self voice-activity `va_start/va_stop` and the PTT cue OFF). WAVs ship from `assets/sounds/`.
  - **Windows (built + verified):** new `VoiceCat.App/Notifications/` (`FeedbackSettings` →
    `%AppData%\VoiceCat\feedback.json`, `SoundPlayerPool` via `System.Media.SoundPlayer`,
    `SpeechAnnouncer` via the **Prismatoid** NuGet 0.3.0, `EventFeedback` dispatcher); hooks in
    `Forms/MainForm.cs`; `Forms/NotificationSettingsForm.cs` under a new **Settings ▸ Notifications**
    menu. `.csproj` adds the Prismatoid PackageRef and copies the WAVs into `sounds\`. `dotnet build`
    clean; WAVs + `Prismatoid.dll` confirmed in output. Note: `SoundPlayer` has no gain control, so
    volume is honoured as a mute gate (0 = silent) — swap to NAudio if finer/overlap control is needed.
  - **macOS + iOS (written, NOT yet built — needs a Mac):** shared `Sources/VoiceCatCore/Feedback/`
    (`SoundEvent`, `EventFeedback` = `AVAudioPlayer` pool + native `AVSpeechSynthesizer`,
    `FeedbackSettings` over `UserDefaults`); WAVs copied into `Sources/VoiceCatCore/Sounds/` and
    bundled via `Package.swift` `resources: [.process("Sounds")]` (`Bundle.module`). Hooks: iOS
    `SessionState.handleEvent` (+ split `userJoined`/`userLeft`, added a `.disconnected` cue case),
    `AppState` auth-success login cue, PTT cue in `setPushToTalk`; macOS `MainWindowController`
    handlers + NSEvent PTT monitor. Settings UI: iOS `SettingsView` Notifications section
    (`@AppStorage`), macOS `SettingsWindowController` checkboxes + volume slider. No `.pbxproj`
    edits needed (shared files are SPM-managed; app files already in the projects).
  - **Next:** on a Mac, `clients/apple/scripts/build-xcframework.sh --all` then build
    VoiceCatMac/VoiceCatiOS; fix compile fallout. **Watch the iOS audio session:** cues/TTS play over
    the live VPIO `playAndRecord` session — verify they mix and don't duck/interrupt the call or get
    silenced by the mute switch (most likely bug site). Then run `ctest --preset dev` (unchanged —
    no core/server code touched).

- **Done (2026-06-22):** **UDP media now shares the TCP port (self-host port-forward fix).** Symptom: a
  remote self-hosted server (`iamtalon.me:8384`, TCP+UDP 8384 forwarded) accepted TCP connections but
  passed no voice. Root cause: `Config::media_port` defaulted to `0` = OS-assigned, and `main.cpp`'s
  `--port` only set `bind_port` (TCP) — so the UDP relay bound a *random high port*, advertised it to
  clients in HELLO (`udp_port`), and clients sent voice there. With only `8384/udp` forwarded those
  packets were dropped → connect OK, no audio. This contradicted `docs/deployment.md` ("Control and media
  share one port number on TCP+UDP"). **Fix (`server/src/server.cpp`):** media follows bind_port when
  `media_port == 0` — `media_want = cfg_.media_port != 0 ? cfg_.media_port : cfg_.bind_port`. The
  `0 = OS-assigned` escape hatch survives when `bind_port` is also 0, so tests that bind ephemeral ports
  are unaffected (kept the logic in server.cpp rather than hardcoding 8384 as the default, which would
  collide parallel tests on UDP 8384). Banner now reads `TCP :8384  UDP :8384`. Build + `ctest --preset
  dev` green (24/24); live-verified banner with `--port 8390` → `UDP :8390`. **Action for self-hosters:**
  redeploy and confirm the startup banner shows matching TCP/UDP ports; the existing single forward rule
  is now correct. If voice still fails, watch the server's rate-limited `[media] dropped frames —
  unmapped-endpoint=…` line (NAT source-port rewrite would be the next suspect).

- **Done (2026-06-23, Swift-only — no core/ABI change; awaiting on-device verification):** **iOS audio
  stack unified — one always-external `AVAudioEngine`, miniaudio dropped on iOS.** The iOS audio path was
  a fragile hybrid: Voice-Chat-class presets ran a native VPIO `AVAudioEngine` (core external) while
  Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every bug lived in the seam
  (lingering miniaudio capture unit fighting VPIO, the `audioRestart` ordering dance, the route-change
  "glitching" loop, stereo↔mono stickiness, "can't hear anyone"), and switching presets/routes mid-call
  routinely dropped input, output, or both. **Fix: drive *all* iOS audio through one `AVAudioEngine` with
  the core fully external at all times** — `vc_set_external_playback(1)` once at connect, every MIC stream
  `external_feed=1`, mic via `vc_stream_feed_pcm`, playback via `vc_set_mixed_output_sink`.
  - `IOSVoiceProcessingEngine.swift` → **`IOSAudioEngine`** (same file): always-on `AVAudioSourceNode`
    playback (runs whenever connected, so remote audio plays before you join voice); conditional mic tap;
    VPIO + AGC toggled per config. One private `rebuild()` (stop → set VPIO → install tap → start) backs
    `startListening`/`stop`/`startMic`/`stopMic`/`reconfigure`/`setCaptureChannels`. Kept the `PCMRing`
    and ring-stats diagnostics.
  - `IOSAudioRouter`: presets cut from seven to **four** — Voice Chat (VPIO mono, system output),
    Stereo Mic / Mono Mic (internal built-in mic regardless of output, A2DP-capable, no VPIO), Advanced
    (manual). New persisted `voiceProcessingEnabled` (master AEC+NS) + `agcEnabled`; setters now call
    `IOSAudioEngine.reconfigure()` instead of `client.audioRestart()` + `reconcileVoicePath`. Kept the
    proven AVAudioSession recipes (category/mode/options, stereo capsule, `applyA2dpSpeakerFallback`).
  - `AudioSessionManager` slimmed (drops `client`/`activeMicStreamId`/`reconcileVoicePath`; adds
    `isActive`); interruption-end & device-change now `reconfigure()` the engine. `SessionState`
    `doStartMicStream`/`stopMicStream` collapsed to start-stream + `startMic`/`stopMic` (no
    `setExternalPlayback`/`audioRestart` toggling); `reconcileVoicePath` deleted. `AppState` sets external
    playback + `startListening` at connect, `stop()` at disconnect. `SettingsView` → four presets +
    Advanced VPIO/AGC toggles.
  - **No core/ABI/test change** — relies on the already-shipped `vc_set_external_playback` /
    `external_feed` / `vc_set_mixed_output_sink` / `vc_stream_feed_pcm` path (`test_external_pcm`,
    `test_external_playback`). `xcodebuild` iOS device Debug **BUILD SUCCEEDED**. **Rebuild the
    xcframework is NOT required** (no new symbols).
  - **Next (user, on device):** two iPhones in a channel — verify BOTH directions survive every
    transition and are never silent unless intended: Voice Chat (no echo, NR), listen-only before joining,
    join↔leave repeatedly, switch Voice Chat↔Stereo↔Mono↔Advanced *while in voice*, A2DP connect/unplug,
    wired connect/unplug, phone-call interruption + resume, screen-audio share.

- **Superseded by the 2026-06-23 unification above (2026-06-22):** **iOS real echo cancellation / noise
  suppression via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
  AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core uses miniaudio's
  plain RemoteIO units — so `.voiceChat` mode alone never engaged AEC. Fix moves both mic capture and
  playback to a native Swift `AVAudioEngine` (`setVoiceProcessingEnabled`) on the AEC presets, with the
  core in external mode.
  - **Core (done, builds + tests green):** new ABI `vc_set_mixed_output_sink` + `vc_set_external_playback`
    (voicecat.h PATCH→2). `AudioEngine` gains a mixer-timer thread that drives `on_playback` (decode+mix)
    on a ~20 ms cadence with NO hardware playback device and ships the final mix to the mixed-output
    sink; `start()` also skips the hardware capture device when the MIC stream is `external_feed`
    (`AudioParams.external_capture`). New white-box test `test_external_playback` (23/24;
    pre-existing `external_pcm` teardown crash on Darwin 25.5 is UNRELATED — original tree crashes too).
  - **Swift (done, builds):** `VoiceCatCore` wrappers (`externalFeed` on `StreamDescriptor`,
    `setMixedOutputSink`, `setExternalPlayback`); new `IOSVoiceProcessingEngine.swift` (VPIO
    `AVAudioEngine`: mic tap→`feedPcm`, mixed-sink lock-free ring→`AVAudioSourceNode`);
    `IOSAudioRouter.currentConfigUsesVoiceProcessing` gates the path per preset; `SessionState`
    join/leave + `reconcileVoicePath()` switch between VPIO and the miniaudio path; Voice Chat defaults
    to speaker; SettingsView shows AEC/NS state. **Rebuild the xcframework** before building the app:
    `clients/apple/scripts/build-xcframework.sh --all` (new ABI symbols). `xcodebuild` iOS sim Debug
    BUILD SUCCEEDED.
  - **Post-verification fixes (2026-06-22, Swift-only — no core/ABI change):** two on-device bugs fixed.
    - **Voice Chat (VPIO) silent playback:** `SessionState.doStartMicStream()` called `audioRestart()`
      BEFORE `startStream`, so when the engine was already running (a remote stream had started it) it
      reopened with `external_capture=false` and opened a hardware miniaudio capture device; the
      announce-result restart then early-returned (engine already running) so that device was never
      dropped and fought the `AVAudioEngine` VPIO unit, silencing playback. Fix: set
      `setExternalPlayback` first, then `startStream` (which stores `external_feed` synchronously), THEN
      `audioRestart()` — the core reopens in full external mode (no hardware devices). Added VPIO
      diagnostics (graph/route formats at start; ring written/read totals at teardown).
    - **Stereo Mic / Studio quiet earpiece:** the `.builtInMicBtA2dp` presets omit `.defaultToSpeaker`
      (it breaks A2DP) and skip `forceSpeaker`, so with no Bluetooth connected output pinned to the quiet
      receiver. New `IOSAudioRouter.applyA2dpSpeakerFallback()` overrides to the built-in speaker when no
      external (A2DP/wired/AirPlay) output is present, clears the override when one is — called after
      activation and on device-change route changes (`AudioSessionManager`).
  - **Next (user, on device):** two iPhones on speaker, Voice Chat preset → confirm (a) no echo, (b)
    background noise suppressed, (c) speaker output by default AND remote audio is now audible; then
    Stereo Mic / Studio with no BT → confirm loud speaker (not earpiece), and A2DP takes over when a BT
    headset connects. Tune the mixer-timer/ring sizing if there's under/overrun.

- **Done (2026-06-21):** **Docker + Linux deployment + GitHub Actions cross-build.** Added the complete Linux server
  deployment story (the only missing platform — Windows and macOS already have native
  binaries):
  - `Dockerfile` — multi-stage (builder: `ubuntu:24.04` + vcpkg + `cmake --preset
    server-release`; runtime: `ubuntu:24.04`, non-root `voicecat` user, `/data` volume,
    TCP+UDP 8384). vcpkg is fetched via the GitHub archive tarball at the exact
    `builtin-baseline` commit (`d46283cf…`), avoiding a full git-history clone. BuildKit
    cache mounts on `/vcpkg/downloads`, `/vcpkg/buildtrees`, `/vcpkg/packages` (scoped by
    `TARGETARCH`) keep rebuilds fast. Both `voicecat-server` and `voicecat-admin` are
    copied into the runtime image.
  - `docker-compose.yml` — single-service compose file with `restart: unless-stopped`,
    named volume `voicecat-data`, and port mappings for TCP+UDP 8384. `command:` shows
    how to set `--name`.
  - `.dockerignore` — excludes `.git/`, `build/`, `clients/` (Swift/C# code), `docs/`,
    markdown, editor config; build context is just `core/`, `server/`, `tools/`, `cmake/`,
    and the three root CMake/vcpkg files.
  - `deploy/linux/voicecat.service` — hardened systemd unit (non-root, `ProtectSystem`,
    `NoNewPrivileges`, `AmbientCapabilities=CAP_NET_BIND_SERVICE`) for bare-metal deploys.
  - Multi-arch: `docker buildx build --platform linux/amd64,linux/arm64 .` works without
    any triplet override — `cmake/voicecat-toolchain.cmake` auto-detects from the host
    arch cmake sees inside the buildx container.
  - Quick start: `docker compose up -d` (or `docker run -d -p 8384:8384/tcp -p
    8384:8384/udp -v voicecat-data:/data voicecat`). First run auto-generates identity
    + cert + DB; check logs for fingerprint + admin password.
  - **GitHub Actions** (`.github/workflows/build-linux.yml`): primary cross-platform
    binary build path — amd64 uses `ubuntu-24.04`, arm64 uses `ubuntu-24.04-arm`
    (native, not QEMU). Triggers on push to main (when C++/cmake files change) and
    manually via `workflow_dispatch`. Downloads land as 90-day artifacts.
    `scripts/build-linux-binaries.sh` is the local Docker fallback (needs ~10–15 GB
    free disk; suits Linux dev machines, not Windows Docker Desktop).

- **Done (2026-06-21):** **Fix permanent voice-loss bug + harden the UDP media path (protocol v2).**
  Field report: two iOS users lost all audio mid-call after a bad-network blip and could not
  recover even by restarting the apps. Root causes found in the UDP media path:
  1. **Anti-replay window poisoned by unauthenticated packets (the trigger).**
     `SodiumMediaCrypto::open()` advanced `recv_highest_` from the plaintext header `seq`
     *before* verifying the AEAD tag and never rolled it back on failure. One corrupted/forged
     frame (a bit-flip on flaky wifi) shoved the high-water mark far ahead, after which every
     legitimate frame was rejected as "too old" — permanently. Fixed by reordering to
     replay-check → authenticate → update (RFC 3711 §3.3): the window is now touched only after
     a successful tag check. Regression test in `test_media_aead.cpp`
     (`test_corrupted_seq_does_not_poison_window`) — fails on the old code, passes now.
  2. **16-bit seq wrap with no rollover counter.** The wire header carried only the low 16 bits
     of the nonce counter (zero-extended on receive); after 65,536 frames the reconstructed
     nonce diverged and all frames failed auth. **Wire format widened to a full u64 seq**
     (`voice_frame.h`: header 14 → 20 bytes, `seq` u16 → u64; `crypto.cpp`, `client.cpp`,
     `media_relay.cpp` updated; `JitterBuffer::Frame::seq` widened). This is a **versioned wire
     change → `VOICECAT_PROTOCOL_VERSION` 1 → 2**; the `Hello` handshake rejects on mismatch
     (`conn_session.cpp`). The voice frame is parsed only in `core/`+`server/`+`tests/`, so the
     Swift/C# clients need only a rebuild — no parser changes.
  3. **Server leaked UDP state on disconnect.** `SessionRegistry::unregister_session()` now also
     frees `udp_endpoints_`/`udp_tokens_`/`ssrc_to_session_` (scan-and-erase by session id).
  4. **Diagnostics.** `MediaRelay` now emits rate-limited dropped-frame counters
     (unmapped-endpoint / no-recv-crypto / open-failed) so a wedged media path is observable.
  - **Verified:** `cmake --build --preset dev` clean; `ctest --preset dev -E external_pcm`
    **22/22 pass** (incl. `m2_voice` e2e relay + the two new AEAD regressions). `external_pcm`
    still aborts on the **pre-existing** CoreAudio shutdown mutex race (confirmed identical on a
    clean baseline checkout under the same harness — unrelated to these changes). Docs updated:
    `voice.md` §2 (header), `protocol.md` (v2 + negotiation), `security.md` (authenticate-then-advance).

- **Done (2026-06-21):** **Expose all channel codec params + guest nickname in every client.**
  - **DRED everywhere + ABI fix.** `dred` (Opus 1.6 Deep REDundancy) existed in the C ABI
    (`vc_audio_config.dred`) and proto but was absent from *both* client marshaling layers — a
    latent ABI mismatch: Swift `AudioConfig` and the C# `VcAudioConfigNative` blittable struct
    were each one `int` short of the native struct passed to `vc_create_channel`/`vc_edit_channel`.
    Added `dred` through Swift (`Models.swift`, `Marshaling.swift`, `VoiceCatClient.toNative`) and
    C# (`Structs.cs`, `Models.cs`, `Marshaling.cs`, `VoiceCatClient.cs`).
  - **Windows:** added the one missing DRED checkbox to `ChannelEditDialog` (all other params
    were already present).
  - **macOS:** `ChannelEditSheet` now exposes the previously-hidden params — application profile,
    sample rate, expected packet loss, complexity, and DRED (was only stereo/bitrate/frame/FEC/DTX).
  - **iOS:** `ChannelEditView` was name+topic only; rebuilt into a full create **and edit** form
    (General: name/topic/parent/password/max-users/sort-order; Audio: stereo/bitrate/sample-rate/
    frame/application/packet-loss/complexity/FEC/DTX/DRED). Added `SessionState.editChannel` and an
    "Edit" swipe action (admins) in `ChannelTreeView` + `ChannelBrowserView` (iOS previously had no
    edit-channel UI at all). Note: the channel list doesn't carry the current audio config, so on
    edit the audio fields start from codec defaults — same limitation as macOS/Windows.
  - **Guest nickname.** Guests could not set a display name on iOS *or* macOS (the field was
    absent/disabled; only Windows had it). Added a dedicated `nickname` to `SavedServer` on both
    (backward-compatible Codable), a Nickname field shown in Guest mode (`AddServerView` /
    `AddServerSheet`), and wired the guest auth path to use it (`AppState`, `ConnectWindowController`).
  - **Verified:** `xcodebuild` Debug — macOS BUILD SUCCEEDED; iOS (sim, `ARCHS=arm64`) BUILD
    SUCCEEDED. Core `ctest --preset dev` 22/23 (only `external_pcm` aborts on a pre-existing
    shutdown mutex race; no C++ was changed). Windows C# not buildable on macOS — changes reviewed.

- **Done (2026-06-21):** **iOS iPhone-layout UX fixes.** (1) Channels are now a **drill-down**
  on iPhone — new `ChannelBrowserView` (root list of top-level channels) → `ChannelDetailView`
  (people in the channel + sub-channels + an explicit "Join Channel" button with password
  prompt). The iPad 3-column `NavigationSplitView` is unchanged. (2) Extracted a self-contained
  `UserRow` (context menu + sheets) from `UserListView` so admin actions are reused in the
  drill-down. (3) Fixed the **off-screen chat compose box**: `MainView` now places
  `VoiceControlsView` via `.safeAreaInset(edge: .bottom)` instead of a floating `.overlay`, so
  it reserves layout space above the tab bar and cooperates with keyboard avoidance. (4)
  **Collapsed Activity into Chat** like macOS/Windows: `ChatView` renders a merged, time-sorted
  timeline of `messages` + `activityLog` (activity rows in gray); the separate Activity tab and
  `ActivityLogView.swift` are removed. `xcodebuild` Debug for `generic/platform=iOS` BUILD
  SUCCEEDED (sim slice still arm64-only → simulator run N/A). Next: on-device check of the
  drill-down + compose box + unified timeline.

- **Done (2026-06-22):** **Windows exclude mode is now a real native exclude + self-echo
  removal.** The "All apps except selected" mode previously captured the *complement of a frozen
  app snapshot* in INCLUDE mode (missed late-launched apps, system sounds; wasted captures on
  silent windows). It now opens a **single `ProcessLoopbackCapture` in EXCLUDE mode**
  (`AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE`) of the one chosen app — true
  system-mix-minus-one, dynamic. `AppAudioPickerDialog` enforces single-selection in exclude
  mode (the API takes one target PID). Added an **"Exclude VoiceCat's own audio (prevents echo)"**
  checkbox (default on, entire-desktop only) that routes the desktop capture through the same
  EXCLUDE path targeting `Environment.ProcessId`, killing the whole-device self-echo loop.
  Touched `ProcessAudioMixer.cs` (`ResolveCaptures`), `AppAudioPickerDialog.cs`, `MainForm.cs`,
  `AudioSessionEnumerator.cs` (`EntireDesktop(bool ExcludeSelf)`); docs in voice.md §9. No C++ /
  ABI changes. `dotnet build` clean. **Still to verify on-device:** exclude actually silences
  the chosen app while the rest plays, late-launched apps appear without restart, and the
  self-exclude checkbox removes the echo.

- **Done (2026-06-21):** **Screen-audio sharing on macOS + iOS.** macOS uses ScreenCaptureKit
  (`ScreenAudioCapture.swift`) → `vc_stream_feed_pcm`; iOS uses a ReplayKit Broadcast Upload
  Extension (`VoiceCatBroadcast`) that forwards captured `.audioApp` PCM through a shared App
  Group SPSC ring (`BroadcastAudioRing.swift`) to the host's `BroadcastAudioPump`, which owns
  the `SCREEN_AUDIO` stream and feeds it — single session, no creds on disk. No C++ changes
  (the core was already ready). macOS `xcodebuild` Debug BUILD SUCCEEDED; iOS app + extension
  build for device (the xcframework sim slice is arm64-only, so x86_64-simulator link is N/A).
  Next: on-device end-to-end verification (two clients hear the shared audio; iOS broadcast
  start/stop). NOTE: `ctest --preset dev` is 22/23 — `external_pcm` passes its assertions but
  aborts at shutdown (`mutex lock failed`), a **pre-existing** teardown crash unrelated to this
  change (no C++ was modified).

- **Done (2026-06-21):** **macOS per-app screen-audio selection.** Before sharing, a new
  `ScreenSharePickerSheet` lets the user choose scope — share Everything / Only selected apps /
  All except selected apps — plus a first-class **"Exclude screen reader (VoiceOver) audio"**
  toggle. `ScreenAudioCapture` now takes a `ScreenAudioSelection` and builds the matching
  `SCContentFilter` (`including:` / `excludingApplications:`); app list comes from
  `SCShareableContent`. macOS `xcodebuild` Debug BUILD SUCCEEDED. iOS deliberately untouched —
  ReplayKit only delivers the mixed system stream, so per-app/VoiceOver filtering is impossible
  there (documented in voice.md §9). **Still to verify on-device:** which process actually
  carries VoiceOver speech (VoiceOver app vs. `com.apple.speech.speechsynthesisd`) — the exclude
  set covers both candidates in `ScreenAudioCapture.screenReaderBundleIDs`; confirm exclusion
  actually silences it in a real share.

- **Done (2026-06-20):** **macOS client UI overhaul** — mirrors the Windows client's UI
  overhaul (commit 97fa659 + 540ec13), adapted to Mac-native conventions. Also fixed and
  verified the previously-uncompiled Swift changes from the external PCM feed/tap commit
  (615d2a8). The main window is now just toolbar + channels + users + chat; audio device
  settings (input mode, VAD, PTT key, device picker, level meter) moved to a modeless
  Settings window (⌘,). Details in M5 section below. `swift test` 10/10; `xcodebuild` Debug
  + Release BUILD SUCCEEDED with 0 Swift warnings.
  Next: live manual verification (toolbar toggles, unified log colors, PM windows, channel
  counts, volume slider, settings window); then iOS ReplayKit and macOS ScreenCaptureKit
  consumers of `vc_stream_feed_pcm`.

- **Awaiting on-device verification:** **iOS stereo mic kills headphone/A2DP output — REAL
  root cause found & fixed** (2026-06-20, on Windows; verify on Mac). All prior "fixes" (the
  2026-06-19 entries below) targeted the Swift `IOSAudioRouter` on the false premise that
  "miniaudio does NOT touch AVAudioSession on iOS." **It does.** The core opened its miniaudio
  devices with `ma_device_init(nullptr, ...)`; with a NULL context, miniaudio (0.11.25) runs an
  iOS "hack" (`miniaudio.h` ~44057) that picks a session category by device type, then
  `ma_context_init__coreaudio` (~36552) calls `setCategory()` + `setActive()` on **every device
  open** — capture → `AVAudioSessionCategoryRecord` with **zero options**. That wiped the
  `.playAndRecord` category, the mode, and `.allowBluetoothA2DP`/`.mixWithOthers`/`.allowAirPlay`
  that `IOSAudioRouter` had just configured → headphone/A2DP (and even wired) output died. The
  stereo presets broke worst because they depend on the A2DP output route the wipe removed.
  TeamTalk never hits this: its SDK opens RemoteIO/VPIO AudioUnits directly and leaves the
  session entirely to the app (`UtilSound.swift`); miniaudio insists on managing it.
  - **Fix (core, cross-platform safe):** `AudioEngine` now owns a `ma_context` built by
    `make_context_config()` with `coreaudio.sessionCategory = ma_ios_session_category_none` +
    `noAudioSessionActivate`/`noAudioSessionDeactivate = MA_TRUE`, and passes it to **all**
    `ma_device_init` calls (playback, capture, loopback) and to `enumerate_devices`'s context.
    miniaudio now never touches AVAudioSession; the Swift `IOSAudioRouter` is the sole owner
    (session is already activated on connect in `AppState.swift:authResult`, before any device
    opens, so removing miniaudio's self-activation is safe). Context is lazily inited in
    `start()`, reused across restarts, uninited in `~AudioEngine`.
    Files: `core/src/audio/audio_engine.{h,cpp}`.
  - **TEMP diagnostics (remove after verification):** `AudioSessionManager.logSessionState(_:)`
    logs category/mode/options/route; called after `ensureSessionActive`, on every route change,
    and on `.streamStarted` (right after the core opens its devices). On Mac, watch the log when
    joining voice with the Stereo Mic preset: category must stay `…PlayAndRecord` with
    `allowBluetoothA2DP` and the output route must remain the headphones/A2DP device — NOT flip
    to `…Record`. If confirmed, delete the `logSessionState` calls + method and the prior
    band-aid comments in `IOSAudioRouter`/`audio_engine.cpp` can be trimmed.
  - **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 23/23
    (22/22 prior + `test_external_pcm` new binary). iOS build & on-device run still to be done by the user on the Mac.

- **Done (2026-06-20):** **External PCM feed/tap API (`vc_stream_feed_pcm` +
  `vc_set_pcm_sink`)** — see detail in M5 section below. `ctest --preset dev` 23/23 (was 22/22 + 1 new test binary with 3 sub-tests).
  Next: iOS ReplayKit and macOS ScreenCaptureKit consumers of this API. A public,
  documented API for driving audio streams with externally-provided PCM instead of (or in
  addition to) miniaudio's hardware device. Motivated by four concrete use cases — all in our
  roadmap — that the current "miniaudio owns the device" model can't serve:
  1. **ReplayKit Broadcast Upload Extension (iOS `SCREEN_AUDIO`)** — the extension is a
     *separate process* with a ~50 MB memory cap and can't link the full `AudioEngine`
     (`ma_device`, capture/playback threads). It needs to feed `CMSampleBuffer` audio (system
     app audio) into the encode path without any audio hardware. The current plan in
     `docs/voice.md §9` says the extension links "a minimal slice of the core (Opus encode +
     media send only)" — a public feed-PCM API *is* that minimal slice. The extension links
     Opus + the feed entry point, no `ma_device` needed.
  2. **ScreenCaptureKit (macOS `SCREEN_AUDIO`)** — `SCStream` delivers `CMSampleBuffer` in a
     callback; convert to int16 and feed. No need to route through miniaudio's device layer.
     **This is how macOS screen-audio actually gets implemented** — today it does NOT work:
     `VOICECAT_HAS_LOOPBACK` is Windows-only (`core/CMakeLists.txt:88-95`), so on macOS
     `AudioEngine::start_loopback_capture()` hits the `#else` stub (`audio_engine.cpp:647-649`)
     and returns `false`. The macOS client's "Share Screen Audio" button
     (`MainWindowController.swift:800-816`) calls `startStream(.screenAudio)` which announces
     the stream to peers but captures **zero audio** — peers hear silence. The button is left
     in place (not touched per user request); it'll work once this API + a ScreenCaptureKit
     tap ship on Mac.
  3. **Bots** — music bot, TTS bot, radio relay, transcription bot. They create a
     `SCREEN_AUDIO`/`AUX_DEVICE` stream and feed synthesized or decoded PCM via the feed API.
     No audio hardware required — runs headless on a server. Today the only way to feed
     external PCM is `vc_test_inject_capture` (TEST-ONLY, name signals "don't ship this") or
     re-implementing Opus encode + AEAD + UDP framing yourself (~500 lines of duplicated
     crypto/codec code per consumer).
  4. **Custom clients / accessibility** — soundboard, DAW integration, TTS of incoming chat,
     recording/transcription of remote audio. Need either feed (send) or tap (receive) or
     both.

  **What we already have (input half, gated as test-only):** `vc_test_inject_capture
  (stream_id, pcm, samples)` (`voicecat.h`, `client.cpp:1452`) feeds raw int16 PCM into the
  encode pipeline via `AudioEngine::inject_capture(kind, pcm, n)`. It works for any stream
  kind, supports multiple concurrent injection taps (one ring buffer per local kind), and
  goes through the full encode → AEAD → UDP path. The encode path already handles
  `channels == 1 || 2` (proven by the WASAPI stereo loopback work, 2026-06-17 entry below).
  The only problems: it's marked TEST-ONLY in the header, the name signals "don't use this in
  production," and it hardcodes mono (no `channels` parameter).

  **What's missing (output half):** today decoded remote audio is mixed and pushed to the
  miniaudio playback device (`on_playback`). There's no way for an external consumer to
  intercept the decoded PCM of a specific remote stream — it all goes to the hardware device.
  A bot that wants to record, transcribe, or re-broadcast remote audio has no hook.

  **Plan (API design — clean, append-only, no struct changes, ABI-stable):**
  - **`vc_stream_feed_pcm`** — promote `vc_test_inject_capture` to a public, documented API
    and add a `channels` parameter:
    ```c
    /* External PCM feed — replaces the hardware capture device for this stream. Caller
       provides interleaved int16 PCM at the stream's sample rate. The core frames it,
       encodes (Opus), seals (AEAD), and sends (UDP). Works for any stream kind
       (MIC/SCREEN_AUDIO/AUX_DEVICE). The stream must be started first (vc_stream_start);
       this just replaces the capture source. channels = 1 (mono) or 2 (stereo interleaved).
       Thread-safe; may be called from any thread including audio callbacks. */
    vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id,
                                 const int16_t* pcm, size_t samples_per_channel,
                                 uint32_t channels);
    ```
  - **`vc_set_pcm_sink`** — symmetric output side: receive decoded remote audio as int16 PCM
    instead of (or in addition to) the hardware playback device:
    ```c
    /* External PCM tap — receive decoded, mixed remote audio as int16 PCM. The callback
       fires on the audio thread with the mixed output for a specific remote stream. Pass
       cb=NULL to disable (default: disabled, hardware playback only). When enabled, PCM is
       delivered to the sink AND the hardware device (dual output) so a bot can record
       without disabling local monitoring. user_id+stream_id identify the source stream.
       The callback MUST NOT block — copy what you need and return (same contract as
       vc_callbacks.on_event). */
    typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
                                   const int16_t* pcm, size_t samples_per_channel,
                                   uint32_t channels, uint32_t sample_rate);
    vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
    ```
  - **Core changes:**
    - `core/include/voicecat.h` — add `vc_pcm_sink_cb` typedef + the two function
      declarations (append-only, after `vc_test_inject_capture`). Full doc comments on both
      (contract, thread-safety, lifetime, use cases).
    - `core/src/voicecat.cpp` — thin C trampolines → `vc_client::stream_feed_pcm` /
      `set_pcm_sink`.
    - `core/src/core/client.{h,cpp}` — `stream_feed_pcm`: validates `stream_id`, looks up
      the `LocalStream`'s kind, calls `audio_engine_.inject_capture(kind, pcm, n)` (existing
      path) with the channel count forwarded. `set_pcm_sink`: stores the callback + user
      pointer; `on_playback` (or a new fan-out in the mixer) invokes it per remote stream
      alongside the existing hardware write. Keep `vc_test_inject_capture` as a deprecated
      alias calling `stream_feed_pcm(..., channels=1)` for source compatibility.
    - `core/src/audio/audio_engine.{h,cpp}` — `inject_capture` already exists per-kind; add
      a `channels` parameter to the ring-buffer write path (or a parallel stereo-aware
      variant). The encode path in `client.cpp::on_capture_frame` already handles
      `channels==2` via the stereo encode branch — just plumb the value through. For the
      sink: add a `pcm_sink_` member (callback + user); in `on_playback` after mixing, if the
      sink is set, copy the mixed PCM for the current stream and invoke the callback. The
      copy must stay off the RT-critical path — document the non-blocking contract.
  - **Skeleton stub path:** update `client.cpp`'s `#else` (no-deps) stub section to add
    `vc_stream_feed_pcm`/`vc_set_pcm_sink` returning `VC_ERR_NOT_IMPLEMENTED` — keeps the
    skeleton preset green.
  - **Swift `VoiceCatCore`:** add `feedPcm(streamId:pcm:samplesPerChannel:channels:)` and
    `setPcmSink(_:user:)` (the Swift wrapper around `vc_pcm_sink_cb` — a
    `@convention(c)` closure + `Unmanaged` context, mirroring `Callbacks.swift`). Wraps both
    new ABI functions.
  - **C# `VoiceCat.Interop`:** add `StreamFeedPcm(streamId, pcm, samples, channels)` (with
    `int16[]` marshaling) and `SetPcmSink` (delegates via `[UnmanagedCallersOnly]` thunk,
    mirroring the event-callback pattern). Wraps both new ABI functions.
  - **Tests:**
    - `tests/test_external_pcm.cpp` (new) — `test_feed_pcm_round_trip`: two clients, A feeds
      a known mono sine wave via `vc_stream_feed_pcm` on a MIC stream, B receives via the
      normal decode path and asserts energy matches. `test_feed_pcm_stereo`: same with
      `channels=2`, assert L≠R end-to-end (mirrors the WASAPI loopback stereo test).
      `test_pcm_sink`: B sets a `vc_pcm_sink_cb`, A feeds PCM, assert the sink callback
      receives the decoded PCM with matching energy. All headless, no audio hardware.
    - `clients/apple/Tests/VoiceCatCoreTests/` — Swift wrapper round-trip for `feedPcm`.
    - `clients/windows/VoiceCat.Interop.Tests/` — C# wrapper round-trip.
  - **Docs:**
    - `docs/architecture.md §4` — new subsection on external PCM feed/tap: the contract
      (caller provides interleaved int16 at the stream's sample rate; core frames/encodes/
      seals/sends for feed; core decodes/mixes/delivers for sink; sink callback must not
      block), the use cases (ReplayKit, ScreenCaptureKit, bots, custom clients), and the
      relationship to `vc_test_inject_capture` (deprecated alias).
    - `docs/voice.md §9` — update the iOS ReplayKit and macOS ScreenCaptureKit rows: both
      now consume `vc_stream_feed_pcm` instead of a "minimal slice of the core." Update the
      iOS detail bullets: the extension links Opus + `vc_stream_feed_pcm` (not a parallel
      media stack). Add a macOS ScreenCaptureKit note: convert `CMSampleBuffer` → int16,
      feed via `vc_stream_feed_pcm` — this is how macOS screen-audio actually ships.
    - `docs/protocol.md` — no protocol changes (the feed/sink are client-local; the wire
      format is identical whether PCM came from miniaudio or an external source). Note this
      explicitly.
    - `docs/roadmap.md` — add a milestone entry; update the iOS ReplayKit and macOS
      ScreenCaptureKit pending items to reference `vc_stream_feed_pcm`.
  - **Implementation order:**
    1. C ABI + core (`voicecat.h`, `voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`) +
       skeleton stub. Verify `ctest --preset dev` green.
    2. `tests/test_external_pcm.cpp` — the three behavior tests. Verify green.
    3. Swift `VoiceCatCore` wrapper + `VoiceCatCoreTests` round-trip.
    4. C# `VoiceCat.Interop` wrapper + `VoiceCatClientSmokeTests` round-trip.
    5. Docs (`architecture.md`, `voice.md`, `protocol.md`, `roadmap.md`, header comments).
    6. **Then** ReplayKit (iOS) and ScreenCaptureKit (macOS) become ~100-line consumers of
       this API instead of parallel media stacks.
  - **Verification:** `ctest --preset dev` green (3 new tests); `swift test` green; `dotnet
    test` green; `xcodebuild` (skeleton) green. The feed/sink tests are fully headless — no
    audio hardware, no simulator, no device — so they run in CI on every platform.
  - **Files to touch:**
    - Core C++: `core/include/voicecat.h`, `core/src/voicecat.cpp`,
      `core/src/core/client.{h,cpp}`, `core/src/audio/audio_engine.{h,cpp}`.
    - Tests: `tests/test_external_pcm.cpp` (new), `tests/CMakeLists.txt`.
    - Swift: `clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift`,
      `clients/apple/Sources/VoiceCatCore/Callbacks.swift`,
      `clients/apple/Tests/VoiceCatCoreTests/ExternalPcmTests.swift` (new).
    - C#: `clients/windows/VoiceCat.Interop/VoiceCatClient.cs`,
      `clients/windows/VoiceCat.Interop/NativeMethods.cs`,
      `clients/windows/VoiceCat.Interop.Tests/ExternalPcmTests.cs` (new).
    - Docs: `docs/architecture.md`, `docs/voice.md`, `docs/protocol.md`, `docs/roadmap.md`.
  - **ABI stability:** append-only — two new functions + one new typedef, no existing
    structs/enums changed. `vc_test_inject_capture` stays as a deprecated alias for source
    compatibility. Treat as a deliberate, versioned ABI event per `docs/protocol.md §8`.
  - **Relationship to the iOS audio routing plan:** orthogonal. The iOS routing layer controls
    *which hardware route* miniaudio opens (AVAudioSession config in Swift). This plan is about
    *bypassing miniaudio's hardware entirely* (external PCM feed/tap). Both ship; they don't
    conflict. ReplayKit/ScreenCaptureKit consume this API; the iOS routing layer controls the
    *mic* path which still uses miniaudio's device.

---

## Recent completed work

All items below are `[x]` done; `ctest --preset dev` 26/26 on Windows after all.

- **Per-channel sample_rate as a bandwidth cap** (2026-06-22): the channel `sample_rate` field
  was inert (the codec is pinned to 48 kHz). Made it meaningful without changing the 48 kHz
  clock: it's carried as `OpusParams::max_bandwidth_hz` and applied via `OPUS_SET_MAX_BANDWIDTH`
  in `OpusEncoder::init` (8000→narrowband … 48000→full). Made it **channel-authoritative** on
  the server (`conn_session.cpp` no longer overrides effective `sample_rate` with the client's
  always-48000 request — like `frame_ms`/`mode`). `vc_get_stream_audio_config` now reports the
  channel's configured rate for own streams too. New ctest `channel_samplerate`: a 7 kHz tone is
  attenuated ~1000× on an 8 kHz channel vs a 48 kHz channel. Files: `opus_codec.{h,cpp}`,
  `client.cpp`, `server/src/conn_session.cpp`, `docs/voice.md`, `tests/test_channel_samplerate.cpp`,
  `tests/CMakeLists.txt`. (Future: a true non-48k stack is possible but unnecessary — 48 kHz is
  what nearly all hard/software runs at; the bandwidth cap covers the narrowband use case.)

- **Non-20ms channel frame_ms fix** (2026-06-22): the AudioEngine capture clock is fixed at
  48 kHz / 20 ms (960-sample frames), but a channel may set any Opus `frame_ms` (2.5…60 ms,
  docs/voice.md §3) and the server enforces it unclamped. The send path handed the engine's
  960-sample frame straight to an encoder configured for the channel's window — silently
  ignoring `frame_ms > 20` and **breaking `frame_ms < 20` entirely** (receiver sized its decode
  buffer too small → `OPUS_BUFFER_TOO_SMALL` → dead audio). Affected the hardware mic AND
  `vc_stream_feed_pcm`. Fix: `vc_client::on_capture_frame` now reframes each captured/fed block
  to `ls.frame_samples` via a per-`LocalStream` accumulator (pre-sized at announce, no RT-thread
  alloc) before `encode_and_send_frame`; the 20 ms case stays a zero-copy fast path. Also pinned
  the codec to 48 kHz internally in `opus_params_from_audio_config` (was honoring a non-48k
  effective sample_rate against a 48k PCM clock). New ctest `frame_ms_reframe` (40 ms accumulate
  + 10 ms split round trips). Files: `client.{h,cpp}`, `voicecat.h` (feed doc), `docs/voice.md`,
  `tests/test_frame_ms_reframe.cpp`, `tests/CMakeLists.txt`.

- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped.
  Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API.
  Adds symmetric PCM sink on the playback thread. Swift wrapper (`feedPcm`/`setPcmSink` in
  `VoiceCatClient.swift`, 4 XCTest smoke tests). C# wrapper (`StreamFeedPcm`/`SetPcmSink` in
  `VoiceCatClient.cs` + `NativeMethods.cs`, 4 xUnit smoke tests in `ExternalPcmTests.cs`).
  Three new headless C++ ctests. Docs: architecture.md §4 new subsection, voice.md §9 updated,
  protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Files: `voicecat.h`,
  `voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`, `tests/test_external_pcm.cpp`,
  `tests/CMakeLists.txt`, Swift + C# wrappers.

- **iOS A2DP + stereo root cause fix** (2026-06-20): miniaudio's NULL-context `ma_device_init`
  was calling `AVAudioSession setCategory(Record)` on every device open, wiping the session
  config `IOSAudioRouter` had set. Fixed by sharing a `ma_context` with
  `sessionCategory=none` + `noAudioSessionActivate/Deactivate=MA_TRUE` — miniaudio never
  touches `AVAudioSession`; `IOSAudioRouter` is the sole owner. Files: `audio_engine.{h,cpp}`.

- **iOS audio routing overhaul** (2026-06-19): Full `IOSAudioRouter` singleton drives all
  `AVAudioSession` config before miniaudio opens devices. Fixed stereo mic polar-pattern setup
  (WWDC20 recipe: `setPreferredInput` + `setInputDataSource` + `.stereo` polar pattern + no
  `setPreferredInputNumberOfChannels`). Added `vc_audio_restart` ABI (full stop+reinit for
  close→reconfigure→reopen ordering). Added `vc_set_capture_channels` ABI (core stereo-mic
  support). AVAudioSession activated proactively on `.authResult`, not lazily on
  `.streamStarted`. Join/Leave Voice button added (parity with macOS). Channel-id sync fixed
  (mic button was permanently dimmed). iOS deployment target raised to 18.0.

- **iOS SwiftUI client** (2026-06-19): `VoiceCatiOS.xcodeproj` at `clients/apple/iOS/`.
  Full feature parity with macOS/Windows: saved server list (JSON + Keychain, App Group
  `group.cat.voice.VoiceCat`), TOFU, connect flow, channel tree, user list with context menus,
  chat, admin sheets, voice controls, settings. `xcodebuild` → BUILD SUCCEEDED.

- **macOS AppKit client** (2026-06-18): `VoiceCatMac.xcodeproj` at `clients/apple/macOS/`.
  Fixed compile errors (`NSAccessibility` call-site arg order, `StreamSummary.id` vs
  `.streamId`) and linker issues (`OTHER_LDFLAGS = -lc++`, `ONLY_ACTIVE_ARCH = YES` for
  Release). Debug + Release both BUILD SUCCEEDED.

- **Swift `VoiceCatCore` package + XCFramework** (2026-06-18): Shared Swift wrapper at
  `clients/apple/`. `build-xcframework.sh` merges `libvoicecat.a` + 107 vcpkg static deps into
  a fat `.a` via `libtool -static`. 6/6 Swift tests green (real server, mirrors C# Interop
  tests). Supports macOS-arm64 + iOS-arm64 + iOS-sim slices.

- **macOS port validated** (2026-06-18): 21/21 on macOS. Three cross-platform bugs fixed:
  missing `<netdb.h>` in POSIX test branch; SIGPIPE kills (added `SIG_IGN`); use-after-free of
  Asio kqueue reactor on server shutdown (fixed `TcpAcceptor` shutdown/connection-drain
  sequence).

- **CMake preset cleanup** (2026-06-18): `m1-dev`→`dev`, `dev`→`skeleton`, `m2-dev` dropped.
  New `release`, `server-release` (stripped), `apple-dev`/`apple-ios`/`apple-ios-sim`. Cross-
  platform triplet auto-resolved by `cmake/voicecat-toolchain.cmake`.

- **Disconnect, keepalive & reaper** (2026-06-18): Client sends `Ping` every 15 s; server
  reaper drops sessions after 45 s; UDP `KEEPALIVE` every 5 s keeps NAT alive. `vc_disconnect`
  sends graceful `Disconnect` proto. Stale-user LEFT broadcast on drop. PLC capped at ~2 s.
  Three new tests: `test_disconnect_left`, `test_plc_cap`, `test_reaper_timeout`.

- **Stereo screen-audio loopback** (2026-06-17): WASAPI loopback opens in channel's
  stereo/mono mode (was hardcoded mono). Real stereo flows end-to-end through loopback → encode
  → decode → mixer. New `test_loopback_stereo_capture`.

- **Windows screen-audio UI wired** (2026-06-17): `btnScreenShareToggle` in `MainForm.cs`.
  No core/proto/ABI changes — all the plumbing was already there. `dotnet test` 4/4 green.

- **Bug fixes** (2026-06-16 – 2026-06-17):
  - *AEAD nonce desync in SFU relay* — relay forwarded sender's `seq` verbatim; recipient
    nonce reconstruction used the wrong counter. Fixed by rewriting the outgoing `seq` field
    to the recipient's `peek_send_counter()`.
  - *Playout clock free-ran* — `playout_ts` advanced even during VAD/PTT silence gaps,
    eventually dropping all frames as too-late. Fixed with resync in `on_playback` via
    `JitterBuffer::peek_front_ts()`.
  - *Stale users after disconnect* — `ConnSession::close()` didn't broadcast `UserEvent::LEFT`
    before erasing. Fixed; PLC cap added as defense-in-depth.
  - *"Randomly bumped to Lobby"* — server excluded the actor from its own state-change
    broadcasts. Fixed: `UserEvent::UPDATED` now goes to all clients including the actor.
  - *Silent playback after join* — `opus_decode` received hardware callback frame count as
    `max_samples` instead of the Opus frame size. Fixed with a decode ring buffer.

---

## Milestones (see [docs/roadmap.md](docs/roadmap.md) for full detail)

- [x] **M0 — Scaffolding** ✓ complete
- [x] **M1 — Control plane** ✓ complete (2026-06-15)
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16)
- [x] **M3 — Multi-stream & per-channel tuning** ✓ complete (2026-06-16)
- [x] **M4 — Native clients** — Windows WinForms ✓ (2026-06-17); macOS AppKit ✓ (2026-06-18); iOS SwiftUI ✓ (2026-06-19)
- [~] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)

---

## M0 — Scaffolding ✓

Repo layout (`core/ server/ tools/ clients/ tests/`), CMake + vcpkg manifest, C ABI header
(`voicecat.h`), proto source of truth, core stubs for all six subsystems, `voicecat-server` +
`vccli` skeletons, smoke CTest, `.clang-format`/`.gitattributes`/`.gitignore`.

---

## M1 — Control plane ✓ (completed 2026-06-15)

**Exit criterion:** `test_m1_integration` — two clients authenticate over TLS 1.3 (guest +
Argon2id), exchange channel + private text. ~1 s.

`FrameCodec`, `TlsContext` (mbedTLS 1.3, ECDSA-P256 self-signed, TOFU pins TLS leaf-cert
SHA-256), `WorkerPool`, `Database` (SQLite + Argon2id), `ServerIdentityManager`,
`ConnSession` state machine, `SessionRegistry`, `vc_client` full M1 C ABI, `voicecat-admin`
CLI, dual-stack `TcpAcceptor`. **Key bug fixed:** `send_frame` double-framing — `encode_envelope`
was pre-framing the protobuf; fixed by passing raw protobuf bytes.

---

## M2 — Voice, single stream ✓ (completed 2026-06-16)

**Exit criterion:** `test_m2_voice` + `test_voice_client_abi` — two headless clients auth, bind
UDP, 50 Opus frames relayed + re-encrypted by SFU, B receives ≥25 and decrypts. ~4 s.

14-byte UDP voice header, `SodiumMediaCrypto` (ChaCha20-Poly1305 + 64-bit anti-replay),
`OpusEncoder`/`OpusDecoder` (FEC, PLC), `UdpMediaChannel`, `JitterBuffer`, `AudioEngine`
(miniaudio), `MediaRelay` SFU. **Key bug fixed:** `on_playback` passed hardware callback frame
count as `opus_decode` max_samples; fixed with a per-stream decode ring buffer.

---

## M3 — Multi-stream & per-channel tuning ✓ (completed 2026-06-16)

**Exit criterion:** `test_m3_multistream` — client A runs two concurrent streams (MIC +
SCREEN_AUDIO); B sees both; per-stream gain/mute/NR independent; effective Opus config matches
channel's server-enforced settings. ~2.4 s.

Fixed server `stream_id` counter bug (always wrote `1`). Per-channel `AudioConfig` populated
(Lobby: mono/24kbps/VOIP + DTX; Music Room: stereo/128kbps/AUDIO). `LocalStream` map,
`pending_announce_kind_`, `run_talk_timer()`, thread-join race in `teardown_voice()` fixed.
New C ABI: `vc_get_stream_audio_config`, `vc_test_inject_capture`.

---

## Post-M3 follow-up ✓ (completed 2026-06-16)

- **Device enumeration** — `vc_list_devices`/`vc_set_input_device`; opaque hex device ids;
  `vc_free_device_list` now frees. Works pre-connect.
- **VAD/PTT gate** — `EnergyVadProcessor` (RMS threshold ~0.025, 300 ms hang-time);
  `vc_set_input_mode`/`vc_set_push_to_talk`; MIC-only (SCREEN_AUDIO/AUX_DEVICE bypass).
- **True stereo playback** — `playback_channels=2`; stereo decoded L→L R→R in mixer; mono
  upmixed L=R; hardware fallback to mono on failure.
- **WASAPI loopback** — `loopback_device_` with `ma_device_type_loopback`;
  `VOICECAT_HAS_LOOPBACK` macro (Windows-only). `vccli --share-screen-audio`.

**Known deferred (still open):** AEC/NS/AGC (no working Windows/MSVC WebRTC APM build);
process-specific WASAPI loopback; RT-thread rule violation in `on_capture_frame` (mutex lock
on audio callback thread — pre-existing, needs lock-free ring-buffer refactor).

---

## M4 — Native clients ✓ (completed 2026-06-17 – 2026-06-19)

**Exit criterion:** `ctest --preset dev` 21/21 green; `dotnet build` 0 warnings; `xcodebuild`
BUILD SUCCEEDED (macOS + iOS); manually verified: connect, TOFU, channel tree, join, voice,
text, device pickers, level meter on each platform.

**New C ABI (additive):** `vc_list_channels`/`vc_list_users`/`vc_list_user_streams`,
`vc_join_channel`, `VC_EVENT_SERVER_IDENTITY` + `vc_confirm_server_identity`,
`vc_config::tofu_store_path`, `VC_INPUT_ALWAYS_ON`, `vc_set_vad_threshold`,
`vc_audio_suspend`/`vc_audio_resume`, `vc_audio_restart`, `vc_set_capture_channels`.

**Windows** (`clients/windows/`): `VoiceCat.Interop` (P/Invoke, `[UnmanagedCallersOnly]`),
`VoiceCat.App` (ConnectDialog, ServerIdentityDialog, MainForm with full M5 moderation UI,
PerUserTuningDialog, PttKeyCaptureDialog), `VoiceCat.Interop.Tests`. PTT can be system-wide
(Raw Input / WM_INPUT) or focus-scoped, toggled in Audio settings (default system-wide).

**macOS** (`clients/apple/macOS/VoiceCatMac.xcodeproj`): NSOutlineView channel tree,
NSTableView user list, NSTextView chat, voice controls, full VoiceOver accessibility, admin
menu, 17 Swift source files. `build-xcframework.sh` produces `VoiceCatCore.xcframework`.

**iOS** (`clients/apple/iOS/VoiceCatiOS.xcodeproj`): SwiftUI, `NavigationSplitView`/`TabView`,
`OutlineGroup` channel tree, `IOSAudioRouter` AVAudioSession driver, 24 Swift source files,
iOS 18.0 deployment target. App Group `group.cat.voice.VoiceCat` for Keychain sharing.

---

## M5 — Moderation, polish, and beyond [~] (in progress 2026-06-17)

**Exit criterion:** four ABI-level tests green (`test_m5_permissions`,
`test_m5_kick_ban_move_mute`, `test_m5_admin_accounts`, `test_m5_channel_crud`);
`vccli` can drive all moderation/admin/channel operations against a live server.

- [x] **Server-side moderation & permissions** — per-session `Permissions`, kick/ban/move/
  server-mute, channel CRUD, DB schema v2 (`channels`, `bans`), BLAKE2b channel passwords.
- [x] **C ABI** — `vc_kick_user`, `vc_ban_user`, `vc_set_permission`, `vc_set_server_mute`,
  `vc_move_user`, `vc_create_channel`, `vc_edit_channel`, `vc_delete_channel`,
  `vc_create_account`, `vc_reset_password`, `vc_delete_account`, `vc_list_accounts`,
  `vc_get_permissions`; events `VC_EVENT_GENERIC_RESULT`, `VC_EVENT_ACCOUNT_LIST`.
- [x] **Four M5 tests** passing — `ctest --preset dev` 21/21.
- [x] **vccli** M5 flags: `--kick`, `--ban`, `--move`, `--server-mute`/`-unmute`/`-deafen`/
  `-undeafen`, `--set-permission`, channel CRUD, account CRUD, `--username`/`--password`.
- [x] **All three client UIs** (Windows WinForms, macOS AppKit, iOS SwiftUI) expose the full
  M5 moderation and admin surface.
- [x] **Docs** — `docs/protocol.md`, `docs/security.md` kept in sync.
- [x] **DRED/audio-quality polish** — done (2026-06-20). `bool dred` added to `AudioConfig`
  proto (field 11) and `vc_audio_config` C ABI. Encoder: `OPUS_SET_DRED_DURATION(2)` when
  enabled (20 ms of ML redundancy per packet). Decoder: `OpusDREDDecoder` + per-stream
  `OpusDRED` scratch pre-allocated; `JitterBuffer::try_copy_front_payload` peeks at the next
  buffered packet on every PLC step; if DRED data is present, `opus_decoder_dred_decode`
  reconstructs the lost frame — otherwise falls back to standard PLC. New test:
  `test_dred_toggle` (ctest 22/22). Files: `voicecat.proto`, `voicecat.h`,
  `opus_codec.{h,cpp}`, `audio_engine.{h,cpp}`, `client.cpp`, `session.{h,cpp}`.
- [x] **In-band FEC decoder wiring** — done (2026-06-22). The encoder set `OPUS_SET_INBAND_FEC`
  all along, but the decoder never invoked it — the loss path went DRED → PLC, so FEC redundancy
  was emitted (and paid for in bitrate) but never consumed. Wired the FEC recovery into
  `AudioEngine::on_playback`'s loss branch between DRED and PLC: copy the next buffered packet
  once, try DRED, else (if the stream negotiated FEC) `decode(next_pkt, …, fec=true)`, else PLC.
  Added per-stream `RemoteStream::fec_enabled_`, captured from `OpusParams` in
  `init_recv_stream`. Recovery priority is now **DRED → FEC → PLC**. ctest 27/27 green. Files:
  `audio_engine.{h,cpp}`, `docs/voice.md`.
- [ ] **DRED toggle in client UIs** — expose the `dred` flag in all three channel-config UIs
  so admins can enable it per channel. Windows: `ChannelEditForm` / `vc_channel_info.audio.dred`
  checkbox. macOS AppKit: channel-edit sheet. iOS SwiftUI: channel-edit form. All three UIs
  already have full channel CRUD wired; this is an additive checkbox on the existing audio-config
  section. (Core/protocol/ABI all done — this is UI-only work.)
- [x] **macOS ScreenCaptureKit screen-audio** — done 2026-06-21. `ScreenAudioCapture.swift`
  drives an `SCStream` (audio-only, `excludesCurrentProcessAudio`), converts Float32 →
  int16 in the channel's mono/stereo mode, and calls `vc_stream_feed_pcm`. Capture starts on
  the self `.streamStarted` event (when the effective config is known); wired into
  `MainWindowController.screenAudioClicked()`.
- [x] **iOS ReplayKit Broadcast Extension** (`VoiceCatBroadcast`) — done 2026-06-21.
  Forward-to-host design: the extension (`SampleHandler.swift`) captures `.audioApp`,
  converts to 48 kHz int16 stereo, and writes a shared App Group SPSC ring
  (`BroadcastAudioRing.swift`); the host's `BroadcastAudioPump` owns the `SCREEN_AUDIO`
  stream and feeds via `vc_stream_feed_pcm` (single session, no creds on disk). UI is an
  `RPSystemBroadcastPickerView` in `VoiceControlsView`. (Replaced the speculative
  `BroadcastCredentials.swift` self-connecting design, now removed.)
- [x] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — done
  2026-06-20. Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public API with
  stereo support. Adds a symmetric PCM sink fired on the playback thread per decoded remote
  stream. Full wrappers for Swift (`feedPcm`/`setPcmSink`) and C# (`StreamFeedPcm`/
  `SetPcmSink`). Three new C++ ctests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`,
  `test_pcm_sink`), 4 Swift XCTest smoke tests, 4 C# xUnit smoke tests. Docs updated
  (architecture.md §4 new subsection, voice.md §9 updated, protocol.md §8 explicit
  no-protocol-change note, roadmap.md M5 entry). `ctest --preset dev` 23/23.
- [x] **macOS client UI overhaul** — done 2026-06-20. Mirrors the Windows client's UI
  overhaul (toolbar, unified log, PM windows, channel counts, output volume, keyboard
  shortcuts), adapted to Mac-native conventions:
  - **NSToolbar**: Join Voice, Share Screen Audio, Mute, Deafen (SF Symbol toggle buttons),
    and Output Volume slider (NSSlider 0–100, default 80). Voice actions + mute/deafen +
    output volume moved out of the bottom voice panel into the toolbar. Bottom panel keeps
    input-mode segmented control / VAD slider / PTT key / device picker / level meter.
  - **Unified log**: chat `NSTextView` + activity `NSTableView` collapsed into a single
    `NSTextView` — activity events in `secondaryLabelColor` (gray), chat in default color.
    Removed `activityTableView` and `activityLog` array.
  - **Private messaging**: scope dropdown removed; compose bar always sends to the current
    channel. Each PM conversation opens in its own modeless `PrivateMessageWindowController`
    (NSWindow). Incoming `.textMessage` with `.private` scope routed to the right window;
    outgoing PMs echoed by server arrive through the same path. "Send Private Message…"
    added to user context menu. "New Private Message…" (⌘⇧N) opens `UserPickerSheet`
    listing all server users.
  - **Channel counts**: outline view renders `"Name (n)"` with live user counts;
    `refreshChannelTree()` called on `.userJoined`/`.userLeft` (was missing).
  - **Voice menu** (⌘⇧V join/leave, ⌘⇧S share screen, ⌘⇧M mute, ⌘⇧D deafen) and **Messages
    menu** (⌘⇧N new PM) added to `NSApp.mainMenu` via `NSMenuItem` key equivalents with
    `[.command, .shift]` mask. Removed on `windowWillClose`. Mac-native: ⌘ not Ctrl, dispatched
    by the responder chain (no custom key monitor needed).
  - **Output volume**: `setOutputVolume(_:)` wrapper added to `VoiceCatClient.swift` (was
    missing — the C ABI + C# wrapper shipped in commit 97fa659 but the Swift wrapper was
    never added). Wired end-to-end: toolbar slider → `client.setOutputVolume(gain)`.
  - **Part A (uncompiled Swift fix)**: the external PCM feed/tap Swift wrapper (commit
    615d2a8) was never compiled — the local xcframework predating the `voicecat.h` PCM
    additions. Fixed: rebuilt xcframework (regenerated module map), fixed `UInt`→`Int` type
    mismatch in `feedPcm` (Swift imports `size_t` as `Int` not `UInt`), added
    `VoiceCatPcmSinkCallback` typealias (Swift-idiomatic alias for the C `vc_pcm_sink_cb`
    so consumers don't need to directly import `VoiceCatC`). `swift test` 10/10 green.
  - **Audio settings moved to Settings window**: the bottom voice panel (input mode, VAD
    slider, PTT key, device picker, level meter) was removed from the main window and moved
    into a new `SettingsWindowController` — a modeless window opened via the app menu's
    "Settings…" (⌘,) item. The main window is now just toolbar + channels + users + chat.
    Source-of-truth for audio settings (`selectedInputMode`, `vadThresholdValue`,
    `selectedInputDeviceId`, `pttKeyCode`) lives in `MainWindowController` so voice start can
    apply them even before the settings window has been opened; `SettingsWindowController`
    reads from and writes back to those properties and applies changes to the client
    immediately when voice is active. The level meter is forwarded from
    `MainWindowController.handleLevel` → `settingsWindowController.updateLevel(rms:)`.
    `keyCodeName` helper deduplicated (was duplicated in `PttKeyCaptureSheet.swift` +
    `MainWindowController.swift` — now shared from `MainWindowController.swift`).
  - Files: `MainWindowController.swift` (overhauled), `PrivateMessageWindowController.swift`
    (new), `UserPickerSheet.swift` (new), `SettingsWindowController.swift` (new),
    `VoiceCatClient.swift` (setOutputVolume + VoiceCatPcmSinkCallback typealias + feedPcm
    type fix), `ExternalPcmTests.swift` (use typealias), `PttKeyCaptureSheet.swift` (removed
    duplicate `keyCodeName`), `VoiceCatMac.xcodeproj/project.pbxproj` (register 3 new files).
  - **Platform-specific adaptations** (vs. Windows): `NSToolbar` instead of `ToolStrip`;
    global menu bar + `NSMenuItem` key equivalents (⌘ not Ctrl, responder-chain dispatched);
    PM windows as modeless `NSWindow`s; picker as Mac sheet; gray = `secondaryLabelColor`;
    SF Symbols for toolbar icons.

---

## Decisions log

All architecture/scope decisions are settled and recorded in
[docs/roadmap.md §2 "Resolved decisions"](docs/roadmap.md) and reflected across `docs/`.
If you make a *new* decision, record it there and link it here.

---

## How to update this file

1. Check off tasks as you complete them; flip a milestone to `[x]` only when its **exit
   criterion test** passes.
2. Keep the **"Where we left off / next action"** block at the top accurate — it's the first
   thing the next agent reads.
3. When you start a milestone, copy its task list from `docs/roadmap.md` into a section here.