voice-cat/PROGRESS.md

# VoiceCat status

Updated: 2026-09-25

## Current state

VoiceCat's supported implementation is .NET 10. The managed protocol, crypto, TLS, server,
CLI, client state, audio engine, Windows client, macOS client, and iOS client are implemented.
Retired implementations and compatibility projects have been removed; this tree contains only
the supported product and its required native media boundaries.

The source-of-truth layout is:

- `proto/voicecat.proto` — wire schema.
- `src/` — protocol, crypto, codec/DSP bindings, server, client core, audio, and CLI.
- `tests/VoiceCat.Tests/` — managed behavior and integration tests.
- `clients/windows/` — WinForms application over the managed core.
- `clients/apple/` — AppKit and UIKit applications over the managed core.
- `native/media/` and `native/rnnoise/` — required Opus/RNNoise shim and vendored RNNoise.
- `native/apple/broadcast/` — required ReplayKit upload extension and shared ring producer.

The physical-device iOS voice path now supports ReplayKit fallback, stable Apple VPIO voice-chat
capture using a paced 20 ms handoff, and true built-in stereo microphone capture. Stereo was
verified on an iPhone 16 Pro Max with a two-channel AVAudioEngine input and distinct left/right
samples; the managed Apple binding requires native use of its otherwise-unmapped stereo polar
pattern constant. The voice-chat preset now leaves speaker routing off so a connected Bluetooth
headset can supply both input and output; speaker routing remains an explicit Advanced setting.
Device-selected inputs are not saved as preferences during route refresh. Bluetooth switching
still needs device validation.

The iOS user list now opens a remote-user detail view with independent tuning for each active
audio stream. Private messages are grouped into per-user conversations with direct access to the
same user and audio controls. Lists reload only when their rendered content actually changed and
only while on screen, and the 20 Hz microphone level is a separate signal from the general model
change, so VoiceOver explore mode no longer re-announces the row under a dragging finger or loses
the element a double tap was aimed at.

The iOS audio graph is rebuilt only when the audio configuration changed. A lost connection
unbinds the client but keeps the session, graph, and route alive, so a reconnect rebinds to a
live Bluetooth HFP link instead of renegotiating one; restoring voice reuses a running capture
tap of the same width; and foregrounding ensures the graph is running rather than rebuilding it.
Route changes, media-services resets, and the stall watchdog still force a full rebuild.

The media path now survives changing networks. A client whose source address changes proves
possession of its media key from the new address with an authenticated `Rebind` frame and the
relay moves its endpoint, instead of the session dying silently in both directions; the client
rebuilds its UDP socket rather than retrying on one pinned to a vanished interface. The control
connection is judged live by server traffic rather than assumed live, so a blackholed TCP path
is detected in 30 s instead of waiting minutes for the OS, and 12 s on iOS. iOS also watches the
system path and fails the control connection the moment the carrying interface changes, so a
handover reconnects in about a second instead of waiting out the silence timeout; access-point
roaming and an unusable-but-unchanged link are deliberately not handovers and are ridden out.
The first reconnect attempt is immediate, and a loss is only announced once an attempt has
actually failed, so a handover reads as a hiccup rather than a dropped call. A control reconnect
still re-authenticates and rejoins: seamless handover needs control-plane session resumption,
because the media keys come from the TLS exporter of the connection that was lost. The receive jitter buffer keeps a
one-frame depth floor, measures late and reordered arrivals, and can deepen mid-call, and a
stalled consumer now costs bounded audio rather than the live talkspurt.

SQLite schema v4 persists DRED and the channel packet-loss mode. Manual loss remains the default;
automatic Fast/Balanced/Stable modes measure each sender's authenticated UDP uplink at the server,
cap the applied Opus hint at 30%, and feed it back over TLS.

## Release gates

- Run real multi-person calls on Windows, macOS, and physical iOS hardware, including adaptive
  20/40/60 ms buffering, duration-aware DRED/FEC, automatic packet-loss feedback, and mismatched
  input/output endpoints.
- Complete NVDA and VoiceOver navigation/announcement passes.
- Verify iOS remote-user tuning and private-conversation navigation with VoiceOver, including
  multiple streams, users without active streams, and users who disconnect while a view is open.
- Exercise iOS background/lock, interruption, Bluetooth, route-change, ReplayKit, and iOS 27
  ScreenCaptureKit paths on devices. The background/lock gate keeps a call active for 15+ minutes
  backgrounded and screen-locked with no periodic glitches and flat `VC_AUDIO` `feedDrops`/`starved`
  counters (the render callback now paces the mix and the 20 ms capture handoff, and a watchdog
  rebuilds a graph that stops calling back). Take a Siri or phone-call interruption while
  backgrounded and confirm audio resumes without foregrounding. Complete a
  30-minute iOS call and Wi-Fi/cellular switching with voice restoration (the switch is covered
  by simulation in `NetworkImpairmentTests`; hardware confirms the real route change), plus extended
  mono/stereo/voice-chat switching while joined. Verify Windows desktop/per-app stereo sharing.
- Complete Developer ID signing/notarization. The iOS host and ReplayKit extension have been
  distribution-signed and packaged locally; upload the IPA for Apple's server-side validation.
- Run the published Linux container and a 30-minute-or-longer server soak.

## Working rule

Keep this file short. It records only current state and open release gates. Git history is the
implementation diary.