voice-cat/docs/roadmap.md

# Roadmap & Open Questions

## 1. Milestones

Each milestone is shippable/testable on its own. The headless C++ test client (`vccli`)
exists from M1 so the protocol can be exercised long before any GUI.

### M0 — Scaffolding
- Repo layout (see architecture.md §6), CMake + vcpkg manifest, CI matrix.
- `core/proto/` skeleton; `protoc` codegen wired for C++ (C#/Swift later).
- Empty `libvoicecat` with the C ABI header and stub implementations.
- **Exit:** core + server + `vccli` compile and link on Linux/macOS/Windows.

### M1 — Control plane (TCP/TLS, no audio yet)
- TLS 1.3 transport; framing; Envelope; ClientHello/ServerHello negotiation.
- Auth: **guest + admin-provisioned local account** (Argon2id, SQLite); server identity
  (TOFU/Ed25519); `voicecat-admin` account add/reset/del/list.
- Channel tree: snapshot + deltas; join/leave; create/edit/delete (perm-checked).
- **Text chat (ephemeral):** channel + private messages, acks; live relay, no history store.
- `vccli` can connect, auth, browse channels, and chat.
- **Exit:** two `vccli` instances chat through a real server over TLS.

### M2 — Voice, single stream
- UDP transport with exported-key + ChaCha20-Poly1305 AEAD (mandatory, no plaintext path);
  UDP token binding; anti-replay.
- miniaudio capture/playback; libopus encode/decode; one `MIC` stream per user.
- **Send-side webrtc APM** (AEC + NS + AGC + VAD) and a **VAD/PTT input gate** (both modes,
  client-configurable) — AEC is in from the start, not deferred.
- Per-ssrc adaptive jitter buffer; mixer; FEC/PLC/DTX.
- Per-channel `AudioConfig` enforced (incl. server `max_bitrate_bps` ceiling); SFU relay.
- **Exit:** talk between two `vccli`/early-GUI clients in a channel; loss resilience visible.

### M3 — Multi-stream & per-channel tuning
- Multiple concurrent streams per user (`MIC`, `SCREEN_AUDIO`, `AUX_DEVICE`).
- Per-stream receiver gain/mute; **listener-side per-user noise reduction** (APM NS on the
  receive path, per ssrc, local-only); talk indicators.
- Full per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX).
- **Exit:** a user shares mic + desktop audio; listeners control each independently.

### M4 — Native clients

**Windows (C#/WinForms, .NET 10) ✓ complete 2026-06-17:**
- Connect, saved-server list (JSON, DPAPI-encrypted passwords), TOFU identity dialog.
- Channel tree (`TreeView`), user list, join (incl. password-protected channels).
- Voice: mic start/stop, mute/deafen, VAD/PTT/**always-on** mode, **VAD sensitivity slider**
  (live threshold update via `vc_set_vad_threshold`), device picker, level meter.
- Per-user gain/mute/NR tuning (`PerUserTuningDialog`).
- Channel + private text chat. Activity log (screen-reader primary path).
- Explicit `AccessibleName`/`AccessibleDescription` on every control; `&` mnemonics;
  `AutomationNotification` curated live announcements.
- Focus-scoped PTT (documented limitation — no system-wide hook in v1).
- Admin/moderation UI **out of scope** — needs server-side dispatch first (M5).

**macOS (Swift/AppKit) — in progress:**
- Same feature set as Windows over the same C ABI (now stable and complete).
- **Shared Swift core (`VoiceCatCore` package) ✓ complete 2026-06-18** — wraps all 38 C ABI
  functions; 6/6 XCTest smoke tests pass against a real server (connect → TOFU → auth →
  channels → moderation → admin CRUD → per-stream recv controls). See
  `clients/apple/README.md`.
- **UI: AppKit** (not SwiftUI) — chosen for the most mature VoiceOver accessibility story
  (per-control `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`,
  `NSAccessibility.post(.announcement)` for live announcements). Same rationale as the
  Windows client's WinForms-over-WinUI-3 decision (see §2 below). macOS 14 (Sonoma)
  deployment target.
- AVAudioSession not needed on macOS (CoreAudio via the core directly).

**iOS (Swift/SwiftUI) — pending:**
- SwiftUI app consuming the same `VoiceCatCore` package.
- ~~AVAudioSession, mic permission, foreground voice.~~ ✓ Done — `IOSAudioRouter` drives
  all iOS audio routing (input ports, orientation/polar patterns, HFP/A2DP, Standard/Raw
  mic mode, stereo capture), `vc_audio_suspend`/`vc_audio_resume` for interruptions.
- ReplayKit broadcast extension for `SCREEN_AUDIO` — feeds `CMSampleBuffer` audio via
  `vc_stream_feed_pcm` (see architecture.md §4).

**Exit:** non-technical user installs a client, saves a server, and joins.

### M5 — Moderation, polish, and beyond
- Permissions/roles, kick/ban/server-mute, channel passwords UI. Windows WinForms UI complete
  (channel CRUD with full Opus config, user moderation, server account management); macOS/iOS
  Swift UI pending.
- DRED toggle, audio-quality polish. (AEC and VAD/PTT already shipped in M2.)
- **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) ✓ shipped
  (2026-06-20) — promotes `vc_test_inject_capture` to a public, stereo-capable API and adds
  a symmetric PCM sink. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, and custom
  clients. See architecture.md §4 "External PCM feed/tap" and protocol.md §8 for the
  no-protocol-change rationale. One new C++ ctest binary (`test_external_pcm`) covering 3
  sub-tests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`, `test_pcm_sink`) — ctest
  23/23; Swift wrapper + 4 XCTest smoke tests; C# wrapper + 4 xUnit smoke tests.
- **Then (post-v1, protocol already reserves space):** file transfer, E2EE option,
  CallKit/PushKit background voice, key-based identity, server-side text history,
  multi-node server.

## 2. Resolved decisions

Settled and reflected throughout the docs:

- **Media crypto:** exported-key + ChaCha20-Poly1305 AEAD from day one, **mandatory** — no
  DTLS, no plaintext mode. (security.md §2)
- **TLS library / licensing:** **mbedTLS** (Apache-2.0) + **libsodium** (ISC). **No GPL/LGPL**
  anywhere; code is redistributable closed-source. wolfSSL is rejected. (tech-stack.md §5)
- **iOS screen/system audio:** supported via a ReplayKit Broadcast Upload Extension
  (`.audioApp`); audio-only stays within the extension memory cap. (voice.md §9)
- **Self-host UX:** zero-config, encrypted-by-default; Docker / single binary / source build.
  (deployment.md)
- **DSP engine:** **webrtc-audio-processing (APM)** — AEC in from the start, plus NS/AGC/VAD.
  (voice.md §8)
- **Input activation:** **VAD *and* PTT**, both modes client-configurable. (voice.md §11)
- **Noise reduction is two-sided:** sender can denoise its mic, *and* each listener can apply
  NS to a **specific** other user, locally, with no protocol traffic. (voice.md §10)
- **Accounts:** **admin-provisioned** (no self-serve registration) via `voicecat-admin` or the
  in-app admin interface. (security.md §4, protocol.md §3, deployment.md §3a)
- **Text:** **ephemeral** — live relay, no server-side history in v1. (protocol.md §5)
- **Connect UX:** pure direct-connect with a **client-side saved-server list** (no central
  directory). (deployment.md §3)
- **Bitrate ceiling:** server-config `opus.limits.max_bitrate_bps`. (deployment.md §2)
- **Name:** "VoiceCat" stays as the internal placeholder.
- **DSP engine, superseded (2026-06-16):** the "webrtc-audio-processing (APM)" decision above
  (AEC + NS/AGC/VAD in one module) could not be carried out — it has no working Windows/MSVC
  build upstream (GCC-only Meson build, MinGW support unfinished, hard `abseil-cpp` dependency,
  Linux-tested only). v1 ships a lightweight, dependency-free energy/RMS VAD instead, behind
  the same `ApmProcessor` interface; there is **no AEC/NS/AGC implementation at all** yet. Real
  `webrtc-audio-processing` stays a tracked future swap (e.g. if/when a Linux build target
  exists). (voice.md §8, §11)
- **Windows client UI framework (2026-06-17):** **WinForms** (.NET 10), not WinUI 3 or
  Avalonia. Reason: Win32 HWND controls have the most mature, predictable screen-reader
  (NVDA/JAWS/Narrator) support of any current .NET UI stack. WinUI 3's accessibility UIA
  tree has known rough edges on .NET 10; Avalonia's accessibility story is thinner still.
  This overrides the earlier WinUI/Avalonia mention in `docs/tech-stack.md §2` and
  `docs/architecture.md §4`. (clients/windows/)
- **TOFU pins TLS leaf cert, not Ed25519 (2026-06-17):** the original design said to pin the
  server's declared Ed25519 identity fingerprint from `ServerHello`. This is circular — the
  Ed25519 key and the TLS cert are generated independently with no cryptographic binding, so
  accepting/rejecting based on a value sent *inside* the channel being trust-decided is
  meaningless. **Decision:** pin the TLS leaf certificate's own SHA-256 fingerprint, which is
  verifiable directly from the TLS handshake before any application data is trusted. The Ed25519
  value is still shown in the identity dialog for human-readable display only (informational).
  See `docs/security.md §1.1`. (core/src/crypto/tofu_store.*, vc_confirm_server_identity)
- **PTT is focus-scoped in v1 (2026-06-17):** PTT hotkey capture uses `Form.KeyDown`/`KeyUp`
  (works only while the VoiceCat window has focus), not a system-wide `WH_KEYBOARD_LL` hook.
  Reason: a low-level keyboard hook requires escalated permissions, risks AV flagging, and is
  disproportionate complexity for a v1 client. Documented in the UI as a known limitation.
  Can be revisited for v2 if users request it. (clients/windows/VoiceCat.App/Forms/MainForm.cs)
- **macOS client UI framework (2026-06-18):** **AppKit**, not SwiftUI. Reason: AppKit has the
  most mature, granular **VoiceOver** accessibility story on macOS — per-control
  `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`, `NSAccessibility.post(.announcement)`
  for curated live announcements, and decades of real-world screen-reader usage. This is the
  same rationale that drove the Windows client to WinForms over WinUI 3 (screen-reader
  support is the deciding factor). SwiftUI's VoiceOver support has improved but still has
  gaps in complex AppKit-bridged control surfaces (outline views, data-table column
  headers, live-region announcements). iOS stays SwiftUI — its control surface is narrower
  and SwiftUI's VoiceOver support is sufficient there. This overrides the earlier
  "SwiftUI for both macOS + iOS" mention in `docs/tech-stack.md §2` and `docs/architecture.md
  §4`. macOS 14 (Sonoma) deployment target. (clients/apple/)

## 3. Open questions

All initial open questions are resolved (§2). Two **second-order considerations** to keep in
mind during implementation — not blockers:

- **APM in constrained contexts.** webrtc-audio-processing is a heavier build; confirm it
  static-links cleanly for the single-binary goal, and note the iOS **broadcast extension only
  does Opus encode + send** (no APM), so it stays under the ~50 MB cap. Listener-side per-user
  NS runs only in the full host app.
- **Receive-side NR cost at scale.** A per-ssrc APM NS instance per flagged user adds CPU on
  busy channels; instantiate lazily (only for flagged streams) and cap concurrent instances.

## 4. What's intentionally deferred

To keep v1 focused (voice + text), these are designed-for but not built: file transfer,
end-to-end encryption, key-based identities, multi-node/federated servers, mobile
background VoIP push, and any server-side audio mixing/transcoding. The protocol's
versioning + feature negotiation + reserved tag ranges (protocol.md §8) ensure each can be
added without breaking deployed clients.