voice-cat/docs/roadmap.md

# Roadmap & Open Questions

## 1. Milestones

Each milestone is shippable/testable on its own. The headless C++ test client (`vccli`)
exists from M1 so the protocol can be exercised long before any GUI.

### M0 — Scaffolding
- Repo layout (see architecture.md §6), CMake + vcpkg manifest, CI matrix.
- `core/proto/` skeleton; `protoc` codegen wired for C++ (C#/Swift later).
- Empty `libvoicecat` with the C ABI header and stub implementations.
- **Exit:** core + server + `vccli` compile and link on Linux/macOS/Windows.

### M1 — Control plane (TCP/TLS, no audio yet)
- TLS 1.3 transport; framing; Envelope; ClientHello/ServerHello negotiation.
- Auth: **guest + admin-provisioned local account** (Argon2id, SQLite); server identity
  (TOFU/Ed25519); `voicecat-admin` account add/reset/del/list.
- Channel tree: snapshot + deltas; join/leave; create/edit/delete (perm-checked).
- **Text chat (ephemeral):** channel + private messages, acks; live relay, no history store.
- `vccli` can connect, auth, browse channels, and chat.
- **Exit:** two `vccli` instances chat through a real server over TLS.

### M2 — Voice, single stream
- UDP transport with exported-key + ChaCha20-Poly1305 AEAD (mandatory, no plaintext path);
  UDP token binding; anti-replay.
- miniaudio capture/playback; libopus encode/decode; one `MIC` stream per user.
- **Send-side webrtc APM** (AEC + NS + AGC + VAD) and a **VAD/PTT input gate** (both modes,
  client-configurable) — AEC is in from the start, not deferred.
- Per-ssrc adaptive jitter buffer; mixer; FEC/PLC/DTX.
- Per-channel `AudioConfig` enforced (incl. server `max_bitrate_bps` ceiling); SFU relay.
- **Exit:** talk between two `vccli`/early-GUI clients in a channel; loss resilience visible.

### M3 — Multi-stream & per-channel tuning
- Multiple concurrent streams per user (`MIC`, `SCREEN_AUDIO`, `AUX_DEVICE`).
- Per-stream receiver gain/mute; **listener-side per-user noise reduction** (APM NS on the
  receive path, per ssrc, local-only); talk indicators.
- Full per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX).
- **Exit:** a user shares mic + desktop audio; listeners control each independently.

### M4 — Native clients
- **Windows (C#/WinUI):** connect, saved-server list, channel tree, voice, text, device
  pickers, meters, VAD/PTT + per-user NR controls.
- **macOS (Swift/SwiftUI):** same.
- **iOS (Swift):** AVAudioSession integration, mic permission, foreground voice; ReplayKit
  broadcast extension for `SCREEN_AUDIO`.
- In-app **admin interface** (account provisioning, bans) for admin users.
- **Exit:** non-technical user installs a client, saves a server, and joins.

### M5 — Moderation, polish, and beyond
- Permissions/roles, kick/ban/server-mute, channel passwords UI.
- DRED toggle, audio-quality polish. (AEC and VAD/PTT already shipped in M2.)
- **Then (post-v1, protocol already reserves space):** file transfer, E2EE option,
  CallKit/PushKit background voice, key-based identity, server-side text history,
  multi-node server.

## 2. Resolved decisions

Settled and reflected throughout the docs:

- **Media crypto:** exported-key + ChaCha20-Poly1305 AEAD from day one, **mandatory** — no
  DTLS, no plaintext mode. (security.md §2)
- **TLS library / licensing:** **mbedTLS** (Apache-2.0) + **libsodium** (ISC). **No GPL/LGPL**
  anywhere; code is redistributable closed-source. wolfSSL is rejected. (tech-stack.md §5)
- **iOS screen/system audio:** supported via a ReplayKit Broadcast Upload Extension
  (`.audioApp`); audio-only stays within the extension memory cap. (voice.md §9)
- **Self-host UX:** zero-config, encrypted-by-default; Docker / single binary / source build.
  (deployment.md)
- **DSP engine:** **webrtc-audio-processing (APM)** — AEC in from the start, plus NS/AGC/VAD.
  (voice.md §8)
- **Input activation:** **VAD *and* PTT**, both modes client-configurable. (voice.md §11)
- **Noise reduction is two-sided:** sender can denoise its mic, *and* each listener can apply
  NS to a **specific** other user, locally, with no protocol traffic. (voice.md §10)
- **Accounts:** **admin-provisioned** (no self-serve registration) via `voicecat-admin` or the
  in-app admin interface. (security.md §4, protocol.md §3, deployment.md §3a)
- **Text:** **ephemeral** — live relay, no server-side history in v1. (protocol.md §5)
- **Connect UX:** pure direct-connect with a **client-side saved-server list** (no central
  directory). (deployment.md §3)
- **Bitrate ceiling:** server-config `opus.limits.max_bitrate_bps`. (deployment.md §2)
- **Name:** "VoiceCat" stays as the internal placeholder.

## 3. Open questions

All initial open questions are resolved (§2). Two **second-order considerations** to keep in
mind during implementation — not blockers:

- **APM in constrained contexts.** webrtc-audio-processing is a heavier build; confirm it
  static-links cleanly for the single-binary goal, and note the iOS **broadcast extension only
  does Opus encode + send** (no APM), so it stays under the ~50 MB cap. Listener-side per-user
  NS runs only in the full host app.
- **Receive-side NR cost at scale.** A per-ssrc APM NS instance per flagged user adds CPU on
  busy channels; instantiate lazily (only for flagged streams) and cap concurrent instances.

## 4. What's intentionally deferred

To keep v1 focused (voice + text), these are designed-for but not built: file transfer,
end-to-end encryption, key-based identities, multi-node/federated servers, mobile
background VoIP push, and any server-side audio mixing/transcoding. The protocol's
versioning + feature negotiation + reserved tag ranges (protocol.md §8) ensure each can be
added without breaking deployed clients.