voice-cat/docs/roadmap.md

Roadmap & Open Questions

1. Milestones

Each milestone is shippable/testable on its own. The headless C++ test client (vccli) exists from M1 so the protocol can be exercised long before any GUI.

M0 — Scaffolding

• Repo layout (see architecture.md §6), CMake + vcpkg manifest, CI matrix.
• core/proto/ skeleton; protoc codegen wired for C++ (C#/Swift later).
• Empty libvoicecat with the C ABI header and stub implementations.
• Exit: core + server + vccli compile and link on Linux/macOS/Windows.

M1 — Control plane (TCP/TLS, no audio yet)

• TLS 1.3 transport; framing; Envelope; ClientHello/ServerHello negotiation.
• Auth: guest + admin-provisioned local account (Argon2id, SQLite); server identity (TOFU/Ed25519); voicecat-admin account add/reset/del/list.
• Channel tree: snapshot + deltas; join/leave; create/edit/delete (perm-checked).
• Text chat (ephemeral): channel + private messages, acks; live relay, no history store.
• vccli can connect, auth, browse channels, and chat.
• Exit: two vccli instances chat through a real server over TLS.

M2 — Voice, single stream

• UDP transport with exported-key + ChaCha20-Poly1305 AEAD (mandatory, no plaintext path); UDP token binding; anti-replay.
• miniaudio capture/playback; libopus encode/decode; one MIC stream per user.
• Send-side webrtc APM (AEC + NS + AGC + VAD) and a VAD/PTT input gate (both modes, client-configurable) — AEC is in from the start, not deferred.
• Per-ssrc adaptive jitter buffer; mixer; FEC/PLC/DTX.
• Per-channel AudioConfig enforced (incl. server max_bitrate_bps ceiling); SFU relay.
• Exit: talk between two vccli/early-GUI clients in a channel; loss resilience visible.

M3 — Multi-stream & per-channel tuning

• Multiple concurrent streams per user (MIC, SCREEN_AUDIO, AUX_DEVICE).
• Per-stream receiver gain/mute; listener-side per-user noise reduction (APM NS on the receive path, per ssrc, local-only); talk indicators.
• Full per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX).
• Exit: a user shares mic + desktop audio; listeners control each independently.

M4 — Native clients

• Windows (C#/WinUI): connect, saved-server list, channel tree, voice, text, device pickers, meters, VAD/PTT + per-user NR controls.
• macOS (Swift/SwiftUI): same.
• iOS (Swift): AVAudioSession integration, mic permission, foreground voice; ReplayKit broadcast extension for SCREEN_AUDIO.
• In-app admin interface (account provisioning, bans) for admin users.
• Exit: non-technical user installs a client, saves a server, and joins.

M5 — Moderation, polish, and beyond

• Permissions/roles, kick/ban/server-mute, channel passwords UI.
• DRED toggle, audio-quality polish. (AEC and VAD/PTT already shipped in M2.)
• Then (post-v1, protocol already reserves space): file transfer, E2EE option, CallKit/PushKit background voice, key-based identity, server-side text history, multi-node server.

2. Resolved decisions

Settled and reflected throughout the docs:

• Media crypto: exported-key + ChaCha20-Poly1305 AEAD from day one, mandatory — no DTLS, no plaintext mode. (security.md §2)
• TLS library / licensing: mbedTLS (Apache-2.0) + libsodium (ISC). No GPL/LGPL anywhere; code is redistributable closed-source. wolfSSL is rejected. (tech-stack.md §5)
• iOS screen/system audio: supported via a ReplayKit Broadcast Upload Extension (.audioApp); audio-only stays within the extension memory cap. (voice.md §9)
• Self-host UX: zero-config, encrypted-by-default; Docker / single binary / source build. (deployment.md)
• DSP engine: webrtc-audio-processing (APM) — AEC in from the start, plus NS/AGC/VAD. (voice.md §8)
• Input activation: VAD and PTT, both modes client-configurable. (voice.md §11)
• Noise reduction is two-sided: sender can denoise its mic, and each listener can apply NS to a specific other user, locally, with no protocol traffic. (voice.md §10)
• Accounts: admin-provisioned (no self-serve registration) via voicecat-admin or the in-app admin interface. (security.md §4, protocol.md §3, deployment.md §3a)
• Text: ephemeral — live relay, no server-side history in v1. (protocol.md §5)
• Connect UX: pure direct-connect with a client-side saved-server list (no central directory). (deployment.md §3)
• Bitrate ceiling: server-config opus.limits.max_bitrate_bps. (deployment.md §2)
• Name: "VoiceCat" stays as the internal placeholder.

3. Open questions

All initial open questions are resolved (§2). Two second-order considerations to keep in mind during implementation — not blockers:

• APM in constrained contexts. webrtc-audio-processing is a heavier build; confirm it static-links cleanly for the single-binary goal, and note the iOS broadcast extension only does Opus encode + send (no APM), so it stays under the ~50 MB cap. Listener-side per-user NS runs only in the full host app.
• Receive-side NR cost at scale. A per-ssrc APM NS instance per flagged user adds CPU on busy channels; instantiate lazily (only for flagged streams) and cap concurrent instances.

4. What's intentionally deferred

To keep v1 focused (voice + text), these are designed-for but not built: file transfer, end-to-end encryption, key-based identities, multi-node/federated servers, mobile background VoIP push, and any server-side audio mixing/transcoding. The protocol's versioning + feature negotiation + reserved tag ranges (protocol.md §8) ensure each can be added without breaking deployed clients.