Talon
694494a5be
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305 AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX, an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain wired through ConnSession/SessionRegistry into a new server-side SFU (MediaRelay) that decrypts and re-encrypts frames per channel member. Exit criterion verified: test_m2_voice — two headless clients relay 50 encrypted Opus frames through the server; ctest --preset m1-dev is 9/9 green. Also corrects protocol.md's UdpBinding diagram, which described the UDP-side binding packet as AEAD-sealed when it is in fact a plaintext bootstrap frame (separate from the TCP/TLS UdpBinding ack). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
116 lines
6.8 KiB
Markdown
116 lines
6.8 KiB
Markdown
# PROGRESS — VoiceCat
|
||
|
||
Living status. **Update this file in the same commit as your work** so the next agent picks
|
||
up instantly. Newest status at the top.
|
||
|
||
- **Date convention:** ISO (YYYY-MM-DD).
|
||
- Statuses: `[ ]` not started · `[~]` in progress · `[x]` done.
|
||
|
||
---
|
||
|
||
## ▶ Where we left off / next action
|
||
|
||
- **Done:** **M2 — voice, single stream** ✓ complete (2026-06-16).
|
||
`ctest --preset m1-dev` — all **9/9 tests** green including the new M2 exit criterion
|
||
(`test_m2_voice`): two headless clients encrypt Opus frames via ChaCha20-Poly1305, bind
|
||
UDP sockets, and the server SFU relay re-encrypts + forwards frames.
|
||
- **Next:** **M3 — multi-stream & per-channel tuning** (screen audio, listener-side per-user
|
||
NR, jitter buffer stats API). See `docs/roadmap.md §M3`.
|
||
|
||
---
|
||
|
||
## Milestones (see [docs/roadmap.md](docs/roadmap.md) for full detail)
|
||
|
||
- [x] **M0 — Scaffolding** ✓ complete
|
||
- [x] **M1 — Control plane** ✓ complete (2026-06-15)
|
||
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16)
|
||
- [ ] **M3 — Multi-stream & per-channel tuning** ← next
|
||
- [ ] **M3 — Multi-stream & per-channel tuning** (screen audio, listener-side per-user NR)
|
||
- [ ] **M4 — Native clients** (Windows C#, macOS/iOS Swift)
|
||
- [ ] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)
|
||
|
||
---
|
||
|
||
## M0 — Scaffolding ✓ (completed)
|
||
|
||
- [x] Repo layout (`core/ server/ tools/ clients/ tests/`), CMake + presets, vcpkg manifest.
|
||
- [x] C ABI header `core/include/voicecat.h` (full surface, stubbed).
|
||
- [x] Protocol source-of-truth `core/proto/voicecat.proto` (matches docs/protocol.md).
|
||
- [x] Core stubs for all six subsystems (net/crypto/codec/protocol/session/audio) + `vc_client`.
|
||
- [x] `voicecat-server` (arg parsing, config, stub run) and `vccli` (drives the C ABI).
|
||
- [x] CTest **smoke test** asserting the C ABI contract (not just "it compiles").
|
||
- [x] `.gitattributes` (LF), `.gitignore`, `.clang-format`, onboarding docs.
|
||
- **Verified:** `cmake --preset dev && cmake --build --preset dev && ctest --preset dev` → green.
|
||
|
||
---
|
||
|
||
## M1 — Control plane ✓ (completed 2026-06-15)
|
||
|
||
**Exit criterion:** ✓ `test_m1_integration` — two clients authenticate over TLS 1.3 (guest
|
||
+ Argon2id password), exchange channel and private text messages. Passes in ~1 s.
|
||
|
||
- [x] vcpkg baseline + `m1-dev` preset; `find_package` for protobuf/mbedTLS/libsodium/asio/sqlite3.
|
||
- [x] `FrameCodec` feed + emit; `encode_envelope` / `decode_envelope`.
|
||
- [x] Asio TCP acceptor + `TcpServerConn` (TLS path: blocking handshake thread + `tls_read_loop`).
|
||
- [x] `TlsContext` (mbedTLS 1.3, server cert/identity, ECDSA-P256 self-signed, TOFU on client).
|
||
- [x] `WorkerPool` (3 threads, used for Argon2id).
|
||
- [x] `Database` — SQLite, Argon2id via libsodium, `create_account` / `authenticate` / bootstrap admin.
|
||
- [x] `voicecat-admin` — account add/reset/del/list against live DB file.
|
||
- [x] `ServerIdentityManager` — generate/persist Ed25519 key + cert; fingerprint display.
|
||
- [x] `ConnSession` — WaitingHello → WaitingAuth → Authenticated state machine; full protocol relay.
|
||
- [x] `SessionRegistry` — channel tree, user map, broadcast, text routing.
|
||
- [x] `vc_client` (`client.cpp`) — full M1 C ABI: connect/TLS/ClientHello/AuthRequest/text/disconnect.
|
||
- [x] `Server::run()` — io_context, acceptor, worker pool, signal handling, `on_ready` callback.
|
||
- [x] `test_m1_integration` — M1 exit criterion. Verified green 2026-06-15.
|
||
|
||
**Key bug fixed:** double-framing in `ConnSession::send_envelope` — `encode_envelope` was
|
||
pre-framing the protobuf, then `TcpServerConn::send_frame` re-framed it. Fixed by serializing
|
||
raw protobuf bytes directly and letting `send_frame` add the single `[4-byte len]` prefix.
|
||
|
||
---
|
||
|
||
---
|
||
|
||
## M2 — Voice, single stream ✓ (completed 2026-06-16)
|
||
|
||
**Exit criterion:** ✓ `test_m2_voice` — two headless clients authenticate over TLS, bind UDP,
|
||
announce a MIC stream, send 50 encrypted Opus frames; server SFU relay re-encrypts + forwards
|
||
to the second client; B receives ≥ 25 frames and all decrypt correctly. Passes in ~4 s.
|
||
|
||
- [x] `m2-dev` preset (inherits `vcpkg-base`, binaryDir `build/m2-dev`); `m1-dev` also builds all M2 code.
|
||
- [x] `core/CMakeLists.txt` — `find_package(Opus)`, `find_path(MINIAUDIO_INCLUDE_DIR)`.
|
||
- [x] `core/src/net/voice_frame.h` — 14-byte UDP header (type/flags/codec/ssrc/seq/ts), serialize/parse, `make_udp_binding_packet`.
|
||
- [x] `SodiumMediaCrypto` — ChaCha20-Poly1305 AEAD; counter-nonce; 64-bit sliding-window anti-replay; `derive_send/recv` from TLS RFC 5705 exporter.
|
||
- [x] `OpusEncoder` / `OpusDecoder` — libopus 1.6, FEC, DTX, PLC (free; nullptr → decoder extrapolates).
|
||
- [x] `UdpMediaChannel` — async UDP socket (asio); thread-safe `send_to`; async recv loop.
|
||
- [x] `JitterBuffer` — per-ssrc, EWMA jitter estimation, adaptive depth 20–200 ms, late-drop at 500 ms.
|
||
- [x] `AudioEngine` — miniaudio capture+playback; `inject_capture()` bypass for headless tests; per-ssrc RemoteStream with OpusDecoder + JitterBuffer.
|
||
- [x] `ApmProcessor` — `ApmPassthrough` stub (VAD always open); WebRTC APM deferred until M3.
|
||
- [x] `on_tls_ready` callback in `TcpChannelCallbacks` — server derives and stores media AEAD keys immediately after TLS handshake.
|
||
- [x] `ConnSession` M2 — `udp_token` generated at construction; included in `AuthResult`; `handle_udp_binding` (verifies token, TCP ack); `handle_stream_announce` (assigns SSRC via registry); `udp_media_port` in `ServerHello`.
|
||
- [x] `SessionRegistry` M2 — `register_udp_token`, `find_by_udp_token`, `register_udp_endpoint`, `find_by_udp_endpoint`, `assign_ssrc`, `find_channel_sessions`, `user_channel`.
|
||
- [x] `MediaRelay` — SFU UDP relay; `kFrameUdpBinding` → endpoint binding; `kFrameVoice` → decrypt/re-encrypt/forward to channel members.
|
||
- [x] `Server::run()` — creates and binds `MediaRelay`; passes media port to `ConnSession`; wires `on_tls_ready` to derive per-connection media AEAD keys.
|
||
- [x] `test_voice_frame` — header round-trip, big-endian layout, binding packet format.
|
||
- [x] `test_media_aead` — seal/open round-trip, anti-replay, tamper detection, multi-packet sequence.
|
||
- [x] `test_opus_codec` — encode/decode round-trip energy check (within 3 dB), PLC, frame-samples helper.
|
||
- [x] `test_m2_voice` — M2 exit criterion. Verified green 2026-06-16.
|
||
|
||
---
|
||
|
||
## Decisions log
|
||
|
||
All architecture/scope decisions are settled and recorded in
|
||
[docs/roadmap.md §2 "Resolved decisions"](docs/roadmap.md) and reflected across `docs/`.
|
||
If you make a *new* decision, record it there and link it here.
|
||
|
||
---
|
||
|
||
## How to update this file
|
||
|
||
1. Check off tasks as you complete them; flip a milestone to `[x]` only when its **exit
|
||
criterion test** passes.
|
||
2. Keep the **"Where we left off / next action"** block at the top accurate — it's the first
|
||
thing the next agent reads.
|
||
3. When you start a milestone, copy its task list from `docs/roadmap.md` into a section here.
|