voice-cat/PROGRESS.md

PROGRESS — VoiceCat

Living status. Update this file in the same commit as your work so the next agent picks up instantly. Newest status at the top.

• Date convention: ISO (YYYY-MM-DD).
• Statuses: [ ] not started · [~] in progress · [x] done.

---

▶ Where we left off / next action

• Done: M2 — voice, single stream ✓ complete (2026-06-16), now genuinely satisfied through the real client library, not just test_m2_voice's raw-socket harness. vc_stream_start/stop, UDP binding, capture→encode→seal→send, and recv→open→decode→ playback were all VC_ERR_NOT_IMPLEMENTED stubs in core/src/core/client.cpp even after test_m2_voice went green — meaning vccli/any GUI client still couldn't actually talk. Implemented for real this session, plus server-side StreamInfo broadcast (SessionRegistry::set_user_stream/clear_user_stream → UserEvent::UPDATED) so a second client's sync_remote_streams() learns about a peer's stream without polling. ctest --test-dir build/m1-dev — 10/10 tests green, including the new test_voice_client_abi (two real vc_client instances, not raw sockets, drive the full UDP-binding → stream-announce → cross-client STREAM_STARTED/STOPPED event path). vccli --voice (new flag, alongside --host/--port/--nick/--channel/--mute/--text) was manually verified live: two instances see each other's mic-stream start/stop in real time. Note: test_m1_integration and test_m2_voice have a pre-existing intermittent flake on Windows in their cleanup paths (thread-join / SQLite-file-handle release race, unrelated to this session's changes) — rerun in isolation if one fails standalone in the full suite.
• Next: M3 — multi-stream & per-channel tuning (screen audio, listener-side per-user NR, jitter buffer stats API — note voicecat.h currently exposes no stats getter beyond on_level's RMS meter, so a stats API needs new C ABI surface). See docs/roadmap.md §M3.

---

Milestones (see docs/roadmap.md for full detail)

• M0 — Scaffolding ✓ complete
• M1 — Control plane ✓ complete (2026-06-15)
• M2 — Voice, single stream ✓ complete (2026-06-16)
• M3 — Multi-stream & per-channel tuning ← next
• M3 — Multi-stream & per-channel tuning (screen audio, listener-side per-user NR)
• M4 — Native clients (Windows C#, macOS/iOS Swift)
• M5 — Moderation, polish, beyond (perms, bans, DRED; then file transfer, E2EE, …)

---

M0 — Scaffolding ✓ (completed)

• Repo layout (core/ server/ tools/ clients/ tests/), CMake + presets, vcpkg manifest.
• C ABI header core/include/voicecat.h (full surface, stubbed).
• Protocol source-of-truth core/proto/voicecat.proto (matches docs/protocol.md).
• Core stubs for all six subsystems (net/crypto/codec/protocol/session/audio) + vc_client.
• voicecat-server (arg parsing, config, stub run) and vccli (drives the C ABI).
• CTest smoke test asserting the C ABI contract (not just "it compiles").
• .gitattributes (LF), .gitignore, .clang-format, onboarding docs.
• Verified: cmake --preset dev && cmake --build --preset dev && ctest --preset dev → green.

---

M1 — Control plane ✓ (completed 2026-06-15)

Exit criterion: ✓ test_m1_integration — two clients authenticate over TLS 1.3 (guest

• Argon2id password), exchange channel and private text messages. Passes in ~1 s.

• vcpkg baseline + m1-dev preset; find_package for protobuf/mbedTLS/libsodium/asio/sqlite3.
• FrameCodec feed + emit; encode_envelope / decode_envelope.
• Asio TCP acceptor + TcpServerConn (TLS path: blocking handshake thread + tls_read_loop).
• TlsContext (mbedTLS 1.3, server cert/identity, ECDSA-P256 self-signed, TOFU on client).
• WorkerPool (3 threads, used for Argon2id).
• Database — SQLite, Argon2id via libsodium, create_account / authenticate / bootstrap admin.
• voicecat-admin — account add/reset/del/list against live DB file.
• ServerIdentityManager — generate/persist Ed25519 key + cert; fingerprint display.
• ConnSession — WaitingHello → WaitingAuth → Authenticated state machine; full protocol relay.
• SessionRegistry — channel tree, user map, broadcast, text routing.
• vc_client (client.cpp) — full M1 C ABI: connect/TLS/ClientHello/AuthRequest/text/disconnect.
• Server::run() — io_context, acceptor, worker pool, signal handling, on_ready callback.
• test_m1_integration — M1 exit criterion. Verified green 2026-06-15.

Key bug fixed: double-framing in ConnSession::send_envelope — encode_envelope was pre-framing the protobuf, then TcpServerConn::send_frame re-framed it. Fixed by serializing raw protobuf bytes directly and letting send_frame add the single [4-byte len] prefix.

---

---

M2 — Voice, single stream ✓ (completed 2026-06-16)

Exit criterion: ✓ test_m2_voice — two headless clients authenticate over TLS, bind UDP, announce a MIC stream, send 50 encrypted Opus frames; server SFU relay re-encrypts + forwards to the second client; B receives ≥ 25 frames and all decrypt correctly. Passes in ~4 s.

• m2-dev preset (inherits vcpkg-base, binaryDir build/m2-dev); m1-dev also builds all M2 code.
• core/CMakeLists.txt — find_package(Opus), find_path(MINIAUDIO_INCLUDE_DIR).
• core/src/net/voice_frame.h — 14-byte UDP header (type/flags/codec/ssrc/seq/ts), serialize/parse, make_udp_binding_packet.
• SodiumMediaCrypto — ChaCha20-Poly1305 AEAD; counter-nonce; 64-bit sliding-window anti-replay; derive_send/recv from TLS RFC 5705 exporter.
• OpusEncoder / OpusDecoder — libopus 1.6, FEC, DTX, PLC (free; nullptr → decoder extrapolates).
• UdpMediaChannel — async UDP socket (asio); thread-safe send_to; async recv loop.
• JitterBuffer — per-ssrc, EWMA jitter estimation, adaptive depth 20–200 ms, late-drop at 500 ms.
• AudioEngine — miniaudio capture+playback; inject_capture() bypass for headless tests; per-ssrc RemoteStream with OpusDecoder + JitterBuffer.
• ApmProcessor — ApmPassthrough stub (VAD always open); WebRTC APM deferred until M3.
• on_tls_ready callback in TcpChannelCallbacks — server derives and stores media AEAD keys immediately after TLS handshake.
• ConnSession M2 — udp_token generated at construction; included in AuthResult; handle_udp_binding (verifies token, TCP ack); handle_stream_announce (assigns SSRC via registry); udp_media_port in ServerHello.
• SessionRegistry M2 — register_udp_token, find_by_udp_token, register_udp_endpoint, find_by_udp_endpoint, assign_ssrc, find_channel_sessions, user_channel.
• MediaRelay — SFU UDP relay; kFrameUdpBinding → endpoint binding; kFrameVoice → decrypt/re-encrypt/forward to channel members.
• Server::run() — creates and binds MediaRelay; passes media port to ConnSession; wires on_tls_ready to derive per-connection media AEAD keys.
• test_voice_frame — header round-trip, big-endian layout, binding packet format.
• test_media_aead — seal/open round-trip, anti-replay, tamper detection, multi-packet sequence.
• test_opus_codec — encode/decode round-trip energy check (within 3 dB), PLC, frame-samples helper.
• test_m2_voice — M2 exit criterion (raw-socket harness). Verified green 2026-06-16.

Follow-up (same day): the above made test_m2_voice pass, but vc_client's public voice methods were still stubs — the actual M2 exit criterion ("two vccli/early-GUI clients talk") wasn't met. Closed the gap:

• core/src/core/client.cpp — real stream_start/stream_stop/set_self_mute/ set_remote_stream; UDP-binding handshake (start_udp_binding/handle_udp_binding_ack/ finish_udp_binding); media key derivation from tls_ (RFC 5705 exporter); run_udp_recv (AEAD-open → JitterBuffer::Frame → audio_engine_.push_recv_frame); on_capture_frame (encode → seal → sendto); sync_remote_streams (diffs a User proto's streams against remote_streams_, wiring up OpusDecoders and emitting STREAM_STARTED/STOPPED). set_input_device/set_input_mode/set_push_to_talk/list_devices remain VC_ERR_NOT_IMPLEMENTED — no device-enumeration backend yet; scoped to M3 (VAD/PTT).
• core/src/session/session.cpp/h — SessionModel::find_user, find_user_by_ssrc, Stream{stream_id, ssrc, kind, label, sample_rate, frame_ms}.
• server/src/conn_session.cpp/h — handle_stream_announce/handle_stream_stop now broadcast via SessionRegistry::set_user_stream/clear_user_stream → UserEvent::UPDATED.
• server/src/session_registry.cpp/h — set_user_stream/clear_user_stream (mutate a user's StreamInfo list, return the updated User proto for broadcast).
• tests/test_voice_client_abi.cpp — drives two real vc_client instances through vc_connect/vc_authenticate_guest/vc_stream_start/vc_stream_stop; asserts client B observes client A's STREAM_STARTED/STOPPED events. Verified green 2026-06-16.
• tools/vccli/src/main.cpp — argv parsing (--host/--port/--nick/--channel/--voice/ --mute/--text); --voice starts a MIC stream and blocks on SIGINT, printing on_event callbacks live (unbuffered stdout — MinGW/MSVCRT treat _IOLBF as full buffering for non-console streams). Dropped the originally-planned --voice-loopback and the tx=N rx=M lost=K jitter=J stats line: voicecat.h exposes no PCM-injection hook or jitter/loss stats getter publicly, only on_event + on_level (RMS). Manually verified: two vccli --voice instances see each other's stream start in real time.

---

Decisions log

All architecture/scope decisions are settled and recorded in docs/roadmap.md §2 "Resolved decisions" and reflected across docs/. If you make a new decision, record it there and link it here.

---

How to update this file

1. Check off tasks as you complete them; flip a milestone to [x] only when its exit criterion test passes.
2. Keep the "Where we left off / next action" block at the top accurate — it's the first thing the next agent reads.
3. When you start a milestone, copy its task list from docs/roadmap.md into a section here.