Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(M2): UDP voice/media plane -- SFU relay, Opus, AEAD, jitter buffer

Browse Source 
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305
AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX,
an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an
APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain
wired through ConnSession/SessionRegistry into a new server-side SFU
(MediaRelay) that decrypts and re-encrypts frames per channel member.

Exit criterion verified: test_m2_voice — two headless clients relay 50
encrypted Opus frames through the server; ctest --preset m1-dev is 9/9
green. Also corrects protocol.md's UdpBinding diagram, which described the
UDP-side binding packet as AEAD-sealed when it is in fact a plaintext
bootstrap frame (separate from the TCP/TLS UdpBinding ack).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-16 01:31:14 +02:00
parent 63f457fc54
commit 694494a5be
29 changed files with 2548 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
server/src/server.cpp
View File

@@ -15,6 +15,7 @@
#include "crypto/crypto.h"
#include "db.h"
#include "identity.h"
#include "media_relay.h"
#include "net/transport.h"
#include "session_registry.h"
@@ -62,6 +63,16 @@ int Server::run() {
// ── Asio io_context ──────────────────────────────────────────────────────
asio::io_context io;
// ── UDP media relay (M2) ─────────────────────────────────────────────────
auto media_relay = std::make_shared<MediaRelay>(io, registry);
if (!media_relay->bind(cfg_.media_port)) {
std::fprintf(stderr, "[server] failed to bind UDP media port %u\n", cfg_.media_port);
return 1;
}
media_relay->start();
uint16_t media_bound = media_relay->media_port();
if (cfg_.on_media_ready) cfg_.on_media_ready(media_bound);
// Capture all locals by reference for the factory lambda (io lifetime is > factory)
voicecat::net::TcpAcceptor acceptor(
io, cfg_.bind_port,
@@ -69,7 +80,8 @@ int Server::run() {
auto session = std::make_shared<ConnSession>(
db, registry, workers,
id_mgr.identity().fingerprint,
cfg_.allow_guests);
cfg_.allow_guests,
media_bound);
// Use shared_ptr (not weak_ptr) so TcpServerConn keeps ConnSession alive.
// cycle is broken by weak_tcp in the send/close fns below.
@@ -83,6 +95,12 @@ int Server::run() {
cbs.on_error = [session](std::error_code) {
session->on_disconnect();
};
// Derive media keys right after TLS handshake (server is not the client).
cbs.on_tls_ready = [session](voicecat::crypto::TlsContext& tls) {
auto send = voicecat::crypto::SodiumMediaCrypto::derive_send(tls, false);
auto recv = voicecat::crypto::SodiumMediaCrypto::derive_recv(tls, false);
if (send && recv) session->set_media_crypto(std::move(send), std::move(recv));
};
// Create a TLS context for this connection (server role).
auto tls = std::make_unique<voicecat::crypto::TlsContext>(
@@ -126,11 +144,12 @@ int Server::run() {
signals.async_wait([&](std::error_code, int sig) {
std::printf("\n[server] signal %d — shutting down\n", sig);
acceptor.stop();
media_relay->stop();
io.stop();
});
std::printf("[voicecat-server] %s — listening on :%u\n",
cfg_.server_name.c_str(), bound);
std::printf("[voicecat-server] %s — TCP :%u UDP :%u\n",
cfg_.server_name.c_str(), bound, media_bound);
std::printf("[voicecat-server] fingerprint: %s\n",
id_mgr.fingerprint_display().c_str());