feat(M2): UDP voice/media plane -- SFU relay, Opus, AEAD, jitter buffer
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305
AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX,
an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an
APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain
wired through ConnSession/SessionRegistry into a new server-side SFU
(MediaRelay) that decrypts and re-encrypts frames per channel member.
Exit criterion verified: test_m2_voice — two headless clients relay 50
encrypted Opus frames through the server; ctest --preset m1-dev is 9/9
green. Also corrects protocol.md's UdpBinding diagram, which described the
UDP-side binding packet as AEAD-sealed when it is in fact a plaintext
bootstrap frame (separate from the TCP/TLS UdpBinding ack).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 01:31:14 +02:00
|
|
|
/*
|
|
|
|
|
* server/media_relay.h — UDP SFU relay for M2 voice.
|
|
|
|
|
*
|
|
|
|
|
* Design: docs/architecture.md §5, docs/voice.md §2.
|
|
|
|
|
* Receives encrypted UDP voice frames from clients, decrypts+authenticates them,
|
|
|
|
|
* re-encrypts for each channel member, and forwards.
|
|
|
|
|
*
|
|
|
|
|
* Flow:
|
|
|
|
|
* 1. Client sends kFrameUdpBinding UDP packet (plaintext) → MediaRelay looks up
|
|
|
|
|
* the 16-byte token in SessionRegistry, associates the sender endpoint with
|
|
|
|
|
* the ConnSession, and calls session->set_udp_endpoint().
|
|
|
|
|
* 2. Client sends kFrameVoice UDP packets → MediaRelay decrypts via recv_crypto(),
|
|
|
|
|
* finds channel members via find_channel_sessions(), re-encrypts via send_crypto(),
|
|
|
|
|
* and sends to each member's UDP endpoint.
|
|
|
|
|
*/
|
|
|
|
|
#ifndef VOICECAT_SERVER_MEDIA_RELAY_H
|
|
|
|
|
#define VOICECAT_SERVER_MEDIA_RELAY_H
|
|
|
|
|
|
|
|
|
|
#ifdef VOICECAT_HAS_NET
|
|
|
|
|
|
|
|
|
|
#include <memory>
|
|
|
|
|
|
|
|
|
|
#define ASIO_STANDALONE 1
|
|
|
|
|
#include <asio.hpp>
|
|
|
|
|
|
|
|
|
|
#include "net/transport.h"
|
|
|
|
|
|
|
|
|
|
namespace voicecat::server {
|
|
|
|
|
|
|
|
|
|
class SessionRegistry;
|
|
|
|
|
|
|
|
|
|
class MediaRelay {
|
|
|
|
|
public:
|
|
|
|
|
MediaRelay(asio::io_context& io, std::shared_ptr<SessionRegistry> registry);
|
|
|
|
|
~MediaRelay();
|
|
|
|
|
|
|
|
|
|
// Bind the UDP socket. port=0 lets the OS pick. Must be called before start().
|
|
|
|
|
bool bind(uint16_t port = 0);
|
|
|
|
|
|
|
|
|
|
// Begin async receive loop. Call once after bind().
|
|
|
|
|
void start();
|
|
|
|
|
|
|
|
|
|
// Stop receiving and close the socket.
|
|
|
|
|
void stop();
|
|
|
|
|
|
|
|
|
|
// Actual bound port (after bind()).
|
|
|
|
|
uint16_t media_port() const;
|
|
|
|
|
|
|
|
|
|
private:
|
|
|
|
|
void on_udp_frame(const uint8_t* data, size_t len, asio::ip::udp::endpoint sender);
|
|
|
|
|
|
2026-06-21 17:45:28 +02:00
|
|
|
// Count a dropped inbound voice frame (by reason) and emit a rate-limited summary
|
|
|
|
|
// to stderr. Runs on the io thread, so plain counters are safe.
|
|
|
|
|
void note_drop(const char* reason);
|
|
|
|
|
|
feat(M2): UDP voice/media plane -- SFU relay, Opus, AEAD, jitter buffer
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305
AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX,
an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an
APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain
wired through ConnSession/SessionRegistry into a new server-side SFU
(MediaRelay) that decrypts and re-encrypts frames per channel member.
Exit criterion verified: test_m2_voice — two headless clients relay 50
encrypted Opus frames through the server; ctest --preset m1-dev is 9/9
green. Also corrects protocol.md's UdpBinding diagram, which described the
UDP-side binding packet as AEAD-sealed when it is in fact a plaintext
bootstrap frame (separate from the TCP/TLS UdpBinding ack).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 01:31:14 +02:00
|
|
|
asio::io_context& io_;
|
|
|
|
|
std::shared_ptr<SessionRegistry> registry_;
|
|
|
|
|
voicecat::net::UdpMediaChannel udp_;
|
|
|
|
|
|
2026-06-21 17:45:28 +02:00
|
|
|
// Diagnostics: dropped-frame counters so a wedged media path is observable.
|
|
|
|
|
uint64_t drop_no_endpoint_ = 0; // voice from an unmapped UDP endpoint
|
|
|
|
|
uint64_t drop_no_crypto_ = 0; // session has no recv_crypto yet
|
|
|
|
|
uint64_t drop_open_failed_ = 0; // AEAD auth failure or replay reject
|
|
|
|
|
int64_t last_drop_log_ms_ = 0;
|
|
|
|
|
|
feat(M2): UDP voice/media plane -- SFU relay, Opus, AEAD, jitter buffer
Adds the full voice pipeline: 14-byte binary frame header, ChaCha20-Poly1305
AEAD keyed from the TLS exporter, libopus encode/decode with FEC/PLC/DTX,
an adaptive per-ssrc jitter buffer, a miniaudio capture/playback engine, an
APM passthrough stub, and the UdpBinding/StreamAnnounce signaling chain
wired through ConnSession/SessionRegistry into a new server-side SFU
(MediaRelay) that decrypts and re-encrypts frames per channel member.
Exit criterion verified: test_m2_voice — two headless clients relay 50
encrypted Opus frames through the server; ctest --preset m1-dev is 9/9
green. Also corrects protocol.md's UdpBinding diagram, which described the
UDP-side binding packet as AEAD-sealed when it is in fact a plaintext
bootstrap frame (separate from the TCP/TLS UdpBinding ack).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 01:31:14 +02:00
|
|
|
// Scratch buffer for re-encrypted payloads (size = max_frame + 16 MAC)
|
|
|
|
|
static constexpr size_t kMaxPayload = 1500;
|
|
|
|
|
std::vector<uint8_t> seal_buf_ = std::vector<uint8_t>(kMaxPayload + 16, uint8_t{0});
|
|
|
|
|
std::vector<uint8_t> plain_buf_ = std::vector<uint8_t>(kMaxPayload, uint8_t{0});
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
} // namespace voicecat::server
|
|
|
|
|
|
|
|
|
|
#endif // VOICECAT_HAS_NET
|
|
|
|
|
#endif // VOICECAT_SERVER_MEDIA_RELAY_H
|