voice-cat/docs/protocol.md

Control Protocol

The control plane runs over TCP, wrapped in TLS 1.3. It carries everything that is not real-time media: handshake, authentication, channel/user/presence state, text chat, and voice signaling (announcing that a media stream is starting/stopping). Real-time voice travels separately over UDP — see voice.md.

1. Framing

Inside the TLS stream, messages are length-prefixed:

┌──────────────┬───────────────────────────────────────────────┐
│  u32 length  │  protobuf-encoded Envelope (length bytes)      │
│  (big-endian)│                                                │
└──────────────┴───────────────────────────────────────────────┘

• length is the byte count of the payload that follows (not including the 4 length bytes). Hard cap (e.g. 16 MiB) to bound memory; oversized frame → protocol error + disconnect.
• The payload is a single Envelope protobuf message. We do not add our own type byte; the type is the oneof discriminator inside the Envelope, which keeps the framing trivial and lets protobuf own all forward/backward compatibility.

TLS already provides record framing, integrity, and ordering; we only add message boundaries on top of the TLS byte stream.

2. Why Protocol Buffers for control

• Schema-driven codegen for C++, C#, and Swift (all first-class) → no hand-rolled parsers, no drift between client and server.
• Forward/backward compatible by construction: unknown fields are preserved/ignored, new fields and new oneof arms are additive. This is exactly the "extensible protocol" requirement.
• Compact enough for a control plane (text/state, not media). We use oneof envelopes rather than Any so the wire stays tight and the switch is exhaustive.

Media frames do not use protobuf — they use a fixed binary header (see voice.md), because per-packet protobuf overhead and allocation are unacceptable on the RT path.

3. The Envelope

syntax = "proto3";
package voicecat.v1;

message Envelope {
  // Monotonic per-connection id set by the sender of a request; echoed in the
  // matching response so async callers can correlate. 0 for unsolicited events.
  uint64 request_id = 1;

  oneof body {
    // ── Session / handshake ───────────────────────────────
    ClientHello       client_hello        = 10;
    ServerHello       server_hello        = 11;
    AuthRequest       auth_request        = 12;
    AuthResult        auth_result         = 13;
    Disconnect        disconnect          = 14;
    Ping              ping                = 15;
    Pong              pong                = 16;

    // ── State sync ────────────────────────────────────────
    ServerStateSnapshot server_state      = 20;
    ChannelEvent      channel_event       = 21;   // created/updated/deleted
    UserEvent         user_event          = 22;   // joined/left/updated
    SubscribeRequest  subscribe           = 23;

    // ── Channel operations ────────────────────────────────
    JoinChannelRequest join_channel       = 30;
    JoinChannelResult  join_channel_result= 31;
    LeaveChannelRequest leave_channel     = 32;
    CreateChannelRequest create_channel   = 33;
    EditChannelRequest edit_channel       = 34;
    DeleteChannelRequest delete_channel   = 35;
    MoveUserRequest   move_user           = 36;
    GenericResult     generic_result      = 37;   // ack/err for the above

    // ── Voice signaling (media is on UDP) ─────────────────
    StreamAnnounce    stream_announce     = 40;
    StreamAnnounceResult stream_announce_result = 41;
    StreamStop        stream_stop         = 42;
    StreamStateUpdate stream_state        = 43;   // talking/muted indicator
    UdpBinding        udp_binding         = 44;   // token to bind the UDP 5-tuple

    // ── Text ──────────────────────────────────────────────
    TextMessage       text_message        = 50;
    TextMessageAck    text_message_ack    = 51;
    TypingIndicator   typing              = 52;

    // ── Moderation / permissions ──────────────────────────
    KickRequest       kick                = 60;
    BanRequest        ban                 = 61;
    SetPermissionRequest set_permission   = 62;
    ServerMuteRequest server_mute         = 63;

    // ── Admin account management (privileged) ─────────────
    // Accounts are admin-provisioned (no self-serve registration in v1).
    // These ride the same TLS control channel and require an admin permission.
    CreateAccountRequest create_account   = 70;
    ResetPasswordRequest reset_password   = 71;
    DeleteAccountRequest delete_account   = 72;
    ListAccountsRequest  list_accounts    = 73;
    ListAccountsResult   list_accounts_result = 74;

    // ── Extension escape hatch ────────────────────────────
    Extension         extension           = 200;  // {string ns; bytes payload;}
  }
}

Reserved tag ranges keep future families from colliding: 10–19 session, 20–29 state, 30–39 channels, 40–49 voice signaling, 50–59 text, 60–99 moderation, 100–199 future (e.g. file transfer = 100–109), 200+ extensions.

4. Connection lifecycle

Client                                            Server
  │  TCP connect ───────────────────────────────────▶│
  │  ◀──────────────── TLS 1.3 handshake ────────────▶│   server cert (TOFU/PKI, see security.md)
  │                                                   │
  │  ClientHello (proto_version, features[], info) ──▶│
  │  ◀── ServerHello (proto_version, features[],      │   feature intersection negotiated here
  │       server_info, auth_methods, udp_port)        │
  │                                                   │
  │  AuthRequest (guest{nick} | user{name,pass}) ────▶│   password verified w/ Argon2id
  │  ◀── AuthResult (ok, session_id, self, perms,     │
  │       udp_token)                                  │
  │                                                   │
  │  ◀── ServerStateSnapshot (channel tree, users) ───│   initial sync
  │                                                   │
  │  UdpBinding(udp_token) [TCP/TLS] ─────────────────▶│   confirms token, no-ops if mismatched
  │  ◀── UdpBinding(ack=true) [TCP/TLS] ───────────────│
  │                                                   │
  │  ===== UDP side (parallel) =====================  │
  │  (media keys derived from TLS exporter — no 2nd   │
  │   handshake; see security.md §2)                  │
  │  UDP_BINDING frame(udp_token) [plaintext] ───────▶│   binds 5-tuple → session_id
  │  ── voice frames (AEAD, exported keys) ──────────▶│
  │                                                   │
  │  JoinChannelRequest(id, password?) ──────────────▶│
  │  ◀── JoinChannelResult(ok, members, audio_cfg) ───│
  │  StreamAnnounce(kind=mic, opus_params) ──────────▶│
  │  ◀── StreamAnnounceResult(ok, ssrc)               │
  │  ── voice frames flow over UDP ──────────────────▶│
  │                                                   │
  │  Ping / Pong (TCP keepalive) ◀──────────────────▶│

Notes:

• Version negotiation. Each side sends proto_version (integer) and a features string list. The effective version is min(client, server); the effective feature set is the intersection. A client that doesn't understand a feature simply never uses it.
• Auth over TLS. Passwords cross the wire only inside TLS 1.3 and are verified against an Argon2id hash at rest (see security.md). auth_methods in ServerHello advertises whether guest is enabled.
• UDP token. AuthResult.udp_token is a short-lived opaque token. The client confirms it over TCP/TLS (UdpBinding request/ack) and also sends it as the payload of a plaintext UDP_BINDING-type media frame so the server can bind the UDP 5-tuple to the authenticated session without trusting the source address. This bootstrap frame is the only UDP message that carries identity material in the clear; everything after (voice frames) is AEAD-sealed and routed purely by the bound tuple + media-AEAD session.
• Snapshot then deltas. After auth the server pushes a ServerStateSnapshot (full channel tree + visible users), then streams incremental ChannelEvent/UserEvent deltas. Clients reconcile by id.

5. Message catalog (selected definitions)

Representative messages; the full .proto is the source of truth in core/proto/.

message ClientHello {
  uint32 proto_version = 1;
  repeated string features = 2;        // e.g. "opus", "fec", "screen-audio"
  string client_name = 3;              // "VoiceCat-macOS"
  string client_version = 4;
  string preferred_locale = 5;
}

message ServerHello {
  uint32 proto_version = 1;
  repeated string features = 2;
  string server_name = 3;
  string server_version = 4;
  repeated string auth_methods = 5;    // "guest", "password"
  uint32 udp_port = 6;
  bytes  server_identity_fingerprint = 7;  // Ed25519 key fp for TOFU display
}

message AuthRequest {
  oneof method {
    GuestAuth guest = 1;               // { string nickname; }
    PasswordAuth password = 2;         // { string username; string password; }
  }
}

message AuthResult {
  bool ok = 1;
  string error = 2;
  uint64 session_id = 3;
  User self = 4;
  Permissions permissions = 5;
  bytes udp_token = 6;                 // bind UDP 5-tuple with this
}

message Channel {
  uint32 id = 1;
  uint32 parent_id = 2;                // 0 = root
  string name = 3;
  string topic = 4;
  bool   password_protected = 5;
  uint32 max_users = 6;
  ChannelType type = 7;                // PERMANENT / TEMPORARY
  AudioConfig audio = 8;               // per-channel Opus settings (see voice.md)
  int32  order = 9;
}

message User {
  uint32 id = 1;
  string nickname = 2;
  bool   is_guest = 3;
  uint32 channel_id = 4;
  bool   self_mic_muted = 5;
  bool   self_deafened = 6;
  bool   server_muted = 7;
  repeated StreamInfo streams = 8;     // active media streams this user publishes
  bool   server_deafened = 9;          // M5: server-imposed deafen
}

message StreamInfo {
  uint32 stream_id = 1;                // unique within the user
  uint32 ssrc = 2;                     // media-plane id assigned by server
  StreamKind kind = 3;                 // MIC / SCREEN_AUDIO / AUX_DEVICE
  AudioConfig audio = 4;
  string label = 5;                    // "Microphone", "Desktop audio"
}

message StreamAnnounce {              // client → server: "I'm about to publish media"
  StreamKind kind = 1;
  AudioConfig requested_audio = 2;     // server may clamp to channel policy
  string label = 3;
}
message StreamAnnounceResult {
  bool ok = 1; string error = 2;
  uint32 stream_id = 3; uint32 ssrc = 4;
  AudioConfig effective_audio = 5;     // authoritative params to encode with
}

message TextMessage {
  TextScope scope = 1;                 // CHANNEL / PRIVATE / SERVER
  uint32 target_id = 2;                // channel_id or user_id depending on scope
  uint32 sender_id = 3;                // set by server on relay
  string body = 4;                     // UTF-8, server-bounded length
  uint64 sent_at_unix_ms = 5;          // server timestamp on relay
  string client_msg_id = 6;            // client-chosen, echoed in ack (dedup)
}

Text is ephemeral (v1). The server relays messages live to currently-connected, subscribed recipients and does not persist history — there is no store and no backfill on join. Clients may keep their own local scrollback for the session. Server-side history is a deliberate non-feature for now (it can be added later behind a capability flag without changing TextMessage).

6. Request / response & errors

• Any message a client expects a direct answer to sets a nonzero request_id; the server echoes it in the response (*Result or GenericResult). Unsolicited server→client events use request_id = 0.
• GenericResult { bool ok; uint32 code; string message; } is the default acknowledgement for operations without a richer reply (create/edit/delete channel, move user, kick, ban, server-mute, set-permission, create/reset/delete account). Error codes are an enumerated, stable list.
• Fatal conditions send Disconnect { code; reason } then close the TLS connection. code ≥ 1 is server-sent (1 = protocol error, 2 = kicked). code = 0 is client-sent graceful disconnect (§7): the server broadcasts UserEvent::LEFT and closes immediately.
• The response is for the request; the broadcast is for the state. A *Result only acknowledges the actor's request (correlation via request_id, error text, and any actor-private payload — e.g. the channel AudioConfig in JoinChannelResult). The resulting state change is delivered to every connected client including the actor via the normal UserEvent / ChannelEvent / relayed TextMessage path. Clients apply those events to their local model and never re-derive their own state from a *Result (doing so drifts: the actor would miss its own change and a later event for another user would surface the stale value).

7. Keepalive & timeouts

• TCP: Ping/Pong every ~15 s; missing 3 consecutive pongs (45 s) → the server's reaper drops the session. Pong echoes the Ping nonce so RTT is measurable. The client sends Ping automatically from its io thread; the server answers with Pong in any state.
• last_seen reaper. Every ConnSession tracks last_seen — bumped on any inbound TCP frame (not just Ping) and on any inbound UDP voice/keepalive frame. A periodic sweep (asio::steady_timer, every 15 s) drops sessions whose last_seen is older than 45 s. Each drop calls close(), which broadcasts UserEvent::LEFT to remaining clients — so half-open connections (NAT timeout, wifi loss without RST, laptop sleep) that never produce a TCP EOF are cleaned up, and peers' audio engines remove_stream and stop PLC. The timeout and sweep interval are configurable via server::Config::reaper_timeout_ms / reaper_sweep_ms (set to 0 to disable).
• UDP: a separate lightweight keepalive on the media channel (voice.md §6) keeps NAT bindings alive and detects media-path failure independently of the control channel.
• Graceful disconnect. A client ending its session sends Disconnect { code = 0; reason } before closing the socket. The server calls close() on receipt — broadcasting UserEvent::LEFT immediately, without waiting for TCP EOF or the reaper. The client's vc_disconnect() queues this message and waits for the io thread to flush it before closing the socket. code = 0 is reserved for client-initiated graceful disconnect; server-sent fatal Disconnect uses code ≥ 1 (1 = protocol error, 2 = kicked).

8. Client-local features (no protocol changes)

Some features are entirely client-side and involve no changes to the wire format:

• External PCM feed (vc_stream_feed_pcm) — the caller supplies interleaved int16 PCM that the core frames, encodes, and sends over the existing UDP media path. From the server and peers' perspective the stream is indistinguishable from a hardware-captured stream. No new messages, fields, or tags are needed.
• PCM tap (vc_set_pcm_sink) — receives decoded per-stream audio before hardware mixing. Entirely local to the listener; no protocol traffic of any kind.

These are noted here to prevent future contributors from looking for corresponding protocol changes: there are none.

9. Extensibility checklist

When adding a feature later (e.g. file transfer), the rules are:

1. Add new oneof arms in the reserved tag range (file transfer = 100–109) — never reuse or renumber existing tags.
2. Advertise a feature string in ClientHello/ServerHello; only use the feature if both peers list it.
3. Prefer extending an existing message with new fields (additive) over inventing a new message where it fits.
4. For experimental/out-of-tree features, ride inside Extension { ns; payload } until it is promoted to a first-class oneof arm.

This guarantees a v1 client and a v3 server interoperate at the negotiated lowest common denominator.