Talon
f3ac779bf4
Media died silently whenever a client's source address changed. The relay bound a peer's endpoint once and refused to move it, and the client stopped offering its binding token after the first bind, so a Wi-Fi/cellular handover stranded the session in both directions. Add an authenticated Rebind media frame: the binding token travels in the clear for peer lookup only, and the AEAD tag over header and token plus the peer's existing replay window are what authorize the move, so a captured rebind cannot be replayed to redirect someone else's downlink. The client rebuilds its UDP socket instead of retrying on one still pinned to a vanished interface. Nothing judged the control connection live: pings were sent and pongs ignored, so a blackholed TCP path went unnoticed for minutes while the UI showed a live session. Treat any server traffic as liveness and fail the connection when it stops, which drives the existing reconnect. The receive jitter buffer had lost its depth floor, so a channel without FEC or DRED played out with no buffer at all and ordinary reordering became concealment. Restore a one-frame floor, observe every arrival rather than only accepted ones — a shallow buffer was rejecting the late arrivals that should have deepened it — and allow playout to hold a frame so depth can follow a degrading link. A stalled consumer now sheds the oldest queued packet instead of refusing the live talkspurt. Add a deterministic network-impairment simulation covering bursty loss, jitter, reordering, duplication, outages and a stalled consumer, a handover test against a real relay, a replay test for the rebind path, and a blackholed control connection driven through a freezable TCP proxy.
80 lines
4.2 KiB
Markdown
80 lines
4.2 KiB
Markdown
# Architecture
|
|
|
|
VoiceCat is a .NET 10 client/server application with a deliberately narrow native media
|
|
boundary. The managed implementation is authoritative.
|
|
|
|
## Runtime components
|
|
|
|
```text
|
|
Windows WinForms ─┐
|
|
macOS AppKit ─────┼─ VoiceCat.Core ─ VoiceCat.Audio ─ native media shim
|
|
iOS UIKit ────────┘ │
|
|
├─ VoiceCat.Protocol
|
|
└─ VoiceCat.Crypto
|
|
|
|
VoiceCat.Server ──────────── VoiceCat.Protocol + VoiceCat.Crypto + SQLite
|
|
```
|
|
|
|
The server terminates TLS control sessions and relays authenticated encrypted Opus packets.
|
|
It does not decode, mix, or transcode media. Clients own capture, encoding, jitter/loss
|
|
recovery, decoding, mixing, and playback.
|
|
|
|
## Managed projects
|
|
|
|
- `VoiceCat.Protocol`: generated protobuf types and bounded length-prefixed framing.
|
|
- `VoiceCat.Crypto`: BouncyCastle TLS 1.3, exporter-derived media secrets, certificate TOFU,
|
|
Ed25519 server identity, ChaCha20-Poly1305, replay windows, and Argon2id.
|
|
- `VoiceCat.Codec`: safe ownership around fixed Opus shim handles.
|
|
- `VoiceCat.Dsp`: RNNoise and energy-VAD ownership.
|
|
- `VoiceCat.Audio`: streams, packet-loss recovery, jitter buffers, mixing, PCM rings, and
|
|
activation policy.
|
|
- `VoiceCat.Core`: connection lifecycle, authentication, state snapshots/events, requests,
|
|
stream negotiation, encrypted UDP, reconnect, and administration helpers.
|
|
- `VoiceCat.Server`: listener/session ownership, SQLite state, permissions, moderation,
|
|
channel management, encrypted UDP routing, and process administration commands.
|
|
- `VoiceCat.Cli`: interactive client and deterministic text/voice behavior driver.
|
|
|
|
Leaf UI projects reference the managed core; platform audio objects translate between native
|
|
device buffers and bounded PCM rings owned by `VoiceCat.Audio`.
|
|
|
|
## Native boundary
|
|
|
|
`native/media` builds `voicecat_media` for desktop and static Apple targets. It contains only
|
|
fixed C entry points for Opus/DRED and RNNoise. Vendored RNNoise is under `native/rnnoise`.
|
|
Networking, TLS, media encryption, session state, jitter, and mixing do not live in native code.
|
|
|
|
`native/apple/broadcast` is a Swift ReplayKit upload extension. It captures application audio,
|
|
converts it to 48 kHz stereo int16 PCM, and writes the frozen App Group ring documented in
|
|
`broadcast-ring-format.md`. The managed iOS host drains that ring and performs encoding and
|
|
networking. The extension deliberately has no managed runtime or VoiceCat protocol stack.
|
|
|
|
The iOS host also has small Objective-C bridges in its managed project for platform APIs that
|
|
need direct native entry points. These are platform adapters, not a second core.
|
|
|
|
## Ownership and concurrency
|
|
|
|
- A `VoiceCatClient` owns one TLS control connection, one media session, and its audio engine.
|
|
- Control/TLS operations have one serialized owner. UI code submits requests and observes
|
|
events; it does not call TLS concurrently.
|
|
- The server publishes immutable routing state to its UDP loop. The UDP loop owns endpoint
|
|
binding, packet authentication, replay checks, and recipient resealing.
|
|
- Audio callbacks consume or produce preallocated ring-buffer memory. They never allocate,
|
|
lock, block, log, or access sockets.
|
|
- Teardown establishes a quiescence barrier before callback-owned state is released.
|
|
- Blocking file, database, TLS, and device lifecycle work stays off real-time callbacks.
|
|
|
|
## Data and contracts
|
|
|
|
- `proto/voicecat.proto` is the control-plane schema.
|
|
- Media uses the fixed header and AEAD construction described in `protocol.md` and
|
|
`security.md`.
|
|
- Media frame types are a versioned contract: `Voice`, `Keepalive`, `UdpBinding`, and `Rebind`.
|
|
`UdpBinding` establishes a peer's endpoint once, in the clear. `Rebind` moves an established
|
|
endpoint after the client's source address changes, as on a Wi-Fi/cellular handover; it
|
|
carries the binding token in the clear for peer lookup only, and authorization comes from the
|
|
AEAD tag over header and token plus the peer's replay window, so a captured rebind cannot be
|
|
replayed to redirect someone else's downlink.
|
|
- SQLite is the server's persistent store; schema changes require explicit migrations.
|
|
- Client profiles and TOFU pins are local platform data.
|
|
- The ReplayKit ring layout is separately versioned and frozen.
|