Retire legacy sources and verify managed iOS deployment
This commit is contained in:
+31
-62
@@ -1,69 +1,38 @@
|
||||
# VoiceCat — Design Documentation
|
||||
# VoiceCat documentation
|
||||
|
||||
VoiceCat is a self-hosted, server-based voice and text chat system in the spirit of
|
||||
classic TeamSpeak / Mumble: channel-based voice, channel and private text chat, and a
|
||||
single server you own and run. It deliberately avoids WebRTC. The media path is plain
|
||||
**UDP**, the control path is plain **TCP**, and both are encrypted.
|
||||
VoiceCat is a self-hosted channel-based voice and text system. Control traffic uses TLS 1.3;
|
||||
media uses authenticated encrypted UDP derived from the TLS session. There is no plaintext
|
||||
mode and no central service.
|
||||
|
||||
This folder is the design spec. No code yet — these documents define the architecture,
|
||||
the wire protocol, the audio pipeline, the security model, and the dependency list, so
|
||||
that implementation can start from a shared, agreed plan.
|
||||
The .NET 10 implementation is the source of truth. Some detailed documents predate the managed
|
||||
rewrite and are being corrected as code changes touch them. When prose conflicts with current
|
||||
managed code or tests, follow the managed implementation and fix the document in the same
|
||||
change.
|
||||
|
||||
## Decisions locked so far
|
||||
## Current documents
|
||||
|
||||
| Area | Decision |
|
||||
|------|----------|
|
||||
| Code architecture | **Shared C++ core** (`libvoicecat`) consumed by native UIs over a **C ABI**. Server reuses the same core. |
|
||||
| Native clients | Managed **C#** WinForms and AppKit replacements are implemented; Swift macOS remains the migration/release oracle pending manual cutover gates, and iOS remains Swift. |
|
||||
| Control transport | **TCP + TLS 1.3** (mbedTLS) |
|
||||
| Media transport | **UDP** secured by **TLS-exported keys + ChaCha20-Poly1305 AEAD** — mandatory, no plaintext mode (see [security.md](security.md)) |
|
||||
| Crypto libraries | **mbedTLS** (TLS 1.3) + **libsodium** (AEAD, Argon2id, Ed25519) — both permissive, **no GPL/LGPL anywhere** |
|
||||
| Voice codec | **Opus** (libopus 1.6), per-channel configurable mono/stereo, bitrate, frame size, FEC/DTX |
|
||||
| Audio DSP | **webrtc-audio-processing (APM)** was the plan for AEC/NS/AGC/VAD, but has no working Windows/MSVC build upstream — v1 ships a lightweight energy/RMS VAD instead, no AEC/NS/AGC yet (see [tech-stack.md](tech-stack.md), [roadmap.md](roadmap.md) §2). NR is **two-sided**: sender can denoise, and each listener can denoise a *specific* other user locally — this plumbing exists but is currently inert pending a real DSP backend. Input gate supports **VAD and PTT**, client-configurable. |
|
||||
| Identity | **Guests + admin-provisioned local accounts** (Argon2id, SQLite). No self-serve registration; guests toggleable per server. |
|
||||
| Text | **Ephemeral** — live relay, no server-side history in v1. |
|
||||
| Serialization | **Protocol Buffers** for the control plane; **custom binary** for voice frames |
|
||||
| Deployment | **One `docker run`, one static binary, or `cmake --build`** — zero-config, secured by default (see [deployment.md](deployment.md)) |
|
||||
- [architecture.md](architecture.md) — components, ownership, concurrency, and native boundary.
|
||||
- [protocol.md](protocol.md) — protobuf control messages and fixed UDP media header.
|
||||
- [security.md](security.md) — TLS, TOFU, media keys, authentication, and threat model.
|
||||
- [voice.md](voice.md) — streams, Opus, loss handling, jitter, mixing, and screen audio.
|
||||
- [api-dotnet.md](api-dotnet.md) — managed APIs and ownership contracts.
|
||||
- [building.md](building.md) — development, platform builds, tests, and publishing.
|
||||
- [deployment.md](deployment.md) — server configuration and packaging.
|
||||
- [ios-deploy.md](ios-deploy.md) — managed iOS physical-device build and deployment.
|
||||
- [tech-stack.md](tech-stack.md) — supported dependencies and licensing.
|
||||
- [broadcast-ring-format.md](broadcast-ring-format.md) — frozen iOS extension/host ring ABI.
|
||||
- [roadmap.md](roadmap.md) — only current release gates and intentionally deferred features.
|
||||
|
||||
## Document index
|
||||
The completed C++-to-.NET migration plan was removed. Git history preserves that work without
|
||||
making every future agent load an obsolete implementation diary.
|
||||
|
||||
1. [architecture.md](architecture.md) — System layers, the shared core, threading model, the C ABI, server design.
|
||||
2. [protocol.md](protocol.md) — The control protocol: framing, connection lifecycle, the full message catalog, encoding, versioning/extensibility.
|
||||
3. [voice.md](voice.md) — The UDP media protocol: voice frame format, Opus configuration, multi-stream model, jitter buffer, packet-loss handling.
|
||||
4. [security.md](security.md) — Mandatory encryption (TLS 1.3 + exported-key media AEAD), server identity (TOFU), authentication, accounts, anti-replay, threat model.
|
||||
5. [tech-stack.md](tech-stack.md) — Concrete libraries with versions and rationale, the permissive-license rule, build tooling, per-platform notes.
|
||||
6. [deployment.md](deployment.md) — The "set it up in a few minutes" story: Docker, single binary, source build, zero-config defaults.
|
||||
7. [roadmap.md](roadmap.md) — Milestones, what ships when, and the list of open questions still to resolve.
|
||||
8. [porting-to-dotnet.md](porting-to-dotnet.md) — **Active migration.** Step-by-step plan and implementation checkpoints for replacing the C++ core, C++ server, and replaceable Swift clients with .NET 10 / C#. Dependency map, TLS exporter, real-time-audio design, and phased cutover gates.
|
||||
## Durable rules
|
||||
|
||||
## Design principles
|
||||
|
||||
- **One core, many faces.** Protocol, crypto, Opus, networking, jitter buffering, and
|
||||
mixing live once in C++. UIs are thin. This keeps behavior identical across platforms
|
||||
and the security-sensitive code reviewed in a single place.
|
||||
- **Boringly simple transport.** TCP for control, UDP for media. No ICE, no SDP, no
|
||||
TURN. A user opens a port (or port-forwards) and runs a server.
|
||||
- **Extensible from day one.** Every message rides in a versioned envelope; capabilities
|
||||
are negotiated at connect time; unknown fields are ignored. File transfer, screen-audio,
|
||||
and moderation slot in without breaking older clients.
|
||||
- **Real-time correctness.** The audio thread never blocks, never allocates, never takes a
|
||||
lock. Network and audio communicate through lock-free ring buffers.
|
||||
- **Encrypted, always.** There is no unencrypted mode to misconfigure. The server has no
|
||||
plaintext listener; encryption is on because it can't be turned off. And it's free to the
|
||||
operator — the server self-provisions its key/cert on first run.
|
||||
- **Stupid-easy to self-host.** The target reaction is "oh, I (or my agent) can stand this up
|
||||
in a few minutes." One `docker run`, or one static binary, or a plain `cmake --build` — no
|
||||
certificate wrangling, no external database, sane defaults out of the box. Permissive
|
||||
licenses only (no GPL/LGPL) so it can be redistributed freely, including closed-source.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Core** — `libvoicecat`, the shared C++ library.
|
||||
- **Control channel** — the TCP/TLS connection carrying protobuf messages.
|
||||
- **Media channel** — the UDP connection carrying voice frames.
|
||||
- **Stream** — one audio source from one user (e.g. mic, screen audio, second device). A
|
||||
user may publish several streams at once; each is independently controllable.
|
||||
- **Channel** — a room in the channel tree. Voice is scoped to a channel.
|
||||
- **Session** — an authenticated connection; ties a TCP control channel to a UDP 5-tuple.
|
||||
- **SFU relay** — the server forwards Opus packets between channel members without decoding
|
||||
them (selective forwarding, no transcoding).
|
||||
- `proto/voicecat.proto` is the control-plane schema.
|
||||
- Encryption is mandatory.
|
||||
- No GPL or LGPL dependencies.
|
||||
- Real-time audio callbacks never allocate, lock, block, or perform I/O.
|
||||
- The server relays encoded media; it does not mix or transcode.
|
||||
- Text is ephemeral in v1.
|
||||
- Accounts are administrator-provisioned; guests are an operator choice.
|
||||
- Wire, database, and shared-ring changes are explicitly versioned.
|
||||
|
||||
+5
-5
@@ -132,7 +132,7 @@ erasure of every runtime/library copy.
|
||||
|
||||
`VoiceCat.Codec` and `VoiceCat.Dsp` call the desktop `voicecat_media` native library
|
||||
through source-generated `LibraryImport`. It links pinned Opus 1.5.2 and the existing
|
||||
vendored RNNoise; it has no dependency on libvoicecat or its C ABI. Fixed C signatures
|
||||
vendored RNNoise; it has no dependency on the retired native core or its C ABI. Fixed C signatures
|
||||
wrap Opus controls so P/Invoke never calls C varargs. SafeHandle owns every native
|
||||
encoder, decoder, DRED parser/state, and denoiser, including failed initialization.
|
||||
|
||||
@@ -168,7 +168,7 @@ can try FEC/PLC. Only consume recovery output on success. Parse/native errors th
|
||||
`RnnoiseProcessor.Process(Span<short>, sampleRate)` operates in place on complete
|
||||
480-sample mono chunks at 48 kHz. Other rates pass through unchanged; partial chunks
|
||||
at 48 kHz throw instead of leaving a tail silently untreated. Float scratch is
|
||||
preallocated, and rounding/clipping matches the C++ processor. Use distinct instances
|
||||
preallocated, and rounding/clipping is covered by canonical vectors. Use distinct instances
|
||||
for stereo channels when the later pipeline supports stereo microphone denoising.
|
||||
Noise reduction does not gate speech.
|
||||
|
||||
@@ -230,7 +230,7 @@ Database v2 has no DRED column; CRUD rejects DRED rather than silently losing it
|
||||
|
||||
Session permissions gate kick/ban/move/mute and account operations. Only administrators
|
||||
can grant permissions; account-administration permission cannot grant administrator status.
|
||||
These two permission restrictions are stricter than the C++ oracle. Moves bypass channel
|
||||
These two permission restrictions are intentional managed-server policy. Moves bypass channel
|
||||
passwords but respect capacity and clear streams. Server mute/deafen immediately updates
|
||||
encrypted routing. Kick/ban retire routing before closure and emit one LEFT with the reason.
|
||||
Account bans persist by username; guest bans persist by address because nicknames are not
|
||||
@@ -281,7 +281,7 @@ The reaper sends a fatal disconnect, removes presence/routing and broadcasts one
|
||||
event. Valid UDP activity keeps a TCP-idle client alive. Shutdown cancels and awaits
|
||||
the accept, reaper, control and media loops before disposing credentials/storage.
|
||||
|
||||
`AccountStore(path)` retains the C++ schema version 2, accepts version 1 migration,
|
||||
`AccountStore(path)` uses schema version 2 and accepts version 1 migration,
|
||||
and rejects unknown revisions. Opening an existing channel table does not reseed it.
|
||||
Account creation/authentication uses parameterized SQL; two password workers bound
|
||||
per-store Argon2 work. Failed authentication leaves `last_login` unchanged. Dispose
|
||||
@@ -293,7 +293,7 @@ the native administration CLI; there is no automatic bootstrap account.
|
||||
Argon2id v19 PHC strings: 16-byte salt, 32-byte output, new-hash parameters
|
||||
64 MiB memory, two iterations, parallelism one. Verification supports up to 128 MiB,
|
||||
ten iterations, parallelism four and 1024 UTF-8 password bytes; malformed or excessive
|
||||
hashes fail closed. Standard C++ interactive-cost accounts are preserved. These
|
||||
hashes fail closed. Standard interactive-cost Argon2id accounts are preserved. These
|
||||
bounds intentionally reject imported hashes above those costs. Native fixtures cover
|
||||
ASCII, Unicode and embedded NUL; the database oracle verifies cross-implementation
|
||||
authentication in both directions.
|
||||
|
||||
+57
-232
@@ -1,251 +1,76 @@
|
||||
# Architecture
|
||||
|
||||
The parallel .NET rewrite under `dotnet/` currently implements shared protocol framing,
|
||||
voice headers, and media crypto. Existing server/client/audio behavior remains in C++.
|
||||
See `docs/api-dotnet.md` for the initial managed contract and
|
||||
`docs/porting-to-dotnet.md` for subsequent migration phases.
|
||||
VoiceCat is a .NET 10 client/server application with a deliberately narrow native media
|
||||
boundary. The managed implementation is authoritative.
|
||||
|
||||
## 1. The shared-core model
|
||||
## Runtime components
|
||||
|
||||
All non-UI logic lives in one C++ library, **`libvoicecat`**. The same library is linked
|
||||
into every client and into the server. Platform UIs are thin and call the core through a
|
||||
stable **C ABI** (`voicecat.h`).
|
||||
```text
|
||||
Windows WinForms ─┐
|
||||
macOS AppKit ─────┼─ VoiceCat.Core ─ VoiceCat.Audio ─ native media shim
|
||||
iOS UIKit ────────┘ │
|
||||
├─ VoiceCat.Protocol
|
||||
└─ VoiceCat.Crypto
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────┐
|
||||
macOS / iOS (Swift) │ │ Windows (C#)
|
||||
┌──────────────────┐ │ libvoicecat (C++) │ ┌──────────────────┐
|
||||
│ SwiftUI views │ │ ┌─────────────────────────────────────┐ │ │ WinForms (.NET 10│
|
||||
│ AVAudioSession │──┼─▶│ C ABI (voicecat.h) │◀─┼──│ LibraryImport │
|
||||
│ Swift↔C++ interop│ │ ├─────────────────────────────────────┤ │ │ P/Invoke │
|
||||
└──────────────────┘ │ │ Session / Protocol state machine │ │ └──────────────────┘
|
||||
│ │ Text + voice signaling │ │
|
||||
Linux/macOS/Windows │ │ Audio engine: capture→encode→send, │ │
|
||||
server │ │ recv→jitter→decode→mix→playback │ │
|
||||
┌──────────────────┐ │ │ Codec layer (Opus 1.6) │ │
|
||||
│ voicecat-server │──┼─▶│ Crypto + transport (TLS 1.3 + AEAD) │ │
|
||||
│ (reuses core) │ │ │ Net I/O (Asio: TCP + UDP + timers) │ │
|
||||
└──────────────────┘ │ └─────────────────────────────────────┘ │
|
||||
└───────────────────────────────────────────┘
|
||||
VoiceCat.Server ──────────── VoiceCat.Protocol + VoiceCat.Crypto + SQLite
|
||||
```
|
||||
|
||||
Why this shape:
|
||||
The server terminates TLS control sessions and relays authenticated encrypted Opus packets.
|
||||
It does not decode, mix, or transcode media. Clients own capture, encoding, jitter/loss
|
||||
recovery, decoding, mixing, and playback.
|
||||
|
||||
- **Swift** (5.9+) can import C++ directly, but we still ship a C ABI because it is the
|
||||
lowest-friction, most stable boundary and it is what **C#** needs (`LibraryImport`/
|
||||
P/Invoke). One ABI serves both.
|
||||
- The **server** is not a separate codebase. It links the same protocol, crypto, and Opus
|
||||
code as the client, so framing/encryption can never drift between the two ends.
|
||||
## Managed projects
|
||||
|
||||
## 2. Layered design inside the core
|
||||
- `VoiceCat.Protocol`: generated protobuf types and bounded length-prefixed framing.
|
||||
- `VoiceCat.Crypto`: BouncyCastle TLS 1.3, exporter-derived media secrets, certificate TOFU,
|
||||
Ed25519 server identity, ChaCha20-Poly1305, replay windows, and Argon2id.
|
||||
- `VoiceCat.Codec`: safe ownership around fixed Opus shim handles.
|
||||
- `VoiceCat.Dsp`: RNNoise and energy-VAD ownership.
|
||||
- `VoiceCat.Audio`: streams, packet-loss recovery, jitter buffers, mixing, PCM rings, and
|
||||
activation policy.
|
||||
- `VoiceCat.Core`: connection lifecycle, authentication, state snapshots/events, requests,
|
||||
stream negotiation, encrypted UDP, reconnect, and administration helpers.
|
||||
- `VoiceCat.Server`: listener/session ownership, SQLite state, permissions, moderation,
|
||||
channel management, encrypted UDP routing, and process administration commands.
|
||||
- `VoiceCat.Cli`: interactive client and deterministic text/voice behavior driver.
|
||||
|
||||
From the OS up:
|
||||
Leaf UI projects reference the managed core; platform audio objects translate between native
|
||||
device buffers and bounded PCM rings owned by `VoiceCat.Audio`.
|
||||
|
||||
| Layer | Responsibility | Key deps |
|
||||
|-------|----------------|----------|
|
||||
| **Platform I/O** | Sockets, timers; audio device capture/playback | Asio, miniaudio |
|
||||
| **Transport** | TLS 1.3 (TCP), exported-key ChaCha20-Poly1305 AEAD (UDP), framing, anti-replay | mbedTLS, libsodium |
|
||||
| **Codec & DSP** | Opus encode/decode; APM (AEC/NS/AGC/VAD) send-side + per-user NR receive-side; resample; jitter buffer; mixer | libopus, webrtc-audio-processing, speexdsp |
|
||||
| **Protocol** | Message (de)serialization, request/response correlation, state machine | protobuf |
|
||||
| **Session/domain** | Channels, users, streams, permissions, text routing | — |
|
||||
| **C ABI façade** | Handle-based API + event callbacks exposed to UIs | — |
|
||||
## Native boundary
|
||||
|
||||
A UI never sees a socket, an Opus packet, or a protobuf message. It sees: "connect",
|
||||
"join channel", "start a stream from this device", "send this text", and a stream of
|
||||
events ("user joined", "user is talking", "message received", "level meter = 0.4").
|
||||
`native/media` builds `voicecat_media` for desktop and static Apple targets. It contains only
|
||||
fixed C entry points for Opus/DRED and RNNoise. Vendored RNNoise is under `native/rnnoise`.
|
||||
Networking, TLS, media encryption, session state, jitter, and mixing do not live in native code.
|
||||
|
||||
## 3. Threading model
|
||||
`native/apple/broadcast` is a Swift ReplayKit upload extension. It captures application audio,
|
||||
converts it to 48 kHz stereo int16 PCM, and writes the frozen App Group ring documented in
|
||||
`broadcast-ring-format.md`. The managed iOS host drains that ring and performs encoding and
|
||||
networking. The extension deliberately has no managed runtime or VoiceCat protocol stack.
|
||||
|
||||
Three classes of thread, with strict rules.
|
||||
The iOS host also has small Objective-C bridges in its managed project for platform APIs that
|
||||
need direct native entry points. These are platform adapters, not a second core.
|
||||
|
||||
```
|
||||
┌──────────────┐ lock-free ┌──────────────┐ lock-free ┌──────────────┐
|
||||
│ Audio capture│ ──ring buffer─▶│ Net thread │ ──ring buffer─▶│Audio playback│
|
||||
│ (RT, miniaudio│ │ (Asio loop) │ │ (RT, miniaudio│
|
||||
│ callback) │◀───ring buffer─│ │◀───ring buffer─│ callback) │
|
||||
│ capture→Opus │ │ TLS + AEAD, │ │ jitter→Opus │
|
||||
│ encode │ │ route, relay │ │ decode→mix │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
│
|
||||
┌─────▼──────┐
|
||||
│ Worker pool│ DB, Argon2id, file I/O,
|
||||
│ (blocking) │ TLS handshakes, codec setup
|
||||
└────────────┘
|
||||
```
|
||||
## Ownership and concurrency
|
||||
|
||||
Rules:
|
||||
- A `VoiceCatClient` owns one TLS control connection, one media session, and its audio engine.
|
||||
- Control/TLS operations have one serialized owner. UI code submits requests and observes
|
||||
events; it does not call TLS concurrently.
|
||||
- The server publishes immutable routing state to its UDP loop. The UDP loop owns endpoint
|
||||
binding, packet authentication, replay checks, and recipient resealing.
|
||||
- Audio callbacks consume or produce preallocated ring-buffer memory. They never allocate,
|
||||
lock, block, log, or access sockets.
|
||||
- Teardown establishes a quiescence barrier before callback-owned state is released.
|
||||
- Blocking file, database, TLS, and device lifecycle work stays off real-time callbacks.
|
||||
|
||||
- **Audio (real-time) threads** are driven by the OS audio callback. They must not
|
||||
allocate, lock, log, or do syscalls beyond the ring-buffer hand-off. Opus encode/decode
|
||||
runs here (it is allocation-free after init).
|
||||
- **Net thread(s)** run the Asio event loop: TLS records, AEAD seal/open, protobuf parse, channel
|
||||
routing, jitter-buffer feed. On the server, this is where the SFU relay copies packets
|
||||
to subscribers.
|
||||
- **Worker pool** absorbs anything that can block: SQLite, Argon2id verification, DNS,
|
||||
TLS handshake CPU, codec (re)configuration.
|
||||
- **Communication** between audio and net is single-producer/single-consumer lock-free
|
||||
ring buffers (one per direction per stream). Control-plane events to the UI go through a
|
||||
thread-safe queue drained on the UI's terms.
|
||||
## Data and contracts
|
||||
|
||||
## 4. The C ABI (`voicecat.h`) — shape
|
||||
- `proto/voicecat.proto` is the control-plane schema.
|
||||
- Media uses the fixed header and AEAD construction described in `protocol.md` and
|
||||
`security.md`.
|
||||
- SQLite is the server's persistent store; schema changes require explicit migrations.
|
||||
- Client profiles and TOFU pins are local platform data.
|
||||
- The ReplayKit ring layout is separately versioned and frozen.
|
||||
|
||||
Handle-based, opaque pointers, C-linkage. Illustrative (final names in implementation):
|
||||
|
||||
```c
|
||||
typedef struct vc_client vc_client;
|
||||
|
||||
typedef struct {
|
||||
void (*on_event)(void* user, const vc_event* ev); // state changes, messages
|
||||
void (*on_level)(void* user, uint32_t stream_id, float rms); // meters (throttled)
|
||||
void* user;
|
||||
} vc_callbacks;
|
||||
|
||||
vc_client* vc_client_create(const vc_config* cfg, vc_callbacks cb);
|
||||
void vc_client_destroy(vc_client*);
|
||||
|
||||
int vc_connect(vc_client*, const char* host, uint16_t port); // async; result via event
|
||||
int vc_authenticate_guest(vc_client*, const char* nickname);
|
||||
int vc_authenticate_user(vc_client*, const char* user, const char* password);
|
||||
|
||||
int vc_join_channel(vc_client*, uint32_t channel_id, const char* password /*nullable*/);
|
||||
int vc_leave_channel(vc_client*);
|
||||
|
||||
// Streams (mic / screen audio / aux device)
|
||||
int vc_stream_start(vc_client*, const vc_stream_desc* desc, uint32_t* out_stream_id);
|
||||
int vc_stream_stop(vc_client*, uint32_t stream_id);
|
||||
int vc_set_input_device(vc_client*, uint32_t stream_id, const char* device_id);
|
||||
int vc_set_self_mute(vc_client*, bool mic_muted, bool deafened);
|
||||
|
||||
// Text
|
||||
int vc_send_text(vc_client*, vc_text_scope scope, uint32_t target_id, const char* utf8);
|
||||
|
||||
// Enumeration helpers for UI device pickers
|
||||
int vc_list_devices(vc_client*, vc_device_kind kind, vc_device_list* out);
|
||||
```
|
||||
|
||||
Design notes:
|
||||
|
||||
- **Async, event-driven.** Calls return immediately; results and state changes arrive via
|
||||
`on_event`. This maps cleanly onto SwiftUI/`async` and C# `event`/`Task` patterns.
|
||||
- **The core owns audio.** Capture, encode, decode, mixing, and playback happen inside the
|
||||
core via miniaudio. The UI only *selects devices*, *starts/stops streams*, and *renders
|
||||
meters/state*. This keeps the real-time path identical on every OS. (iOS is the one
|
||||
exception that needs UI-side cooperation — see below.)
|
||||
- **Device enumeration works pre-connect.** `vc_list_devices` needs no live session — device
|
||||
pickers can populate before `vc_connect`. `vc_device.id` is an opaque, internally-encoded
|
||||
handle (currently a hex-encoded `ma_device_id`) — always round-trip an id that came from
|
||||
`vc_list_devices`/`vc_get_stream_audio_config`; never construct one by hand. Tolerate an
|
||||
empty list (a machine can legitimately have zero input or output devices).
|
||||
- **Strings are UTF-8 `const char*`; ownership is explicit.** Output buffers are
|
||||
caller-allocated or returned with a paired `vc_free`.
|
||||
|
||||
### Per-platform binding notes
|
||||
|
||||
- **Swift / Apple.** Import the C ABI via a **module map** (`module VoiceCatC { header "voicecat.h" }`) staged into the XCFramework headers by `clients/apple/scripts/build-xcframework.sh` — Swift gets a clean `import VoiceCatC` with all C enums/structs/functions available directly (no manual redeclaration, unlike the C# P/Invoke layer). A **Swift wrapper** (`VoiceCatCore` package at `clients/apple/`) provides Swift-idiomatic types (`VoiceCatResult`, `VoiceCatEvent`, `Channel`, `User`, etc.) on top, mirroring the C# `VoiceCat.Interop` layer. Callbacks use `@convention(c)` closures (plain C function pointers, not ARC-managed closures) + `Unmanaged.passUnretained(self)` as the `user` context (the Swift analog of C#'s `[UnmanagedCallersOnly]` + `GCHandle`). Events are delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (one async block scheduled at a time) — the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump. `deinit` calls `vc_client_destroy` (joins all threads) then frees native CString config storage (the core stores raw pointers, doesn't copy). **macOS UI: AppKit** (chosen over SwiftUI for the most mature VoiceOver accessibility story — same rationale as the Windows client's WinForms choice); **iOS UI: SwiftUI** (narrower control surface, sufficient VoiceOver support). On **iOS** the app owns `AVAudioSession` (category `.playAndRecord`), requests mic permission, and handles interruptions/route changes — the core exposes hooks (`vc_audio_suspend`/`vc_audio_resume`/`vc_audio_restart`, implemented) the Swift layer calls from `AVAudioSession` notifications and `IOSAudioRouter` setting changes. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture) is driven from the Swift `IOSAudioRouter` singleton via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The core is told the capture channel count via `vc_set_capture_channels` (append-only ABI). `vc_audio_restart` does a full stop + re-init (unlike `suspend`/`resume` which only stop/start) so devices reopen against a new route after `AVAudioSession` reconfiguration. iOS 18.0 deployment target. Background voice and VoIP push (CallKit/PushKit) are a later milestone. The XCFramework carries a **fat static library** (`libvoicecat-fat.a`) bundling `libvoicecat.a` + all vcpkg static deps so the Swift Package links a single self-contained `.a` per slice.
|
||||
- **iOS screen / system-audio sharing** is supported via a **ReplayKit Broadcast Upload
|
||||
Extension** (the same mechanism Discord uses; triggered from Control Center's screen-record
|
||||
button via `RPSystemBroadcastPickerView`). The extension receives
|
||||
`RPSampleBufferType.audioApp` (system/app audio) and `.audioMic`. We capture **`.audioApp`**
|
||||
for the `SCREEN_AUDIO` "listen together" stream and ignore video. The extension runs in a
|
||||
**separate process with a ~50 MB memory cap** — that cap is a problem only for video
|
||||
frames, so audio-only stays well within budget. It links a *minimal* slice of the core
|
||||
(Opus encode + media send), shares the session/credentials with the host app through an
|
||||
**App Group**, and re-derives its own media keys. This is detailed in [voice.md](voice.md) §9.
|
||||
- **C# / Windows.** `[LibraryImport]` (source-generated P/Invoke, .NET 7+) over the C ABI.
|
||||
`[UnmanagedCallersOnly]` static methods for `on_event`/`on_level` to avoid delegate-lifetime
|
||||
pitfalls. UI in **WinForms (.NET 10)** — chosen over WinUI 3/Avalonia for its mature,
|
||||
predictable screen-reader (NVDA/JAWS/Narrator) UIA support (see roadmap.md §2).
|
||||
Events are delivered via `System.Threading.Channels.Channel<VoiceCatEvent>`, drained by a
|
||||
30ms `System.Windows.Forms.Timer` on the UI thread — simpler than a message-only HWND +
|
||||
`PostMessage` with no meaningful latency cost. `VoiceCatClientHandle : SafeHandle` wraps
|
||||
the `vc_client*` and guarantees `vc_client_destroy` runs on GC/Dispose.
|
||||
|
||||
### External PCM feed/tap
|
||||
|
||||
Two API functions let callers bypass miniaudio entirely for a stream:
|
||||
|
||||
| Function | Direction | Contract |
|
||||
|----------|-----------|----------|
|
||||
| `vc_stream_feed_pcm(c, stream_id, pcm, samples_per_channel, channels)` | **Send** — caller → network | Caller supplies interleaved int16 at the stream's sample rate (`channels` = 1 mono, 2 stereo). The core frames, Opus-encodes, AEAD-seals, and sends over UDP — identical wire path to hardware capture. The stream must already be started with `vc_stream_start`. Thread-safe; may be called from any thread (audio callback, ReplayKit delegate, SCStream callback). |
|
||||
| `vc_set_pcm_sink(c, cb, user)` | **Receive** — network → caller | `cb` is called on the audio (playback) thread once per decoded Opus frame per remote stream, with `(user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)`. PCM is delivered to the sink **and** the hardware device — dual output; the hardware mix is unaffected. Pass `cb=NULL` to disable (default); once that call returns, no callback using the old `user` pointer remains in flight. **Must not block** — copy what you need and return. |
|
||||
|
||||
`vc_test_inject_capture` (the old TEST-ONLY mono-only predecessor) is a deprecated alias
|
||||
for `vc_stream_feed_pcm(..., channels=1)` — kept for source compatibility.
|
||||
|
||||
**Use cases:** ReplayKit Broadcast Extension (iOS `SCREEN_AUDIO`), ScreenCaptureKit (macOS
|
||||
`SCREEN_AUDIO`), music/TTS/relay bots, soundboards, transcription clients. The extension or
|
||||
bot links Opus + the feed entry point — no `ma_device`, no hardware, headless.
|
||||
|
||||
**Threading:** the feed path is thread-safe (ring buffer, no lock on the RT path). The sink
|
||||
callback runs on the miniaudio playback thread — observe the same rules as the capture
|
||||
callback: no allocations, no blocking calls.
|
||||
|
||||
## 5. Server architecture
|
||||
|
||||
`voicecat-server` is a headless process linking the core.
|
||||
|
||||
```
|
||||
TCP/TLS 1.3 UDP + media AEAD
|
||||
│ │
|
||||
┌────────▼─────────┐ ┌─────────▼──────────┐
|
||||
│ Connection mgr │ │ UDP demux │
|
||||
│ (accept, TLS, │ │ 5-tuple → session │
|
||||
│ per-conn state) │ │ anti-replay window │
|
||||
└────────┬─────────┘ └─────────┬──────────┘
|
||||
│ │
|
||||
┌────────▼───────────────────────────────────▼──────────┐
|
||||
│ Session registry (session_id ↔ TCP conn ↔ UDP tuple) │
|
||||
└────────┬───────────────────────────────┬───────────────┘
|
||||
│ │
|
||||
┌────────▼─────────┐ ┌──────────────┐ ┌▼─────────────────┐
|
||||
│ Channel manager │ │ Text router │ │ Voice router/SFU │
|
||||
│ tree, configs, │ │ channel + PM │ │ relay Opus to │
|
||||
│ membership, perms│ │ │ │ channel members │
|
||||
└────────┬─────────┘ └──────────────┘ └──────────────────┘
|
||||
│
|
||||
┌────────▼─────────┐
|
||||
│ Persistence │ accounts (Argon2id), channels, bans, config
|
||||
│ SQLite │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
- **Voice router is a relay, not a mixer.** For each incoming voice frame it looks up the
|
||||
sender's channel and forwards the *unmodified encoded Opus bytes* to other members.
|
||||
It authenticates/decrypts incoming media, then reseals with each recipient's directional
|
||||
key and counter. SSRC/timestamp/flags/codec pass through; sequence and ciphertext/tag change.
|
||||
No server-side decode/transcode → low CPU,
|
||||
low latency, and end-to-content is just Opus. Per-channel Opus params are enforced so all
|
||||
members are mutually decodable.
|
||||
- **Subscriptions.** Clients implicitly subscribe to their current channel's voice; text
|
||||
and presence can be subscribed more broadly. This keeps fan-out bounded on big servers.
|
||||
- **Stateless-ish media.** UDP carries no auth per packet beyond the media-AEAD session;
|
||||
the 5-tuple→session binding is established once via a token (see protocol.md §4).
|
||||
- **Keepalive reaper.** An `asio::steady_timer` sweeps every 15 s and drops any session
|
||||
whose `last_seen` (bumped on every inbound TCP or UDP frame) is older than 45 s. Each
|
||||
drop broadcasts `UserEvent::LEFT` so peers clean up immediately. This catches half-open
|
||||
connections that never produce a TCP EOF. Configurable via `server::Config`.
|
||||
- **Single process, scalable later.** v1 is one process, one machine. The session registry
|
||||
and router are written behind interfaces so a future build can sit them behind a shared
|
||||
bus for multi-node, but that is explicitly out of scope for now.
|
||||
|
||||
## 6. Repository layout (proposed)
|
||||
|
||||
```
|
||||
voice-cat/
|
||||
├── docs/ # this folder
|
||||
├── core/ # libvoicecat (C++)
|
||||
│ ├── include/voicecat.h # the C ABI
|
||||
│ ├── src/{net,crypto,codec,protocol,session,audio}/
|
||||
│ └── proto/ # .proto definitions (shared source of truth)
|
||||
├── server/ # voicecat-server (C++, links core)
|
||||
├── clients/
|
||||
│ ├── apple/ # Swift package + Xcode project (macOS + iOS)
|
||||
│ └── windows/ # .NET solution (C#)
|
||||
├── tools/
|
||||
│ └── vccli/ # headless test client (C++), for protocol bring-up
|
||||
├── third_party/ # vendored / vcpkg manifest
|
||||
└── CMakeLists.txt
|
||||
```
|
||||
|
||||
Build is **CMake** with **vcpkg** (manifest mode) for C/C++ deps; the Apple and Windows UI
|
||||
projects consume the built core as a binary + headers. See [tech-stack.md](tech-stack.md).
|
||||
Unsupported C++ and Swift applications may remain temporarily during repository cleanup, but
|
||||
they are not dependencies, compatibility targets, or design authorities.
|
||||
|
||||
+52
-515
@@ -1,550 +1,87 @@
|
||||
# Building & Manual Testing
|
||||
# Building and testing
|
||||
|
||||
## .NET rewrite
|
||||
## Prerequisites
|
||||
|
||||
The managed wire/crypto, TLS, and codec/DSP slices are under `dotnet/`, targeting .NET 10.
|
||||
From the root (CMake and a C compiler are required for codec/DSP):
|
||||
- .NET SDK selected by `dotnet/global.json`
|
||||
- CMake and a C compiler for the Opus/RNNoise shim
|
||||
- PowerShell for the cross-platform build scripts
|
||||
- Xcode plus the pinned .NET macOS/iOS workloads for Apple clients
|
||||
|
||||
```powershell
|
||||
Dependencies and NuGet lock files are committed. GPL/LGPL dependencies are forbidden.
|
||||
|
||||
## Managed core, server, CLI, and tests
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
./dotnet/build-native.ps1
|
||||
dotnet restore dotnet/VoiceCat.slnx --locked-mode
|
||||
dotnet build dotnet/VoiceCat.slnx -c Release --no-restore
|
||||
dotnet test dotnet/VoiceCat.slnx -c Release --no-build
|
||||
./dotnet/check-licenses.ps1
|
||||
```
|
||||
|
||||
See `dotnet/README.md` for conformance fixtures and conventions. The C++ commands
|
||||
below remain required while the existing implementation is the migration oracle.
|
||||
The native script builds `native/media`, fetches checksum-pinned Opus 1.5.2, compiles vendored
|
||||
RNNoise, and stages the resulting library and notices in `dotnet/artifacts/native`.
|
||||
|
||||
This doc explains what each CMake preset in [`CMakePresets.json`](../CMakePresets.json) is
|
||||
*for*, which one to actually use day-to-day, and the commands to stand up a real server +
|
||||
`vccli` clients against each other for manual testing. For the one-paragraph quick-start see
|
||||
[`CLAUDE.md`](../CLAUDE.md); for `ctest` targets see [`AGENTS.md`](../AGENTS.md). This doc is
|
||||
the missing middle: *how the presets relate to each other*, *which platform each targets*,
|
||||
and *how to drive the binaries by hand*.
|
||||
|
||||
**Quick navigation:**
|
||||
|
||||
| What you want to build | Section | Key command |
|
||||
|------------------------|---------|-------------|
|
||||
| Server + `vccli` + tests (all platforms) | [§3](#3-build--test-the-loop-youll-run-constantly) | `cmake --preset dev && cmake --build --preset dev && ctest --preset dev` |
|
||||
| Production server (stripped, no tests) | [§5](#5-server-release-production-shaped-build) | `cmake --preset server-release && cmake --build --preset server-release` |
|
||||
| Windows client (C# / WinForms) | [§7](#7-windows-client-c--winforms) | `dotnet build clients/windows/VoiceCat.slnx` |
|
||||
| Managed macOS client (C# AppKit) | [§8](#8-macos-client-appkit) | `dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx` |
|
||||
| Swift macOS migration oracle | [§8](#8-macos-client-appkit) | `scripts/build-macos-client.sh` |
|
||||
| Managed iOS client (C# UIKit / simulator) | [§9](#9-ios-client-uikit) | `./dotnet/build-native-ios.sh && dotnet build clients/apple/dotnet/VoiceCat.iOS/VoiceCat.iOS.csproj` |
|
||||
| Launch iOS app on simulator | [§9](#9-ios-client-swiftui) | `scripts/run-ios-simulator.sh` |
|
||||
| Swift core + tests | [§8](#8-macos-client-appkit) | `cd clients/apple && swift test` |
|
||||
|
||||
## 1. What each preset is for
|
||||
|
||||
| Preset | Binary dir | Deps | Build type | Server | Tools | Tests | Strip | Platform | What it's for |
|
||||
|--------|-----------|------|------------|--------|-------|-------|-------|----------|---------------|
|
||||
| `vcpkg-common` | — | vcpkg | — | — | — | — | — | all | Hidden base. Sets the vcpkg toolchain wrapper ([`cmake/voicecat-toolchain.cmake`](../cmake/voicecat-toolchain.cmake)) which auto-resolves the triplet from the host platform. Not used directly. |
|
||||
| `dev` | `build/dev` | vcpkg | Debug | ON | ON | ON | no | all | **The one you actually want.** Day-to-day development: real protocol, crypto, voice, server — everything. Builds server + tools + tests (29 tests). Works on Windows, Linux, and macOS (triplet auto-resolved). |
|
||||
| `release` | `build/release` | vcpkg | Release | ON | ON | ON | no | all | Optimized build with the full test suite. Use to run tests against optimized code, profile, or catch optimizer-sensitive bugs. Symbols kept (not stripped) so stack traces and profiling remain useful. |
|
||||
| `server-release` | `build/server-release` | vcpkg | Release | ON | ON | OFF | **yes** | all | Production-shaped build for deployment. Optimized + stripped binaries (`-s`), no tests. This is what you'd ship/run — see [docs/deployment.md](deployment.md). |
|
||||
| `windows-client` | `build/windows-client` | vcpkg | Release | OFF | OFF | OFF | no | Windows | Produces a redistributable `voicecat.dll` for the C# WinForms client (M4). Static MinGW runtime — no `libgcc_s_seh-1.dll` etc. See [clients/windows/README.md](../clients/windows/README.md). |
|
||||
| `apple-dev` | `build/apple-dev` | vcpkg | Release | OFF | OFF | OFF | no | macOS | Static `libvoicecat.a` for the Swift Package / XCFramework (macOS slice). Validated on macOS 26.5 / Apple Silicon — builds green, produces valid arm64 `.a` + XCFramework. See [clients/apple/README.md](../clients/apple/README.md). |
|
||||
| `apple-ios` | `build/apple-ios` | vcpkg | Release | OFF | OFF | OFF | no | macOS→iOS | **Scaffolding** — cross-compiled static `libvoicecat.a` for iOS device (`arm64-ios`). One XCFramework slice. Not yet CI-validated. |
|
||||
| `apple-ios-sim` | `build/apple-ios-sim` | vcpkg | Release | OFF | OFF | OFF | no | macOS→iOS sim | **Scaffolding** — cross-compiled static `libvoicecat.a` for iOS simulator (`arm64-ios-sim`). One XCFramework slice. Not yet CI-validated. |
|
||||
|
||||
So in practice there are two presets that matter for day-to-day work:
|
||||
- **`dev`** — everything: real protocol, real voice, real manual testing. This is the loop you run constantly.
|
||||
- **`release`** — same suite, optimized. Run it when you want to check optimized behavior or profile.
|
||||
|
||||
The rest are purpose-specific: `server-release` for deployment, `windows-client` for the DLL,
|
||||
`apple-*` for Apple platform slices.
|
||||
|
||||
### Platform matrix
|
||||
|
||||
The vcpkg presets (`dev`, `release`, `server-release`, `windows-client`, `apple-*`) auto-resolve
|
||||
the vcpkg triplet via [`cmake/voicecat-toolchain.cmake`](../cmake/voicecat-toolchain.cmake):
|
||||
|
||||
| Host platform | Auto-resolved triplet | Notes |
|
||||
|---------------|----------------------|-------|
|
||||
| Windows (MinGW/MSYS2) | `x64-mingw-static` | The project's Windows toolchain. MSVC users must set `VCPKG_TARGET_TRIPLET=x64-windows` explicitly. |
|
||||
| Linux x64 | `x64-linux` | Server's primary deployment target (Docker, systemd). |
|
||||
| Linux arm64 | `arm64-linux` | Raspberry Pi / ARM VPS. |
|
||||
| macOS (Apple Silicon) | `arm64-osx` | `apple-dev` uses this automatically. |
|
||||
| macOS (Intel) | `x64-osx` | `apple-dev` uses this automatically. |
|
||||
|
||||
Cross-compile presets (`apple-ios`, `apple-ios-sim`) override `VCPKG_TARGET_TRIPLET` explicitly;
|
||||
`VCPKG_HOST_TRIPLET` stays the host's (e.g. `arm64-osx` when building iOS on Apple Silicon).
|
||||
|
||||
### Preset history
|
||||
|
||||
The preset set was cleaned up on 2026-06-18 (see `PROGRESS.md`). The old names map as follows:
|
||||
|
||||
| Old name | New name | Notes |
|
||||
|----------|----------|-------|
|
||||
| `m1-dev` | `dev` | Renamed — the project is past M5, so milestone-named presets were misleading. This is now the default development preset. |
|
||||
| `m2-dev` | *(dropped)* | Was cache-identical to `m1-dev` (same flags, same triplet, only the binary dir differed). Removed. |
|
||||
| `skeleton` | *(dropped)* | Removed — was a no-deps stub build mode used during M0. All subsystems are now fully implemented; the stub `#ifdef` scaffolding has been deleted. |
|
||||
| `server-release` | `server-release` | Unchanged name; now stripped (`-s`) and auto-triplet. |
|
||||
| *(new)* | `release` | New: optimized build with tests on, symbols kept. |
|
||||
| `windows-client` | `windows-client` | Unchanged name; triplet now auto-resolved. |
|
||||
| *(new)* | `apple-dev`, `apple-ios`, `apple-ios-sim` | New: Apple platform scaffolding. |
|
||||
|
||||
If you see `m1-dev` or `m2-dev` in old scripts, commits, or `PROGRESS.md` history entries,
|
||||
use `dev` instead. Historical `PROGRESS.md` entries are left intact as a true record of what
|
||||
was run.
|
||||
|
||||
## 2. One-time setup for the real-deps presets
|
||||
|
||||
vcpkg is bundled as a git submodule at [`vcpkg/`](../vcpkg), pinned to the exact commit in
|
||||
[`vcpkg.json`](../vcpkg.json)'s `builtin-baseline` — so the bundled checkout and the manifest's
|
||||
resolved port versions can never drift apart. All presets except `vcpkg-common` need it
|
||||
bootstrapped:
|
||||
Equivalent direct native build:
|
||||
|
||||
```bash
|
||||
# once, after cloning:
|
||||
git submodule update --init vcpkg
|
||||
./vcpkg/bootstrap-vcpkg.sh # .bat on Windows
|
||||
cmake -S native/media -B dotnet/artifacts/native-build -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build dotnet/artifacts/native-build --target voicecat_media --parallel 2
|
||||
cmake --install dotnet/artifacts/native-build --component DotnetMedia \
|
||||
--prefix dotnet/artifacts/native
|
||||
```
|
||||
|
||||
`cmake/voicecat-toolchain.cmake` resolves the vcpkg root itself — no environment variable
|
||||
needed. If you'd rather use an external vcpkg checkout (e.g. one shared across several
|
||||
projects), set `VCPKG_ROOT` and it takes priority over the bundled submodule:
|
||||
## Run locally
|
||||
|
||||
```bash
|
||||
# PowerShell:
|
||||
$env:VCPKG_ROOT = "D:\path\to\vcpkg"
|
||||
|
||||
# Linux/macOS:
|
||||
export VCPKG_ROOT=/path/to/vcpkg
|
||||
dotnet run --project dotnet/src/VoiceCat.Server -- --data-dir ./voicecat-data
|
||||
dotnet run --project dotnet/src/VoiceCat.Cli -- \
|
||||
--host 127.0.0.1 --port 8384 --nickname Alice --trust-first
|
||||
```
|
||||
|
||||
An external checkout must still be bootstrapped, and should be at (or compatible with)
|
||||
`vcpkg.json`'s `builtin-baseline` commit to resolve the same port versions.
|
||||
Both commands support `--help`. The CLI also supports deterministic two-process text and tone
|
||||
checks used by the managed test suite.
|
||||
|
||||
`vcpkg.json` (manifest mode) pins every dependency (protobuf, mbedTLS, libsodium, asio,
|
||||
sqlite3, spdlog, opus, miniaudio) — `cmake --preset dev` resolves and builds them
|
||||
automatically on first configure. That first configure is slow (vcpkg building from source);
|
||||
subsequent ones are cached.
|
||||
|
||||
## 3. Build + test (the loop you'll run constantly)
|
||||
|
||||
```bash
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev
|
||||
ctest --preset dev
|
||||
```
|
||||
|
||||
Binaries land in `build/dev/bin/` (`.exe` suffix on Windows):
|
||||
|
||||
- `build/dev/bin/voicecat-server`
|
||||
- `build/dev/bin/vccli`
|
||||
- `build/dev/bin/voicecat-admin`
|
||||
|
||||
To run the same suite against optimized code:
|
||||
|
||||
```bash
|
||||
cmake --preset release
|
||||
cmake --build --preset release
|
||||
ctest --preset release
|
||||
```
|
||||
|
||||
## 4. Manual testing: server + two clients
|
||||
|
||||
### Start the server
|
||||
|
||||
```bash
|
||||
./build/dev/bin/voicecat-server --name "Test Server" --data-dir ./voicecat-data
|
||||
```
|
||||
|
||||
First run generates an Ed25519 identity + self-signed cert under `--data-dir`, creates the
|
||||
SQLite store, and creates a default "Lobby" channel. Other server flags:
|
||||
|
||||
```
|
||||
--port <n> control+media port (default 8384)
|
||||
--data-dir <path> data directory (default ./voicecat-data)
|
||||
--name <name> server name
|
||||
--no-guests disable guest access (then provision accounts via voicecat-admin)
|
||||
--print-config print effective config and exit
|
||||
--version print version and exit
|
||||
```
|
||||
|
||||
### Drive it with `vccli`
|
||||
|
||||
`vccli` is the headless client used to exercise the protocol by hand. Full flag list:
|
||||
|
||||
```
|
||||
vccli [--host H] [--port P] [--nick NAME] [--channel ID]
|
||||
[--voice] [--mute] [--text MSG] [--list-devices]
|
||||
[--input-device ID] [--input-mode vad|ptt] [--share-screen-audio]
|
||||
|
||||
--host H server host (default 127.0.0.1)
|
||||
--port P server TCP port (default 8384)
|
||||
--nick NAME guest nickname (default vccli-test)
|
||||
--channel ID channel to join after auth (default 1, Lobby)
|
||||
--voice start a MIC stream and stay connected until Ctrl+C
|
||||
--mute start with the mic muted (only meaningful with --voice)
|
||||
--text MSG send MSG to the channel, then exit
|
||||
--list-devices print input/output devices (vc_list_devices) and exit
|
||||
--input-device ID use device ID (from --list-devices) for the MIC stream
|
||||
--input-mode vad|ptt send-side input gate mode (default vad)
|
||||
--share-screen-audio also start a SCREEN_AUDIO stream (WASAPI loopback on Windows)
|
||||
```
|
||||
|
||||
While `--voice` is running, stdin accepts `ptt on`, `ptt off`, `mode vad`, `mode ptt` to
|
||||
toggle the input gate live.
|
||||
|
||||
**Smoke test — two clients talking:**
|
||||
|
||||
```bash
|
||||
# terminal A
|
||||
./build/dev/bin/vccli --nick Alice --text "hello from Alice"
|
||||
|
||||
# terminal B (separate window, after A confirms it sent)
|
||||
./build/dev/bin/vccli --nick Bob --text "hello from Bob"
|
||||
```
|
||||
|
||||
**Real voice between two clients (needs working mic/speakers, two terminals):**
|
||||
|
||||
```bash
|
||||
# terminal A
|
||||
./build/dev/bin/vccli --nick Alice --voice
|
||||
|
||||
# terminal B
|
||||
./build/dev/bin/vccli --nick Bob --voice
|
||||
```
|
||||
|
||||
Speak into the mic on one side; you should hear it on the other. Ctrl+C to disconnect.
|
||||
|
||||
**Enumerate audio devices before picking one:**
|
||||
|
||||
```bash
|
||||
./build/dev/bin/vccli --list-devices
|
||||
./build/dev/bin/vccli --nick Alice --voice --input-device <ID> --input-mode ptt
|
||||
```
|
||||
|
||||
**Provisioning a non-guest account** (if the server was started with `--no-guests`):
|
||||
|
||||
```bash
|
||||
./build/dev/bin/voicecat-admin --data-dir ./voicecat-data account add alice --password secret
|
||||
./build/dev/bin/voicecat-admin --data-dir ./voicecat-data account list
|
||||
```
|
||||
|
||||
## 5. `server-release` (production-shaped build)
|
||||
|
||||
Same dependency story, but `Release` build type, stripped binaries, and no tests — this is the
|
||||
closest local analogue to what [docs/deployment.md](deployment.md)'s "from source" path
|
||||
produces:
|
||||
|
||||
```bash
|
||||
cmake --preset server-release
|
||||
cmake --build --preset server-release
|
||||
./build/server-release/bin/voicecat-server
|
||||
```
|
||||
|
||||
Use this to sanity-check release-mode behavior (e.g. perf, optimized codepaths) — not for
|
||||
day-to-day development, since it has no test target wired up. The `-s` linker flag strips
|
||||
symbol tables from the binaries, producing smaller executables suitable for distribution.
|
||||
|
||||
## 6. Apple platform builds
|
||||
|
||||
The `apple-dev`, `apple-ios`, and `apple-ios-sim` presets produce static `libvoicecat.a`
|
||||
slices for the Swift Package / XCFramework. All three are validated on macOS 26.5 / Apple
|
||||
Silicon. Build on macOS:
|
||||
|
||||
```bash
|
||||
# macOS slice (arm64-osx on Apple Silicon, x64-osx on Intel)
|
||||
cmake --preset apple-dev
|
||||
cmake --build --preset apple-dev
|
||||
# → build/apple-dev/lib/libvoicecat.a
|
||||
|
||||
# iOS device slice (arm64-ios)
|
||||
cmake --preset apple-ios
|
||||
cmake --build --preset apple-ios
|
||||
# → build/apple-ios/lib/libvoicecat.a
|
||||
|
||||
# iOS simulator slice (arm64-ios-simulator)
|
||||
cmake --preset apple-ios-sim
|
||||
cmake --build --preset apple-ios-sim
|
||||
# → build/apple-ios-sim/lib/libvoicecat.a
|
||||
```
|
||||
|
||||
You rarely need to run these CMake commands directly. The
|
||||
`clients/apple/scripts/build-xcframework.sh` script drives them internally and stitches
|
||||
the result into a self-contained `VoiceCatCore.xcframework`:
|
||||
|
||||
```bash
|
||||
# macOS slice only (default — used by macOS AppKit client)
|
||||
clients/apple/scripts/build-xcframework.sh
|
||||
|
||||
# All 3 slices (macOS + iOS device + iOS sim — required for iOS client)
|
||||
clients/apple/scripts/build-xcframework.sh --all
|
||||
```
|
||||
|
||||
See [clients/apple/README.md](../clients/apple/README.md) for the XCFramework internals
|
||||
(fat static lib merge, module map staging).
|
||||
|
||||
## 7. Windows client (C# / WinForms)
|
||||
|
||||
The Windows client is a .NET 10 WinForms app that loads `voicecat.dll` (the MinGW-built
|
||||
shared library from the `windows-client` preset) via P/Invoke. Full details in
|
||||
[`clients/windows/README.md`](../clients/windows/README.md).
|
||||
|
||||
**Prerequisites:** .NET SDK 10, MinGW-w64 / MSYS2 UCRT64 (GCC 13+), vcpkg.
|
||||
|
||||
### Build the DLL
|
||||
## Windows client
|
||||
|
||||
```powershell
|
||||
cmake --preset windows-client
|
||||
cmake --build --preset windows-client
|
||||
# → build/windows-client/bin/voicecat.dll
|
||||
./dotnet/build-native.ps1
|
||||
dotnet restore clients/windows/VoiceCat.slnx --locked-mode
|
||||
dotnet build clients/windows/VoiceCat.slnx -c Release --no-restore
|
||||
./clients/windows/publish-client.ps1
|
||||
```
|
||||
|
||||
### Build the C# solution
|
||||
The supported app references `VoiceCat.Managed` and the managed core. Published output must
|
||||
contain `voicecat_media.dll` and must not contain the retired `voicecat.dll`.
|
||||
|
||||
```powershell
|
||||
cd clients/windows
|
||||
dotnet build VoiceCat.slnx
|
||||
```
|
||||
## Apple clients
|
||||
|
||||
`Directory.Build.props` copies `voicecat.dll` into the output directory automatically.
|
||||
|
||||
### Run the app
|
||||
|
||||
```powershell
|
||||
# Terminal 1 — start the server (built with the dev preset)
|
||||
./build/dev/bin/voicecat-server.exe --name "My Server"
|
||||
|
||||
# Terminal 2 — launch the client
|
||||
dotnet run --project clients/windows/VoiceCat.App/VoiceCat.App.csproj
|
||||
```
|
||||
|
||||
### Run the C# interop tests
|
||||
|
||||
```powershell
|
||||
dotnet test clients/windows/VoiceCat.slnx
|
||||
```
|
||||
|
||||
## 8. macOS client (AppKit)
|
||||
|
||||
The replacement client is a C# AppKit application that consumes the managed core and the
|
||||
small Opus/RNNoise media shim. The Swift/Xcode app remains the migration oracle until the
|
||||
managed app passes its VoiceOver, live-call and notarization gates. Managed-client details are
|
||||
in [`clients/apple/dotnet/README.md`](../clients/apple/dotnet/README.md); Swift-oracle details
|
||||
remain in [`clients/apple/README.md`](../clients/apple/README.md).
|
||||
|
||||
**Managed prerequisites:** Xcode 27, .NET SDK/workload set 10.0.401, Homebrew `protobuf`, and
|
||||
macOS 14+ (the current bindings target `net10.0-macos27.0`).
|
||||
|
||||
### Build and validate the managed app
|
||||
|
||||
```bash
|
||||
cmake -S dotnet/native -B dotnet/artifacts/native-build \
|
||||
-DCMAKE_BUILD_TYPE=Release -DVOICECAT_DOTNET_RID=osx-arm64
|
||||
cmake --build dotnet/artifacts/native-build --config Release \
|
||||
--target voicecat_media --parallel 2
|
||||
cmake --install dotnet/artifacts/native-build --config Release \
|
||||
--component DotnetMedia --prefix dotnet/artifacts/native
|
||||
dotnet restore clients/apple/dotnet/VoiceCat.Apple.slnx
|
||||
dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx -c Debug
|
||||
open clients/apple/dotnet/VoiceCat.Mac/bin/Debug/net10.0-macos27.0/osx-arm64/VoiceCat.app
|
||||
```
|
||||
|
||||
Produce a Release bundle and verify its complete ad-hoc signature graph with:
|
||||
|
||||
```bash
|
||||
zsh clients/apple/dotnet/publish-macos.sh --dry-run
|
||||
```
|
||||
|
||||
For distribution, set `VOICECAT_CODESIGN_IDENTITY` to a Developer ID Application identity.
|
||||
Setting `APPLE_ID`, `APPLE_TEAM_ID`, and `APPLE_APP_PASSWORD` also submits the archive for
|
||||
notarization, staples it, and runs Gatekeeper assessment.
|
||||
|
||||
### Build the Swift migration oracle
|
||||
|
||||
**Prerequisites:** Xcode, vcpkg (`VCPKG_ROOT` set), macOS 14+ (deployment target).
|
||||
|
||||
### Build the XCFramework
|
||||
|
||||
The XCFramework is a local build artifact (gitignored, like the Windows DLL). It bundles
|
||||
`libvoicecat.a` + all vcpkg static deps into a single fat `.a` per slice, plus staged
|
||||
headers with a module map so Swift gets `import VoiceCatC`.
|
||||
|
||||
```bash
|
||||
# macOS slice only (default, validated)
|
||||
clients/apple/scripts/build-xcframework.sh
|
||||
# → clients/apple/VoiceCatCore.xcframework/
|
||||
|
||||
# All 3 slices (macOS + iOS device + iOS sim — iOS still scaffolding)
|
||||
clients/apple/scripts/build-xcframework.sh --all
|
||||
```
|
||||
|
||||
The script runs `cmake --preset apple-dev` + `cmake --build --preset apple-dev` internally,
|
||||
then merges vcpkg's static deps with `libtool -static` and stitches the XCFramework with
|
||||
`xcodebuild -create-xcframework`.
|
||||
|
||||
### Build the Swift core (SPM)
|
||||
|
||||
```bash
|
||||
cd clients/apple
|
||||
swift build # builds VoiceCatCore library
|
||||
swift test # 6 smoke tests against a real voicecat-server
|
||||
```
|
||||
|
||||
`swift test` requires the `dev` CMake preset to be built
|
||||
(`build/dev/bin/voicecat-server` + `voicecat-admin`).
|
||||
|
||||
### Build the macOS app (Xcode)
|
||||
|
||||
```bash
|
||||
xcodebuild -project clients/apple/macOS/VoiceCatMac.xcodeproj \
|
||||
-scheme VoiceCatMac -configuration Debug build
|
||||
# → ~/Library/Developer/Xcode/DerivedData/VoiceCatMac-*/Build/Products/Debug/VoiceCatMac.app
|
||||
```
|
||||
|
||||
Or open the project in Xcode and build from the UI:
|
||||
|
||||
```bash
|
||||
open clients/apple/macOS/VoiceCatMac.xcodeproj
|
||||
```
|
||||
|
||||
### Run the app
|
||||
|
||||
```bash
|
||||
# Terminal 1 — start the server (built with the dev preset)
|
||||
./build/dev/bin/voicecat-server --name "My Server"
|
||||
|
||||
# Terminal 2 — launch the client
|
||||
open ~/Library/Developer/Xcode/DerivedData/VoiceCatMac-*/Build/Products/Debug/VoiceCatMac.app
|
||||
```
|
||||
|
||||
Or use the per-artifact script which builds and stages to `dist/macos-client/`:
|
||||
|
||||
```bash
|
||||
scripts/build-macos-client.sh
|
||||
```
|
||||
|
||||
## 9. iOS client (UIKit)
|
||||
|
||||
The replacement iOS client is a native C# UIKit app at
|
||||
`clients/apple/dotnet/VoiceCat.iOS`. Build its static media shim and simulator bundle with:
|
||||
On Apple Silicon with the SDK/workload versions documented in
|
||||
`clients/apple/dotnet/README.md`:
|
||||
|
||||
```bash
|
||||
./dotnet/build-native.ps1
|
||||
./dotnet/build-native-ios.sh
|
||||
dotnet restore clients/apple/dotnet/VoiceCat.Apple.slnx
|
||||
dotnet build clients/apple/dotnet/VoiceCat.iOS/VoiceCat.iOS.csproj \
|
||||
-c Debug -r iossimulator-arm64 --no-restore
|
||||
dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx -c Debug --no-restore
|
||||
```
|
||||
|
||||
The build invokes Xcode for the retained Swift ReplayKit extension and embeds the resulting
|
||||
appex. The older SwiftUI application below remains the migration oracle during parity and
|
||||
physical-device accessibility testing.
|
||||
The iOS build invokes the standalone project in `native/apple/broadcast` and embeds its appex.
|
||||
Use `clients/apple/dotnet/build-ios-device.sh` and `deploy-ios-device.sh` for signed device
|
||||
builds. Use `clients/apple/dotnet/publish-macos.sh --dry-run` for an ad-hoc validated macOS
|
||||
bundle; its environment variables enable Developer ID signing and notarization.
|
||||
|
||||
The iOS client is an Xcode project (`clients/apple/iOS/VoiceCatiOS.xcodeproj`) that
|
||||
links `libvoicecat` via the same `VoiceCatCore` Swift Package as the macOS client.
|
||||
Full details in [`clients/apple/README.md`](../clients/apple/README.md).
|
||||
## Server publishing
|
||||
|
||||
**Prerequisites:** Xcode, vcpkg (`VCPKG_ROOT` set), iOS Simulator runtime installed
|
||||
(Xcode > Settings > Platforms > iOS). iOS deployment target: 18.0.
|
||||
`dotnet/publish-server.ps1` produces locked self-contained Windows and Linux artifacts.
|
||||
`Dockerfile`, Compose configuration, and systemd packaging use the managed server. Validate a
|
||||
published binary with its TLS `--health-check`, preferably including `--expect-fingerprint`.
|
||||
|
||||
### Build the XCFramework (all slices)
|
||||
## CI and release expectations
|
||||
|
||||
The iOS build requires the `ios-arm64-simulator` slice in the XCFramework — not just the
|
||||
macOS slice. Use the `--all` flag to produce all three slices:
|
||||
|
||||
```bash
|
||||
clients/apple/scripts/build-xcframework.sh --all
|
||||
# → clients/apple/VoiceCatCore.xcframework/
|
||||
# ├── macos-arm64/
|
||||
# ├── ios-arm64/
|
||||
# └── ios-arm64-simulator/
|
||||
```
|
||||
|
||||
### Build the iOS simulator app
|
||||
|
||||
```bash
|
||||
# Via the convenience script (recommended):
|
||||
scripts/build-ios-client.sh
|
||||
# → clients/apple/iOS/build/Debug-iphonesimulator/VoiceCatiOS.app
|
||||
# → dist/ios-client/VoiceCatiOS.app
|
||||
|
||||
# Or as a single command (what the script does under the hood):
|
||||
SIM_SDK="iphonesimulator$(xcrun --sdk iphonesimulator --show-sdk-version)"
|
||||
BUILD_DIR="$(pwd)/clients/apple/iOS/build"
|
||||
xcodebuild \
|
||||
-project clients/apple/iOS/VoiceCatiOS.xcodeproj \
|
||||
-target VoiceCatiOS \
|
||||
-sdk "$SIM_SDK" \
|
||||
-configuration Debug \
|
||||
CODE_SIGNING_ALLOWED=NO \
|
||||
ARCHS=arm64 \
|
||||
ONLY_ACTIVE_ARCH=YES \
|
||||
SYMROOT="$BUILD_DIR" \
|
||||
OBJROOT="$BUILD_DIR" \
|
||||
build
|
||||
```
|
||||
|
||||
**Why `-target` instead of `-scheme -destination`?** Using `-scheme VoiceCatiOS
|
||||
-destination 'platform=iOS Simulator,OS=latest'` requires a simulator runtime whose iOS
|
||||
version exactly matches the SDK version (`iphonesimulatorX.Y`). If you have an older
|
||||
runtime installed (common when the SDK ships ahead of runtime availability in Xcode), the
|
||||
build fails with "Unable to find a destination matching the provided destination specifier."
|
||||
Using `-target` bypasses destination matching and builds against the SDK directly.
|
||||
|
||||
**Why `SYMROOT=OBJROOT=clients/apple/iOS/build`?** When building with `-target` (not
|
||||
`-scheme`), the local Swift Package (`VoiceCatCore`) resolves its build products relative
|
||||
to `OBJROOT`. By default, SPM resolves into `clients/apple/build/`, while the app target
|
||||
looks in `clients/apple/iOS/build/`. Pointing both to the same directory fixes the
|
||||
"unable to resolve module dependency: 'VoiceCatCore'" error.
|
||||
|
||||
### Run on the iOS Simulator
|
||||
|
||||
```bash
|
||||
# Via the script (finds or boots an iPhone simulator, installs, launches):
|
||||
scripts/run-ios-simulator.sh
|
||||
|
||||
# Build and run in one step:
|
||||
scripts/run-ios-simulator.sh --build
|
||||
|
||||
# Stream app logs after launch:
|
||||
scripts/run-ios-simulator.sh --log
|
||||
|
||||
# Target a specific device by name or UDID:
|
||||
scripts/run-ios-simulator.sh --device "iPhone 16 Pro"
|
||||
scripts/run-ios-simulator.sh --udid XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
|
||||
```
|
||||
|
||||
Under the hood the script uses `xcrun simctl` commands:
|
||||
|
||||
```bash
|
||||
# Boot a simulator (if not already running):
|
||||
xcrun simctl boot <UDID>
|
||||
open -a Simulator
|
||||
xcrun simctl bootstatus <UDID> -b # wait until boot is complete before installing
|
||||
|
||||
# Install the built .app:
|
||||
xcrun simctl install <UDID> clients/apple/iOS/build/Debug-iphonesimulator/VoiceCatiOS.app
|
||||
|
||||
# Launch the app:
|
||||
xcrun simctl launch <UDID> cat.voice.VoiceCatiOS
|
||||
|
||||
# Stream logs (Ctrl+C to stop — does not kill the app):
|
||||
xcrun simctl spawn <UDID> log stream --predicate 'subsystem contains "VoiceCat"'
|
||||
```
|
||||
|
||||
### Run the full stack (server + iOS simulator)
|
||||
|
||||
```bash
|
||||
# Terminal 1 — start the server
|
||||
./build/dev/bin/voicecat-server --name "My Server"
|
||||
|
||||
# Terminal 2 — build + launch iOS client on simulator
|
||||
scripts/run-ios-simulator.sh --build
|
||||
```
|
||||
|
||||
On first connect the app will show a TOFU identity sheet — accept it, then join a channel.
|
||||
|
||||
### Open in Xcode
|
||||
|
||||
```bash
|
||||
open clients/apple/iOS/VoiceCatiOS.xcodeproj
|
||||
```
|
||||
|
||||
Xcode can build and run on the simulator directly. The XCFramework must already exist
|
||||
(`clients/apple/VoiceCatCore.xcframework/`) — run `build-xcframework.sh --all` once
|
||||
before opening Xcode.
|
||||
CI builds the native shim, managed solution, tests, licenses, and managed Apple clients. The
|
||||
old C++ implementation is not a conformance target. Release validation additionally includes
|
||||
real devices, screen readers, sustained calls, signing/notarization, container execution, and
|
||||
a server soak; see `roadmap.md`.
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
# Deploying the managed iOS app
|
||||
|
||||
VoiceCat's supported iOS client is the .NET 10 UIKit application in
|
||||
`clients/apple/dotnet/VoiceCat.iOS`. The checked-in scripts build its native Opus/RNNoise
|
||||
dependency, compile and sign the managed app and Swift ReplayKit extension, stage the bundle,
|
||||
install it, and launch it on a paired physical device.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- macOS 27, Xcode 27, .NET SDK 10.0.401, and the matching iOS workload.
|
||||
- An unlocked, trusted iPhone connected by USB or visible to Xcode over the network.
|
||||
- Apple Development profiles for `me.iamtalon.voicecat` and
|
||||
`me.iamtalon.voicecat.broadcast`, both with App Group `group.me.iamtalon.voicecat`.
|
||||
|
||||
List available devices:
|
||||
|
||||
```bash
|
||||
clients/apple/dotnet/deploy-ios-device.sh --list
|
||||
```
|
||||
|
||||
## Build, install, and launch
|
||||
|
||||
Set the Apple developer team explicitly. The .NET host and Xcode-built extension use separate
|
||||
build systems, and Xcode needs the team to select the extension profile.
|
||||
|
||||
```bash
|
||||
export VOICECAT_DEVELOPMENT_TEAM=FJV8L966W4
|
||||
clients/apple/dotnet/build-ios-device.sh --configuration Debug
|
||||
clients/apple/dotnet/deploy-ios-device.sh \
|
||||
--device "Talon’s iPhone" \
|
||||
--configuration Debug \
|
||||
--no-build
|
||||
```
|
||||
|
||||
The verified bundle is staged at `dist/ios-managed-device/VoiceCat.iOS.app`. Omit `--no-build`
|
||||
to build and deploy in one command. Add `--console` to attach the launch to device logs.
|
||||
|
||||
Confirm that the process stayed alive:
|
||||
|
||||
```bash
|
||||
xcrun devicectl device info processes --device "Talon’s iPhone" | rg VoiceCat
|
||||
```
|
||||
|
||||
## Signing findings
|
||||
|
||||
The development team ID is `FJV8L966W4`. Do not infer it from the parenthesized suffix in an
|
||||
Apple Development certificate name; that value can identify the certificate holder and need
|
||||
not equal a profile's `TeamIdentifier`.
|
||||
|
||||
The extension wrapper uses automatic signing with installed profiles. If a profile is absent
|
||||
and Xcode has a valid signed-in developer account, permit Xcode to create or download it:
|
||||
|
||||
```bash
|
||||
export VOICECAT_ALLOW_PROVISIONING_UPDATES=1
|
||||
```
|
||||
|
||||
Leave that variable unset when valid profiles are already installed. A stale command-line Xcode
|
||||
account can otherwise make provisioning updates fail even though offline signing can succeed.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Stale CMake source path
|
||||
|
||||
After moving the native shim from `dotnet/native` to `native/media`, an old CMake cache can
|
||||
report a source-directory mismatch. Move the generated directories aside and rebuild:
|
||||
|
||||
```bash
|
||||
mv dotnet/artifacts/native-build-ios-arm64-cmake /tmp/
|
||||
mv dotnet/artifacts/native-build-iossimulator-arm64-cmake /tmp/
|
||||
```
|
||||
|
||||
### Locked restore reports changed runtime identifiers
|
||||
|
||||
The managed libraries' lock files must include `ios-arm64` and `iossimulator-arm64`. Regenerate
|
||||
them after runtime changes, then verify that dependency versions did not change:
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet restore \
|
||||
clients/apple/dotnet/VoiceCat.iOS/VoiceCat.iOS.csproj \
|
||||
-p:VoiceCatIosStatic=true \
|
||||
--force-evaluate
|
||||
```
|
||||
|
||||
### Developer disk image cannot be mounted
|
||||
|
||||
CoreDevice errors 10003 or 12040 mean the phone locked. Unlock it, keep the display awake, and
|
||||
rerun deployment with `--no-build`.
|
||||
|
||||
## Verified hardware result
|
||||
|
||||
On 2026-09-19, the Debug build completed with .NET 10.0.401 and Xcode 27.0. The host and
|
||||
ReplayKit extension were signed under team `FJV8L966W4`, installed wirelessly on Talon's iPhone,
|
||||
launched as `me.iamtalon.voicecat`, and remained in the device process list. Audio,
|
||||
background/lock behavior, Bluetooth, ReplayKit, ScreenCaptureKit, and VoiceOver remain separate
|
||||
manual hardware gates.
|
||||
@@ -1,965 +0,0 @@
|
||||
# Porting VoiceCat to pure .NET / C#
|
||||
|
||||
**Status:** the managed protocol, crypto, server, CLI, audio/client core, Windows client and
|
||||
macOS functional surface are implemented under `dotnet/`, with C++ interoperability retained
|
||||
as a migration oracle. The managed macOS client is awaiting its manual VoiceOver, live-call and
|
||||
credentialed notarization gates before it replaces the Swift release client. The iOS rewrite
|
||||
remains planned.
|
||||
See `dotnet/README.md`, `docs/api-dotnet.md`, and `PROGRESS.md` for verification and next steps.
|
||||
**Target runtime:** .NET 10 LTS (in-service to Nov 2028), with .NET 11 as the follow-on.
|
||||
**Scope:** replace the C++ core (`libvoicecat`), the C++ server, the C++ `vccli`, and the
|
||||
Swift macOS/iOS clients with a single C# codebase. The Windows WinForms client is already C#
|
||||
and is mostly *kept*.
|
||||
|
||||
This document is the map for that work: what maps 1:1, what has no .NET equivalent, what has
|
||||
to stay native, and the order to do it in so the tree is testable at every step.
|
||||
|
||||
---
|
||||
|
||||
## 0. Executive summary
|
||||
|
||||
**The port is feasible.** Roughly 80 % of the C++ core is protocol/state-machine/buffer code
|
||||
that translates to C# almost mechanically and gets *shorter*. The risk is concentrated in
|
||||
four places, and only one of them is a genuine design change:
|
||||
|
||||
| Risk | Verdict |
|
||||
|------|---------|
|
||||
| **TLS keying-material exporter (RFC 5705)** — the media-key derivation the whole UDP path depends on | ⚠️ **`SslStream` cannot do this.** [The API is an unapproved proposal](https://github.com/dotnet/runtime/issues/112529) targeting "Future", and SChannel structurally can't export secrets. **Must** use BouncyCastle's managed TLS stack, or change the protocol. See §3. |
|
||||
| **Real-time audio + GC** | Manageable, but needs deliberate design. Native audio callbacks must not enter managed code. See §5. |
|
||||
| **Opus 1.6 / DRED, RNNoise** | No managed equivalent exists. Stay native via P/Invoke. See §4. |
|
||||
| **iOS ReplayKit Broadcast Upload Extension** | ⚠️ **Keep this in Swift.** 50 MB jetsam cap + a managed runtime in an appex that .NET for iOS does not officially support. See §8.4. |
|
||||
|
||||
Everything else — protobuf, SQLite, sockets, Argon2id, ChaCha20-Poly1305, X.509 generation,
|
||||
jitter buffers, the mixer, the session model, the server — is either built into .NET or
|
||||
covered by a permissive, well-maintained NuGet package.
|
||||
|
||||
**Net effect on native dependencies:** from 8 vcpkg deps + 1 vendored, down to **3 native
|
||||
libraries** (libopus, RNNoise, miniaudio) — all three tiny, all three already vendored or
|
||||
trivially buildable, and all three optional to *replace* later.
|
||||
|
||||
---
|
||||
|
||||
## 1. What exists today (baseline inventory)
|
||||
|
||||
Sizes are source bytes, to calibrate effort.
|
||||
|
||||
### Core — `core/` (~310 KB C++)
|
||||
|
||||
| File | Bytes | What it is | Port difficulty |
|
||||
|------|-------|-----------|-----------------|
|
||||
| `core/src/core/client.cpp` + `.h` | 107 K | `vc_client` — the entire client state machine: connect/auth/TOFU, channel + user model, stream lifecycle, reframing, encode path, event queue | **Medium**, but large. Mostly mechanical. |
|
||||
| `core/src/audio/audio_engine.cpp` + `.h` | 69 K | miniaudio devices, `JitterBuffer`, per-ssrc decode, DRED/FEC/PLC ladder, mixer, level meters, talk detection, external feed/tap/mixed-sink | **Hard** — the RT-sensitive part |
|
||||
| `core/src/net/transport.cpp` + `.h` | 25 K | Asio TCP framing `[u32 len][payload]`, UDP socket | **Easy** — `Socket`/`System.IO.Pipelines` is nicer |
|
||||
| `core/src/crypto/` | 27 K | mbedTLS TLS 1.3 wrapper, exporter, self-signed cert gen, Ed25519 identity, ChaCha20-Poly1305 + anti-replay, TOFU pin store | **Hard** — see §3 |
|
||||
| `core/src/codec/opus_codec.*` | 9 K | libopus encode/decode wrapper incl. DRED | **Easy** — P/Invoke shim |
|
||||
| `core/src/protocol/`, `core/src/session/` | 12 K | Envelope (de)serialize, dispatch, channel/user/stream registry | **Easy** |
|
||||
| `core/src/audio/apm_processor.*` | 5 K | RNNoise wrapper + energy VAD | **Easy** |
|
||||
| `core/src/voicecat.cpp` | 13 K | C ABI façade over `vc_client` | **Deleted** — no ABI needed any more |
|
||||
| `core/include/voicecat.h` | 27 K | The C ABI | **Becomes a C# interface**, not an ABI |
|
||||
| `core/proto/voicecat.proto` | 9 K | Wire format, source of truth | **Unchanged** |
|
||||
|
||||
### Server — `server/` (~80 KB C++)
|
||||
|
||||
`conn_session.cpp` (34 K, per-connection protocol handling), `db.cpp` (26 K, SQLite:
|
||||
accounts, channels, bans), `session_registry.cpp` (20 K), `media_relay.cpp` (8 K, the SFU
|
||||
relay), `server.cpp`, `identity.cpp`. All **easy-to-medium** — this is ordinary async network
|
||||
server code and translates very well.
|
||||
|
||||
### Clients
|
||||
|
||||
| Client | Today | After the port |
|
||||
|--------|-------|----------------|
|
||||
| Windows | C#, WinForms, `net10.0-windows`, ~40 files | **Kept.** Swap `VoiceCat.Interop` P/Invoke for a direct project reference. |
|
||||
| macOS | Swift + AppKit, `MainWindowController.swift` alone is 70 K | Rewrite as C# AppKit on `net10.0-macos` — near-mechanical, AppKit maps 1:1 |
|
||||
| iOS | Swift + SwiftUI, ~150 K across views/audio | Rewrite as C# UIKit on `net10.0-ios` (or MAUI — see §8.3). **No 1:1 SwiftUI equivalent.** |
|
||||
| iOS broadcast appex | Swift, 12 K (`SampleHandler.swift` + `BroadcastAudioRing.swift`) | **Stays Swift.** See §8.4 |
|
||||
| `tools/vccli` | C++, 32 K | Rewrite as a C# console app — this becomes the primary conformance harness |
|
||||
|
||||
### Tests — `tests/` (29 files, ~290 KB, all green)
|
||||
|
||||
These are the real specification. Every one of them must be ported to xUnit and stay green;
|
||||
the milestone exit criteria in `docs/roadmap.md` are encoded here.
|
||||
|
||||
---
|
||||
|
||||
## 2. Target solution layout
|
||||
|
||||
```
|
||||
voice-cat/
|
||||
├── proto/voicecat.proto # unchanged, single source of truth
|
||||
├── native/ # the only C left
|
||||
│ ├── opus/ # libopus 1.6 build scripts
|
||||
│ ├── rnnoise/ # moved from third_party/
|
||||
│ ├── miniaudio/ # miniaudio.h + voicecat_audio_shim.c (§5.2)
|
||||
│ └── build-native.{ps1,sh} # produces per-RID binaries
|
||||
├── src/
|
||||
│ ├── VoiceCat.Protocol/ # Google.Protobuf codegen + Envelope framing
|
||||
│ ├── VoiceCat.Crypto/ # TLS, media AEAD, anti-replay, identity, TOFU store
|
||||
│ ├── VoiceCat.Codec/ # libopus P/Invoke + OpusEncoder/OpusDecoder
|
||||
│ ├── VoiceCat.Dsp/ # RNNoise P/Invoke, energy VAD, resample helpers
|
||||
│ ├── VoiceCat.Audio/ # devices, jitter buffer, mixer, RT ring buffers
|
||||
│ ├── VoiceCat.Core/ # VoiceCatClient — replaces vc_client + the C ABI
|
||||
│ ├── VoiceCat.Server/ # replaces server/
|
||||
│ ├── VoiceCat.Cli/ # replaces tools/vccli + voicecat-admin
|
||||
│ └── clients/
|
||||
│ ├── VoiceCat.Windows/ # net10.0-windows, WinForms (kept, retargeted)
|
||||
│ ├── VoiceCat.Mac/ # net10.0-macos, AppKit
|
||||
│ ├── VoiceCat.iOS/ # net10.0-ios, UIKit
|
||||
│ └── VoiceCatBroadcast/ # Swift appex — the one non-C# artifact
|
||||
└── tests/VoiceCat.Tests/ # xUnit, ports all 29 ctest cases
|
||||
```
|
||||
|
||||
**Target frameworks.** `VoiceCat.Core` and everything below it target plain `net10.0` — no
|
||||
platform TFM — so the same assembly loads into the server, the WinForms app, the macOS app,
|
||||
the iOS app, and the test host. Only the four leaf client projects carry a platform TFM.
|
||||
|
||||
**Why not one big assembly:** the layering is what keeps the "core owns audio, UI is thin"
|
||||
rule enforceable. It also lets the server reference `VoiceCat.Protocol` + `VoiceCat.Crypto`
|
||||
without dragging in miniaudio.
|
||||
|
||||
---
|
||||
|
||||
## 3. The TLS problem — read this before anything else
|
||||
|
||||
### 3.1 What breaks
|
||||
|
||||
`docs/security.md` §2 is built on one mbedTLS call:
|
||||
|
||||
```c
|
||||
mbedtls_ssl_export_keying_material("voicecat media v1", ...) → media keys
|
||||
```
|
||||
|
||||
**.NET has no equivalent.** `SslStream` exposes no RFC 5705 exporter. The API proposal
|
||||
([dotnet/runtime#112529](https://github.com/dotnet/runtime/issues/112529)) is labelled
|
||||
`api-suggestion` ("NOT ready for implementation"), milestone *Future*, and the platform notes
|
||||
on it are discouraging: *"Windows – needs verification"* (SChannel runs TLS in a separate
|
||||
privileged process and deliberately refuses to hand back secrets), *"OSX – Not implemented
|
||||
for Secure Transport"*. Do not plan around this shipping.
|
||||
|
||||
This is not a small dependency swap. The media key derivation is the root of the entire UDP
|
||||
path: AEAD keys, nonce discipline, anti-replay, and the UDP binding token all hang off it.
|
||||
|
||||
### 3.2 Option A (recommended) — BouncyCastle managed TLS
|
||||
|
||||
[BouncyCastle for .NET](https://www.nuget.org/packages/BouncyCastle.Cryptography) (MIT,
|
||||
actively maintained) ships a **complete managed TLS 1.3 client and server** in
|
||||
`Org.BouncyCastle.Tls`, and `TlsContext` exposes exactly the method we need:
|
||||
|
||||
```csharp
|
||||
byte[] ExportKeyingMaterial(string asciiLabel, byte[] contextValue, int length);
|
||||
// "Export keying material according to RFC 5705" — TLS 1.3 (RFC 8446 §7.5) aware
|
||||
```
|
||||
|
||||
*(verified against [bc-csharp `crypto/src/tls/TlsContext.cs`](https://github.com/bcgit/bc-csharp/blob/master/crypto/src/tls/TlsContext.cs); the C# port is feature-matched to bc-java here.)*
|
||||
|
||||
**Consequences — mostly good:**
|
||||
|
||||
- ✅ **Byte-identical wire compatibility with the existing C++ implementation.** This is the
|
||||
single biggest de-risking factor in the whole project: it means a .NET client can talk to
|
||||
the shipped C++ server (and vice versa) at *every* step of the port, and the C++ side
|
||||
becomes a conformance oracle. See §11.
|
||||
- ✅ **Identical TLS behaviour on all five platforms.** No SChannel-vs-OpenSSL-vs-
|
||||
SecureTransport variance, no per-OS cipher-suite policy surprises, no "TLS 1.3 on macOS
|
||||
only since .NET 10" caveat. For a self-hosted product shipping to unknown machines this is
|
||||
worth a lot on its own.
|
||||
- ✅ MIT, permissive — satisfies the hard no-GPL rule.
|
||||
- ✅ Also gives us **Ed25519** and **BLAKE2b** (see §4), which .NET lacks.
|
||||
|
||||
**Costs:**
|
||||
|
||||
- ⚠️ Pure-managed TLS is slower than SChannel/OpenSSL. **This does not matter here** — the
|
||||
TLS channel carries only control messages (a handshake plus a few KB/s of protobuf). Media
|
||||
is UDP + ChaCha20-Poly1305, which uses the fast built-in .NET AEAD, not BouncyCastle.
|
||||
- ⚠️ You implement `TlsClient` / `TlsServer` callback subclasses yourself (~200–300 lines for
|
||||
both sides): cipher-suite selection, certificate handling, `TlsCrypto` provider. Well-trodden
|
||||
— BC ships `DefaultTlsClient`/`DefaultTlsServer` bases and `BcTlsCrypto`.
|
||||
- ⚠️ You own the cert-validation logic (that's actually a *plus* for TOFU — see §3.4).
|
||||
- ⚠️ One more external dependency in the trust base. It's Bouncy Castle; acceptable.
|
||||
|
||||
### 3.3 Option B — `SslStream` + in-band media keys (protocol v3)
|
||||
|
||||
Abandon the exporter. Since the TLS 1.3 channel is already confidential, authenticated and
|
||||
forward-secret, the server can simply **generate the media keys and send them inside it**:
|
||||
|
||||
```proto
|
||||
message AuthResult {
|
||||
// ...
|
||||
bytes udp_token = 6;
|
||||
bytes media_key_c2s = 7; // 32 bytes, server-generated CSPRNG (NEW, v3)
|
||||
bytes media_key_s2c = 8; // 32 bytes (NEW, v3)
|
||||
}
|
||||
```
|
||||
|
||||
This is what Mumble does (its OCB2/AES key exchange happens inside its TLS control channel),
|
||||
and it is what SDES-SRTP-over-TLS does. Security posture is equivalent: an attacker who can
|
||||
read these can already read everything.
|
||||
|
||||
- ✅ Uses only built-in `System.Net.Security.SslStream` — zero TLS dependencies.
|
||||
- ✅ Simplest possible code; best per-platform TLS performance.
|
||||
- ❌ **Breaks wire compatibility** → no cross-testing against the C++ implementation during
|
||||
the port, which throws away the best safety net available.
|
||||
- ❌ Protocol version bump to 3, forced-update for the shipped iOS/macOS/Windows clients.
|
||||
- ❌ Loses the exporter's nice property that media keys are never *transmitted* at all.
|
||||
- ⚠️ Server-side `SslStream` on Windows has a real footgun: a cert created with
|
||||
`CertificateRequest.CreateSelfSigned` must be round-tripped through
|
||||
`X509CertificateLoader.LoadPkcs12(cert.Export(X509ContentType.Pfx), null)` before SChannel
|
||||
will accept it — an ephemeral/CNG-only key produces a confusing handshake failure.
|
||||
|
||||
### 3.4 Recommendation
|
||||
|
||||
**Take Option A (BouncyCastle) for the port. Keep Option B as an optional later
|
||||
simplification** once the C++ tree is retired and you no longer need it as an oracle — at
|
||||
which point it's a contained protocol-v3 change, not a rewrite.
|
||||
|
||||
TOFU is *easier* under Option A: BouncyCastle hands you the peer's DER certificate chain
|
||||
directly in `TlsAuthentication.NotifyServerCertificate`, so
|
||||
`SHA256.HashData(leafDer)` — the exact value `docs/security.md` §1.1 says is pinned — falls
|
||||
out with no ceremony. (Under `SslStream` you'd get it from
|
||||
`RemoteCertificateValidationCallback` via `cert.GetRawCertData()`; also fine, just less
|
||||
direct.)
|
||||
|
||||
**Worth fixing while you're in here:** `security.md` §1.1 documents a known limitation — the
|
||||
Ed25519 identity key is not bound to the TLS cert, so it's display-only. When generating the
|
||||
self-signed cert in C#, **embed the Ed25519 public key as a X.509 extension or SAN URI**.
|
||||
`CertificateRequest.CertificateExtensions` makes this trivial, and it closes the gap the doc
|
||||
has been carrying.
|
||||
|
||||
---
|
||||
|
||||
## 4. Dependency map — C++ → .NET
|
||||
|
||||
| Concern | Today | .NET replacement | License | Notes / pitfalls |
|
||||
|---------|-------|------------------|---------|------------------|
|
||||
| Sockets, timers | Asio | **`System.Net.Sockets`** + `System.IO.Pipelines` + `PeriodicTimer` | built-in | Strictly better. `Socket.ReceiveFromAsync(SocketAddress)` (net8+) is the allocation-free UDP receive path — use it, not `UdpClient`. |
|
||||
| Control framing | hand-rolled `[u32 len]` | `System.IO.Pipelines` `SequenceReader` | built-in | Removes a class of bugs. Keep the same 16 MiB frame cap. |
|
||||
| TLS 1.3 | mbedTLS | **BouncyCastle `Org.BouncyCastle.Tls`** | MIT | See §3. **Not `SslStream`.** |
|
||||
| Media AEAD | libsodium ChaCha20-Poly1305 | **`System.Security.Cryptography.ChaCha20Poly1305`** | built-in | ⚠️ Check `ChaCha20Poly1305.IsSupported` at startup — it is OS-backed (Windows 10 1903+ / OpenSSL 1.1+). Fall back to BouncyCastle's `ChaCha20Poly1305` if false. Same 12-byte nonce, 16-byte tag → identical wire bytes. |
|
||||
| Anti-replay window | hand-rolled 64-bit | port verbatim | — | ~40 lines. Keep the RFC 3711 §3.3 ordering (replay-check → authenticate → *then* advance). This ordering is load-bearing; `test_media_aead.cpp` covers it. |
|
||||
| Argon2id | libsodium `crypto_pwhash` | **BouncyCastle `Argon2BytesGenerator`** | MIT | Implemented with a strict libsodium PHC parser and original costs; native database tests prove existing-account import and managed-account verification by C++. See `docs/api-dotnet.md` for cost bounds. |
|
||||
| BLAKE2b (channel passwords) | libsodium `crypto_generichash` | **`Blake2Fast`** (MIT) or BouncyCastle `Blake2bDigest` | MIT | Salted BLAKE2b-256, must produce identical digests to keep existing channel passwords working. Blake2Fast is SIMD and fast enough for the net thread, preserving the reason BLAKE2b was chosen over Argon2 here. |
|
||||
| Ed25519 identity | libsodium | **BouncyCastle `Ed25519Signer`** | MIT | ⚠️ **Not in .NET 10.** [dotnet/runtime#63174](https://github.com/dotnet/runtime/issues/63174) is api-approved but milestoned **11.0.0**. Since Option A already pulls in BouncyCastle, this is free. |
|
||||
| Self-signed cert gen | mbedTLS x509write | **`CertificateRequest.CreateSelfSigned`** | built-in | Much nicer than the C++ version. ECDSA-P256, same as today. Add the Ed25519 SAN (§3.4). |
|
||||
| CSPRNG | libsodium | **`RandomNumberGenerator`** | built-in | — |
|
||||
| Opus codec | libopus 1.6 | **P/Invoke libopus 1.6** | BSD | **Keep native.** [Concentus](https://github.com/lostromb/concentus) is a pure-C# Opus port but it tracks **Opus 1.1** — it has no DRED, no 1.6 features. `docs/voice.md` §4 and `test_dred_toggle.cpp` depend on DRED. Concentus is a viable *fallback* for a future platform where native linking is impossible, not the primary. |
|
||||
| Noise suppression | RNNoise (vendored) | **P/Invoke RNNoise** | BSD-3 + CC0 | **Keep native.** No managed port exists. It's ~5 exported functions; the binding is trivial. Already vendored at `third_party/rnnoise/`. |
|
||||
| Audio capture/playback | miniaudio | **P/Invoke miniaudio via a shim** | MIT-0/PD | **Keep native**, see §5.2. Managed alternatives exist ([SoundFlow](https://www.nuget.org/packages/SoundFlow), [MiniaudioSharp](https://www.nuget.org/packages/MiniaudioSharp), NAudio/CSCore for Windows-only) but auto-generated bindings marshal the callback into managed code, which is exactly what you must avoid (§5.1). Write the shim yourself. |
|
||||
| Energy VAD | hand-rolled | port verbatim | — | ~60 lines. Trivial. |
|
||||
| Protobuf | protobuf-lite (C++) | **`Google.Protobuf`** + `Grpc.Tools` | BSD | ⚠️ Reference `Grpc.Tools` for the `protoc` MSBuild integration even though there is no gRPC here — it is the standard way to codegen `.proto` in a `.csproj`. `<Protobuf Include="../../proto/voicecat.proto" GrpcServices="None" />`. The `.proto` needs **zero changes**. |
|
||||
| SQLite | sqlite3 | **`Microsoft.Data.Sqlite.Core` + SQLitePCLRaw** | MIT / public domain | Implemented with provider 10.0.5, bundle 3.0.2 and pinned SQLite 3.50.4.2. Same schema/file; native database import is tested. NativeAOT publishing remains to be validated. |
|
||||
| Logging | spdlog | **`Microsoft.Extensions.Logging`** (+ Serilog console sink) | MIT/Apache | Use `LoggerMessage` source generators on any path near the hot loop. Never log from an audio path. |
|
||||
| Server config | `server.toml` | **`Tomlyn`** (MIT) or switch to JSON + `System.Text.Json` | MIT | Tomlyn keeps `server.toml` compatible; recommended, since operator-facing config shouldn't churn. |
|
||||
| CLI arg parsing | hand-rolled | **`System.CommandLine`** | MIT | For `VoiceCat.Cli` and the server. |
|
||||
| Build | CMake + vcpkg | **`dotnet build`** + a small native build script | — | vcpkg disappears entirely except for the 3 native libs, which you can vendor as sources and build with a 20-line CMakeLists or even `cl`/`gcc` directly. |
|
||||
|
||||
### 4.1 License check
|
||||
|
||||
Every replacement is MIT / BSD / Apache-2.0 / built-in. **The hard no-GPL/LGPL rule in
|
||||
`docs/tech-stack.md` §5 holds.** BouncyCastle is MIT. Konscious is MIT. Blake2Fast is MIT.
|
||||
Tomlyn is MIT. The .NET runtime itself is MIT.
|
||||
|
||||
---
|
||||
|
||||
## 5. Real-time audio — the hard part
|
||||
|
||||
This is where a naive port fails. `docs/architecture.md` §3 states the rule: *"Audio
|
||||
(real-time) threads must not allocate, lock, log, or do syscalls."* A managed runtime adds a
|
||||
second rule: **they must not be subject to GC pauses, and they must not be managed threads at
|
||||
all if avoidable.**
|
||||
|
||||
### 5.1 Why you cannot just P/Invoke miniaudio and use `[UnmanagedCallersOnly]`
|
||||
|
||||
miniaudio calls your `ma_device_data_proc` on an OS-owned real-time audio thread (WASAPI's
|
||||
MMCSS thread, a CoreAudio IOThread, an ALSA thread). If that callback is a managed method:
|
||||
|
||||
1. **The thread must attach to the runtime.** First entry does thread registration; every
|
||||
entry does a managed↔native transition.
|
||||
2. **The thread becomes GC-suspendable.** A gen-0 collection anywhere in the process can
|
||||
suspend it mid-callback. WASAPI in exclusive/low-latency mode will glitch on a 2 ms stall;
|
||||
a gen-2 blocking collection is fatal to the audio.
|
||||
3. **Delegate lifetime.** Even with `[UnmanagedCallersOnly]` (which correctly avoids the
|
||||
marshalling stub and the `GCHandle` dance) the *reachability* problem is solved but the
|
||||
suspension problem is not.
|
||||
|
||||
The existing Windows client sidesteps all of this by keeping the entire audio pipeline in
|
||||
C++. A pure-.NET port has to solve it deliberately.
|
||||
|
||||
### 5.2 Recommended design: a native shim that owns the RT thread
|
||||
|
||||
Write **one small C file** (`native/miniaudio/voicecat_audio_shim.c`, est. 300–400 lines)
|
||||
that compiles miniaudio and exposes a *pull/push ring-buffer API* instead of a callback API:
|
||||
|
||||
```c
|
||||
// The audio callback lives entirely in C. It only ever touches lock-free ring buffers.
|
||||
// Managed code polls. No managed frame is ever on an RT stack.
|
||||
|
||||
vcsh_device* vcsh_capture_open (const char* device_id, uint32_t channels, uint32_t rate);
|
||||
size_t vcsh_capture_read (vcsh_device*, int16_t* dst, size_t frames); // non-blocking
|
||||
vcsh_device* vcsh_playback_open(const char* device_id, uint32_t channels, uint32_t rate);
|
||||
size_t vcsh_playback_write(vcsh_device*, const int16_t* src, size_t frames);
|
||||
int vcsh_playback_wait(vcsh_device*, int timeout_ms); // eventfd/Event, wakes the mixer
|
||||
void vcsh_enumerate(int capture, vcsh_device_info** out, size_t* n);
|
||||
```
|
||||
|
||||
Managed side then runs a **normal, dedicated, non-RT `Thread`** at
|
||||
`ThreadPriority.Highest`, woken by `vcsh_playback_wait`, that does: drain jitter buffers →
|
||||
Opus decode → NR → gain/mute → mix → `vcsh_playback_write`. The ring absorbs GC pauses; size
|
||||
it for ~120 ms (6 × 20 ms frames), which is well within the latency budget the jitter buffer
|
||||
already targets (`target_depth_ms_` starts at 40).
|
||||
|
||||
This is *the same architecture the code already has* — `AudioEngine` already runs a
|
||||
`mixer_timer_thread_` for the iOS external-playback path and already has per-stream ring
|
||||
buffers (`RemoteStream::ring`). You are generalising the iOS path to every platform. That is
|
||||
a pleasing simplification, and it means the iOS design needs no special case at all.
|
||||
|
||||
**Bonus:** it makes `vc_set_external_playback` / `vc_set_mixed_output_sink` disappear as
|
||||
special modes. Everything is external playback; the shim is just one more sink.
|
||||
|
||||
### 5.3 GC and allocation discipline in the managed audio path
|
||||
|
||||
Even off the RT thread, the decode/mix loop runs 50×/second per stream and must not churn:
|
||||
|
||||
- `<ServerGarbageCollector>false</ServerGarbageCollector>` and
|
||||
`<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>` on client apps.
|
||||
Set `GCSettings.LatencyMode = GCLatencyMode.SustainedLowLatency` while a call is active.
|
||||
- **Preallocate everything at stream init**, exactly as `RemoteStream::init_ring` does today.
|
||||
Use `int16[]` fields, not `new` per frame.
|
||||
- Use `Span<short>` / `ReadOnlySpan<short>` throughout the DSP; `ArrayPool<short>.Shared` for
|
||||
the rare variable-size case. Never LINQ, never `IEnumerable`, never `List<T>` growth on
|
||||
this path.
|
||||
- Marshal to native with `fixed` + raw pointers, or declare P/Invokes as
|
||||
`[LibraryImport]` taking `ref short` / `ReadOnlySpan<short>` — the source generator emits
|
||||
pinning without a marshalling stub. **Do not** use `Marshal.Copy` per frame.
|
||||
- **Add an allocation regression test.** `GC.GetAllocatedBytesForCurrentThread()` before/after
|
||||
1000 simulated mix cycles must be ~0. This is a cheap, high-value test the C++ code can't
|
||||
even express.
|
||||
- The mixer's soft limiter, the RMS level meter, and the RNNoise call are all
|
||||
fixed-work-per-frame — they port directly.
|
||||
|
||||
### 5.4 Jitter buffer
|
||||
|
||||
`JitterBuffer` uses `std::map<uint32_t, Frame>` keyed by timestamp with wraparound handling,
|
||||
plus `try_lock` everywhere so the RT thread never blocks. In C#:
|
||||
|
||||
- `SortedDictionary<uint,Frame>` allocates per insert. Prefer a **fixed-capacity circular
|
||||
array of pre-allocated frame slots** indexed by `(ts / frameSamples) % capacity` — the
|
||||
buffer is bounded at 500 ms anyway (`kLateDropSamples`), so a ring is the natural shape and
|
||||
removes all allocation. This is a genuine improvement over the current C++.
|
||||
- Replace `try_lock` with `Monitor.TryEnter` or, better, make the ring single-producer
|
||||
(net thread) / single-consumer (mixer thread) with `Volatile`/`Interlocked` indices and drop
|
||||
the lock entirely.
|
||||
- Keep the EWMA jitter estimation, the leading-edge reseed, and the frame-skip catch-up logic
|
||||
**verbatim** — that logic is subtle, hard-won, and covered by `test_jitter_depth.cpp`.
|
||||
|
||||
### 5.5 Opus P/Invoke
|
||||
|
||||
```csharp
|
||||
[LibraryImport("opus")]
|
||||
internal static partial int opus_encode(IntPtr st, ReadOnlySpan<short> pcm, int frameSize,
|
||||
Span<byte> data, int maxDataBytes);
|
||||
```
|
||||
|
||||
- `opus_encoder_ctl` is **varargs** — P/Invoke cannot do C varargs portably. The implemented
|
||||
desktop binding uses fixed C entry points in `dotnet/native/media.c`; C calls the
|
||||
varargs function with the correct ABI. This also handles Apple arm64's different
|
||||
varargs calling convention. Only whitelisted single-int controls are accepted.
|
||||
- DRED (`opus_dred_alloc`, `opus_dred_parse`, `opus_decoder_dred_decode`) binds the same way.
|
||||
Guard with a runtime feature check as `opus_codec.cpp` does today.
|
||||
- **iOS requires static linking**: use `[LibraryImport("__Internal")]` and link
|
||||
`libopus.a` via `<NativeReference>` in the `.csproj`. Multi-target the DllImport name with a
|
||||
`const string` behind `#if IOS`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Core client port — `VoiceCat.Core`
|
||||
|
||||
`vc_client` (107 KB) is the biggest single unit. It becomes `VoiceCatClient : IAsyncDisposable`.
|
||||
|
||||
### 6.1 The C ABI goes away — and the API gets much better
|
||||
|
||||
The 60-odd `vc_*` functions were shaped by C ABI constraints. In C#:
|
||||
|
||||
| C ABI pattern | C# replacement |
|
||||
|---------------|----------------|
|
||||
| `vc_result` enum returns | Exceptions for programmer errors; `VoiceCatResult` for protocol outcomes |
|
||||
| `vc_callbacks.on_event` + `vc_event` union-ish struct | **`IAsyncEnumerable<VoiceCatEvent>`** or typed `event` handlers per event type. Kill the `u32a` generic-payload field — use a discriminated hierarchy (`record UserJoined(uint UserId, uint ChannelId, string Nick)`). |
|
||||
| `VC_EVENT_JOIN_RESULT` correlating with `vc_join_channel` | **`Task<JoinResult> JoinChannelAsync(uint id, string? pw, CancellationToken ct)`** — request/response correlation via `TaskCompletionSource` keyed on `Envelope.request_id`. This removes an entire class of "which reply was mine" bugs and shrinks every client's code. |
|
||||
| `vc_list_channels` + `vc_free_channel_list` | `IReadOnlyList<Channel> Channels { get; }` — no ownership contract at all |
|
||||
| `vc_get_server_identity_display(buf, cap, out len)` two-call idiom | `string ServerIdentityDisplay { get; }` |
|
||||
| `vc_set_pcm_sink` / `vc_set_mixed_output_sink` / `vc_stream_feed_pcm` | Keep as-is conceptually — they're the bot/extension API. `Action<PcmFrame>` or a `ChannelWriter<T>`. Document the no-blocking rule just as loudly. |
|
||||
| `vc_test_inject_capture` | `internal` test hook, not public API |
|
||||
|
||||
**Do this deliberately, not accidentally.** Write `docs/api-dotnet.md` as the successor to
|
||||
`voicecat.h`, and keep the same rule from `CLAUDE.md`: changing it is a versioned act.
|
||||
|
||||
### 6.2 Threading model in C#
|
||||
|
||||
| C++ | C# |
|
||||
|-----|-----|
|
||||
| Asio `io_context` on `io_thread_` | A single `async` read loop over `PipeReader` per connection; no explicit thread |
|
||||
| `WorkerPool` (blocking work) | Default `ThreadPool` — `Task.Run` for Argon2id, SQLite, DNS |
|
||||
| Event queue drained by UI | `System.Threading.Channels.Channel<VoiceCatEvent>` (already what the Windows client does) |
|
||||
| Mixer timer thread | Dedicated `Thread` (§5.2) — **not** a `Task`, the thread pool is not for this |
|
||||
| Capture/encode thread | Dedicated `Thread`, fed by the shim's capture ring |
|
||||
|
||||
`WorkerPool` (861 bytes) simply deletes.
|
||||
|
||||
### 6.3 State model
|
||||
|
||||
`client.cpp` holds channel/user/stream maps guarded by mutexes and exposes them through
|
||||
pull-based `vc_list_*`. In C#, hold them as immutable snapshots swapped with
|
||||
`Volatile.Write` — readers get a consistent view with no locking, and the UI can bind to it
|
||||
directly. `VC_EVENT_CHANNEL_LIST` becomes "a new snapshot is available", which is what it
|
||||
already means.
|
||||
|
||||
---
|
||||
|
||||
## 7. Server port — `VoiceCat.Server`
|
||||
|
||||
The most mechanical part of the project. Straight `async`/`await` network code.
|
||||
|
||||
| Component | Port notes |
|
||||
|-----------|-----------|
|
||||
| `server.cpp` — accept loop | `Socket.AcceptAsync` loop + `Task` per connection. Trivial. |
|
||||
| `conn_session.cpp` (34 K) — per-conn protocol | The bulk. A big `switch` on `Envelope.BodyCase`. Mechanical; write it against the ported xUnit tests. |
|
||||
| `session_registry.cpp` | `ConcurrentDictionary<ulong, Session>` + a channel-membership index. Simpler than the C++. |
|
||||
| `media_relay.cpp` — the SFU | **The server hot path.** Authenticate/decrypt using the sender's directional key, then reseal for each recipient with its directional key and next counter. Preserve SSRC, timestamp, flags, and encoded Opus bytes; replace sequence and ciphertext/tag. Use pooled buffers and `Socket.ReceiveFromAsync(Memory<byte>, SocketAddress)`. Never decode audio. Benchmark fan-out and allocations. |
|
||||
| `db.cpp` (26 K) — SQLite | `Microsoft.Data.Sqlite`, same schema, same file. Keep raw SQL — do not introduce EF Core; the schema is 4 tables and EF's startup cost hurts the "single binary, instant start" goal. |
|
||||
| `identity.cpp` | `CertificateRequest` + BouncyCastle Ed25519. Reads the same on-disk files. |
|
||||
| Keepalive reaper | `PeriodicTimer` — cleaner than the `asio::steady_timer`. |
|
||||
| `server.toml` | Tomlyn, unchanged format. |
|
||||
|
||||
### 7.1 Deployment — keeping the "single static binary" promise
|
||||
|
||||
`docs/deployment.md` promises a single statically-linked executable with no runtime to
|
||||
install. **NativeAOT preserves this:**
|
||||
|
||||
```xml
|
||||
<PublishAot>true</PublishAot>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<StripSymbols>true</StripSymbols>
|
||||
```
|
||||
|
||||
- ✅ SQLitePCLRaw, Google.Protobuf, and BouncyCastle are all AOT-compatible.
|
||||
- ✅ Startup drops to ~5 ms; binary lands around 15–25 MB (vs. the current C++ static binary
|
||||
— comparable order of magnitude).
|
||||
- ⚠️ **No reflection-based JSON/config.** Use `System.Text.Json` source generators
|
||||
(`JsonSerializerContext`) if you use JSON anywhere. Tomlyn's model binding uses reflection —
|
||||
either use its low-level `DocumentSyntax` API or add trim descriptors.
|
||||
- ⚠️ Cross-compilation is per-RID; you need a build machine per target (`linux-x64`,
|
||||
`linux-arm64`, `win-x64`, `osx-arm64`). Same as today with vcpkg, so no regression.
|
||||
- The Docker image gets *simpler*: `FROM scratch`-ish with a NativeAOT binary, or
|
||||
`mcr.microsoft.com/dotnet/runtime-deps:10.0-noble`.
|
||||
|
||||
**Alternative if AOT fights you:** self-contained single-file publish
|
||||
(`PublishSingleFile` + `SelfContained`) — bigger (~70 MB) and slower to start, but no AOT
|
||||
constraints. Keep as a fallback per-RID, not the default.
|
||||
|
||||
---
|
||||
|
||||
## 8. Clients
|
||||
|
||||
### 8.1 Windows — the easy one
|
||||
|
||||
The `net10.0-windows` WinForms app is already C# and already structured around
|
||||
`Channel<VoiceCatEvent>` + a 30 ms UI-thread pump. The port is:
|
||||
|
||||
1. Delete `VoiceCat.Interop` (the P/Invoke layer) and `VoiceCat.Interop.Tests`.
|
||||
2. `<ProjectReference Include="VoiceCat.Core" />`.
|
||||
3. Update ~40 call sites from `VcResult r = Native.vc_join_channel(...)` to
|
||||
`await client.JoinChannelAsync(...)`. Mostly a find/replace plus `async void` →
|
||||
`async Task` hygiene on event handlers.
|
||||
4. `Audio/ProcessLoopbackCapture.cs`, `ProcessAudioMixer.cs`, `AudioSessionEnumerator.cs`
|
||||
(WASAPI process loopback, `AUDIOCLIENT_ACTIVATION_PARAMS`) — **unchanged**. They already
|
||||
feed `vc_stream_feed_pcm`; they'll feed `client.FeedPcm(...)`.
|
||||
5. `Native/RawInput.cs` (PTT hotkeys), `Models/PasswordProtector.cs` (DPAPI),
|
||||
`Notifications/*` (SAPI announcer, sound pool) — **unchanged**.
|
||||
|
||||
**WinForms accessibility (the reason it was chosen over WinUI 3) is unaffected.** Keep it.
|
||||
|
||||
**Estimated effort: 1–2 weeks.** This client is nearly free.
|
||||
|
||||
### 8.2 macOS — AppKit in C#
|
||||
|
||||
`net10.0-macos` gives full AppKit bindings via [dotnet/macios](https://github.com/dotnet/macios).
|
||||
The Swift AppKit code maps almost line-for-line:
|
||||
|
||||
| Swift | C# |
|
||||
|-------|-----|
|
||||
| `NSWindowController`, `NSOutlineView`, `NSTableViewDataSource` | Same types, same selectors, PascalCase |
|
||||
| `@objc func handleClick(_ sender: Any)` | `[Export("handleClick:")] void HandleClick(NSObject sender)` |
|
||||
| `accessibilityLabel`, `NSAccessibility.post(.announcement)` | `AccessibilityLabel`, `NSAccessibility.PostNotification(...)` — **full VoiceOver parity, the reason AppKit was chosen holds** |
|
||||
| `ScreenAudioCapture.swift` — ScreenCaptureKit | ✅ **ScreenCaptureKit is bound** in `net10.0-macos` (`SCStream`, `SCContentFilter`, `SCStreamConfiguration` incl. `CapturesAudio` / `ExcludesCurrentProcessAudio`). Per-app include/exclude filters and the VoiceOver-exclusion set port directly. |
|
||||
| `InputDeviceCapture.swift` — CoreAudio | AVFoundation/CoreAudio bound; or just use the miniaudio shim on macOS |
|
||||
|
||||
`MainWindowController.swift` is 70 KB — this is the single largest UI rewrite. Budget for it.
|
||||
|
||||
⚠️ **Distribution:** a `net10.0-macos` app bundle needs codesigning + notarization, and
|
||||
NativeAOT for macOS app bundles is supported but adds a step. Nothing blocking; just not
|
||||
free.
|
||||
|
||||
### 8.3 iOS — the SwiftUI gap (decision implemented 2026-09-19)
|
||||
|
||||
This is the only client with no mechanical path, because **SwiftUI has no C# equivalent.**
|
||||
Three options:
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| **A. UIKit in C#** (`net10.0-ios`, hand-written) | Full API access, best accessibility control, matches the macOS/AppKit approach, no extra framework | The ~150 KB of SwiftUI views (`SettingsView`, `ChannelTreeView`, `ChatView`, …) must be re-authored as UIKit — a real rewrite, not a translation |
|
||||
| **B. .NET MAUI** | Fastest to write; XAML declarative style is closest in spirit to SwiftUI; one codebase could later cover macOS too (Mac Catalyst) | ⚠️ Accessibility is weaker than native UIKit — and the project *explicitly* chose native toolkits for screen-reader quality (`roadmap.md` §2). Extra abstraction layer over the audio-sensitive app lifecycle. |
|
||||
| **C. Avalonia** | One UI codebase for Windows + macOS + iOS | ⚠️ Same accessibility objection as MAUI, *and* it would mean abandoning WinForms/AppKit — contradicts two settled decisions |
|
||||
|
||||
**Recommendation: Option A (UIKit).** It's more work but it is the only choice consistent
|
||||
with the accessibility commitments already made twice in the docs. Budget it as the largest
|
||||
single client task.
|
||||
|
||||
The decision is now implemented in `clients/apple/dotnet/VoiceCat.iOS`: a `net10.0-ios`
|
||||
UIKit host uses the managed core, statically links the Opus/RNNoise shim through
|
||||
`__Internal`, and embeds the retained Swift ReplayKit extension. The Swift app remains the
|
||||
migration oracle until the UIKit client completes its device, live-call and VoiceOver gates.
|
||||
|
||||
What ports cleanly regardless:
|
||||
- `IOSAudioRouter.swift` (31 KB) — `AVAudioSession` is fully bound. `SetPreferredDataSource`,
|
||||
`SetPreferredPolarPattern`, `AllowBluetoothA2DP`, `MeasurementMode` all exist in C#. The
|
||||
re-entrancy guards and route-change filtering (`.categoryChange` / `.routeConfigurationChange`
|
||||
/ `.override`) port verbatim. **Keep the invariants in `voice.md` §8 exactly.**
|
||||
- `IOSVoiceProcessingEngine.swift` (24 KB) — `AVAudioEngine`, `AVAudioSourceNode`,
|
||||
`SetVoiceProcessingEnabled(true)`, `VoiceProcessingAgcEnabled` are all bound.
|
||||
- Under §5.2's design, iOS stops being a special case: the core is *always* externally
|
||||
driven, and `AVAudioEngine` is simply the iOS "shim" implementation.
|
||||
|
||||
⚠️ **iOS + NativeAOT:** .NET for iOS ships Mono AOT by default; NativeAOT for iOS is
|
||||
[still experimental](https://learn.microsoft.com/en-us/dotnet/maui/deployment/nativeaot).
|
||||
Mono AOT is fine for the host app (it's what every Xamarin/MAUI app ships). Do not depend on
|
||||
NativeAOT on iOS.
|
||||
|
||||
⚠️ **App Store:** you already ship `me.iamtalon.voicecat`. A runtime change is invisible to
|
||||
review, but re-validate background-audio behaviour (`UIBackgroundModes: audio`) under Mono —
|
||||
managed finalizers and the GC must not stall the audio render callback while backgrounded.
|
||||
§5.2's native-ring design is what protects you here.
|
||||
|
||||
### 8.4 iOS screen sharing — **keep this in Swift**
|
||||
|
||||
You anticipated this correctly. The ReplayKit Broadcast Upload Extension should **not** be
|
||||
ported.
|
||||
|
||||
**Why:**
|
||||
1. **The 50 MB jetsam cap.** A managed runtime (Mono AOT + metadata + GC heap) inside a
|
||||
separate appex process eats a meaningful fraction of that before your code runs. The
|
||||
current Swift `SampleHandler` is 4.4 KB and does one `AVAudioConverter` call per buffer.
|
||||
2. **.NET for iOS does not officially support broadcast upload extensions.** Microsoft's own
|
||||
guidance is that this is a [known gap with no documentation](https://learn.microsoft.com/en-sg/answers/questions/2006706/issues-with-bundling-ios-broadcast-extension-in-ne);
|
||||
the supported extension types are enumerated and this isn't reliably among them.
|
||||
3. **There is nothing to gain.** The extension deliberately does *not* link the core
|
||||
(`voice.md` §9) — it converts PCM and writes to a shared ring. It is already a
|
||||
language-agnostic boundary.
|
||||
|
||||
**The boundary is already clean.** `BroadcastAudioRing.swift` is an mmap'd file in an App
|
||||
Group with an SPSC ring layout. C# reads it with `MemoryMappedFile.CreateFromFile` +
|
||||
`MemoryMappedViewAccessor`, and `CFNotificationCenter` (Darwin notifications) is bound in
|
||||
`net10.0-ios`. **Action: freeze `BroadcastAudioRing`'s binary layout as a documented struct**
|
||||
(magic, version, capacity, head, tail, activeFlag, sample format) in
|
||||
`docs/broadcast-ring-format.md`, so the Swift writer and the C# reader are contractually
|
||||
pinned. The C# `BroadcastAudioPump` is then ~150 lines.
|
||||
|
||||
This leaves the repo with exactly **one Swift file plus one shared Swift ring** — an
|
||||
acceptable, well-justified exception to "pure C#", and dramatically less than the current
|
||||
three Swift codebases.
|
||||
|
||||
### 8.5 What about a Linux client?
|
||||
|
||||
Not in scope today, but worth noting: once the core is `net10.0` with a miniaudio shim
|
||||
(which has ALSA/PulseAudio backends), a Linux client becomes a UI-only problem for the first
|
||||
time. Avalonia would be the natural choice *there specifically*, without disturbing the
|
||||
Windows/macOS/iOS decisions. Mention it in `roadmap.md`; don't build it now.
|
||||
|
||||
---
|
||||
|
||||
## 9. Tests
|
||||
|
||||
The 29 ctest cases **are** the specification. Port every one to xUnit in
|
||||
`tests/VoiceCat.Tests/`. Grouping:
|
||||
|
||||
| Group | Tests | Notes |
|
||||
|-------|-------|-------|
|
||||
| Wire format | `test_envelope`, `test_voice_frame`, `test_frame_codec`, `test_frame_ms_reframe` | Port first. These are pure functions — fastest possible feedback on the protobuf + framing layers. |
|
||||
| Crypto | `test_media_aead`, `test_tls_loopback`, `test_tofu_flow` | The AEAD test must produce **byte-identical** ciphertext to the C++ for a fixed key+nonce+AAD. Add that as a golden-vector test — it's your proof the port is wire-compatible. |
|
||||
| Codec/DSP | `test_opus_codec`, `test_dred_toggle`, `test_noise_suppression`, `test_recv_noise_reduction`, `test_plc_cap` | Depend on the native P/Invokes; run them as soon as those exist. |
|
||||
| Audio engine | `test_jitter_depth`, `test_channel_samplerate`, `test_external_pcm`, `test_external_playback`, `test_vad_ptt_devices` | The subtle ones. `test_jitter_depth` guards the bounded-depth invariant — do not weaken it. |
|
||||
| Integration | `test_m1_integration`, `test_m2_voice`, `test_m3_multistream`, `test_tcp_loopback`, `test_disconnect_left`, `test_reaper_timeout` | Real client + real server in-process. |
|
||||
| Moderation/admin | `test_m5_permissions`, `test_m5_kick_ban_move_mute`, `test_m5_channel_crud`, `test_m5_admin_accounts` | Server-side; port with `VoiceCat.Server`. |
|
||||
| ABI surface | `test_voice_client_abi`, `test_channel_user_list_abi` | These test the C ABI specifically — **rewrite as API-shape tests** against the new C# surface, don't port literally. |
|
||||
|
||||
**New tests the port should add:**
|
||||
- Allocation regression on the mix loop (§5.3).
|
||||
- AEAD golden vectors vs. C++ output.
|
||||
- A **cross-implementation test**: C# client ↔ C++ server, and C++ `vccli` ↔ C# server, run
|
||||
in CI for as long as both trees exist (§11).
|
||||
|
||||
`ctest --preset dev` → `dotnet test`. The house rule in `CLAUDE.md` ("every commit builds and
|
||||
passes") carries over unchanged.
|
||||
|
||||
---
|
||||
|
||||
## 10. Documentation changes
|
||||
|
||||
Per the `CLAUDE.md` rule that docs and code stay in sync:
|
||||
|
||||
| Doc | Change |
|
||||
|-----|--------|
|
||||
| `docs/architecture.md` | Rewrite §1 (shared-core model — it's now a shared *assembly*), §4 (C ABI → C# API), §3 (threading — the shim design). Keep §5 (server) and §2 (layers) nearly as-is. |
|
||||
| `docs/tech-stack.md` | Replace the whole dependency table. Re-run the license audit (§5) — the no-GPL rule still passes. |
|
||||
| `docs/security.md` | ⚠️ **§2 needs rewriting** to describe BouncyCastle's exporter rather than mbedTLS's, and §1.1 should be updated when the Ed25519↔cert binding lands (§3.4). §3–§6 unchanged. |
|
||||
| `docs/voice.md` | §5 (jitter), §8 (pipeline), §10 (NR) get implementation-detail updates. The *protocol* sections (§2 frame format, §3 config, §4 loss resilience) are **unchanged** — that's the point. |
|
||||
| `docs/protocol.md` | Unchanged unless you take Option B (§3.3), which adds two `AuthResult` fields and bumps to v3. |
|
||||
| `docs/building.md` | Full rewrite: `dotnet build` + the native build script replace the CMake preset matrix. **Delete the "run ctest in PowerShell not Git Bash" warning** — that MinGW pathology disappears with the toolchain. |
|
||||
| `docs/deployment.md` | §1.B/C update for NativeAOT publish; Docker base image changes. The zero-config promises hold. |
|
||||
| `docs/roadmap.md` | Add the port as its own milestone; note the Linux-client possibility (§8.5). |
|
||||
| `CLAUDE.md` | New build commands, new subsystem map, new house rules (no allocation in the audio path becomes explicit). |
|
||||
| **new** `docs/api-dotnet.md` | The successor to `voicecat.h` — the versioned client API contract. |
|
||||
| **new** `docs/broadcast-ring-format.md` | The frozen Swift↔C# App Group ring layout (§8.4). |
|
||||
|
||||
---
|
||||
|
||||
## 11. Migration strategy — the actual step-by-step
|
||||
|
||||
The guiding principle: **the C++ tree stays working and becomes the conformance oracle.** Do
|
||||
not delete anything until the C# equivalent passes the same test against it. This is only
|
||||
possible because Option A (§3.2) preserves wire compatibility — which is the main reason to
|
||||
choose it.
|
||||
|
||||
The rewrite lives under `dotnet/`; initial implementation branch: `dotnet/foundations`,
|
||||
created from `cs-port`. Keep the existing schema at `core/proto/voicecat.proto` during migration.
|
||||
Native packaging is deferred until the codec/audio phase rather than blocking the wire slice.
|
||||
Each phase ends with a green build,
|
||||
green tests, and an updated `PROGRESS.md` entry.
|
||||
|
||||
---
|
||||
|
||||
### Phase 0 — Foundations (est. 1 week)
|
||||
|
||||
1. Create the solution skeleton from §2. `Directory.Build.props` with
|
||||
`net10.0`, `<Nullable>enable</Nullable>`, `<TreatWarningsAsErrors>true</TreatWarningsAsErrors>`,
|
||||
`<AnalysisLevel>latest-all</AnalysisLevel>`, `<InvariantGlobalization>true</InvariantGlobalization>`.
|
||||
2. `native/build-native.{ps1,sh}`: build libopus 1.6, RNNoise, and the (empty for now)
|
||||
miniaudio shim into `runtimes/{rid}/native/`. Vendor the sources — drop vcpkg.
|
||||
3. `VoiceCat.Protocol`: add `Google.Protobuf` + `Grpc.Tools`, point at the **existing**
|
||||
`core/proto/voicecat.proto`, verify generated C# types compile.
|
||||
4. CI: GitHub Actions matrix building both trees (C++ and C#) side by side.
|
||||
|
||||
**Exit criterion:** `dotnet build` produces empty-but-real assemblies; generated protobuf
|
||||
types are present; native libs land in the right RID folders.
|
||||
|
||||
---
|
||||
|
||||
### Phase 1 — Wire format, provably compatible (est. 1 week)
|
||||
|
||||
1. Port `envelope.cpp` (frame `[u32 len][payload]`) using `System.IO.Pipelines`.
|
||||
2. Port `voice_frame.h` — the 20-byte big-endian header. Use
|
||||
`BinaryPrimitives.WriteUInt64BigEndian` etc.
|
||||
3. Port `SodiumMediaCrypto` → `MediaCrypto` on `System.Security.Cryptography.ChaCha20Poly1305`,
|
||||
including the nonce scheme and the sliding replay window. **Preserve the RFC 3711 §3.3
|
||||
ordering.**
|
||||
4. Port `test_envelope`, `test_voice_frame`, `test_frame_codec`, `test_media_aead`.
|
||||
5. **Generate golden vectors from the C++ build** (a throwaway C++ main dumping sealed frames
|
||||
for fixed inputs) and assert the C# produces identical bytes.
|
||||
|
||||
**Exit criterion:** golden-vector tests green. From here on, byte-compatibility is measured,
|
||||
not assumed.
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — TLS + the exporter (est. 1.5 weeks — highest-risk phase, do it early)
|
||||
|
||||
1. Spike first, in isolation: a BouncyCastle `TlsClientProtocol` ↔ `TlsServerProtocol`
|
||||
loopback over a `Socket` pair, both calling
|
||||
`ExportKeyingMaterial("voicecat media v1", ...)` and asserting the two sides agree.
|
||||
2. **Then the real proof:** a C# BouncyCastle client handshaking against the **existing C++
|
||||
mbedTLS server**, both exporting with the same label, and asserting the derived media keys
|
||||
are identical. *This is the single most important checkpoint in the whole project.* If it
|
||||
fails, stop and reconsider Option B before writing anything else.
|
||||
3. Port `ServerCert`/`ServerIdentity` (`CertificateRequest` + BC Ed25519), the TOFU pin store
|
||||
(`tofu_store.cpp` — a trivial text file), and cert-fingerprint pinning.
|
||||
4. Port `test_tls_loopback`, `test_tofu_flow`, `test_tcp_loopback`.
|
||||
|
||||
**Exit criterion:** C# client completes a TLS 1.3 handshake with the C++ server, derives
|
||||
matching media keys, and pins the leaf fingerprint.
|
||||
|
||||
**Checkpoint (2026-09-15):** implemented nonblocking managed TLS, handshake-time
|
||||
exporters, explicit certificate acceptance, persisted TOFU, and native-compatible
|
||||
credentials. The C++ TLS oracle authenticates a media challenge in both directions
|
||||
over an actual socket, proving exporter compatibility. Tests also cover managed
|
||||
fragmented loopback, first-connect acceptance, changed-pin rejection, TLS 1.2 rejection,
|
||||
close_notify/abrupt EOF, restart persistence, and import of C++ credential files.
|
||||
Socket orchestration remains a transport-owner responsibility; the complete managed
|
||||
server and client are later phases. See `dotnet/README.md` for the required native
|
||||
interoperability test command.
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — Codec + DSP (est. 1 week)
|
||||
|
||||
1. `VoiceCat.Codec`: libopus `[LibraryImport]`, `OpusEncoder`/`OpusDecoder`, the varargs-CTL
|
||||
workaround (§5.5), DRED.
|
||||
2. `VoiceCat.Dsp`: RNNoise binding, `EnergyVadProcessor`.
|
||||
3. Port `test_opus_codec`, `test_dred_toggle`, `test_noise_suppression`.
|
||||
|
||||
**Exit criterion:** encode→decode round-trip at every supported frame size; DRED recovery
|
||||
test green; RNNoise output matches the C++ within tolerance.
|
||||
|
||||
**Checkpoint (2026-09-15):** implemented `VoiceCat.Codec`, `VoiceCat.Dsp`, safe native
|
||||
handles, fixed-signature C bindings, and independent desktop native staging. The native
|
||||
build pins the upstream Opus 1.5.2 release/checksum (matching the actual vcpkg baseline,
|
||||
despite older code comments referring to 1.6), enables DRED, and shares the existing
|
||||
vendored RNNoise sources/model. Tests cover 40 rate/channel/frame-size round trips,
|
||||
PLC, 40 dropped-frame DRED recovery formats, C++ denoising within one PCM unit, VAD
|
||||
hang time, and zero managed allocations over 1,000 combined processing cycles.
|
||||
At 8/12 kHz, DRED tests encode at 16 kHz and decode at the requested rate: this pinned
|
||||
encoder's activity analysis cannot emit DRED at 8/12 kHz. These encoding configurations
|
||||
are explicitly rejected. Its redundancy floor is 30 ms because two chunks are required
|
||||
to emit DRED; actual redundancy remains adaptive. Desktop CI builds/stages the library
|
||||
before testing. iOS static native packaging and device audio remain later phases.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — Server (est. 3–4 weeks)
|
||||
|
||||
Do the server before the client: it lets you point the **existing, trusted C++ `vccli`** at
|
||||
it, which is a far better test client than a half-built C# one.
|
||||
|
||||
**Implemented checkpoint (2026-09-15):** bounded async TLS socket orchestration,
|
||||
guest/password authentication, existing SQLite account/channel import, snapshots,
|
||||
unprotected channel joins, channel/private/server text, ping and disconnect events.
|
||||
The existing C++ CLI authenticates and sends text through this server. Argon2id uses
|
||||
the existing BouncyCastle dependency with a strict libsodium PHC parser, not a new
|
||||
Konscious dependency. Native database tests prove password compatibility in both
|
||||
directions without resets. SQLite uses `Microsoft.Data.Sqlite.Core` 10.0.5,
|
||||
SQLitePCLRaw bundle 3.0.2 and explicitly pinned SourceGear SQLite 3.50.4.2.
|
||||
See `docs/api-dotnet.md` for limits. This first checkpoint did not include UDP voice,
|
||||
streams, protected joins, moderation, admin handlers or production configuration.
|
||||
The subsequent voice checkpoint is described below.
|
||||
|
||||
**Voice checkpoint:** the managed server now advertises UDP, issues session-bound
|
||||
tokens, implements voice subscription and multi-stream signaling, and relays encrypted
|
||||
Opus with recipient-specific counters. The first UDP endpoint is fixed for the session;
|
||||
reconnect for endpoint changes. Immutable routing snapshots separate control handlers
|
||||
from the UDP crypto owner. Real-socket tests cover replay/forgery/SSRC rejection,
|
||||
channel/subscription isolation, stream stop and disconnect. A native client oracle
|
||||
exercises bidirectional microphone and screen audio in mono and stereo. The fan-out
|
||||
core has a 50-subscriber allocation regression test; transport scheduling and the
|
||||
BouncyCastle crypto fallback are excluded from its zero-allocation guarantee.
|
||||
Two real C++ `vccli` processes also pass join/text/bidirectional voice tests using
|
||||
finite `--test-tone-ms` external capture/playback. The transport load test delivers
|
||||
all 2,500 recipient packets from a sender paced at 50 pps to 50 subscribers.
|
||||
**Reaper checkpoint:** configurable 45-second idle expiry / 15-second sweep replaces
|
||||
the TCP-only idle timeout. Control envelopes, authenticated voice and bound-endpoint
|
||||
keepalives refresh shared monotonic activity; invalid media does not. Reaping removes
|
||||
presence and media routing, and can be disabled. Tests inject a clock to cover silent
|
||||
clients, UDP-only activity, forged media, single departure events and disabled expiry.
|
||||
**Channel/administration checkpoint:** protected joins and channel CRUD now persist using
|
||||
the native BLAKE2b password format (native verification in both directions). Permissions
|
||||
gate moderation and account create/reset/delete/list. Mute/deafen/move update encrypted
|
||||
routing; kick/ban retire media and emit one reason-bearing departure. Guest bans use
|
||||
addresses, account bans use usernames, and wire milliseconds convert to database seconds.
|
||||
Temporary-channel permission only creates temporary channels and only administrators
|
||||
grant permissions; these deliberately tighten native policy. Tree validation and Lobby
|
||||
protection prevent invalid mutations. Existing streams stop on channel edits/moves/deletion.
|
||||
The C++ CLI creates protected channels and administers accounts against the managed server;
|
||||
its channel argument lifetimes and default audio config were corrected. See api-dotnet.md
|
||||
for limits, persistence and policy differences. Production configuration/publishing and
|
||||
the remaining server readiness checks still precede Phase 4 completion.
|
||||
|
||||
1. `VoiceCat.Server`: accept loop, `ConnSession` protocol handling, session registry.
|
||||
2. `Db` on `Microsoft.Data.Sqlite` — same schema. **Resolve the Argon2id hash-compat
|
||||
question here** (§4).
|
||||
3. `MediaRelay` — the allocation-free SFU fan-out.
|
||||
4. Keepalive reaper, moderation/admin handlers.
|
||||
5. Port `test_m1_integration`, `test_m5_*`, `test_disconnect_left`, `test_reaper_timeout`.
|
||||
|
||||
**Exit criterion:** ▶ **C++ `vccli` connects to the C# server, authenticates, joins a
|
||||
channel, sends text, and exchanges voice with a second C++ `vccli`.** That is the M1+M2 exit
|
||||
criterion from `roadmap.md`, re-proven against the new server.
|
||||
|
||||
---
|
||||
|
||||
### Phase 5 — Audio engine (est. 4–5 weeks — the hardest phase)
|
||||
|
||||
1. Write and validate `voicecat_audio_shim.c` standalone (a C test that loops mic→speaker
|
||||
through the rings, no .NET involved).
|
||||
2. `VoiceCat.Audio`: device enumeration, the allocation-free jitter buffer (§5.4), per-ssrc
|
||||
decode with the **DRED → FEC → PLC** ladder, per-stream NR/gain/mute, the mixer, level
|
||||
meters, talk-state edge detection.
|
||||
3. The dedicated mixer thread + capture thread.
|
||||
4. External feed/tap/mixed-sink — now the *normal* path, not special modes.
|
||||
5. Port `test_jitter_depth`, `test_external_pcm`, `test_external_playback`,
|
||||
`test_channel_samplerate`, `test_plc_cap`, `test_recv_noise_reduction`,
|
||||
`test_frame_ms_reframe`.
|
||||
6. **Add the allocation-regression test.**
|
||||
|
||||
**Exit criterion:** `test_jitter_depth`'s bounded-depth invariant holds; zero allocations per
|
||||
mix cycle; real multi-human calls on Windows and macOS have no reproducible audio-quality
|
||||
defects once each client is feature-complete.
|
||||
|
||||
**Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every
|
||||
protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream
|
||||
controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero
|
||||
managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation
|
||||
uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device
|
||||
shim; the codec/DSP shim remains the only native component. A real device smoke passed;
|
||||
final listen validation is performed with real multi-human calls after feature completion,
|
||||
not a synthetic ten-minute sine-wave gate.
|
||||
|
||||
---
|
||||
|
||||
### Phase 6 — Client core (est. 3–4 weeks)
|
||||
|
||||
1. `VoiceCatClient`: connect/TOFU/auth state machine, request/response correlation via
|
||||
`TaskCompletionSource`, channel/user/stream snapshots, stream lifecycle, reframing, the
|
||||
send path, the event stream.
|
||||
2. `VoiceCat.Cli` — the `vccli` replacement, plus `voicecat-admin`.
|
||||
3. Port `test_m2_voice`, `test_m3_multistream`, `test_vad_ptt_devices`; rewrite the two ABI
|
||||
tests as API-shape tests.
|
||||
|
||||
**Exit criterion:** ▶ **Two C# `vccli` instances hold a multi-channel voice + text
|
||||
conversation through the C# server**, and a C# `vccli` interoperates with a C++ `vccli` on
|
||||
the same server. This is the full M0–M3 criterion re-proven end to end.
|
||||
|
||||
**Complete (2026-09-16):** `VoiceCat.Core` implements TOFU-gated TLS, concurrent correlated
|
||||
requests, snapshots/events, reconnects, encrypted UDP binding and send/receive stream
|
||||
lifecycle. Two managed clients exchange text and decoded PCM through the managed server.
|
||||
`VoiceCat.Cli` supports interactive channel text and deterministic headless text/voice runs.
|
||||
Process tests prove two managed CLIs converse with decoded voice in both configured channels,
|
||||
and a managed CLI exchanges text and bidirectional voice with the existing C++ CLI.
|
||||
|
||||
---
|
||||
|
||||
### Phase 7 — Windows client (est. 1–2 weeks)
|
||||
|
||||
Per §8.1. Ship this first of the three GUIs — it's the cheapest and it validates the C# API
|
||||
shape against a real, complete UI before you commit to two rewrites.
|
||||
|
||||
**Exit criterion:** feature parity with the current WinForms build, NVDA smoke-tested.
|
||||
|
||||
**Checkpoint (2026-09-16):** the shipped WinForms project references `VoiceCat.Managed`, not
|
||||
the P/Invoke core. The compatibility facade preserves UI-thread event pumping and stable
|
||||
capture IDs while delegating all protocol and audio state to the idiomatic managed projects.
|
||||
Channel moves automatically renegotiate active streams. A published build passed real WASAPI
|
||||
capture/playback and form-startup smoke tests and contains `voicecat_media.dll` but no
|
||||
`voicecat.dll`. Automated text, bidirectional PCM voice, multi-frame audio and stream-move
|
||||
tests pass. NVDA and the manual endurance/listen pass remain before the Phase 7 exit criterion.
|
||||
|
||||
---
|
||||
|
||||
### Phase 8 — macOS client (est. 4–5 weeks)
|
||||
|
||||
Per §8.2. AppKit port, ScreenCaptureKit per-app audio selection, VoiceOver parity.
|
||||
|
||||
**Exit criterion:** feature parity with `VoiceCatMac`, VoiceOver smoke-tested, notarized
|
||||
build produced.
|
||||
|
||||
**Checkpoint (2026-09-18):** `clients/apple/dotnet/VoiceCat.Mac` is a separate .NET 10
|
||||
AppKit application that consumes `VoiceCat.Core` directly. It implements guest and account
|
||||
connection, validated saved server profiles whose JSON never contains passwords,
|
||||
interactive TOFU approval, protected-channel password prompts, channel browsing, roster,
|
||||
channel/private text,
|
||||
voice subscription, native accessibility labels and default-device microphone/playback.
|
||||
Account passwords can be stored in macOS Keychain after successful authentication and are
|
||||
removed when the user disables remembering or deletes the profile.
|
||||
Core Audio capture is resampled
|
||||
and converted through `AVAudioConverter` before entering the managed audio engine. Stereo
|
||||
playback uses `AVAudioSourceNode` and the shared bounded PCM ring; its render callback does
|
||||
not allocate, lock or block. Voice stream lifetime follows subscription, disconnect and
|
||||
channel changes. The app now targets Microsoft's Xcode 27 preview bindings from workload
|
||||
set 10.0.401. On Apple Silicon it selects a native Homebrew `protoc`, deduplicates the
|
||||
transitive media dylib before AppKit bundling, uses the macOS 27 error-returning Core Audio
|
||||
overloads, and keeps hardened runtime Release-only so local ad-hoc Debug builds can load
|
||||
their runtime libraries. It enumerates and selects Core Audio input/output devices, requests
|
||||
microphone permission explicitly, and renders through Core Audio's native planar Float32
|
||||
layout. The arm64 app builds, signs, launches and displays on macOS 27. Live testing proved
|
||||
physical-microphone transmission and clean peer playback; Lobby's expected DTX comfort noise
|
||||
was distinguished from corruption by repeating the tone in the DTX-disabled Music Room. A
|
||||
macOS CI job builds the native codec/DSP shim and Apple solution. The existing Swift app
|
||||
remains the release client until the manual release gates below are complete.
|
||||
|
||||
**Functional-parity checkpoint (2026-09-19):** the managed app now covers persistent audio
|
||||
and notification settings, VAD/configurable focus-scoped PTT/always-on input, stereo capture,
|
||||
RNNoise, input/output/auxiliary gain, selectable auxiliary capture, self mute/deafen, speaking
|
||||
state, event sounds and speech, modeless private conversations, full channel configuration,
|
||||
per-user receive tuning, moderation, permissions and account administration. ScreenCaptureKit
|
||||
publishes desktop audio with whole-desktop, application-only and application-exclusion scopes
|
||||
and can exclude VoiceOver/speech processes. Legacy Swift profiles, TOFU pins and Keychain
|
||||
passwords migrate without discarding the original profile JSON. A publishing script validates
|
||||
an isolated ad-hoc distribution copy without mutating incremental build output and supports
|
||||
Developer ID signing, notarization, stapling and Gatekeeper assessment. Debug and Release
|
||||
builds, 190 managed tests, 29 native CTests and a
|
||||
strictly verified/running ad-hoc Release bundle pass. Do not remove the Swift macOS app yet:
|
||||
cutover still requires a manual VoiceOver navigation/announcement pass, ScreenCaptureKit plus
|
||||
real multi-human call validation, and an actual credentialed notarization run.
|
||||
|
||||
---
|
||||
|
||||
### Phase 9 — iOS client (est. 5–7 weeks)
|
||||
|
||||
Per §8.3/§8.4. UIKit rewrite, `AVAudioSession` router port, `AVAudioEngine` VPIO path, and
|
||||
the C# `BroadcastAudioPump` reading the **unchanged Swift** extension's ring.
|
||||
|
||||
**Exit criterion:** feature parity with `VoiceCatiOS`; screen-audio sharing works with the
|
||||
Swift extension untouched; A2DP/stereo/VPIO preset matrix re-verified (this is where the
|
||||
known stereo-A2DP class of bug lives — re-test it explicitly).
|
||||
|
||||
---
|
||||
|
||||
### Phase 10 — Cutover (est. 1–2 weeks)
|
||||
|
||||
1. Run both trees in CI for one full release cycle.
|
||||
2. Delete `core/`, `server/`, `tools/`, `clients/apple/` (except `VoiceCatBroadcast/` and
|
||||
`Shared/BroadcastAudioRing.swift`), `vcpkg/`, `CMakePresets.json`, root `CMakeLists.txt`.
|
||||
3. Update every doc per §10.
|
||||
4. Tag the last C++ commit so the oracle stays reachable.
|
||||
|
||||
---
|
||||
|
||||
### 11.5 Total estimate
|
||||
|
||||
**~7–9 months of focused single-developer work**, front-loaded with risk (Phase 2) and
|
||||
back-loaded with volume (Phases 8–9). The server + core (Phases 0–6) is roughly 4 months and
|
||||
is the part that removes the most complexity; the three GUIs are roughly half the calendar
|
||||
time and almost none of the difficulty.
|
||||
|
||||
---
|
||||
|
||||
## 12. Risk register
|
||||
|
||||
| # | Risk | Severity | Mitigation |
|
||||
|---|------|----------|------------|
|
||||
| 1 | **BouncyCastle's exporter doesn't interoperate with mbedTLS's** | 🔴 Critical | Prove it in Phase 2 step 2, before any other work depends on it. Both implement RFC 8446 §7.5, so it should — but *verify*, don't assume. Fallback: Option B (§3.3). |
|
||||
| 2 | **GC pauses cause audio glitches** | 🔴 High | Native shim owns the RT thread (§5.2); ~120 ms ring; allocation-regression test; `SustainedLowLatency`. This is the design's whole answer. |
|
||||
| 3 | **iOS audio regressions under Mono AOT** | 🟠 Medium-High | The iOS audio path is already the most delicate part of the product (see the A2DP/VPIO invariants in `voice.md` §8). Re-test the full preset × route matrix. Do not port the invariants "roughly". |
|
||||
| 4 | **Argon2id hashes don't verify → existing accounts locked out** | 🟠 Medium | Decide in Phase 4. Preferred: parse libsodium's PHC string and pass m/t/p to Konscious; verify against real hashes from an existing `voicecat.db` *before* writing the rest of `Db`. |
|
||||
| 5 | **SFU relay throughput regression** | 🟠 Medium | Benchmark early (Phase 4): N=50 subscribers × 50 pps. Allocation-free `SocketAddress` receive + pooled buffers. .NET's socket layer is good; this should be fine, but measure. |
|
||||
| 6 | **`ChaCha20Poly1305.IsSupported == false`** on some target | 🟡 Low | Startup check + BouncyCastle fallback. One-line risk. |
|
||||
| 7 | **`opus_encoder_ctl` varargs breaks on a future ABI** | 🟡 Low | Only single-`int` CTLs are used; document it, add a test that exercises every CTL used. |
|
||||
| 8 | **NativeAOT trimming breaks protobuf/SQLite reflection** | 🟡 Low | All three are AOT-tested upstream. Add an AOT-published smoke test to CI from Phase 4. |
|
||||
| 9 | **macOS/iOS bindings lag a new Xcode** | 🟡 Low | dotnet/macios tracks Xcode closely (bindings exist through Xcode 26). Pin the workload version. |
|
||||
| 10 | **Scope creep — "while we're rewriting, let's also…"** | 🟠 Medium | The port is a *translation*. The API-shape improvements in §6.1 are the only sanctioned redesign. Everything else goes in `roadmap.md`. |
|
||||
|
||||
---
|
||||
|
||||
## 13. What you gain
|
||||
|
||||
Worth being explicit, since this is 7+ months:
|
||||
|
||||
- **One language, one toolchain, one debugger.** No more CMake presets, vcpkg triplets,
|
||||
MinGW-vs-PowerShell execution pathologies, XCFramework fat-static-lib packaging, or a C ABI
|
||||
that has to be hand-mirrored into both Swift and C#.
|
||||
- **Three UI codebases instead of three UI codebases *plus* a core plus two binding layers.**
|
||||
The `VoiceCat.Interop` P/Invoke layer, the `VoiceCatCore` Swift wrapper, the module map, and
|
||||
the whole `voicecat.h` ABI surface all cease to exist.
|
||||
- **Better API.** `await client.JoinChannelAsync()` instead of "call this, then wait for
|
||||
`VC_EVENT_JOIN_RESULT`, and hope it's yours."
|
||||
- **Memory safety** across the entire protocol-parsing surface — the part most exposed to
|
||||
hostile input.
|
||||
- **8 native dependencies → 3**, each small and vendored.
|
||||
- **Tests that can assert things C++ couldn't**, notably zero-allocation invariants.
|
||||
|
||||
And what you keep: the protocol, the wire format, the security model, the audio design, the
|
||||
accessibility-first UI toolkit choices, and every single one of the 29 behavioural tests.
|
||||
+1
-1
@@ -28,7 +28,7 @@ boundaries on top of the TLS byte stream.
|
||||
|
||||
## 2. Why Protocol Buffers for control
|
||||
|
||||
- Schema-driven codegen for **C++, C#, and Swift** (all first-class) → no hand-rolled
|
||||
- Schema-driven codegen for the supported **C#** implementation → no hand-rolled
|
||||
parsers, no drift between client and server.
|
||||
- **Forward/backward compatible by construction**: unknown fields are preserved/ignored,
|
||||
new fields and new `oneof` arms are additive. This is exactly the "extensible protocol"
|
||||
|
||||
+25
-187
@@ -1,195 +1,33 @@
|
||||
# Roadmap & Open Questions
|
||||
# Roadmap
|
||||
|
||||
## 1. Milestones
|
||||
The managed rewrite is functionally complete. Work is now release validation, retirement of
|
||||
the old implementation, and focused product development.
|
||||
|
||||
### .NET port — initial slice
|
||||
## Release gates
|
||||
|
||||
**Complete 2026-09-15:** managed Release build and 34/34 xUnit tests, C++ golden
|
||||
fixtures for both crypto backends, fresh native build and 29/29 CTest tests. Native
|
||||
packaging and TLS/server/client migration remain later checkpoints.
|
||||
- Windows: published-client smoke test, NVDA navigation, and sustained real calls.
|
||||
- macOS: VoiceOver matrix, multi-person call, Developer ID signing, notarization, Gatekeeper.
|
||||
- iOS: physical-device VoiceOver, background/lock, interruption and route recovery, Bluetooth,
|
||||
ReplayKit on iOS 18–26, and ScreenCaptureKit audio on iOS 27+.
|
||||
- Server: build and run the container, validate graceful shutdown, and complete a 30-minute or
|
||||
longer concurrent text/voice soak.
|
||||
|
||||
- `dotnet/` contains .NET 10 protocol and crypto assemblies plus xUnit conformance tests.
|
||||
- Preserve the existing protobuf and 20-byte media wire formats; keep C++ as the oracle.
|
||||
- **Exit:** managed framing, headers, and ciphertext match fixtures generated by C++;
|
||||
managed tests and the existing C++ behavior suite pass.
|
||||
- **Subsequent checkpoints:** TLS/exporter and credential interoperability, codec/DSP
|
||||
desktop packaging, and managed control/UDP server slices are implemented. Two C++
|
||||
`vccli` processes authenticate, join, chat and exchange mono/stereo voice through
|
||||
the managed server. The 50-subscriber fan-out core has an allocation regression test.
|
||||
- **Media-aware reaping:** monotonic control/valid-UDP activity, configurable 45-second
|
||||
idle timeout / 15-second sweep, and graceful shutdown are implemented and tested.
|
||||
- **Next:** finish managed server administration, protected joins and production configuration,
|
||||
then audio/client core, Windows cutover, C# AppKit and UIKit.
|
||||
Keep the Swift ReplayKit extension and its shared ring; defer C++ removal until parity.
|
||||
- See `docs/porting-to-dotnet.md` and `dotnet/README.md`.
|
||||
## Cleanup
|
||||
|
||||
Each milestone is shippable/testable on its own. The headless C++ test client (`vccli`)
|
||||
exists from M1 so the protocol can be exercised long before any GUI.
|
||||
- Remove the retired C++ core, server, CLI, tests, CMake presets, and vcpkg tree.
|
||||
- Remove the old Swift macOS/iOS applications after retained assets are detached.
|
||||
- Keep `native/media`, `native/rnnoise`, and `native/apple/broadcast`.
|
||||
- Remove the old Windows P/Invoke project after shared model types are moved into the managed
|
||||
compatibility facade.
|
||||
- Continue reducing historical documentation to current contracts and operating instructions.
|
||||
|
||||
### M0 — Scaffolding
|
||||
- Repo layout (see architecture.md §6), CMake + vcpkg manifest, CI matrix.
|
||||
- `core/proto/` skeleton; `protoc` codegen wired for C++ (C#/Swift later).
|
||||
- Empty `libvoicecat` with the C ABI header and stub implementations.
|
||||
- **Exit:** core + server + `vccli` compile and link on Linux/macOS/Windows.
|
||||
## Deferred product features
|
||||
|
||||
### M1 — Control plane (TCP/TLS, no audio yet)
|
||||
- TLS 1.3 transport; framing; Envelope; ClientHello/ServerHello negotiation.
|
||||
- Auth: **guest + admin-provisioned local account** (Argon2id, SQLite); server identity
|
||||
(TOFU/Ed25519); `voicecat-admin` account add/reset/del/list.
|
||||
- Channel tree: snapshot + deltas; join/leave; create/edit/delete (perm-checked).
|
||||
- **Text chat (ephemeral):** channel + private messages, acks; live relay, no history store.
|
||||
- `vccli` can connect, auth, browse channels, and chat.
|
||||
- **Exit:** two `vccli` instances chat through a real server over TLS.
|
||||
- file transfer
|
||||
- end-to-end media encryption beyond the server-terminated transport encryption
|
||||
- persistent server-side text history
|
||||
- key-based user identities
|
||||
- multi-node/federated servers
|
||||
- PushKit/CallKit incoming-call behavior
|
||||
|
||||
### M2 — Voice, single stream
|
||||
- UDP transport with exported-key + ChaCha20-Poly1305 AEAD (mandatory, no plaintext path);
|
||||
UDP token binding; anti-replay.
|
||||
- miniaudio capture/playback; libopus encode/decode; one `MIC` stream per user.
|
||||
- **Send-side webrtc APM** (AEC + NS + AGC + VAD) and a **VAD/PTT input gate** (both modes,
|
||||
client-configurable) — AEC is in from the start, not deferred.
|
||||
- Per-ssrc adaptive jitter buffer; mixer; FEC/PLC/DTX.
|
||||
- Per-channel `AudioConfig` enforced (incl. server `max_bitrate_bps` ceiling); SFU relay.
|
||||
- **Exit:** talk between two `vccli`/early-GUI clients in a channel; loss resilience visible.
|
||||
|
||||
### M3 — Multi-stream & per-channel tuning
|
||||
- Multiple concurrent streams per user (`MIC`, `SCREEN_AUDIO`, `AUX_DEVICE`).
|
||||
- Per-stream receiver gain/mute; **listener-side per-user noise reduction** (APM NS on the
|
||||
receive path, per ssrc, local-only); talk indicators.
|
||||
- Full per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX).
|
||||
- **Exit:** a user shares mic + desktop audio; listeners control each independently.
|
||||
|
||||
### M4 — Native clients
|
||||
|
||||
**Windows (C#/WinForms, .NET 10) ✓ complete 2026-06-17:**
|
||||
- Connect, saved-server list (JSON, DPAPI-encrypted passwords), TOFU identity dialog.
|
||||
- Channel tree (`TreeView`), user list, join (incl. password-protected channels).
|
||||
- Voice: mic start/stop, mute/deafen, VAD/PTT/**always-on** mode, **VAD sensitivity slider**
|
||||
(live threshold update via `vc_set_vad_threshold`), device picker, level meter.
|
||||
- Per-user gain/mute/NR tuning (`PerUserTuningDialog`).
|
||||
- Channel + private text chat. Activity log (screen-reader primary path).
|
||||
- Explicit `AccessibleName`/`AccessibleDescription` on every control; `&` mnemonics;
|
||||
`AutomationNotification` curated live announcements.
|
||||
- Focus-scoped PTT (documented limitation — no system-wide hook in v1).
|
||||
- Admin/moderation UI **out of scope** — needs server-side dispatch first (M5).
|
||||
|
||||
**macOS (Swift/AppKit) — in progress:**
|
||||
- Same feature set as Windows over the same C ABI (now stable and complete).
|
||||
- **Shared Swift core (`VoiceCatCore` package) ✓ complete 2026-06-18** — wraps all 38 C ABI
|
||||
functions; 6/6 XCTest smoke tests pass against a real server (connect → TOFU → auth →
|
||||
channels → moderation → admin CRUD → per-stream recv controls). See
|
||||
`clients/apple/README.md`.
|
||||
- **UI: AppKit** (not SwiftUI) — chosen for the most mature VoiceOver accessibility story
|
||||
(per-control `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`,
|
||||
`NSAccessibility.post(.announcement)` for live announcements). Same rationale as the
|
||||
Windows client's WinForms-over-WinUI-3 decision (see §2 below). macOS 14 (Sonoma)
|
||||
deployment target.
|
||||
- AVAudioSession not needed on macOS (CoreAudio via the core directly).
|
||||
|
||||
**iOS (.NET/UIKit) — replacement in progress 2026-09-19:**
|
||||
- Native C# UIKit app consuming `VoiceCat.Core`; UIKit was chosen over MAUI for direct audio
|
||||
lifecycle control and the strongest VoiceOver surface.
|
||||
- ~~AVAudioSession, mic permission, foreground voice.~~ ✓ Done — `IOSAudioRouter` drives
|
||||
all iOS audio routing (input ports, orientation/polar patterns, HFP/A2DP, Standard/Raw
|
||||
mic mode, stereo capture), `vc_audio_suspend`/`vc_audio_resume` for interruptions.
|
||||
- The small Swift ReplayKit broadcast extension remains and writes its versioned App Group
|
||||
PCM ring; the C# host drains it into `SCREEN_AUDIO`. See `broadcast-ring-format.md`.
|
||||
|
||||
**Exit:** non-technical user installs a client, saves a server, and joins.
|
||||
|
||||
### M5 — Moderation, polish, and beyond
|
||||
- Permissions/roles, kick/ban/server-mute, channel passwords UI. Windows WinForms UI complete
|
||||
(channel CRUD with full Opus config, user moderation, server account management); macOS/iOS
|
||||
Swift UI pending.
|
||||
- DRED toggle, audio-quality polish. (AEC and VAD/PTT already shipped in M2.)
|
||||
- **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) ✓ shipped
|
||||
(2026-06-20) — promotes `vc_test_inject_capture` to a public, stereo-capable API and adds
|
||||
a symmetric PCM sink. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, and custom
|
||||
clients. See architecture.md §4 "External PCM feed/tap" and protocol.md §8 for the
|
||||
no-protocol-change rationale. One new C++ ctest binary (`test_external_pcm`) covering 3
|
||||
sub-tests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`, `test_pcm_sink`) — ctest
|
||||
23/23; Swift wrapper + 4 XCTest smoke tests; C# wrapper + 4 xUnit smoke tests.
|
||||
- **Then (post-v1, protocol already reserves space):** file transfer, E2EE option,
|
||||
CallKit/PushKit background voice, key-based identity, server-side text history,
|
||||
multi-node server.
|
||||
|
||||
## 2. Resolved decisions
|
||||
|
||||
Settled and reflected throughout the docs:
|
||||
|
||||
- **Media crypto:** exported-key + ChaCha20-Poly1305 AEAD from day one, **mandatory** — no
|
||||
DTLS, no plaintext mode. (security.md §2)
|
||||
- **TLS library / licensing:** **mbedTLS** (Apache-2.0) + **libsodium** (ISC). **No GPL/LGPL**
|
||||
anywhere; code is redistributable closed-source. wolfSSL is rejected. (tech-stack.md §5)
|
||||
- **iOS screen/system audio:** supported via a ReplayKit Broadcast Upload Extension
|
||||
(`.audioApp`); audio-only stays within the extension memory cap. (voice.md §9)
|
||||
- **Self-host UX:** zero-config, encrypted-by-default; Docker / single binary / source build.
|
||||
(deployment.md)
|
||||
- **DSP engine:** **webrtc-audio-processing (APM)** — AEC in from the start, plus NS/AGC/VAD.
|
||||
(voice.md §8)
|
||||
- **Input activation:** **VAD *and* PTT**, both modes client-configurable. (voice.md §11)
|
||||
- **Noise reduction is two-sided:** sender can denoise its mic, *and* each listener can apply
|
||||
NS to a **specific** other user, locally, with no protocol traffic. (voice.md §10)
|
||||
- **Accounts:** **admin-provisioned** (no self-serve registration) via `voicecat-admin` or the
|
||||
in-app admin interface. (security.md §4, protocol.md §3, deployment.md §3a)
|
||||
- **Text:** **ephemeral** — live relay, no server-side history in v1. (protocol.md §5)
|
||||
- **Connect UX:** pure direct-connect with a **client-side saved-server list** (no central
|
||||
directory). (deployment.md §3)
|
||||
- **Bitrate ceiling:** server-config `opus.limits.max_bitrate_bps`. (deployment.md §2)
|
||||
- **Name:** "VoiceCat" stays as the internal placeholder.
|
||||
- **DSP engine, superseded (2026-06-16):** the "webrtc-audio-processing (APM)" decision above
|
||||
(AEC + NS/AGC/VAD in one module) could not be carried out — it has no working Windows/MSVC
|
||||
build upstream (GCC-only Meson build, MinGW support unfinished, hard `abseil-cpp` dependency,
|
||||
Linux-tested only). v1 ships a lightweight, dependency-free energy/RMS VAD instead, behind
|
||||
the same `ApmProcessor` interface; there is **no AEC/NS/AGC implementation at all** yet. Real
|
||||
`webrtc-audio-processing` stays a tracked future swap (e.g. if/when a Linux build target
|
||||
exists). (voice.md §8, §11)
|
||||
- **Windows client UI framework (2026-06-17):** **WinForms** (.NET 10), not WinUI 3 or
|
||||
Avalonia. Reason: Win32 HWND controls have the most mature, predictable screen-reader
|
||||
(NVDA/JAWS/Narrator) support of any current .NET UI stack. WinUI 3's accessibility UIA
|
||||
tree has known rough edges on .NET 10; Avalonia's accessibility story is thinner still.
|
||||
This overrides the earlier WinUI/Avalonia mention in `docs/tech-stack.md §2` and
|
||||
`docs/architecture.md §4`. (clients/windows/)
|
||||
- **TOFU pins TLS leaf cert, not Ed25519 (2026-06-17):** the original design said to pin the
|
||||
server's declared Ed25519 identity fingerprint from `ServerHello`. This is circular — the
|
||||
Ed25519 key and the TLS cert are generated independently with no cryptographic binding, so
|
||||
accepting/rejecting based on a value sent *inside* the channel being trust-decided is
|
||||
meaningless. **Decision:** pin the TLS leaf certificate's own SHA-256 fingerprint, which is
|
||||
verifiable directly from the TLS handshake before any application data is trusted. The Ed25519
|
||||
value is still shown in the identity dialog for human-readable display only (informational).
|
||||
See `docs/security.md §1.1`. (core/src/crypto/tofu_store.*, vc_confirm_server_identity)
|
||||
- **PTT is focus-scoped in v1 (2026-06-17):** PTT hotkey capture uses `Form.KeyDown`/`KeyUp`
|
||||
(works only while the VoiceCat window has focus), not a system-wide `WH_KEYBOARD_LL` hook.
|
||||
Reason: a low-level keyboard hook requires escalated permissions, risks AV flagging, and is
|
||||
disproportionate complexity for a v1 client. Documented in the UI as a known limitation.
|
||||
Can be revisited for v2 if users request it. (clients/windows/VoiceCat.App/Forms/MainForm.cs)
|
||||
- **macOS client UI framework (2026-06-18):** **AppKit**, not SwiftUI. Reason: AppKit has the
|
||||
most mature, granular **VoiceOver** accessibility story on macOS — per-control
|
||||
`accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`, `NSAccessibility.post(.announcement)`
|
||||
for curated live announcements, and decades of real-world screen-reader usage. This is the
|
||||
same rationale that drove the Windows client to WinForms over WinUI 3 (screen-reader
|
||||
support is the deciding factor). SwiftUI's VoiceOver support has improved but still has
|
||||
gaps in complex AppKit-bridged control surfaces (outline views, data-table column
|
||||
headers, live-region announcements). iOS stays SwiftUI — its control surface is narrower
|
||||
and SwiftUI's VoiceOver support is sufficient there. This overrides the earlier
|
||||
"SwiftUI for both macOS + iOS" mention in `docs/tech-stack.md §2` and `docs/architecture.md
|
||||
§4`. macOS 14 (Sonoma) deployment target. (clients/apple/)
|
||||
|
||||
## 3. Open questions
|
||||
|
||||
All initial open questions are resolved (§2). Two **second-order considerations** to keep in
|
||||
mind during implementation — not blockers:
|
||||
|
||||
- **APM in constrained contexts.** webrtc-audio-processing is a heavier build; confirm it
|
||||
static-links cleanly for the single-binary goal, and note the iOS **broadcast extension only
|
||||
does Opus encode + send** (no APM), so it stays under the ~50 MB cap. Listener-side per-user
|
||||
NS runs only in the full host app.
|
||||
- **Receive-side NR cost at scale.** A per-ssrc APM NS instance per flagged user adds CPU on
|
||||
busy channels; instantiate lazily (only for flagged streams) and cap concurrent instances.
|
||||
|
||||
## 4. What's intentionally deferred
|
||||
|
||||
To keep v1 focused (voice + text), these are designed-for but not built: file transfer,
|
||||
end-to-end encryption, key-based identities, multi-node/federated servers, mobile
|
||||
background VoIP push, and any server-side audio mixing/transcoding. The protocol's
|
||||
versioning + feature negotiation + reserved tag ranges (protocol.md §8) ensure each can be
|
||||
added without breaking deployed clients.
|
||||
These are not blockers for the current release.
|
||||
|
||||
+3
-3
@@ -53,7 +53,7 @@ future improvement. Until then, clients display both values but gate on the cert
|
||||
captures directional exporters during handshake completion. Its client requires an
|
||||
explicit leaf-fingerprint acceptance callback; PKI validation remains unimplemented.
|
||||
New managed server certificates include the Ed25519 public key in SAN URI
|
||||
`urn:voicecat:identity:ed25519:<lowercase-public-key-hex>`. Existing C++ credentials
|
||||
`urn:voicecat:identity:ed25519:<lowercase-public-key-hex>`. Existing pre-rewrite credentials
|
||||
are imported unchanged. Verifying that URI against the declared ServerHello identity
|
||||
is still deferred to the managed session layer; leaf-certificate TOFU remains the
|
||||
trust gate. Missing members of a persisted credential set cause startup rejection
|
||||
@@ -82,7 +82,7 @@ mandatory from the first build. This was chosen over DTLS after weighing two fin
|
||||
`0x01` for server→client. Each export yields a 32-byte directional media key.
|
||||
No second handshake, no certificates on the UDP path — the UDP channel inherits the
|
||||
authenticated, MITM-resistant TLS session's trust.
|
||||
2. Each UDP voice frame is sealed with **ChaCha20-Poly1305** (libsodium in C++;
|
||||
2. Each UDP voice frame is sealed with **ChaCha20-Poly1305** (managed platform crypto;
|
||||
platform cryptography with a BouncyCastle fallback in .NET).
|
||||
3. The full 20-byte header is AEAD **associated data**. The server authenticates/decrypts
|
||||
inbound media and reseals for each recipient, replacing the sequence with that
|
||||
@@ -124,7 +124,7 @@ Binding works as:
|
||||
accepts the first endpoint only; further bootstrap packets cannot replace it.
|
||||
Endpoint changes require a new authenticated session. The token remains available
|
||||
for TLS confirmation but cannot establish a second binding. Session removal retires
|
||||
its endpoint, token and directional keys. The C++ oracle currently permits rebinding
|
||||
its endpoint, token and directional keys. Unsupported old releases permitted rebinding
|
||||
with the same token; this differs in policy, not in the packet format.
|
||||
4. Thereafter, frames are accepted only on that bound tuple; ssrcs are checked against the
|
||||
streams the session announced. Source-address spoofing can't hijack a session because the
|
||||
|
||||
+31
-101
@@ -1,113 +1,43 @@
|
||||
# Tech Stack & Dependencies
|
||||
# Technology stack
|
||||
|
||||
## Initial .NET rewrite
|
||||
## Managed runtime
|
||||
|
||||
The parallel rewrite under `dotnet/` targets .NET 10. Its initial dependencies are
|
||||
Google.Protobuf 3.36.1 (BSD-3-Clause), build-only Grpc.Tools 2.83.0 (Apache-2.0), and
|
||||
BouncyCastle.Cryptography 2.6.2 (MIT). Media AEAD prefers the platform implementation;
|
||||
BouncyCastle provides the managed fallback and is the planned TLS/exporter provider.
|
||||
No managed server or audio replacement is shipped yet.
|
||||
VoiceCat targets .NET 10. Shared libraries use plain `net10.0`; WinForms, AppKit, and UIKit are
|
||||
leaf platform projects.
|
||||
|
||||
Project files and NuGet lock files pin versions. `dotnet/check-licenses.ps1` checks
|
||||
all restored direct/transitive packages against a permissive license allowlist in CI;
|
||||
unknown or copyleft licenses fail. See `dotnet/README.md` for build and test commands.
|
||||
The existing implementation's dependency choices follow below.
|
||||
| Area | Technology | Notes |
|
||||
|---|---|---|
|
||||
| Control schema | Protocol Buffers / Google.Protobuf | `proto/voicecat.proto` is authoritative |
|
||||
| TLS 1.3/exporters | BouncyCastle.Cryptography | Managed TLS is required for exporter-derived media keys |
|
||||
| Media AEAD | System.Security.Cryptography ChaCha20Poly1305 | Managed fallback is behavior-tested |
|
||||
| Passwords | Konscious Argon2id | Bounded parameters, PHC strings |
|
||||
| Persistence | Microsoft.Data.Sqlite | Server-local SQLite with explicit migrations |
|
||||
| Tests | xUnit | Unit, socket integration, allocation, CLI, and production-package checks |
|
||||
|
||||
Concrete library choices with versions and rationale. Everything in the **core** is C++
|
||||
(C++20). UIs are Swift and C#. Build is CMake + vcpkg.
|
||||
NuGet packages and their lock files are committed. `dotnet/check-licenses.ps1` enforces the
|
||||
permissive-license policy.
|
||||
|
||||
## 1. Core library (`libvoicecat`, C++20)
|
||||
## Native media
|
||||
|
||||
| Concern | Choice | Version (as of 2026-06) | Why / notes |
|
||||
|---------|--------|-------------------------|-------------|
|
||||
| Sockets, timers, async | **Standalone Asio** | 1.30.x | Header-only, no Boost dependency, cross-platform TCP+UDP+timers, one reactor for client and server. (Boost.Asio is interchangeable if we already pull Boost.) |
|
||||
| TLS 1.3 (control) | **mbedTLS 3.6 LTS** | 3.6.x (LTS ≥ Mar 2027) | **Apache-2.0** (permissive — clean for eventual closed-source distribution). TLS 1.3 client+server, plus `mbedtls_ssl_export_keying_material()` to seed the media AEAD. **Static-links cleanly → single self-host binary.** OpenSSL 3.x (Apache-2.0) is an interchangeable alternative. No DTLS/wolfSSL (GPL) — see [security.md](security.md) §2. |
|
||||
| Crypto primitives + password hashing + media AEAD | **libsodium** | 1.0.20 | **ISC.** Argon2id (`crypto_pwhash`), ChaCha20-Poly1305 (per-frame media encryption), Ed25519 server identity, X25519, CSPRNG. Audited, hard to misuse. |
|
||||
| Audio codec | **libopus** | **1.6** (2025-12) | Per-channel mono/stereo, bitrate, frame size; in-band FEC, DTX, PLC, and optional **DRED** deep redundancy; Opus HD/96 kHz available. The whole reason the design is codec-flexible. |
|
||||
| Audio capture/playback | **miniaudio** | 0.11.x | Single-header, public-domain, backends for **WASAPI / CoreAudio / ALSA / PulseAudio**. One real-time abstraction across all desktop targets; keeps the RT path identical. |
|
||||
| Audio DSP — noise suppression (NS) | **RNNoise** (vendored, `third_party/rnnoise/`) | xiph @ `70f1d25` (2026-06) | **BSD-3-Clause + CC0-1.0** (model). Hybrid DSP/RNN speech denoiser, mono/48 kHz, ~60× real time, no deps. The shipped NS backend behind `ApmProcessor` (`RnnoiseProcessor`), used by both send-side mic NR (`vc_set_input_noise_reduction`) and per-listener receive NR (`vc_set_remote_stream`). Vendored (not vcpkg) because the vcpkg port is `!windows !arm`. See [voice.md](voice.md) §10. |
|
||||
| Audio DSP — AEC/AGC/VAD | **webrtc-audio-processing** (APM) — **planned, not built** | 1.x (standalone APM) | **BSD-3**, but has no working Windows/MSVC build upstream (GCC-only Meson, MinGW support unfinished, hard `abseil-cpp` dep — see roadmap.md §2). v1 ships a lightweight, dependency-free energy/RMS VAD (`EnergyVadProcessor`, `core/src/audio/apm_processor.cpp`); **NS now exists via RNNoise (row above)**, but there is still **no AEC or AGC** (iOS gets AEC/NS/AGC natively from VPIO). Real APM stays a tracked future swap behind the same `ApmProcessor` interface. |
|
||||
| Resampling + jitter ref | **speexdsp** | 1.2.x | BSD. Resampler for non-48 kHz devices; lightweight jitter-buffer reference. (No longer the NS/AGC/VAD source — APM replaces it.) |
|
||||
| Control serialization | **Protocol Buffers** (protobuf-lite) | 5.x (proto3) | Codegen for C++/C#/Swift; additive, forward/backward compatible; `oneof` envelopes. `nanopb` is a fallback if footprint matters. |
|
||||
| Server persistence | **SQLite** | 3.4x | Accounts, channels, bans, config. Zero-admin, single file, ships everywhere. |
|
||||
| Logging | **spdlog** | 1.14.x | Fast, async-capable; off the RT path. |
|
||||
`native/media` builds one narrow library named `voicecat_media`:
|
||||
|
||||
Resampling note: Opus runs internally at 48 kHz; miniaudio can deliver 48 kHz directly, so
|
||||
explicit resampling (speexdsp/libsamplerate) is only needed when a device can't do 48 kHz.
|
||||
- Opus 1.5.2, checksum-pinned and built with DRED/Deep PLC support.
|
||||
- RNNoise, vendored under `native/rnnoise` at the repository-pinned source/model revision.
|
||||
|
||||
## 2. Clients
|
||||
The shim contains codec and denoiser entry points only. Managed code owns networking, crypto,
|
||||
jitter, mixing, state, and lifecycle. CMake is retained solely for this boundary and its Apple
|
||||
static archives.
|
||||
|
||||
### Apple clients — .NET native UI with a Swift ReplayKit exception
|
||||
## Platform clients
|
||||
|
||||
| Concern | Choice | Notes |
|
||||
|---------|--------|-------|
|
||||
| Language | **Swift 5.9+** | Direct **Swift↔C interop** — the C ABI (`voicecat.h`) is imported as a Clang module (`import VoiceCatC`) via a module map in the XCFramework headers; no manual struct/function redeclaration (unlike the C# P/Invoke layer). A Swift wrapper (`VoiceCatCore` package) provides Swift-idiomatic types on top. |
|
||||
| UI — macOS | **AppKit** | Chosen over SwiftUI for the most mature, granular **VoiceOver** accessibility story (per-control `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`, `NSAccessibility.post(.announcement)` for live announcements) — the same rationale that drove the Windows client to WinForms over WinUI 3 for screen-reader (NVDA/JAWS/Narrator) UIA support (resolved decision in `docs/roadmap.md`). macOS 14 (Sonoma) deployment target. |
|
||||
| UI — iOS | **C# / UIKit (`net10.0-ios`)** | Native UIKit keeps direct lifecycle/audio control and predictable VoiceOver semantics. MAUI was rejected because a cross-platform abstraction provides no benefit for this platform-specific client. iOS 18.0 deployment target. |
|
||||
| Shared core | **VoiceCatCore** Swift Package | One Swift library wrapping the C ABI, consumed by both the macOS AppKit app and the iOS SwiftUI app. Mirrors the C# `VoiceCat.Interop` layer. Events delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump). |
|
||||
| Audio session (iOS) | **AVAudioSession + AVAudioEngine from C#** | The UIKit host owns category, permission, interruptions, routes, VPIO capture and planar playback. Converted PCM crosses bounded managed rings; callbacks allocate no managed objects, lock, or block. |
|
||||
| Packaging | .NET Apple workloads + Xcode appex | The managed iOS host statically links merged Opus/RNNoise archives. MSBuild invokes Xcode to build/embed the small Swift ReplayKit extension, which communicates through the versioned App Group PCM ring. |
|
||||
| Future | CallKit / PushKit | For background VoIP + incoming-call UX on iOS. Post-v1. |
|
||||
- Windows: .NET WinForms for mature UI Automation and screen-reader behavior.
|
||||
- macOS: .NET AppKit for direct accessibility and Core Audio/ScreenCaptureKit control.
|
||||
- iOS: .NET UIKit for direct VoiceOver and AVAudioSession lifecycle control.
|
||||
- iOS broadcast upload: Swift/ReplayKit under `native/apple/broadcast`; it writes PCM to the
|
||||
frozen App Group ring and deliberately does not host .NET or the protocol stack.
|
||||
|
||||
### Windows — C# (shipped in M4, 2026-06-17)
|
||||
## Licensing
|
||||
|
||||
| Concern | Choice | Notes |
|
||||
|---------|--------|-------|
|
||||
| Runtime | **.NET 10 LTS** (`net10.0-windows`) | In-service until 2028. |
|
||||
| Interop | **`[LibraryImport]`** (source-gen P/Invoke) over the C ABI | `[UnmanagedCallersOnly]` static methods for `on_event`/`on_level`; `VoiceCatClientHandle : SafeHandle` owns the `vc_client*` lifetime. |
|
||||
| Event delivery | **`System.Threading.Channels.Channel<VoiceCatEvent>`** | Single-writer/reader, unbounded; drained by a 30ms `System.Windows.Forms.Timer` on the UI thread. Simpler than a message-only HWND with no meaningful latency cost. |
|
||||
| UI | **WinForms** | Chosen over WinUI 3 / Avalonia for mature, predictable NVDA/JAWS/Narrator UIA support. Win32 HWND controls have the most complete accessibility story on .NET 10 today. See roadmap.md §2. |
|
||||
| Persistence | **`System.Text.Json`** (`servers.json`), **`ProtectedData`** (DPAPI) | Saved-server list in `%AppData%\VoiceCat\`; passwords DPAPI-encrypted at rest, opt-in, `CurrentUser` scope. |
|
||||
| Audio | Handled by the core (miniaudio/WASAPI) | C# only drives device selection + meters. |
|
||||
|
||||
## 3. Server (`voicecat-server`)
|
||||
|
||||
- Pure C++ linking the core; **no GUI**. Runs on **Linux** (primary), **macOS**, **Windows**.
|
||||
- Config via a `server.toml` (`allow_guests`, ports, channel defaults, Opus policy, TLS cert
|
||||
paths or auto-self-signed + Ed25519 identity, Argon2id cost params, rate limits).
|
||||
- SQLite for state. Single process for v1; interfaces drawn so a multi-node build is
|
||||
*possible* later but explicitly out of scope.
|
||||
- Packaging: static-ish binary per OS; systemd unit + Docker image for Linux.
|
||||
|
||||
## 4. Build & tooling
|
||||
|
||||
| Tool | Use |
|
||||
|------|-----|
|
||||
| **CMake** (3.25+) | One build graph for core + server + test CLI; UI projects consume the built core. |
|
||||
| **vcpkg** (manifest mode) | Pin C/C++ deps (opus, libsodium, mbedtls, protobuf, sqlite3, spdlog, asio, miniaudio — see `vcpkg.json`). `webrtc-audio-processing`/`speexdsp` are **not** in the manifest: no working vcpkg port / no working Windows/MSVC build exists upstream for the former; the latter was never actually wired up (the lightweight VAD needs no resampler). Reproducible across OSes. Triplet auto-resolved from the host platform by [`cmake/voicecat-toolchain.cmake`](../cmake/voicecat-toolchain.cmake) — `x64-mingw-static` on Windows, `x64-linux` on Linux, `arm64-osx` on Apple Silicon. Apple platform scaffolding presets (`apple-dev`/`apple-ios`/`apple-ios-sim`) produce static `libvoicecat.a` slices for XCFramework consumption. vcpkg itself is bundled as a git submodule at `vcpkg/`, pinned to `vcpkg.json`'s `builtin-baseline` commit — `VCPKG_ROOT` overrides it for an external checkout. See [building.md §2](building.md#2-one-time-setup-for-the-real-deps-presets). |
|
||||
| **protoc** | Generate C++/C#/Swift from `core/proto/*.proto` (single source of truth). |
|
||||
| **clang-format / clang-tidy** | Style + static analysis on the core. |
|
||||
| **CTest + a fuzz target** | Unit/integration tests; fuzz the frame parser and protobuf boundary (security-sensitive). |
|
||||
| **GitHub Actions** (or similar) | Matrix CI: Linux/macOS/Windows core+server; Xcode build for Apple; `dotnet` build for Windows. |
|
||||
|
||||
## 5. Licensing — permissive only (hard rule)
|
||||
|
||||
The code will eventually be distributed in **closed-source** form, so **no GPL/LGPL
|
||||
dependencies are permitted.** Every dependency below is BSD / MIT / ISC / Apache-2.0 /
|
||||
public-domain:
|
||||
|
||||
- **mbedTLS** — Apache-2.0 ✅ · **libsodium** — ISC ✅ · **libopus** — BSD ✅ ·
|
||||
**miniaudio** — public domain / MIT-0 ✅ · **protobuf** — BSD ✅ ·
|
||||
**SQLite** — public domain ✅ · **Asio** (standalone) — Boost ✅ · **spdlog** — MIT ✅ ·
|
||||
**RNNoise** — BSD-3-Clause (code) + CC0-1.0 (model) ✅, vendored in `third_party/rnnoise/`
|
||||
(not vcpkg — the port is `!windows !arm`; see [`third_party/README.md`](../third_party/README.md)).
|
||||
**webrtc-audio-processing** would be BSD-3 ✅ if/when it's actually built in (see §1) —
|
||||
not a live dependency today, so not part of the resolved vcpkg graph the license scanner
|
||||
below checks.
|
||||
- **Explicitly rejected:** **wolfSSL** (GPLv2/commercial) and any DTLS stack that would drag
|
||||
in copyleft. The exported-keys + AEAD media design (security.md §2) removes the need for
|
||||
one entirely.
|
||||
- CI runs a license scanner over the resolved vcpkg graph and **fails the build on any
|
||||
GPL/LGPL transitive dependency**, so this rule can't silently regress.
|
||||
|
||||
## 6. Why not the obvious alternatives
|
||||
|
||||
- **WebRTC** — explicitly rejected: ICE/SDP/TURN complexity, huge dependency, opaque. We
|
||||
want plain TCP+UDP we fully control.
|
||||
- **QUIC** — capable (reliable streams + datagrams + TLS 1.3 in one), but heavier and drifts
|
||||
toward the complexity we're avoiding. Revisit only if NAT traversal/multiplexing pain
|
||||
appears.
|
||||
- **gRPC** for control — pulls HTTP/2 and a lot of surface for what is a simple framed
|
||||
message stream over TLS. Plain protobuf-over-framed-TLS is enough.
|
||||
- **A Rust core** — viable and memory-safe, but the user prefers C++ and the Swift/C#
|
||||
binding story is marginally simpler from C++ (Swift can even consume C++ directly).
|
||||
GPL and LGPL dependencies are forbidden. Current production dependencies are permissively
|
||||
licensed; RNNoise code is BSD-3-Clause and its model data is CC0. Preserve license notices in
|
||||
published native artifacts.
|
||||
|
||||
+4
-4
@@ -324,7 +324,7 @@ independent.
|
||||
chose not to denoise his mic, Sam can locally suppress Alex's background noise without
|
||||
affecting how anyone else hears Alex.
|
||||
|
||||
**Backend: RNNoise** (vendored in [`third_party/rnnoise/`](../third_party/rnnoise), BSD-3 + CC0).
|
||||
**Backend: RNNoise** (vendored in [`native/rnnoise/`](../native/rnnoise), BSD-3 + CC0).
|
||||
The original plan was WebRTC's APM, but `webrtc-audio-processing` has no working Windows/MSVC
|
||||
build (see §8). RNNoise is a small, dependency-free C library — a hybrid DSP/RNN speech denoiser
|
||||
that runs ~60× faster than real time. Both NR paths share one `ApmProcessor` implementation
|
||||
@@ -437,14 +437,14 @@ user (exactly like macOS/Windows), and no credentials are ever persisted to disk
|
||||
|
||||
- The user starts a broadcast from Control Center's screen-record button; we surface it via
|
||||
`RPSystemBroadcastPickerView` from inside the app (`VoiceControlsView`) for one-tap start.
|
||||
- The **Broadcast Upload Extension** (`clients/apple/iOS/VoiceCatBroadcast/SampleHandler.swift`)
|
||||
- The **Broadcast Upload Extension** (`native/apple/broadcast/SampleHandler.swift`)
|
||||
receives `RPSampleBufferType.audioApp` (system/app audio), `.audioMic`, and `.video`. We
|
||||
consume **`.audioApp`** only and drop video + mic — video is what blows the **~50 MB**
|
||||
extension memory budget, so an audio-only consumer stays comfortably inside it. The extension
|
||||
does **not** link `libvoicecat`.
|
||||
does **not** link the managed host or native media shim.
|
||||
- The extension converts each chunk to the core's canonical format (48 kHz int16 stereo, via
|
||||
`AVAudioConverter`) and writes it into a lock-free single-producer/single-consumer ring in a
|
||||
shared **App Group** mmap'd file (`clients/apple/iOS/Shared/BroadcastAudioRing.swift`). It
|
||||
shared **App Group** mmap'd file (`native/apple/broadcast/BroadcastAudioRing.swift`). It
|
||||
posts Darwin notifications on start/stop so the host reacts promptly.
|
||||
- The **host app** owns the stream: its `BroadcastAudioPump` announces the `SCREEN_AUDIO`
|
||||
stream over the control channel (`StreamAnnounce`), drains the ring, and calls
|
||||
|
||||
Reference in New Issue
Block a user