Files
voice-cat/docs/porting-to-dotnet.md
T

966 lines
62 KiB
Markdown
Raw Normal View History

2026-08-05 03:19:59 +02:00
# Porting VoiceCat to pure .NET / C#
**Status:** the managed protocol, crypto, server, CLI, audio/client core, Windows client and
macOS functional surface are implemented under `dotnet/`, with C++ interoperability retained
as a migration oracle. The managed macOS client is awaiting its manual VoiceOver, live-call and
credentialed notarization gates before it replaces the Swift release client. The iOS rewrite
remains planned.
See `dotnet/README.md`, `docs/api-dotnet.md`, and `PROGRESS.md` for verification and next steps.
2026-08-05 03:19:59 +02:00
**Target runtime:** .NET 10 LTS (in-service to Nov 2028), with .NET 11 as the follow-on.
**Scope:** replace the C++ core (`libvoicecat`), the C++ server, the C++ `vccli`, and the
Swift macOS/iOS clients with a single C# codebase. The Windows WinForms client is already C#
and is mostly *kept*.
This document is the map for that work: what maps 1:1, what has no .NET equivalent, what has
to stay native, and the order to do it in so the tree is testable at every step.
---
## 0. Executive summary
**The port is feasible.** Roughly 80 % of the C++ core is protocol/state-machine/buffer code
that translates to C# almost mechanically and gets *shorter*. The risk is concentrated in
four places, and only one of them is a genuine design change:
| Risk | Verdict |
|------|---------|
| **TLS keying-material exporter (RFC 5705)** — the media-key derivation the whole UDP path depends on | ⚠️ **`SslStream` cannot do this.** [The API is an unapproved proposal](https://github.com/dotnet/runtime/issues/112529) targeting "Future", and SChannel structurally can't export secrets. **Must** use BouncyCastle's managed TLS stack, or change the protocol. See §3. |
| **Real-time audio + GC** | Manageable, but needs deliberate design. Native audio callbacks must not enter managed code. See §5. |
| **Opus 1.6 / DRED, RNNoise** | No managed equivalent exists. Stay native via P/Invoke. See §4. |
| **iOS ReplayKit Broadcast Upload Extension** | ⚠️ **Keep this in Swift.** 50 MB jetsam cap + a managed runtime in an appex that .NET for iOS does not officially support. See §8.4. |
Everything else — protobuf, SQLite, sockets, Argon2id, ChaCha20-Poly1305, X.509 generation,
jitter buffers, the mixer, the session model, the server — is either built into .NET or
covered by a permissive, well-maintained NuGet package.
**Net effect on native dependencies:** from 8 vcpkg deps + 1 vendored, down to **3 native
libraries** (libopus, RNNoise, miniaudio) — all three tiny, all three already vendored or
trivially buildable, and all three optional to *replace* later.
---
## 1. What exists today (baseline inventory)
Sizes are source bytes, to calibrate effort.
### Core — `core/` (~310 KB C++)
| File | Bytes | What it is | Port difficulty |
|------|-------|-----------|-----------------|
| `core/src/core/client.cpp` + `.h` | 107 K | `vc_client` — the entire client state machine: connect/auth/TOFU, channel + user model, stream lifecycle, reframing, encode path, event queue | **Medium**, but large. Mostly mechanical. |
| `core/src/audio/audio_engine.cpp` + `.h` | 69 K | miniaudio devices, `JitterBuffer`, per-ssrc decode, DRED/FEC/PLC ladder, mixer, level meters, talk detection, external feed/tap/mixed-sink | **Hard** — the RT-sensitive part |
| `core/src/net/transport.cpp` + `.h` | 25 K | Asio TCP framing `[u32 len][payload]`, UDP socket | **Easy**`Socket`/`System.IO.Pipelines` is nicer |
| `core/src/crypto/` | 27 K | mbedTLS TLS 1.3 wrapper, exporter, self-signed cert gen, Ed25519 identity, ChaCha20-Poly1305 + anti-replay, TOFU pin store | **Hard** — see §3 |
| `core/src/codec/opus_codec.*` | 9 K | libopus encode/decode wrapper incl. DRED | **Easy** — P/Invoke shim |
| `core/src/protocol/`, `core/src/session/` | 12 K | Envelope (de)serialize, dispatch, channel/user/stream registry | **Easy** |
| `core/src/audio/apm_processor.*` | 5 K | RNNoise wrapper + energy VAD | **Easy** |
| `core/src/voicecat.cpp` | 13 K | C ABI façade over `vc_client` | **Deleted** — no ABI needed any more |
| `core/include/voicecat.h` | 27 K | The C ABI | **Becomes a C# interface**, not an ABI |
| `core/proto/voicecat.proto` | 9 K | Wire format, source of truth | **Unchanged** |
### Server — `server/` (~80 KB C++)
`conn_session.cpp` (34 K, per-connection protocol handling), `db.cpp` (26 K, SQLite:
accounts, channels, bans), `session_registry.cpp` (20 K), `media_relay.cpp` (8 K, the SFU
relay), `server.cpp`, `identity.cpp`. All **easy-to-medium** — this is ordinary async network
server code and translates very well.
### Clients
| Client | Today | After the port |
|--------|-------|----------------|
| Windows | C#, WinForms, `net10.0-windows`, ~40 files | **Kept.** Swap `VoiceCat.Interop` P/Invoke for a direct project reference. |
| macOS | Swift + AppKit, `MainWindowController.swift` alone is 70 K | Rewrite as C# AppKit on `net10.0-macos` — near-mechanical, AppKit maps 1:1 |
| iOS | Swift + SwiftUI, ~150 K across views/audio | Rewrite as C# UIKit on `net10.0-ios` (or MAUI — see §8.3). **No 1:1 SwiftUI equivalent.** |
| iOS broadcast appex | Swift, 12 K (`SampleHandler.swift` + `BroadcastAudioRing.swift`) | **Stays Swift.** See §8.4 |
| `tools/vccli` | C++, 32 K | Rewrite as a C# console app — this becomes the primary conformance harness |
### Tests — `tests/` (29 files, ~290 KB, all green)
These are the real specification. Every one of them must be ported to xUnit and stay green;
the milestone exit criteria in `docs/roadmap.md` are encoded here.
---
## 2. Target solution layout
```
voice-cat/
├── proto/voicecat.proto # unchanged, single source of truth
├── native/ # the only C left
│ ├── opus/ # libopus 1.6 build scripts
│ ├── rnnoise/ # moved from third_party/
│ ├── miniaudio/ # miniaudio.h + voicecat_audio_shim.c (§5.2)
│ └── build-native.{ps1,sh} # produces per-RID binaries
├── src/
│ ├── VoiceCat.Protocol/ # Google.Protobuf codegen + Envelope framing
│ ├── VoiceCat.Crypto/ # TLS, media AEAD, anti-replay, identity, TOFU store
│ ├── VoiceCat.Codec/ # libopus P/Invoke + OpusEncoder/OpusDecoder
│ ├── VoiceCat.Dsp/ # RNNoise P/Invoke, energy VAD, resample helpers
│ ├── VoiceCat.Audio/ # devices, jitter buffer, mixer, RT ring buffers
│ ├── VoiceCat.Core/ # VoiceCatClient — replaces vc_client + the C ABI
│ ├── VoiceCat.Server/ # replaces server/
│ ├── VoiceCat.Cli/ # replaces tools/vccli + voicecat-admin
│ └── clients/
│ ├── VoiceCat.Windows/ # net10.0-windows, WinForms (kept, retargeted)
│ ├── VoiceCat.Mac/ # net10.0-macos, AppKit
│ ├── VoiceCat.iOS/ # net10.0-ios, UIKit
│ └── VoiceCatBroadcast/ # Swift appex — the one non-C# artifact
└── tests/VoiceCat.Tests/ # xUnit, ports all 29 ctest cases
```
**Target frameworks.** `VoiceCat.Core` and everything below it target plain `net10.0` — no
platform TFM — so the same assembly loads into the server, the WinForms app, the macOS app,
the iOS app, and the test host. Only the four leaf client projects carry a platform TFM.
**Why not one big assembly:** the layering is what keeps the "core owns audio, UI is thin"
rule enforceable. It also lets the server reference `VoiceCat.Protocol` + `VoiceCat.Crypto`
without dragging in miniaudio.
---
## 3. The TLS problem — read this before anything else
### 3.1 What breaks
`docs/security.md` §2 is built on one mbedTLS call:
```c
mbedtls_ssl_export_keying_material("voicecat media v1", ...) media keys
```
**.NET has no equivalent.** `SslStream` exposes no RFC 5705 exporter. The API proposal
([dotnet/runtime#112529](https://github.com/dotnet/runtime/issues/112529)) is labelled
`api-suggestion` ("NOT ready for implementation"), milestone *Future*, and the platform notes
on it are discouraging: *"Windows needs verification"* (SChannel runs TLS in a separate
privileged process and deliberately refuses to hand back secrets), *"OSX Not implemented
for Secure Transport"*. Do not plan around this shipping.
This is not a small dependency swap. The media key derivation is the root of the entire UDP
path: AEAD keys, nonce discipline, anti-replay, and the UDP binding token all hang off it.
### 3.2 Option A (recommended) — BouncyCastle managed TLS
[BouncyCastle for .NET](https://www.nuget.org/packages/BouncyCastle.Cryptography) (MIT,
actively maintained) ships a **complete managed TLS 1.3 client and server** in
`Org.BouncyCastle.Tls`, and `TlsContext` exposes exactly the method we need:
```csharp
byte[] ExportKeyingMaterial(string asciiLabel, byte[] contextValue, int length);
// "Export keying material according to RFC 5705" — TLS 1.3 (RFC 8446 §7.5) aware
```
*(verified against [bc-csharp `crypto/src/tls/TlsContext.cs`](https://github.com/bcgit/bc-csharp/blob/master/crypto/src/tls/TlsContext.cs); the C# port is feature-matched to bc-java here.)*
**Consequences — mostly good:**
-**Byte-identical wire compatibility with the existing C++ implementation.** This is the
single biggest de-risking factor in the whole project: it means a .NET client can talk to
the shipped C++ server (and vice versa) at *every* step of the port, and the C++ side
becomes a conformance oracle. See §11.
-**Identical TLS behaviour on all five platforms.** No SChannel-vs-OpenSSL-vs-
SecureTransport variance, no per-OS cipher-suite policy surprises, no "TLS 1.3 on macOS
only since .NET 10" caveat. For a self-hosted product shipping to unknown machines this is
worth a lot on its own.
- ✅ MIT, permissive — satisfies the hard no-GPL rule.
- ✅ Also gives us **Ed25519** and **BLAKE2b** (see §4), which .NET lacks.
**Costs:**
- ⚠️ Pure-managed TLS is slower than SChannel/OpenSSL. **This does not matter here** — the
TLS channel carries only control messages (a handshake plus a few KB/s of protobuf). Media
is UDP + ChaCha20-Poly1305, which uses the fast built-in .NET AEAD, not BouncyCastle.
- ⚠️ You implement `TlsClient` / `TlsServer` callback subclasses yourself (~200300 lines for
both sides): cipher-suite selection, certificate handling, `TlsCrypto` provider. Well-trodden
— BC ships `DefaultTlsClient`/`DefaultTlsServer` bases and `BcTlsCrypto`.
- ⚠️ You own the cert-validation logic (that's actually a *plus* for TOFU — see §3.4).
- ⚠️ One more external dependency in the trust base. It's Bouncy Castle; acceptable.
### 3.3 Option B — `SslStream` + in-band media keys (protocol v3)
Abandon the exporter. Since the TLS 1.3 channel is already confidential, authenticated and
forward-secret, the server can simply **generate the media keys and send them inside it**:
```proto
message AuthResult {
// ...
bytes udp_token = 6;
bytes media_key_c2s = 7; // 32 bytes, server-generated CSPRNG (NEW, v3)
bytes media_key_s2c = 8; // 32 bytes (NEW, v3)
}
```
This is what Mumble does (its OCB2/AES key exchange happens inside its TLS control channel),
and it is what SDES-SRTP-over-TLS does. Security posture is equivalent: an attacker who can
read these can already read everything.
- ✅ Uses only built-in `System.Net.Security.SslStream` — zero TLS dependencies.
- ✅ Simplest possible code; best per-platform TLS performance.
-**Breaks wire compatibility** → no cross-testing against the C++ implementation during
the port, which throws away the best safety net available.
- ❌ Protocol version bump to 3, forced-update for the shipped iOS/macOS/Windows clients.
- ❌ Loses the exporter's nice property that media keys are never *transmitted* at all.
- ⚠️ Server-side `SslStream` on Windows has a real footgun: a cert created with
`CertificateRequest.CreateSelfSigned` must be round-tripped through
`X509CertificateLoader.LoadPkcs12(cert.Export(X509ContentType.Pfx), null)` before SChannel
will accept it — an ephemeral/CNG-only key produces a confusing handshake failure.
### 3.4 Recommendation
**Take Option A (BouncyCastle) for the port. Keep Option B as an optional later
simplification** once the C++ tree is retired and you no longer need it as an oracle — at
which point it's a contained protocol-v3 change, not a rewrite.
TOFU is *easier* under Option A: BouncyCastle hands you the peer's DER certificate chain
directly in `TlsAuthentication.NotifyServerCertificate`, so
`SHA256.HashData(leafDer)` — the exact value `docs/security.md` §1.1 says is pinned — falls
out with no ceremony. (Under `SslStream` you'd get it from
`RemoteCertificateValidationCallback` via `cert.GetRawCertData()`; also fine, just less
direct.)
**Worth fixing while you're in here:** `security.md` §1.1 documents a known limitation — the
Ed25519 identity key is not bound to the TLS cert, so it's display-only. When generating the
self-signed cert in C#, **embed the Ed25519 public key as a X.509 extension or SAN URI**.
`CertificateRequest.CertificateExtensions` makes this trivial, and it closes the gap the doc
has been carrying.
---
## 4. Dependency map — C++ → .NET
| Concern | Today | .NET replacement | License | Notes / pitfalls |
|---------|-------|------------------|---------|------------------|
| Sockets, timers | Asio | **`System.Net.Sockets`** + `System.IO.Pipelines` + `PeriodicTimer` | built-in | Strictly better. `Socket.ReceiveFromAsync(SocketAddress)` (net8+) is the allocation-free UDP receive path — use it, not `UdpClient`. |
| Control framing | hand-rolled `[u32 len]` | `System.IO.Pipelines` `SequenceReader` | built-in | Removes a class of bugs. Keep the same 16 MiB frame cap. |
| TLS 1.3 | mbedTLS | **BouncyCastle `Org.BouncyCastle.Tls`** | MIT | See §3. **Not `SslStream`.** |
| Media AEAD | libsodium ChaCha20-Poly1305 | **`System.Security.Cryptography.ChaCha20Poly1305`** | built-in | ⚠️ Check `ChaCha20Poly1305.IsSupported` at startup — it is OS-backed (Windows 10 1903+ / OpenSSL 1.1+). Fall back to BouncyCastle's `ChaCha20Poly1305` if false. Same 12-byte nonce, 16-byte tag → identical wire bytes. |
| Anti-replay window | hand-rolled 64-bit | port verbatim | — | ~40 lines. Keep the RFC 3711 §3.3 ordering (replay-check → authenticate → *then* advance). This ordering is load-bearing; `test_media_aead.cpp` covers it. |
| Argon2id | libsodium `crypto_pwhash` | **BouncyCastle `Argon2BytesGenerator`** | MIT | Implemented with a strict libsodium PHC parser and original costs; native database tests prove existing-account import and managed-account verification by C++. See `docs/api-dotnet.md` for cost bounds. |
2026-08-05 03:19:59 +02:00
| BLAKE2b (channel passwords) | libsodium `crypto_generichash` | **`Blake2Fast`** (MIT) or BouncyCastle `Blake2bDigest` | MIT | Salted BLAKE2b-256, must produce identical digests to keep existing channel passwords working. Blake2Fast is SIMD and fast enough for the net thread, preserving the reason BLAKE2b was chosen over Argon2 here. |
| Ed25519 identity | libsodium | **BouncyCastle `Ed25519Signer`** | MIT | ⚠️ **Not in .NET 10.** [dotnet/runtime#63174](https://github.com/dotnet/runtime/issues/63174) is api-approved but milestoned **11.0.0**. Since Option A already pulls in BouncyCastle, this is free. |
| Self-signed cert gen | mbedTLS x509write | **`CertificateRequest.CreateSelfSigned`** | built-in | Much nicer than the C++ version. ECDSA-P256, same as today. Add the Ed25519 SAN (§3.4). |
| CSPRNG | libsodium | **`RandomNumberGenerator`** | built-in | — |
| Opus codec | libopus 1.6 | **P/Invoke libopus 1.6** | BSD | **Keep native.** [Concentus](https://github.com/lostromb/concentus) is a pure-C# Opus port but it tracks **Opus 1.1** — it has no DRED, no 1.6 features. `docs/voice.md` §4 and `test_dred_toggle.cpp` depend on DRED. Concentus is a viable *fallback* for a future platform where native linking is impossible, not the primary. |
| Noise suppression | RNNoise (vendored) | **P/Invoke RNNoise** | BSD-3 + CC0 | **Keep native.** No managed port exists. It's ~5 exported functions; the binding is trivial. Already vendored at `third_party/rnnoise/`. |
| Audio capture/playback | miniaudio | **P/Invoke miniaudio via a shim** | MIT-0/PD | **Keep native**, see §5.2. Managed alternatives exist ([SoundFlow](https://www.nuget.org/packages/SoundFlow), [MiniaudioSharp](https://www.nuget.org/packages/MiniaudioSharp), NAudio/CSCore for Windows-only) but auto-generated bindings marshal the callback into managed code, which is exactly what you must avoid (§5.1). Write the shim yourself. |
| Energy VAD | hand-rolled | port verbatim | — | ~60 lines. Trivial. |
| Protobuf | protobuf-lite (C++) | **`Google.Protobuf`** + `Grpc.Tools` | BSD | ⚠️ Reference `Grpc.Tools` for the `protoc` MSBuild integration even though there is no gRPC here — it is the standard way to codegen `.proto` in a `.csproj`. `<Protobuf Include="../../proto/voicecat.proto" GrpcServices="None" />`. The `.proto` needs **zero changes**. |
| SQLite | sqlite3 | **`Microsoft.Data.Sqlite.Core` + SQLitePCLRaw** | MIT / public domain | Implemented with provider 10.0.5, bundle 3.0.2 and pinned SQLite 3.50.4.2. Same schema/file; native database import is tested. NativeAOT publishing remains to be validated. |
2026-08-05 03:19:59 +02:00
| Logging | spdlog | **`Microsoft.Extensions.Logging`** (+ Serilog console sink) | MIT/Apache | Use `LoggerMessage` source generators on any path near the hot loop. Never log from an audio path. |
| Server config | `server.toml` | **`Tomlyn`** (MIT) or switch to JSON + `System.Text.Json` | MIT | Tomlyn keeps `server.toml` compatible; recommended, since operator-facing config shouldn't churn. |
| CLI arg parsing | hand-rolled | **`System.CommandLine`** | MIT | For `VoiceCat.Cli` and the server. |
| Build | CMake + vcpkg | **`dotnet build`** + a small native build script | — | vcpkg disappears entirely except for the 3 native libs, which you can vendor as sources and build with a 20-line CMakeLists or even `cl`/`gcc` directly. |
### 4.1 License check
Every replacement is MIT / BSD / Apache-2.0 / built-in. **The hard no-GPL/LGPL rule in
`docs/tech-stack.md` §5 holds.** BouncyCastle is MIT. Konscious is MIT. Blake2Fast is MIT.
Tomlyn is MIT. The .NET runtime itself is MIT.
---
## 5. Real-time audio — the hard part
This is where a naive port fails. `docs/architecture.md` §3 states the rule: *"Audio
(real-time) threads must not allocate, lock, log, or do syscalls."* A managed runtime adds a
second rule: **they must not be subject to GC pauses, and they must not be managed threads at
all if avoidable.**
### 5.1 Why you cannot just P/Invoke miniaudio and use `[UnmanagedCallersOnly]`
miniaudio calls your `ma_device_data_proc` on an OS-owned real-time audio thread (WASAPI's
MMCSS thread, a CoreAudio IOThread, an ALSA thread). If that callback is a managed method:
1. **The thread must attach to the runtime.** First entry does thread registration; every
entry does a managed↔native transition.
2. **The thread becomes GC-suspendable.** A gen-0 collection anywhere in the process can
suspend it mid-callback. WASAPI in exclusive/low-latency mode will glitch on a 2 ms stall;
a gen-2 blocking collection is fatal to the audio.
3. **Delegate lifetime.** Even with `[UnmanagedCallersOnly]` (which correctly avoids the
marshalling stub and the `GCHandle` dance) the *reachability* problem is solved but the
suspension problem is not.
The existing Windows client sidesteps all of this by keeping the entire audio pipeline in
C++. A pure-.NET port has to solve it deliberately.
### 5.2 Recommended design: a native shim that owns the RT thread
Write **one small C file** (`native/miniaudio/voicecat_audio_shim.c`, est. 300400 lines)
that compiles miniaudio and exposes a *pull/push ring-buffer API* instead of a callback API:
```c
// The audio callback lives entirely in C. It only ever touches lock-free ring buffers.
// Managed code polls. No managed frame is ever on an RT stack.
vcsh_device* vcsh_capture_open (const char* device_id, uint32_t channels, uint32_t rate);
size_t vcsh_capture_read (vcsh_device*, int16_t* dst, size_t frames); // non-blocking
vcsh_device* vcsh_playback_open(const char* device_id, uint32_t channels, uint32_t rate);
size_t vcsh_playback_write(vcsh_device*, const int16_t* src, size_t frames);
int vcsh_playback_wait(vcsh_device*, int timeout_ms); // eventfd/Event, wakes the mixer
void vcsh_enumerate(int capture, vcsh_device_info** out, size_t* n);
```
Managed side then runs a **normal, dedicated, non-RT `Thread`** at
`ThreadPriority.Highest`, woken by `vcsh_playback_wait`, that does: drain jitter buffers →
Opus decode → NR → gain/mute → mix → `vcsh_playback_write`. The ring absorbs GC pauses; size
it for ~120 ms (6 × 20 ms frames), which is well within the latency budget the jitter buffer
already targets (`target_depth_ms_` starts at 40).
This is *the same architecture the code already has*`AudioEngine` already runs a
`mixer_timer_thread_` for the iOS external-playback path and already has per-stream ring
buffers (`RemoteStream::ring`). You are generalising the iOS path to every platform. That is
a pleasing simplification, and it means the iOS design needs no special case at all.
**Bonus:** it makes `vc_set_external_playback` / `vc_set_mixed_output_sink` disappear as
special modes. Everything is external playback; the shim is just one more sink.
### 5.3 GC and allocation discipline in the managed audio path
Even off the RT thread, the decode/mix loop runs 50×/second per stream and must not churn:
- `<ServerGarbageCollector>false</ServerGarbageCollector>` and
`<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>` on client apps.
Set `GCSettings.LatencyMode = GCLatencyMode.SustainedLowLatency` while a call is active.
- **Preallocate everything at stream init**, exactly as `RemoteStream::init_ring` does today.
Use `int16[]` fields, not `new` per frame.
- Use `Span<short>` / `ReadOnlySpan<short>` throughout the DSP; `ArrayPool<short>.Shared` for
the rare variable-size case. Never LINQ, never `IEnumerable`, never `List<T>` growth on
this path.
- Marshal to native with `fixed` + raw pointers, or declare P/Invokes as
`[LibraryImport]` taking `ref short` / `ReadOnlySpan<short>` — the source generator emits
pinning without a marshalling stub. **Do not** use `Marshal.Copy` per frame.
- **Add an allocation regression test.** `GC.GetAllocatedBytesForCurrentThread()` before/after
1000 simulated mix cycles must be ~0. This is a cheap, high-value test the C++ code can't
even express.
- The mixer's soft limiter, the RMS level meter, and the RNNoise call are all
fixed-work-per-frame — they port directly.
### 5.4 Jitter buffer
`JitterBuffer` uses `std::map<uint32_t, Frame>` keyed by timestamp with wraparound handling,
plus `try_lock` everywhere so the RT thread never blocks. In C#:
- `SortedDictionary<uint,Frame>` allocates per insert. Prefer a **fixed-capacity circular
array of pre-allocated frame slots** indexed by `(ts / frameSamples) % capacity` — the
buffer is bounded at 500 ms anyway (`kLateDropSamples`), so a ring is the natural shape and
removes all allocation. This is a genuine improvement over the current C++.
- Replace `try_lock` with `Monitor.TryEnter` or, better, make the ring single-producer
(net thread) / single-consumer (mixer thread) with `Volatile`/`Interlocked` indices and drop
the lock entirely.
- Keep the EWMA jitter estimation, the leading-edge reseed, and the frame-skip catch-up logic
**verbatim** — that logic is subtle, hard-won, and covered by `test_jitter_depth.cpp`.
### 5.5 Opus P/Invoke
```csharp
[LibraryImport("opus")]
internal static partial int opus_encode(IntPtr st, ReadOnlySpan<short> pcm, int frameSize,
Span<byte> data, int maxDataBytes);
```
- `opus_encoder_ctl` is **varargs** — P/Invoke cannot do C varargs portably. The implemented
desktop binding uses fixed C entry points in `dotnet/native/media.c`; C calls the
varargs function with the correct ABI. This also handles Apple arm64's different
varargs calling convention. Only whitelisted single-int controls are accepted.
2026-08-05 03:19:59 +02:00
- DRED (`opus_dred_alloc`, `opus_dred_parse`, `opus_decoder_dred_decode`) binds the same way.
Guard with a runtime feature check as `opus_codec.cpp` does today.
- **iOS requires static linking**: use `[LibraryImport("__Internal")]` and link
`libopus.a` via `<NativeReference>` in the `.csproj`. Multi-target the DllImport name with a
`const string` behind `#if IOS`.
---
## 6. Core client port — `VoiceCat.Core`
`vc_client` (107 KB) is the biggest single unit. It becomes `VoiceCatClient : IAsyncDisposable`.
### 6.1 The C ABI goes away — and the API gets much better
The 60-odd `vc_*` functions were shaped by C ABI constraints. In C#:
| C ABI pattern | C# replacement |
|---------------|----------------|
| `vc_result` enum returns | Exceptions for programmer errors; `VoiceCatResult` for protocol outcomes |
| `vc_callbacks.on_event` + `vc_event` union-ish struct | **`IAsyncEnumerable<VoiceCatEvent>`** or typed `event` handlers per event type. Kill the `u32a` generic-payload field — use a discriminated hierarchy (`record UserJoined(uint UserId, uint ChannelId, string Nick)`). |
| `VC_EVENT_JOIN_RESULT` correlating with `vc_join_channel` | **`Task<JoinResult> JoinChannelAsync(uint id, string? pw, CancellationToken ct)`** — request/response correlation via `TaskCompletionSource` keyed on `Envelope.request_id`. This removes an entire class of "which reply was mine" bugs and shrinks every client's code. |
| `vc_list_channels` + `vc_free_channel_list` | `IReadOnlyList<Channel> Channels { get; }` — no ownership contract at all |
| `vc_get_server_identity_display(buf, cap, out len)` two-call idiom | `string ServerIdentityDisplay { get; }` |
| `vc_set_pcm_sink` / `vc_set_mixed_output_sink` / `vc_stream_feed_pcm` | Keep as-is conceptually — they're the bot/extension API. `Action<PcmFrame>` or a `ChannelWriter<T>`. Document the no-blocking rule just as loudly. |
| `vc_test_inject_capture` | `internal` test hook, not public API |
**Do this deliberately, not accidentally.** Write `docs/api-dotnet.md` as the successor to
`voicecat.h`, and keep the same rule from `CLAUDE.md`: changing it is a versioned act.
### 6.2 Threading model in C#
| C++ | C# |
|-----|-----|
| Asio `io_context` on `io_thread_` | A single `async` read loop over `PipeReader` per connection; no explicit thread |
| `WorkerPool` (blocking work) | Default `ThreadPool``Task.Run` for Argon2id, SQLite, DNS |
| Event queue drained by UI | `System.Threading.Channels.Channel<VoiceCatEvent>` (already what the Windows client does) |
| Mixer timer thread | Dedicated `Thread` (§5.2) — **not** a `Task`, the thread pool is not for this |
| Capture/encode thread | Dedicated `Thread`, fed by the shim's capture ring |
`WorkerPool` (861 bytes) simply deletes.
### 6.3 State model
`client.cpp` holds channel/user/stream maps guarded by mutexes and exposes them through
pull-based `vc_list_*`. In C#, hold them as immutable snapshots swapped with
`Volatile.Write` — readers get a consistent view with no locking, and the UI can bind to it
directly. `VC_EVENT_CHANNEL_LIST` becomes "a new snapshot is available", which is what it
already means.
---
## 7. Server port — `VoiceCat.Server`
The most mechanical part of the project. Straight `async`/`await` network code.
| Component | Port notes |
|-----------|-----------|
| `server.cpp` — accept loop | `Socket.AcceptAsync` loop + `Task` per connection. Trivial. |
| `conn_session.cpp` (34 K) — per-conn protocol | The bulk. A big `switch` on `Envelope.BodyCase`. Mechanical; write it against the ported xUnit tests. |
| `session_registry.cpp` | `ConcurrentDictionary<ulong, Session>` + a channel-membership index. Simpler than the C++. |
| `media_relay.cpp` — the SFU | **The server hot path.** Authenticate/decrypt using the sender's directional key, then reseal for each recipient with its directional key and next counter. Preserve SSRC, timestamp, flags, and encoded Opus bytes; replace sequence and ciphertext/tag. Use pooled buffers and `Socket.ReceiveFromAsync(Memory<byte>, SocketAddress)`. Never decode audio. Benchmark fan-out and allocations. |
2026-08-05 03:19:59 +02:00
| `db.cpp` (26 K) — SQLite | `Microsoft.Data.Sqlite`, same schema, same file. Keep raw SQL — do not introduce EF Core; the schema is 4 tables and EF's startup cost hurts the "single binary, instant start" goal. |
| `identity.cpp` | `CertificateRequest` + BouncyCastle Ed25519. Reads the same on-disk files. |
| Keepalive reaper | `PeriodicTimer` — cleaner than the `asio::steady_timer`. |
| `server.toml` | Tomlyn, unchanged format. |
### 7.1 Deployment — keeping the "single static binary" promise
`docs/deployment.md` promises a single statically-linked executable with no runtime to
install. **NativeAOT preserves this:**
```xml
<PublishAot>true</PublishAot>
<InvariantGlobalization>true</InvariantGlobalization>
<StripSymbols>true</StripSymbols>
```
- ✅ SQLitePCLRaw, Google.Protobuf, and BouncyCastle are all AOT-compatible.
- ✅ Startup drops to ~5 ms; binary lands around 1525 MB (vs. the current C++ static binary
— comparable order of magnitude).
- ⚠️ **No reflection-based JSON/config.** Use `System.Text.Json` source generators
(`JsonSerializerContext`) if you use JSON anywhere. Tomlyn's model binding uses reflection —
either use its low-level `DocumentSyntax` API or add trim descriptors.
- ⚠️ Cross-compilation is per-RID; you need a build machine per target (`linux-x64`,
`linux-arm64`, `win-x64`, `osx-arm64`). Same as today with vcpkg, so no regression.
- The Docker image gets *simpler*: `FROM scratch`-ish with a NativeAOT binary, or
`mcr.microsoft.com/dotnet/runtime-deps:10.0-noble`.
**Alternative if AOT fights you:** self-contained single-file publish
(`PublishSingleFile` + `SelfContained`) — bigger (~70 MB) and slower to start, but no AOT
constraints. Keep as a fallback per-RID, not the default.
---
## 8. Clients
### 8.1 Windows — the easy one
The `net10.0-windows` WinForms app is already C# and already structured around
`Channel<VoiceCatEvent>` + a 30 ms UI-thread pump. The port is:
1. Delete `VoiceCat.Interop` (the P/Invoke layer) and `VoiceCat.Interop.Tests`.
2. `<ProjectReference Include="VoiceCat.Core" />`.
3. Update ~40 call sites from `VcResult r = Native.vc_join_channel(...)` to
`await client.JoinChannelAsync(...)`. Mostly a find/replace plus `async void`
`async Task` hygiene on event handlers.
4. `Audio/ProcessLoopbackCapture.cs`, `ProcessAudioMixer.cs`, `AudioSessionEnumerator.cs`
(WASAPI process loopback, `AUDIOCLIENT_ACTIVATION_PARAMS`) — **unchanged**. They already
feed `vc_stream_feed_pcm`; they'll feed `client.FeedPcm(...)`.
5. `Native/RawInput.cs` (PTT hotkeys), `Models/PasswordProtector.cs` (DPAPI),
`Notifications/*` (SAPI announcer, sound pool) — **unchanged**.
**WinForms accessibility (the reason it was chosen over WinUI 3) is unaffected.** Keep it.
**Estimated effort: 12 weeks.** This client is nearly free.
### 8.2 macOS — AppKit in C#
`net10.0-macos` gives full AppKit bindings via [dotnet/macios](https://github.com/dotnet/macios).
The Swift AppKit code maps almost line-for-line:
| Swift | C# |
|-------|-----|
| `NSWindowController`, `NSOutlineView`, `NSTableViewDataSource` | Same types, same selectors, PascalCase |
| `@objc func handleClick(_ sender: Any)` | `[Export("handleClick:")] void HandleClick(NSObject sender)` |
| `accessibilityLabel`, `NSAccessibility.post(.announcement)` | `AccessibilityLabel`, `NSAccessibility.PostNotification(...)`**full VoiceOver parity, the reason AppKit was chosen holds** |
| `ScreenAudioCapture.swift` — ScreenCaptureKit | ✅ **ScreenCaptureKit is bound** in `net10.0-macos` (`SCStream`, `SCContentFilter`, `SCStreamConfiguration` incl. `CapturesAudio` / `ExcludesCurrentProcessAudio`). Per-app include/exclude filters and the VoiceOver-exclusion set port directly. |
| `InputDeviceCapture.swift` — CoreAudio | AVFoundation/CoreAudio bound; or just use the miniaudio shim on macOS |
`MainWindowController.swift` is 70 KB — this is the single largest UI rewrite. Budget for it.
⚠️ **Distribution:** a `net10.0-macos` app bundle needs codesigning + notarization, and
NativeAOT for macOS app bundles is supported but adds a step. Nothing blocking; just not
free.
2026-09-19 15:43:37 +02:00
### 8.3 iOS — the SwiftUI gap (decision implemented 2026-09-19)
2026-08-05 03:19:59 +02:00
This is the only client with no mechanical path, because **SwiftUI has no C# equivalent.**
Three options:
| Option | Pros | Cons |
|--------|------|------|
| **A. UIKit in C#** (`net10.0-ios`, hand-written) | Full API access, best accessibility control, matches the macOS/AppKit approach, no extra framework | The ~150 KB of SwiftUI views (`SettingsView`, `ChannelTreeView`, `ChatView`, …) must be re-authored as UIKit — a real rewrite, not a translation |
| **B. .NET MAUI** | Fastest to write; XAML declarative style is closest in spirit to SwiftUI; one codebase could later cover macOS too (Mac Catalyst) | ⚠️ Accessibility is weaker than native UIKit — and the project *explicitly* chose native toolkits for screen-reader quality (`roadmap.md` §2). Extra abstraction layer over the audio-sensitive app lifecycle. |
| **C. Avalonia** | One UI codebase for Windows + macOS + iOS | ⚠️ Same accessibility objection as MAUI, *and* it would mean abandoning WinForms/AppKit — contradicts two settled decisions |
**Recommendation: Option A (UIKit).** It's more work but it is the only choice consistent
with the accessibility commitments already made twice in the docs. Budget it as the largest
single client task.
2026-09-19 15:43:37 +02:00
The decision is now implemented in `clients/apple/dotnet/VoiceCat.iOS`: a `net10.0-ios`
UIKit host uses the managed core, statically links the Opus/RNNoise shim through
`__Internal`, and embeds the retained Swift ReplayKit extension. The Swift app remains the
migration oracle until the UIKit client completes its device, live-call and VoiceOver gates.
2026-08-05 03:19:59 +02:00
What ports cleanly regardless:
- `IOSAudioRouter.swift` (31 KB) — `AVAudioSession` is fully bound. `SetPreferredDataSource`,
`SetPreferredPolarPattern`, `AllowBluetoothA2DP`, `MeasurementMode` all exist in C#. The
re-entrancy guards and route-change filtering (`.categoryChange` / `.routeConfigurationChange`
/ `.override`) port verbatim. **Keep the invariants in `voice.md` §8 exactly.**
- `IOSVoiceProcessingEngine.swift` (24 KB) — `AVAudioEngine`, `AVAudioSourceNode`,
`SetVoiceProcessingEnabled(true)`, `VoiceProcessingAgcEnabled` are all bound.
- Under §5.2's design, iOS stops being a special case: the core is *always* externally
driven, and `AVAudioEngine` is simply the iOS "shim" implementation.
⚠️ **iOS + NativeAOT:** .NET for iOS ships Mono AOT by default; NativeAOT for iOS is
[still experimental](https://learn.microsoft.com/en-us/dotnet/maui/deployment/nativeaot).
Mono AOT is fine for the host app (it's what every Xamarin/MAUI app ships). Do not depend on
NativeAOT on iOS.
⚠️ **App Store:** you already ship `me.iamtalon.voicecat`. A runtime change is invisible to
review, but re-validate background-audio behaviour (`UIBackgroundModes: audio`) under Mono —
managed finalizers and the GC must not stall the audio render callback while backgrounded.
§5.2's native-ring design is what protects you here.
### 8.4 iOS screen sharing — **keep this in Swift**
You anticipated this correctly. The ReplayKit Broadcast Upload Extension should **not** be
ported.
**Why:**
1. **The 50 MB jetsam cap.** A managed runtime (Mono AOT + metadata + GC heap) inside a
separate appex process eats a meaningful fraction of that before your code runs. The
current Swift `SampleHandler` is 4.4 KB and does one `AVAudioConverter` call per buffer.
2. **.NET for iOS does not officially support broadcast upload extensions.** Microsoft's own
guidance is that this is a [known gap with no documentation](https://learn.microsoft.com/en-sg/answers/questions/2006706/issues-with-bundling-ios-broadcast-extension-in-ne);
the supported extension types are enumerated and this isn't reliably among them.
3. **There is nothing to gain.** The extension deliberately does *not* link the core
(`voice.md` §9) — it converts PCM and writes to a shared ring. It is already a
language-agnostic boundary.
**The boundary is already clean.** `BroadcastAudioRing.swift` is an mmap'd file in an App
Group with an SPSC ring layout. C# reads it with `MemoryMappedFile.CreateFromFile` +
`MemoryMappedViewAccessor`, and `CFNotificationCenter` (Darwin notifications) is bound in
`net10.0-ios`. **Action: freeze `BroadcastAudioRing`'s binary layout as a documented struct**
(magic, version, capacity, head, tail, activeFlag, sample format) in
`docs/broadcast-ring-format.md`, so the Swift writer and the C# reader are contractually
pinned. The C# `BroadcastAudioPump` is then ~150 lines.
This leaves the repo with exactly **one Swift file plus one shared Swift ring** — an
acceptable, well-justified exception to "pure C#", and dramatically less than the current
three Swift codebases.
### 8.5 What about a Linux client?
Not in scope today, but worth noting: once the core is `net10.0` with a miniaudio shim
(which has ALSA/PulseAudio backends), a Linux client becomes a UI-only problem for the first
time. Avalonia would be the natural choice *there specifically*, without disturbing the
Windows/macOS/iOS decisions. Mention it in `roadmap.md`; don't build it now.
---
## 9. Tests
The 29 ctest cases **are** the specification. Port every one to xUnit in
`tests/VoiceCat.Tests/`. Grouping:
| Group | Tests | Notes |
|-------|-------|-------|
| Wire format | `test_envelope`, `test_voice_frame`, `test_frame_codec`, `test_frame_ms_reframe` | Port first. These are pure functions — fastest possible feedback on the protobuf + framing layers. |
| Crypto | `test_media_aead`, `test_tls_loopback`, `test_tofu_flow` | The AEAD test must produce **byte-identical** ciphertext to the C++ for a fixed key+nonce+AAD. Add that as a golden-vector test — it's your proof the port is wire-compatible. |
| Codec/DSP | `test_opus_codec`, `test_dred_toggle`, `test_noise_suppression`, `test_recv_noise_reduction`, `test_plc_cap` | Depend on the native P/Invokes; run them as soon as those exist. |
| Audio engine | `test_jitter_depth`, `test_channel_samplerate`, `test_external_pcm`, `test_external_playback`, `test_vad_ptt_devices` | The subtle ones. `test_jitter_depth` guards the bounded-depth invariant — do not weaken it. |
| Integration | `test_m1_integration`, `test_m2_voice`, `test_m3_multistream`, `test_tcp_loopback`, `test_disconnect_left`, `test_reaper_timeout` | Real client + real server in-process. |
| Moderation/admin | `test_m5_permissions`, `test_m5_kick_ban_move_mute`, `test_m5_channel_crud`, `test_m5_admin_accounts` | Server-side; port with `VoiceCat.Server`. |
| ABI surface | `test_voice_client_abi`, `test_channel_user_list_abi` | These test the C ABI specifically — **rewrite as API-shape tests** against the new C# surface, don't port literally. |
**New tests the port should add:**
- Allocation regression on the mix loop (§5.3).
- AEAD golden vectors vs. C++ output.
- A **cross-implementation test**: C# client ↔ C++ server, and C++ `vccli` ↔ C# server, run
in CI for as long as both trees exist (§11).
`ctest --preset dev``dotnet test`. The house rule in `CLAUDE.md` ("every commit builds and
passes") carries over unchanged.
---
## 10. Documentation changes
Per the `CLAUDE.md` rule that docs and code stay in sync:
| Doc | Change |
|-----|--------|
| `docs/architecture.md` | Rewrite §1 (shared-core model — it's now a shared *assembly*), §4 (C ABI → C# API), §3 (threading — the shim design). Keep §5 (server) and §2 (layers) nearly as-is. |
| `docs/tech-stack.md` | Replace the whole dependency table. Re-run the license audit (§5) — the no-GPL rule still passes. |
| `docs/security.md` | ⚠️ **§2 needs rewriting** to describe BouncyCastle's exporter rather than mbedTLS's, and §1.1 should be updated when the Ed25519↔cert binding lands (§3.4). §3–§6 unchanged. |
| `docs/voice.md` | §5 (jitter), §8 (pipeline), §10 (NR) get implementation-detail updates. The *protocol* sections (§2 frame format, §3 config, §4 loss resilience) are **unchanged** — that's the point. |
| `docs/protocol.md` | Unchanged unless you take Option B (§3.3), which adds two `AuthResult` fields and bumps to v3. |
| `docs/building.md` | Full rewrite: `dotnet build` + the native build script replace the CMake preset matrix. **Delete the "run ctest in PowerShell not Git Bash" warning** — that MinGW pathology disappears with the toolchain. |
| `docs/deployment.md` | §1.B/C update for NativeAOT publish; Docker base image changes. The zero-config promises hold. |
| `docs/roadmap.md` | Add the port as its own milestone; note the Linux-client possibility (§8.5). |
| `CLAUDE.md` | New build commands, new subsystem map, new house rules (no allocation in the audio path becomes explicit). |
| **new** `docs/api-dotnet.md` | The successor to `voicecat.h` — the versioned client API contract. |
| **new** `docs/broadcast-ring-format.md` | The frozen Swift↔C# App Group ring layout (§8.4). |
---
## 11. Migration strategy — the actual step-by-step
The guiding principle: **the C++ tree stays working and becomes the conformance oracle.** Do
not delete anything until the C# equivalent passes the same test against it. This is only
possible because Option A (§3.2) preserves wire compatibility — which is the main reason to
choose it.
The rewrite lives under `dotnet/`; initial implementation branch: `dotnet/foundations`,
created from `cs-port`. Keep the existing schema at `core/proto/voicecat.proto` during migration.
Native packaging is deferred until the codec/audio phase rather than blocking the wire slice.
Each phase ends with a green build,
2026-08-05 03:19:59 +02:00
green tests, and an updated `PROGRESS.md` entry.
---
### Phase 0 — Foundations (est. 1 week)
1. Create the solution skeleton from §2. `Directory.Build.props` with
`net10.0`, `<Nullable>enable</Nullable>`, `<TreatWarningsAsErrors>true</TreatWarningsAsErrors>`,
`<AnalysisLevel>latest-all</AnalysisLevel>`, `<InvariantGlobalization>true</InvariantGlobalization>`.
2. `native/build-native.{ps1,sh}`: build libopus 1.6, RNNoise, and the (empty for now)
miniaudio shim into `runtimes/{rid}/native/`. Vendor the sources — drop vcpkg.
3. `VoiceCat.Protocol`: add `Google.Protobuf` + `Grpc.Tools`, point at the **existing**
`core/proto/voicecat.proto`, verify generated C# types compile.
4. CI: GitHub Actions matrix building both trees (C++ and C#) side by side.
**Exit criterion:** `dotnet build` produces empty-but-real assemblies; generated protobuf
types are present; native libs land in the right RID folders.
---
### Phase 1 — Wire format, provably compatible (est. 1 week)
1. Port `envelope.cpp` (frame `[u32 len][payload]`) using `System.IO.Pipelines`.
2. Port `voice_frame.h` — the 20-byte big-endian header. Use
`BinaryPrimitives.WriteUInt64BigEndian` etc.
3. Port `SodiumMediaCrypto``MediaCrypto` on `System.Security.Cryptography.ChaCha20Poly1305`,
including the nonce scheme and the sliding replay window. **Preserve the RFC 3711 §3.3
ordering.**
4. Port `test_envelope`, `test_voice_frame`, `test_frame_codec`, `test_media_aead`.
5. **Generate golden vectors from the C++ build** (a throwaway C++ main dumping sealed frames
for fixed inputs) and assert the C# produces identical bytes.
**Exit criterion:** golden-vector tests green. From here on, byte-compatibility is measured,
not assumed.
---
### Phase 2 — TLS + the exporter (est. 1.5 weeks — highest-risk phase, do it early)
1. Spike first, in isolation: a BouncyCastle `TlsClientProtocol``TlsServerProtocol`
loopback over a `Socket` pair, both calling
`ExportKeyingMaterial("voicecat media v1", ...)` and asserting the two sides agree.
2. **Then the real proof:** a C# BouncyCastle client handshaking against the **existing C++
mbedTLS server**, both exporting with the same label, and asserting the derived media keys
are identical. *This is the single most important checkpoint in the whole project.* If it
fails, stop and reconsider Option B before writing anything else.
3. Port `ServerCert`/`ServerIdentity` (`CertificateRequest` + BC Ed25519), the TOFU pin store
(`tofu_store.cpp` — a trivial text file), and cert-fingerprint pinning.
4. Port `test_tls_loopback`, `test_tofu_flow`, `test_tcp_loopback`.
**Exit criterion:** C# client completes a TLS 1.3 handshake with the C++ server, derives
matching media keys, and pins the leaf fingerprint.
**Checkpoint (2026-09-15):** implemented nonblocking managed TLS, handshake-time
exporters, explicit certificate acceptance, persisted TOFU, and native-compatible
credentials. The C++ TLS oracle authenticates a media challenge in both directions
over an actual socket, proving exporter compatibility. Tests also cover managed
fragmented loopback, first-connect acceptance, changed-pin rejection, TLS 1.2 rejection,
close_notify/abrupt EOF, restart persistence, and import of C++ credential files.
Socket orchestration remains a transport-owner responsibility; the complete managed
server and client are later phases. See `dotnet/README.md` for the required native
interoperability test command.
2026-08-05 03:19:59 +02:00
---
### Phase 3 — Codec + DSP (est. 1 week)
1. `VoiceCat.Codec`: libopus `[LibraryImport]`, `OpusEncoder`/`OpusDecoder`, the varargs-CTL
workaround (§5.5), DRED.
2. `VoiceCat.Dsp`: RNNoise binding, `EnergyVadProcessor`.
3. Port `test_opus_codec`, `test_dred_toggle`, `test_noise_suppression`.
**Exit criterion:** encode→decode round-trip at every supported frame size; DRED recovery
test green; RNNoise output matches the C++ within tolerance.
**Checkpoint (2026-09-15):** implemented `VoiceCat.Codec`, `VoiceCat.Dsp`, safe native
handles, fixed-signature C bindings, and independent desktop native staging. The native
build pins the upstream Opus 1.5.2 release/checksum (matching the actual vcpkg baseline,
despite older code comments referring to 1.6), enables DRED, and shares the existing
vendored RNNoise sources/model. Tests cover 40 rate/channel/frame-size round trips,
PLC, 40 dropped-frame DRED recovery formats, C++ denoising within one PCM unit, VAD
hang time, and zero managed allocations over 1,000 combined processing cycles.
At 8/12 kHz, DRED tests encode at 16 kHz and decode at the requested rate: this pinned
encoder's activity analysis cannot emit DRED at 8/12 kHz. These encoding configurations
are explicitly rejected. Its redundancy floor is 30 ms because two chunks are required
to emit DRED; actual redundancy remains adaptive. Desktop CI builds/stages the library
before testing. iOS static native packaging and device audio remain later phases.
2026-08-05 03:19:59 +02:00
---
### Phase 4 — Server (est. 34 weeks)
Do the server before the client: it lets you point the **existing, trusted C++ `vccli`** at
it, which is a far better test client than a half-built C# one.
**Implemented checkpoint (2026-09-15):** bounded async TLS socket orchestration,
guest/password authentication, existing SQLite account/channel import, snapshots,
unprotected channel joins, channel/private/server text, ping and disconnect events.
The existing C++ CLI authenticates and sends text through this server. Argon2id uses
the existing BouncyCastle dependency with a strict libsodium PHC parser, not a new
Konscious dependency. Native database tests prove password compatibility in both
directions without resets. SQLite uses `Microsoft.Data.Sqlite.Core` 10.0.5,
SQLitePCLRaw bundle 3.0.2 and explicitly pinned SourceGear SQLite 3.50.4.2.
See `docs/api-dotnet.md` for limits. This first checkpoint did not include UDP voice,
streams, protected joins, moderation, admin handlers or production configuration.
The subsequent voice checkpoint is described below.
**Voice checkpoint:** the managed server now advertises UDP, issues session-bound
tokens, implements voice subscription and multi-stream signaling, and relays encrypted
Opus with recipient-specific counters. The first UDP endpoint is fixed for the session;
reconnect for endpoint changes. Immutable routing snapshots separate control handlers
from the UDP crypto owner. Real-socket tests cover replay/forgery/SSRC rejection,
channel/subscription isolation, stream stop and disconnect. A native client oracle
exercises bidirectional microphone and screen audio in mono and stereo. The fan-out
core has a 50-subscriber allocation regression test; transport scheduling and the
BouncyCastle crypto fallback are excluded from its zero-allocation guarantee.
Two real C++ `vccli` processes also pass join/text/bidirectional voice tests using
finite `--test-tone-ms` external capture/playback. The transport load test delivers
all 2,500 recipient packets from a sender paced at 50 pps to 50 subscribers.
**Reaper checkpoint:** configurable 45-second idle expiry / 15-second sweep replaces
the TCP-only idle timeout. Control envelopes, authenticated voice and bound-endpoint
keepalives refresh shared monotonic activity; invalid media does not. Reaping removes
presence and media routing, and can be disabled. Tests inject a clock to cover silent
clients, UDP-only activity, forged media, single departure events and disabled expiry.
**Channel/administration checkpoint:** protected joins and channel CRUD now persist using
the native BLAKE2b password format (native verification in both directions). Permissions
gate moderation and account create/reset/delete/list. Mute/deafen/move update encrypted
routing; kick/ban retire media and emit one reason-bearing departure. Guest bans use
addresses, account bans use usernames, and wire milliseconds convert to database seconds.
Temporary-channel permission only creates temporary channels and only administrators
grant permissions; these deliberately tighten native policy. Tree validation and Lobby
protection prevent invalid mutations. Existing streams stop on channel edits/moves/deletion.
The C++ CLI creates protected channels and administers accounts against the managed server;
its channel argument lifetimes and default audio config were corrected. See api-dotnet.md
for limits, persistence and policy differences. Production configuration/publishing and
the remaining server readiness checks still precede Phase 4 completion.
2026-08-05 03:19:59 +02:00
1. `VoiceCat.Server`: accept loop, `ConnSession` protocol handling, session registry.
2. `Db` on `Microsoft.Data.Sqlite` — same schema. **Resolve the Argon2id hash-compat
question here** (§4).
3. `MediaRelay` — the allocation-free SFU fan-out.
4. Keepalive reaper, moderation/admin handlers.
5. Port `test_m1_integration`, `test_m5_*`, `test_disconnect_left`, `test_reaper_timeout`.
**Exit criterion:** ▶ **C++ `vccli` connects to the C# server, authenticates, joins a
channel, sends text, and exchanges voice with a second C++ `vccli`.** That is the M1+M2 exit
criterion from `roadmap.md`, re-proven against the new server.
---
### Phase 5 — Audio engine (est. 45 weeks — the hardest phase)
1. Write and validate `voicecat_audio_shim.c` standalone (a C test that loops mic→speaker
through the rings, no .NET involved).
2. `VoiceCat.Audio`: device enumeration, the allocation-free jitter buffer (§5.4), per-ssrc
decode with the **DRED → FEC → PLC** ladder, per-stream NR/gain/mute, the mixer, level
meters, talk-state edge detection.
3. The dedicated mixer thread + capture thread.
4. External feed/tap/mixed-sink — now the *normal* path, not special modes.
5. Port `test_jitter_depth`, `test_external_pcm`, `test_external_playback`,
`test_channel_samplerate`, `test_plc_cap`, `test_recv_noise_reduction`,
`test_frame_ms_reframe`.
6. **Add the allocation-regression test.**
**Exit criterion:** `test_jitter_depth`'s bounded-depth invariant holds; zero allocations per
2026-09-18 18:51:53 +02:00
mix cycle; real multi-human calls on Windows and macOS have no reproducible audio-quality
defects once each client is feature-complete.
2026-08-05 03:19:59 +02:00
**Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every
protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream
controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero
managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation
uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device
2026-09-18 18:51:53 +02:00
shim; the codec/DSP shim remains the only native component. A real device smoke passed;
final listen validation is performed with real multi-human calls after feature completion,
not a synthetic ten-minute sine-wave gate.
2026-08-05 03:19:59 +02:00
---
### Phase 6 — Client core (est. 34 weeks)
1. `VoiceCatClient`: connect/TOFU/auth state machine, request/response correlation via
`TaskCompletionSource`, channel/user/stream snapshots, stream lifecycle, reframing, the
send path, the event stream.
2. `VoiceCat.Cli` — the `vccli` replacement, plus `voicecat-admin`.
3. Port `test_m2_voice`, `test_m3_multistream`, `test_vad_ptt_devices`; rewrite the two ABI
tests as API-shape tests.
**Exit criterion:** ▶ **Two C# `vccli` instances hold a multi-channel voice + text
conversation through the C# server**, and a C# `vccli` interoperates with a C++ `vccli` on
the same server. This is the full M0M3 criterion re-proven end to end.
**Complete (2026-09-16):** `VoiceCat.Core` implements TOFU-gated TLS, concurrent correlated
requests, snapshots/events, reconnects, encrypted UDP binding and send/receive stream
lifecycle. Two managed clients exchange text and decoded PCM through the managed server.
`VoiceCat.Cli` supports interactive channel text and deterministic headless text/voice runs.
Process tests prove two managed CLIs converse with decoded voice in both configured channels,
and a managed CLI exchanges text and bidirectional voice with the existing C++ CLI.
2026-08-05 03:19:59 +02:00
---
### Phase 7 — Windows client (est. 12 weeks)
Per §8.1. Ship this first of the three GUIs — it's the cheapest and it validates the C# API
shape against a real, complete UI before you commit to two rewrites.
**Exit criterion:** feature parity with the current WinForms build, NVDA smoke-tested.
**Checkpoint (2026-09-16):** the shipped WinForms project references `VoiceCat.Managed`, not
the P/Invoke core. The compatibility facade preserves UI-thread event pumping and stable
capture IDs while delegating all protocol and audio state to the idiomatic managed projects.
Channel moves automatically renegotiate active streams. A published build passed real WASAPI
capture/playback and form-startup smoke tests and contains `voicecat_media.dll` but no
`voicecat.dll`. Automated text, bidirectional PCM voice, multi-frame audio and stream-move
tests pass. NVDA and the manual endurance/listen pass remain before the Phase 7 exit criterion.
2026-08-05 03:19:59 +02:00
---
### Phase 8 — macOS client (est. 45 weeks)
Per §8.2. AppKit port, ScreenCaptureKit per-app audio selection, VoiceOver parity.
**Exit criterion:** feature parity with `VoiceCatMac`, VoiceOver smoke-tested, notarized
build produced.
2026-09-18 18:51:53 +02:00
**Checkpoint (2026-09-18):** `clients/apple/dotnet/VoiceCat.Mac` is a separate .NET 10
AppKit application that consumes `VoiceCat.Core` directly. It implements guest and account
connection, validated saved server profiles whose JSON never contains passwords,
interactive TOFU approval, protected-channel password prompts, channel browsing, roster,
channel/private text,
2026-09-16 22:16:18 +02:00
voice subscription, native accessibility labels and default-device microphone/playback.
2026-09-16 22:22:03 +02:00
Account passwords can be stored in macOS Keychain after successful authentication and are
removed when the user disables remembering or deletes the profile.
2026-09-16 22:16:18 +02:00
Core Audio capture is resampled
2026-09-16 22:05:40 +02:00
and converted through `AVAudioConverter` before entering the managed audio engine. Stereo
playback uses `AVAudioSourceNode` and the shared bounded PCM ring; its render callback does
not allocate, lock or block. Voice stream lifetime follows subscription, disconnect and
2026-09-18 18:51:53 +02:00
channel changes. The app now targets Microsoft's Xcode 27 preview bindings from workload
set 10.0.401. On Apple Silicon it selects a native Homebrew `protoc`, deduplicates the
transitive media dylib before AppKit bundling, uses the macOS 27 error-returning Core Audio
overloads, and keeps hardened runtime Release-only so local ad-hoc Debug builds can load
their runtime libraries. It enumerates and selects Core Audio input/output devices, requests
microphone permission explicitly, and renders through Core Audio's native planar Float32
layout. The arm64 app builds, signs, launches and displays on macOS 27. Live testing proved
physical-microphone transmission and clean peer playback; Lobby's expected DTX comfort noise
was distinguished from corruption by repeating the tone in the DTX-disabled Music Room. A
macOS CI job builds the native codec/DSP shim and Apple solution. The existing Swift app
remains the release client until the manual release gates below are complete.
**Functional-parity checkpoint (2026-09-19):** the managed app now covers persistent audio
and notification settings, VAD/configurable focus-scoped PTT/always-on input, stereo capture,
RNNoise, input/output/auxiliary gain, selectable auxiliary capture, self mute/deafen, speaking
state, event sounds and speech, modeless private conversations, full channel configuration,
per-user receive tuning, moderation, permissions and account administration. ScreenCaptureKit
publishes desktop audio with whole-desktop, application-only and application-exclusion scopes
and can exclude VoiceOver/speech processes. Legacy Swift profiles, TOFU pins and Keychain
passwords migrate without discarding the original profile JSON. A publishing script validates
an isolated ad-hoc distribution copy without mutating incremental build output and supports
Developer ID signing, notarization, stapling and Gatekeeper assessment. Debug and Release
builds, 190 managed tests, 29 native CTests and a
strictly verified/running ad-hoc Release bundle pass. Do not remove the Swift macOS app yet:
cutover still requires a manual VoiceOver navigation/announcement pass, ScreenCaptureKit plus
real multi-human call validation, and an actual credentialed notarization run.
2026-09-16 17:34:16 +02:00
2026-08-05 03:19:59 +02:00
---
### Phase 9 — iOS client (est. 57 weeks)
Per §8.3/§8.4. UIKit rewrite, `AVAudioSession` router port, `AVAudioEngine` VPIO path, and
the C# `BroadcastAudioPump` reading the **unchanged Swift** extension's ring.
**Exit criterion:** feature parity with `VoiceCatiOS`; screen-audio sharing works with the
Swift extension untouched; A2DP/stereo/VPIO preset matrix re-verified (this is where the
known stereo-A2DP class of bug lives — re-test it explicitly).
---
### Phase 10 — Cutover (est. 12 weeks)
1. Run both trees in CI for one full release cycle.
2. Delete `core/`, `server/`, `tools/`, `clients/apple/` (except `VoiceCatBroadcast/` and
`Shared/BroadcastAudioRing.swift`), `vcpkg/`, `CMakePresets.json`, root `CMakeLists.txt`.
3. Update every doc per §10.
4. Tag the last C++ commit so the oracle stays reachable.
---
### 11.5 Total estimate
**~79 months of focused single-developer work**, front-loaded with risk (Phase 2) and
back-loaded with volume (Phases 89). The server + core (Phases 06) is roughly 4 months and
is the part that removes the most complexity; the three GUIs are roughly half the calendar
time and almost none of the difficulty.
---
## 12. Risk register
| # | Risk | Severity | Mitigation |
|---|------|----------|------------|
| 1 | **BouncyCastle's exporter doesn't interoperate with mbedTLS's** | 🔴 Critical | Prove it in Phase 2 step 2, before any other work depends on it. Both implement RFC 8446 §7.5, so it should — but *verify*, don't assume. Fallback: Option B (§3.3). |
| 2 | **GC pauses cause audio glitches** | 🔴 High | Native shim owns the RT thread (§5.2); ~120 ms ring; allocation-regression test; `SustainedLowLatency`. This is the design's whole answer. |
| 3 | **iOS audio regressions under Mono AOT** | 🟠 Medium-High | The iOS audio path is already the most delicate part of the product (see the A2DP/VPIO invariants in `voice.md` §8). Re-test the full preset × route matrix. Do not port the invariants "roughly". |
| 4 | **Argon2id hashes don't verify → existing accounts locked out** | 🟠 Medium | Decide in Phase 4. Preferred: parse libsodium's PHC string and pass m/t/p to Konscious; verify against real hashes from an existing `voicecat.db` *before* writing the rest of `Db`. |
| 5 | **SFU relay throughput regression** | 🟠 Medium | Benchmark early (Phase 4): N=50 subscribers × 50 pps. Allocation-free `SocketAddress` receive + pooled buffers. .NET's socket layer is good; this should be fine, but measure. |
| 6 | **`ChaCha20Poly1305.IsSupported == false`** on some target | 🟡 Low | Startup check + BouncyCastle fallback. One-line risk. |
| 7 | **`opus_encoder_ctl` varargs breaks on a future ABI** | 🟡 Low | Only single-`int` CTLs are used; document it, add a test that exercises every CTL used. |
| 8 | **NativeAOT trimming breaks protobuf/SQLite reflection** | 🟡 Low | All three are AOT-tested upstream. Add an AOT-published smoke test to CI from Phase 4. |
| 9 | **macOS/iOS bindings lag a new Xcode** | 🟡 Low | dotnet/macios tracks Xcode closely (bindings exist through Xcode 26). Pin the workload version. |
| 10 | **Scope creep — "while we're rewriting, let's also…"** | 🟠 Medium | The port is a *translation*. The API-shape improvements in §6.1 are the only sanctioned redesign. Everything else goes in `roadmap.md`. |
---
## 13. What you gain
Worth being explicit, since this is 7+ months:
- **One language, one toolchain, one debugger.** No more CMake presets, vcpkg triplets,
MinGW-vs-PowerShell execution pathologies, XCFramework fat-static-lib packaging, or a C ABI
that has to be hand-mirrored into both Swift and C#.
- **Three UI codebases instead of three UI codebases *plus* a core plus two binding layers.**
The `VoiceCat.Interop` P/Invoke layer, the `VoiceCatCore` Swift wrapper, the module map, and
the whole `voicecat.h` ABI surface all cease to exist.
- **Better API.** `await client.JoinChannelAsync()` instead of "call this, then wait for
`VC_EVENT_JOIN_RESULT`, and hope it's yours."
- **Memory safety** across the entire protocol-parsing surface — the part most exposed to
hostile input.
- **8 native dependencies → 3**, each small and vendored.
- **Tests that can assert things C++ couldn't**, notably zero-allocation invariants.
And what you keep: the protocol, the wire format, the security model, the audio design, the
accessibility-first UI toolkit choices, and every single one of the 29 behavioural tests.