Add managed codec DSP and initial control server
This commit is contained in:
@@ -115,3 +115,99 @@ On Unix new files use owner read/write permissions; Windows inherits directory A
|
||||
The credential directory must have one provisioning owner. PEM strings and crypto
|
||||
library internal copies are managed memory; owned-array clearing does not promise
|
||||
erasure of every runtime/library copy.
|
||||
|
||||
## Codec and DSP
|
||||
|
||||
`VoiceCat.Codec` and `VoiceCat.Dsp` call the desktop `voicecat_media` native library
|
||||
through source-generated `LibraryImport`. It links pinned Opus 1.5.2 and the existing
|
||||
vendored RNNoise; it has no dependency on libvoicecat or its C ABI. Fixed C signatures
|
||||
wrap Opus controls so P/Invoke never calls C varargs. SafeHandle owns every native
|
||||
encoder, decoder, DRED parser/state, and denoiser, including failed initialization.
|
||||
|
||||
`OpusOptions` is an immutable record. Supported PCM rates are 8/12/16/24/48 kHz,
|
||||
one or two interleaved channels, and integral 10/20/40/60 ms frames. These match the
|
||||
current VoiceCat protocol's integer frame duration; fractional Opus frame durations
|
||||
are not exposed. Low-delay application mode requires at most 20 ms. Channel capture
|
||||
bandwidth is controlled separately by `MaximumBandwidthHz`; the production audio
|
||||
clock will remain 48 kHz. Options are validated before native creation, and native
|
||||
control failures throw `OpusException` with the libopus error code.
|
||||
|
||||
`OpusEncoder.Encode(ReadOnlySpan<short>, Span<byte>)` accepts exactly one frame
|
||||
and returns encoded bytes. `OpusDecoder.Decode(packet, pcm, samplesPerChannel,
|
||||
recoverPreviousFrame)` returns samples **per channel**, not total interleaved samples.
|
||||
An empty packet requests PLC. Passing the next packet with `recoverPreviousFrame`
|
||||
requests in-band FEC; absence of FEC permits libopus's PLC fallback. Decode that next
|
||||
packet normally afterward. Capacity/overlap errors throw before native processing.
|
||||
|
||||
DRED is explicit. Unsupported native builds reject `DeepRedundancy = true` rather
|
||||
than silently disabling it. With pinned Opus 1.5.2, DRED encoding requires PCM at
|
||||
16/24/48 kHz; its activity analysis cannot emit DRED at 8/12 kHz. Such configurations
|
||||
are rejected. DRED packets can still be decoded at all five rates. The encoder uses
|
||||
a 30 ms minimum redundancy duration because this release needs two redundancy chunks;
|
||||
the old 20 ms setting produces no DRED packets. Actual redundancy remains adaptive
|
||||
to bitrate, loss estimate, and activity; it is not guaranteed in every packet.
|
||||
|
||||
`OpusDeepRedundancy.TryRecover(audioDecoder, nextPacket, pcm, samplesPerChannel,
|
||||
offset)` parses the next packet and reconstructs a missing frame. Default offset is
|
||||
one missing frame's samples per channel before the next packet's start, matching
|
||||
libopus's offset convention. A packet without DRED returns false; then the owner
|
||||
can try FEC/PLC. Only consume recovery output on success. Parse/native errors throw.
|
||||
|
||||
`RnnoiseProcessor.Process(Span<short>, sampleRate)` operates in place on complete
|
||||
480-sample mono chunks at 48 kHz. Other rates pass through unchanged; partial chunks
|
||||
at 48 kHz throw instead of leaving a tail silently untreated. Float scratch is
|
||||
preallocated, and rounding/clipping matches the C++ processor. Use distinct instances
|
||||
for stereo channels when the later pipeline supports stereo microphone denoising.
|
||||
Noise reduction does not gate speech.
|
||||
|
||||
`EnergyVadProcessor.Process(ReadOnlySpan<short>)` compares normalized RMS against
|
||||
`Threshold`, retains speech for `HangTime`, and starts closed. It uses monotonic
|
||||
`TimeProvider` timestamps; tests inject a clock. Threshold changes are atomic; all
|
||||
processing state otherwise has one owner. Codec/DSP processing methods allocate no
|
||||
managed memory after initialization, verified across 1,000 combined cycles. They run
|
||||
on a managed worker, never the native real-time device callback. Native device rings,
|
||||
jitter, mixer, and audio scheduling remain later work.
|
||||
|
||||
## Initial managed server
|
||||
|
||||
`VoiceServer(directory, endpoint, allowGuests, name)` owns credentials, the SQLite
|
||||
store, a TCP listener and its connection tasks. `EndPoint` reports the actual bound
|
||||
port (zero requests an ephemeral port). Dispose asynchronously to stop the listener
|
||||
and wait for all connections. The CLI currently binds loopback and accepts optional
|
||||
data-directory/port positional arguments.
|
||||
|
||||
All control traffic uses TLS 1.3 with the existing v2 protobuf. A single async loop
|
||||
owns each `TlsSession`; handlers exchange envelopes through bounded queues.
|
||||
This checkpoint caps connections at 64, queued input at 32 envelopes, queued output
|
||||
at 64 envelopes, and each control payload at 64 KiB (stricter than the shared framer's
|
||||
16 MiB limit). Queue exhaustion disconnects slow consumers. Handshake timeout is
|
||||
15 seconds; completed TLS connections have a 60-second receive-idle timeout.
|
||||
|
||||
Authentication starts users in unprotected Lobby (id 1), subject to its capacity.
|
||||
Success returns permissions, then a cloned snapshot; peers receive joined/updated/left
|
||||
events. Server-authoritative text replaces supplied sender ids/timestamps, limits
|
||||
bodies to 4096 UTF-8 bytes, and acknowledges valid or rejected routing. Channel text
|
||||
requires membership; private text echoes to sender and recipient. Protected channel
|
||||
joins and all admin/moderation handlers are pending. No UDP port or media features
|
||||
are advertised; voice subscription explicitly fails until the SFU is implemented.
|
||||
|
||||
`AccountStore(path)` retains the C++ schema version 2, accepts version 1 migration,
|
||||
and rejects unknown revisions. Opening an existing channel table does not reseed it.
|
||||
Account creation/authentication uses parameterized SQL; two password workers bound
|
||||
per-store Argon2 work. Failed authentication leaves `last_login` unchanged. Dispose
|
||||
after its operations finish. Account provisioning currently uses this API or the
|
||||
existing native administration path; there is no automatic bootstrap account.
|
||||
|
||||
`PasswordHasher` uses strict UTF-8 without normalization and libsodium-compatible
|
||||
Argon2id v19 PHC strings: 16-byte salt, 32-byte output, new-hash parameters
|
||||
64 MiB memory, two iterations, parallelism one. Verification supports up to 128 MiB,
|
||||
ten iterations, parallelism four and 1024 UTF-8 password bytes; malformed or excessive
|
||||
hashes fail closed. Standard C++ interactive-cost accounts are preserved. These
|
||||
bounds intentionally reject imported hashes above those costs. Native fixtures cover
|
||||
ASCII, Unicode and embedded NUL; the database oracle verifies cross-implementation
|
||||
authentication in both directions.
|
||||
|
||||
SQLite's MIT provider/bundle uses the pinned public-domain SourceGear SQLite build.
|
||||
The license audit checks that exact package version and repository identity because
|
||||
the native package lacks a NuGet license expression; other dependencies still require
|
||||
an approved permissive expression.
|
||||
|
||||
+3
-1
@@ -2,9 +2,11 @@
|
||||
|
||||
## .NET rewrite
|
||||
|
||||
The initial managed wire/crypto slice is under `dotnet/`, targeting .NET 10. From the root:
|
||||
The managed wire/crypto, TLS, and codec/DSP slices are under `dotnet/`, targeting .NET 10.
|
||||
From the root (CMake and a C compiler are required for codec/DSP):
|
||||
|
||||
```powershell
|
||||
./dotnet/build-native.ps1
|
||||
dotnet restore dotnet/VoiceCat.slnx --locked-mode
|
||||
dotnet build dotnet/VoiceCat.slnx -c Release --no-restore
|
||||
dotnet test dotnet/VoiceCat.slnx -c Release --no-build
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
# Porting VoiceCat to pure .NET / C#
|
||||
|
||||
**Status:** wire/media crypto and TLS/exporter foundations implemented under `dotnet/`,
|
||||
including C++ interoperability, persisted TOFU, and compatible server credentials.
|
||||
Codec/audio, managed server/client state, and UI phases remain planned.
|
||||
including C++ interoperability, persisted TOFU, compatible server credentials,
|
||||
codec/DSP wrappers, desktop native staging, and the initial managed TLS control server.
|
||||
Phase 4 remains in progress; UDP relay, complete session administration, device audio,
|
||||
managed client state, and UI phases remain planned.
|
||||
See `dotnet/README.md`, `docs/api-dotnet.md`, and `PROGRESS.md` for verification and next steps.
|
||||
**Target runtime:** .NET 10 LTS (in-service to Nov 2028), with .NET 11 as the follow-on.
|
||||
**Scope:** replace the C++ core (`libvoicecat`), the C++ server, the C++ `vccli`, and the
|
||||
@@ -233,7 +235,7 @@ has been carrying.
|
||||
| TLS 1.3 | mbedTLS | **BouncyCastle `Org.BouncyCastle.Tls`** | MIT | See §3. **Not `SslStream`.** |
|
||||
| Media AEAD | libsodium ChaCha20-Poly1305 | **`System.Security.Cryptography.ChaCha20Poly1305`** | built-in | ⚠️ Check `ChaCha20Poly1305.IsSupported` at startup — it is OS-backed (Windows 10 1903+ / OpenSSL 1.1+). Fall back to BouncyCastle's `ChaCha20Poly1305` if false. Same 12-byte nonce, 16-byte tag → identical wire bytes. |
|
||||
| Anti-replay window | hand-rolled 64-bit | port verbatim | — | ~40 lines. Keep the RFC 3711 §3.3 ordering (replay-check → authenticate → *then* advance). This ordering is load-bearing; `test_media_aead.cpp` covers it. |
|
||||
| Argon2id | libsodium `crypto_pwhash` | **`Konscious.Security.Cryptography.Argon2`** | MIT | Pure managed. ⚠️ **Existing password hashes will not verify** — libsodium emits `$argon2id$...` PHC strings with its own tuned m/t/p. Either implement a PHC-string parser and feed those params to Konscious (doable, recommended), or force a password reset on migration. Decide early; `db.cpp` migration depends on it. |
|
||||
| Argon2id | libsodium `crypto_pwhash` | **BouncyCastle `Argon2BytesGenerator`** | MIT | Implemented with a strict libsodium PHC parser and original costs; native database tests prove existing-account import and managed-account verification by C++. See `docs/api-dotnet.md` for cost bounds. |
|
||||
| BLAKE2b (channel passwords) | libsodium `crypto_generichash` | **`Blake2Fast`** (MIT) or BouncyCastle `Blake2bDigest` | MIT | Salted BLAKE2b-256, must produce identical digests to keep existing channel passwords working. Blake2Fast is SIMD and fast enough for the net thread, preserving the reason BLAKE2b was chosen over Argon2 here. |
|
||||
| Ed25519 identity | libsodium | **BouncyCastle `Ed25519Signer`** | MIT | ⚠️ **Not in .NET 10.** [dotnet/runtime#63174](https://github.com/dotnet/runtime/issues/63174) is api-approved but milestoned **11.0.0**. Since Option A already pulls in BouncyCastle, this is free. |
|
||||
| Self-signed cert gen | mbedTLS x509write | **`CertificateRequest.CreateSelfSigned`** | built-in | Much nicer than the C++ version. ECDSA-P256, same as today. Add the Ed25519 SAN (§3.4). |
|
||||
@@ -243,7 +245,7 @@ has been carrying.
|
||||
| Audio capture/playback | miniaudio | **P/Invoke miniaudio via a shim** | MIT-0/PD | **Keep native**, see §5.2. Managed alternatives exist ([SoundFlow](https://www.nuget.org/packages/SoundFlow), [MiniaudioSharp](https://www.nuget.org/packages/MiniaudioSharp), NAudio/CSCore for Windows-only) but auto-generated bindings marshal the callback into managed code, which is exactly what you must avoid (§5.1). Write the shim yourself. |
|
||||
| Energy VAD | hand-rolled | port verbatim | — | ~60 lines. Trivial. |
|
||||
| Protobuf | protobuf-lite (C++) | **`Google.Protobuf`** + `Grpc.Tools` | BSD | ⚠️ Reference `Grpc.Tools` for the `protoc` MSBuild integration even though there is no gRPC here — it is the standard way to codegen `.proto` in a `.csproj`. `<Protobuf Include="../../proto/voicecat.proto" GrpcServices="None" />`. The `.proto` needs **zero changes**. |
|
||||
| SQLite | sqlite3 | **`Microsoft.Data.Sqlite`** | MIT | Bundles SQLitePCLRaw; works with NativeAOT. Same schema, same file — an existing `voicecat.db` opens unchanged. |
|
||||
| SQLite | sqlite3 | **`Microsoft.Data.Sqlite.Core` + SQLitePCLRaw** | MIT / public domain | Implemented with provider 10.0.5, bundle 3.0.2 and pinned SQLite 3.50.4.2. Same schema/file; native database import is tested. NativeAOT publishing remains to be validated. |
|
||||
| Logging | spdlog | **`Microsoft.Extensions.Logging`** (+ Serilog console sink) | MIT/Apache | Use `LoggerMessage` source generators on any path near the hot loop. Never log from an audio path. |
|
||||
| Server config | `server.toml` | **`Tomlyn`** (MIT) or switch to JSON + `System.Text.Json` | MIT | Tomlyn keeps `server.toml` compatible; recommended, since operator-facing config shouldn't churn. |
|
||||
| CLI arg parsing | hand-rolled | **`System.CommandLine`** | MIT | For `VoiceCat.Cli` and the server. |
|
||||
@@ -356,11 +358,10 @@ internal static partial int opus_encode(IntPtr st, ReadOnlySpan<short> pcm, int
|
||||
Span<byte> data, int maxDataBytes);
|
||||
```
|
||||
|
||||
- `opus_encoder_ctl` is **varargs** — P/Invoke cannot do C varargs portably. Declare one
|
||||
overload per argument shape (`int`, `out int`) with `EntryPoint = "opus_encoder_ctl"`. This
|
||||
works on all the ABIs we target (x64 SysV, x64 Win, arm64 AAPCS) because all the CTLs we use
|
||||
take a single `int`/`int*`. **Note this explicitly in code comments** — it's a real
|
||||
portability caveat if a future CTL takes a different shape.
|
||||
- `opus_encoder_ctl` is **varargs** — P/Invoke cannot do C varargs portably. The implemented
|
||||
desktop binding uses fixed C entry points in `dotnet/native/media.c`; C calls the
|
||||
varargs function with the correct ABI. This also handles Apple arm64's different
|
||||
varargs calling convention. Only whitelisted single-int controls are accepted.
|
||||
- DRED (`opus_dred_alloc`, `opus_dred_parse`, `opus_decoder_dred_decode`) binds the same way.
|
||||
Guard with a runtime feature check as `opus_codec.cpp` does today.
|
||||
- **iOS requires static linking**: use `[LibraryImport("__Internal")]` and link
|
||||
@@ -700,6 +701,19 @@ interoperability test command.
|
||||
**Exit criterion:** encode→decode round-trip at every supported frame size; DRED recovery
|
||||
test green; RNNoise output matches the C++ within tolerance.
|
||||
|
||||
**Checkpoint (2026-09-15):** implemented `VoiceCat.Codec`, `VoiceCat.Dsp`, safe native
|
||||
handles, fixed-signature C bindings, and independent desktop native staging. The native
|
||||
build pins the upstream Opus 1.5.2 release/checksum (matching the actual vcpkg baseline,
|
||||
despite older code comments referring to 1.6), enables DRED, and shares the existing
|
||||
vendored RNNoise sources/model. Tests cover 40 rate/channel/frame-size round trips,
|
||||
PLC, 40 dropped-frame DRED recovery formats, C++ denoising within one PCM unit, VAD
|
||||
hang time, and zero managed allocations over 1,000 combined processing cycles.
|
||||
At 8/12 kHz, DRED tests encode at 16 kHz and decode at the requested rate: this pinned
|
||||
encoder's activity analysis cannot emit DRED at 8/12 kHz. These encoding configurations
|
||||
are explicitly rejected. Its redundancy floor is 30 ms because two chunks are required
|
||||
to emit DRED; actual redundancy remains adaptive. Desktop CI builds/stages the library
|
||||
before testing. iOS static native packaging and device audio remain later phases.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — Server (est. 3–4 weeks)
|
||||
@@ -707,6 +721,18 @@ test green; RNNoise output matches the C++ within tolerance.
|
||||
Do the server before the client: it lets you point the **existing, trusted C++ `vccli`** at
|
||||
it, which is a far better test client than a half-built C# one.
|
||||
|
||||
**Implemented checkpoint (2026-09-15):** bounded async TLS socket orchestration,
|
||||
guest/password authentication, existing SQLite account/channel import, snapshots,
|
||||
unprotected channel joins, channel/private/server text, ping and disconnect events.
|
||||
The existing C++ CLI authenticates and sends text through this server. Argon2id uses
|
||||
the existing BouncyCastle dependency with a strict libsodium PHC parser, not a new
|
||||
Konscious dependency. Native database tests prove password compatibility in both
|
||||
directions without resets. SQLite uses `Microsoft.Data.Sqlite.Core` 10.0.5,
|
||||
SQLitePCLRaw bundle 3.0.2 and explicitly pinned SourceGear SQLite 3.50.4.2.
|
||||
See `docs/api-dotnet.md` for limits. UDP voice, streams, protected joins, moderation,
|
||||
admin handlers and production configuration remain pending. The exit criterion
|
||||
below has not yet been met.
|
||||
|
||||
1. `VoiceCat.Server`: accept loop, `ConnSession` protocol handling, session registry.
|
||||
2. `Db` on `Microsoft.Data.Sqlite` — same schema. **Resolve the Argon2id hash-compat
|
||||
question here** (§4).
|
||||
|
||||
Reference in New Issue
Block a user