Port managed client audio and Windows application
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s

This commit is contained in:
2026-09-16 16:48:06 +02:00
parent 5a226ba543
commit 82ad4c2811
56 changed files with 2304 additions and 250 deletions
+3 -1
View File
@@ -39,7 +39,9 @@ dotnet test dotnet/VoiceCat.slnx -c Release --no-build
See `dotnet/README.md` for C# conventions and required native voice/CLI conformance, See `dotnet/README.md` for C# conventions and required native voice/CLI conformance,
and `docs/api-dotnet.md` for managed interfaces. Phase 4 remains in progress; and `docs/api-dotnet.md` for managed interfaces. Phase 4 remains in progress;
media-aware reaping and server administration are implemented. Server deployment hardening media-aware reaping and server administration are implemented. Server deployment hardening
and managed audio/client/Windows cutover are in progress; Apple GUI rewrites follow Windows. and the managed audio/client core plus Windows cutover are implemented. The Windows publish
contains only the permitted media shim. Accessibility/manual endurance validation and broad
server deployment work remain; Apple GUI rewrites follow Windows.
The default development preset is **`dev`** — it builds everything (server + tools + tests) The default development preset is **`dev`** — it builds everything (server + tools + tests)
with real vcpkg deps. The `skeleton` preset (no deps, stubs only) is a fast smoke check; see with real vcpkg deps. The `skeleton` preset (no deps, stubs only) is a fast smoke check; see
+21
View File
@@ -10,6 +10,27 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action ## ▶ Where we left off / next action
- **Done (2026-09-16): Managed client/audio and Windows cutover checkpoint.** Added
`VoiceCat.Core` with TOFU-gated TLS, correlated concurrent requests, immutable snapshots,
reconnects, bounded client events, UDP binding and authenticated encrypted media. Added
`VoiceCat.Audio` with allocation-free real-time cycles, bounded PCM/jitter queues,
5/10/20/40/60 ms Opus reframing, VAD/PTT, DTX, DRED → FEC → PLC, RNNoise, per-stream
controls and stereo mixing. Shared TLS/media primitives moved to `VoiceCat.Crypto`.
The WinForms app now references a managed compatibility facade over these libraries and
uses C# WASAPI capture, loopback and playback. Active streams renegotiate after channel
moves while capture-facing IDs stay stable. Per-app mixing no longer allocates or locks in
its real-time loop. The self-contained Windows publish includes only the permitted
`voicecat_media.dll`; a real default-device capture/playback plus main-form smoke passed.
Managed tests cover two-way decoded PCM voice, text, TOFU, concurrent requests, reconnects,
bounded jitter, zero-allocation cycles, recovery order and stream continuity across a
mono→stereo channel move. The old Interop project remains only as a migration oracle and
its nine native ABI tests remain green. **Verified:** 190 managed tests with every native
conformance/published-server variable enabled, nine Windows native-oracle tests, 29 CTest
tests, locked restores, both self-contained publishes and the permissive license audit.
**Next:** add the managed CLI and explicit managed
client ↔ C++ CLI conversation, then finish production deployment work. The Phase 5/7
manual ten-minute listen and NVDA gates are not yet signed off; Apple rewrites follow.
- **Done (2026-09-15): Server deployment hardening checkpoint.** Pushed existing work - **Done (2026-09-15): Server deployment hardening checkpoint.** Pushed existing work
through `653131b` to `origin/dotnet/foundations`. Added CLI/environment configuration, through `653131b` to `origin/dotnet/foundations`. Added CLI/environment configuration,
all-interface port 8384 defaults, config/fingerprint commands, local administrator all-interface port 8384 defaults, config/fingerprint commands, local administrator
+20 -72
View File
@@ -1,95 +1,43 @@
# VoiceCat — Windows client # VoiceCat — Windows client
WinForms (.NET 10 LTS) UI over `voicecat.dll` (MinGW-built `libvoicecat` shared library). The WinForms .NET 10 client uses `VoiceCat.Core` for TLS, TOFU, protocol state and encrypted UDP, and `VoiceCat.Audio` for streams, jitter, Opus and mixing. Its only native runtime component is `voicecat_media.dll`, the narrow Opus/RNNoise C shim. The old `voicecat.dll` core is no longer loaded or published.
## Prerequisites ## Build
| Tool | Version | Notes | Stage the pinned native media dependencies once, then build the solution:
|------|---------|-------|
| .NET SDK | 10.0.x | `dotnet --version` should report `10.0.*` |
| CMake | 3.25+ | For building the C++ DLL |
| MinGW-w64 / MSYS2 UCRT64 | GCC 13+ | `C:\tools\msys64\ucrt64` is the expected location |
| vcpkg | any | `VCPKG_ROOT` env var must point to a bootstrapped clone |
## Build order
### 1. Build the server (for testing)
```powershell ```powershell
cmake --preset dev ./dotnet/build-native.ps1
cmake --build --preset dev --target voicecat-server dotnet restore clients/windows/VoiceCat.slnx --locked-mode
dotnet build clients/windows/VoiceCat.slnx -c Release --no-restore
``` ```
### 2. Build the DLL The original `VoiceCat.Interop` project remains in the repository as a migration oracle. The app references `VoiceCat.Managed`, whose compatibility facade lets the existing accessible WinForms UI keep its event-pump shape while all networking and audio state live in the idiomatic managed libraries.
## Publish
```powershell ```powershell
cmake --preset windows-client ./clients/windows/publish-client.ps1
cmake --build --preset windows-client
``` ```
Output: `build/windows-client/bin/voicecat.dll` This produces a self-contained `win-x64` distribution in `dotnet/artifacts/client/win-x64`. The script requires `voicecat_media.dll` and fails if the legacy `voicecat.dll` appears.
**Verify no MinGW runtime dependencies remain:** The noninteractive startup and real WASAPI device check is:
```powershell
& "C:\tools\msys64\ucrt64\bin\objdump.exe" -p build/windows-client/bin/voicecat.dll |
Select-String "DLL Name"
```
Expected: only Windows system DLLs (`KERNEL32.dll`, `WS2_32.dll`, `BCRYPT.dll`, etc.).
If `libgcc_s_seh-1.dll`, `libstdc++-6.dll`, or `libwinpthread-1.dll` appear, the
`-static-libgcc -static-libstdc++ -static -lwinpthread` link flags in `core/CMakeLists.txt`
are not taking effect — check the CMake log for the `VOICECAT_BUILD_SHARED+WIN32` branch.
### 3. Build the C# solution
```powershell ```powershell
cd clients/windows ./dotnet/artifacts/client/win-x64/VoiceCat.App.exe --smoke-test --audio
dotnet build VoiceCat.slnx
``` ```
The app's `Directory.Build.props` copies `voicecat.dll` from `../../build/windows-client/bin/` `--smoke-test` constructs the real main form and pumps the managed client. `--audio` additionally opens the default WASAPI capture and render endpoints, moves PCM through both for three seconds, and fails if capture produces no samples.
into the output directory automatically on every build.
## Running manually ## Run manually
```powershell ```powershell
# Terminal 1 — start the server # Terminal 1
./build/dev/bin/voicecat-server.exe --name "My Server" ./dotnet/artifacts/server/win-x64/VoiceCat.Server.exe
# Terminal 2 — launch the client # Terminal 2
dotnet run --project clients/windows/VoiceCat.App/VoiceCat.App.csproj dotnet run --project clients/windows/VoiceCat.App/VoiceCat.App.csproj
``` ```
On first connect to a new server: Manual release validation still includes NVDA navigation and a ten-minute two-client listen test, as required by `docs/porting-to-dotnet.md`.
- Enter `127.0.0.1` as the host (not `localhost` — Windows resolves `localhost` to `::1`
first, and while the server now dual-stacks, `127.0.0.1` is cleaner for local testing).
- The server identity dialog will appear. The TLS leaf-cert SHA-256 fingerprint is shown;
accept to pin it. Subsequent connects to the same server will be silent (MATCHED).
## M5 — Moderation & admin UI
The WinForms client now exposes all M5 operations through the main menu and context menus:
- **Admin → Server accounts…** — create, reset password, and delete server accounts
(requires `can_admin_accounts`).
- **Channel tree right-click** — create, edit, and delete channels. The edit dialog exposes the
full per-channel Opus configuration: mono/stereo, sample rate, bitrate, frame size,
application mode, FEC, expected packet loss, DTX, and complexity.
- **User list right-click** — move, kick, ban, server mute/deafen, and set permissions
(items are gated by your own permissions).
- **Activity log** shows async `GenericResult` feedback for every moderation request.
- **User list** shows text indicators for self-mute, self-deafen, server-mute, and
server-deafen states.
These operations require an admin-provisioned account with the appropriate permissions; the
connect dialog already supports username/password auth.
## Known limitations
- **PTT is focus-scoped** — the push-to-talk key only works while the VoiceCat window has
focus. A system-wide `WH_KEYBOARD_LL` hook is not used in v1 (permissions + AV risk).
- **Receive-side noise reduction** checkbox in per-user tuning is wired end-to-end but is a
passthrough no-op until a real APM/NS backend is built (no working Windows/MSVC port of
`webrtc-audio-processing` upstream — see `docs/tech-stack.md §1`).
- **TOFU pins the TLS leaf cert**, not the declared Ed25519 identity fingerprint. Both are
shown in the identity dialog, but the cert fingerprint is the value that is actually
verified on reconnect. See `docs/security.md §1.1`.
@@ -16,7 +16,7 @@ public sealed record InputDeviceInfo(string Id, string Name, bool IsDefault)
/// the aux device is opened client-side and needs a WASAPI id, not a miniaudio one.</summary> /// the aux device is opened client-side and needs a WASAPI id, not a miniaudio one.</summary>
public static class InputDeviceEnumerator public static class InputDeviceEnumerator
{ {
public static IReadOnlyList<InputDeviceInfo> List() public static IReadOnlyList<InputDeviceInfo> List(bool input = true)
{ {
var result = new List<InputDeviceInfo>(); var result = new List<InputDeviceInfo>();
InputDeviceCapture.IMMDeviceEnumerator? enumerator = null; InputDeviceCapture.IMMDeviceEnumerator? enumerator = null;
@@ -29,7 +29,7 @@ public static class InputDeviceEnumerator
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!; Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
// Resolve the default capture endpoint id so the picker can flag it. // Resolve the default capture endpoint id so the picker can flag it.
if (enumerator.GetDefaultAudioEndpoint(1 /*eCapture*/, 0 /*eConsole*/, if (enumerator.GetDefaultAudioEndpoint(input ? 1 : 0, 0 /*eConsole*/,
out var defDev) == 0 && defDev != null) out var defDev) == 0 && defDev != null)
{ {
try { if (defDev.GetId(out string id) == 0) defaultId = id; } try { if (defDev.GetId(out string id) == 0) defaultId = id; }
@@ -37,7 +37,7 @@ public static class InputDeviceEnumerator
} }
// DEVICE_STATE_ACTIVE = 0x1 — only currently-usable endpoints. // DEVICE_STATE_ACTIVE = 0x1 — only currently-usable endpoints.
if (enumerator.EnumAudioEndpoints(1 /*eCapture*/, 0x1, out collectionPtr) != 0 if (enumerator.EnumAudioEndpoints(input ? 1 : 0, 0x1, out collectionPtr) != 0
|| collectionPtr == IntPtr.Zero) || collectionPtr == IntPtr.Zero)
return result; return result;
@@ -107,6 +107,7 @@ public sealed class InputDeviceCapture : IDisposable
private const int FrameSamples = 960; // 20 ms private const int FrameSamples = 960; // 20 ms
private readonly string? _deviceId; // null = system default capture endpoint private readonly string? _deviceId; // null = system default capture endpoint
private readonly bool _loopback;
private IAudioClient? _audioClient; private IAudioClient? _audioClient;
private IAudioCaptureClient? _captureClient; private IAudioCaptureClient? _captureClient;
@@ -123,8 +124,9 @@ public sealed class InputDeviceCapture : IDisposable
// Init-done signal: Set() by the capture thread after activation completes. // Init-done signal: Set() by the capture thread after activation completes.
private readonly ManualResetEventSlim _initDone = new(false); private readonly ManualResetEventSlim _initDone = new(false);
private bool _initOk; private bool _initOk;
private int _disposed;
public InputDeviceCapture(string? deviceId) => _deviceId = deviceId; public InputDeviceCapture(string? deviceId, bool loopback = false) { _deviceId = deviceId; _loopback = loopback; }
/// <summary>Starts capture. Blocks until WASAPI activation completes (typically &lt;100 ms). /// <summary>Starts capture. Blocks until WASAPI activation completes (typically &lt;100 ms).
/// Returns false if the device cannot be opened.</summary> /// Returns false if the device cannot be opened.</summary>
@@ -148,15 +150,13 @@ public sealed class InputDeviceCapture : IDisposable
{ {
_running = false; _running = false;
_bufferEvent?.Set(); _bufferEvent?.Set();
_captureThread?.Join(500); if (_captureThread is not null && !_captureThread.Join(5000)) throw new TimeoutException("Capture did not stop.");
try { _audioClient?.Stop(); } catch { /* device already gone */ }
} }
public void Dispose() public void Dispose()
{ {
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
Stop(); Stop();
if (_captureClient != null) { Marshal.ReleaseComObject(_captureClient); _captureClient = null; }
if (_audioClient != null) { Marshal.ReleaseComObject(_audioClient); _audioClient = null; }
_bufferEvent?.Dispose(); _bufferEvent?.Dispose();
_initDone.Dispose(); _initDone.Dispose();
} }
@@ -164,11 +164,20 @@ public sealed class InputDeviceCapture : IDisposable
// ── Capture thread (MTA) ────────────────────────────────────────────────── // ── Capture thread (MTA) ──────────────────────────────────────────────────
private void CaptureThreadProc() private void CaptureThreadProc()
{
try
{ {
_initOk = ActivateAndStart(); _initOk = ActivateAndStart();
_initDone.Set(); _initDone.Set();
if (!_initOk) return; if (_initOk && _running) CaptureLoop();
CaptureLoop(); }
catch { _initOk = false; _initDone.Set(); }
finally
{
try { _audioClient?.Stop(); } catch { }
if (_captureClient != null) { Marshal.ReleaseComObject(_captureClient); _captureClient = null; }
if (_audioClient != null) { Marshal.ReleaseComObject(_audioClient); _audioClient = null; }
}
} }
private bool ActivateAndStart() private bool ActivateAndStart()
@@ -192,7 +201,7 @@ public sealed class InputDeviceCapture : IDisposable
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!; Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
int hr = _deviceId is null int hr = _deviceId is null
? enumerator.GetDefaultAudioEndpoint(1 /*eCapture*/, 0 /*eConsole*/, out device) ? enumerator.GetDefaultAudioEndpoint(_loopback ? 0 : 1, 0 /*eConsole*/, out device)
: enumerator.GetDevice(_deviceId, out device); : enumerator.GetDevice(_deviceId, out device);
if (hr != 0 || device == null) return false; if (hr != 0 || device == null) return false;
@@ -220,7 +229,7 @@ public sealed class InputDeviceCapture : IDisposable
// AUDCLNT_STREAMFLAGS_EVENTCALLBACK = 0x00040000 // AUDCLNT_STREAMFLAGS_EVENTCALLBACK = 0x00040000
// AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM = 0x80000000 // AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM = 0x80000000
// AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY = 0x08000000 // AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY = 0x08000000
const uint streamFlags = 0x00040000u | 0x80000000u | 0x08000000u; uint streamFlags = 0x00040000u | 0x80000000u | 0x08000000u | (_loopback ? 0x00020000u : 0u);
foreach (int ch in new[] { 2, 1 }) foreach (int ch in new[] { 2, 1 })
{ {
@@ -327,9 +336,7 @@ public sealed class InputDeviceCapture : IDisposable
private void FlushFrame() private void FlushFrame()
{ {
var copy = new short[_accumBuf.Length]; PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels); // Borrowed until callback returns.
_accumBuf.AsSpan().CopyTo(copy);
PcmFrameReady?.Invoke(copy, FrameSamples, _channels);
_accumCount = 0; _accumCount = 0;
} }
@@ -1,148 +1,85 @@
using VoiceCat.Audio;
using VoiceCat.Interop; using VoiceCat.Interop;
// Owns N ProcessLoopbackCapture instances, mixes their PCM every 20 ms, and feeds
// the result to the core via vc_stream_feed_pcm. Used for per-app audio sharing.
namespace VoiceCat.App.Audio; namespace VoiceCat.App.Audio;
// Capture threads own their ring producers; the mix thread alone consumes them.
public sealed class ProcessAudioMixer : IDisposable public sealed class ProcessAudioMixer : IDisposable
{ {
private const int SampleRate = 48000; private sealed class Input
private const int FrameSamples = 960; {
private const int Channels = 2; // stereo; captures fall back to mono if needed internal readonly PcmRing Ring = new(16384);
private readonly short[] stereo = new short[1920];
private readonly List<ProcessLoopbackCapture> _captures = []; internal void Feed(short[] pcm, int channels)
{
// Per-capture latest frame, protected by _frameLock. if (channels == 2) { Ring.TryWrite(pcm); return; }
private readonly object _frameLock = new(); if (channels != 1 || pcm.Length > 960) return;
private List<short[]> _latestFrames = []; for (int i = 0; i < pcm.Length; i++) stereo[2 * i] = stereo[2 * i + 1] = pcm[i];
private int _activeChannels = Channels; Ring.TryWrite(stereo.AsSpan(0, pcm.Length * 2));
}
private Thread? _mixThread; }
private volatile bool _running; private readonly List<ProcessLoopbackCapture> captures = [];
private VoiceCatClient? _client; private Input[] inputs = [];
private uint _streamId; private Thread? thread;
private volatile bool running;
private VoiceCatClient? client;
private uint streamId;
public void Start(AppAudioScope scope, VoiceCatClient client, uint streamId) public void Start(AppAudioScope scope, VoiceCatClient client, uint streamId)
{ {
if (_running) return; if (running) return;
_client = client; this.client = client; this.streamId = streamId;
_streamId = streamId;
var specs = ResolveCaptures(scope); var specs = ResolveCaptures(scope);
if (specs.Count == 0) inputs = specs.Select(_ => new Input()).ToArray();
try
{ {
// nothing to capture — scope resolved to empty set
return;
}
lock (_frameLock)
{
_latestFrames = new List<short[]>(new short[specs.Count][]);
_activeChannels = Channels;
}
for (int i = 0; i < specs.Count; i++) for (int i = 0; i < specs.Count; i++)
{ {
int captureIndex = i; Input input = inputs[i]; var (pid, mode) = specs[i];
var (pid, mode) = specs[i]; var capture = new ProcessLoopbackCapture(pid, mode);
var cap = new ProcessLoopbackCapture(pid, mode); capture.PcmFrameReady += (pcm, _, channels) => input.Feed(pcm, channels);
cap.PcmFrameReady += (pcm, spc, ch) => OnCaptureFrame(captureIndex, pcm, ch); captures.Add(capture);
_captures.Add(cap); if (!capture.Start()) throw new InvalidOperationException("Process audio capture could not start.");
} }
if (inputs.Length == 0) return;
foreach (var c in _captures) c.Start(); running = true;
thread = new Thread(MixLoop) { IsBackground = true, Name = "ProcessAudioMixer" }; thread.Start();
_running = true; }
_mixThread = new Thread(MixLoop) { IsBackground = true, Name = "ProcessAudioMixer" }; catch { Stop(); throw; }
_mixThread.Start();
} }
public void Stop() public void Stop()
{ {
_running = false; running = false;
_mixThread?.Join(500); if (thread is not null && !thread.Join(5000)) throw new TimeoutException("Process audio mixer did not stop.");
foreach (var c in _captures) { c.Stop(); c.Dispose(); } foreach (var capture in captures) capture.Dispose();
_captures.Clear(); captures.Clear(); inputs = []; thread = null;
} }
public void Dispose() => Stop(); public void Dispose() => Stop();
// ── Capture callback ──────────────────────────────────────────────────────
private void OnCaptureFrame(int index, short[] pcm, int channels)
{
lock (_frameLock)
{
// Upmix mono → stereo interleave if the capture fell back to mono.
if (channels == 1 && _activeChannels == 2)
pcm = MonoToStereo(pcm);
if (index < _latestFrames.Count)
_latestFrames[index] = pcm;
}
}
// ── Mix loop (20 ms timer) ────────────────────────────────────────────────
private void MixLoop() private void MixLoop()
{ {
// Use a target period close to 20 ms; small under-shoot avoids accumulating drift. var frame = new short[1920]; var mix = new short[1920]; var sums = new int[1920];
const int periodMs = 19; long deadline = System.Diagnostics.Stopwatch.GetTimestamp();
while (_running) while (running)
{ {
Thread.Sleep(periodMs); sums.AsSpan().Clear();
if (!_running) break; foreach (Input input in inputs)
short[] mix;
lock (_frameLock)
{ {
int len = FrameSamples * _activeChannels; while (input.Ring.Count > 11520) input.Ring.Read(frame);
mix = new short[len]; frame.AsSpan().Clear(); input.Ring.Read(frame);
for (int i = 0; i < frame.Length; i++) sums[i] += frame[i];
foreach (var frame in _latestFrames) }
{ for (int i = 0; i < mix.Length; i++) mix[i] = (short)Math.Clamp(sums[i], short.MinValue, short.MaxValue);
if (frame == null) continue; client?.StreamFeedPcm(streamId, mix, 960, 2);
int frameLen = Math.Min(frame.Length, len); deadline += System.Diagnostics.Stopwatch.Frequency / 50;
for (int i = 0; i < frameLen; i++) double wait = (deadline - System.Diagnostics.Stopwatch.GetTimestamp()) * 1000.0 / System.Diagnostics.Stopwatch.Frequency;
{ if (wait > 0) Thread.Sleep((int)Math.Ceiling(wait));
int sum = mix[i] + frame[i]; else if (wait < -100) deadline = System.Diagnostics.Stopwatch.GetTimestamp();
mix[i] = (short)Math.Clamp(sum, short.MinValue, short.MaxValue);
} }
} }
} private static List<(int pid, ProcessLoopbackCapture.Mode mode)> ResolveCaptures(AppAudioScope scope) => scope switch
_client?.StreamFeedPcm(_streamId, mix, FrameSamples, (uint)_activeChannels);
}
}
// ── Helpers ───────────────────────────────────────────────────────────────
// Resolve the scope to the WASAPI captures to open:
// OnlyApps → one INCLUDE capture per selected process tree.
// AllExceptApps → one EXCLUDE capture of the single selected process tree, which
// natively captures the whole system render mix minus that tree
// (dynamic — apps launched later are included automatically).
private static List<(int pid, ProcessLoopbackCapture.Mode mode)> ResolveCaptures(AppAudioScope scope)
{
return scope switch
{ {
OnlyApps o => o.Pids.Select(p => (p, ProcessLoopbackCapture.Mode.Include)).ToList(), OnlyApps o => o.Pids.Select(p => (p, ProcessLoopbackCapture.Mode.Include)).ToList(),
AllExceptApps a when a.Pids.Count > 0 => AllExceptApps a when a.Pids.Count > 0 => [(a.Pids[0], ProcessLoopbackCapture.Mode.Exclude)],
[(a.Pids[0], ProcessLoopbackCapture.Mode.Exclude)], EntireDesktop { ExcludeSelf: true } => [(Environment.ProcessId, ProcessLoopbackCapture.Mode.Exclude)],
// Entire desktop minus VoiceCat itself: EXCLUDE our own process tree.
EntireDesktop { ExcludeSelf: true } =>
[(Environment.ProcessId, ProcessLoopbackCapture.Mode.Exclude)],
_ => [], _ => [],
}; };
} }
private static short[] MonoToStereo(short[] mono)
{
var stereo = new short[mono.Length * 2];
for (int i = 0; i < mono.Length; i++)
{
stereo[i * 2] = mono[i];
stereo[i * 2 + 1] = mono[i];
}
return stereo;
}
}
@@ -64,15 +64,15 @@ public sealed class ProcessLoopbackCapture : IDisposable
{ {
_running = false; _running = false;
_bufferEvent?.Set(); _bufferEvent?.Set();
_captureThread?.Join(500); if (_captureThread is not null && !_captureThread.Join(5000))
if (_audioClientPtr != IntPtr.Zero) AC_Stop(_audioClientPtr); throw new TimeoutException("Process capture did not stop.");
} }
private int _disposed;
public void Dispose() public void Dispose()
{ {
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
Stop(); Stop();
ComRelease(ref _captureClientPtr);
ComRelease(ref _audioClientPtr);
_bufferEvent?.Dispose(); _bufferEvent?.Dispose();
_initDone.Dispose(); _initDone.Dispose();
} }
@@ -80,11 +80,24 @@ public sealed class ProcessLoopbackCapture : IDisposable
// ── Capture thread (MTA) ────────────────────────────────────────────────── // ── Capture thread (MTA) ──────────────────────────────────────────────────
private void CaptureThreadProc() private void CaptureThreadProc()
{
try
{ {
_initOk = ActivateAndStart(); _initOk = ActivateAndStart();
_initDone.Set(); _initDone.Set();
if (!_initOk) return; if (_initOk && _running) CaptureLoop();
CaptureLoop(); }
catch
{
_initOk = false;
_initDone.Set();
}
finally
{
if (_audioClientPtr != IntPtr.Zero) AC_Stop(_audioClientPtr);
ComRelease(ref _captureClientPtr);
ComRelease(ref _audioClientPtr);
}
} }
private bool ActivateAndStart() private bool ActivateAndStart()
@@ -274,9 +287,9 @@ public sealed class ProcessLoopbackCapture : IDisposable
private void FlushFrame() private void FlushFrame()
{ {
var copy = new short[_accumBuf.Length]; // The callback borrows this buffer until it returns. This capture owner cannot
_accumBuf.AsSpan().CopyTo(copy); // overwrite it concurrently, which avoids one allocation every 20 ms.
PcmFrameReady?.Invoke(copy, FrameSamples, _channels); PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels);
_accumCount = 0; _accumCount = 0;
} }
@@ -0,0 +1,113 @@
using System.Runtime.InteropServices;
using VoiceCat.Audio;
using static VoiceCat.App.Audio.InputDeviceCapture;
namespace VoiceCat.App.Audio;
public sealed class WasapiAudioBackend : IAudioDeviceBackend
{
public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => InputDeviceEnumerator.List(input).Select(d => new AudioDeviceInfo(d.Id, d.Name, d.IsDefault)).ToArray();
public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler handler)
{
var capture = new InputDeviceCapture(NormalizeDeviceId(deviceId), loopback);
capture.PcmFrameReady += (pcm, _, channels) => handler(pcm, channels);
if (!capture.Start()) { capture.Dispose(); throw new InvalidOperationException("WASAPI capture could not start."); }
return new Capture(capture);
}
public IAudioPlayback OpenPlayback(string? deviceId = null) => new Playback(deviceId);
// The old miniaudio ABI persisted its raw device union as hex. Its Windows member
// is a null-terminated UTF-16 WASAPI endpoint id; preserve saved selections.
internal static string? NormalizeDeviceId(string? id)
{
if (string.IsNullOrEmpty(id)) return null;
if (id.StartsWith('{')) return id;
try
{
byte[] bytes = Convert.FromHexString(id);
string decoded = System.Text.Encoding.Unicode.GetString(bytes).Split('\0')[0];
return decoded.StartsWith('{') ? decoded : null;
}
catch (FormatException) { return id; }
}
private sealed class Capture(InputDeviceCapture capture) : IAudioCapture { public void Dispose() => capture.Dispose(); }
private sealed class Playback : IAudioPlayback
{
private readonly string? deviceId;
private readonly PcmRing pcm = new(32768);
private readonly ManualResetEventSlim initialized = new(false);
private readonly Thread thread;
private volatile bool running = true;
private bool ready;
private int disposed;
internal Playback(string? deviceId)
{
this.deviceId = deviceId;
thread = new Thread(Work) { IsBackground = true, Name = "VoiceCat WASAPI playback" };
thread.Start();
if (!initialized.Wait(5000) || !ready) { Dispose(); throw new InvalidOperationException("WASAPI playback could not start."); }
}
public void Write(ReadOnlySpan<short> stereoPcm) => pcm.TryWrite(stereoPcm);
private unsafe void Work()
{
IMMDeviceEnumerator? enumerator = null; IMMDevice? device = null; IAudioClient? client = null; IAudioRenderClient? render = null;
using var bufferReady = new AutoResetEvent(false);
try
{
enumerator = (IMMDeviceEnumerator)Activator.CreateInstance(Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
int result = deviceId is null ? enumerator.GetDefaultAudioEndpoint(0, 0, out device) : enumerator.GetDevice(deviceId, out device);
if (result < 0 || device is null) return;
Guid iid = typeof(IAudioClient).GUID;
if (device.Activate(ref iid, 0x17, 0, out object audio) < 0) return;
client = (IAudioClient)audio;
WaveFormat format = new() { Format = 1, Channels = 2, Samples = 48000, Bytes = 192000, Align = 4, Bits = 16 };
if (client.Initialize(0, 0x00040000u | 0x80000000u | 0x08000000u, 600000, 0, (nint)(&format), 0) < 0) return;
if (client.GetBufferSize(out uint capacity) < 0 || client.SetEventHandle(bufferReady.SafeWaitHandle.DangerousGetHandle()) < 0) return;
iid = typeof(IAudioRenderClient).GUID;
if (client.GetService(ref iid, out object output) < 0) return;
render = (IAudioRenderClient)output;
if (client.Start() < 0) return;
ready = true; initialized.Set();
Span<short> discard = stackalloc short[1920];
while (running)
{
bufferReady.WaitOne(20); // Scheduling wait is outside the buffer-fill cycle.
if (!running || client.GetCurrentPadding(out uint padding) < 0) break;
uint frames = capacity - Math.Min(capacity, padding);
if (frames == 0) continue;
if (render.GetBuffer(frames, out nint buffer) < 0) break;
var destination = new Span<short>((void*)buffer, checked((int)frames * 2));
// Keep queued playback bounded to ~120 ms, then fill underflow with silence.
while (pcm.Count > 11520) pcm.Read(discard);
int count = pcm.Read(destination); destination[count..].Clear();
if (render.ReleaseBuffer(frames, 0) < 0) break;
}
}
catch { ready = false; }
finally
{
initialized.Set();
try { client?.Stop(); } catch { }
if (render is not null) Marshal.ReleaseComObject(render);
if (client is not null) Marshal.ReleaseComObject(client);
if (device is not null) Marshal.ReleaseComObject(device);
if (enumerator is not null) Marshal.ReleaseComObject(enumerator);
}
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
running = false;
if (!thread.Join(5000)) throw new TimeoutException("WASAPI playback did not stop.");
initialized.Dispose();
}
}
[StructLayout(LayoutKind.Sequential, Pack = 2)]
private struct WaveFormat { internal ushort Format, Channels; internal uint Samples, Bytes; internal ushort Align, Bits, Extra; }
[ComImport, Guid("F294ACFC-3146-4483-A7BF-ADDCA7C260E2"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IAudioRenderClient
{
[PreserveSig] int GetBuffer(uint frames, out nint data);
[PreserveSig] int ReleaseBuffer(uint frames, uint flags);
}
}
@@ -119,7 +119,7 @@ public partial class ConnectDialog : Form
Directory.CreateDirectory(tofuDir); Directory.CreateDirectory(tofuDir);
_client = new VoiceCatClient("VoiceCat-Windows", VoiceCatClient.VersionString, _client = new VoiceCatClient("VoiceCat-Windows", VoiceCatClient.VersionString,
VcLogLevel.Info, ServerListStore.TofuStorePath); VcLogLevel.Info, ServerListStore.TofuStorePath, new VoiceCat.App.Audio.WasapiAudioBackend());
_client.EventReceived += OnEvent; _client.EventReceived += OnEvent;
_identityDialogShown = false; _identityDialogShown = false;
_pumpTimer.Start(); _pumpTimer.Start();
+27 -2
View File
@@ -5,7 +5,7 @@ namespace VoiceCat.App;
internal static class Program internal static class Program
{ {
[STAThread] [STAThread]
private static void Main() private static int Main(string[] args)
{ {
// Surface exceptions that WinForms' default message-loop handling would otherwise // Surface exceptions that WinForms' default message-loop handling would otherwise
// swallow silently (or crash with no visible cause). // swallow silently (or crash with no visible cause).
@@ -16,12 +16,37 @@ internal static class Program
ApplicationConfiguration.Initialize(); ApplicationConfiguration.Initialize();
if (args.Contains("--smoke-test"))
{
try
{
using var client = new VoiceCat.Interop.VoiceCatClient("Smoke", "test");
using var form = new MainForm(client, 0, "Smoke", "Managed core");
form.CreateControl();
if (form.Controls.Count == 0 || string.IsNullOrEmpty(form.Text)) return 1;
client.PumpEvents();
if (args.Contains("--audio"))
{
var backend = new Audio.WasapiAudioBackend();
using var playback = backend.OpenPlayback();
int samples = 0;
using var capture = backend.OpenCapture(null, false, (pcm, _) => Interlocked.Add(ref samples, pcm.Length));
short[] silence = new short[1920];
for (int i = 0; i < 150; i++) { playback.Write(silence); Thread.Sleep(20); }
if (samples == 0) return 2;
}
return 0;
}
catch { return 1; }
}
using var connectDialog = new ConnectDialog(); using var connectDialog = new ConnectDialog();
var result = connectDialog.ShowDialog(); var result = connectDialog.ShowDialog();
if (result != DialogResult.OK || connectDialog.ConnectedClient is null) if (result != DialogResult.OK || connectDialog.ConnectedClient is null)
return; return 0;
Application.Run(new MainForm(connectDialog.ConnectedClient, connectDialog.SelfUserId, Application.Run(new MainForm(connectDialog.ConnectedClient, connectDialog.SelfUserId,
connectDialog.Nickname, connectDialog.ServerName)); connectDialog.Nickname, connectDialog.ServerName));
return 0;
} }
} }
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\VoiceCat.Interop\VoiceCat.Interop.csproj" /> <ProjectReference Include="..\VoiceCat.Managed\VoiceCat.Managed.csproj" />
</ItemGroup> </ItemGroup>
<!-- Prismatoid — .NET bindings for the Prism speech library, used for spoken event <!-- Prismatoid — .NET bindings for the Prism speech library, used for spoken event
@@ -42,18 +42,6 @@
as part of the Windows Desktop shared framework on net10.0-windows — no PackageReference as part of the Windows Desktop shared framework on net10.0-windows — no PackageReference
needed (one was tried and NuGet flagged it as redundant/unprunable, NU1510). --> needed (one was tried and NuGet flagged it as redundant/unprunable, NU1510). -->
<!-- voicecat.dll must exist (build the `windows-client` CMake preset first — see
clients/windows/README.md). -->
<ItemGroup>
<Content Include="$(VoiceCatNativeDir)\voicecat.dll" Condition="Exists('$(VoiceCatNativeDir)\voicecat.dll')">
<Link>voicecat.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<Target Name="VoiceCatCheckNativeDll" BeforeTargets="Build">
<Error Condition="!Exists('$(VoiceCatNativeDir)\voicecat.dll')"
Text="voicecat.dll not found at '$(VoiceCatNativeDir)'. Build it first: cmake --preset windows-client &amp;&amp; cmake --build --preset windows-client (see clients/windows/README.md)." />
</Target>
</Project> </Project>
@@ -0,0 +1,77 @@
{
"version": 1,
"dependencies": {
"net10.0-windows7.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
},
"Prismatoid": {
"type": "Direct",
"requested": "[0.3.0, )",
"resolved": "0.3.0",
"contentHash": "mUOgtmFtjbLxydrdAImkfP3u0U8qrYkvfx2NlQqdQQ8TB9qe59BobxBCcNDo21g8xSefkBJB1VdGPbVTfiE7zw=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.managed": {
"type": "Project",
"dependencies": {
"VoiceCat.Core": "[1.0.0, )"
}
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0-windows7.0/win-x64": {
"Prismatoid": {
"type": "Direct",
"requested": "[0.3.0, )",
"resolved": "0.3.0",
"contentHash": "mUOgtmFtjbLxydrdAImkfP3u0U8qrYkvfx2NlQqdQQ8TB9qe59BobxBCcNDo21g8xSefkBJB1VdGPbVTfiE7zw=="
}
}
}
}
@@ -272,6 +272,8 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
Assert.Equal(VcResult.Ok, events.First(e => e.Type == VcEventType.AuthResult).Result); Assert.Equal(VcResult.Ok, events.First(e => e.Type == VcEventType.AuthResult).Result);
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ChannelList), 3000)); Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ChannelList), 3000));
Assert.Equal(VcResult.Ok, client.JoinVoice());
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
// Give the async UDP binding handshake a moment to land before announcing a stream // Give the async UDP binding handshake a moment to land before announcing a stream
// (mirrors vccli's 500ms sleep after auth). // (mirrors vccli's 500ms sleep after auth).
Thread.Sleep(500); Thread.Sleep(500);
@@ -340,6 +342,10 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
Assert.True(PumpUntil(b, () => eventsB.Any(e => e.Type == VcEventType.JoinResult), 5000), Assert.True(PumpUntil(b, () => eventsB.Any(e => e.Type == VcEventType.JoinResult), 5000),
"B did not receive VC_EVENT_JOIN_RESULT"); "B did not receive VC_EVENT_JOIN_RESULT");
Assert.Equal(VcResult.Ok, a.JoinVoice());
Assert.Equal(VcResult.Ok, b.JoinVoice());
Assert.True(PumpUntil(a, () => eventsA.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
Assert.True(PumpUntil(b, () => eventsB.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
// UDP binding handshake is async; give it a moment (mirrors ScreenAudio test). // UDP binding handshake is async; give it a moment (mirrors ScreenAudio test).
Thread.Sleep(500); Thread.Sleep(500);
@@ -9,5 +9,6 @@
<AllowUnsafeBlocks>true</AllowUnsafeBlocks> <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<RootNamespace>VoiceCat.Interop</RootNamespace> <RootNamespace>VoiceCat.Interop</RootNamespace>
</PropertyGroup> </PropertyGroup>
<ItemGroup><InternalsVisibleTo Include="VoiceCat.Interop.Tests" /></ItemGroup>
</Project> </Project>
@@ -0,0 +1,32 @@
using Voicecat.V1;
namespace VoiceCat.Interop;
public sealed partial class VoiceCatClient
{
public VcResult KickUser(uint id, string? reason = null) => Request(new() { Kick = new() { UserId = id, Reason = reason ?? "" } });
public VcResult BanUser(uint id, string? reason = null, ulong expiresUnixMs = 0) => Request(new() { Ban = new() { UserId = id, Reason = reason ?? "", ExpiresUnixMs = expiresUnixMs } });
public VcResult MoveUser(uint id, uint channelId) => Request(new() { MoveUser = new() { UserId = id, ChannelId = channelId } });
public VcResult SetServerMute(uint id, bool muted, bool deafened) => Request(new() { ServerMute = new() { UserId = id, Muted = muted, Deafened = deafened } });
public VcResult SetPermission(uint id, PermissionsInfo permissions) => Request(new() { SetPermission = new() { UserId = id, Permissions = new()
{ IsAdmin = permissions.IsAdmin, CanCreateTempChannel = permissions.CanCreateTempChannel, CanAdminAccounts = permissions.CanAdminAccounts, CanBan = permissions.CanBan, CanKick = permissions.CanKick, CanMoveUsers = permissions.CanMoveUsers } } });
public VcResult CreateAccount(string username, string password) => Request(new() { CreateAccount = new() { Username = username, Password = password } });
public VcResult ResetPassword(string username, string password) => Request(new() { ResetPassword = new() { Username = username, NewPassword = password } });
public VcResult DeleteAccount(string username) => Request(new() { DeleteAccount = new() { Username = username } });
public VcResult RequestAccountList() => Request(new() { ListAccounts = new() });
public VcResult CreateChannel(ChannelEditInfo info) => Request(new() { CreateChannel = new() { Channel = Channel(info), Password = info.Password ?? "" } });
public VcResult EditChannel(ChannelEditInfo info) => Request(new() { EditChannel = new() { Channel = Channel(info), Password = info.Password ?? "" } });
public VcResult DeleteChannel(uint id) => Request(new() { DeleteChannel = new() { ChannelId = id } });
public VcResult SendText(VcTextScope scope, uint targetId, string body)
{
try { core.Send(new() { TextMessage = new() { Scope = (TextScope)scope, TargetId = targetId, Body = body, ClientMsgId = Guid.NewGuid().ToString("N") } }); return VcResult.Ok; }
catch { return VcResult.NotConnected; }
}
private static Voicecat.V1.Channel Channel(ChannelEditInfo info) => new()
{
Id = info.Id, ParentId = info.ParentId, Name = info.Name, Topic = info.Topic, MaxUsers = info.MaxUsers, Order = unchecked((int)info.SortOrder),
Audio = new() { Codec = info.Audio.Codec, Mode = info.Audio.Stereo ? ChannelMode.ModeStereo : ChannelMode.ModeMono, SampleRate = info.Audio.SampleRate,
BitrateBps = info.Audio.BitrateBps, FrameMs = info.Audio.FrameMs, Application = (OpusApplication)info.Audio.Application, Complexity = info.Audio.Complexity,
Fec = info.Audio.Fec, ExpectedPacketLoss = info.Audio.ExpectedPacketLoss, Dtx = info.Audio.Dtx, Dred = info.Audio.Dred }
};
}
@@ -0,0 +1,106 @@
using VoiceCat.Audio;
using Voicecat.V1;
namespace VoiceCat.Interop;
public sealed partial class VoiceCatClient
{
private sealed record PcmRegistration(nint Callback, nint User);
private PcmRegistration? pcm;
public (VcResult Result, uint StreamId) StartStream(VcStreamKind kind, string label) => Start(kind, label, false);
public (VcResult Result, uint StreamId) StartStreamExternalFeed(VcStreamKind kind, string label) => Start(kind, label, true);
private (VcResult, uint) Start(VcStreamKind kind, string label, bool external)
{
StreamInfo? stream = null;
try
{
stream = core.StartStreamAsync((StreamKind)kind, label).GetAwaiter().GetResult();
local[stream.StreamId] = new(stream, external);
if (!external && backend is not null) captures[stream.StreamId] = Capture(stream);
return (VcResult.Ok, stream.StreamId);
}
catch (Exception exception)
{
if (stream is not null) { local.TryRemove(stream.StreamId, out _); core.StopStream(stream.StreamId); }
Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); return (VcResult.Audio, 0);
}
}
private IAudioCapture Capture(StreamInfo stream) => backend!.OpenCapture(devices.GetValueOrDefault(stream.StreamId), stream.Kind == StreamKind.StreamScreenAudio,
(samples, channels) => StreamFeedPcm(stream.StreamId, samples, samples.Length / channels, (uint)channels));
public VcResult StopStream(uint id)
{
if (captures.Remove(id, out var capture)) capture.Dispose();
if (!local.TryRemove(id, out LocalStream? stream)) return VcResult.InvalidArg;
try { core.StopStream(stream.Info.StreamId); return VcResult.Ok; } catch { return VcResult.NotConnected; }
}
public VcResult SetInputDevice(uint id, string? device)
{
devices[id] = device;
if (!captures.Remove(id, out var previous)) return VcResult.Ok;
previous.Dispose();
try { var info = local[id].Info.Clone(); info.StreamId = id; captures[id] = Capture(info); return VcResult.Ok; }
catch { return VcResult.Audio; }
}
public VcResult SetCaptureChannels(uint id, uint channels)
{ try { LocalStream stream = local[id]; core.Audio.SetCaptureChannels(stream.Info.StreamId, checked((int)channels)); stream.CaptureChannels = (int)channels; return VcResult.Ok; } catch { return VcResult.InvalidArg; } }
public VcResult AudioRestart()
{
foreach (uint id in captures.Keys.ToArray()) { VcResult result = SetInputDevice(id, devices.GetValueOrDefault(id)); if (result != VcResult.Ok) return result; }
return VcResult.Ok;
}
public VcResult SetInputMode(VcInputMode mode) { if (!Enum.IsDefined(mode)) return VcResult.InvalidArg; core.Audio.InputMode = (AudioInputMode)mode; return VcResult.Ok; }
public VcResult SetVadThreshold(float threshold) { if (!float.IsFinite(threshold) || threshold is < 0 or > 1) return VcResult.InvalidArg; core.Audio.VadThreshold = threshold; return VcResult.Ok; }
public VcResult SetPushToTalk(bool active) { core.Audio.PushToTalk = active; return VcResult.Ok; }
public VcResult SetSelfMute(bool muted, bool deafened) { core.Audio.MicMuted = muted; core.Audio.Deafened = deafened; return VcResult.Ok; }
public VcResult SetOutputVolume(float gain) { if (!float.IsFinite(gain) || gain is < 0 or > 4) return VcResult.InvalidArg; core.Audio.OutputGain = gain; return VcResult.Ok; }
public VcResult SetInputGain(float gain) { if (!float.IsFinite(gain) || gain is < 0 or > 4) return VcResult.InvalidArg; core.Audio.InputGain = gain; return VcResult.Ok; }
public VcResult SetInputNoiseReduction(bool enabled) { core.Audio.InputNoiseReduction = enabled; return VcResult.Ok; }
public VcResult SetRemoteStream(uint userId, uint streamId, float gain, bool muted, bool nr)
{ try { core.Audio.SetRemotePlayback(userId, streamId, gain, muted, nr); return VcResult.Ok; } catch { return VcResult.InvalidArg; } }
public (VcResult Result, RemoteStreamState? State) GetRemoteStream(uint userId, uint streamId)
{
var state = core.Audio.GetRemotePlayback(userId, streamId);
return state is { } value ? (VcResult.Ok, new(value.Gain, value.Muted, value.NoiseReduction)) : (VcResult.InvalidArg, null);
}
public (VcResult Result, AudioConfigInfo? Config) GetStreamAudioConfig(uint userId, uint streamId)
{
var info = core.Users.FirstOrDefault(u => u.Id == userId)?.Streams.FirstOrDefault(s => s.StreamId == streamId);
if (core.Authentication?.Self.Id == userId && local.TryGetValue(streamId, out LocalStream? own)) info = own.Info;
return info is null ? (VcResult.InvalidArg, null) : (VcResult.Ok, Audio(info.Audio));
}
public VcResult StreamFeedPcm(uint id, ReadOnlySpan<short> samples, int samplesPerChannel, uint channels)
{
if (samplesPerChannel < 0 || channels is not (1 or 2) || samples.Length != (long)samplesPerChannel * channels) return VcResult.InvalidArg;
if (!local.TryGetValue(id, out LocalStream? stream)) return VcResult.InvalidArg;
core.Audio.FeedPcm(Volatile.Read(ref stream.Info).StreamId, samples, (int)channels); return VcResult.Ok; // A full real-time ring drops, never waits.
}
public VcResult SetPcmSink(nint callback, nint user) { Volatile.Write(ref pcm, callback == 0 ? null : new(callback, user)); return VcResult.Ok; }
private unsafe void ForwardPcm(uint userId, uint streamId, ReadOnlySpan<short> samples, int channels)
{
var target = Volatile.Read(ref pcm); if (target is null) return;
fixed (short* input = samples)
((delegate* unmanaged[Cdecl]<nint, uint, uint, short*, nuint, uint, uint, void>)target.Callback)(target.User, userId, streamId, input, (nuint)(samples.Length / channels), (uint)channels, 48000);
}
// Server-authoritative moves/edits stop old SSRCs. Reannounce with the new channel
// configuration while keeping UI/external-feed ids stable; captures continue feeding
// the alias and switch to the new audio owner only when negotiation completes.
private void ReconcileStreams()
{
var active = core.LocalStreams;
foreach (LocalStream stream in local.Values)
if (!active.Any(s => s.StreamId == stream.Info.StreamId) && Interlocked.CompareExchange(ref stream.Restarting, 1, 0) == 0) _ = RestartAsync(stream);
}
private async Task RestartAsync(LocalStream stream)
{
try
{
StreamInfo previous = stream.Info;
StreamInfo next = await core.StartStreamAsync(previous.Kind, previous.Label, stream.CaptureChannels).ConfigureAwait(false);
if (!local.ContainsKey(stream.Alias)) core.StopStream(next.StreamId);
else Volatile.Write(ref stream.Info, next);
}
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); }
finally { Volatile.Write(ref stream.Restarting, 0); }
}
}
@@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" />
<Compile Include="../VoiceCat.Interop/Enums.cs" Link="Enums.cs" />
<Compile Include="../VoiceCat.Interop/Models.cs" Link="Models.cs" />
</ItemGroup>
</Project>
@@ -0,0 +1,193 @@
using System.Runtime.InteropServices;
using System.Threading.Channels;
using System.Collections.Concurrent;
using VoiceCat.Audio;
using VoiceCat.Core;
using VoiceCat.Crypto;
using Voicecat.V1;
using CoreClient = VoiceCat.Core.VoiceCatClient;
namespace VoiceCat.Interop;
// Preserves the shipped WinForms call/event surface while its implementation moves to
// the async managed core. Events still reach controls only through PumpEvents on the UI thread.
public sealed partial class VoiceCatClient : IDisposable
{
private readonly CoreClient core;
private readonly IAudioDeviceBackend? backend;
private readonly System.Threading.Channels.Channel<VoiceCatEvent> events = System.Threading.Channels.Channel.CreateUnbounded<VoiceCatEvent>();
private readonly Dictionary<uint, IAudioCapture> captures = [];
private readonly Dictionary<uint, string?> devices = [];
private readonly Dictionary<uint, StreamSummary[]> remoteStreams = [];
private readonly Dictionary<uint, bool> talkState = [];
private sealed class LocalStream(StreamInfo info, bool external)
{
internal readonly uint Alias = info.StreamId;
internal StreamInfo Info = info;
internal readonly bool External = external;
internal int CaptureChannels = 1;
internal int Restarting;
}
private readonly ConcurrentDictionary<uint, LocalStream> local = new();
private IAudioPlayback? playback;
private Task connecting = Task.CompletedTask;
private TaskCompletionSource<bool>? identity;
private int disposed;
private List<AccountInfo> accounts = [];
private bool audioFailureReported;
public event Action<VoiceCatEvent>? EventReceived;
public event Action<uint, float>? LevelChanged;
public CoreClient ManagedClient => core;
public VoiceCatClient(string clientName, string clientVersion, VcLogLevel logLevel = VcLogLevel.Info, string? tofuStorePath = null, IAudioDeviceBackend? audioBackend = null)
{
backend = audioBackend;
core = new(clientName, clientVersion, tofuStorePath);
core.ConnectionStateChanged += state =>
{
VcConnectionState mapped = state switch { ClientConnectionState.Connecting => VcConnectionState.Connecting, ClientConnectionState.VerifyingIdentity => VcConnectionState.VerifyingIdentity,
ClientConnectionState.Authenticating => VcConnectionState.Authenticating, ClientConnectionState.Connected => VcConnectionState.Connected, _ => VcConnectionState.Disconnected };
Queue(new(VcEventType.ConnectionState, ConnectionState: mapped));
if (mapped == VcConnectionState.Disconnected) Queue(new(VcEventType.Disconnected));
};
core.Audio.MixedPcm += pcm => Volatile.Read(ref playback)?.Write(pcm);
core.Audio.StreamPcm += ForwardPcm;
}
private void Queue(VoiceCatEvent message) => events.Writer.TryWrite(message);
public VcResult Connect(string host, ushort port)
{
if (!connecting.IsCompleted || core.State != ClientConnectionState.Disconnected) return VcResult.Already;
connecting = ConnectAsync(host, port);
return VcResult.Ok;
}
private async Task ConnectAsync(string host, ushort port)
{
try
{
await core.ConnectAsync(host, port, async (challenge, token) =>
{
identity = new(TaskCreationOptions.RunContinuationsAsynchronously);
Queue(new(VcEventType.ServerIdentity, U32a: (uint)challenge.Status, Text: challenge.CertificateFingerprint));
return await identity.Task.WaitAsync(token).ConfigureAwait(false);
}).ConfigureAwait(false);
}
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Io, Text: exception.Message)); }
}
public VcResult ConfirmServerIdentity(bool accept) { identity?.TrySetResult(accept); return VcResult.Ok; }
public VcResult AuthenticateGuest(string nickname) { _ = AuthenticateAsync(() => core.AuthenticateGuestAsync(nickname)); return VcResult.Ok; }
public VcResult AuthenticateUser(string username, string password) { _ = AuthenticateAsync(() => core.AuthenticateUserAsync(username, password)); return VcResult.Ok; }
private async Task AuthenticateAsync(Func<Task<AuthResult>> authenticate)
{
try { await connecting.ConfigureAwait(false); await authenticate().ConfigureAwait(false); }
catch (Exception exception) { Queue(new(VcEventType.AuthResult, Result: VcResult.AuthFailed, Text: exception.Message)); }
}
public VcResult Disconnect()
{
StopDevices(); core.DisconnectAsync().GetAwaiter().GetResult(); return VcResult.Ok;
}
public string GetServerIdentityDisplay() => core.ServerHello is { } hello ? Convert.ToHexString(hello.ServerIdentityFingerprint.Span) : "";
public void PumpEvents()
{
if (!audioFailureReported && core.Audio.Failure is { } failure) { audioFailureReported = true; Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: failure.Message)); }
while (core.TryReadEvent(out Envelope? message)) Translate(message!);
while (events.Reader.TryRead(out VoiceCatEvent? message)) EventReceived?.Invoke(message);
foreach (LocalStream stream in local.Values)
{
var info = Volatile.Read(ref stream.Info);
var level = core.Audio.GetLocalLevel(info.StreamId); LevelChanged?.Invoke(stream.Alias, level.Level);
if (talkState.GetValueOrDefault(stream.Alias) != level.Talking)
{
talkState[stream.Alias] = level.Talking;
if (core.State == ClientConnectionState.Connected) core.Send(new() { StreamState = new() { StreamId = info.StreamId, Talking = level.Talking, Muted = core.Audio.MicMuted } });
EventReceived?.Invoke(new(VcEventType.TalkState, UserId: core.Authentication?.Self.Id ?? 0, StreamId: stream.Alias, U32a: level.Talking ? 1U : 0));
}
}
}
private void Translate(Envelope message)
{
if (message.AuthResult is not null) Queue(new(VcEventType.AuthResult, Result: message.AuthResult.Ok ? VcResult.Ok : VcResult.AuthFailed, UserId: message.AuthResult.Self?.Id ?? 0, Text: message.AuthResult.Error));
if (message.ServerState is not null) { remoteStreams.Clear(); foreach (User user in message.ServerState.Users) UpdateStreams(user); Queue(new(VcEventType.ChannelList)); }
if (message.ChannelEvent is not null) Queue(new(VcEventType.ChannelList));
if (message.UserEvent is not null)
{
var change = message.UserEvent;
if (change.User is { VoiceSubscribed: true } self && self.Id == core.Authentication?.Self.Id) ReconcileStreams();
if (change.User is not null) UpdateStreams(change.User);
if (change.Kind == UserEvent.Types.Kind.Left) remoteStreams.Remove(change.LeftId);
Queue(new(change.Kind switch { UserEvent.Types.Kind.Joined => VcEventType.UserJoined, UserEvent.Types.Kind.Left => VcEventType.UserLeft, _ => VcEventType.UserUpdated }, UserId: change.User?.Id ?? change.LeftId,
ChannelId: change.User?.ChannelId ?? 0, Text: change.Kind == UserEvent.Types.Kind.Joined ? change.User?.Nickname : change.Reason));
}
if (message.JoinChannelResult is not null) Queue(new(VcEventType.JoinResult, Result: message.JoinChannelResult.Ok ? VcResult.Ok : VcResult.InvalidArg, ChannelId: message.JoinChannelResult.ChannelId, Text: message.JoinChannelResult.Error));
if (message.VoiceSubscriptionResult is not null) Queue(new(VcEventType.VoiceState, U32a: message.VoiceSubscriptionResult.Subscribed ? 1U : 0));
if (message.TextMessage is not null) Queue(new(VcEventType.TextMessage, UserId: message.TextMessage.SenderId, ChannelId: message.TextMessage.TargetId, TextScope: (VcTextScope)message.TextMessage.Scope, Text: message.TextMessage.Body, TimestampUnixMs: message.TextMessage.SentAtUnixMs));
if (message.StreamState is not null) Queue(new(VcEventType.TalkState, UserId: message.StreamState.UserId, StreamId: message.StreamState.UserId == core.Authentication?.Self.Id ? local.Values.FirstOrDefault(s => s.Info.StreamId == message.StreamState.StreamId)?.Alias ?? message.StreamState.StreamId : message.StreamState.StreamId, U32a: message.StreamState.Talking ? 1U : 0));
if (message.GenericResult is not null) Queue(new(VcEventType.GenericResult, Result: message.GenericResult.Ok ? VcResult.Ok : message.GenericResult.Code == 6 ? VcResult.PermissionDenied : VcResult.InvalidArg, U32a: message.GenericResult.Code, Text: message.GenericResult.Message));
if (message.ListAccountsResult is not null)
{
accounts = message.ListAccountsResult.Accounts.Select(a => new AccountInfo(a.Username, a.IsAdmin, a.CreatedAtUnixMs, a.LastLoginUnixMs)).ToList();
Queue(new(VcEventType.AccountList));
}
if (message.Disconnect is not null) Queue(new(VcEventType.Error, Result: VcResult.Io, Text: message.Disconnect.Reason));
}
private void UpdateStreams(User user)
{
StreamSummary[] previous = remoteStreams.GetValueOrDefault(user.Id, []);
StreamSummary[] next = user.Id == core.Authentication?.Self.Id ? ListUserStreams(user.Id).ToArray() : user.Streams.Select(s => new StreamSummary(s.StreamId, (VcStreamKind)s.Kind, s.Label)).ToArray();
foreach (var stream in previous) if (!next.Any(s => s.StreamId == stream.StreamId)) Queue(new(VcEventType.StreamStopped, UserId: user.Id, StreamId: stream.StreamId));
foreach (var stream in next) if (!previous.Any(s => s.StreamId == stream.StreamId)) Queue(new(VcEventType.StreamStarted, UserId: user.Id, StreamId: stream.StreamId, U32a: (uint)stream.Kind, Text: stream.Label));
remoteStreams[user.Id] = next;
}
private VcResult Request(Envelope request)
{
if (core.State != ClientConnectionState.Connected) return VcResult.NotConnected;
try { _ = RequestAsync(request); return VcResult.Ok; }
catch { return VcResult.NotConnected; }
}
private async Task RequestAsync(Envelope request)
{
try { await core.RequestAsync(request).ConfigureAwait(false); }
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Io, Text: exception.Message)); }
}
public VcResult JoinChannel(uint id, string? password = null) => Request(new() { JoinChannel = new() { ChannelId = id, Password = password ?? "" } });
public VcResult LeaveChannel() => Request(new() { LeaveChannel = new() });
public VcResult JoinVoice()
{
try
{
if (backend is not null && playback is null) playback = backend.OpenPlayback();
var result = core.SubscribeVoiceAsync().GetAwaiter().GetResult();
if (!result.Ok) { Interlocked.Exchange(ref playback, null)?.Dispose(); }
return result.Ok ? VcResult.Ok : VcResult.Audio;
}
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); return VcResult.Audio; }
}
public VcResult LeaveVoice() { StopDevices(); local.Clear(); return Request(new() { UnsubscribeVoice = new() }); }
public List<ChannelInfo> ListChannels() => core.Channels.Select(c => new ChannelInfo(c.Id, c.ParentId, c.Name, c.Topic, c.PasswordProtected, c.MaxUsers, unchecked((uint)c.Order), Audio(c.Audio))).ToList();
public List<UserInfo> ListUsers() => core.Users.Select(u => new UserInfo(u.Id, u.Nickname, u.IsGuest, u.ChannelId, u.SelfMicMuted, u.SelfDeafened, u.ServerMuted, u.ServerDeafened, u.VoiceSubscribed)).ToList();
public List<StreamSummary> ListUserStreams(uint id) => id == core.Authentication?.Self.Id
? local.Values.Select(s => new StreamSummary(s.Alias, (VcStreamKind)s.Info.Kind, s.Info.Label)).ToList()
: core.Users.FirstOrDefault(u => u.Id == id)?.Streams.Select(s => new StreamSummary(s.StreamId, (VcStreamKind)s.Kind, s.Label)).ToList() ?? [];
public PermissionsInfo GetPermissions() { Permissions p = core.Authentication?.Permissions ?? new(); return new(p.CanCreateTempChannel, p.CanKick, p.CanBan, p.CanMoveUsers, p.CanAdminAccounts, p.IsAdmin); }
public List<AccountInfo> ListAccounts() => accounts.ToList();
public List<DeviceInfo> ListDevices(VcDeviceKind kind) => backend?.Enumerate(kind == VcDeviceKind.Input).Select(d => new DeviceInfo(d.Id, d.Name, d.IsDefault)).ToList() ?? [];
public static string VersionString => "VoiceCat managed core 0.1.0 (protocol v2)";
public static string ResultString(VcResult result) => result.ToString();
private static AudioConfigInfo Audio(AudioConfig a) => new(a.Codec, a.Mode == ChannelMode.ModeStereo, a.SampleRate, a.BitrateBps, a.FrameMs, (uint)a.Application, a.Fec, a.ExpectedPacketLoss, a.Dtx, a.Complexity, a.Dred);
private void StopDevices()
{
foreach (var capture in captures.Values) capture.Dispose(); captures.Clear(); local.Clear();
IAudioPlayback? previous = Interlocked.Exchange(ref playback, null); previous?.Dispose();
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
identity?.TrySetResult(false); StopDevices(); core.DisposeAsync().AsTask().GetAwaiter().GetResult();
}
}
@@ -0,0 +1,7 @@
namespace VoiceCat.Interop;
// Compatibility event consumed by WinForms. The managed core owns protocol and audio;
// this assembly contains no libvoicecat bindings or native handles.
public sealed record VoiceCatEvent(VcEventType Type, VcConnectionState ConnectionState = VcConnectionState.Disconnected,
VcResult Result = VcResult.Ok, uint UserId = 0, uint ChannelId = 0, uint StreamId = 0, VcTextScope TextScope = VcTextScope.Channel,
uint U32a = 0, string? Text = null, ulong TimestampUnixMs = 0);
@@ -0,0 +1,51 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
}
}
}
@@ -0,0 +1,58 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0/win-x64": {}
}
}
+9
View File
@@ -1,5 +1,14 @@
<Solution> <Solution>
<Folder Name="/Managed core/">
<Project Path="../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" />
<Project Path="../../dotnet/src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
<Project Path="../../dotnet/src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
<Project Path="../../dotnet/src/VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
<Project Path="../../dotnet/src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
<Project Path="../../dotnet/src/VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
</Folder>
<Project Path="VoiceCat.App/VoiceCat.App.csproj" /> <Project Path="VoiceCat.App/VoiceCat.App.csproj" />
<Project Path="VoiceCat.Interop.Tests/VoiceCat.Interop.Tests.csproj" /> <Project Path="VoiceCat.Interop.Tests/VoiceCat.Interop.Tests.csproj" />
<Project Path="VoiceCat.Interop/VoiceCat.Interop.csproj" /> <Project Path="VoiceCat.Interop/VoiceCat.Interop.csproj" />
<Project Path="VoiceCat.Managed/VoiceCat.Managed.csproj" />
</Solution> </Solution>
+15
View File
@@ -0,0 +1,15 @@
param(
[string]$RuntimeIdentifier = "win-x64",
[string]$Output = ""
)
$ErrorActionPreference = "Stop"
$repoRoot = (Resolve-Path "$PSScriptRoot/../..").Path
if ([string]::IsNullOrWhiteSpace($Output)) { $Output = "$repoRoot/dotnet/artifacts/client/$RuntimeIdentifier" }
dotnet publish "$PSScriptRoot/VoiceCat.App/VoiceCat.App.csproj" -c Release -r $RuntimeIdentifier --self-contained true `
-p:PublishSingleFile=true -p:PublishTrimmed=false -p:IncludeNativeLibrariesForSelfExtract=false `
-p:RestorePackagesWithLockFile=true -p:RestoreLockedMode=true `
"-p:NuGetLockFilePath=packages.publish.$RuntimeIdentifier.lock.json" -o $Output
if ($LASTEXITCODE -ne 0) { throw "Managed Windows client publish failed." }
if (-not (Test-Path "$Output/voicecat_media.dll")) { throw "Managed media shim was not published." }
if (Test-Path "$Output/voicecat.dll") { throw "Legacy VoiceCat native core must not be published." }
Write-Host "Published managed Windows client to $Output"
+1 -1
View File
@@ -125,7 +125,7 @@ wrap Opus controls so P/Invoke never calls C varargs. SafeHandle owns every nati
encoder, decoder, DRED parser/state, and denoiser, including failed initialization. encoder, decoder, DRED parser/state, and denoiser, including failed initialization.
`OpusOptions` is an immutable record. Supported PCM rates are 8/12/16/24/48 kHz, `OpusOptions` is an immutable record. Supported PCM rates are 8/12/16/24/48 kHz,
one or two interleaved channels, and integral 10/20/40/60 ms frames. These match the one or two interleaved channels, and integral 5/10/20/40/60 ms frames. These match the
current VoiceCat protocol's integer frame duration; fractional Opus frame durations current VoiceCat protocol's integer frame duration; fractional Opus frame durations
are not exposed. Low-delay application mode requires at most 20 ms. Channel capture are not exposed. Low-delay application mode requires at most 20 ms. Channel capture
bandwidth is controlled separately by `MaximumBandwidthHz`; the production audio bandwidth is controlled separately by `MaximumBandwidthHz`; the production audio
+22
View File
@@ -794,6 +794,14 @@ criterion from `roadmap.md`, re-proven against the new server.
mix cycle; a manual listen test on Windows and macOS with no audible glitching over 10 mix cycle; a manual listen test on Windows and macOS with no audible glitching over 10
minutes. minutes.
**Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every
protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream
controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero
managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation
uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device
shim; the codec/DSP shim remains the only native component. A real device smoke passed, but
the required ten-minute Windows/macOS listen test is still a manual release gate.
--- ---
### Phase 6 — Client core (est. 34 weeks) ### Phase 6 — Client core (est. 34 weeks)
@@ -809,6 +817,12 @@ minutes.
conversation through the C# server**, and a C# `vccli` interoperates with a C++ `vccli` on conversation through the C# server**, and a C# `vccli` interoperates with a C++ `vccli` on
the same server. This is the full M0M3 criterion re-proven end to end. the same server. This is the full M0M3 criterion re-proven end to end.
**Checkpoint (2026-09-16):** `VoiceCat.Core` implements TOFU-gated TLS, concurrent correlated
requests, snapshots/events, reconnects, encrypted UDP binding and send/receive stream
lifecycle. Two managed clients exchange text and decoded PCM through the managed server.
The remaining Phase 6 item is the managed console client and its explicit C++ CLI
interoperability scenario.
--- ---
### Phase 7 — Windows client (est. 12 weeks) ### Phase 7 — Windows client (est. 12 weeks)
@@ -818,6 +832,14 @@ shape against a real, complete UI before you commit to two rewrites.
**Exit criterion:** feature parity with the current WinForms build, NVDA smoke-tested. **Exit criterion:** feature parity with the current WinForms build, NVDA smoke-tested.
**Checkpoint (2026-09-16):** the shipped WinForms project references `VoiceCat.Managed`, not
the P/Invoke core. The compatibility facade preserves UI-thread event pumping and stable
capture IDs while delegating all protocol and audio state to the idiomatic managed projects.
Channel moves automatically renegotiate active streams. A published build passed real WASAPI
capture/playback and form-startup smoke tests and contains `voicecat_media.dll` but no
`voicecat.dll`. Automated text, bidirectional PCM voice, multi-frame audio and stream-move
tests pass. NVDA and the manual endurance/listen pass remain before the Phase 7 exit criterion.
--- ---
### Phase 8 — macOS client (est. 45 weeks) ### Phase 8 — macOS client (est. 45 weeks)
+5
View File
@@ -5,8 +5,13 @@
<Project Path="src/VoiceCat.Codec/VoiceCat.Codec.csproj" /> <Project Path="src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
<Project Path="src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" /> <Project Path="src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
<Project Path="src/VoiceCat.Server/VoiceCat.Server.csproj" /> <Project Path="src/VoiceCat.Server/VoiceCat.Server.csproj" />
<Project Path="src/VoiceCat.Core/VoiceCat.Core.csproj" />
<Project Path="src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
</Folder> </Folder>
<Folder Name="/tests/"> <Folder Name="/tests/">
<Project Path="tests/VoiceCat.Tests/VoiceCat.Tests.csproj" /> <Project Path="tests/VoiceCat.Tests/VoiceCat.Tests.csproj" />
</Folder> </Folder>
<Folder Name="/clients/">
<Project Path="../clients/windows/VoiceCat.Managed/VoiceCat.Managed.csproj" />
</Folder>
</Solution> </Solution>
@@ -0,0 +1,12 @@
namespace VoiceCat.Audio;
public sealed record AudioDeviceInfo(string Id, string Name, bool IsDefault);
public delegate void CapturePcmHandler(ReadOnlySpan<short> pcm, int channels);
public interface IAudioCapture : IDisposable { }
public interface IAudioPlayback : IDisposable { void Write(ReadOnlySpan<short> stereoPcm); }
public interface IAudioDeviceBackend
{
IReadOnlyList<AudioDeviceInfo> Enumerate(bool input);
IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler pcm);
IAudioPlayback OpenPlayback(string? deviceId = null);
}
+170
View File
@@ -0,0 +1,170 @@
using System.Diagnostics;
using VoiceCat.Protocol;
using Voicecat.V1;
namespace VoiceCat.Audio;
public sealed class AudioEngine : IDisposable
{
private readonly object gate = new();
private readonly EncodedVoiceSender sender;
private readonly int[] mixed = new int[1920];
private readonly short[] output = new short[1920];
private Routes routes = new([], [], 0);
private readonly List<(IDisposable Stream, long Epoch)> retired = [];
private long completedEpoch;
private readonly CancellationTokenSource stop = new();
private readonly Task maintenance;
private Thread? worker;
private int disposed;
public uint SampleClock { get; private set; }
public event MixedPcmHandler? MixedPcm;
public event PcmStreamHandler? StreamPcm;
public volatile float InputGain = 1, OutputGain = 1, VadThreshold = 0.02f;
public volatile bool InputNoiseReduction, MicMuted, Deafened, PushToTalk;
public volatile AudioInputMode InputMode = AudioInputMode.VoiceActivation;
public Exception? Failure { get; private set; }
public AudioEngine(EncodedVoiceSender sender, bool startWorker = true)
{
this.sender = sender;
maintenance = MaintainAsync();
if (startWorker)
{
worker = new Thread(Work) { IsBackground = true, Name = "VoiceCat managed audio" };
worker.Start();
}
}
public void AddLocalStream(StreamInfo info, int captureChannels = 1)
{
if (captureChannels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(captureChannels));
lock (gate)
{
ObjectDisposedException.ThrowIf(disposed != 0, this);
var stream = new LocalStream(info, captureChannels);
Routes previous = routes;
var locals = previous.Local.Where(s => s.Info.StreamId != info.StreamId).Append(stream).ToArray();
Publish(locals, previous.Remote);
}
}
public void RemoveLocalStream(uint streamId)
{
lock (gate) Publish(routes.Local.Where(s => s.Info.StreamId != streamId).ToArray(), routes.Remote);
}
public void SetCaptureChannels(uint streamId, int channels)
{
if (channels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(channels));
lock (gate)
{
var current = routes.Local.FirstOrDefault(s => s.Info.StreamId == streamId) ?? throw new ArgumentException("Stream not found.");
if (current.CaptureChannels != channels) Publish(routes.Local.Select(s => s == current ? new LocalStream(s.Info, channels) : s).ToArray(), routes.Remote);
}
}
public void SetRemoteStreams(IReadOnlyList<User> users, uint selfId, uint channelId)
{
lock (gate)
{
var next = new List<ReceiveStream>();
foreach (User user in users.Where(u => u.Id != selfId && u.ChannelId == channelId))
foreach (StreamInfo info in user.Streams)
{
var previous = routes.Remote.FirstOrDefault(s => s.UserId == user.Id && s.Info.Equals(info));
next.Add(previous ?? new ReceiveStream(user.Id, info));
}
Publish(routes.Local, next.ToArray());
}
}
public bool FeedPcm(uint streamId, ReadOnlySpan<short> pcm, int channels)
{
foreach (LocalStream stream in Volatile.Read(ref routes).Local)
if (stream.Info.StreamId == streamId) return stream.Feed(pcm, channels);
return false;
}
public void Receive(VoiceFrameHeader header, ReadOnlySpan<byte> packet)
{
foreach (ReceiveStream stream in Volatile.Read(ref routes).Remote) if (stream.Info.Ssrc == header.Ssrc) { stream.Enqueue(header, packet); return; }
}
public (float Level, bool Talking) GetLocalLevel(uint streamId)
{
foreach (LocalStream stream in Volatile.Read(ref routes).Local) if (stream.Info.StreamId == streamId) return (stream.Level, stream.Talking);
return default;
}
public void SetRemotePlayback(uint userId, uint streamId, float gain, bool muted, bool noiseReduction)
{
if (!float.IsFinite(gain) || gain < 0 || gain > 4) throw new ArgumentOutOfRangeException(nameof(gain));
foreach (var stream in Volatile.Read(ref routes).Remote)
if (stream.UserId == userId && stream.Info.StreamId == streamId) { stream.Gain = gain; stream.Muted = muted; stream.NoiseReduction = noiseReduction; return; }
}
public (float Gain, bool Muted, bool NoiseReduction)? GetRemotePlayback(uint userId, uint streamId)
{
foreach (var stream in Volatile.Read(ref routes).Remote) if (stream.UserId == userId && stream.Info.StreamId == streamId) return (stream.Gain, stream.Muted, stream.NoiseReduction);
return null;
}
private void Publish(LocalStream[] local, ReceiveStream[] remote)
{
Routes previous = routes; var next = new Routes(local, remote, previous.Epoch + 1);
foreach (var stream in previous.Local) if (!local.Contains(stream)) retired.Add((stream, next.Epoch));
foreach (var stream in previous.Remote) if (!remote.Contains(stream)) retired.Add((stream, next.Epoch));
Volatile.Write(ref routes, next);
}
// One audio owner calls this. No allocation, waiting, lock, registry mutation or disposal
// occurs inside a mix cycle. Callbacks receive borrowed spans and must follow that rule.
internal void ProcessCycle()
{
Routes current = Volatile.Read(ref routes);
mixed.AsSpan().Clear();
foreach (var stream in current.Local) stream.Process(this, sender);
foreach (var stream in current.Remote) stream.Mix(mixed, Deafened, StreamPcm);
float gain = Deafened ? 0 : OutputGain;
for (int i = 0; i < output.Length; i++) output[i] = (short)Math.Clamp((int)(mixed[i] * gain), short.MinValue, short.MaxValue);
MixedPcm?.Invoke(output);
SampleClock = unchecked(SampleClock + 960);
Volatile.Write(ref completedEpoch, current.Epoch);
}
private void Work()
{
long deadline = Stopwatch.GetTimestamp();
try
{
while (!stop.IsCancellationRequested)
{
ProcessCycle();
deadline += Stopwatch.Frequency / 50;
double remaining = (deadline - Stopwatch.GetTimestamp()) * 1000.0 / Stopwatch.Frequency;
if (remaining > 0) Thread.Sleep((int)Math.Ceiling(remaining));
else if (remaining < -100) deadline = Stopwatch.GetTimestamp();
}
}
catch (Exception exception) { Failure = exception; stop.Cancel(); }
}
private async Task MaintainAsync()
{
try
{
using var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(20));
while (await timer.WaitForNextTickAsync(stop.Token).ConfigureAwait(false))
lock (gate)
for (int i = retired.Count - 1; i >= 0; i--)
if (retired[i].Epoch <= Volatile.Read(ref completedEpoch)) { retired[i].Stream.Dispose(); retired.RemoveAt(i); }
}
catch (OperationCanceledException) when (stop.IsCancellationRequested) { }
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
stop.Cancel(); worker?.Join(); maintenance.GetAwaiter().GetResult();
lock (gate)
{
foreach (var stream in routes.Local) stream.Dispose(); foreach (var stream in routes.Remote) stream.Dispose();
foreach (var stream in retired) stream.Stream.Dispose(); retired.Clear();
routes = new([], [], routes.Epoch + 1);
}
}
private sealed record Routes(LocalStream[] Local, ReceiveStream[] Remote, long Epoch);
}
+115
View File
@@ -0,0 +1,115 @@
using VoiceCat.Codec;
using VoiceCat.Dsp;
using VoiceCat.Protocol;
using Voicecat.V1;
using OpusApplication = Voicecat.V1.OpusApplication;
namespace VoiceCat.Audio;
public enum AudioInputMode { VoiceActivation, PushToTalk, AlwaysOn }
public delegate bool EncodedVoiceSender(uint ssrc, uint timestamp, ReadOnlySpan<byte> payload, VoiceFrameFlags flags);
public delegate void PcmStreamHandler(uint userId, uint streamId, ReadOnlySpan<short> pcm, int channels);
public delegate void MixedPcmHandler(ReadOnlySpan<short> stereoPcm);
internal sealed class LocalStream : IDisposable
{
internal readonly StreamInfo Info;
internal readonly int CaptureChannels;
internal readonly PcmRing Input = new(16384);
internal volatile float Level;
internal volatile bool Talking;
private readonly OpusEncoder encoder;
private readonly RnnoiseProcessor left, right;
private readonly EnergyVadProcessor vad = new();
private readonly short[] capture = new short[1920], wire = new short[5760 * 2], mono = new short[960];
private readonly byte[] packet = new byte[1275];
private readonly short[] converted = new short[16384];
private int feeding;
private int buffered;
private uint timestamp;
private bool wasTransmitting, marker;
internal bool Feed(ReadOnlySpan<short> pcm, int channels)
{
if (channels is not (1 or 2) || pcm.Length % channels != 0 || pcm.Length / channels * CaptureChannels > converted.Length || Interlocked.CompareExchange(ref feeding, 1, 0) != 0) return false;
try
{
if (channels == CaptureChannels) return Input.TryWrite(pcm);
int frames = pcm.Length / channels;
for (int i = 0; i < frames; i++)
if (CaptureChannels == 1) converted[i] = (short)(((int)pcm[i * 2] + pcm[i * 2 + 1]) / 2);
else { converted[2 * i] = pcm[i]; converted[2 * i + 1] = pcm[i]; }
return Input.TryWrite(converted.AsSpan(0, frames * CaptureChannels));
}
finally { Volatile.Write(ref feeding, 0); }
}
internal LocalStream(StreamInfo stream, int captureChannels)
{
Info = stream.Clone(); CaptureChannels = captureChannels;
encoder = new(new()
{
Channels = stream.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1,
FrameDurationMilliseconds = checked((int)stream.Audio.FrameMs), MaximumBandwidthHz = checked((int)stream.Audio.SampleRate),
Bitrate = checked((int)stream.Audio.BitrateBps), Complexity = checked((int)stream.Audio.Complexity),
ExpectedPacketLossPercent = checked((int)stream.Audio.ExpectedPacketLoss), ForwardErrorCorrection = stream.Audio.Fec,
DiscontinuousTransmission = stream.Audio.Dtx, DeepRedundancy = stream.Audio.Dred,
Application = stream.Audio.Application switch { OpusApplication.OpusAudio => VoiceCat.Codec.OpusApplication.Audio, OpusApplication.OpusLowdelay => VoiceCat.Codec.OpusApplication.LowDelay, _ => VoiceCat.Codec.OpusApplication.Voip }
});
try { left = new(); } catch { encoder.Dispose(); throw; }
try { right = new(); } catch { left.Dispose(); encoder.Dispose(); throw; }
}
internal void Process(AudioEngine engine, EncodedVoiceSender sender)
{
var input = capture.AsSpan(0, 960 * CaptureChannels);
if (Input.Count < input.Length) { Level = 0; Talking = false; buffered = 0; wasTransmitting = false; return; }
while (Input.Count > input.Length * 6) Input.Read(input);
if (buffered == 0) timestamp = engine.SampleClock;
Input.Read(input);
bool mic = Info.Kind == StreamKind.StreamMic;
if (mic && engine.InputNoiseReduction)
{
if (CaptureChannels == 1) left.Process(input);
else
{
for (int i = 0; i < 960; i++) mono[i] = input[2 * i]; left.Process(mono);
for (int i = 0; i < 960; i++) input[2 * i] = mono[i];
for (int i = 0; i < 960; i++) mono[i] = input[2 * i + 1]; right.Process(mono);
for (int i = 0; i < 960; i++) input[2 * i + 1] = mono[i];
}
}
float gain = mic ? engine.InputGain : 1;
double energy = 0;
for (int i = 0; i < input.Length; i++) { input[i] = (short)Math.Clamp((int)(input[i] * gain), short.MinValue, short.MaxValue); energy += (double)input[i] * input[i]; }
Level = (float)(Math.Sqrt(energy / input.Length) / 32768);
vad.Threshold = engine.VadThreshold;
bool transmit = !mic || !engine.MicMuted && engine.InputMode switch
{
AudioInputMode.AlwaysOn => true, AudioInputMode.PushToTalk => engine.PushToTalk,
_ => vad.Process(input)
};
Talking = transmit && Level > 0.001f;
if (!transmit) { buffered = 0; wasTransmitting = false; return; }
if (!wasTransmitting) marker = true;
wasTransmitting = true;
int channels = encoder.Options.Channels;
for (int i = 0; i < 960; i++)
{
if (channels == 1) wire[buffered + i] = CaptureChannels == 1 ? input[i] : (short)(((int)input[2 * i] + input[2 * i + 1]) / 2);
else { wire[buffered + 2 * i] = input[i * CaptureChannels]; wire[buffered + 2 * i + 1] = input[i * CaptureChannels + CaptureChannels - 1]; }
}
buffered += 960 * channels;
int frame = encoder.Options.SamplesPerChannel * channels;
while (buffered >= frame)
{
int length = encoder.Encode(wire.AsSpan(0, frame), packet);
VoiceFrameFlags flags = (Info.Audio.Fec ? VoiceFrameFlags.FecPresent : VoiceFrameFlags.None) | (marker ? VoiceFrameFlags.Marker : VoiceFrameFlags.None);
sender(Info.Ssrc, timestamp, packet.AsSpan(0, length), flags); marker = false;
timestamp = unchecked(timestamp + (uint)encoder.Options.SamplesPerChannel);
buffered -= frame;
wire.AsSpan(frame, buffered).CopyTo(wire);
}
}
public void Dispose() { encoder.Dispose(); left.Dispose(); right.Dispose(); }
}
+34
View File
@@ -0,0 +1,34 @@
namespace VoiceCat.Audio;
// Single consumer, non-waiting producer gate. Whole writes either fit or drop, so
// channels remain aligned and a capture thread never waits for a mixer/network owner.
public sealed class PcmRing
{
private readonly short[] samples;
private readonly int mask;
private int read, written, producer;
public PcmRing(int capacity = 32768)
{
if (capacity < 2 || (capacity & (capacity - 1)) != 0) throw new ArgumentOutOfRangeException(nameof(capacity));
samples = new short[capacity]; mask = capacity - 1;
}
public int Count => unchecked(Volatile.Read(ref written) - Volatile.Read(ref read));
public bool TryWrite(ReadOnlySpan<short> source)
{
if (Interlocked.CompareExchange(ref producer, 1, 0) != 0) return false;
try
{
int index = written;
if (source.Length > samples.Length - unchecked(index - Volatile.Read(ref read))) return false;
for (int i = 0; i < source.Length; i++) samples[(index + i) & mask] = source[i];
Volatile.Write(ref written, unchecked(index + source.Length)); return true;
}
finally { Volatile.Write(ref producer, 0); }
}
public int Read(Span<short> destination)
{
int index = read, count = Math.Min(destination.Length, unchecked(Volatile.Read(ref written) - index));
for (int i = 0; i < count; i++) destination[i] = samples[(index + i) & mask];
Volatile.Write(ref read, unchecked(index + count)); return count;
}
}
+163
View File
@@ -0,0 +1,163 @@
using VoiceCat.Codec;
using VoiceCat.Dsp;
using VoiceCat.Protocol;
using Voicecat.V1;
namespace VoiceCat.Audio;
internal sealed class ReceiveStream : IDisposable
{
internal readonly uint UserId;
internal readonly StreamInfo Info;
internal volatile float Gain = 1;
internal volatile bool Muted, NoiseReduction;
private readonly OpusDecoder decoder;
private readonly OpusDeepRedundancy? dred;
private readonly RnnoiseProcessor left, right;
private readonly short[] pcm = new short[5760 * 2];
private readonly short[] mono = new short[960];
private readonly short[] block = new short[1920];
private readonly byte[][] packets = Enumerable.Range(0, 64).Select(_ => new byte[1275]).ToArray();
private readonly VoiceFrameHeader[] headers = new VoiceFrameHeader[64];
private readonly int[] lengths = new int[64];
private int read, written;
private readonly byte[][] jitter = Enumerable.Range(0, 6).Select(_ => new byte[1275]).ToArray();
private readonly uint[] timestamps = new uint[6];
private readonly int[] sizes = new int[6];
private int count, available, offset, missing, waiting;
private uint expected;
private bool started, hasTimestamp;
private bool hasMarker;
private uint lastMarker;
private readonly int channels, frameSamples, maximumDepth;
internal int Depth => count;
internal int ConcealedFrames { get; private set; }
internal int DredFrames { get; private set; }
internal int FecFrames { get; private set; }
internal ReceiveStream(uint userId, StreamInfo info)
{
if (info.Audio.FrameMs is not (5 or 10 or 20 or 40 or 60) || !Enum.IsDefined(info.Audio.Mode)) throw new ArgumentException("Unsupported remote audio configuration.", nameof(info));
UserId = userId; Info = info.Clone();
channels = info.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1;
frameSamples = checked((int)info.Audio.FrameMs * 48);
maximumDepth = Math.Clamp(120 / (int)info.Audio.FrameMs, 2, 6);
decoder = new(48000, channels);
try { left = new(); } catch { decoder.Dispose(); throw; }
try { right = new(); } catch { left.Dispose(); decoder.Dispose(); throw; }
try { if (info.Audio.Dred) dred = new(); } catch { right.Dispose(); left.Dispose(); decoder.Dispose(); throw; }
}
// Only the network receive owner calls this; mixer alone consumes.
internal bool Enqueue(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
{
int index = written;
if (payload.Length is < 1 or > 1275 || unchecked(index - Volatile.Read(ref read)) >= 64) return false;
int slot = index & 63; payload.CopyTo(packets[slot]); headers[slot] = header; lengths[slot] = payload.Length;
Volatile.Write(ref written, unchecked(index + 1)); return true;
}
private void Drain()
{
while (read != Volatile.Read(ref written))
{
int source = read & 63; uint timestamp = headers[source].Timestamp;
if (!hasTimestamp) { hasTimestamp = true; expected = timestamp; }
int delta = unchecked((int)(timestamp - expected));
if ((headers[source].Flags & VoiceFrameFlags.Marker) != 0 && (!hasMarker || unchecked((int)(timestamp - lastMarker)) > 0))
{
hasMarker = true; lastMarker = timestamp;
sizes.AsSpan().Clear(); count = available = offset = missing = waiting = 0;
expected = timestamp; started = false; delta = 0;
}
bool duplicate = false;
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == timestamp) duplicate = true;
if ((!started || delta >= 0) && delta % frameSamples == 0 && !duplicate)
{
if (count >= maximumDepth)
{
int oldest = Oldest(); sizes[oldest] = 0; count--;
}
int target = Array.IndexOf(sizes, 0);
timestamps[target] = timestamp; sizes[target] = lengths[source]; packets[source].AsSpan(0, lengths[source]).CopyTo(jitter[target]); count++;
int oldestRemaining = Oldest();
if (count >= maximumDepth && unchecked((int)(timestamps[oldestRemaining] - expected)) > 0) expected = timestamps[oldestRemaining];
}
Volatile.Write(ref read, unchecked(read + 1));
}
}
private int Oldest()
{
int oldest = -1;
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && (oldest < 0 || unchecked((int)(timestamps[i] - timestamps[oldest])) < 0)) oldest = i;
return oldest;
}
private void Decode()
{
available = frameSamples; offset = 0;
int found = -1;
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == expected) { found = i; break; }
bool decoded = false;
if (found >= 0)
{
decoded = decoder.TryDecode(jitter[found].AsSpan(0, sizes[found]), pcm, frameSamples, out int result) && result == frameSamples;
sizes[found] = 0; count--; missing = decoded ? 0 : missing + 1;
}
else
{
int next = Oldest(); missing++;
if (next >= 0 && unchecked((int)(timestamps[next] - expected)) == frameSamples)
{
var packet = jitter[next].AsSpan(0, sizes[next]);
if (dred?.TryRecover(decoder, packet, pcm, frameSamples) == true) { decoded = true; DredFrames++; }
else if (Info.Audio.Fec && decoder.TryDecode(packet, pcm, frameSamples, out int recovered, true) && recovered == frameSamples) { decoded = true; FecFrames++; }
}
}
int maximumConcealment = Math.Max(1, 200 / (int)Info.Audio.FrameMs);
if (!decoded && missing <= maximumConcealment)
{
decoded = decoder.TryDecode([], pcm, frameSamples, out _); ConcealedFrames++;
}
if (!decoded || missing > maximumConcealment) pcm.AsSpan(0, frameSamples * channels).Clear();
expected = unchecked(expected + (uint)frameSamples);
}
internal void Mix(Span<int> output, bool deafened, PcmStreamHandler? sink)
{
Drain();
if (!started)
{
waiting++;
if (!hasTimestamp || count < Math.Min(3, maximumDepth) && waiting < 3) return;
expected = timestamps[Oldest()]; started = true;
}
int copied = 0;
while (copied < 960)
{
if (available == 0) Decode();
int take = Math.Min(960 - copied, available);
var decoded = pcm.AsSpan(offset * channels, take * channels);
decoded.CopyTo(block.AsSpan(copied * channels));
available -= take; offset += take; copied += take;
}
var samples = block.AsSpan(0, 960 * channels);
if (NoiseReduction && Info.Kind == StreamKind.StreamMic)
{
if (channels == 1) left.Process(samples);
else
{
for (int i = 0; i < 960; i++) mono[i] = samples[2 * i]; left.Process(mono);
for (int i = 0; i < 960; i++) samples[2 * i] = mono[i];
for (int i = 0; i < 960; i++) mono[i] = samples[2 * i + 1]; right.Process(mono);
for (int i = 0; i < 960; i++) samples[2 * i + 1] = mono[i];
}
}
float gain = Muted || deafened ? 0 : Gain;
for (int i = 0; i < samples.Length; i++) samples[i] = (short)Math.Clamp((int)(samples[i] * gain), short.MinValue, short.MaxValue);
sink?.Invoke(UserId, Info.StreamId, samples, channels);
for (int i = 0; i < 960; i++) { output[i * 2] += samples[i * channels]; output[i * 2 + 1] += samples[i * channels + channels - 1]; }
}
public void Dispose() { decoder.Dispose(); dred?.Dispose(); left.Dispose(); right.Dispose(); }
}
@@ -0,0 +1,8 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<ProjectReference Include="../VoiceCat.Codec/VoiceCat.Codec.csproj" />
<ProjectReference Include="../VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
<ProjectReference Include="../VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
<InternalsVisibleTo Include="VoiceCat.Tests" />
</ItemGroup>
</Project>
@@ -0,0 +1,24 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
}
}
}
@@ -0,0 +1,31 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0/win-x64": {}
}
}
+12
View File
@@ -41,5 +41,17 @@ public sealed class OpusDecoder : IDisposable
return OpusException.Check(NativeMethods.Decode(handle, input, packet.Length, output, samplesPerChannel, recoverPreviousFrame ? 1 : 0)); return OpusException.Check(NativeMethods.Decode(handle, input, packet.Length, output, samplesPerChannel, recoverPreviousFrame ? 1 : 0));
} }
public unsafe bool TryDecode(ReadOnlySpan<byte> packet, Span<short> pcm, int samplesPerChannel, out int decodedSamples, bool recoverPreviousFrame = false)
{
ValidateOutput(pcm, samplesPerChannel);
if (packet.Overlaps(MemoryMarshal.AsBytes(pcm))) throw new ArgumentException("Packet and PCM storage must not overlap.");
fixed (byte* input = packet)
fixed (short* output = pcm)
{
decodedSamples = NativeMethods.Decode(handle, input, packet.Length, output, samplesPerChannel, recoverPreviousFrame ? 1 : 0);
return decodedSamples >= 0;
}
}
public void Dispose() => handle.Dispose(); public void Dispose() => handle.Dispose();
} }
@@ -40,11 +40,10 @@ public sealed class OpusDeepRedundancy : IDisposable
fixed (byte* packet = nextPacket) fixed (byte* packet = nextPacket)
fixed (short* output = pcm) fixed (short* output = pcm)
{ {
int parsed = OpusException.Check(NativeMethods.DredParse(decoder, dred, packet, nextPacket.Length, int parsed = NativeMethods.DredParse(decoder, dred, packet, nextPacket.Length,
checked(samplesPerChannel + recoveryOffset), audioDecoder.SampleRate, out _)); checked(samplesPerChannel + recoveryOffset), audioDecoder.SampleRate, out _);
if (parsed == 0) return false; if (parsed <= 0) return false;
OpusException.Check(NativeMethods.DredDecode(audioDecoder.Handle, dred, recoveryOffset, output, samplesPerChannel)); return NativeMethods.DredDecode(audioDecoder.Handle, dred, recoveryOffset, output, samplesPerChannel) >= 0;
return true;
} }
} }
+1 -1
View File
@@ -21,7 +21,7 @@ public sealed record OpusOptions
{ {
if (SampleRate is not (8000 or 12000 or 16000 or 24000 or 48000)) throw new ArgumentOutOfRangeException(nameof(SampleRate)); if (SampleRate is not (8000 or 12000 or 16000 or 24000 or 48000)) throw new ArgumentOutOfRangeException(nameof(SampleRate));
if (Channels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(Channels)); if (Channels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(Channels));
if (FrameDurationMilliseconds is not (10 or 20 or 40 or 60)) throw new ArgumentOutOfRangeException(nameof(FrameDurationMilliseconds)); if (FrameDurationMilliseconds is not (5 or 10 or 20 or 40 or 60)) throw new ArgumentOutOfRangeException(nameof(FrameDurationMilliseconds));
if (Application == OpusApplication.LowDelay && FrameDurationMilliseconds > 20) throw new ArgumentException("Low-delay Opus requires frames of at most 20 ms."); if (Application == OpusApplication.LowDelay && FrameDurationMilliseconds > 20) throw new ArgumentException("Low-delay Opus requires frames of at most 20 ms.");
if (!Enum.IsDefined(Application)) throw new ArgumentOutOfRangeException(nameof(Application)); if (!Enum.IsDefined(Application)) throw new ArgumentOutOfRangeException(nameof(Application));
if (Bitrate is < 500 or > 512000) throw new ArgumentOutOfRangeException(nameof(Bitrate)); if (Bitrate is < 500 or > 512000) throw new ArgumentOutOfRangeException(nameof(Bitrate));
@@ -0,0 +1,14 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
}
},
"net10.0/win-x64": {}
}
}
@@ -0,0 +1,128 @@
using System.Net;
using System.Net.Sockets;
using VoiceCat.Protocol;
using VoiceCat.Crypto;
using VoiceCat.Transport;
namespace VoiceCat.Core;
public delegate void EncodedVoiceHandler(VoiceFrameHeader header, ReadOnlySpan<byte> payload);
internal sealed class ClientMediaTransport : IAsyncDisposable
{
private readonly Socket socket;
private readonly MediaSessionCrypto crypto;
private readonly CancellationTokenSource stop;
private readonly byte[] binding = new byte[VoiceFrameHeader.Size + 16];
private readonly byte[] keepalive = new byte[VoiceFrameHeader.Size];
private readonly PacketQueue packets = new();
private readonly Task sending;
private readonly Task receiving;
private readonly TaskCompletionSource bound = new(TaskCreationOptions.RunContinuationsAsynchronously);
internal event EncodedVoiceHandler? Received;
internal Task Bound => bound.Task;
internal ClientMediaTransport(IPEndPoint endpoint, ReadOnlySpan<byte> token, MediaSessionCrypto crypto, CancellationToken cancellationToken)
{
if (token.Length != 16) throw new IOException("Invalid UDP binding token.");
this.crypto = crypto;
stop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
socket = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
try { socket.Connect(endpoint); }
catch { socket.Dispose(); stop.Dispose(); throw; }
new VoiceFrameHeader(MediaFrameType.UdpBinding, 0, 0, 0, 0, 0).Write(binding);
token.CopyTo(binding.AsSpan(VoiceFrameHeader.Size));
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
receiving = ReceiveAsync();
sending = SendAsync();
}
internal bool TrySend(VoiceFrameHeader header, ReadOnlySpan<byte> payload) => packets.TryWrite(header, payload);
private async Task SendAsync()
{
byte[] plain = new byte[1275], packet = new byte[1275 + VoiceFrameHeader.Size + MediaEncryptor.TagSize];
long nextKeepalive = 0;
try
{
while (!stop.IsCancellationRequested)
{
if (Environment.TickCount64 >= nextKeepalive)
{
if (!bound.Task.IsCompleted) await socket.SendAsync(binding, SocketFlags.None, stop.Token).ConfigureAwait(false);
await socket.SendAsync(keepalive, SocketFlags.None, stop.Token).ConfigureAwait(false);
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? 5000 : 250);
}
while (packets.TryRead(plain, out VoiceFrameHeader header, out int length))
{
int size = crypto.Encryptor.Encrypt(header, plain.AsSpan(0, length), packet);
await socket.SendAsync(packet.AsMemory(0, size), SocketFlags.None, stop.Token).ConfigureAwait(false);
}
await Task.Delay(5, stop.Token).ConfigureAwait(false);
}
}
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException)
{ if (!stop.IsCancellationRequested) bound.TrySetException(exception); }
finally { stop.Cancel(); }
}
private async Task ReceiveAsync()
{
byte[] packet = new byte[65535], plain = new byte[65535];
try
{
while (true)
{
int length;
try { length = await socket.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
catch (SocketException exception) when (exception.SocketErrorCode is SocketError.ConnectionReset or SocketError.MessageSize) { continue; }
if (!VoiceFrameHeader.TryRead(packet.AsSpan(0, length), out var candidate)) continue;
if (candidate.Type == MediaFrameType.Keepalive && length == VoiceFrameHeader.Size) { bound.TrySetResult(); continue; }
if (candidate.Type != MediaFrameType.Voice || candidate.Codec != 0 ||
!crypto.Decryptor.TryDecrypt(packet.AsSpan(0, length), plain, out var header, out int size)) continue;
Received?.Invoke(header, plain.AsSpan(0, size));
}
}
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException)
{ if (!stop.IsCancellationRequested) bound.TrySetException(exception); }
finally { bound.TrySetCanceled(); stop.Cancel(); }
}
public async ValueTask DisposeAsync()
{
stop.Cancel(); socket.Dispose();
try { await Task.WhenAll(sending, receiving).ConfigureAwait(false); }
finally { System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding); stop.Dispose(); }
}
// A bounded, allocation-free packet handoff. A contending producer drops instead of
// waiting; the network owner alone consumes and encrypts. Audio never enters a Channel lock.
private sealed class PacketQueue
{
private readonly byte[][] payloads = Enumerable.Range(0, 64).Select(_ => new byte[1275]).ToArray();
private readonly VoiceFrameHeader[] headers = new VoiceFrameHeader[64];
private readonly int[] lengths = new int[64];
private int read, written, producer;
internal bool TryWrite(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
{
if (payload.Length is < 1 or > 1275 || Interlocked.CompareExchange(ref producer, 1, 0) != 0) return false;
try
{
int index = written;
if (unchecked(index - Volatile.Read(ref read)) >= 64) return false;
int slot = index & 63;
payload.CopyTo(payloads[slot]); headers[slot] = header; lengths[slot] = payload.Length;
Volatile.Write(ref written, unchecked(index + 1)); return true;
}
finally { Volatile.Write(ref producer, 0); }
}
internal bool TryRead(Span<byte> payload, out VoiceFrameHeader header, out int length)
{
int index = read; header = default; length = 0;
if (index == Volatile.Read(ref written)) return false;
int slot = index & 63; header = headers[slot]; length = lengths[slot];
payloads[slot].AsSpan(0, length).CopyTo(payload);
Volatile.Write(ref read, unchecked(index + 1)); return true;
}
}
}
@@ -0,0 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<ProjectReference Include="../VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
<ProjectReference Include="../VoiceCat.Audio/VoiceCat.Audio.csproj" />
<InternalsVisibleTo Include="VoiceCat.Tests" />
</ItemGroup>
</Project>
+292
View File
@@ -0,0 +1,292 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using System.Threading.Channels;
using VoiceCat.Crypto;
using VoiceCat.Transport;
using VoiceCat.Protocol;
using VoiceCat.Audio;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.Core;
public enum ClientConnectionState { Disconnected, Connecting, VerifyingIdentity, Authenticating, Connected }
public sealed record ServerIdentityChallenge(string Host, ushort Port, string CertificateFingerprint, TofuStatus Status);
public sealed partial class VoiceCatClient : IAsyncDisposable
{
private readonly string clientName;
private readonly string clientVersion;
private readonly TofuStore pins;
private readonly SemaphoreSlim lifecycle = new(1);
private readonly CancellationTokenSource disposed = new();
private readonly object stateGate = new();
private readonly ConcurrentDictionary<ulong, TaskCompletionSource<Envelope>> pending = new();
private readonly System.Threading.Channels.Channel<Envelope> events = System.Threading.Channels.Channel.CreateBounded<Envelope>(128);
private readonly Dictionary<uint, Channel> channels = [];
private readonly Dictionary<uint, User> users = [];
private readonly Dictionary<uint, StreamInfo> localStreams = [];
public AudioEngine Audio { get; }
public IReadOnlyList<StreamInfo> LocalStreams { get { lock (stateGate) return localStreams.Values.Select(s => s.Clone()).ToArray(); } }
private TlsControlConnection? control;
private MediaSessionCrypto? mediaCrypto;
private ClientMediaTransport? media;
private Task keepalive = Task.CompletedTask;
public event EncodedVoiceHandler? VoiceReceived;
private CancellationTokenSource? connectionLifetime;
private Task reader = Task.CompletedTask;
private long nextRequest;
private AuthResult? authentication;
private ServerHello? hello;
private ClientConnectionState state;
public event Action<ClientConnectionState>? ConnectionStateChanged;
public ClientConnectionState State { get { lock (stateGate) return state; } }
public Task Completion => reader;
public AuthResult? Authentication { get { lock (stateGate) return authentication?.Clone(); } }
public ServerHello? ServerHello { get { lock (stateGate) return hello?.Clone(); } }
public IReadOnlyList<Channel> Channels { get { lock (stateGate) return channels.Values.Select(c => c.Clone()).ToArray(); } }
public IReadOnlyList<User> Users { get { lock (stateGate) return users.Values.Select(u => u.Clone()).ToArray(); } }
public bool TryReadEvent(out Envelope? envelope) => events.Reader.TryRead(out envelope);
public IAsyncEnumerable<Envelope> ReadEventsAsync(CancellationToken cancellationToken = default) => events.Reader.ReadAllAsync(cancellationToken);
public VoiceCatClient(string clientName = "VoiceCat .NET", string clientVersion = "0.1.0", string? tofuStorePath = null)
{
this.clientName = clientName;
this.clientVersion = clientVersion;
pins = new(tofuStorePath ?? Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "VoiceCat", "tofu.txt"));
Audio = new(TrySendEncodedVoice);
VoiceReceived += Audio.Receive;
}
public async Task ConnectAsync(string host, ushort port, Func<ServerIdentityChallenge, CancellationToken, ValueTask<bool>>? confirmIdentity = null, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(host);
ArgumentOutOfRangeException.ThrowIfZero(port);
await lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false);
Socket? socket = null;
bool started = false;
try
{
ObjectDisposedException.ThrowIf(disposed.IsCancellationRequested, this);
if (control is not null) throw new InvalidOperationException("Disconnect before reconnecting.");
started = true;
connectionLifetime = CancellationTokenSource.CreateLinkedTokenSource(disposed.Token);
using var connecting = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, connectionLifetime.Token);
CancellationToken token = connecting.Token;
SetState(ClientConnectionState.Connecting);
socket = new(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
await socket.ConnectAsync(host, port, token).ConfigureAwait(false);
string? fingerprint = null;
control = new(socket, TlsSession.CreateClient(value => { fingerprint = value; return true; }), connectionLifetime.Token);
socket = null; // Transport owns it from here.
mediaCrypto = await control.TakeMediaCryptoAsync(token).ConfigureAwait(false);
string certificatePin = fingerprint ?? throw new IOException("TLS did not report a certificate fingerprint.");
TofuStatus pinStatus = pins.Check(host, port, certificatePin);
if (pinStatus != TofuStatus.Matched)
{
SetState(ClientConnectionState.VerifyingIdentity);
if (confirmIdentity is null || !await confirmIdentity(new(host, port, certificatePin, pinStatus), token).ConfigureAwait(false))
throw new System.Security.Authentication.AuthenticationException("Server identity was rejected.");
pins.Pin(host, port, certificatePin);
}
reader = ReadAsync(control, connectionLifetime.Token);
Envelope response = await RequestAsync(new() { ClientHello = new() { ProtoVersion = 2, ClientName = clientName, ClientVersion = clientVersion } }, token).ConfigureAwait(false);
if (response.ServerHello?.ProtoVersion != 2) throw new IOException("Unsupported server protocol.");
lock (stateGate) hello = response.ServerHello.Clone();
keepalive = KeepaliveAsync(connectionLifetime.Token);
SetState(ClientConnectionState.Authenticating);
}
catch
{
socket?.Dispose();
if (started) await CloseAsync().ConfigureAwait(false);
throw;
}
finally { lifecycle.Release(); }
}
public Task<AuthResult> AuthenticateGuestAsync(string nickname, CancellationToken cancellationToken = default) =>
AuthenticateAsync(new() { Guest = new() { Nickname = nickname } }, cancellationToken);
public Task<AuthResult> AuthenticateUserAsync(string username, string password, CancellationToken cancellationToken = default) =>
AuthenticateAsync(new() { Password = new() { Username = username, Password = password } }, cancellationToken);
private async Task<AuthResult> AuthenticateAsync(AuthRequest request, CancellationToken cancellationToken)
{
if (State != ClientConnectionState.Authenticating) throw new InvalidOperationException("Authentication requires a connected TLS session.");
Envelope response = await RequestAsync(new() { AuthRequest = request }, cancellationToken).ConfigureAwait(false);
AuthResult result = response.AuthResult ?? throw new IOException("Unexpected authentication response.");
if (result.Ok)
{
var endpoint = (IPEndPoint)control!.RemoteEndPoint;
IPAddress address = endpoint.Address.IsIPv4MappedToIPv6 ? endpoint.Address.MapToIPv4() : endpoint.Address;
media = new(new(address, checked((int)ServerHello!.UdpPort)), result.UdpToken.Span, mediaCrypto!, connectionLifetime!.Token);
media.Received += (header, packet) => VoiceReceived?.Invoke(header, packet);
SetState(ClientConnectionState.Connected);
}
return result;
}
public async Task<VoiceSubscriptionResult> SubscribeVoiceAsync(bool subscribe = true, CancellationToken cancellationToken = default)
{
if (subscribe && media is not null) await media.Bound.WaitAsync(TimeSpan.FromSeconds(5), cancellationToken).ConfigureAwait(false);
return (await RequestAsync(subscribe ? new() { SubscribeVoice = new() } : new() { UnsubscribeVoice = new() }, cancellationToken).ConfigureAwait(false)).VoiceSubscriptionResult;
}
public bool TrySendEncodedVoice(uint ssrc, uint timestamp, ReadOnlySpan<byte> payload, VoiceFrameFlags flags = VoiceFrameFlags.None) =>
media?.TrySend(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload) == true;
public async Task<StreamInfo> StartStreamAsync(StreamKind kind, string label = "", int captureChannels = 1, CancellationToken cancellationToken = default)
{
if (State != ClientConnectionState.Connected) throw new InvalidOperationException("Client is disconnected.");
var response = (await RequestAsync(new() { StreamAnnounce = new() { Kind = kind, Label = label } }, cancellationToken).ConfigureAwait(false)).StreamAnnounceResult;
if (!response.Ok) throw new InvalidOperationException(response.Error);
var info = new StreamInfo { StreamId = response.StreamId, Ssrc = response.Ssrc, Kind = kind, Audio = response.EffectiveAudio.Clone(), Label = label };
try
{
lock (stateGate) { if (State != ClientConnectionState.Connected) throw new InvalidOperationException("Client disconnected during stream negotiation."); Audio.AddLocalStream(info, captureChannels); localStreams[info.StreamId] = info; }
return info.Clone();
}
catch { if (State == ClientConnectionState.Connected) Send(new() { StreamStop = new() { StreamId = info.StreamId } }); throw; }
}
public void StopStream(uint streamId)
{
lock (stateGate) { localStreams.Remove(streamId); Audio.RemoveLocalStream(streamId); }
Send(new() { StreamStop = new() { StreamId = streamId } });
}
private async Task KeepaliveAsync(CancellationToken cancellationToken)
{
try
{
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(10));
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false)) Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
}
catch (Exception exception) when (exception is OperationCanceledException or IOException or InvalidOperationException) { }
}
public async Task<Envelope> RequestAsync(Envelope request, CancellationToken cancellationToken = default)
{
TlsControlConnection connection = control ?? throw new InvalidOperationException("Client is disconnected.");
var completion = new TaskCompletionSource<Envelope>(TaskCreationOptions.RunContinuationsAsynchronously);
ulong id = checked((ulong)Interlocked.Increment(ref nextRequest));
Envelope outbound = request.Clone(); outbound.RequestId = id;
if (!pending.TryAdd(id, completion)) throw new InvalidOperationException("Request ids exhausted.");
try
{
if (!connection.TrySend(outbound)) throw new IOException("Control queue is full or closed.");
return await completion.Task.WaitAsync(TimeSpan.FromSeconds(15), cancellationToken).ConfigureAwait(false);
}
finally { pending.TryRemove(id, out _); }
}
public void Send(Envelope message)
{
TlsControlConnection connection = control ?? throw new InvalidOperationException("Client is disconnected.");
if (!connection.TrySend(message.Clone())) throw new IOException("Control queue is full or closed.");
}
private async Task ReadAsync(TlsControlConnection connection, CancellationToken cancellationToken)
{
Exception? failure = null;
try
{
await foreach (Envelope message in connection.ReadAsync(cancellationToken).ConfigureAwait(false))
{
Apply(message);
if (message.RequestId != 0 && pending.TryRemove(message.RequestId, out var completion)) completion.TrySetResult(message.Clone());
if (!events.Writer.TryWrite(message.Clone())) throw new IOException("Client event queue exhausted; consume events regularly.");
if (message.Disconnect is not null) { connection.CompleteWrites(); break; }
}
}
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { failure = exception; }
finally
{
connectionLifetime?.Cancel();
foreach (var operation in pending.Values) operation.TrySetException(failure ?? new IOException("Connection closed."));
SetState(ClientConnectionState.Disconnected);
}
}
private void Apply(Envelope message)
{
lock (stateGate)
{
if (message.AuthResult?.Ok == true) authentication = message.AuthResult.Clone();
if (message.ServerState is not null)
{
channels.Clear(); users.Clear();
foreach (var channel in message.ServerState.Channels) channels[channel.Id] = channel.Clone();
foreach (var user in message.ServerState.Users) users[user.Id] = user.Clone();
}
if (message.ChannelEvent is not null)
{
if (message.ChannelEvent.Kind == ChannelEvent.Types.Kind.Deleted) channels.Remove(message.ChannelEvent.DeletedId);
else if (message.ChannelEvent.Channel is not null) channels[message.ChannelEvent.Channel.Id] = message.ChannelEvent.Channel.Clone();
}
if (message.UserEvent is not null)
{
if (message.UserEvent.Kind == UserEvent.Types.Kind.Left) users.Remove(message.UserEvent.LeftId);
else if (message.UserEvent.User is not null) users[message.UserEvent.User.Id] = message.UserEvent.User.Clone();
}
if (authentication is not null && (message.ServerState is not null || message.UserEvent is not null))
{
User self = users.GetValueOrDefault(authentication.Self.Id, authentication.Self);
Audio.SetRemoteStreams(users.Values.ToArray(), self.Id, self.ChannelId);
foreach (var id in localStreams.Keys.Where(id => !self.Streams.Any(s => s.StreamId == id)).ToArray())
{ Audio.RemoveLocalStream(id); localStreams.Remove(id); }
}
}
}
private void SetState(ClientConnectionState value)
{
lock (stateGate) state = value;
ConnectionStateChanged?.Invoke(value);
}
public async Task DisconnectAsync()
{
connectionLifetime?.Cancel();
await lifecycle.WaitAsync().ConfigureAwait(false);
try { await CloseAsync().ConfigureAwait(false); }
finally { lifecycle.Release(); }
}
private async Task CloseAsync()
{
connectionLifetime?.Cancel();
try { await reader.ConfigureAwait(false); }
finally
{
try { await keepalive.ConfigureAwait(false); }
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
try { if (media is not null) await media.DisposeAsync().ConfigureAwait(false); }
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
try { if (control is not null) await control.DisposeAsync().ConfigureAwait(false); }
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
control = null;
media = null;
mediaCrypto?.Dispose(); mediaCrypto = null;
connectionLifetime?.Dispose(); connectionLifetime = null;
lock (stateGate) { authentication = null; hello = null; channels.Clear(); users.Clear(); }
lock (stateGate)
{
foreach (var id in localStreams.Keys) Audio.RemoveLocalStream(id);
localStreams.Clear(); Audio.SetRemoteStreams([], 0, 0);
}
SetState(ClientConnectionState.Disconnected);
}
}
public async ValueTask DisposeAsync()
{
if (disposed.IsCancellationRequested) return;
disposed.Cancel();
await DisconnectAsync().ConfigureAwait(false);
events.Writer.TryComplete();
Audio.Dispose();
}
}
@@ -0,0 +1,44 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
}
}
}
@@ -0,0 +1,51 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0/win-x64": {}
}
}
@@ -1,6 +1,6 @@
using VoiceCat.Crypto; using VoiceCat.Crypto;
namespace VoiceCat.Server.Transport; namespace VoiceCat.Transport;
internal sealed class MediaSessionCrypto(MediaEncryptor encryptor, MediaDecryptor decryptor) : IDisposable internal sealed class MediaSessionCrypto(MediaEncryptor encryptor, MediaDecryptor decryptor) : IDisposable
{ {
@@ -7,7 +7,7 @@ using VoiceCat.Crypto;
using VoiceCat.Protocol; using VoiceCat.Protocol;
using Voicecat.V1; using Voicecat.V1;
namespace VoiceCat.Server.Transport; namespace VoiceCat.Transport;
internal sealed class TlsControlConnection : IAsyncDisposable internal sealed class TlsControlConnection : IAsyncDisposable
{ {
@@ -25,6 +25,7 @@ internal sealed class TlsControlConnection : IAsyncDisposable
private MediaSessionCrypto? mediaCrypto; private MediaSessionCrypto? mediaCrypto;
public Task Completion { get; } public Task Completion { get; }
internal System.Net.EndPoint RemoteEndPoint => socket.RemoteEndPoint!;
public CancellationToken CancellationToken => lifetime.Token; public CancellationToken CancellationToken => lifetime.Token;
internal TlsControlConnection(Socket socket, TlsSession tls, CancellationToken cancellationToken, TimeSpan? handshakeTimeout = null) internal TlsControlConnection(Socket socket, TlsSession tls, CancellationToken cancellationToken, TimeSpan? handshakeTimeout = null)
@@ -5,5 +5,7 @@
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="VoiceCat.Tests" /> <InternalsVisibleTo Include="VoiceCat.Tests" />
<InternalsVisibleTo Include="VoiceCat.Server" />
<InternalsVisibleTo Include="VoiceCat.Core" />
</ItemGroup> </ItemGroup>
</Project> </Project>
@@ -0,0 +1,14 @@
{
"version": 1,
"dependencies": {
"net10.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.7, )",
"resolved": "10.0.7",
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
}
},
"net10.0/win-x64": {}
}
}
@@ -1,3 +1,4 @@
using VoiceCat.Transport;
using System.Collections.Concurrent; using System.Collections.Concurrent;
using System.Net; using System.Net;
using System.Net.Sockets; using System.Net.Sockets;
@@ -1,3 +1,4 @@
using VoiceCat.Transport;
using System.Net; using System.Net;
using System.Net.Sockets; using System.Net.Sockets;
using System.Security.Cryptography; using System.Security.Cryptography;
@@ -0,0 +1,117 @@
using VoiceCat.Audio;
using VoiceCat.Codec;
using VoiceCat.Protocol;
using Voicecat.V1;
namespace VoiceCat.Tests;
public class AudioEngineTests
{
[Theory]
[InlineData(true)]
[InlineData(false)]
public void LostFramesUseDredThenFecBeforeBoundedPlc(bool useDred)
{
using var receive = new ReceiveStream(2, Stream(dred: useDred));
using var encoder = new OpusEncoder(new() { DeepRedundancy = useDred, ForwardErrorCorrection = true, ExpectedPacketLossPercent = 30, Complexity = 10, Bitrate = 64000 });
byte[] packet = new byte[1275]; short[] tone = Tone(); int[] output = new int[1920];
for (uint i = 0; i < 40; i++)
{
CodecTests.FillTone(tone, 960, 1, 48000, (int)i);
int size = encoder.Encode(tone, packet);
if (i != 25 && i != 30 && i != 35) receive.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, size));
output.AsSpan().Clear(); receive.Mix(output, false, null);
}
if (useDred) Assert.True(receive.DredFrames > 0); else Assert.True(receive.FecFrames > 0);
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); receive.Mix(output, false, null); }
Assert.True(receive.ConcealedFrames > 0); Assert.All(output, sample => Assert.Equal(0, sample));
}
[Fact]
public void PcmRingDropsWholeFramesWhenFullAndPreservesOrderAcrossWraps()
{
var ring = new PcmRing(8); short[] output = new short[8];
for (int i = 0; i < 100; i++)
{
Assert.True(ring.TryWrite([1, 2, 3, 4, 5, 6])); Assert.False(ring.TryWrite([7, 8, 9]));
Assert.Equal(4, ring.Read(output.AsSpan(0, 4))); Assert.Equal(new short[] { 1, 2, 3, 4 }, output[..4]);
Assert.True(ring.TryWrite([7, 8])); Assert.Equal(4, ring.Read(output)); Assert.Equal(new short[] { 5, 6, 7, 8 }, output[..4]); Assert.Equal(0, ring.Count);
}
}
internal static StreamInfo Stream(int frame = 20, bool stereo = false, bool dred = false) => new()
{
StreamId = 1, Ssrc = 42, Kind = StreamKind.StreamMic,
Audio = new() { SampleRate = 48000, BitrateBps = 32000, FrameMs = (uint)frame, Complexity = 5,
Mode = stereo ? ChannelMode.ModeStereo : ChannelMode.ModeMono, Fec = true, ExpectedPacketLoss = 20, Dred = dred }
};
private static short[] Tone(int channels = 1)
{
var pcm = new short[960 * channels];
for (int i = 0; i < 960; i++) for (int c = 0; c < channels; c++) pcm[i * channels + c] = (short)(Math.Sin(i * 2 * Math.PI * (c == 0 ? 440 : 660) / 48000) * 8000);
return pcm;
}
[Theory]
[InlineData(5, false)] [InlineData(10, false)] [InlineData(20, false)] [InlineData(40, false)] [InlineData(60, false)] [InlineData(20, true)]
public void ReframedEncodedPcmIsDecodedAndMixedForMonoAndStereo(int frame, bool stereo)
{
StreamInfo stream = Stream(frame, stereo);
using var receive = new AudioEngine((_, _, _, _) => true, false);
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { stream } }], 1, 1);
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
send.InputMode = AudioInputMode.AlwaysOn; send.AddLocalStream(stream, stereo ? 2 : 1);
long energy = 0; int sinkChannels = 0;
receive.MixedPcm += pcm => { foreach (short sample in pcm) energy += Math.Abs((int)sample); };
receive.StreamPcm += (_, _, _, channels) => sinkChannels = channels;
short[] tone = Tone(stereo ? 2 : 1);
for (int i = 0; i < 30; i++) { Assert.True(send.FeedPcm(1, tone, stereo ? 2 : 1)); send.ProcessCycle(); receive.ProcessCycle(); }
Assert.True(energy > 100000); Assert.Equal(stereo ? 2 : 1, sinkChannels);
receive.SetRemotePlayback(2, 1, 1, true, false); energy = 0;
for (int i = 0; i < 5; i++) { send.FeedPcm(1, tone, stereo ? 2 : 1); send.ProcessCycle(); receive.ProcessCycle(); }
Assert.Equal(0, energy);
}
[Fact]
public void AudioCyclesAllocateZeroBytesWithEncodeDecodeStereoNoiseReductionAndMixing()
{
StreamInfo stream = Stream(20, true);
using var receive = new AudioEngine((_, _, _, _) => true, false);
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { stream } }], 1, 1);
receive.SetRemotePlayback(2, 1, 0.8f, false, true);
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
send.InputMode = AudioInputMode.AlwaysOn; send.InputNoiseReduction = true; send.AddLocalStream(stream, 2);
short[] tone = Tone(2);
for (int i = 0; i < 30; i++) { send.FeedPcm(1, tone, 2); send.ProcessCycle(); receive.ProcessCycle(); }
long before = GC.GetAllocatedBytesForCurrentThread();
for (int i = 0; i < 100; i++) { send.FeedPcm(1, tone, 2); send.ProcessCycle(); receive.ProcessCycle(); }
Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before);
}
[Fact]
public void JitterBacklogIsBoundedAndPlcEventuallyBecomesSilence()
{
var info = Stream(); using var stream = new ReceiveStream(2, info); using var encoder = new OpusEncoder(new() { Bitrate = 32000 });
byte[] packet = new byte[1275]; int length = encoder.Encode(Tone(), packet); int[] output = new int[1920];
for (uint i = 0; i < 64; i++) stream.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, length));
stream.Mix(output, false, null); Assert.InRange(stream.Depth, 0, 6);
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); stream.Mix(output, false, null); }
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 10);
}
[Fact]
public void PttResumeAndCaptureChannelChangesKeepTimestampProgressAndAudio()
{
var info = Stream(60, true); using var receive = new AudioEngine((_, _, _, _) => true, false);
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { info } }], 1, 1);
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
send.InputMode = AudioInputMode.PushToTalk; send.PushToTalk = true; send.AddLocalStream(info);
short[] mono = Tone(), stereo = Tone(2); long energy = 0;
receive.MixedPcm += pcm => { foreach (short value in pcm) energy += Math.Abs((int)value); };
for (int i = 0; i < 15; i++) { send.FeedPcm(1, mono, 1); send.ProcessCycle(); receive.ProcessCycle(); }
send.PushToTalk = false;
for (int i = 0; i < 16; i++) { send.FeedPcm(1, mono, 1); send.ProcessCycle(); receive.ProcessCycle(); }
send.SetCaptureChannels(1, 2); send.PushToTalk = true; energy = 0;
for (int i = 0; i < 15; i++) { send.FeedPcm(1, stereo, 2); send.ProcessCycle(); receive.ProcessCycle(); }
Assert.True(energy > 100000);
}
}
@@ -0,0 +1,80 @@
using VoiceCat.Core;
using VoiceCat.Crypto;
using Voicecat.V1;
using static VoiceCat.Tests.ServerTests;
using static VoiceCat.Tests.MediaRelayTests;
namespace VoiceCat.Tests;
public class ManagedClientTests
{
private static VoiceCatClient NewClient(ServerFixture fixture, string name) => new(name, "test", Path.Combine(fixture.Directory, name + ".pins"));
private static Task Connect(VoiceCatClient client, ServerFixture fixture) => client.ConnectAsync("127.0.0.1", (ushort)fixture.Server.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
private static async Task<Envelope> Event(VoiceCatClient client, Func<Envelope, bool> predicate)
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10));
await foreach (Envelope message in client.ReadEventsAsync(timeout.Token)) if (predicate(message)) return message;
throw new IOException("Expected client event was not received.");
}
[Fact]
public async Task ManagedClientsAuthenticateChatAndCorrelateConcurrentRequests()
{
await using var fixture = new ServerFixture();
await using var alice = NewClient(fixture, "Alice"); await using var bob = NewClient(fixture, "Bob");
await Connect(alice, fixture); await Connect(bob, fixture);
Assert.True((await alice.AuthenticateGuestAsync("Alice")).Ok); Assert.True((await bob.AuthenticateGuestAsync("Bob")).Ok);
await Event(bob, e => e.ServerState is not null); await Event(alice, e => e.UserEvent?.User?.Nickname == "Bob");
Assert.Equal(2, alice.Users.Count);
var copy = alice.Users[0]; copy.Nickname = "Mutated"; Assert.DoesNotContain(alice.Users, u => u.Nickname == "Mutated");
alice.Send(new() { TextMessage = new() { Scope = TextScope.TextChannel, TargetId = 1, Body = "Managed conversation", ClientMsgId = "a1" } });
Assert.Equal("Managed conversation", (await Event(bob, e => e.TextMessage is not null)).TextMessage.Body);
var requests = Enumerable.Range(1, 20).Select(async i =>
{
Envelope response = await alice.RequestAsync(new() { Ping = new() { Nonce = (ulong)i } });
Assert.Equal((ulong)i, response.Pong.Nonce); return response.RequestId;
});
Assert.Equal(20, (await Task.WhenAll(requests)).Distinct().Count());
await Assert.ThrowsAsync<InvalidOperationException>(() => Connect(alice, fixture));
Assert.Equal(ClientConnectionState.Connected, alice.State);
await alice.DisconnectAsync();
await Event(bob, e => e.UserEvent?.Kind == UserEvent.Types.Kind.Left);
await Connect(alice, fixture); Assert.True((await alice.AuthenticateGuestAsync("Returned")).Ok);
}
[Fact]
public async Task TofuRequiresApprovalPinsAcceptedCertificateAndRejectsChanges()
{
await using var first = new ServerFixture(); await using var second = new ServerFixture();
await using var client = NewClient(first, "Tofu");
await Assert.ThrowsAsync<System.Security.Authentication.AuthenticationException>(() => client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port));
await client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port, (challenge, _) =>
{ Assert.Equal(TofuStatus.FirstConnect, challenge.Status); return ValueTask.FromResult(true); });
await client.DisconnectAsync();
await client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port); await client.DisconnectAsync();
// Pin the other server's certificate to this endpoint, simulating a changed server certificate.
using var credentials = ServerCredentials.LoadOrCreate(second.Directory, "VoiceCat Server");
new TofuStore(Path.Combine(first.Directory, "Other.pins")).Pin("127.0.0.1", (ushort)first.Server.EndPoint.Port, credentials.CertificateFingerprint);
await using var changed = new VoiceCatClient(tofuStorePath: Path.Combine(first.Directory, "Other.pins"));
await Assert.ThrowsAsync<System.Security.Authentication.AuthenticationException>(() => changed.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port,
(challenge, _) => { Assert.Equal(TofuStatus.Mismatch, challenge.Status); return ValueTask.FromResult(false); }));
}
[Fact]
public async Task ManagedClientSendsAndReceivesAuthenticatedEncodedVoice()
{
await using var fixture = new ServerFixture();
await using var managed = NewClient(fixture, "Managed"); await Connect(managed, fixture); await managed.AuthenticateGuestAsync("Managed");
Assert.True((await managed.SubscribeVoiceAsync()).Ok);
await using var peer = await VoicePeer.ConnectAsync(fixture, "Peer");
var remote = await peer.AnnounceAsync(StreamKind.StreamMic);
var local = (await managed.RequestAsync(new() { StreamAnnounce = new() { Kind = StreamKind.StreamMic } })).StreamAnnounceResult;
Assert.True(local.Ok);
var received = new TaskCompletionSource<byte[]>(TaskCreationOptions.RunContinuationsAsynchronously);
managed.VoiceReceived += (header, payload) => { Assert.Equal(remote.Ssrc, header.Ssrc); received.TrySetResult(payload.ToArray()); };
await peer.SendAsync(peer.Seal(remote.Ssrc, [1, 2, 3]));
Assert.Equal(new byte[] { 1, 2, 3 }, await received.Task.WaitAsync(TimeSpan.FromSeconds(5)));
Assert.True(managed.TrySendEncodedVoice(local.Ssrc, 960, [4, 5, 6]));
Assert.Equal(new byte[] { 4, 5, 6 }, (await peer.ReceiveVoiceAsync()).Payload);
}
}
@@ -1,3 +1,4 @@
using VoiceCat.Transport;
using System.Net; using System.Net;
using System.Diagnostics; using System.Diagnostics;
using System.Net.Sockets; using System.Net.Sockets;
@@ -1,3 +1,4 @@
using VoiceCat.Transport;
using System.Net; using System.Net;
using System.Net.Sockets; using System.Net.Sockets;
using System.Text.Json; using System.Text.Json;
@@ -1,3 +1,4 @@
using VoiceCat.Transport;
using System.Diagnostics; using System.Diagnostics;
using System.Net; using System.Net;
using System.Net.Sockets; using System.Net.Sockets;
@@ -4,6 +4,8 @@
<IsTestProject>true</IsTestProject> <IsTestProject>true</IsTestProject>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="../../../clients/windows/VoiceCat.Managed/VoiceCat.Managed.csproj" />
<ProjectReference Include="../../src/VoiceCat.Core/VoiceCat.Core.csproj" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" /> <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageReference Include="xunit" Version="2.9.3" /> <PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.1" PrivateAssets="all" /> <PackageReference Include="xunit.runner.visualstudio" Version="3.1.1" PrivateAssets="all" />
@@ -0,0 +1,58 @@
using VoiceCat.Interop;
using VoiceCat.Server.Data;
using Voicecat.V1;
using static VoiceCat.Tests.ServerTests;
using Client = VoiceCat.Interop.VoiceCatClient;
namespace VoiceCat.Tests;
public class WindowsManagedClientTests
{
private static async Task Until(Client client, Func<bool> predicate)
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10));
do { client.PumpEvents(); if (predicate()) return; await Task.Delay(10, timeout.Token); } while (true);
}
private static async Task Login(Client client, ServerFixture fixture, bool admin = false)
{
client.EventReceived += e => { if (e.Type == VcEventType.ServerIdentity) client.ConfirmServerIdentity(true); };
Assert.Equal(VcResult.Ok, client.Connect("127.0.0.1", (ushort)fixture.Server.EndPoint.Port));
if (admin) client.AuthenticateUser("Admin", "secret"); else client.AuthenticateGuest("Guest");
await Until(client, () => client.ListUsers().Count > 0);
}
[Fact]
public async Task ShippedWindowsFacadeChatsExchangesPcmAndKeepsCaptureIdsAcrossChannelMoves()
{
await using var fixture = new ServerFixture();
using (var accounts = new AccountStore(Path.Combine(fixture.Directory, "voicecat.db"))) await accounts.CreateAccountAsync("Admin", "secret", true);
using var alice = new Client("Alice", "test", tofuStorePath: Path.Combine(fixture.Directory, "alice.pins"));
using var bob = new Client("Bob", "test", tofuStorePath: Path.Combine(fixture.Directory, "bob.pins"));
await Login(alice, fixture, true); await Login(bob, fixture);
Assert.True(alice.GetPermissions().IsAdmin);
string? body = null; bob.EventReceived += e => { if (e.Type == VcEventType.TextMessage) body = e.Text; };
Assert.Equal(VcResult.Ok, alice.SendText(VcTextScope.Channel, 1, "Managed Windows chat"));
await Until(bob, () => body is not null); Assert.Equal("Managed Windows chat", body);
Assert.Equal(VcResult.Ok, alice.JoinVoice()); Assert.Equal(VcResult.Ok, bob.JoinVoice());
alice.SetInputMode(VcInputMode.AlwaysOn); bob.SetInputMode(VcInputMode.AlwaysOn);
var a = alice.StartStreamExternalFeed(VcStreamKind.Mic, "Mic"); var b = bob.StartStreamExternalFeed(VcStreamKind.Mic, "Mic");
Assert.Equal(VcResult.Ok, a.Result); Assert.Equal(VcResult.Ok, b.Result);
await Until(bob, () => bob.ManagedClient.Users.Any(u => u.Streams.Count > 0 && u.Id != bob.ManagedClient.Authentication!.Self.Id));
long aliceEnergy = 0, bobEnergy = 0;
alice.ManagedClient.Audio.MixedPcm += pcm => { long sum = 0; foreach (short sample in pcm) sum += Math.Abs((int)sample); Interlocked.Add(ref aliceEnergy, sum); };
bob.ManagedClient.Audio.MixedPcm += pcm => { long sum = 0; foreach (short sample in pcm) sum += Math.Abs((int)sample); Interlocked.Add(ref bobEnergy, sum); };
short[] tone = Enumerable.Range(0, 960).Select(i => (short)(8000 * Math.Sin(i * Math.PI * 880 / 48000))).ToArray();
for (int i = 0; i < 40; i++) { alice.StreamFeedPcm(a.StreamId, tone, 960, 1); bob.StreamFeedPcm(b.StreamId, tone, 960, 1); alice.PumpEvents(); bob.PumpEvents(); await Task.Delay(20); }
Assert.True(Interlocked.Read(ref aliceEnergy) > 100000); Assert.True(Interlocked.Read(ref bobEnergy) > 100000);
uint oldId = alice.ManagedClient.LocalStreams.Single().StreamId;
await alice.ManagedClient.RequestAsync(new() { CreateChannel = new() { Channel = new() { Name = "Stereo", ParentId = 1, Audio = AudioEngineTests.Stream(20, true).Audio } } });
await Until(alice, () => alice.ListChannels().Any(c => c.Name == "Stereo"));
uint channel = alice.ListChannels().Single(c => c.Name == "Stereo").Id;
Assert.Equal(VcResult.Ok, alice.JoinChannel(channel));
await Until(alice, () => alice.ManagedClient.LocalStreams.Any(s => s.StreamId != oldId));
Assert.Equal(a.StreamId, Assert.Single(alice.ListUserStreams(alice.ManagedClient.Authentication!.Self.Id)).StreamId);
Assert.True(alice.GetStreamAudioConfig(alice.ManagedClient.Authentication!.Self.Id, a.StreamId).Config!.Stereo);
Assert.Equal(VcResult.Ok, alice.StreamFeedPcm(a.StreamId, tone, 960, 1));
Assert.Equal(VcResult.Ok, alice.StopStream(a.StreamId));
Assert.Empty(alice.ManagedClient.LocalStreams);
}
}
@@ -146,9 +146,24 @@
"xunit.extensibility.core": "[2.9.3]" "xunit.extensibility.core": "[2.9.3]"
} }
}, },
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": { "voicecat.codec": {
"type": "Project" "type": "Project"
}, },
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": { "voicecat.crypto": {
"type": "Project", "type": "Project",
"dependencies": { "dependencies": {
@@ -159,6 +174,12 @@
"voicecat.dsp": { "voicecat.dsp": {
"type": "Project" "type": "Project"
}, },
"voicecat.managed": {
"type": "Project",
"dependencies": {
"VoiceCat.Core": "[1.0.0, )"
}
},
"voicecat.protocol": { "voicecat.protocol": {
"type": "Project", "type": "Project",
"dependencies": { "dependencies": {