Port managed client audio and Windows application
This commit is contained in:
@@ -39,7 +39,9 @@ dotnet test dotnet/VoiceCat.slnx -c Release --no-build
|
||||
See `dotnet/README.md` for C# conventions and required native voice/CLI conformance,
|
||||
and `docs/api-dotnet.md` for managed interfaces. Phase 4 remains in progress;
|
||||
media-aware reaping and server administration are implemented. Server deployment hardening
|
||||
and managed audio/client/Windows cutover are in progress; Apple GUI rewrites follow Windows.
|
||||
and the managed audio/client core plus Windows cutover are implemented. The Windows publish
|
||||
contains only the permitted media shim. Accessibility/manual endurance validation and broad
|
||||
server deployment work remain; Apple GUI rewrites follow Windows.
|
||||
|
||||
The default development preset is **`dev`** — it builds everything (server + tools + tests)
|
||||
with real vcpkg deps. The `skeleton` preset (no deps, stubs only) is a fast smoke check; see
|
||||
|
||||
+21
@@ -10,6 +10,27 @@ up instantly. Newest status at the top.
|
||||
|
||||
## ▶ Where we left off / next action
|
||||
|
||||
- **Done (2026-09-16): Managed client/audio and Windows cutover checkpoint.** Added
|
||||
`VoiceCat.Core` with TOFU-gated TLS, correlated concurrent requests, immutable snapshots,
|
||||
reconnects, bounded client events, UDP binding and authenticated encrypted media. Added
|
||||
`VoiceCat.Audio` with allocation-free real-time cycles, bounded PCM/jitter queues,
|
||||
5/10/20/40/60 ms Opus reframing, VAD/PTT, DTX, DRED → FEC → PLC, RNNoise, per-stream
|
||||
controls and stereo mixing. Shared TLS/media primitives moved to `VoiceCat.Crypto`.
|
||||
The WinForms app now references a managed compatibility facade over these libraries and
|
||||
uses C# WASAPI capture, loopback and playback. Active streams renegotiate after channel
|
||||
moves while capture-facing IDs stay stable. Per-app mixing no longer allocates or locks in
|
||||
its real-time loop. The self-contained Windows publish includes only the permitted
|
||||
`voicecat_media.dll`; a real default-device capture/playback plus main-form smoke passed.
|
||||
Managed tests cover two-way decoded PCM voice, text, TOFU, concurrent requests, reconnects,
|
||||
bounded jitter, zero-allocation cycles, recovery order and stream continuity across a
|
||||
mono→stereo channel move. The old Interop project remains only as a migration oracle and
|
||||
its nine native ABI tests remain green. **Verified:** 190 managed tests with every native
|
||||
conformance/published-server variable enabled, nine Windows native-oracle tests, 29 CTest
|
||||
tests, locked restores, both self-contained publishes and the permissive license audit.
|
||||
**Next:** add the managed CLI and explicit managed
|
||||
client ↔ C++ CLI conversation, then finish production deployment work. The Phase 5/7
|
||||
manual ten-minute listen and NVDA gates are not yet signed off; Apple rewrites follow.
|
||||
|
||||
- **Done (2026-09-15): Server deployment hardening checkpoint.** Pushed existing work
|
||||
through `653131b` to `origin/dotnet/foundations`. Added CLI/environment configuration,
|
||||
all-interface port 8384 defaults, config/fingerprint commands, local administrator
|
||||
|
||||
+20
-72
@@ -1,95 +1,43 @@
|
||||
# VoiceCat — Windows client
|
||||
# VoiceCat — Windows client
|
||||
|
||||
WinForms (.NET 10 LTS) UI over `voicecat.dll` (MinGW-built `libvoicecat` shared library).
|
||||
The WinForms .NET 10 client uses `VoiceCat.Core` for TLS, TOFU, protocol state and encrypted UDP, and `VoiceCat.Audio` for streams, jitter, Opus and mixing. Its only native runtime component is `voicecat_media.dll`, the narrow Opus/RNNoise C shim. The old `voicecat.dll` core is no longer loaded or published.
|
||||
|
||||
## Prerequisites
|
||||
## Build
|
||||
|
||||
| Tool | Version | Notes |
|
||||
|------|---------|-------|
|
||||
| .NET SDK | 10.0.x | `dotnet --version` should report `10.0.*` |
|
||||
| CMake | 3.25+ | For building the C++ DLL |
|
||||
| MinGW-w64 / MSYS2 UCRT64 | GCC 13+ | `C:\tools\msys64\ucrt64` is the expected location |
|
||||
| vcpkg | any | `VCPKG_ROOT` env var must point to a bootstrapped clone |
|
||||
|
||||
## Build order
|
||||
|
||||
### 1. Build the server (for testing)
|
||||
Stage the pinned native media dependencies once, then build the solution:
|
||||
|
||||
```powershell
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev --target voicecat-server
|
||||
./dotnet/build-native.ps1
|
||||
dotnet restore clients/windows/VoiceCat.slnx --locked-mode
|
||||
dotnet build clients/windows/VoiceCat.slnx -c Release --no-restore
|
||||
```
|
||||
|
||||
### 2. Build the DLL
|
||||
The original `VoiceCat.Interop` project remains in the repository as a migration oracle. The app references `VoiceCat.Managed`, whose compatibility facade lets the existing accessible WinForms UI keep its event-pump shape while all networking and audio state live in the idiomatic managed libraries.
|
||||
|
||||
## Publish
|
||||
|
||||
```powershell
|
||||
cmake --preset windows-client
|
||||
cmake --build --preset windows-client
|
||||
./clients/windows/publish-client.ps1
|
||||
```
|
||||
|
||||
Output: `build/windows-client/bin/voicecat.dll`
|
||||
This produces a self-contained `win-x64` distribution in `dotnet/artifacts/client/win-x64`. The script requires `voicecat_media.dll` and fails if the legacy `voicecat.dll` appears.
|
||||
|
||||
**Verify no MinGW runtime dependencies remain:**
|
||||
```powershell
|
||||
& "C:\tools\msys64\ucrt64\bin\objdump.exe" -p build/windows-client/bin/voicecat.dll |
|
||||
Select-String "DLL Name"
|
||||
```
|
||||
Expected: only Windows system DLLs (`KERNEL32.dll`, `WS2_32.dll`, `BCRYPT.dll`, etc.).
|
||||
If `libgcc_s_seh-1.dll`, `libstdc++-6.dll`, or `libwinpthread-1.dll` appear, the
|
||||
`-static-libgcc -static-libstdc++ -static -lwinpthread` link flags in `core/CMakeLists.txt`
|
||||
are not taking effect — check the CMake log for the `VOICECAT_BUILD_SHARED+WIN32` branch.
|
||||
|
||||
### 3. Build the C# solution
|
||||
The noninteractive startup and real WASAPI device check is:
|
||||
|
||||
```powershell
|
||||
cd clients/windows
|
||||
dotnet build VoiceCat.slnx
|
||||
./dotnet/artifacts/client/win-x64/VoiceCat.App.exe --smoke-test --audio
|
||||
```
|
||||
|
||||
The app's `Directory.Build.props` copies `voicecat.dll` from `../../build/windows-client/bin/`
|
||||
into the output directory automatically on every build.
|
||||
`--smoke-test` constructs the real main form and pumps the managed client. `--audio` additionally opens the default WASAPI capture and render endpoints, moves PCM through both for three seconds, and fails if capture produces no samples.
|
||||
|
||||
## Running manually
|
||||
## Run manually
|
||||
|
||||
```powershell
|
||||
# Terminal 1 — start the server
|
||||
./build/dev/bin/voicecat-server.exe --name "My Server"
|
||||
# Terminal 1
|
||||
./dotnet/artifacts/server/win-x64/VoiceCat.Server.exe
|
||||
|
||||
# Terminal 2 — launch the client
|
||||
# Terminal 2
|
||||
dotnet run --project clients/windows/VoiceCat.App/VoiceCat.App.csproj
|
||||
```
|
||||
|
||||
On first connect to a new server:
|
||||
- Enter `127.0.0.1` as the host (not `localhost` — Windows resolves `localhost` to `::1`
|
||||
first, and while the server now dual-stacks, `127.0.0.1` is cleaner for local testing).
|
||||
- The server identity dialog will appear. The TLS leaf-cert SHA-256 fingerprint is shown;
|
||||
accept to pin it. Subsequent connects to the same server will be silent (MATCHED).
|
||||
|
||||
## M5 — Moderation & admin UI
|
||||
|
||||
The WinForms client now exposes all M5 operations through the main menu and context menus:
|
||||
|
||||
- **Admin → Server accounts…** — create, reset password, and delete server accounts
|
||||
(requires `can_admin_accounts`).
|
||||
- **Channel tree right-click** — create, edit, and delete channels. The edit dialog exposes the
|
||||
full per-channel Opus configuration: mono/stereo, sample rate, bitrate, frame size,
|
||||
application mode, FEC, expected packet loss, DTX, and complexity.
|
||||
- **User list right-click** — move, kick, ban, server mute/deafen, and set permissions
|
||||
(items are gated by your own permissions).
|
||||
- **Activity log** shows async `GenericResult` feedback for every moderation request.
|
||||
- **User list** shows text indicators for self-mute, self-deafen, server-mute, and
|
||||
server-deafen states.
|
||||
|
||||
These operations require an admin-provisioned account with the appropriate permissions; the
|
||||
connect dialog already supports username/password auth.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- **PTT is focus-scoped** — the push-to-talk key only works while the VoiceCat window has
|
||||
focus. A system-wide `WH_KEYBOARD_LL` hook is not used in v1 (permissions + AV risk).
|
||||
- **Receive-side noise reduction** checkbox in per-user tuning is wired end-to-end but is a
|
||||
passthrough no-op until a real APM/NS backend is built (no working Windows/MSVC port of
|
||||
`webrtc-audio-processing` upstream — see `docs/tech-stack.md §1`).
|
||||
- **TOFU pins the TLS leaf cert**, not the declared Ed25519 identity fingerprint. Both are
|
||||
shown in the identity dialog, but the cert fingerprint is the value that is actually
|
||||
verified on reconnect. See `docs/security.md §1.1`.
|
||||
Manual release validation still includes NVDA navigation and a ten-minute two-client listen test, as required by `docs/porting-to-dotnet.md`.
|
||||
|
||||
@@ -16,7 +16,7 @@ public sealed record InputDeviceInfo(string Id, string Name, bool IsDefault)
|
||||
/// the aux device is opened client-side and needs a WASAPI id, not a miniaudio one.</summary>
|
||||
public static class InputDeviceEnumerator
|
||||
{
|
||||
public static IReadOnlyList<InputDeviceInfo> List()
|
||||
public static IReadOnlyList<InputDeviceInfo> List(bool input = true)
|
||||
{
|
||||
var result = new List<InputDeviceInfo>();
|
||||
InputDeviceCapture.IMMDeviceEnumerator? enumerator = null;
|
||||
@@ -29,7 +29,7 @@ public static class InputDeviceEnumerator
|
||||
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
|
||||
|
||||
// Resolve the default capture endpoint id so the picker can flag it.
|
||||
if (enumerator.GetDefaultAudioEndpoint(1 /*eCapture*/, 0 /*eConsole*/,
|
||||
if (enumerator.GetDefaultAudioEndpoint(input ? 1 : 0, 0 /*eConsole*/,
|
||||
out var defDev) == 0 && defDev != null)
|
||||
{
|
||||
try { if (defDev.GetId(out string id) == 0) defaultId = id; }
|
||||
@@ -37,7 +37,7 @@ public static class InputDeviceEnumerator
|
||||
}
|
||||
|
||||
// DEVICE_STATE_ACTIVE = 0x1 — only currently-usable endpoints.
|
||||
if (enumerator.EnumAudioEndpoints(1 /*eCapture*/, 0x1, out collectionPtr) != 0
|
||||
if (enumerator.EnumAudioEndpoints(input ? 1 : 0, 0x1, out collectionPtr) != 0
|
||||
|| collectionPtr == IntPtr.Zero)
|
||||
return result;
|
||||
|
||||
@@ -107,6 +107,7 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
private const int FrameSamples = 960; // 20 ms
|
||||
|
||||
private readonly string? _deviceId; // null = system default capture endpoint
|
||||
private readonly bool _loopback;
|
||||
|
||||
private IAudioClient? _audioClient;
|
||||
private IAudioCaptureClient? _captureClient;
|
||||
@@ -123,8 +124,9 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
// Init-done signal: Set() by the capture thread after activation completes.
|
||||
private readonly ManualResetEventSlim _initDone = new(false);
|
||||
private bool _initOk;
|
||||
private int _disposed;
|
||||
|
||||
public InputDeviceCapture(string? deviceId) => _deviceId = deviceId;
|
||||
public InputDeviceCapture(string? deviceId, bool loopback = false) { _deviceId = deviceId; _loopback = loopback; }
|
||||
|
||||
/// <summary>Starts capture. Blocks until WASAPI activation completes (typically <100 ms).
|
||||
/// Returns false if the device cannot be opened.</summary>
|
||||
@@ -148,15 +150,13 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
{
|
||||
_running = false;
|
||||
_bufferEvent?.Set();
|
||||
_captureThread?.Join(500);
|
||||
try { _audioClient?.Stop(); } catch { /* device already gone */ }
|
||||
if (_captureThread is not null && !_captureThread.Join(5000)) throw new TimeoutException("Capture did not stop.");
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
|
||||
Stop();
|
||||
if (_captureClient != null) { Marshal.ReleaseComObject(_captureClient); _captureClient = null; }
|
||||
if (_audioClient != null) { Marshal.ReleaseComObject(_audioClient); _audioClient = null; }
|
||||
_bufferEvent?.Dispose();
|
||||
_initDone.Dispose();
|
||||
}
|
||||
@@ -164,11 +164,20 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
// ── Capture thread (MTA) ──────────────────────────────────────────────────
|
||||
|
||||
private void CaptureThreadProc()
|
||||
{
|
||||
try
|
||||
{
|
||||
_initOk = ActivateAndStart();
|
||||
_initDone.Set();
|
||||
if (!_initOk) return;
|
||||
CaptureLoop();
|
||||
if (_initOk && _running) CaptureLoop();
|
||||
}
|
||||
catch { _initOk = false; _initDone.Set(); }
|
||||
finally
|
||||
{
|
||||
try { _audioClient?.Stop(); } catch { }
|
||||
if (_captureClient != null) { Marshal.ReleaseComObject(_captureClient); _captureClient = null; }
|
||||
if (_audioClient != null) { Marshal.ReleaseComObject(_audioClient); _audioClient = null; }
|
||||
}
|
||||
}
|
||||
|
||||
private bool ActivateAndStart()
|
||||
@@ -192,7 +201,7 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
|
||||
|
||||
int hr = _deviceId is null
|
||||
? enumerator.GetDefaultAudioEndpoint(1 /*eCapture*/, 0 /*eConsole*/, out device)
|
||||
? enumerator.GetDefaultAudioEndpoint(_loopback ? 0 : 1, 0 /*eConsole*/, out device)
|
||||
: enumerator.GetDevice(_deviceId, out device);
|
||||
if (hr != 0 || device == null) return false;
|
||||
|
||||
@@ -220,7 +229,7 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
// AUDCLNT_STREAMFLAGS_EVENTCALLBACK = 0x00040000
|
||||
// AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM = 0x80000000
|
||||
// AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY = 0x08000000
|
||||
const uint streamFlags = 0x00040000u | 0x80000000u | 0x08000000u;
|
||||
uint streamFlags = 0x00040000u | 0x80000000u | 0x08000000u | (_loopback ? 0x00020000u : 0u);
|
||||
|
||||
foreach (int ch in new[] { 2, 1 })
|
||||
{
|
||||
@@ -327,9 +336,7 @@ public sealed class InputDeviceCapture : IDisposable
|
||||
|
||||
private void FlushFrame()
|
||||
{
|
||||
var copy = new short[_accumBuf.Length];
|
||||
_accumBuf.AsSpan().CopyTo(copy);
|
||||
PcmFrameReady?.Invoke(copy, FrameSamples, _channels);
|
||||
PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels); // Borrowed until callback returns.
|
||||
_accumCount = 0;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,148 +1,85 @@
|
||||
using VoiceCat.Audio;
|
||||
using VoiceCat.Interop;
|
||||
|
||||
// Owns N ProcessLoopbackCapture instances, mixes their PCM every 20 ms, and feeds
|
||||
// the result to the core via vc_stream_feed_pcm. Used for per-app audio sharing.
|
||||
namespace VoiceCat.App.Audio;
|
||||
|
||||
// Capture threads own their ring producers; the mix thread alone consumes them.
|
||||
public sealed class ProcessAudioMixer : IDisposable
|
||||
{
|
||||
private const int SampleRate = 48000;
|
||||
private const int FrameSamples = 960;
|
||||
private const int Channels = 2; // stereo; captures fall back to mono if needed
|
||||
|
||||
private readonly List<ProcessLoopbackCapture> _captures = [];
|
||||
|
||||
// Per-capture latest frame, protected by _frameLock.
|
||||
private readonly object _frameLock = new();
|
||||
private List<short[]> _latestFrames = [];
|
||||
private int _activeChannels = Channels;
|
||||
|
||||
private Thread? _mixThread;
|
||||
private volatile bool _running;
|
||||
private VoiceCatClient? _client;
|
||||
private uint _streamId;
|
||||
|
||||
private sealed class Input
|
||||
{
|
||||
internal readonly PcmRing Ring = new(16384);
|
||||
private readonly short[] stereo = new short[1920];
|
||||
internal void Feed(short[] pcm, int channels)
|
||||
{
|
||||
if (channels == 2) { Ring.TryWrite(pcm); return; }
|
||||
if (channels != 1 || pcm.Length > 960) return;
|
||||
for (int i = 0; i < pcm.Length; i++) stereo[2 * i] = stereo[2 * i + 1] = pcm[i];
|
||||
Ring.TryWrite(stereo.AsSpan(0, pcm.Length * 2));
|
||||
}
|
||||
}
|
||||
private readonly List<ProcessLoopbackCapture> captures = [];
|
||||
private Input[] inputs = [];
|
||||
private Thread? thread;
|
||||
private volatile bool running;
|
||||
private VoiceCatClient? client;
|
||||
private uint streamId;
|
||||
public void Start(AppAudioScope scope, VoiceCatClient client, uint streamId)
|
||||
{
|
||||
if (_running) return;
|
||||
_client = client;
|
||||
_streamId = streamId;
|
||||
|
||||
if (running) return;
|
||||
this.client = client; this.streamId = streamId;
|
||||
var specs = ResolveCaptures(scope);
|
||||
if (specs.Count == 0)
|
||||
inputs = specs.Select(_ => new Input()).ToArray();
|
||||
try
|
||||
{
|
||||
// nothing to capture — scope resolved to empty set
|
||||
return;
|
||||
}
|
||||
|
||||
lock (_frameLock)
|
||||
{
|
||||
_latestFrames = new List<short[]>(new short[specs.Count][]);
|
||||
_activeChannels = Channels;
|
||||
}
|
||||
|
||||
for (int i = 0; i < specs.Count; i++)
|
||||
{
|
||||
int captureIndex = i;
|
||||
var (pid, mode) = specs[i];
|
||||
var cap = new ProcessLoopbackCapture(pid, mode);
|
||||
cap.PcmFrameReady += (pcm, spc, ch) => OnCaptureFrame(captureIndex, pcm, ch);
|
||||
_captures.Add(cap);
|
||||
Input input = inputs[i]; var (pid, mode) = specs[i];
|
||||
var capture = new ProcessLoopbackCapture(pid, mode);
|
||||
capture.PcmFrameReady += (pcm, _, channels) => input.Feed(pcm, channels);
|
||||
captures.Add(capture);
|
||||
if (!capture.Start()) throw new InvalidOperationException("Process audio capture could not start.");
|
||||
}
|
||||
|
||||
foreach (var c in _captures) c.Start();
|
||||
|
||||
_running = true;
|
||||
_mixThread = new Thread(MixLoop) { IsBackground = true, Name = "ProcessAudioMixer" };
|
||||
_mixThread.Start();
|
||||
if (inputs.Length == 0) return;
|
||||
running = true;
|
||||
thread = new Thread(MixLoop) { IsBackground = true, Name = "ProcessAudioMixer" }; thread.Start();
|
||||
}
|
||||
catch { Stop(); throw; }
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
_running = false;
|
||||
_mixThread?.Join(500);
|
||||
foreach (var c in _captures) { c.Stop(); c.Dispose(); }
|
||||
_captures.Clear();
|
||||
running = false;
|
||||
if (thread is not null && !thread.Join(5000)) throw new TimeoutException("Process audio mixer did not stop.");
|
||||
foreach (var capture in captures) capture.Dispose();
|
||||
captures.Clear(); inputs = []; thread = null;
|
||||
}
|
||||
|
||||
public void Dispose() => Stop();
|
||||
|
||||
// ── Capture callback ──────────────────────────────────────────────────────
|
||||
|
||||
private void OnCaptureFrame(int index, short[] pcm, int channels)
|
||||
{
|
||||
lock (_frameLock)
|
||||
{
|
||||
// Upmix mono → stereo interleave if the capture fell back to mono.
|
||||
if (channels == 1 && _activeChannels == 2)
|
||||
pcm = MonoToStereo(pcm);
|
||||
if (index < _latestFrames.Count)
|
||||
_latestFrames[index] = pcm;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Mix loop (20 ms timer) ────────────────────────────────────────────────
|
||||
|
||||
private void MixLoop()
|
||||
{
|
||||
// Use a target period close to 20 ms; small under-shoot avoids accumulating drift.
|
||||
const int periodMs = 19;
|
||||
while (_running)
|
||||
var frame = new short[1920]; var mix = new short[1920]; var sums = new int[1920];
|
||||
long deadline = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
while (running)
|
||||
{
|
||||
Thread.Sleep(periodMs);
|
||||
if (!_running) break;
|
||||
|
||||
short[] mix;
|
||||
lock (_frameLock)
|
||||
sums.AsSpan().Clear();
|
||||
foreach (Input input in inputs)
|
||||
{
|
||||
int len = FrameSamples * _activeChannels;
|
||||
mix = new short[len];
|
||||
|
||||
foreach (var frame in _latestFrames)
|
||||
{
|
||||
if (frame == null) continue;
|
||||
int frameLen = Math.Min(frame.Length, len);
|
||||
for (int i = 0; i < frameLen; i++)
|
||||
{
|
||||
int sum = mix[i] + frame[i];
|
||||
mix[i] = (short)Math.Clamp(sum, short.MinValue, short.MaxValue);
|
||||
while (input.Ring.Count > 11520) input.Ring.Read(frame);
|
||||
frame.AsSpan().Clear(); input.Ring.Read(frame);
|
||||
for (int i = 0; i < frame.Length; i++) sums[i] += frame[i];
|
||||
}
|
||||
for (int i = 0; i < mix.Length; i++) mix[i] = (short)Math.Clamp(sums[i], short.MinValue, short.MaxValue);
|
||||
client?.StreamFeedPcm(streamId, mix, 960, 2);
|
||||
deadline += System.Diagnostics.Stopwatch.Frequency / 50;
|
||||
double wait = (deadline - System.Diagnostics.Stopwatch.GetTimestamp()) * 1000.0 / System.Diagnostics.Stopwatch.Frequency;
|
||||
if (wait > 0) Thread.Sleep((int)Math.Ceiling(wait));
|
||||
else if (wait < -100) deadline = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
_client?.StreamFeedPcm(_streamId, mix, FrameSamples, (uint)_activeChannels);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
// Resolve the scope to the WASAPI captures to open:
|
||||
// OnlyApps → one INCLUDE capture per selected process tree.
|
||||
// AllExceptApps → one EXCLUDE capture of the single selected process tree, which
|
||||
// natively captures the whole system render mix minus that tree
|
||||
// (dynamic — apps launched later are included automatically).
|
||||
private static List<(int pid, ProcessLoopbackCapture.Mode mode)> ResolveCaptures(AppAudioScope scope)
|
||||
{
|
||||
return scope switch
|
||||
private static List<(int pid, ProcessLoopbackCapture.Mode mode)> ResolveCaptures(AppAudioScope scope) => scope switch
|
||||
{
|
||||
OnlyApps o => o.Pids.Select(p => (p, ProcessLoopbackCapture.Mode.Include)).ToList(),
|
||||
AllExceptApps a when a.Pids.Count > 0 =>
|
||||
[(a.Pids[0], ProcessLoopbackCapture.Mode.Exclude)],
|
||||
// Entire desktop minus VoiceCat itself: EXCLUDE our own process tree.
|
||||
EntireDesktop { ExcludeSelf: true } =>
|
||||
[(Environment.ProcessId, ProcessLoopbackCapture.Mode.Exclude)],
|
||||
AllExceptApps a when a.Pids.Count > 0 => [(a.Pids[0], ProcessLoopbackCapture.Mode.Exclude)],
|
||||
EntireDesktop { ExcludeSelf: true } => [(Environment.ProcessId, ProcessLoopbackCapture.Mode.Exclude)],
|
||||
_ => [],
|
||||
};
|
||||
}
|
||||
|
||||
private static short[] MonoToStereo(short[] mono)
|
||||
{
|
||||
var stereo = new short[mono.Length * 2];
|
||||
for (int i = 0; i < mono.Length; i++)
|
||||
{
|
||||
stereo[i * 2] = mono[i];
|
||||
stereo[i * 2 + 1] = mono[i];
|
||||
}
|
||||
return stereo;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -64,15 +64,15 @@ public sealed class ProcessLoopbackCapture : IDisposable
|
||||
{
|
||||
_running = false;
|
||||
_bufferEvent?.Set();
|
||||
_captureThread?.Join(500);
|
||||
if (_audioClientPtr != IntPtr.Zero) AC_Stop(_audioClientPtr);
|
||||
if (_captureThread is not null && !_captureThread.Join(5000))
|
||||
throw new TimeoutException("Process capture did not stop.");
|
||||
}
|
||||
|
||||
private int _disposed;
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
|
||||
Stop();
|
||||
ComRelease(ref _captureClientPtr);
|
||||
ComRelease(ref _audioClientPtr);
|
||||
_bufferEvent?.Dispose();
|
||||
_initDone.Dispose();
|
||||
}
|
||||
@@ -80,11 +80,24 @@ public sealed class ProcessLoopbackCapture : IDisposable
|
||||
// ── Capture thread (MTA) ──────────────────────────────────────────────────
|
||||
|
||||
private void CaptureThreadProc()
|
||||
{
|
||||
try
|
||||
{
|
||||
_initOk = ActivateAndStart();
|
||||
_initDone.Set();
|
||||
if (!_initOk) return;
|
||||
CaptureLoop();
|
||||
if (_initOk && _running) CaptureLoop();
|
||||
}
|
||||
catch
|
||||
{
|
||||
_initOk = false;
|
||||
_initDone.Set();
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (_audioClientPtr != IntPtr.Zero) AC_Stop(_audioClientPtr);
|
||||
ComRelease(ref _captureClientPtr);
|
||||
ComRelease(ref _audioClientPtr);
|
||||
}
|
||||
}
|
||||
|
||||
private bool ActivateAndStart()
|
||||
@@ -274,9 +287,9 @@ public sealed class ProcessLoopbackCapture : IDisposable
|
||||
|
||||
private void FlushFrame()
|
||||
{
|
||||
var copy = new short[_accumBuf.Length];
|
||||
_accumBuf.AsSpan().CopyTo(copy);
|
||||
PcmFrameReady?.Invoke(copy, FrameSamples, _channels);
|
||||
// The callback borrows this buffer until it returns. This capture owner cannot
|
||||
// overwrite it concurrently, which avoids one allocation every 20 ms.
|
||||
PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels);
|
||||
_accumCount = 0;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
using System.Runtime.InteropServices;
|
||||
using VoiceCat.Audio;
|
||||
using static VoiceCat.App.Audio.InputDeviceCapture;
|
||||
|
||||
namespace VoiceCat.App.Audio;
|
||||
|
||||
public sealed class WasapiAudioBackend : IAudioDeviceBackend
|
||||
{
|
||||
public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => InputDeviceEnumerator.List(input).Select(d => new AudioDeviceInfo(d.Id, d.Name, d.IsDefault)).ToArray();
|
||||
public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler handler)
|
||||
{
|
||||
var capture = new InputDeviceCapture(NormalizeDeviceId(deviceId), loopback);
|
||||
capture.PcmFrameReady += (pcm, _, channels) => handler(pcm, channels);
|
||||
if (!capture.Start()) { capture.Dispose(); throw new InvalidOperationException("WASAPI capture could not start."); }
|
||||
return new Capture(capture);
|
||||
}
|
||||
public IAudioPlayback OpenPlayback(string? deviceId = null) => new Playback(deviceId);
|
||||
// The old miniaudio ABI persisted its raw device union as hex. Its Windows member
|
||||
// is a null-terminated UTF-16 WASAPI endpoint id; preserve saved selections.
|
||||
internal static string? NormalizeDeviceId(string? id)
|
||||
{
|
||||
if (string.IsNullOrEmpty(id)) return null;
|
||||
if (id.StartsWith('{')) return id;
|
||||
try
|
||||
{
|
||||
byte[] bytes = Convert.FromHexString(id);
|
||||
string decoded = System.Text.Encoding.Unicode.GetString(bytes).Split('\0')[0];
|
||||
return decoded.StartsWith('{') ? decoded : null;
|
||||
}
|
||||
catch (FormatException) { return id; }
|
||||
}
|
||||
private sealed class Capture(InputDeviceCapture capture) : IAudioCapture { public void Dispose() => capture.Dispose(); }
|
||||
|
||||
private sealed class Playback : IAudioPlayback
|
||||
{
|
||||
private readonly string? deviceId;
|
||||
private readonly PcmRing pcm = new(32768);
|
||||
private readonly ManualResetEventSlim initialized = new(false);
|
||||
private readonly Thread thread;
|
||||
private volatile bool running = true;
|
||||
private bool ready;
|
||||
private int disposed;
|
||||
internal Playback(string? deviceId)
|
||||
{
|
||||
this.deviceId = deviceId;
|
||||
thread = new Thread(Work) { IsBackground = true, Name = "VoiceCat WASAPI playback" };
|
||||
thread.Start();
|
||||
if (!initialized.Wait(5000) || !ready) { Dispose(); throw new InvalidOperationException("WASAPI playback could not start."); }
|
||||
}
|
||||
public void Write(ReadOnlySpan<short> stereoPcm) => pcm.TryWrite(stereoPcm);
|
||||
private unsafe void Work()
|
||||
{
|
||||
IMMDeviceEnumerator? enumerator = null; IMMDevice? device = null; IAudioClient? client = null; IAudioRenderClient? render = null;
|
||||
using var bufferReady = new AutoResetEvent(false);
|
||||
try
|
||||
{
|
||||
enumerator = (IMMDeviceEnumerator)Activator.CreateInstance(Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
|
||||
int result = deviceId is null ? enumerator.GetDefaultAudioEndpoint(0, 0, out device) : enumerator.GetDevice(deviceId, out device);
|
||||
if (result < 0 || device is null) return;
|
||||
Guid iid = typeof(IAudioClient).GUID;
|
||||
if (device.Activate(ref iid, 0x17, 0, out object audio) < 0) return;
|
||||
client = (IAudioClient)audio;
|
||||
WaveFormat format = new() { Format = 1, Channels = 2, Samples = 48000, Bytes = 192000, Align = 4, Bits = 16 };
|
||||
if (client.Initialize(0, 0x00040000u | 0x80000000u | 0x08000000u, 600000, 0, (nint)(&format), 0) < 0) return;
|
||||
if (client.GetBufferSize(out uint capacity) < 0 || client.SetEventHandle(bufferReady.SafeWaitHandle.DangerousGetHandle()) < 0) return;
|
||||
iid = typeof(IAudioRenderClient).GUID;
|
||||
if (client.GetService(ref iid, out object output) < 0) return;
|
||||
render = (IAudioRenderClient)output;
|
||||
if (client.Start() < 0) return;
|
||||
ready = true; initialized.Set();
|
||||
Span<short> discard = stackalloc short[1920];
|
||||
while (running)
|
||||
{
|
||||
bufferReady.WaitOne(20); // Scheduling wait is outside the buffer-fill cycle.
|
||||
if (!running || client.GetCurrentPadding(out uint padding) < 0) break;
|
||||
uint frames = capacity - Math.Min(capacity, padding);
|
||||
if (frames == 0) continue;
|
||||
if (render.GetBuffer(frames, out nint buffer) < 0) break;
|
||||
var destination = new Span<short>((void*)buffer, checked((int)frames * 2));
|
||||
// Keep queued playback bounded to ~120 ms, then fill underflow with silence.
|
||||
while (pcm.Count > 11520) pcm.Read(discard);
|
||||
int count = pcm.Read(destination); destination[count..].Clear();
|
||||
if (render.ReleaseBuffer(frames, 0) < 0) break;
|
||||
}
|
||||
}
|
||||
catch { ready = false; }
|
||||
finally
|
||||
{
|
||||
initialized.Set();
|
||||
try { client?.Stop(); } catch { }
|
||||
if (render is not null) Marshal.ReleaseComObject(render);
|
||||
if (client is not null) Marshal.ReleaseComObject(client);
|
||||
if (device is not null) Marshal.ReleaseComObject(device);
|
||||
if (enumerator is not null) Marshal.ReleaseComObject(enumerator);
|
||||
}
|
||||
}
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
|
||||
running = false;
|
||||
if (!thread.Join(5000)) throw new TimeoutException("WASAPI playback did not stop.");
|
||||
initialized.Dispose();
|
||||
}
|
||||
}
|
||||
[StructLayout(LayoutKind.Sequential, Pack = 2)]
|
||||
private struct WaveFormat { internal ushort Format, Channels; internal uint Samples, Bytes; internal ushort Align, Bits, Extra; }
|
||||
[ComImport, Guid("F294ACFC-3146-4483-A7BF-ADDCA7C260E2"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
|
||||
private interface IAudioRenderClient
|
||||
{
|
||||
[PreserveSig] int GetBuffer(uint frames, out nint data);
|
||||
[PreserveSig] int ReleaseBuffer(uint frames, uint flags);
|
||||
}
|
||||
}
|
||||
@@ -119,7 +119,7 @@ public partial class ConnectDialog : Form
|
||||
Directory.CreateDirectory(tofuDir);
|
||||
|
||||
_client = new VoiceCatClient("VoiceCat-Windows", VoiceCatClient.VersionString,
|
||||
VcLogLevel.Info, ServerListStore.TofuStorePath);
|
||||
VcLogLevel.Info, ServerListStore.TofuStorePath, new VoiceCat.App.Audio.WasapiAudioBackend());
|
||||
_client.EventReceived += OnEvent;
|
||||
_identityDialogShown = false;
|
||||
_pumpTimer.Start();
|
||||
|
||||
@@ -5,7 +5,7 @@ namespace VoiceCat.App;
|
||||
internal static class Program
|
||||
{
|
||||
[STAThread]
|
||||
private static void Main()
|
||||
private static int Main(string[] args)
|
||||
{
|
||||
// Surface exceptions that WinForms' default message-loop handling would otherwise
|
||||
// swallow silently (or crash with no visible cause).
|
||||
@@ -16,12 +16,37 @@ internal static class Program
|
||||
|
||||
ApplicationConfiguration.Initialize();
|
||||
|
||||
if (args.Contains("--smoke-test"))
|
||||
{
|
||||
try
|
||||
{
|
||||
using var client = new VoiceCat.Interop.VoiceCatClient("Smoke", "test");
|
||||
using var form = new MainForm(client, 0, "Smoke", "Managed core");
|
||||
form.CreateControl();
|
||||
if (form.Controls.Count == 0 || string.IsNullOrEmpty(form.Text)) return 1;
|
||||
client.PumpEvents();
|
||||
if (args.Contains("--audio"))
|
||||
{
|
||||
var backend = new Audio.WasapiAudioBackend();
|
||||
using var playback = backend.OpenPlayback();
|
||||
int samples = 0;
|
||||
using var capture = backend.OpenCapture(null, false, (pcm, _) => Interlocked.Add(ref samples, pcm.Length));
|
||||
short[] silence = new short[1920];
|
||||
for (int i = 0; i < 150; i++) { playback.Write(silence); Thread.Sleep(20); }
|
||||
if (samples == 0) return 2;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
catch { return 1; }
|
||||
}
|
||||
|
||||
using var connectDialog = new ConnectDialog();
|
||||
var result = connectDialog.ShowDialog();
|
||||
if (result != DialogResult.OK || connectDialog.ConnectedClient is null)
|
||||
return;
|
||||
return 0;
|
||||
|
||||
Application.Run(new MainForm(connectDialog.ConnectedClient, connectDialog.SelfUserId,
|
||||
connectDialog.Nickname, connectDialog.ServerName));
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\VoiceCat.Interop\VoiceCat.Interop.csproj" />
|
||||
<ProjectReference Include="..\VoiceCat.Managed\VoiceCat.Managed.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
<!-- Prismatoid — .NET bindings for the Prism speech library, used for spoken event
|
||||
@@ -42,18 +42,6 @@
|
||||
as part of the Windows Desktop shared framework on net10.0-windows — no PackageReference
|
||||
needed (one was tried and NuGet flagged it as redundant/unprunable, NU1510). -->
|
||||
|
||||
<!-- voicecat.dll must exist (build the `windows-client` CMake preset first — see
|
||||
clients/windows/README.md). -->
|
||||
<ItemGroup>
|
||||
<Content Include="$(VoiceCatNativeDir)\voicecat.dll" Condition="Exists('$(VoiceCatNativeDir)\voicecat.dll')">
|
||||
<Link>voicecat.dll</Link>
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</Content>
|
||||
</ItemGroup>
|
||||
|
||||
<Target Name="VoiceCatCheckNativeDll" BeforeTargets="Build">
|
||||
<Error Condition="!Exists('$(VoiceCatNativeDir)\voicecat.dll')"
|
||||
Text="voicecat.dll not found at '$(VoiceCatNativeDir)'. Build it first: cmake --preset windows-client && cmake --build --preset windows-client (see clients/windows/README.md)." />
|
||||
</Target>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0-windows7.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
},
|
||||
"Prismatoid": {
|
||||
"type": "Direct",
|
||||
"requested": "[0.3.0, )",
|
||||
"resolved": "0.3.0",
|
||||
"contentHash": "mUOgtmFtjbLxydrdAImkfP3u0U8qrYkvfx2NlQqdQQ8TB9qe59BobxBCcNDo21g8xSefkBJB1VdGPbVTfiE7zw=="
|
||||
},
|
||||
"BouncyCastle.Cryptography": {
|
||||
"type": "Transitive",
|
||||
"resolved": "2.6.2",
|
||||
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.core": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Audio": "[1.0.0, )",
|
||||
"VoiceCat.Crypto": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"BouncyCastle.Cryptography": "[2.6.2, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.managed": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Core": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
},
|
||||
"net10.0-windows7.0/win-x64": {
|
||||
"Prismatoid": {
|
||||
"type": "Direct",
|
||||
"requested": "[0.3.0, )",
|
||||
"resolved": "0.3.0",
|
||||
"contentHash": "mUOgtmFtjbLxydrdAImkfP3u0U8qrYkvfx2NlQqdQQ8TB9qe59BobxBCcNDo21g8xSefkBJB1VdGPbVTfiE7zw=="
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -272,6 +272,8 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
|
||||
Assert.Equal(VcResult.Ok, events.First(e => e.Type == VcEventType.AuthResult).Result);
|
||||
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ChannelList), 3000));
|
||||
|
||||
Assert.Equal(VcResult.Ok, client.JoinVoice());
|
||||
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
|
||||
// Give the async UDP binding handshake a moment to land before announcing a stream
|
||||
// (mirrors vccli's 500ms sleep after auth).
|
||||
Thread.Sleep(500);
|
||||
@@ -340,6 +342,10 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
|
||||
Assert.True(PumpUntil(b, () => eventsB.Any(e => e.Type == VcEventType.JoinResult), 5000),
|
||||
"B did not receive VC_EVENT_JOIN_RESULT");
|
||||
|
||||
Assert.Equal(VcResult.Ok, a.JoinVoice());
|
||||
Assert.Equal(VcResult.Ok, b.JoinVoice());
|
||||
Assert.True(PumpUntil(a, () => eventsA.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
|
||||
Assert.True(PumpUntil(b, () => eventsB.Any(e => e.Type == VcEventType.VoiceState && e.U32a == 1), 5000));
|
||||
// UDP binding handshake is async; give it a moment (mirrors ScreenAudio test).
|
||||
Thread.Sleep(500);
|
||||
|
||||
|
||||
@@ -9,5 +9,6 @@
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
<RootNamespace>VoiceCat.Interop</RootNamespace>
|
||||
</PropertyGroup>
|
||||
<ItemGroup><InternalsVisibleTo Include="VoiceCat.Interop.Tests" /></ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Interop;
|
||||
|
||||
public sealed partial class VoiceCatClient
|
||||
{
|
||||
public VcResult KickUser(uint id, string? reason = null) => Request(new() { Kick = new() { UserId = id, Reason = reason ?? "" } });
|
||||
public VcResult BanUser(uint id, string? reason = null, ulong expiresUnixMs = 0) => Request(new() { Ban = new() { UserId = id, Reason = reason ?? "", ExpiresUnixMs = expiresUnixMs } });
|
||||
public VcResult MoveUser(uint id, uint channelId) => Request(new() { MoveUser = new() { UserId = id, ChannelId = channelId } });
|
||||
public VcResult SetServerMute(uint id, bool muted, bool deafened) => Request(new() { ServerMute = new() { UserId = id, Muted = muted, Deafened = deafened } });
|
||||
public VcResult SetPermission(uint id, PermissionsInfo permissions) => Request(new() { SetPermission = new() { UserId = id, Permissions = new()
|
||||
{ IsAdmin = permissions.IsAdmin, CanCreateTempChannel = permissions.CanCreateTempChannel, CanAdminAccounts = permissions.CanAdminAccounts, CanBan = permissions.CanBan, CanKick = permissions.CanKick, CanMoveUsers = permissions.CanMoveUsers } } });
|
||||
public VcResult CreateAccount(string username, string password) => Request(new() { CreateAccount = new() { Username = username, Password = password } });
|
||||
public VcResult ResetPassword(string username, string password) => Request(new() { ResetPassword = new() { Username = username, NewPassword = password } });
|
||||
public VcResult DeleteAccount(string username) => Request(new() { DeleteAccount = new() { Username = username } });
|
||||
public VcResult RequestAccountList() => Request(new() { ListAccounts = new() });
|
||||
public VcResult CreateChannel(ChannelEditInfo info) => Request(new() { CreateChannel = new() { Channel = Channel(info), Password = info.Password ?? "" } });
|
||||
public VcResult EditChannel(ChannelEditInfo info) => Request(new() { EditChannel = new() { Channel = Channel(info), Password = info.Password ?? "" } });
|
||||
public VcResult DeleteChannel(uint id) => Request(new() { DeleteChannel = new() { ChannelId = id } });
|
||||
public VcResult SendText(VcTextScope scope, uint targetId, string body)
|
||||
{
|
||||
try { core.Send(new() { TextMessage = new() { Scope = (TextScope)scope, TargetId = targetId, Body = body, ClientMsgId = Guid.NewGuid().ToString("N") } }); return VcResult.Ok; }
|
||||
catch { return VcResult.NotConnected; }
|
||||
}
|
||||
private static Voicecat.V1.Channel Channel(ChannelEditInfo info) => new()
|
||||
{
|
||||
Id = info.Id, ParentId = info.ParentId, Name = info.Name, Topic = info.Topic, MaxUsers = info.MaxUsers, Order = unchecked((int)info.SortOrder),
|
||||
Audio = new() { Codec = info.Audio.Codec, Mode = info.Audio.Stereo ? ChannelMode.ModeStereo : ChannelMode.ModeMono, SampleRate = info.Audio.SampleRate,
|
||||
BitrateBps = info.Audio.BitrateBps, FrameMs = info.Audio.FrameMs, Application = (OpusApplication)info.Audio.Application, Complexity = info.Audio.Complexity,
|
||||
Fec = info.Audio.Fec, ExpectedPacketLoss = info.Audio.ExpectedPacketLoss, Dtx = info.Audio.Dtx, Dred = info.Audio.Dred }
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
using VoiceCat.Audio;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Interop;
|
||||
|
||||
public sealed partial class VoiceCatClient
|
||||
{
|
||||
private sealed record PcmRegistration(nint Callback, nint User);
|
||||
private PcmRegistration? pcm;
|
||||
public (VcResult Result, uint StreamId) StartStream(VcStreamKind kind, string label) => Start(kind, label, false);
|
||||
public (VcResult Result, uint StreamId) StartStreamExternalFeed(VcStreamKind kind, string label) => Start(kind, label, true);
|
||||
private (VcResult, uint) Start(VcStreamKind kind, string label, bool external)
|
||||
{
|
||||
StreamInfo? stream = null;
|
||||
try
|
||||
{
|
||||
stream = core.StartStreamAsync((StreamKind)kind, label).GetAwaiter().GetResult();
|
||||
local[stream.StreamId] = new(stream, external);
|
||||
if (!external && backend is not null) captures[stream.StreamId] = Capture(stream);
|
||||
return (VcResult.Ok, stream.StreamId);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
if (stream is not null) { local.TryRemove(stream.StreamId, out _); core.StopStream(stream.StreamId); }
|
||||
Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); return (VcResult.Audio, 0);
|
||||
}
|
||||
}
|
||||
private IAudioCapture Capture(StreamInfo stream) => backend!.OpenCapture(devices.GetValueOrDefault(stream.StreamId), stream.Kind == StreamKind.StreamScreenAudio,
|
||||
(samples, channels) => StreamFeedPcm(stream.StreamId, samples, samples.Length / channels, (uint)channels));
|
||||
public VcResult StopStream(uint id)
|
||||
{
|
||||
if (captures.Remove(id, out var capture)) capture.Dispose();
|
||||
if (!local.TryRemove(id, out LocalStream? stream)) return VcResult.InvalidArg;
|
||||
try { core.StopStream(stream.Info.StreamId); return VcResult.Ok; } catch { return VcResult.NotConnected; }
|
||||
}
|
||||
public VcResult SetInputDevice(uint id, string? device)
|
||||
{
|
||||
devices[id] = device;
|
||||
if (!captures.Remove(id, out var previous)) return VcResult.Ok;
|
||||
previous.Dispose();
|
||||
try { var info = local[id].Info.Clone(); info.StreamId = id; captures[id] = Capture(info); return VcResult.Ok; }
|
||||
catch { return VcResult.Audio; }
|
||||
}
|
||||
public VcResult SetCaptureChannels(uint id, uint channels)
|
||||
{ try { LocalStream stream = local[id]; core.Audio.SetCaptureChannels(stream.Info.StreamId, checked((int)channels)); stream.CaptureChannels = (int)channels; return VcResult.Ok; } catch { return VcResult.InvalidArg; } }
|
||||
public VcResult AudioRestart()
|
||||
{
|
||||
foreach (uint id in captures.Keys.ToArray()) { VcResult result = SetInputDevice(id, devices.GetValueOrDefault(id)); if (result != VcResult.Ok) return result; }
|
||||
return VcResult.Ok;
|
||||
}
|
||||
public VcResult SetInputMode(VcInputMode mode) { if (!Enum.IsDefined(mode)) return VcResult.InvalidArg; core.Audio.InputMode = (AudioInputMode)mode; return VcResult.Ok; }
|
||||
public VcResult SetVadThreshold(float threshold) { if (!float.IsFinite(threshold) || threshold is < 0 or > 1) return VcResult.InvalidArg; core.Audio.VadThreshold = threshold; return VcResult.Ok; }
|
||||
public VcResult SetPushToTalk(bool active) { core.Audio.PushToTalk = active; return VcResult.Ok; }
|
||||
public VcResult SetSelfMute(bool muted, bool deafened) { core.Audio.MicMuted = muted; core.Audio.Deafened = deafened; return VcResult.Ok; }
|
||||
public VcResult SetOutputVolume(float gain) { if (!float.IsFinite(gain) || gain is < 0 or > 4) return VcResult.InvalidArg; core.Audio.OutputGain = gain; return VcResult.Ok; }
|
||||
public VcResult SetInputGain(float gain) { if (!float.IsFinite(gain) || gain is < 0 or > 4) return VcResult.InvalidArg; core.Audio.InputGain = gain; return VcResult.Ok; }
|
||||
public VcResult SetInputNoiseReduction(bool enabled) { core.Audio.InputNoiseReduction = enabled; return VcResult.Ok; }
|
||||
public VcResult SetRemoteStream(uint userId, uint streamId, float gain, bool muted, bool nr)
|
||||
{ try { core.Audio.SetRemotePlayback(userId, streamId, gain, muted, nr); return VcResult.Ok; } catch { return VcResult.InvalidArg; } }
|
||||
public (VcResult Result, RemoteStreamState? State) GetRemoteStream(uint userId, uint streamId)
|
||||
{
|
||||
var state = core.Audio.GetRemotePlayback(userId, streamId);
|
||||
return state is { } value ? (VcResult.Ok, new(value.Gain, value.Muted, value.NoiseReduction)) : (VcResult.InvalidArg, null);
|
||||
}
|
||||
public (VcResult Result, AudioConfigInfo? Config) GetStreamAudioConfig(uint userId, uint streamId)
|
||||
{
|
||||
var info = core.Users.FirstOrDefault(u => u.Id == userId)?.Streams.FirstOrDefault(s => s.StreamId == streamId);
|
||||
if (core.Authentication?.Self.Id == userId && local.TryGetValue(streamId, out LocalStream? own)) info = own.Info;
|
||||
return info is null ? (VcResult.InvalidArg, null) : (VcResult.Ok, Audio(info.Audio));
|
||||
}
|
||||
public VcResult StreamFeedPcm(uint id, ReadOnlySpan<short> samples, int samplesPerChannel, uint channels)
|
||||
{
|
||||
if (samplesPerChannel < 0 || channels is not (1 or 2) || samples.Length != (long)samplesPerChannel * channels) return VcResult.InvalidArg;
|
||||
if (!local.TryGetValue(id, out LocalStream? stream)) return VcResult.InvalidArg;
|
||||
core.Audio.FeedPcm(Volatile.Read(ref stream.Info).StreamId, samples, (int)channels); return VcResult.Ok; // A full real-time ring drops, never waits.
|
||||
}
|
||||
public VcResult SetPcmSink(nint callback, nint user) { Volatile.Write(ref pcm, callback == 0 ? null : new(callback, user)); return VcResult.Ok; }
|
||||
private unsafe void ForwardPcm(uint userId, uint streamId, ReadOnlySpan<short> samples, int channels)
|
||||
{
|
||||
var target = Volatile.Read(ref pcm); if (target is null) return;
|
||||
fixed (short* input = samples)
|
||||
((delegate* unmanaged[Cdecl]<nint, uint, uint, short*, nuint, uint, uint, void>)target.Callback)(target.User, userId, streamId, input, (nuint)(samples.Length / channels), (uint)channels, 48000);
|
||||
}
|
||||
|
||||
// Server-authoritative moves/edits stop old SSRCs. Reannounce with the new channel
|
||||
// configuration while keeping UI/external-feed ids stable; captures continue feeding
|
||||
// the alias and switch to the new audio owner only when negotiation completes.
|
||||
private void ReconcileStreams()
|
||||
{
|
||||
var active = core.LocalStreams;
|
||||
foreach (LocalStream stream in local.Values)
|
||||
if (!active.Any(s => s.StreamId == stream.Info.StreamId) && Interlocked.CompareExchange(ref stream.Restarting, 1, 0) == 0) _ = RestartAsync(stream);
|
||||
}
|
||||
private async Task RestartAsync(LocalStream stream)
|
||||
{
|
||||
try
|
||||
{
|
||||
StreamInfo previous = stream.Info;
|
||||
StreamInfo next = await core.StartStreamAsync(previous.Kind, previous.Label, stream.CaptureChannels).ConfigureAwait(false);
|
||||
if (!local.ContainsKey(stream.Alias)) core.StopStream(next.StreamId);
|
||||
else Volatile.Write(ref stream.Info, next);
|
||||
}
|
||||
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); }
|
||||
finally { Volatile.Write(ref stream.Restarting, 0); }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="../../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" />
|
||||
<Compile Include="../VoiceCat.Interop/Enums.cs" Link="Enums.cs" />
|
||||
<Compile Include="../VoiceCat.Interop/Models.cs" Link="Models.cs" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -0,0 +1,193 @@
|
||||
using System.Runtime.InteropServices;
|
||||
using System.Threading.Channels;
|
||||
using System.Collections.Concurrent;
|
||||
using VoiceCat.Audio;
|
||||
using VoiceCat.Core;
|
||||
using VoiceCat.Crypto;
|
||||
using Voicecat.V1;
|
||||
using CoreClient = VoiceCat.Core.VoiceCatClient;
|
||||
|
||||
namespace VoiceCat.Interop;
|
||||
|
||||
// Preserves the shipped WinForms call/event surface while its implementation moves to
|
||||
// the async managed core. Events still reach controls only through PumpEvents on the UI thread.
|
||||
public sealed partial class VoiceCatClient : IDisposable
|
||||
{
|
||||
private readonly CoreClient core;
|
||||
private readonly IAudioDeviceBackend? backend;
|
||||
private readonly System.Threading.Channels.Channel<VoiceCatEvent> events = System.Threading.Channels.Channel.CreateUnbounded<VoiceCatEvent>();
|
||||
private readonly Dictionary<uint, IAudioCapture> captures = [];
|
||||
private readonly Dictionary<uint, string?> devices = [];
|
||||
private readonly Dictionary<uint, StreamSummary[]> remoteStreams = [];
|
||||
private readonly Dictionary<uint, bool> talkState = [];
|
||||
private sealed class LocalStream(StreamInfo info, bool external)
|
||||
{
|
||||
internal readonly uint Alias = info.StreamId;
|
||||
internal StreamInfo Info = info;
|
||||
internal readonly bool External = external;
|
||||
internal int CaptureChannels = 1;
|
||||
internal int Restarting;
|
||||
}
|
||||
private readonly ConcurrentDictionary<uint, LocalStream> local = new();
|
||||
private IAudioPlayback? playback;
|
||||
private Task connecting = Task.CompletedTask;
|
||||
private TaskCompletionSource<bool>? identity;
|
||||
private int disposed;
|
||||
private List<AccountInfo> accounts = [];
|
||||
private bool audioFailureReported;
|
||||
public event Action<VoiceCatEvent>? EventReceived;
|
||||
public event Action<uint, float>? LevelChanged;
|
||||
public CoreClient ManagedClient => core;
|
||||
|
||||
public VoiceCatClient(string clientName, string clientVersion, VcLogLevel logLevel = VcLogLevel.Info, string? tofuStorePath = null, IAudioDeviceBackend? audioBackend = null)
|
||||
{
|
||||
backend = audioBackend;
|
||||
core = new(clientName, clientVersion, tofuStorePath);
|
||||
core.ConnectionStateChanged += state =>
|
||||
{
|
||||
VcConnectionState mapped = state switch { ClientConnectionState.Connecting => VcConnectionState.Connecting, ClientConnectionState.VerifyingIdentity => VcConnectionState.VerifyingIdentity,
|
||||
ClientConnectionState.Authenticating => VcConnectionState.Authenticating, ClientConnectionState.Connected => VcConnectionState.Connected, _ => VcConnectionState.Disconnected };
|
||||
Queue(new(VcEventType.ConnectionState, ConnectionState: mapped));
|
||||
if (mapped == VcConnectionState.Disconnected) Queue(new(VcEventType.Disconnected));
|
||||
};
|
||||
core.Audio.MixedPcm += pcm => Volatile.Read(ref playback)?.Write(pcm);
|
||||
core.Audio.StreamPcm += ForwardPcm;
|
||||
}
|
||||
|
||||
private void Queue(VoiceCatEvent message) => events.Writer.TryWrite(message);
|
||||
public VcResult Connect(string host, ushort port)
|
||||
{
|
||||
if (!connecting.IsCompleted || core.State != ClientConnectionState.Disconnected) return VcResult.Already;
|
||||
connecting = ConnectAsync(host, port);
|
||||
return VcResult.Ok;
|
||||
}
|
||||
private async Task ConnectAsync(string host, ushort port)
|
||||
{
|
||||
try
|
||||
{
|
||||
await core.ConnectAsync(host, port, async (challenge, token) =>
|
||||
{
|
||||
identity = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
Queue(new(VcEventType.ServerIdentity, U32a: (uint)challenge.Status, Text: challenge.CertificateFingerprint));
|
||||
return await identity.Task.WaitAsync(token).ConfigureAwait(false);
|
||||
}).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Io, Text: exception.Message)); }
|
||||
}
|
||||
public VcResult ConfirmServerIdentity(bool accept) { identity?.TrySetResult(accept); return VcResult.Ok; }
|
||||
public VcResult AuthenticateGuest(string nickname) { _ = AuthenticateAsync(() => core.AuthenticateGuestAsync(nickname)); return VcResult.Ok; }
|
||||
public VcResult AuthenticateUser(string username, string password) { _ = AuthenticateAsync(() => core.AuthenticateUserAsync(username, password)); return VcResult.Ok; }
|
||||
private async Task AuthenticateAsync(Func<Task<AuthResult>> authenticate)
|
||||
{
|
||||
try { await connecting.ConfigureAwait(false); await authenticate().ConfigureAwait(false); }
|
||||
catch (Exception exception) { Queue(new(VcEventType.AuthResult, Result: VcResult.AuthFailed, Text: exception.Message)); }
|
||||
}
|
||||
public VcResult Disconnect()
|
||||
{
|
||||
StopDevices(); core.DisconnectAsync().GetAwaiter().GetResult(); return VcResult.Ok;
|
||||
}
|
||||
public string GetServerIdentityDisplay() => core.ServerHello is { } hello ? Convert.ToHexString(hello.ServerIdentityFingerprint.Span) : "";
|
||||
|
||||
public void PumpEvents()
|
||||
{
|
||||
if (!audioFailureReported && core.Audio.Failure is { } failure) { audioFailureReported = true; Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: failure.Message)); }
|
||||
while (core.TryReadEvent(out Envelope? message)) Translate(message!);
|
||||
while (events.Reader.TryRead(out VoiceCatEvent? message)) EventReceived?.Invoke(message);
|
||||
foreach (LocalStream stream in local.Values)
|
||||
{
|
||||
var info = Volatile.Read(ref stream.Info);
|
||||
var level = core.Audio.GetLocalLevel(info.StreamId); LevelChanged?.Invoke(stream.Alias, level.Level);
|
||||
if (talkState.GetValueOrDefault(stream.Alias) != level.Talking)
|
||||
{
|
||||
talkState[stream.Alias] = level.Talking;
|
||||
if (core.State == ClientConnectionState.Connected) core.Send(new() { StreamState = new() { StreamId = info.StreamId, Talking = level.Talking, Muted = core.Audio.MicMuted } });
|
||||
EventReceived?.Invoke(new(VcEventType.TalkState, UserId: core.Authentication?.Self.Id ?? 0, StreamId: stream.Alias, U32a: level.Talking ? 1U : 0));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void Translate(Envelope message)
|
||||
{
|
||||
if (message.AuthResult is not null) Queue(new(VcEventType.AuthResult, Result: message.AuthResult.Ok ? VcResult.Ok : VcResult.AuthFailed, UserId: message.AuthResult.Self?.Id ?? 0, Text: message.AuthResult.Error));
|
||||
if (message.ServerState is not null) { remoteStreams.Clear(); foreach (User user in message.ServerState.Users) UpdateStreams(user); Queue(new(VcEventType.ChannelList)); }
|
||||
if (message.ChannelEvent is not null) Queue(new(VcEventType.ChannelList));
|
||||
if (message.UserEvent is not null)
|
||||
{
|
||||
var change = message.UserEvent;
|
||||
if (change.User is { VoiceSubscribed: true } self && self.Id == core.Authentication?.Self.Id) ReconcileStreams();
|
||||
if (change.User is not null) UpdateStreams(change.User);
|
||||
if (change.Kind == UserEvent.Types.Kind.Left) remoteStreams.Remove(change.LeftId);
|
||||
Queue(new(change.Kind switch { UserEvent.Types.Kind.Joined => VcEventType.UserJoined, UserEvent.Types.Kind.Left => VcEventType.UserLeft, _ => VcEventType.UserUpdated }, UserId: change.User?.Id ?? change.LeftId,
|
||||
ChannelId: change.User?.ChannelId ?? 0, Text: change.Kind == UserEvent.Types.Kind.Joined ? change.User?.Nickname : change.Reason));
|
||||
}
|
||||
if (message.JoinChannelResult is not null) Queue(new(VcEventType.JoinResult, Result: message.JoinChannelResult.Ok ? VcResult.Ok : VcResult.InvalidArg, ChannelId: message.JoinChannelResult.ChannelId, Text: message.JoinChannelResult.Error));
|
||||
if (message.VoiceSubscriptionResult is not null) Queue(new(VcEventType.VoiceState, U32a: message.VoiceSubscriptionResult.Subscribed ? 1U : 0));
|
||||
if (message.TextMessage is not null) Queue(new(VcEventType.TextMessage, UserId: message.TextMessage.SenderId, ChannelId: message.TextMessage.TargetId, TextScope: (VcTextScope)message.TextMessage.Scope, Text: message.TextMessage.Body, TimestampUnixMs: message.TextMessage.SentAtUnixMs));
|
||||
if (message.StreamState is not null) Queue(new(VcEventType.TalkState, UserId: message.StreamState.UserId, StreamId: message.StreamState.UserId == core.Authentication?.Self.Id ? local.Values.FirstOrDefault(s => s.Info.StreamId == message.StreamState.StreamId)?.Alias ?? message.StreamState.StreamId : message.StreamState.StreamId, U32a: message.StreamState.Talking ? 1U : 0));
|
||||
if (message.GenericResult is not null) Queue(new(VcEventType.GenericResult, Result: message.GenericResult.Ok ? VcResult.Ok : message.GenericResult.Code == 6 ? VcResult.PermissionDenied : VcResult.InvalidArg, U32a: message.GenericResult.Code, Text: message.GenericResult.Message));
|
||||
if (message.ListAccountsResult is not null)
|
||||
{
|
||||
accounts = message.ListAccountsResult.Accounts.Select(a => new AccountInfo(a.Username, a.IsAdmin, a.CreatedAtUnixMs, a.LastLoginUnixMs)).ToList();
|
||||
Queue(new(VcEventType.AccountList));
|
||||
}
|
||||
if (message.Disconnect is not null) Queue(new(VcEventType.Error, Result: VcResult.Io, Text: message.Disconnect.Reason));
|
||||
}
|
||||
|
||||
private void UpdateStreams(User user)
|
||||
{
|
||||
StreamSummary[] previous = remoteStreams.GetValueOrDefault(user.Id, []);
|
||||
StreamSummary[] next = user.Id == core.Authentication?.Self.Id ? ListUserStreams(user.Id).ToArray() : user.Streams.Select(s => new StreamSummary(s.StreamId, (VcStreamKind)s.Kind, s.Label)).ToArray();
|
||||
foreach (var stream in previous) if (!next.Any(s => s.StreamId == stream.StreamId)) Queue(new(VcEventType.StreamStopped, UserId: user.Id, StreamId: stream.StreamId));
|
||||
foreach (var stream in next) if (!previous.Any(s => s.StreamId == stream.StreamId)) Queue(new(VcEventType.StreamStarted, UserId: user.Id, StreamId: stream.StreamId, U32a: (uint)stream.Kind, Text: stream.Label));
|
||||
remoteStreams[user.Id] = next;
|
||||
}
|
||||
|
||||
private VcResult Request(Envelope request)
|
||||
{
|
||||
if (core.State != ClientConnectionState.Connected) return VcResult.NotConnected;
|
||||
try { _ = RequestAsync(request); return VcResult.Ok; }
|
||||
catch { return VcResult.NotConnected; }
|
||||
}
|
||||
private async Task RequestAsync(Envelope request)
|
||||
{
|
||||
try { await core.RequestAsync(request).ConfigureAwait(false); }
|
||||
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Io, Text: exception.Message)); }
|
||||
}
|
||||
public VcResult JoinChannel(uint id, string? password = null) => Request(new() { JoinChannel = new() { ChannelId = id, Password = password ?? "" } });
|
||||
public VcResult LeaveChannel() => Request(new() { LeaveChannel = new() });
|
||||
public VcResult JoinVoice()
|
||||
{
|
||||
try
|
||||
{
|
||||
if (backend is not null && playback is null) playback = backend.OpenPlayback();
|
||||
var result = core.SubscribeVoiceAsync().GetAwaiter().GetResult();
|
||||
if (!result.Ok) { Interlocked.Exchange(ref playback, null)?.Dispose(); }
|
||||
return result.Ok ? VcResult.Ok : VcResult.Audio;
|
||||
}
|
||||
catch (Exception exception) { Queue(new(VcEventType.Error, Result: VcResult.Audio, Text: exception.Message)); return VcResult.Audio; }
|
||||
}
|
||||
public VcResult LeaveVoice() { StopDevices(); local.Clear(); return Request(new() { UnsubscribeVoice = new() }); }
|
||||
|
||||
public List<ChannelInfo> ListChannels() => core.Channels.Select(c => new ChannelInfo(c.Id, c.ParentId, c.Name, c.Topic, c.PasswordProtected, c.MaxUsers, unchecked((uint)c.Order), Audio(c.Audio))).ToList();
|
||||
public List<UserInfo> ListUsers() => core.Users.Select(u => new UserInfo(u.Id, u.Nickname, u.IsGuest, u.ChannelId, u.SelfMicMuted, u.SelfDeafened, u.ServerMuted, u.ServerDeafened, u.VoiceSubscribed)).ToList();
|
||||
public List<StreamSummary> ListUserStreams(uint id) => id == core.Authentication?.Self.Id
|
||||
? local.Values.Select(s => new StreamSummary(s.Alias, (VcStreamKind)s.Info.Kind, s.Info.Label)).ToList()
|
||||
: core.Users.FirstOrDefault(u => u.Id == id)?.Streams.Select(s => new StreamSummary(s.StreamId, (VcStreamKind)s.Kind, s.Label)).ToList() ?? [];
|
||||
public PermissionsInfo GetPermissions() { Permissions p = core.Authentication?.Permissions ?? new(); return new(p.CanCreateTempChannel, p.CanKick, p.CanBan, p.CanMoveUsers, p.CanAdminAccounts, p.IsAdmin); }
|
||||
public List<AccountInfo> ListAccounts() => accounts.ToList();
|
||||
public List<DeviceInfo> ListDevices(VcDeviceKind kind) => backend?.Enumerate(kind == VcDeviceKind.Input).Select(d => new DeviceInfo(d.Id, d.Name, d.IsDefault)).ToList() ?? [];
|
||||
public static string VersionString => "VoiceCat managed core 0.1.0 (protocol v2)";
|
||||
public static string ResultString(VcResult result) => result.ToString();
|
||||
private static AudioConfigInfo Audio(AudioConfig a) => new(a.Codec, a.Mode == ChannelMode.ModeStereo, a.SampleRate, a.BitrateBps, a.FrameMs, (uint)a.Application, a.Fec, a.ExpectedPacketLoss, a.Dtx, a.Complexity, a.Dred);
|
||||
|
||||
private void StopDevices()
|
||||
{
|
||||
foreach (var capture in captures.Values) capture.Dispose(); captures.Clear(); local.Clear();
|
||||
IAudioPlayback? previous = Interlocked.Exchange(ref playback, null); previous?.Dispose();
|
||||
}
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
|
||||
identity?.TrySetResult(false); StopDevices(); core.DisposeAsync().AsTask().GetAwaiter().GetResult();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
namespace VoiceCat.Interop;
|
||||
|
||||
// Compatibility event consumed by WinForms. The managed core owns protocol and audio;
|
||||
// this assembly contains no libvoicecat bindings or native handles.
|
||||
public sealed record VoiceCatEvent(VcEventType Type, VcConnectionState ConnectionState = VcConnectionState.Disconnected,
|
||||
VcResult Result = VcResult.Ok, uint UserId = 0, uint ChannelId = 0, uint StreamId = 0, VcTextScope TextScope = VcTextScope.Channel,
|
||||
uint U32a = 0, string? Text = null, ulong TimestampUnixMs = 0);
|
||||
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"BouncyCastle.Cryptography": {
|
||||
"type": "Transitive",
|
||||
"resolved": "2.6.2",
|
||||
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.core": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Audio": "[1.0.0, )",
|
||||
"VoiceCat.Crypto": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"BouncyCastle.Cryptography": "[2.6.2, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
},
|
||||
"BouncyCastle.Cryptography": {
|
||||
"type": "Transitive",
|
||||
"resolved": "2.6.2",
|
||||
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.core": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Audio": "[1.0.0, )",
|
||||
"VoiceCat.Crypto": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"BouncyCastle.Cryptography": "[2.6.2, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
},
|
||||
"net10.0/win-x64": {}
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,14 @@
|
||||
<Solution>
|
||||
<Folder Name="/Managed core/">
|
||||
<Project Path="../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" />
|
||||
<Project Path="../../dotnet/src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
|
||||
<Project Path="../../dotnet/src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
|
||||
<Project Path="../../dotnet/src/VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
|
||||
<Project Path="../../dotnet/src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
|
||||
<Project Path="../../dotnet/src/VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
|
||||
</Folder>
|
||||
<Project Path="VoiceCat.App/VoiceCat.App.csproj" />
|
||||
<Project Path="VoiceCat.Interop.Tests/VoiceCat.Interop.Tests.csproj" />
|
||||
<Project Path="VoiceCat.Interop/VoiceCat.Interop.csproj" />
|
||||
<Project Path="VoiceCat.Managed/VoiceCat.Managed.csproj" />
|
||||
</Solution>
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
param(
|
||||
[string]$RuntimeIdentifier = "win-x64",
|
||||
[string]$Output = ""
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
$repoRoot = (Resolve-Path "$PSScriptRoot/../..").Path
|
||||
if ([string]::IsNullOrWhiteSpace($Output)) { $Output = "$repoRoot/dotnet/artifacts/client/$RuntimeIdentifier" }
|
||||
dotnet publish "$PSScriptRoot/VoiceCat.App/VoiceCat.App.csproj" -c Release -r $RuntimeIdentifier --self-contained true `
|
||||
-p:PublishSingleFile=true -p:PublishTrimmed=false -p:IncludeNativeLibrariesForSelfExtract=false `
|
||||
-p:RestorePackagesWithLockFile=true -p:RestoreLockedMode=true `
|
||||
"-p:NuGetLockFilePath=packages.publish.$RuntimeIdentifier.lock.json" -o $Output
|
||||
if ($LASTEXITCODE -ne 0) { throw "Managed Windows client publish failed." }
|
||||
if (-not (Test-Path "$Output/voicecat_media.dll")) { throw "Managed media shim was not published." }
|
||||
if (Test-Path "$Output/voicecat.dll") { throw "Legacy VoiceCat native core must not be published." }
|
||||
Write-Host "Published managed Windows client to $Output"
|
||||
+1
-1
@@ -125,7 +125,7 @@ wrap Opus controls so P/Invoke never calls C varargs. SafeHandle owns every nati
|
||||
encoder, decoder, DRED parser/state, and denoiser, including failed initialization.
|
||||
|
||||
`OpusOptions` is an immutable record. Supported PCM rates are 8/12/16/24/48 kHz,
|
||||
one or two interleaved channels, and integral 10/20/40/60 ms frames. These match the
|
||||
one or two interleaved channels, and integral 5/10/20/40/60 ms frames. These match the
|
||||
current VoiceCat protocol's integer frame duration; fractional Opus frame durations
|
||||
are not exposed. Low-delay application mode requires at most 20 ms. Channel capture
|
||||
bandwidth is controlled separately by `MaximumBandwidthHz`; the production audio
|
||||
|
||||
@@ -794,6 +794,14 @@ criterion from `roadmap.md`, re-proven against the new server.
|
||||
mix cycle; a manual listen test on Windows and macOS with no audible glitching over 10
|
||||
minutes.
|
||||
|
||||
**Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every
|
||||
protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream
|
||||
controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero
|
||||
managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation
|
||||
uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device
|
||||
shim; the codec/DSP shim remains the only native component. A real device smoke passed, but
|
||||
the required ten-minute Windows/macOS listen test is still a manual release gate.
|
||||
|
||||
---
|
||||
|
||||
### Phase 6 — Client core (est. 3–4 weeks)
|
||||
@@ -809,6 +817,12 @@ minutes.
|
||||
conversation through the C# server**, and a C# `vccli` interoperates with a C++ `vccli` on
|
||||
the same server. This is the full M0–M3 criterion re-proven end to end.
|
||||
|
||||
**Checkpoint (2026-09-16):** `VoiceCat.Core` implements TOFU-gated TLS, concurrent correlated
|
||||
requests, snapshots/events, reconnects, encrypted UDP binding and send/receive stream
|
||||
lifecycle. Two managed clients exchange text and decoded PCM through the managed server.
|
||||
The remaining Phase 6 item is the managed console client and its explicit C++ CLI
|
||||
interoperability scenario.
|
||||
|
||||
---
|
||||
|
||||
### Phase 7 — Windows client (est. 1–2 weeks)
|
||||
@@ -818,6 +832,14 @@ shape against a real, complete UI before you commit to two rewrites.
|
||||
|
||||
**Exit criterion:** feature parity with the current WinForms build, NVDA smoke-tested.
|
||||
|
||||
**Checkpoint (2026-09-16):** the shipped WinForms project references `VoiceCat.Managed`, not
|
||||
the P/Invoke core. The compatibility facade preserves UI-thread event pumping and stable
|
||||
capture IDs while delegating all protocol and audio state to the idiomatic managed projects.
|
||||
Channel moves automatically renegotiate active streams. A published build passed real WASAPI
|
||||
capture/playback and form-startup smoke tests and contains `voicecat_media.dll` but no
|
||||
`voicecat.dll`. Automated text, bidirectional PCM voice, multi-frame audio and stream-move
|
||||
tests pass. NVDA and the manual endurance/listen pass remain before the Phase 7 exit criterion.
|
||||
|
||||
---
|
||||
|
||||
### Phase 8 — macOS client (est. 4–5 weeks)
|
||||
|
||||
@@ -5,8 +5,13 @@
|
||||
<Project Path="src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
|
||||
<Project Path="src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
|
||||
<Project Path="src/VoiceCat.Server/VoiceCat.Server.csproj" />
|
||||
<Project Path="src/VoiceCat.Core/VoiceCat.Core.csproj" />
|
||||
<Project Path="src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/tests/">
|
||||
<Project Path="tests/VoiceCat.Tests/VoiceCat.Tests.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/clients/">
|
||||
<Project Path="../clients/windows/VoiceCat.Managed/VoiceCat.Managed.csproj" />
|
||||
</Folder>
|
||||
</Solution>
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
namespace VoiceCat.Audio;
|
||||
|
||||
public sealed record AudioDeviceInfo(string Id, string Name, bool IsDefault);
|
||||
public delegate void CapturePcmHandler(ReadOnlySpan<short> pcm, int channels);
|
||||
public interface IAudioCapture : IDisposable { }
|
||||
public interface IAudioPlayback : IDisposable { void Write(ReadOnlySpan<short> stereoPcm); }
|
||||
public interface IAudioDeviceBackend
|
||||
{
|
||||
IReadOnlyList<AudioDeviceInfo> Enumerate(bool input);
|
||||
IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler pcm);
|
||||
IAudioPlayback OpenPlayback(string? deviceId = null);
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
using System.Diagnostics;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Audio;
|
||||
|
||||
public sealed class AudioEngine : IDisposable
|
||||
{
|
||||
private readonly object gate = new();
|
||||
private readonly EncodedVoiceSender sender;
|
||||
private readonly int[] mixed = new int[1920];
|
||||
private readonly short[] output = new short[1920];
|
||||
private Routes routes = new([], [], 0);
|
||||
private readonly List<(IDisposable Stream, long Epoch)> retired = [];
|
||||
private long completedEpoch;
|
||||
private readonly CancellationTokenSource stop = new();
|
||||
private readonly Task maintenance;
|
||||
private Thread? worker;
|
||||
private int disposed;
|
||||
public uint SampleClock { get; private set; }
|
||||
public event MixedPcmHandler? MixedPcm;
|
||||
public event PcmStreamHandler? StreamPcm;
|
||||
public volatile float InputGain = 1, OutputGain = 1, VadThreshold = 0.02f;
|
||||
public volatile bool InputNoiseReduction, MicMuted, Deafened, PushToTalk;
|
||||
public volatile AudioInputMode InputMode = AudioInputMode.VoiceActivation;
|
||||
public Exception? Failure { get; private set; }
|
||||
|
||||
public AudioEngine(EncodedVoiceSender sender, bool startWorker = true)
|
||||
{
|
||||
this.sender = sender;
|
||||
maintenance = MaintainAsync();
|
||||
if (startWorker)
|
||||
{
|
||||
worker = new Thread(Work) { IsBackground = true, Name = "VoiceCat managed audio" };
|
||||
worker.Start();
|
||||
}
|
||||
}
|
||||
|
||||
public void AddLocalStream(StreamInfo info, int captureChannels = 1)
|
||||
{
|
||||
if (captureChannels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(captureChannels));
|
||||
lock (gate)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(disposed != 0, this);
|
||||
var stream = new LocalStream(info, captureChannels);
|
||||
Routes previous = routes;
|
||||
var locals = previous.Local.Where(s => s.Info.StreamId != info.StreamId).Append(stream).ToArray();
|
||||
Publish(locals, previous.Remote);
|
||||
}
|
||||
}
|
||||
public void RemoveLocalStream(uint streamId)
|
||||
{
|
||||
lock (gate) Publish(routes.Local.Where(s => s.Info.StreamId != streamId).ToArray(), routes.Remote);
|
||||
}
|
||||
public void SetCaptureChannels(uint streamId, int channels)
|
||||
{
|
||||
if (channels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(channels));
|
||||
lock (gate)
|
||||
{
|
||||
var current = routes.Local.FirstOrDefault(s => s.Info.StreamId == streamId) ?? throw new ArgumentException("Stream not found.");
|
||||
if (current.CaptureChannels != channels) Publish(routes.Local.Select(s => s == current ? new LocalStream(s.Info, channels) : s).ToArray(), routes.Remote);
|
||||
}
|
||||
}
|
||||
|
||||
public void SetRemoteStreams(IReadOnlyList<User> users, uint selfId, uint channelId)
|
||||
{
|
||||
lock (gate)
|
||||
{
|
||||
var next = new List<ReceiveStream>();
|
||||
foreach (User user in users.Where(u => u.Id != selfId && u.ChannelId == channelId))
|
||||
foreach (StreamInfo info in user.Streams)
|
||||
{
|
||||
var previous = routes.Remote.FirstOrDefault(s => s.UserId == user.Id && s.Info.Equals(info));
|
||||
next.Add(previous ?? new ReceiveStream(user.Id, info));
|
||||
}
|
||||
Publish(routes.Local, next.ToArray());
|
||||
}
|
||||
}
|
||||
|
||||
public bool FeedPcm(uint streamId, ReadOnlySpan<short> pcm, int channels)
|
||||
{
|
||||
foreach (LocalStream stream in Volatile.Read(ref routes).Local)
|
||||
if (stream.Info.StreamId == streamId) return stream.Feed(pcm, channels);
|
||||
return false;
|
||||
}
|
||||
public void Receive(VoiceFrameHeader header, ReadOnlySpan<byte> packet)
|
||||
{
|
||||
foreach (ReceiveStream stream in Volatile.Read(ref routes).Remote) if (stream.Info.Ssrc == header.Ssrc) { stream.Enqueue(header, packet); return; }
|
||||
}
|
||||
public (float Level, bool Talking) GetLocalLevel(uint streamId)
|
||||
{
|
||||
foreach (LocalStream stream in Volatile.Read(ref routes).Local) if (stream.Info.StreamId == streamId) return (stream.Level, stream.Talking);
|
||||
return default;
|
||||
}
|
||||
public void SetRemotePlayback(uint userId, uint streamId, float gain, bool muted, bool noiseReduction)
|
||||
{
|
||||
if (!float.IsFinite(gain) || gain < 0 || gain > 4) throw new ArgumentOutOfRangeException(nameof(gain));
|
||||
foreach (var stream in Volatile.Read(ref routes).Remote)
|
||||
if (stream.UserId == userId && stream.Info.StreamId == streamId) { stream.Gain = gain; stream.Muted = muted; stream.NoiseReduction = noiseReduction; return; }
|
||||
}
|
||||
public (float Gain, bool Muted, bool NoiseReduction)? GetRemotePlayback(uint userId, uint streamId)
|
||||
{
|
||||
foreach (var stream in Volatile.Read(ref routes).Remote) if (stream.UserId == userId && stream.Info.StreamId == streamId) return (stream.Gain, stream.Muted, stream.NoiseReduction);
|
||||
return null;
|
||||
}
|
||||
|
||||
private void Publish(LocalStream[] local, ReceiveStream[] remote)
|
||||
{
|
||||
Routes previous = routes; var next = new Routes(local, remote, previous.Epoch + 1);
|
||||
foreach (var stream in previous.Local) if (!local.Contains(stream)) retired.Add((stream, next.Epoch));
|
||||
foreach (var stream in previous.Remote) if (!remote.Contains(stream)) retired.Add((stream, next.Epoch));
|
||||
Volatile.Write(ref routes, next);
|
||||
}
|
||||
|
||||
// One audio owner calls this. No allocation, waiting, lock, registry mutation or disposal
|
||||
// occurs inside a mix cycle. Callbacks receive borrowed spans and must follow that rule.
|
||||
internal void ProcessCycle()
|
||||
{
|
||||
Routes current = Volatile.Read(ref routes);
|
||||
mixed.AsSpan().Clear();
|
||||
foreach (var stream in current.Local) stream.Process(this, sender);
|
||||
foreach (var stream in current.Remote) stream.Mix(mixed, Deafened, StreamPcm);
|
||||
float gain = Deafened ? 0 : OutputGain;
|
||||
for (int i = 0; i < output.Length; i++) output[i] = (short)Math.Clamp((int)(mixed[i] * gain), short.MinValue, short.MaxValue);
|
||||
MixedPcm?.Invoke(output);
|
||||
SampleClock = unchecked(SampleClock + 960);
|
||||
Volatile.Write(ref completedEpoch, current.Epoch);
|
||||
}
|
||||
|
||||
private void Work()
|
||||
{
|
||||
long deadline = Stopwatch.GetTimestamp();
|
||||
try
|
||||
{
|
||||
while (!stop.IsCancellationRequested)
|
||||
{
|
||||
ProcessCycle();
|
||||
deadline += Stopwatch.Frequency / 50;
|
||||
double remaining = (deadline - Stopwatch.GetTimestamp()) * 1000.0 / Stopwatch.Frequency;
|
||||
if (remaining > 0) Thread.Sleep((int)Math.Ceiling(remaining));
|
||||
else if (remaining < -100) deadline = Stopwatch.GetTimestamp();
|
||||
}
|
||||
}
|
||||
catch (Exception exception) { Failure = exception; stop.Cancel(); }
|
||||
}
|
||||
private async Task MaintainAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
using var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(20));
|
||||
while (await timer.WaitForNextTickAsync(stop.Token).ConfigureAwait(false))
|
||||
lock (gate)
|
||||
for (int i = retired.Count - 1; i >= 0; i--)
|
||||
if (retired[i].Epoch <= Volatile.Read(ref completedEpoch)) { retired[i].Stream.Dispose(); retired.RemoveAt(i); }
|
||||
}
|
||||
catch (OperationCanceledException) when (stop.IsCancellationRequested) { }
|
||||
}
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
|
||||
stop.Cancel(); worker?.Join(); maintenance.GetAwaiter().GetResult();
|
||||
lock (gate)
|
||||
{
|
||||
foreach (var stream in routes.Local) stream.Dispose(); foreach (var stream in routes.Remote) stream.Dispose();
|
||||
foreach (var stream in retired) stream.Stream.Dispose(); retired.Clear();
|
||||
routes = new([], [], routes.Epoch + 1);
|
||||
}
|
||||
}
|
||||
private sealed record Routes(LocalStream[] Local, ReceiveStream[] Remote, long Epoch);
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
using VoiceCat.Codec;
|
||||
using VoiceCat.Dsp;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
using OpusApplication = Voicecat.V1.OpusApplication;
|
||||
|
||||
namespace VoiceCat.Audio;
|
||||
|
||||
public enum AudioInputMode { VoiceActivation, PushToTalk, AlwaysOn }
|
||||
public delegate bool EncodedVoiceSender(uint ssrc, uint timestamp, ReadOnlySpan<byte> payload, VoiceFrameFlags flags);
|
||||
public delegate void PcmStreamHandler(uint userId, uint streamId, ReadOnlySpan<short> pcm, int channels);
|
||||
public delegate void MixedPcmHandler(ReadOnlySpan<short> stereoPcm);
|
||||
|
||||
internal sealed class LocalStream : IDisposable
|
||||
{
|
||||
internal readonly StreamInfo Info;
|
||||
internal readonly int CaptureChannels;
|
||||
internal readonly PcmRing Input = new(16384);
|
||||
internal volatile float Level;
|
||||
internal volatile bool Talking;
|
||||
private readonly OpusEncoder encoder;
|
||||
private readonly RnnoiseProcessor left, right;
|
||||
private readonly EnergyVadProcessor vad = new();
|
||||
private readonly short[] capture = new short[1920], wire = new short[5760 * 2], mono = new short[960];
|
||||
private readonly byte[] packet = new byte[1275];
|
||||
private readonly short[] converted = new short[16384];
|
||||
private int feeding;
|
||||
private int buffered;
|
||||
private uint timestamp;
|
||||
private bool wasTransmitting, marker;
|
||||
|
||||
internal bool Feed(ReadOnlySpan<short> pcm, int channels)
|
||||
{
|
||||
if (channels is not (1 or 2) || pcm.Length % channels != 0 || pcm.Length / channels * CaptureChannels > converted.Length || Interlocked.CompareExchange(ref feeding, 1, 0) != 0) return false;
|
||||
try
|
||||
{
|
||||
if (channels == CaptureChannels) return Input.TryWrite(pcm);
|
||||
int frames = pcm.Length / channels;
|
||||
for (int i = 0; i < frames; i++)
|
||||
if (CaptureChannels == 1) converted[i] = (short)(((int)pcm[i * 2] + pcm[i * 2 + 1]) / 2);
|
||||
else { converted[2 * i] = pcm[i]; converted[2 * i + 1] = pcm[i]; }
|
||||
return Input.TryWrite(converted.AsSpan(0, frames * CaptureChannels));
|
||||
}
|
||||
finally { Volatile.Write(ref feeding, 0); }
|
||||
}
|
||||
|
||||
internal LocalStream(StreamInfo stream, int captureChannels)
|
||||
{
|
||||
Info = stream.Clone(); CaptureChannels = captureChannels;
|
||||
encoder = new(new()
|
||||
{
|
||||
Channels = stream.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1,
|
||||
FrameDurationMilliseconds = checked((int)stream.Audio.FrameMs), MaximumBandwidthHz = checked((int)stream.Audio.SampleRate),
|
||||
Bitrate = checked((int)stream.Audio.BitrateBps), Complexity = checked((int)stream.Audio.Complexity),
|
||||
ExpectedPacketLossPercent = checked((int)stream.Audio.ExpectedPacketLoss), ForwardErrorCorrection = stream.Audio.Fec,
|
||||
DiscontinuousTransmission = stream.Audio.Dtx, DeepRedundancy = stream.Audio.Dred,
|
||||
Application = stream.Audio.Application switch { OpusApplication.OpusAudio => VoiceCat.Codec.OpusApplication.Audio, OpusApplication.OpusLowdelay => VoiceCat.Codec.OpusApplication.LowDelay, _ => VoiceCat.Codec.OpusApplication.Voip }
|
||||
});
|
||||
try { left = new(); } catch { encoder.Dispose(); throw; }
|
||||
try { right = new(); } catch { left.Dispose(); encoder.Dispose(); throw; }
|
||||
}
|
||||
|
||||
internal void Process(AudioEngine engine, EncodedVoiceSender sender)
|
||||
{
|
||||
var input = capture.AsSpan(0, 960 * CaptureChannels);
|
||||
if (Input.Count < input.Length) { Level = 0; Talking = false; buffered = 0; wasTransmitting = false; return; }
|
||||
while (Input.Count > input.Length * 6) Input.Read(input);
|
||||
if (buffered == 0) timestamp = engine.SampleClock;
|
||||
Input.Read(input);
|
||||
bool mic = Info.Kind == StreamKind.StreamMic;
|
||||
if (mic && engine.InputNoiseReduction)
|
||||
{
|
||||
if (CaptureChannels == 1) left.Process(input);
|
||||
else
|
||||
{
|
||||
for (int i = 0; i < 960; i++) mono[i] = input[2 * i]; left.Process(mono);
|
||||
for (int i = 0; i < 960; i++) input[2 * i] = mono[i];
|
||||
for (int i = 0; i < 960; i++) mono[i] = input[2 * i + 1]; right.Process(mono);
|
||||
for (int i = 0; i < 960; i++) input[2 * i + 1] = mono[i];
|
||||
}
|
||||
}
|
||||
float gain = mic ? engine.InputGain : 1;
|
||||
double energy = 0;
|
||||
for (int i = 0; i < input.Length; i++) { input[i] = (short)Math.Clamp((int)(input[i] * gain), short.MinValue, short.MaxValue); energy += (double)input[i] * input[i]; }
|
||||
Level = (float)(Math.Sqrt(energy / input.Length) / 32768);
|
||||
vad.Threshold = engine.VadThreshold;
|
||||
bool transmit = !mic || !engine.MicMuted && engine.InputMode switch
|
||||
{
|
||||
AudioInputMode.AlwaysOn => true, AudioInputMode.PushToTalk => engine.PushToTalk,
|
||||
_ => vad.Process(input)
|
||||
};
|
||||
Talking = transmit && Level > 0.001f;
|
||||
if (!transmit) { buffered = 0; wasTransmitting = false; return; }
|
||||
if (!wasTransmitting) marker = true;
|
||||
wasTransmitting = true;
|
||||
int channels = encoder.Options.Channels;
|
||||
for (int i = 0; i < 960; i++)
|
||||
{
|
||||
if (channels == 1) wire[buffered + i] = CaptureChannels == 1 ? input[i] : (short)(((int)input[2 * i] + input[2 * i + 1]) / 2);
|
||||
else { wire[buffered + 2 * i] = input[i * CaptureChannels]; wire[buffered + 2 * i + 1] = input[i * CaptureChannels + CaptureChannels - 1]; }
|
||||
}
|
||||
buffered += 960 * channels;
|
||||
int frame = encoder.Options.SamplesPerChannel * channels;
|
||||
while (buffered >= frame)
|
||||
{
|
||||
int length = encoder.Encode(wire.AsSpan(0, frame), packet);
|
||||
VoiceFrameFlags flags = (Info.Audio.Fec ? VoiceFrameFlags.FecPresent : VoiceFrameFlags.None) | (marker ? VoiceFrameFlags.Marker : VoiceFrameFlags.None);
|
||||
sender(Info.Ssrc, timestamp, packet.AsSpan(0, length), flags); marker = false;
|
||||
timestamp = unchecked(timestamp + (uint)encoder.Options.SamplesPerChannel);
|
||||
buffered -= frame;
|
||||
wire.AsSpan(frame, buffered).CopyTo(wire);
|
||||
}
|
||||
}
|
||||
public void Dispose() { encoder.Dispose(); left.Dispose(); right.Dispose(); }
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
namespace VoiceCat.Audio;
|
||||
|
||||
// Single consumer, non-waiting producer gate. Whole writes either fit or drop, so
|
||||
// channels remain aligned and a capture thread never waits for a mixer/network owner.
|
||||
public sealed class PcmRing
|
||||
{
|
||||
private readonly short[] samples;
|
||||
private readonly int mask;
|
||||
private int read, written, producer;
|
||||
public PcmRing(int capacity = 32768)
|
||||
{
|
||||
if (capacity < 2 || (capacity & (capacity - 1)) != 0) throw new ArgumentOutOfRangeException(nameof(capacity));
|
||||
samples = new short[capacity]; mask = capacity - 1;
|
||||
}
|
||||
public int Count => unchecked(Volatile.Read(ref written) - Volatile.Read(ref read));
|
||||
public bool TryWrite(ReadOnlySpan<short> source)
|
||||
{
|
||||
if (Interlocked.CompareExchange(ref producer, 1, 0) != 0) return false;
|
||||
try
|
||||
{
|
||||
int index = written;
|
||||
if (source.Length > samples.Length - unchecked(index - Volatile.Read(ref read))) return false;
|
||||
for (int i = 0; i < source.Length; i++) samples[(index + i) & mask] = source[i];
|
||||
Volatile.Write(ref written, unchecked(index + source.Length)); return true;
|
||||
}
|
||||
finally { Volatile.Write(ref producer, 0); }
|
||||
}
|
||||
public int Read(Span<short> destination)
|
||||
{
|
||||
int index = read, count = Math.Min(destination.Length, unchecked(Volatile.Read(ref written) - index));
|
||||
for (int i = 0; i < count; i++) destination[i] = samples[(index + i) & mask];
|
||||
Volatile.Write(ref read, unchecked(index + count)); return count;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
using VoiceCat.Codec;
|
||||
using VoiceCat.Dsp;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Audio;
|
||||
|
||||
internal sealed class ReceiveStream : IDisposable
|
||||
{
|
||||
internal readonly uint UserId;
|
||||
internal readonly StreamInfo Info;
|
||||
internal volatile float Gain = 1;
|
||||
internal volatile bool Muted, NoiseReduction;
|
||||
private readonly OpusDecoder decoder;
|
||||
private readonly OpusDeepRedundancy? dred;
|
||||
private readonly RnnoiseProcessor left, right;
|
||||
private readonly short[] pcm = new short[5760 * 2];
|
||||
private readonly short[] mono = new short[960];
|
||||
private readonly short[] block = new short[1920];
|
||||
private readonly byte[][] packets = Enumerable.Range(0, 64).Select(_ => new byte[1275]).ToArray();
|
||||
private readonly VoiceFrameHeader[] headers = new VoiceFrameHeader[64];
|
||||
private readonly int[] lengths = new int[64];
|
||||
private int read, written;
|
||||
private readonly byte[][] jitter = Enumerable.Range(0, 6).Select(_ => new byte[1275]).ToArray();
|
||||
private readonly uint[] timestamps = new uint[6];
|
||||
private readonly int[] sizes = new int[6];
|
||||
private int count, available, offset, missing, waiting;
|
||||
private uint expected;
|
||||
private bool started, hasTimestamp;
|
||||
private bool hasMarker;
|
||||
private uint lastMarker;
|
||||
private readonly int channels, frameSamples, maximumDepth;
|
||||
internal int Depth => count;
|
||||
internal int ConcealedFrames { get; private set; }
|
||||
internal int DredFrames { get; private set; }
|
||||
internal int FecFrames { get; private set; }
|
||||
|
||||
internal ReceiveStream(uint userId, StreamInfo info)
|
||||
{
|
||||
if (info.Audio.FrameMs is not (5 or 10 or 20 or 40 or 60) || !Enum.IsDefined(info.Audio.Mode)) throw new ArgumentException("Unsupported remote audio configuration.", nameof(info));
|
||||
UserId = userId; Info = info.Clone();
|
||||
channels = info.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1;
|
||||
frameSamples = checked((int)info.Audio.FrameMs * 48);
|
||||
maximumDepth = Math.Clamp(120 / (int)info.Audio.FrameMs, 2, 6);
|
||||
decoder = new(48000, channels);
|
||||
try { left = new(); } catch { decoder.Dispose(); throw; }
|
||||
try { right = new(); } catch { left.Dispose(); decoder.Dispose(); throw; }
|
||||
try { if (info.Audio.Dred) dred = new(); } catch { right.Dispose(); left.Dispose(); decoder.Dispose(); throw; }
|
||||
}
|
||||
|
||||
// Only the network receive owner calls this; mixer alone consumes.
|
||||
internal bool Enqueue(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
|
||||
{
|
||||
int index = written;
|
||||
if (payload.Length is < 1 or > 1275 || unchecked(index - Volatile.Read(ref read)) >= 64) return false;
|
||||
int slot = index & 63; payload.CopyTo(packets[slot]); headers[slot] = header; lengths[slot] = payload.Length;
|
||||
Volatile.Write(ref written, unchecked(index + 1)); return true;
|
||||
}
|
||||
|
||||
private void Drain()
|
||||
{
|
||||
while (read != Volatile.Read(ref written))
|
||||
{
|
||||
int source = read & 63; uint timestamp = headers[source].Timestamp;
|
||||
if (!hasTimestamp) { hasTimestamp = true; expected = timestamp; }
|
||||
int delta = unchecked((int)(timestamp - expected));
|
||||
if ((headers[source].Flags & VoiceFrameFlags.Marker) != 0 && (!hasMarker || unchecked((int)(timestamp - lastMarker)) > 0))
|
||||
{
|
||||
hasMarker = true; lastMarker = timestamp;
|
||||
sizes.AsSpan().Clear(); count = available = offset = missing = waiting = 0;
|
||||
expected = timestamp; started = false; delta = 0;
|
||||
}
|
||||
bool duplicate = false;
|
||||
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == timestamp) duplicate = true;
|
||||
if ((!started || delta >= 0) && delta % frameSamples == 0 && !duplicate)
|
||||
{
|
||||
if (count >= maximumDepth)
|
||||
{
|
||||
int oldest = Oldest(); sizes[oldest] = 0; count--;
|
||||
}
|
||||
int target = Array.IndexOf(sizes, 0);
|
||||
timestamps[target] = timestamp; sizes[target] = lengths[source]; packets[source].AsSpan(0, lengths[source]).CopyTo(jitter[target]); count++;
|
||||
int oldestRemaining = Oldest();
|
||||
if (count >= maximumDepth && unchecked((int)(timestamps[oldestRemaining] - expected)) > 0) expected = timestamps[oldestRemaining];
|
||||
}
|
||||
Volatile.Write(ref read, unchecked(read + 1));
|
||||
}
|
||||
}
|
||||
|
||||
private int Oldest()
|
||||
{
|
||||
int oldest = -1;
|
||||
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && (oldest < 0 || unchecked((int)(timestamps[i] - timestamps[oldest])) < 0)) oldest = i;
|
||||
return oldest;
|
||||
}
|
||||
|
||||
private void Decode()
|
||||
{
|
||||
available = frameSamples; offset = 0;
|
||||
int found = -1;
|
||||
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == expected) { found = i; break; }
|
||||
bool decoded = false;
|
||||
if (found >= 0)
|
||||
{
|
||||
decoded = decoder.TryDecode(jitter[found].AsSpan(0, sizes[found]), pcm, frameSamples, out int result) && result == frameSamples;
|
||||
sizes[found] = 0; count--; missing = decoded ? 0 : missing + 1;
|
||||
}
|
||||
else
|
||||
{
|
||||
int next = Oldest(); missing++;
|
||||
if (next >= 0 && unchecked((int)(timestamps[next] - expected)) == frameSamples)
|
||||
{
|
||||
var packet = jitter[next].AsSpan(0, sizes[next]);
|
||||
if (dred?.TryRecover(decoder, packet, pcm, frameSamples) == true) { decoded = true; DredFrames++; }
|
||||
else if (Info.Audio.Fec && decoder.TryDecode(packet, pcm, frameSamples, out int recovered, true) && recovered == frameSamples) { decoded = true; FecFrames++; }
|
||||
}
|
||||
}
|
||||
int maximumConcealment = Math.Max(1, 200 / (int)Info.Audio.FrameMs);
|
||||
if (!decoded && missing <= maximumConcealment)
|
||||
{
|
||||
decoded = decoder.TryDecode([], pcm, frameSamples, out _); ConcealedFrames++;
|
||||
}
|
||||
if (!decoded || missing > maximumConcealment) pcm.AsSpan(0, frameSamples * channels).Clear();
|
||||
expected = unchecked(expected + (uint)frameSamples);
|
||||
}
|
||||
|
||||
internal void Mix(Span<int> output, bool deafened, PcmStreamHandler? sink)
|
||||
{
|
||||
Drain();
|
||||
if (!started)
|
||||
{
|
||||
waiting++;
|
||||
if (!hasTimestamp || count < Math.Min(3, maximumDepth) && waiting < 3) return;
|
||||
expected = timestamps[Oldest()]; started = true;
|
||||
}
|
||||
int copied = 0;
|
||||
while (copied < 960)
|
||||
{
|
||||
if (available == 0) Decode();
|
||||
int take = Math.Min(960 - copied, available);
|
||||
var decoded = pcm.AsSpan(offset * channels, take * channels);
|
||||
decoded.CopyTo(block.AsSpan(copied * channels));
|
||||
available -= take; offset += take; copied += take;
|
||||
}
|
||||
var samples = block.AsSpan(0, 960 * channels);
|
||||
if (NoiseReduction && Info.Kind == StreamKind.StreamMic)
|
||||
{
|
||||
if (channels == 1) left.Process(samples);
|
||||
else
|
||||
{
|
||||
for (int i = 0; i < 960; i++) mono[i] = samples[2 * i]; left.Process(mono);
|
||||
for (int i = 0; i < 960; i++) samples[2 * i] = mono[i];
|
||||
for (int i = 0; i < 960; i++) mono[i] = samples[2 * i + 1]; right.Process(mono);
|
||||
for (int i = 0; i < 960; i++) samples[2 * i + 1] = mono[i];
|
||||
}
|
||||
}
|
||||
float gain = Muted || deafened ? 0 : Gain;
|
||||
for (int i = 0; i < samples.Length; i++) samples[i] = (short)Math.Clamp((int)(samples[i] * gain), short.MinValue, short.MaxValue);
|
||||
sink?.Invoke(UserId, Info.StreamId, samples, channels);
|
||||
for (int i = 0; i < 960; i++) { output[i * 2] += samples[i * channels]; output[i * 2 + 1] += samples[i * channels + channels - 1]; }
|
||||
}
|
||||
public void Dispose() { decoder.Dispose(); dred?.Dispose(); left.Dispose(); right.Dispose(); }
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="../VoiceCat.Codec/VoiceCat.Codec.csproj" />
|
||||
<ProjectReference Include="../VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
|
||||
<ProjectReference Include="../VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
|
||||
<InternalsVisibleTo Include="VoiceCat.Tests" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
},
|
||||
"net10.0/win-x64": {}
|
||||
}
|
||||
}
|
||||
@@ -41,5 +41,17 @@ public sealed class OpusDecoder : IDisposable
|
||||
return OpusException.Check(NativeMethods.Decode(handle, input, packet.Length, output, samplesPerChannel, recoverPreviousFrame ? 1 : 0));
|
||||
}
|
||||
|
||||
public unsafe bool TryDecode(ReadOnlySpan<byte> packet, Span<short> pcm, int samplesPerChannel, out int decodedSamples, bool recoverPreviousFrame = false)
|
||||
{
|
||||
ValidateOutput(pcm, samplesPerChannel);
|
||||
if (packet.Overlaps(MemoryMarshal.AsBytes(pcm))) throw new ArgumentException("Packet and PCM storage must not overlap.");
|
||||
fixed (byte* input = packet)
|
||||
fixed (short* output = pcm)
|
||||
{
|
||||
decodedSamples = NativeMethods.Decode(handle, input, packet.Length, output, samplesPerChannel, recoverPreviousFrame ? 1 : 0);
|
||||
return decodedSamples >= 0;
|
||||
}
|
||||
}
|
||||
|
||||
public void Dispose() => handle.Dispose();
|
||||
}
|
||||
|
||||
@@ -40,11 +40,10 @@ public sealed class OpusDeepRedundancy : IDisposable
|
||||
fixed (byte* packet = nextPacket)
|
||||
fixed (short* output = pcm)
|
||||
{
|
||||
int parsed = OpusException.Check(NativeMethods.DredParse(decoder, dred, packet, nextPacket.Length,
|
||||
checked(samplesPerChannel + recoveryOffset), audioDecoder.SampleRate, out _));
|
||||
if (parsed == 0) return false;
|
||||
OpusException.Check(NativeMethods.DredDecode(audioDecoder.Handle, dred, recoveryOffset, output, samplesPerChannel));
|
||||
return true;
|
||||
int parsed = NativeMethods.DredParse(decoder, dred, packet, nextPacket.Length,
|
||||
checked(samplesPerChannel + recoveryOffset), audioDecoder.SampleRate, out _);
|
||||
if (parsed <= 0) return false;
|
||||
return NativeMethods.DredDecode(audioDecoder.Handle, dred, recoveryOffset, output, samplesPerChannel) >= 0;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ public sealed record OpusOptions
|
||||
{
|
||||
if (SampleRate is not (8000 or 12000 or 16000 or 24000 or 48000)) throw new ArgumentOutOfRangeException(nameof(SampleRate));
|
||||
if (Channels is not (1 or 2)) throw new ArgumentOutOfRangeException(nameof(Channels));
|
||||
if (FrameDurationMilliseconds is not (10 or 20 or 40 or 60)) throw new ArgumentOutOfRangeException(nameof(FrameDurationMilliseconds));
|
||||
if (FrameDurationMilliseconds is not (5 or 10 or 20 or 40 or 60)) throw new ArgumentOutOfRangeException(nameof(FrameDurationMilliseconds));
|
||||
if (Application == OpusApplication.LowDelay && FrameDurationMilliseconds > 20) throw new ArgumentException("Low-delay Opus requires frames of at most 20 ms.");
|
||||
if (!Enum.IsDefined(Application)) throw new ArgumentOutOfRangeException(nameof(Application));
|
||||
if (Bitrate is < 500 or > 512000) throw new ArgumentOutOfRangeException(nameof(Bitrate));
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
}
|
||||
},
|
||||
"net10.0/win-x64": {}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using VoiceCat.Protocol;
|
||||
using VoiceCat.Crypto;
|
||||
using VoiceCat.Transport;
|
||||
|
||||
namespace VoiceCat.Core;
|
||||
|
||||
public delegate void EncodedVoiceHandler(VoiceFrameHeader header, ReadOnlySpan<byte> payload);
|
||||
|
||||
internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
{
|
||||
private readonly Socket socket;
|
||||
private readonly MediaSessionCrypto crypto;
|
||||
private readonly CancellationTokenSource stop;
|
||||
private readonly byte[] binding = new byte[VoiceFrameHeader.Size + 16];
|
||||
private readonly byte[] keepalive = new byte[VoiceFrameHeader.Size];
|
||||
private readonly PacketQueue packets = new();
|
||||
private readonly Task sending;
|
||||
private readonly Task receiving;
|
||||
private readonly TaskCompletionSource bound = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
internal event EncodedVoiceHandler? Received;
|
||||
internal Task Bound => bound.Task;
|
||||
|
||||
internal ClientMediaTransport(IPEndPoint endpoint, ReadOnlySpan<byte> token, MediaSessionCrypto crypto, CancellationToken cancellationToken)
|
||||
{
|
||||
if (token.Length != 16) throw new IOException("Invalid UDP binding token.");
|
||||
this.crypto = crypto;
|
||||
stop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||
socket = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
|
||||
try { socket.Connect(endpoint); }
|
||||
catch { socket.Dispose(); stop.Dispose(); throw; }
|
||||
new VoiceFrameHeader(MediaFrameType.UdpBinding, 0, 0, 0, 0, 0).Write(binding);
|
||||
token.CopyTo(binding.AsSpan(VoiceFrameHeader.Size));
|
||||
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
|
||||
receiving = ReceiveAsync();
|
||||
sending = SendAsync();
|
||||
}
|
||||
|
||||
internal bool TrySend(VoiceFrameHeader header, ReadOnlySpan<byte> payload) => packets.TryWrite(header, payload);
|
||||
|
||||
private async Task SendAsync()
|
||||
{
|
||||
byte[] plain = new byte[1275], packet = new byte[1275 + VoiceFrameHeader.Size + MediaEncryptor.TagSize];
|
||||
long nextKeepalive = 0;
|
||||
try
|
||||
{
|
||||
while (!stop.IsCancellationRequested)
|
||||
{
|
||||
if (Environment.TickCount64 >= nextKeepalive)
|
||||
{
|
||||
if (!bound.Task.IsCompleted) await socket.SendAsync(binding, SocketFlags.None, stop.Token).ConfigureAwait(false);
|
||||
await socket.SendAsync(keepalive, SocketFlags.None, stop.Token).ConfigureAwait(false);
|
||||
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? 5000 : 250);
|
||||
}
|
||||
while (packets.TryRead(plain, out VoiceFrameHeader header, out int length))
|
||||
{
|
||||
int size = crypto.Encryptor.Encrypt(header, plain.AsSpan(0, length), packet);
|
||||
await socket.SendAsync(packet.AsMemory(0, size), SocketFlags.None, stop.Token).ConfigureAwait(false);
|
||||
}
|
||||
await Task.Delay(5, stop.Token).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException)
|
||||
{ if (!stop.IsCancellationRequested) bound.TrySetException(exception); }
|
||||
finally { stop.Cancel(); }
|
||||
}
|
||||
|
||||
private async Task ReceiveAsync()
|
||||
{
|
||||
byte[] packet = new byte[65535], plain = new byte[65535];
|
||||
try
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
int length;
|
||||
try { length = await socket.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
|
||||
catch (SocketException exception) when (exception.SocketErrorCode is SocketError.ConnectionReset or SocketError.MessageSize) { continue; }
|
||||
if (!VoiceFrameHeader.TryRead(packet.AsSpan(0, length), out var candidate)) continue;
|
||||
if (candidate.Type == MediaFrameType.Keepalive && length == VoiceFrameHeader.Size) { bound.TrySetResult(); continue; }
|
||||
if (candidate.Type != MediaFrameType.Voice || candidate.Codec != 0 ||
|
||||
!crypto.Decryptor.TryDecrypt(packet.AsSpan(0, length), plain, out var header, out int size)) continue;
|
||||
Received?.Invoke(header, plain.AsSpan(0, size));
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException)
|
||||
{ if (!stop.IsCancellationRequested) bound.TrySetException(exception); }
|
||||
finally { bound.TrySetCanceled(); stop.Cancel(); }
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
stop.Cancel(); socket.Dispose();
|
||||
try { await Task.WhenAll(sending, receiving).ConfigureAwait(false); }
|
||||
finally { System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding); stop.Dispose(); }
|
||||
}
|
||||
|
||||
// A bounded, allocation-free packet handoff. A contending producer drops instead of
|
||||
// waiting; the network owner alone consumes and encrypts. Audio never enters a Channel lock.
|
||||
private sealed class PacketQueue
|
||||
{
|
||||
private readonly byte[][] payloads = Enumerable.Range(0, 64).Select(_ => new byte[1275]).ToArray();
|
||||
private readonly VoiceFrameHeader[] headers = new VoiceFrameHeader[64];
|
||||
private readonly int[] lengths = new int[64];
|
||||
private int read, written, producer;
|
||||
internal bool TryWrite(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
|
||||
{
|
||||
if (payload.Length is < 1 or > 1275 || Interlocked.CompareExchange(ref producer, 1, 0) != 0) return false;
|
||||
try
|
||||
{
|
||||
int index = written;
|
||||
if (unchecked(index - Volatile.Read(ref read)) >= 64) return false;
|
||||
int slot = index & 63;
|
||||
payload.CopyTo(payloads[slot]); headers[slot] = header; lengths[slot] = payload.Length;
|
||||
Volatile.Write(ref written, unchecked(index + 1)); return true;
|
||||
}
|
||||
finally { Volatile.Write(ref producer, 0); }
|
||||
}
|
||||
internal bool TryRead(Span<byte> payload, out VoiceFrameHeader header, out int length)
|
||||
{
|
||||
int index = read; header = default; length = 0;
|
||||
if (index == Volatile.Read(ref written)) return false;
|
||||
int slot = index & 63; header = headers[slot]; length = lengths[slot];
|
||||
payloads[slot].AsSpan(0, length).CopyTo(payload);
|
||||
Volatile.Write(ref read, unchecked(index + 1)); return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="../VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
|
||||
<ProjectReference Include="../VoiceCat.Audio/VoiceCat.Audio.csproj" />
|
||||
<InternalsVisibleTo Include="VoiceCat.Tests" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -0,0 +1,292 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using System.Threading.Channels;
|
||||
using VoiceCat.Crypto;
|
||||
using VoiceCat.Transport;
|
||||
using VoiceCat.Protocol;
|
||||
using VoiceCat.Audio;
|
||||
using Voicecat.V1;
|
||||
using Channel = Voicecat.V1.Channel;
|
||||
|
||||
namespace VoiceCat.Core;
|
||||
|
||||
public enum ClientConnectionState { Disconnected, Connecting, VerifyingIdentity, Authenticating, Connected }
|
||||
public sealed record ServerIdentityChallenge(string Host, ushort Port, string CertificateFingerprint, TofuStatus Status);
|
||||
|
||||
public sealed partial class VoiceCatClient : IAsyncDisposable
|
||||
{
|
||||
private readonly string clientName;
|
||||
private readonly string clientVersion;
|
||||
private readonly TofuStore pins;
|
||||
private readonly SemaphoreSlim lifecycle = new(1);
|
||||
private readonly CancellationTokenSource disposed = new();
|
||||
private readonly object stateGate = new();
|
||||
private readonly ConcurrentDictionary<ulong, TaskCompletionSource<Envelope>> pending = new();
|
||||
private readonly System.Threading.Channels.Channel<Envelope> events = System.Threading.Channels.Channel.CreateBounded<Envelope>(128);
|
||||
private readonly Dictionary<uint, Channel> channels = [];
|
||||
private readonly Dictionary<uint, User> users = [];
|
||||
private readonly Dictionary<uint, StreamInfo> localStreams = [];
|
||||
public AudioEngine Audio { get; }
|
||||
public IReadOnlyList<StreamInfo> LocalStreams { get { lock (stateGate) return localStreams.Values.Select(s => s.Clone()).ToArray(); } }
|
||||
private TlsControlConnection? control;
|
||||
private MediaSessionCrypto? mediaCrypto;
|
||||
private ClientMediaTransport? media;
|
||||
private Task keepalive = Task.CompletedTask;
|
||||
public event EncodedVoiceHandler? VoiceReceived;
|
||||
private CancellationTokenSource? connectionLifetime;
|
||||
private Task reader = Task.CompletedTask;
|
||||
private long nextRequest;
|
||||
private AuthResult? authentication;
|
||||
private ServerHello? hello;
|
||||
private ClientConnectionState state;
|
||||
|
||||
public event Action<ClientConnectionState>? ConnectionStateChanged;
|
||||
public ClientConnectionState State { get { lock (stateGate) return state; } }
|
||||
public Task Completion => reader;
|
||||
public AuthResult? Authentication { get { lock (stateGate) return authentication?.Clone(); } }
|
||||
public ServerHello? ServerHello { get { lock (stateGate) return hello?.Clone(); } }
|
||||
public IReadOnlyList<Channel> Channels { get { lock (stateGate) return channels.Values.Select(c => c.Clone()).ToArray(); } }
|
||||
public IReadOnlyList<User> Users { get { lock (stateGate) return users.Values.Select(u => u.Clone()).ToArray(); } }
|
||||
public bool TryReadEvent(out Envelope? envelope) => events.Reader.TryRead(out envelope);
|
||||
public IAsyncEnumerable<Envelope> ReadEventsAsync(CancellationToken cancellationToken = default) => events.Reader.ReadAllAsync(cancellationToken);
|
||||
|
||||
public VoiceCatClient(string clientName = "VoiceCat .NET", string clientVersion = "0.1.0", string? tofuStorePath = null)
|
||||
{
|
||||
this.clientName = clientName;
|
||||
this.clientVersion = clientVersion;
|
||||
pins = new(tofuStorePath ?? Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "VoiceCat", "tofu.txt"));
|
||||
Audio = new(TrySendEncodedVoice);
|
||||
VoiceReceived += Audio.Receive;
|
||||
}
|
||||
|
||||
public async Task ConnectAsync(string host, ushort port, Func<ServerIdentityChallenge, CancellationToken, ValueTask<bool>>? confirmIdentity = null, CancellationToken cancellationToken = default)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(host);
|
||||
ArgumentOutOfRangeException.ThrowIfZero(port);
|
||||
await lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
Socket? socket = null;
|
||||
bool started = false;
|
||||
try
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(disposed.IsCancellationRequested, this);
|
||||
if (control is not null) throw new InvalidOperationException("Disconnect before reconnecting.");
|
||||
started = true;
|
||||
connectionLifetime = CancellationTokenSource.CreateLinkedTokenSource(disposed.Token);
|
||||
using var connecting = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, connectionLifetime.Token);
|
||||
CancellationToken token = connecting.Token;
|
||||
SetState(ClientConnectionState.Connecting);
|
||||
socket = new(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
|
||||
await socket.ConnectAsync(host, port, token).ConfigureAwait(false);
|
||||
string? fingerprint = null;
|
||||
control = new(socket, TlsSession.CreateClient(value => { fingerprint = value; return true; }), connectionLifetime.Token);
|
||||
socket = null; // Transport owns it from here.
|
||||
mediaCrypto = await control.TakeMediaCryptoAsync(token).ConfigureAwait(false);
|
||||
string certificatePin = fingerprint ?? throw new IOException("TLS did not report a certificate fingerprint.");
|
||||
TofuStatus pinStatus = pins.Check(host, port, certificatePin);
|
||||
if (pinStatus != TofuStatus.Matched)
|
||||
{
|
||||
SetState(ClientConnectionState.VerifyingIdentity);
|
||||
if (confirmIdentity is null || !await confirmIdentity(new(host, port, certificatePin, pinStatus), token).ConfigureAwait(false))
|
||||
throw new System.Security.Authentication.AuthenticationException("Server identity was rejected.");
|
||||
pins.Pin(host, port, certificatePin);
|
||||
}
|
||||
reader = ReadAsync(control, connectionLifetime.Token);
|
||||
Envelope response = await RequestAsync(new() { ClientHello = new() { ProtoVersion = 2, ClientName = clientName, ClientVersion = clientVersion } }, token).ConfigureAwait(false);
|
||||
if (response.ServerHello?.ProtoVersion != 2) throw new IOException("Unsupported server protocol.");
|
||||
lock (stateGate) hello = response.ServerHello.Clone();
|
||||
keepalive = KeepaliveAsync(connectionLifetime.Token);
|
||||
SetState(ClientConnectionState.Authenticating);
|
||||
}
|
||||
catch
|
||||
{
|
||||
socket?.Dispose();
|
||||
if (started) await CloseAsync().ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
finally { lifecycle.Release(); }
|
||||
}
|
||||
|
||||
public Task<AuthResult> AuthenticateGuestAsync(string nickname, CancellationToken cancellationToken = default) =>
|
||||
AuthenticateAsync(new() { Guest = new() { Nickname = nickname } }, cancellationToken);
|
||||
public Task<AuthResult> AuthenticateUserAsync(string username, string password, CancellationToken cancellationToken = default) =>
|
||||
AuthenticateAsync(new() { Password = new() { Username = username, Password = password } }, cancellationToken);
|
||||
|
||||
private async Task<AuthResult> AuthenticateAsync(AuthRequest request, CancellationToken cancellationToken)
|
||||
{
|
||||
if (State != ClientConnectionState.Authenticating) throw new InvalidOperationException("Authentication requires a connected TLS session.");
|
||||
Envelope response = await RequestAsync(new() { AuthRequest = request }, cancellationToken).ConfigureAwait(false);
|
||||
AuthResult result = response.AuthResult ?? throw new IOException("Unexpected authentication response.");
|
||||
if (result.Ok)
|
||||
{
|
||||
var endpoint = (IPEndPoint)control!.RemoteEndPoint;
|
||||
IPAddress address = endpoint.Address.IsIPv4MappedToIPv6 ? endpoint.Address.MapToIPv4() : endpoint.Address;
|
||||
media = new(new(address, checked((int)ServerHello!.UdpPort)), result.UdpToken.Span, mediaCrypto!, connectionLifetime!.Token);
|
||||
media.Received += (header, packet) => VoiceReceived?.Invoke(header, packet);
|
||||
SetState(ClientConnectionState.Connected);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
public async Task<VoiceSubscriptionResult> SubscribeVoiceAsync(bool subscribe = true, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (subscribe && media is not null) await media.Bound.WaitAsync(TimeSpan.FromSeconds(5), cancellationToken).ConfigureAwait(false);
|
||||
return (await RequestAsync(subscribe ? new() { SubscribeVoice = new() } : new() { UnsubscribeVoice = new() }, cancellationToken).ConfigureAwait(false)).VoiceSubscriptionResult;
|
||||
}
|
||||
|
||||
public bool TrySendEncodedVoice(uint ssrc, uint timestamp, ReadOnlySpan<byte> payload, VoiceFrameFlags flags = VoiceFrameFlags.None) =>
|
||||
media?.TrySend(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload) == true;
|
||||
|
||||
public async Task<StreamInfo> StartStreamAsync(StreamKind kind, string label = "", int captureChannels = 1, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (State != ClientConnectionState.Connected) throw new InvalidOperationException("Client is disconnected.");
|
||||
var response = (await RequestAsync(new() { StreamAnnounce = new() { Kind = kind, Label = label } }, cancellationToken).ConfigureAwait(false)).StreamAnnounceResult;
|
||||
if (!response.Ok) throw new InvalidOperationException(response.Error);
|
||||
var info = new StreamInfo { StreamId = response.StreamId, Ssrc = response.Ssrc, Kind = kind, Audio = response.EffectiveAudio.Clone(), Label = label };
|
||||
try
|
||||
{
|
||||
lock (stateGate) { if (State != ClientConnectionState.Connected) throw new InvalidOperationException("Client disconnected during stream negotiation."); Audio.AddLocalStream(info, captureChannels); localStreams[info.StreamId] = info; }
|
||||
return info.Clone();
|
||||
}
|
||||
catch { if (State == ClientConnectionState.Connected) Send(new() { StreamStop = new() { StreamId = info.StreamId } }); throw; }
|
||||
}
|
||||
|
||||
public void StopStream(uint streamId)
|
||||
{
|
||||
lock (stateGate) { localStreams.Remove(streamId); Audio.RemoveLocalStream(streamId); }
|
||||
Send(new() { StreamStop = new() { StreamId = streamId } });
|
||||
}
|
||||
|
||||
private async Task KeepaliveAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
try
|
||||
{
|
||||
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(10));
|
||||
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false)) Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or IOException or InvalidOperationException) { }
|
||||
}
|
||||
|
||||
public async Task<Envelope> RequestAsync(Envelope request, CancellationToken cancellationToken = default)
|
||||
{
|
||||
TlsControlConnection connection = control ?? throw new InvalidOperationException("Client is disconnected.");
|
||||
var completion = new TaskCompletionSource<Envelope>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
ulong id = checked((ulong)Interlocked.Increment(ref nextRequest));
|
||||
Envelope outbound = request.Clone(); outbound.RequestId = id;
|
||||
if (!pending.TryAdd(id, completion)) throw new InvalidOperationException("Request ids exhausted.");
|
||||
try
|
||||
{
|
||||
if (!connection.TrySend(outbound)) throw new IOException("Control queue is full or closed.");
|
||||
return await completion.Task.WaitAsync(TimeSpan.FromSeconds(15), cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
finally { pending.TryRemove(id, out _); }
|
||||
}
|
||||
|
||||
public void Send(Envelope message)
|
||||
{
|
||||
TlsControlConnection connection = control ?? throw new InvalidOperationException("Client is disconnected.");
|
||||
if (!connection.TrySend(message.Clone())) throw new IOException("Control queue is full or closed.");
|
||||
}
|
||||
|
||||
private async Task ReadAsync(TlsControlConnection connection, CancellationToken cancellationToken)
|
||||
{
|
||||
Exception? failure = null;
|
||||
try
|
||||
{
|
||||
await foreach (Envelope message in connection.ReadAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
Apply(message);
|
||||
if (message.RequestId != 0 && pending.TryRemove(message.RequestId, out var completion)) completion.TrySetResult(message.Clone());
|
||||
if (!events.Writer.TryWrite(message.Clone())) throw new IOException("Client event queue exhausted; consume events regularly.");
|
||||
if (message.Disconnect is not null) { connection.CompleteWrites(); break; }
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { failure = exception; }
|
||||
finally
|
||||
{
|
||||
connectionLifetime?.Cancel();
|
||||
foreach (var operation in pending.Values) operation.TrySetException(failure ?? new IOException("Connection closed."));
|
||||
SetState(ClientConnectionState.Disconnected);
|
||||
}
|
||||
}
|
||||
|
||||
private void Apply(Envelope message)
|
||||
{
|
||||
lock (stateGate)
|
||||
{
|
||||
if (message.AuthResult?.Ok == true) authentication = message.AuthResult.Clone();
|
||||
if (message.ServerState is not null)
|
||||
{
|
||||
channels.Clear(); users.Clear();
|
||||
foreach (var channel in message.ServerState.Channels) channels[channel.Id] = channel.Clone();
|
||||
foreach (var user in message.ServerState.Users) users[user.Id] = user.Clone();
|
||||
}
|
||||
if (message.ChannelEvent is not null)
|
||||
{
|
||||
if (message.ChannelEvent.Kind == ChannelEvent.Types.Kind.Deleted) channels.Remove(message.ChannelEvent.DeletedId);
|
||||
else if (message.ChannelEvent.Channel is not null) channels[message.ChannelEvent.Channel.Id] = message.ChannelEvent.Channel.Clone();
|
||||
}
|
||||
if (message.UserEvent is not null)
|
||||
{
|
||||
if (message.UserEvent.Kind == UserEvent.Types.Kind.Left) users.Remove(message.UserEvent.LeftId);
|
||||
else if (message.UserEvent.User is not null) users[message.UserEvent.User.Id] = message.UserEvent.User.Clone();
|
||||
}
|
||||
if (authentication is not null && (message.ServerState is not null || message.UserEvent is not null))
|
||||
{
|
||||
User self = users.GetValueOrDefault(authentication.Self.Id, authentication.Self);
|
||||
Audio.SetRemoteStreams(users.Values.ToArray(), self.Id, self.ChannelId);
|
||||
foreach (var id in localStreams.Keys.Where(id => !self.Streams.Any(s => s.StreamId == id)).ToArray())
|
||||
{ Audio.RemoveLocalStream(id); localStreams.Remove(id); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void SetState(ClientConnectionState value)
|
||||
{
|
||||
lock (stateGate) state = value;
|
||||
ConnectionStateChanged?.Invoke(value);
|
||||
}
|
||||
|
||||
public async Task DisconnectAsync()
|
||||
{
|
||||
connectionLifetime?.Cancel();
|
||||
await lifecycle.WaitAsync().ConfigureAwait(false);
|
||||
try { await CloseAsync().ConfigureAwait(false); }
|
||||
finally { lifecycle.Release(); }
|
||||
}
|
||||
|
||||
private async Task CloseAsync()
|
||||
{
|
||||
connectionLifetime?.Cancel();
|
||||
try { await reader.ConfigureAwait(false); }
|
||||
finally
|
||||
{
|
||||
try { await keepalive.ConfigureAwait(false); }
|
||||
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
|
||||
try { if (media is not null) await media.DisposeAsync().ConfigureAwait(false); }
|
||||
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
|
||||
try { if (control is not null) await control.DisposeAsync().ConfigureAwait(false); }
|
||||
catch (Exception exception) when (exception is IOException or OperationCanceledException or SocketException or ObjectDisposedException) { }
|
||||
control = null;
|
||||
media = null;
|
||||
mediaCrypto?.Dispose(); mediaCrypto = null;
|
||||
connectionLifetime?.Dispose(); connectionLifetime = null;
|
||||
lock (stateGate) { authentication = null; hello = null; channels.Clear(); users.Clear(); }
|
||||
lock (stateGate)
|
||||
{
|
||||
foreach (var id in localStreams.Keys) Audio.RemoveLocalStream(id);
|
||||
localStreams.Clear(); Audio.SetRemoteStreams([], 0, 0);
|
||||
}
|
||||
SetState(ClientConnectionState.Disconnected);
|
||||
}
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (disposed.IsCancellationRequested) return;
|
||||
disposed.Cancel();
|
||||
await DisconnectAsync().ConfigureAwait(false);
|
||||
events.Writer.TryComplete();
|
||||
Audio.Dispose();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"BouncyCastle.Cryptography": {
|
||||
"type": "Transitive",
|
||||
"resolved": "2.6.2",
|
||||
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"BouncyCastle.Cryptography": "[2.6.2, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
},
|
||||
"BouncyCastle.Cryptography": {
|
||||
"type": "Transitive",
|
||||
"resolved": "2.6.2",
|
||||
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
|
||||
},
|
||||
"Google.Protobuf": {
|
||||
"type": "Transitive",
|
||||
"resolved": "3.36.1",
|
||||
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"BouncyCastle.Cryptography": "[2.6.2, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"Google.Protobuf": "[3.36.1, )"
|
||||
}
|
||||
}
|
||||
},
|
||||
"net10.0/win-x64": {}
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
using VoiceCat.Crypto;
|
||||
|
||||
namespace VoiceCat.Server.Transport;
|
||||
namespace VoiceCat.Transport;
|
||||
|
||||
internal sealed class MediaSessionCrypto(MediaEncryptor encryptor, MediaDecryptor decryptor) : IDisposable
|
||||
{
|
||||
+2
-1
@@ -7,7 +7,7 @@ using VoiceCat.Crypto;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Server.Transport;
|
||||
namespace VoiceCat.Transport;
|
||||
|
||||
internal sealed class TlsControlConnection : IAsyncDisposable
|
||||
{
|
||||
@@ -25,6 +25,7 @@ internal sealed class TlsControlConnection : IAsyncDisposable
|
||||
private MediaSessionCrypto? mediaCrypto;
|
||||
|
||||
public Task Completion { get; }
|
||||
internal System.Net.EndPoint RemoteEndPoint => socket.RemoteEndPoint!;
|
||||
public CancellationToken CancellationToken => lifetime.Token;
|
||||
|
||||
internal TlsControlConnection(Socket socket, TlsSession tls, CancellationToken cancellationToken, TimeSpan? handshakeTimeout = null)
|
||||
@@ -5,5 +5,7 @@
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<InternalsVisibleTo Include="VoiceCat.Tests" />
|
||||
<InternalsVisibleTo Include="VoiceCat.Server" />
|
||||
<InternalsVisibleTo Include="VoiceCat.Core" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"version": 1,
|
||||
"dependencies": {
|
||||
"net10.0": {
|
||||
"Microsoft.NET.ILLink.Tasks": {
|
||||
"type": "Direct",
|
||||
"requested": "[10.0.7, )",
|
||||
"resolved": "10.0.7",
|
||||
"contentHash": "AA/yhzFHNtQZXLdqjzujPy25G8EWwGWsAnxOE2zYSBoT/8QHP6ketN3CToD3DFreO653ipUwnKHo22B8AlBMCw=="
|
||||
}
|
||||
},
|
||||
"net10.0/win-x64": {}
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using System.Security.Cryptography;
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
using VoiceCat.Audio;
|
||||
using VoiceCat.Codec;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
|
||||
public class AudioEngineTests
|
||||
{
|
||||
[Theory]
|
||||
[InlineData(true)]
|
||||
[InlineData(false)]
|
||||
public void LostFramesUseDredThenFecBeforeBoundedPlc(bool useDred)
|
||||
{
|
||||
using var receive = new ReceiveStream(2, Stream(dred: useDred));
|
||||
using var encoder = new OpusEncoder(new() { DeepRedundancy = useDred, ForwardErrorCorrection = true, ExpectedPacketLossPercent = 30, Complexity = 10, Bitrate = 64000 });
|
||||
byte[] packet = new byte[1275]; short[] tone = Tone(); int[] output = new int[1920];
|
||||
for (uint i = 0; i < 40; i++)
|
||||
{
|
||||
CodecTests.FillTone(tone, 960, 1, 48000, (int)i);
|
||||
int size = encoder.Encode(tone, packet);
|
||||
if (i != 25 && i != 30 && i != 35) receive.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, size));
|
||||
output.AsSpan().Clear(); receive.Mix(output, false, null);
|
||||
}
|
||||
if (useDred) Assert.True(receive.DredFrames > 0); else Assert.True(receive.FecFrames > 0);
|
||||
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); receive.Mix(output, false, null); }
|
||||
Assert.True(receive.ConcealedFrames > 0); Assert.All(output, sample => Assert.Equal(0, sample));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PcmRingDropsWholeFramesWhenFullAndPreservesOrderAcrossWraps()
|
||||
{
|
||||
var ring = new PcmRing(8); short[] output = new short[8];
|
||||
for (int i = 0; i < 100; i++)
|
||||
{
|
||||
Assert.True(ring.TryWrite([1, 2, 3, 4, 5, 6])); Assert.False(ring.TryWrite([7, 8, 9]));
|
||||
Assert.Equal(4, ring.Read(output.AsSpan(0, 4))); Assert.Equal(new short[] { 1, 2, 3, 4 }, output[..4]);
|
||||
Assert.True(ring.TryWrite([7, 8])); Assert.Equal(4, ring.Read(output)); Assert.Equal(new short[] { 5, 6, 7, 8 }, output[..4]); Assert.Equal(0, ring.Count);
|
||||
}
|
||||
}
|
||||
internal static StreamInfo Stream(int frame = 20, bool stereo = false, bool dred = false) => new()
|
||||
{
|
||||
StreamId = 1, Ssrc = 42, Kind = StreamKind.StreamMic,
|
||||
Audio = new() { SampleRate = 48000, BitrateBps = 32000, FrameMs = (uint)frame, Complexity = 5,
|
||||
Mode = stereo ? ChannelMode.ModeStereo : ChannelMode.ModeMono, Fec = true, ExpectedPacketLoss = 20, Dred = dred }
|
||||
};
|
||||
private static short[] Tone(int channels = 1)
|
||||
{
|
||||
var pcm = new short[960 * channels];
|
||||
for (int i = 0; i < 960; i++) for (int c = 0; c < channels; c++) pcm[i * channels + c] = (short)(Math.Sin(i * 2 * Math.PI * (c == 0 ? 440 : 660) / 48000) * 8000);
|
||||
return pcm;
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(5, false)] [InlineData(10, false)] [InlineData(20, false)] [InlineData(40, false)] [InlineData(60, false)] [InlineData(20, true)]
|
||||
public void ReframedEncodedPcmIsDecodedAndMixedForMonoAndStereo(int frame, bool stereo)
|
||||
{
|
||||
StreamInfo stream = Stream(frame, stereo);
|
||||
using var receive = new AudioEngine((_, _, _, _) => true, false);
|
||||
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { stream } }], 1, 1);
|
||||
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
|
||||
send.InputMode = AudioInputMode.AlwaysOn; send.AddLocalStream(stream, stereo ? 2 : 1);
|
||||
long energy = 0; int sinkChannels = 0;
|
||||
receive.MixedPcm += pcm => { foreach (short sample in pcm) energy += Math.Abs((int)sample); };
|
||||
receive.StreamPcm += (_, _, _, channels) => sinkChannels = channels;
|
||||
short[] tone = Tone(stereo ? 2 : 1);
|
||||
for (int i = 0; i < 30; i++) { Assert.True(send.FeedPcm(1, tone, stereo ? 2 : 1)); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
Assert.True(energy > 100000); Assert.Equal(stereo ? 2 : 1, sinkChannels);
|
||||
receive.SetRemotePlayback(2, 1, 1, true, false); energy = 0;
|
||||
for (int i = 0; i < 5; i++) { send.FeedPcm(1, tone, stereo ? 2 : 1); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
Assert.Equal(0, energy);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AudioCyclesAllocateZeroBytesWithEncodeDecodeStereoNoiseReductionAndMixing()
|
||||
{
|
||||
StreamInfo stream = Stream(20, true);
|
||||
using var receive = new AudioEngine((_, _, _, _) => true, false);
|
||||
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { stream } }], 1, 1);
|
||||
receive.SetRemotePlayback(2, 1, 0.8f, false, true);
|
||||
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
|
||||
send.InputMode = AudioInputMode.AlwaysOn; send.InputNoiseReduction = true; send.AddLocalStream(stream, 2);
|
||||
short[] tone = Tone(2);
|
||||
for (int i = 0; i < 30; i++) { send.FeedPcm(1, tone, 2); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
long before = GC.GetAllocatedBytesForCurrentThread();
|
||||
for (int i = 0; i < 100; i++) { send.FeedPcm(1, tone, 2); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void JitterBacklogIsBoundedAndPlcEventuallyBecomesSilence()
|
||||
{
|
||||
var info = Stream(); using var stream = new ReceiveStream(2, info); using var encoder = new OpusEncoder(new() { Bitrate = 32000 });
|
||||
byte[] packet = new byte[1275]; int length = encoder.Encode(Tone(), packet); int[] output = new int[1920];
|
||||
for (uint i = 0; i < 64; i++) stream.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, length));
|
||||
stream.Mix(output, false, null); Assert.InRange(stream.Depth, 0, 6);
|
||||
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); stream.Mix(output, false, null); }
|
||||
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 10);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PttResumeAndCaptureChannelChangesKeepTimestampProgressAndAudio()
|
||||
{
|
||||
var info = Stream(60, true); using var receive = new AudioEngine((_, _, _, _) => true, false);
|
||||
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { info } }], 1, 1);
|
||||
using var send = new AudioEngine((ssrc, timestamp, payload, flags) => { receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload); return true; }, false);
|
||||
send.InputMode = AudioInputMode.PushToTalk; send.PushToTalk = true; send.AddLocalStream(info);
|
||||
short[] mono = Tone(), stereo = Tone(2); long energy = 0;
|
||||
receive.MixedPcm += pcm => { foreach (short value in pcm) energy += Math.Abs((int)value); };
|
||||
for (int i = 0; i < 15; i++) { send.FeedPcm(1, mono, 1); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
send.PushToTalk = false;
|
||||
for (int i = 0; i < 16; i++) { send.FeedPcm(1, mono, 1); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
send.SetCaptureChannels(1, 2); send.PushToTalk = true; energy = 0;
|
||||
for (int i = 0; i < 15; i++) { send.FeedPcm(1, stereo, 2); send.ProcessCycle(); receive.ProcessCycle(); }
|
||||
Assert.True(energy > 100000);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
using VoiceCat.Core;
|
||||
using VoiceCat.Crypto;
|
||||
using Voicecat.V1;
|
||||
using static VoiceCat.Tests.ServerTests;
|
||||
using static VoiceCat.Tests.MediaRelayTests;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
|
||||
public class ManagedClientTests
|
||||
{
|
||||
private static VoiceCatClient NewClient(ServerFixture fixture, string name) => new(name, "test", Path.Combine(fixture.Directory, name + ".pins"));
|
||||
private static Task Connect(VoiceCatClient client, ServerFixture fixture) => client.ConnectAsync("127.0.0.1", (ushort)fixture.Server.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
|
||||
private static async Task<Envelope> Event(VoiceCatClient client, Func<Envelope, bool> predicate)
|
||||
{
|
||||
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10));
|
||||
await foreach (Envelope message in client.ReadEventsAsync(timeout.Token)) if (predicate(message)) return message;
|
||||
throw new IOException("Expected client event was not received.");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ManagedClientsAuthenticateChatAndCorrelateConcurrentRequests()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var alice = NewClient(fixture, "Alice"); await using var bob = NewClient(fixture, "Bob");
|
||||
await Connect(alice, fixture); await Connect(bob, fixture);
|
||||
Assert.True((await alice.AuthenticateGuestAsync("Alice")).Ok); Assert.True((await bob.AuthenticateGuestAsync("Bob")).Ok);
|
||||
await Event(bob, e => e.ServerState is not null); await Event(alice, e => e.UserEvent?.User?.Nickname == "Bob");
|
||||
Assert.Equal(2, alice.Users.Count);
|
||||
var copy = alice.Users[0]; copy.Nickname = "Mutated"; Assert.DoesNotContain(alice.Users, u => u.Nickname == "Mutated");
|
||||
alice.Send(new() { TextMessage = new() { Scope = TextScope.TextChannel, TargetId = 1, Body = "Managed conversation", ClientMsgId = "a1" } });
|
||||
Assert.Equal("Managed conversation", (await Event(bob, e => e.TextMessage is not null)).TextMessage.Body);
|
||||
var requests = Enumerable.Range(1, 20).Select(async i =>
|
||||
{
|
||||
Envelope response = await alice.RequestAsync(new() { Ping = new() { Nonce = (ulong)i } });
|
||||
Assert.Equal((ulong)i, response.Pong.Nonce); return response.RequestId;
|
||||
});
|
||||
Assert.Equal(20, (await Task.WhenAll(requests)).Distinct().Count());
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(() => Connect(alice, fixture));
|
||||
Assert.Equal(ClientConnectionState.Connected, alice.State);
|
||||
await alice.DisconnectAsync();
|
||||
await Event(bob, e => e.UserEvent?.Kind == UserEvent.Types.Kind.Left);
|
||||
await Connect(alice, fixture); Assert.True((await alice.AuthenticateGuestAsync("Returned")).Ok);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TofuRequiresApprovalPinsAcceptedCertificateAndRejectsChanges()
|
||||
{
|
||||
await using var first = new ServerFixture(); await using var second = new ServerFixture();
|
||||
await using var client = NewClient(first, "Tofu");
|
||||
await Assert.ThrowsAsync<System.Security.Authentication.AuthenticationException>(() => client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port));
|
||||
await client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port, (challenge, _) =>
|
||||
{ Assert.Equal(TofuStatus.FirstConnect, challenge.Status); return ValueTask.FromResult(true); });
|
||||
await client.DisconnectAsync();
|
||||
await client.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port); await client.DisconnectAsync();
|
||||
// Pin the other server's certificate to this endpoint, simulating a changed server certificate.
|
||||
using var credentials = ServerCredentials.LoadOrCreate(second.Directory, "VoiceCat Server");
|
||||
new TofuStore(Path.Combine(first.Directory, "Other.pins")).Pin("127.0.0.1", (ushort)first.Server.EndPoint.Port, credentials.CertificateFingerprint);
|
||||
await using var changed = new VoiceCatClient(tofuStorePath: Path.Combine(first.Directory, "Other.pins"));
|
||||
await Assert.ThrowsAsync<System.Security.Authentication.AuthenticationException>(() => changed.ConnectAsync("127.0.0.1", (ushort)first.Server.EndPoint.Port,
|
||||
(challenge, _) => { Assert.Equal(TofuStatus.Mismatch, challenge.Status); return ValueTask.FromResult(false); }));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ManagedClientSendsAndReceivesAuthenticatedEncodedVoice()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var managed = NewClient(fixture, "Managed"); await Connect(managed, fixture); await managed.AuthenticateGuestAsync("Managed");
|
||||
Assert.True((await managed.SubscribeVoiceAsync()).Ok);
|
||||
await using var peer = await VoicePeer.ConnectAsync(fixture, "Peer");
|
||||
var remote = await peer.AnnounceAsync(StreamKind.StreamMic);
|
||||
var local = (await managed.RequestAsync(new() { StreamAnnounce = new() { Kind = StreamKind.StreamMic } })).StreamAnnounceResult;
|
||||
Assert.True(local.Ok);
|
||||
var received = new TaskCompletionSource<byte[]>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
managed.VoiceReceived += (header, payload) => { Assert.Equal(remote.Ssrc, header.Ssrc); received.TrySetResult(payload.ToArray()); };
|
||||
await peer.SendAsync(peer.Seal(remote.Ssrc, [1, 2, 3]));
|
||||
Assert.Equal(new byte[] { 1, 2, 3 }, await received.Task.WaitAsync(TimeSpan.FromSeconds(5)));
|
||||
Assert.True(managed.TrySendEncodedVoice(local.Ssrc, 960, [4, 5, 6]));
|
||||
Assert.Equal(new byte[] { 4, 5, 6 }, (await peer.ReceiveVoiceAsync()).Payload);
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Net;
|
||||
using System.Diagnostics;
|
||||
using System.Net.Sockets;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using System.Text.Json;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Diagnostics;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
<IsTestProject>true</IsTestProject>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="../../../clients/windows/VoiceCat.Managed/VoiceCat.Managed.csproj" />
|
||||
<ProjectReference Include="../../src/VoiceCat.Core/VoiceCat.Core.csproj" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
|
||||
<PackageReference Include="xunit" Version="2.9.3" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.1" PrivateAssets="all" />
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
using VoiceCat.Interop;
|
||||
using VoiceCat.Server.Data;
|
||||
using Voicecat.V1;
|
||||
using static VoiceCat.Tests.ServerTests;
|
||||
using Client = VoiceCat.Interop.VoiceCatClient;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
|
||||
public class WindowsManagedClientTests
|
||||
{
|
||||
private static async Task Until(Client client, Func<bool> predicate)
|
||||
{
|
||||
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10));
|
||||
do { client.PumpEvents(); if (predicate()) return; await Task.Delay(10, timeout.Token); } while (true);
|
||||
}
|
||||
private static async Task Login(Client client, ServerFixture fixture, bool admin = false)
|
||||
{
|
||||
client.EventReceived += e => { if (e.Type == VcEventType.ServerIdentity) client.ConfirmServerIdentity(true); };
|
||||
Assert.Equal(VcResult.Ok, client.Connect("127.0.0.1", (ushort)fixture.Server.EndPoint.Port));
|
||||
if (admin) client.AuthenticateUser("Admin", "secret"); else client.AuthenticateGuest("Guest");
|
||||
await Until(client, () => client.ListUsers().Count > 0);
|
||||
}
|
||||
[Fact]
|
||||
public async Task ShippedWindowsFacadeChatsExchangesPcmAndKeepsCaptureIdsAcrossChannelMoves()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
using (var accounts = new AccountStore(Path.Combine(fixture.Directory, "voicecat.db"))) await accounts.CreateAccountAsync("Admin", "secret", true);
|
||||
using var alice = new Client("Alice", "test", tofuStorePath: Path.Combine(fixture.Directory, "alice.pins"));
|
||||
using var bob = new Client("Bob", "test", tofuStorePath: Path.Combine(fixture.Directory, "bob.pins"));
|
||||
await Login(alice, fixture, true); await Login(bob, fixture);
|
||||
Assert.True(alice.GetPermissions().IsAdmin);
|
||||
string? body = null; bob.EventReceived += e => { if (e.Type == VcEventType.TextMessage) body = e.Text; };
|
||||
Assert.Equal(VcResult.Ok, alice.SendText(VcTextScope.Channel, 1, "Managed Windows chat"));
|
||||
await Until(bob, () => body is not null); Assert.Equal("Managed Windows chat", body);
|
||||
Assert.Equal(VcResult.Ok, alice.JoinVoice()); Assert.Equal(VcResult.Ok, bob.JoinVoice());
|
||||
alice.SetInputMode(VcInputMode.AlwaysOn); bob.SetInputMode(VcInputMode.AlwaysOn);
|
||||
var a = alice.StartStreamExternalFeed(VcStreamKind.Mic, "Mic"); var b = bob.StartStreamExternalFeed(VcStreamKind.Mic, "Mic");
|
||||
Assert.Equal(VcResult.Ok, a.Result); Assert.Equal(VcResult.Ok, b.Result);
|
||||
await Until(bob, () => bob.ManagedClient.Users.Any(u => u.Streams.Count > 0 && u.Id != bob.ManagedClient.Authentication!.Self.Id));
|
||||
long aliceEnergy = 0, bobEnergy = 0;
|
||||
alice.ManagedClient.Audio.MixedPcm += pcm => { long sum = 0; foreach (short sample in pcm) sum += Math.Abs((int)sample); Interlocked.Add(ref aliceEnergy, sum); };
|
||||
bob.ManagedClient.Audio.MixedPcm += pcm => { long sum = 0; foreach (short sample in pcm) sum += Math.Abs((int)sample); Interlocked.Add(ref bobEnergy, sum); };
|
||||
short[] tone = Enumerable.Range(0, 960).Select(i => (short)(8000 * Math.Sin(i * Math.PI * 880 / 48000))).ToArray();
|
||||
for (int i = 0; i < 40; i++) { alice.StreamFeedPcm(a.StreamId, tone, 960, 1); bob.StreamFeedPcm(b.StreamId, tone, 960, 1); alice.PumpEvents(); bob.PumpEvents(); await Task.Delay(20); }
|
||||
Assert.True(Interlocked.Read(ref aliceEnergy) > 100000); Assert.True(Interlocked.Read(ref bobEnergy) > 100000);
|
||||
uint oldId = alice.ManagedClient.LocalStreams.Single().StreamId;
|
||||
await alice.ManagedClient.RequestAsync(new() { CreateChannel = new() { Channel = new() { Name = "Stereo", ParentId = 1, Audio = AudioEngineTests.Stream(20, true).Audio } } });
|
||||
await Until(alice, () => alice.ListChannels().Any(c => c.Name == "Stereo"));
|
||||
uint channel = alice.ListChannels().Single(c => c.Name == "Stereo").Id;
|
||||
Assert.Equal(VcResult.Ok, alice.JoinChannel(channel));
|
||||
await Until(alice, () => alice.ManagedClient.LocalStreams.Any(s => s.StreamId != oldId));
|
||||
Assert.Equal(a.StreamId, Assert.Single(alice.ListUserStreams(alice.ManagedClient.Authentication!.Self.Id)).StreamId);
|
||||
Assert.True(alice.GetStreamAudioConfig(alice.ManagedClient.Authentication!.Self.Id, a.StreamId).Config!.Stereo);
|
||||
Assert.Equal(VcResult.Ok, alice.StreamFeedPcm(a.StreamId, tone, 960, 1));
|
||||
Assert.Equal(VcResult.Ok, alice.StopStream(a.StreamId));
|
||||
Assert.Empty(alice.ManagedClient.LocalStreams);
|
||||
}
|
||||
}
|
||||
@@ -146,9 +146,24 @@
|
||||
"xunit.extensibility.core": "[2.9.3]"
|
||||
}
|
||||
},
|
||||
"voicecat.audio": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Codec": "[1.0.0, )",
|
||||
"VoiceCat.Dsp": "[1.0.0, )",
|
||||
"VoiceCat.Protocol": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.codec": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.core": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Audio": "[1.0.0, )",
|
||||
"VoiceCat.Crypto": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.crypto": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
@@ -159,6 +174,12 @@
|
||||
"voicecat.dsp": {
|
||||
"type": "Project"
|
||||
},
|
||||
"voicecat.managed": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
"VoiceCat.Core": "[1.0.0, )"
|
||||
}
|
||||
},
|
||||
"voicecat.protocol": {
|
||||
"type": "Project",
|
||||
"dependencies": {
|
||||
|
||||
Reference in New Issue
Block a user