Complete managed macOS platform bring-up
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s

This commit is contained in:
2026-09-18 18:51:53 +02:00
parent ab460ae12b
commit 310c0f09dd
21 changed files with 421 additions and 83 deletions
+1 -1
View File
@@ -40,7 +40,7 @@ jobs:
cache: true cache: true
cache-dependency-path: dotnet/**/packages.lock.json cache-dependency-path: dotnet/**/packages.lock.json
- name: Install macOS workload - name: Install macOS workload
run: dotnet workload install macos --skip-manifest-update run: dotnet workload install macos --version 10.0.401
- name: Build and stage native codec/DSP - name: Build and stage native codec/DSP
shell: pwsh shell: pwsh
run: ./dotnet/build-native.ps1 run: ./dotnet/build-native.ps1
+2
View File
@@ -41,6 +41,8 @@ Thumbs.db
# Apple / Windows client build artifacts # Apple / Windows client build artifacts
clients/apple/**/build/ clients/apple/**/build/
clients/apple/**/bin/
clients/apple/**/obj/
clients/apple/**/*.xcodeproj/xcuserdata/ clients/apple/**/*.xcodeproj/xcuserdata/
clients/apple/**/*.xcodeproj/project.xcworkspace/ clients/apple/**/*.xcodeproj/project.xcworkspace/
clients/apple/**/*.xcframework/ clients/apple/**/*.xcframework/
+26 -10
View File
@@ -10,16 +10,16 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action ## ▶ Where we left off / next action
- **In progress (2026-09-16): managed macOS client and default audio path.** Added a - **In progress (2026-09-18): managed macOS client and selectable audio path.** Added a
separate .NET 10 AppKit solution with application/menu lifecycle, guest/account connection, separate .NET 10 AppKit solution with application/menu lifecycle, guest/account connection,
explicit TOFU approval, channel selection, roster/chat, disconnect state, accessibility explicit TOFU approval, channel selection, roster/chat, disconnect state, accessibility
labels, sandbox entitlements and deterministic disposal. Voice join now opens bounded labels, sandbox entitlements and deterministic disposal. Voice join now opens bounded
stereo playback, subscribes, announces a managed microphone stream, converts the default stereo playback, subscribes, announces a managed microphone stream, converts the selected
Core Audio input to interleaved 48 kHz int16 and feeds `VoiceCat.Audio`; leave/disconnect Core Audio input to interleaved 48 kHz int16 and feeds `VoiceCat.Audio`; leave/disconnect
tears the path down. Channel changes suspend and resume voice so streams renegotiate. tears the path down. Channel changes suspend and resume voice so streams renegotiate.
Playback uses `AVAudioSourceNode` plus the shared tested `PcmRing`; the render callback Playback uses `AVAudioSourceNode` plus the shared tested `PcmRing`, converting to Core Audio's
allocates no managed memory and never locks or blocks. Current AppKit/AVFoundation calls native planar Float32 layout; the render callback allocates no managed memory and never locks
compile warning-free against Microsoft's 26.4.10259 macOS reference assembly. Added a or blocks. Added a
macOS CI gate for the real workload, native Opus/RNNoise shim and Apple solution. Local macOS CI gate for the real workload, native Opus/RNNoise shim and Apple solution. Local
server profiles now persist validated host, port, authentication mode and identity through server profiles now persist validated host, port, authentication mode and identity through
`VoiceCat.Core`; malformed profile files do not prevent startup, writes replace atomically, `VoiceCat.Core`; malformed profile files do not prevent startup, writes replace atomically,
@@ -31,11 +31,27 @@ up instantly. Newest status at the top.
successful authentication and removed when remembering is disabled or the profile is deleted. successful authentication and removed when remembering is disabled or the profile is deleted.
The composer can now target the current channel or any connected user, and labels private The composer can now target the current channel or any connected user, and labels private
messages with the other participant. messages with the other participant.
Workload installation remains On Apple Silicon macOS 27, SDK/workload set 10.0.401 now builds against the explicit
blocked by this Windows machine's unrelated Visual Studio `net10.0-macos27.0` preview target. The build selects native arm64 Homebrew `protoc`,
iOS/Android MSI repair failure, so device execution is not claimed. **Next:** run the deduplicates the media dylib during app bundling and uses macOS 27's error-returning
macOS CI/device capture-playback test and ten-minute listen gate, then add non-default devices, AVFoundation overloads while preserving macOS 1426 calls. Debug disables hardened runtime
moderation/settings, ScreenCaptureKit, so its ad-hoc-signed .NET dylibs load without a developer identity; Release remains hardened.
The arm64 bundle is valid, launches successfully and displays the connection window on a
physical macOS 27 host. The app now creates missing private-file parent directories on first
launch, requests microphone authorization explicitly, and enumerates/selects actual Core Audio
input and output devices; changing either device while joined restarts voice cleanly. Native
validation on macOS also exposed a PCM-tap teardown race:
disabling either external sink is now a quiescence barrier, so callback state can be released
safely without adding locks or waits to the real-time path. **Verified:** managed locked
restore and warning-free Release build; 188/196 managed tests passed with eight optional
native/published-server oracle tests skipped; native `dev` build and 29/29 CTest tests passed;
both PCM teardown regressions plus external playback passed five consecutive runs. Live testing
selected the internal MacBook microphone, proved nonzero mic energy at a peer, and played a
clean sustained peer tone in the DTX-disabled Music Room. Lobby's later hiss was confirmed as
expected Opus DTX comfort noise for a synthetic sine, not playback corruption. Per user
direction, final listen quality will be exercised in real multi-human calls after feature
completion rather than an artificial ten-minute tone. **Next:** add moderation/settings,
ScreenCaptureKit,
VoiceOver verification, signing and notarization. VoiceOver verification, signing and notarization.
- **Done (2026-09-16): Linux production packaging checkpoint.** Added a real TLS 1.3 - **Done (2026-09-16): Linux production packaging checkpoint.** Added a real TLS 1.3
+10 -6
View File
@@ -1,16 +1,20 @@
# Managed Apple clients # Managed Apple clients
`VoiceCat.Mac` is the native AppKit C# port. It targets `net10.0-macos` and references the same `VoiceCat.Core` and `VoiceCat.Audio` assemblies used by the Windows client and managed CLI. `VoiceCat.Mac` is the native AppKit C# port. It targets `net10.0-macos27.0` and references the same `VoiceCat.Core` and `VoiceCat.Audio` assemblies used by the Windows client and managed CLI. The explicit platform version selects Microsoft's Xcode 27 preview bindings; the deployed app still supports macOS 14 and later.
The current checkpoint is a functional client: AppKit launch/menu lifecycle, saved server profiles, guest and account authentication, explicit TOFU approval (including changed-key warning), protected-channel password prompts, channel selection, user roster, channel and private text, voice subscription, managed microphone-stream lifecycle, default-device Core Audio capture/playback, disconnect reporting, native accessibility labels, sandbox/network/audio entitlements, and deterministic disposal. Profiles save the host, port, authentication mode, username or guest nickname. An account password can be remembered as a macOS Keychain generic-password item only after successful authentication; it is never written to profile JSON and is removed when remembering is disabled or the profile is deleted. Capture is converted to interleaved 48 kHz int16 PCM through `AVAudioConverter`; playback uses an `AVAudioSourceNode` and the shared bounded `PcmRing`, so its render callback does not allocate, lock, or block. It does not yet replace the Swift release. Non-default device selection, moderation sheets, settings, ScreenCaptureKit sharing, VoiceOver verification, signing and notarization remain. The current checkpoint is a functional client: AppKit launch/menu lifecycle, saved server profiles, guest and account authentication, explicit TOFU approval (including changed-key warning), protected-channel password prompts, channel selection, user roster, channel and private text, voice subscription, managed microphone-stream lifecycle, selectable Core Audio input/output devices, disconnect reporting, native accessibility labels, sandbox/network/audio entitlements, and deterministic disposal. Profiles save the host, port, authentication mode, username or guest nickname. An account password can be remembered as a macOS Keychain generic-password item only after successful authentication; it is never written to profile JSON and is removed when remembering is disabled or the profile is deleted. Joining voice requests microphone authorization explicitly. Capture is converted to interleaved 48 kHz int16 PCM through `AVAudioConverter`; playback converts the shared bounded `PcmRing` into Core Audio's native planar Float32 layout inside an allocation-free, non-blocking `AVAudioSourceNode` callback. It does not yet replace the Swift release. Moderation sheets, settings, ScreenCaptureKit sharing, VoiceOver verification, signing and notarization remain.
Build on Apple Silicon macOS 15.6+ with Xcode 26 and the .NET 10 macOS workload: Build on Apple Silicon macOS 27 with Xcode 27, .NET SDK 10.0.401 and workload set 10.0.401. Homebrew `protobuf` supplies a native arm64 `protoc`; the current `Grpc.Tools` package contains only an x64 macOS compiler.
```bash ```bash
dotnet workload install macos sudo dotnet workload install macos ios --version 10.0.401
pwsh ./dotnet/build-native.ps1 brew install protobuf # if /opt/homebrew/bin/protoc is not already present
cmake -S dotnet/native -B dotnet/artifacts/native-build -DCMAKE_BUILD_TYPE=Release -DVOICECAT_DOTNET_RID=osx-arm64
cmake --build dotnet/artifacts/native-build --config Release --target voicecat_media --parallel 2
cmake --install dotnet/artifacts/native-build --config Release --component DotnetMedia --prefix dotnet/artifacts/native
dotnet restore clients/apple/dotnet/VoiceCat.Apple.slnx dotnet restore clients/apple/dotnet/VoiceCat.Apple.slnx
dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx -c Debug dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx -c Debug
open clients/apple/dotnet/VoiceCat.Mac/bin/Debug/net10.0-macos27.0/osx-arm64/VoiceCat.app
``` ```
The native build stages an `osx-arm64` `libvoicecat_media.dylib`; that shim contains only Opus/RNNoise. The AppKit and AVFoundation calls are also compiled against Microsoft's current macOS reference assembly during development, but only a macOS host can link, launch, grant microphone access and verify live devices. The macOS CI build and manual listen test are required release gates. The native build stages an `osx-arm64` `libvoicecat_media.dylib`; that shim contains only Opus/RNNoise. Debug builds deliberately omit hardened runtime so an ad-hoc-signed local app can load the separately ad-hoc-signed .NET runtime libraries without an Apple Development identity. Release builds retain hardened runtime for Developer ID signing and notarization. Only a macOS host can link, launch, grant microphone access and verify live devices. Final audio quality is validated with real multi-human calls after the feature surface is complete; a synthetic ten-minute sine-wave listen is deliberately not a release gate.
@@ -0,0 +1,113 @@
using System.Runtime.InteropServices;
using Foundation;
using ObjCRuntime;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal static class CoreAudioDevices
{
private const string CoreAudio = "/System/Library/Frameworks/CoreAudio.framework/CoreAudio";
private const uint SystemObject = 1;
private const uint Devices = 0x64657623; // 'dev#'
private const uint DefaultInput = 0x64496e20; // 'dIn '
private const uint DefaultOutput = 0x644f7574; // 'dOut'
private const uint StreamConfiguration = 0x736c6179; // 'slay'
private const uint ObjectName = 0x6c6e616d; // 'lnam'
private const uint ScopeGlobal = 0x676c6f62; // 'glob'
private const uint ScopeInput = 0x696e7074; // 'inpt'
private const uint ScopeOutput = 0x6f757470; // 'outp'
[StructLayout(LayoutKind.Sequential)]
private struct PropertyAddress(uint selector, uint scope)
{
public uint Selector = selector;
public uint Scope = scope;
public uint Element;
}
[DllImport(CoreAudio)]
private static extern int AudioObjectGetPropertyDataSize(uint objectId, ref PropertyAddress address,
uint qualifierDataSize, nint qualifierData, out uint dataSize);
[DllImport(CoreAudio)]
private static extern int AudioObjectGetPropertyData(uint objectId, ref PropertyAddress address,
uint qualifierDataSize, nint qualifierData, ref uint dataSize, nint data);
internal static IReadOnlyList<AudioDeviceInfo> List(bool input)
{
uint defaultId = ReadUInt(SystemObject, input ? DefaultInput : DefaultOutput, ScopeGlobal);
var devices = new List<AudioDeviceInfo>();
foreach (uint id in ReadUIntArray(SystemObject, Devices, ScopeGlobal))
{
if (ChannelCount(id, input ? ScopeInput : ScopeOutput) == 0) continue;
string name = ReadString(id, ObjectName, ScopeGlobal) ?? $"Core Audio device {id}";
devices.Add(new(id.ToString(System.Globalization.CultureInfo.InvariantCulture), name, id == defaultId));
}
return devices.OrderByDescending(device => device.IsDefault).ThenBy(device => device.Name, StringComparer.CurrentCultureIgnoreCase).ToArray();
}
internal static uint ParseId(string? value)
{
if (!uint.TryParse(value, System.Globalization.NumberStyles.None, System.Globalization.CultureInfo.InvariantCulture, out uint id) || id == 0)
throw new ArgumentException("The selected Core Audio device is no longer available.", nameof(value));
return id;
}
private static uint ReadUInt(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
uint size = sizeof(uint);
nint memory = Marshal.AllocHGlobal(sizeof(uint));
try { return AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) == 0 ? unchecked((uint)Marshal.ReadInt32(memory)) : 0; }
finally { Marshal.FreeHGlobal(memory); }
}
private static uint[] ReadUIntArray(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
if (AudioObjectGetPropertyDataSize(objectId, ref address, 0, 0, out uint size) != 0 || size < sizeof(uint)) return [];
nint memory = Marshal.AllocHGlobal(checked((int)size));
try
{
if (AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) != 0) return [];
var result = new uint[size / sizeof(uint)];
for (int i = 0; i < result.Length; i++) result[i] = unchecked((uint)Marshal.ReadInt32(memory, i * sizeof(uint)));
return result;
}
finally { Marshal.FreeHGlobal(memory); }
}
private static int ChannelCount(uint deviceId, uint scope)
{
var address = new PropertyAddress(StreamConfiguration, scope);
if (AudioObjectGetPropertyDataSize(deviceId, ref address, 0, 0, out uint size) != 0 || size < 8) return 0;
nint memory = Marshal.AllocHGlobal(checked((int)size));
try
{
if (AudioObjectGetPropertyData(deviceId, ref address, 0, 0, ref size, memory) != 0) return 0;
int count = Marshal.ReadInt32(memory);
int first = IntPtr.Size == 8 ? 8 : 4;
int stride = IntPtr.Size == 8 ? 16 : 12;
int channels = 0;
for (int i = 0; i < count && first + i * stride + sizeof(uint) <= size; i++)
channels += Marshal.ReadInt32(memory, first + i * stride);
return channels;
}
finally { Marshal.FreeHGlobal(memory); }
}
private static string? ReadString(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
uint size = checked((uint)IntPtr.Size);
nint memory = Marshal.AllocHGlobal(IntPtr.Size);
try
{
if (AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) != 0) return null;
nint handle = Marshal.ReadIntPtr(memory);
return handle == 0 ? null : Runtime.GetNSObject<NSString>(handle, owns: false)?.ToString();
}
finally { Marshal.FreeHGlobal(memory); }
}
}
@@ -1,4 +1,5 @@
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using AudioUnit;
using AVFoundation; using AVFoundation;
using Foundation; using Foundation;
using VoiceCat.Audio; using VoiceCat.Audio;
@@ -7,25 +8,21 @@ namespace VoiceCat.Mac;
internal sealed class MacAudioBackend : IAudioDeviceBackend internal sealed class MacAudioBackend : IAudioDeviceBackend
{ {
private static readonly AudioDeviceInfo[] DefaultInput = [new("", "System default microphone", true)];
private static readonly AudioDeviceInfo[] DefaultOutput = [new("", "System default output", true)];
private Exception? failure; private Exception? failure;
internal Exception? Failure => Volatile.Read(ref failure); internal Exception? Failure => Volatile.Read(ref failure);
public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => input ? DefaultInput : DefaultOutput; public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => CoreAudioDevices.List(input);
public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler pcm) public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler pcm)
{ {
if (loopback) throw new NotSupportedException("Screen audio uses ScreenCaptureKit, not microphone capture."); if (loopback) throw new NotSupportedException("Screen audio uses ScreenCaptureKit, not microphone capture.");
if (!string.IsNullOrEmpty(deviceId)) throw new NotSupportedException("Selecting a non-default Core Audio input is not implemented yet."); return new Capture(deviceId, pcm, ReportFailure);
return new Capture(pcm, ReportFailure);
} }
public IAudioPlayback OpenPlayback(string? deviceId = null) public IAudioPlayback OpenPlayback(string? deviceId = null)
{ {
if (!string.IsNullOrEmpty(deviceId)) throw new NotSupportedException("Selecting a non-default Core Audio output is not implemented yet."); return new Playback(deviceId, ReportFailure);
return new Playback(ReportFailure);
} }
private void ReportFailure(Exception exception) => Interlocked.CompareExchange(ref failure, exception, null); private void ReportFailure(Exception exception) => Interlocked.CompareExchange(ref failure, exception, null);
@@ -43,11 +40,16 @@ internal sealed class MacAudioBackend : IAudioDeviceBackend
private bool inputProvided; private bool inputProvided;
private int disposed; private int disposed;
internal Capture(CapturePcmHandler handler, Action<Exception> reportFailure) internal Capture(string? deviceId, CapturePcmHandler handler, Action<Exception> reportFailure)
{ {
this.handler = handler; this.handler = handler;
this.reportFailure = reportFailure; this.reportFailure = reportFailure;
AVAudioInputNode input = engine.InputNode; AVAudioInputNode input = engine.InputNode;
if (!string.IsNullOrEmpty(deviceId))
{
AudioUnitStatus status = input.AudioUnit!.SetCurrentDevice(CoreAudioDevices.ParseId(deviceId), AudioUnitScopeType.Global, 0);
if (status != AudioUnitStatus.NoError) throw new InvalidOperationException($"Core Audio input selection failed ({status}).");
}
AVAudioFormat inputFormat = input.GetBusOutputFormat(0); AVAudioFormat inputFormat = input.GetBusOutputFormat(0);
uint channels = Math.Clamp(inputFormat.ChannelCount, 1u, 2u); uint channels = Math.Clamp(inputFormat.ChannelCount, 1u, 2u);
outputFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, channels, true); outputFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, channels, true);
@@ -55,7 +57,19 @@ internal sealed class MacAudioBackend : IAudioDeviceBackend
uint outputCapacity = checked((uint)Math.Ceiling(4_096 * 48_000 / inputFormat.SampleRate) + 64); uint outputCapacity = checked((uint)Math.Ceiling(4_096 * 48_000 / inputFormat.SampleRate) + 64);
converted = new(outputFormat, outputCapacity); converted = new(outputFormat, outputCapacity);
inputProvider = ProvideInput; inputProvider = ProvideInput;
NSError? tapError = null;
if (OperatingSystem.IsMacOSVersionAtLeast(27))
input.InstallTapOnBus(0, 960, inputFormat, out tapError, Convert);
else
input.InstallTapOnBus(0, 960, inputFormat, Convert); input.InstallTapOnBus(0, 960, inputFormat, Convert);
if (tapError is not null)
{
converted.Dispose();
converter.Dispose();
outputFormat.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio capture tap could not be installed: " + tapError.LocalizedDescription);
}
engine.Prepare(); engine.Prepare();
if (!engine.StartAndReturnError(out var error)) if (!engine.StartAndReturnError(out var error))
{ {
@@ -126,6 +140,7 @@ internal sealed class MacAudioBackend : IAudioDeviceBackend
private sealed class Playback : IAudioPlayback private sealed class Playback : IAudioPlayback
{ {
private readonly PcmRing pcm = new(32_768); private readonly PcmRing pcm = new(32_768);
private readonly short[] renderScratch = new short[8_192];
private readonly AVAudioEngine engine = new(); private readonly AVAudioEngine engine = new();
private readonly AVAudioFormat format; private readonly AVAudioFormat format;
private readonly AVAudioSourceNode source; private readonly AVAudioSourceNode source;
@@ -133,13 +148,33 @@ internal sealed class MacAudioBackend : IAudioDeviceBackend
private int callbackFailed; private int callbackFailed;
private int disposed; private int disposed;
internal Playback(Action<Exception> reportFailure) internal Playback(string? deviceId, Action<Exception> reportFailure)
{ {
this.reportFailure = reportFailure; this.reportFailure = reportFailure;
format = new(AVAudioCommonFormat.PCMInt16, 48_000, 2, true); if (!string.IsNullOrEmpty(deviceId))
{
AudioUnitStatus status = engine.OutputNode.AudioUnit!.SetCurrentDevice(CoreAudioDevices.ParseId(deviceId), AudioUnitScopeType.Global, 0);
if (status != AudioUnitStatus.NoError) throw new InvalidOperationException($"Core Audio output selection failed ({status}).");
}
// AVAudioEngine's native mixer path is planar Float32. Convert from the managed
// Int16 ring directly in this allocation-free callback, avoiding an extra graph
// converter and matching the established Swift/VPIO renderer.
format = new(AVAudioCommonFormat.PCMFloat32, 48_000, 2, false);
source = new(format, Render); source = new(format, Render);
engine.AttachNode(source); engine.AttachNode(source);
NSError? connectionError = null;
if (OperatingSystem.IsMacOSVersionAtLeast(27))
engine.Connect(source, engine.MainMixerNode, format, out connectionError);
else
engine.Connect(source, engine.MainMixerNode, format); engine.Connect(source, engine.MainMixerNode, format);
if (connectionError is not null)
{
engine.DetachNode(source);
source.Dispose();
format.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio playback could not be connected: " + connectionError.LocalizedDescription);
}
engine.Prepare(); engine.Prepare();
if (!engine.StartAndReturnError(out var error)) if (!engine.StartAndReturnError(out var error))
{ {
@@ -159,17 +194,26 @@ internal sealed class MacAudioBackend : IAudioDeviceBackend
{ {
int bufferCount = Marshal.ReadInt32(outputData); int bufferCount = Marshal.ReadInt32(outputData);
int firstBuffer = IntPtr.Size == 8 ? 8 : 4; int firstBuffer = IntPtr.Size == 8 ? 8 : 4;
if (bufferCount != 1) return Fail("Core Audio returned a non-interleaved playback buffer."); const int bufferStride64 = 16;
int channels = Marshal.ReadInt32(outputData, firstBuffer); int stride = IntPtr.Size == 8 ? bufferStride64 : 12;
int byteCount = Marshal.ReadInt32(outputData, firstBuffer + 4); int frames = checked((int)frameCount);
nint data = Marshal.ReadIntPtr(outputData, firstBuffer + 8); int requested = checked(frames * 2);
int requested = checked((int)frameCount * 2); if (bufferCount != 2 || requested > renderScratch.Length) return Fail("Core Audio returned an invalid planar stereo playback buffer.");
if (channels != 2 || data == 0 || byteCount < requested * sizeof(short)) return Fail("Core Audio returned an invalid stereo playback buffer.");
var output = new Span<short>((void*)data, requested);
Span<short> discard = stackalloc short[1_920]; Span<short> discard = stackalloc short[1_920];
while (pcm.Count > 11_520) pcm.Read(discard[..Math.Min(discard.Length, pcm.Count - 11_520)]); while (pcm.Count > 11_520) pcm.Read(discard[..Math.Min(discard.Length, pcm.Count - 11_520)]);
int read = pcm.Read(output); Span<short> source = renderScratch.AsSpan(0, requested);
output[read..].Clear(); int read = pcm.Read(source);
source[read..].Clear();
for (int channel = 0; channel < 2; channel++)
{
int offset = firstBuffer + channel * stride;
int channels = Marshal.ReadInt32(outputData, offset);
int byteCount = Marshal.ReadInt32(outputData, offset + 4);
nint data = Marshal.ReadIntPtr(outputData, offset + 8);
if (channels != 1 || data == 0 || byteCount < frames * sizeof(float)) return Fail("Core Audio returned an invalid Float32 playback channel.");
var output = new Span<float>((void*)data, frames);
for (int frame = 0; frame < frames; frame++) output[frame] = source[frame * 2 + channel] * (1.0f / 32_768.0f);
}
if (isSilence != IntPtr.Zero) Marshal.WriteByte(isSilence, read == 0 ? (byte)1 : (byte)0); if (isSilence != IntPtr.Zero) Marshal.WriteByte(isSilence, read == 0 ? (byte)1 : (byte)0);
return 0; return 0;
} }
@@ -1,4 +1,5 @@
using AppKit; using AppKit;
using AVFoundation;
using CoreGraphics; using CoreGraphics;
using Foundation; using Foundation;
using VoiceCat.Audio; using VoiceCat.Audio;
@@ -12,7 +13,9 @@ internal sealed class MainWindowController : NSWindowController
private readonly VoiceCatClient client; private readonly VoiceCatClient client;
private readonly MacAudioBackend audioBackend = new(); private readonly MacAudioBackend audioBackend = new();
private readonly uint selfId; private readonly uint selfId;
private readonly NSPopUpButton channels = new(new CGRect(20, 515, 300, 28), false); private readonly NSPopUpButton channels = new(new CGRect(20, 515, 245, 28), false);
private readonly NSPopUpButton inputDevice = new(new CGRect(275, 515, 165, 28), false);
private readonly NSPopUpButton outputDevice = new(new CGRect(450, 515, 165, 28), false);
private readonly NSTextView users = new(new CGRect(0, 0, 190, 430)) { Editable = false, Selectable = true }; private readonly NSTextView users = new(new CGRect(0, 0, 190, 430)) { Editable = false, Selectable = true };
private readonly NSTextView chat = new(new CGRect(0, 0, 510, 390)) { Editable = false, Selectable = true }; private readonly NSTextView chat = new(new CGRect(0, 0, 510, 390)) { Editable = false, Selectable = true };
private readonly NSPopUpButton messageTarget = new(new CGRect(230, 55, 160, 28), false); private readonly NSPopUpButton messageTarget = new(new CGRect(230, 55, 160, 28), false);
@@ -27,6 +30,7 @@ internal sealed class MainWindowController : NSWindowController
private bool changingChannel; private bool changingChannel;
private IAudioCapture? microphone; private IAudioCapture? microphone;
private IAudioPlayback? playback; private IAudioPlayback? playback;
private long lastRemoteAudioTick;
internal MainWindowController(VoiceCatClient client, uint selfId, string nickname) : base(new NSWindow(new CGRect(0, 0, 760, 570), internal MainWindowController(VoiceCatClient client, uint selfId, string nickname) : base(new NSWindow(new CGRect(0, 0, 760, 570),
NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable | NSWindowStyle.Miniaturizable, NSBackingStore.Buffered, false)) NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable | NSWindowStyle.Miniaturizable, NSBackingStore.Buffered, false))
@@ -35,6 +39,9 @@ internal sealed class MainWindowController : NSWindowController
Window!.Title = $"VoiceCat — {nickname}"; Window.Center(); Window.MinSize = new CGSize(680, 480); Window!.Title = $"VoiceCat — {nickname}"; Window.Center(); Window.MinSize = new CGSize(680, 480);
var content = Window.ContentView!; var content = Window.ContentView!;
((INSAccessibility)channels).AccessibilityLabel = "Channel"; channels.Activated += ChangeChannel; content.AddSubview(channels); ((INSAccessibility)channels).AccessibilityLabel = "Channel"; channels.Activated += ChangeChannel; content.AddSubview(channels);
PopulateAudioDevices(inputDevice, true); PopulateAudioDevices(outputDevice, false);
((INSAccessibility)inputDevice).AccessibilityLabel = "Microphone input device"; inputDevice.Activated += ChangeAudioDevice; content.AddSubview(inputDevice);
((INSAccessibility)outputDevice).AccessibilityLabel = "Audio output device"; outputDevice.Activated += ChangeAudioDevice; content.AddSubview(outputDevice);
((INSAccessibility)voice).AccessibilityLabel = "Join or leave voice"; voice.Activated += ToggleVoice; content.AddSubview(voice); ((INSAccessibility)voice).AccessibilityLabel = "Join or leave voice"; voice.Activated += ToggleVoice; content.AddSubview(voice);
var userScroll = new NSScrollView(new CGRect(20, 90, 190, 410)) { HasVerticalScroller = true, DocumentView = users }; userScroll.AccessibilityLabel = "Users in channel"; content.AddSubview(userScroll); var userScroll = new NSScrollView(new CGRect(20, 90, 190, 410)) { HasVerticalScroller = true, DocumentView = users }; userScroll.AccessibilityLabel = "Users in channel"; content.AddSubview(userScroll);
var chatScroll = new NSScrollView(new CGRect(230, 90, 500, 410)) { HasVerticalScroller = true, DocumentView = chat }; chatScroll.AccessibilityLabel = "Channel messages"; content.AddSubview(chatScroll); var chatScroll = new NSScrollView(new CGRect(230, 90, 500, 410)) { HasVerticalScroller = true, DocumentView = chat }; chatScroll.AccessibilityLabel = "Channel messages"; content.AddSubview(chatScroll);
@@ -58,8 +65,14 @@ internal sealed class MainWindowController : NSWindowController
status.StringValue = "Disconnected: " + disconnected.Reason; voice.Enabled = send.Enabled = messageTarget.Enabled = false; status.StringValue = "Disconnected: " + disconnected.Reason; voice.Enabled = send.Enabled = messageTarget.Enabled = false;
} }
} }
if (client.Audio.Failure is { } failure) status.StringValue = "Audio stopped: " + failure.Message; if (client.Audio.Failure is { } failure) { status.StringValue = "Audio stopped: " + failure.Message; return; }
if (audioBackend.Failure is { } deviceFailure) status.StringValue = "Audio device stopped: " + deviceFailure.Message; if (audioBackend.Failure is { } deviceFailure) { status.StringValue = "Audio device stopped: " + deviceFailure.Message; return; }
if (joinedVoice)
{
(float level, bool talking) = client.Audio.GetLocalLevel(microphoneStreamId);
bool receiving = Environment.TickCount64 - Volatile.Read(ref lastRemoteAudioTick) < 1_000;
status.StringValue = $"Voice connected · Mic {(talking ? "sending" : "idle")} {level:P0} · Remote audio {(receiving ? "active" : "idle")}";
}
} }
private void RefreshState() private void RefreshState()
{ {
@@ -148,17 +161,46 @@ internal sealed class MainWindowController : NSWindowController
finally { voice.Enabled = true; } finally { voice.Enabled = true; }
} }
private void PopulateAudioDevices(NSPopUpButton menu, bool input)
{
IReadOnlyList<AudioDeviceInfo> devices = audioBackend.Enumerate(input);
foreach (AudioDeviceInfo device in devices)
{
menu.AddItem(device.Name + (device.IsDefault ? " (default)" : ""));
menu.LastItem!.RepresentedObject = new NSString(device.Id);
}
int selected = devices.ToList().FindIndex(device => device.IsDefault);
if (selected >= 0) menu.SelectItem(selected);
menu.Enabled = devices.Count > 0;
}
private async void ChangeAudioDevice(object? sender, EventArgs args)
{
if (!joinedVoice) return;
try { await LeaveVoice(); await JoinVoice(); }
catch (Exception exception) { status.StringValue = exception.Message; }
}
private async Task JoinVoice() private async Task JoinVoice()
{ {
AVAuthorizationStatus permission = AVCaptureDevice.GetAuthorizationStatus(AVAuthorizationMediaType.Audio);
if (permission == AVAuthorizationStatus.NotDetermined)
{
bool granted = await AVCaptureDevice.RequestAccessForMediaTypeAsync(AVAuthorizationMediaType.Audio);
permission = granted ? AVAuthorizationStatus.Authorized : AVAuthorizationStatus.Denied;
}
if (permission != AVAuthorizationStatus.Authorized)
throw new UnauthorizedAccessException("Microphone access is disabled. Enable VoiceCat in System Settings → Privacy & Security → Microphone, then join voice again.");
var result = await client.SubscribeVoiceAsync(); var result = await client.SubscribeVoiceAsync();
if (!result.Ok) throw new InvalidOperationException(result.Error); if (!result.Ok) throw new InvalidOperationException(result.Error);
try try
{ {
playback = audioBackend.OpenPlayback(); playback = audioBackend.OpenPlayback(SelectedDevice(outputDevice));
client.Audio.MixedPcm += PlayMixedPcm; client.Audio.MixedPcm += PlayMixedPcm;
StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamMic, "Microphone"); StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamMic, "Microphone");
microphoneStreamId = stream.StreamId; microphoneStreamId = stream.StreamId;
microphone = audioBackend.OpenCapture(null, false, FeedMicrophone); microphone = audioBackend.OpenCapture(SelectedDevice(inputDevice), false, FeedMicrophone);
joinedVoice = true; voice.Title = "Leave voice"; status.StringValue = "Voice connected"; joinedVoice = true; voice.Title = "Leave voice"; status.StringValue = "Voice connected";
} }
catch catch
@@ -186,11 +228,21 @@ internal sealed class MainWindowController : NSWindowController
if (streamId != 0) client.Audio.FeedPcm(streamId, pcm, channels); if (streamId != 0) client.Audio.FeedPcm(streamId, pcm, channels);
} }
private void PlayMixedPcm(ReadOnlySpan<short> pcm) => Volatile.Read(ref playback)?.Write(pcm); private static string? SelectedDevice(NSPopUpButton menu) => menu.SelectedItem?.RepresentedObject?.ToString();
private void PlayMixedPcm(ReadOnlySpan<short> pcm)
{
bool signal = false;
foreach (short sample in pcm)
if (sample is > 64 or < -64) { signal = true; break; }
if (signal) Volatile.Write(ref lastRemoteAudioTick, Environment.TickCount64);
Volatile.Read(ref playback)?.Write(pcm);
}
private void StopAudioDevices() private void StopAudioDevices()
{ {
microphoneStreamId = 0; microphoneStreamId = 0;
Volatile.Write(ref lastRemoteAudioTick, 0);
Interlocked.Exchange(ref microphone, null)?.Dispose(); Interlocked.Exchange(ref microphone, null)?.Dispose();
client.Audio.MixedPcm -= PlayMixedPcm; client.Audio.MixedPcm -= PlayMixedPcm;
Interlocked.Exchange(ref playback, null)?.Dispose(); Interlocked.Exchange(ref playback, null)?.Dispose();
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<TargetFramework>net10.0-macos</TargetFramework> <TargetFramework>net10.0-macos27.0</TargetFramework>
<RuntimeIdentifier>osx-arm64</RuntimeIdentifier> <RuntimeIdentifier>osx-arm64</RuntimeIdentifier>
<SupportedOSPlatformVersion>14.0</SupportedOSPlatformVersion> <SupportedOSPlatformVersion>14.0</SupportedOSPlatformVersion>
<Nullable>enable</Nullable> <Nullable>enable</Nullable>
@@ -9,11 +9,37 @@
<AllowUnsafeBlocks>true</AllowUnsafeBlocks> <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<ApplicationTitle>VoiceCat</ApplicationTitle> <ApplicationTitle>VoiceCat</ApplicationTitle>
<ApplicationId>net.iamtalon.voicecat</ApplicationId> <ApplicationId>net.iamtalon.voicecat</ApplicationId>
<UseHardenedRuntime>true</UseHardenedRuntime> <UseHardenedRuntime Condition="'$(Configuration)' == 'Release'">true</UseHardenedRuntime>
<UseHardenedRuntime Condition="'$(Configuration)' != 'Release'">false</UseHardenedRuntime>
<ApplicationManifest>Info.plist</ApplicationManifest> <ApplicationManifest>Info.plist</ApplicationManifest>
<CodesignEntitlements>VoiceCat.Mac.entitlements</CodesignEntitlements> <CodesignEntitlements>VoiceCat.Mac.entitlements</CodesignEntitlements>
<NoWarn>$(NoWarn);XCODE_27_0_PREVIEW</NoWarn>
<VoiceCatNativeMediaPath>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../../../dotnet/artifacts/native/runtimes/osx-arm64/native/libvoicecat_media.dylib'))</VoiceCatNativeMediaPath>
<VoiceCatNativeLicenseDirectory>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../../../dotnet/artifacts/native/licenses'))</VoiceCatNativeLicenseDirectory>
<_ComputePublishLocationDependsOn>VoiceCatPrepareNativeAssets;$(_ComputePublishLocationDependsOn)</_ComputePublishLocationDependsOn>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="../../../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" /> <ProjectReference Include="../../../../dotnet/src/VoiceCat.Core/VoiceCat.Core.csproj" />
</ItemGroup> </ItemGroup>
<!-- Every managed media project stages the same dylib for ordinary .NET consumers.
The macOS bundler preserves those transitive items separately and otherwise runs
install_name_tool against the same temporary file concurrently. Collapse them to
the single native reference the application actually ships. -->
<Target Name="VoiceCatPrepareNativeAssets">
<ItemGroup>
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'libvoicecat_media.dylib'" />
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'NOTICE.txt' or '%(Filename)%(Extension)' == 'Opus.txt' or '%(Filename)%(Extension)' == 'RNNoise.txt'" />
<ResolvedFileToPublish Include="$(VoiceCatNativeMediaPath)" Condition="Exists('$(VoiceCatNativeMediaPath)')">
<RelativePath>libvoicecat_media.dylib</RelativePath>
<PublishFolderType>DynamicLibrary</PublishFolderType>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</ResolvedFileToPublish>
<ResolvedFileToPublish Include="$(VoiceCatNativeLicenseDirectory)/*.txt">
<RelativePath>licenses/%(Filename)%(Extension)</RelativePath>
<PublishFolderType>Assembly</PublishFolderType>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</ResolvedFileToPublish>
</ItemGroup>
</Target>
</Project> </Project>
+5 -2
View File
@@ -453,7 +453,9 @@ VC_API vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id,
* channels : 1 or 2, matching the sender's stream configuration. * channels : 1 or 2, matching the sender's stream configuration.
* sample_rate : always 48000 in the current implementation. * sample_rate : always 48000 in the current implementation.
* *
* Pass cb = NULL to disable (default: disabled; hardware playback only). * Pass cb = NULL to disable (default: disabled; hardware playback only). Disabling is a
* quiescence barrier: when vc_set_pcm_sink(c, NULL, NULL) returns, no callback using the old
* user pointer is still running, so the caller may safely release that state.
* The callback MUST NOT block, lock, or allocate copy what you need and return. * The callback MUST NOT block, lock, or allocate copy what you need and return.
* PCM is still delivered to the hardware playback device regardless (dual output). */ * PCM is still delivered to the hardware playback device regardless (dual output). */
typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id, typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
@@ -483,7 +485,8 @@ VC_API vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
* steadily even with no remote streams (a silent block), so the renderer has a continuous * steadily even with no remote streams (a silent block), so the renderer has a continuous
* clock. The callback MUST NOT block, lock, or allocate copy into a lock-free ring and * clock. The callback MUST NOT block, lock, or allocate copy into a lock-free ring and
* return. Independent of vc_set_pcm_sink (the per-stream tap), which still works. Pass cb=NULL * return. Independent of vc_set_pcm_sink (the per-stream tap), which still works. Pass cb=NULL
* to disable (default: disabled). */ * to disable (default: disabled). Disabling is a quiescence barrier with the same callback-state
* lifetime guarantee as vc_set_pcm_sink. */
typedef void (*vc_mixed_output_cb)(void* user, const int16_t* pcm, typedef void (*vc_mixed_output_cb)(void* user, const int16_t* pcm,
size_t samples_per_channel, uint32_t channels, size_t samples_per_channel, uint32_t channels,
uint32_t sample_rate); uint32_t sample_rate);
+21 -11
View File
@@ -566,13 +566,19 @@ void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p,
} }
void AudioEngine::set_pcm_sink(PcmSink cb, void* user) { void AudioEngine::set_pcm_sink(PcmSink cb, void* user) {
pcm_sink_user_.store(user, std::memory_order_relaxed); std::lock_guard setter_lk(pcm_sink_set_mu_);
pcm_sink_.store(cb, std::memory_order_release); pcm_sink_.store(nullptr);
while (pcm_sink_active_.load() != 0) std::this_thread::yield();
pcm_sink_user_.store(user);
pcm_sink_.store(cb);
} }
void AudioEngine::set_mixed_output_sink(MixedSink cb, void* user) { void AudioEngine::set_mixed_output_sink(MixedSink cb, void* user) {
mixed_sink_user_.store(user, std::memory_order_relaxed); std::lock_guard setter_lk(mixed_sink_set_mu_);
mixed_sink_.store(cb, std::memory_order_release); mixed_sink_.store(nullptr);
while (mixed_sink_active_.load() != 0) std::this_thread::yield();
mixed_sink_user_.store(user);
mixed_sink_.store(cb);
} }
void AudioEngine::capture_data_cb(ma_device* dev, void* /*out*/, void AudioEngine::capture_data_cb(ma_device* dev, void* /*out*/,
@@ -734,17 +740,19 @@ void AudioEngine::on_playback(int16_t* out, ma_uint32 frames) {
} }
// PCM sink: deliver decoded per-stream audio to external consumer (bots, // PCM sink: deliver decoded per-stream audio to external consumer (bots,
// transcription, recording) before it enters the hardware mix. Atomic relaxed- // transcription, recording) before it enters the hardware mix. The reader count
// load is safe on the RT thread — the fn-ptr and user-ptr are independent // makes disabling the callback a quiescence barrier while keeping this RT path
// pointer-sized values written together by set_pcm_sink (release store). // lock-free and non-blocking.
if (auto sink = pcm_sink_.load(std::memory_order_relaxed)) { pcm_sink_active_.fetch_add(1);
sink(pcm_sink_user_.load(std::memory_order_relaxed), if (auto sink = pcm_sink_.load()) {
sink(pcm_sink_user_.load(),
stream.user_id, stream.stream_id, stream.user_id, stream.stream_id,
stream.decode_scratch.data(), stream.decode_scratch.data(),
static_cast<size_t>(n), static_cast<size_t>(n),
static_cast<uint32_t>(dec_channels), static_cast<uint32_t>(dec_channels),
params_.sample_rate); params_.sample_rate);
} }
pcm_sink_active_.fetch_sub(1);
stream.push_ring(stream.decode_scratch.data(), static_cast<size_t>(n)); stream.push_ring(stream.decode_scratch.data(), static_cast<size_t>(n));
stream.playout_ts += static_cast<uint32_t>(n); stream.playout_ts += static_cast<uint32_t>(n);
@@ -797,10 +805,12 @@ void AudioEngine::run_mixer_timer() {
auto next = clock::now(); auto next = clock::now();
while (!mixer_timer_stop_.load(std::memory_order_acquire)) { while (!mixer_timer_stop_.load(std::memory_order_acquire)) {
on_playback(mixer_scratch_.data(), spc); on_playback(mixer_scratch_.data(), spc);
if (auto sink = mixed_sink_.load(std::memory_order_relaxed)) { mixed_sink_active_.fetch_add(1);
sink(mixed_sink_user_.load(std::memory_order_relaxed), mixer_scratch_.data(), spc, ch, if (auto sink = mixed_sink_.load()) {
sink(mixed_sink_user_.load(), mixer_scratch_.data(), spc, ch,
params_.sample_rate); params_.sample_rate);
} }
mixed_sink_active_.fetch_sub(1);
next += period; next += period;
// If we fell badly behind (e.g. the thread was descheduled), reset the deadline rather // If we fell badly behind (e.g. the thread was descheduled), reset the deadline rather
// than spin to catch up — the VPIO renderer rides its own clock + jitter ring. // than spin to catch up — the VPIO renderer rides its own clock + jitter ring.
+8 -2
View File
@@ -514,16 +514,22 @@ class AudioEngine {
int frame_samples_ = 960; // 20 ms @48 kHz int frame_samples_ = 960; // 20 ms @48 kHz
// External PCM tap: atomic fn-ptr + user-ptr pair. Written by set_pcm_sink (any thread); // External PCM tap. The in-flight counter lets set_pcm_sink(nullptr, nullptr) wait for an
// read by on_playback (RT thread) via relaxed load — safe for pointer-sized atomics. // already-entered callback without making the RT thread lock or block. Sequentially
// consistent operations are intentional here: once the setter observes zero readers, a
// later reader must observe the disabled callback before it can dereference the old user.
std::atomic<PcmSink> pcm_sink_{nullptr}; std::atomic<PcmSink> pcm_sink_{nullptr};
std::atomic<void*> pcm_sink_user_{nullptr}; std::atomic<void*> pcm_sink_user_{nullptr};
std::atomic<uint32_t> pcm_sink_active_{0};
std::mutex pcm_sink_set_mu_;
// External mixed-output sink + mode flag (iOS VPIO). mixed_sink_ is written by // External mixed-output sink + mode flag (iOS VPIO). mixed_sink_ is written by
// set_mixed_output_sink (any thread); read by run_mixer_timer via relaxed load. // set_mixed_output_sink (any thread); read by run_mixer_timer via relaxed load.
// external_playback_ is read in start() to gate hardware-playback-device creation. // external_playback_ is read in start() to gate hardware-playback-device creation.
std::atomic<MixedSink> mixed_sink_{nullptr}; std::atomic<MixedSink> mixed_sink_{nullptr};
std::atomic<void*> mixed_sink_user_{nullptr}; std::atomic<void*> mixed_sink_user_{nullptr};
std::atomic<uint32_t> mixed_sink_active_{0};
std::mutex mixed_sink_set_mu_;
bool external_playback_ = false; bool external_playback_ = false;
::OpusDREDDecoder* dred_dec_ = nullptr; // shared DRED decoder; null if unsupported ::OpusDREDDecoder* dred_dec_ = nullptr; // shared DRED decoder; null if unsupported
+1 -1
View File
@@ -167,7 +167,7 @@ Two API functions let callers bypass miniaudio entirely for a stream:
| Function | Direction | Contract | | Function | Direction | Contract |
|----------|-----------|----------| |----------|-----------|----------|
| `vc_stream_feed_pcm(c, stream_id, pcm, samples_per_channel, channels)` | **Send** — caller → network | Caller supplies interleaved int16 at the stream's sample rate (`channels` = 1 mono, 2 stereo). The core frames, Opus-encodes, AEAD-seals, and sends over UDP — identical wire path to hardware capture. The stream must already be started with `vc_stream_start`. Thread-safe; may be called from any thread (audio callback, ReplayKit delegate, SCStream callback). | | `vc_stream_feed_pcm(c, stream_id, pcm, samples_per_channel, channels)` | **Send** — caller → network | Caller supplies interleaved int16 at the stream's sample rate (`channels` = 1 mono, 2 stereo). The core frames, Opus-encodes, AEAD-seals, and sends over UDP — identical wire path to hardware capture. The stream must already be started with `vc_stream_start`. Thread-safe; may be called from any thread (audio callback, ReplayKit delegate, SCStream callback). |
| `vc_set_pcm_sink(c, cb, user)` | **Receive** — network → caller | `cb` is called on the audio (playback) thread once per decoded Opus frame per remote stream, with `(user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)`. PCM is delivered to the sink **and** the hardware device — dual output; the hardware mix is unaffected. Pass `cb=NULL` to disable (default). **Must not block** — copy what you need and return. | | `vc_set_pcm_sink(c, cb, user)` | **Receive** — network → caller | `cb` is called on the audio (playback) thread once per decoded Opus frame per remote stream, with `(user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)`. PCM is delivered to the sink **and** the hardware device — dual output; the hardware mix is unaffected. Pass `cb=NULL` to disable (default); once that call returns, no callback using the old `user` pointer remains in flight. **Must not block** — copy what you need and return. |
`vc_test_inject_capture` (the old TEST-ONLY mono-only predecessor) is a deprecated alias `vc_test_inject_capture` (the old TEST-ONLY mono-only predecessor) is a deprecated alias
for `vc_stream_feed_pcm(..., channels=1)` — kept for source compatibility. for `vc_stream_feed_pcm(..., channels=1)` — kept for source compatibility.
+17 -10
View File
@@ -791,16 +791,17 @@ criterion from `roadmap.md`, re-proven against the new server.
6. **Add the allocation-regression test.** 6. **Add the allocation-regression test.**
**Exit criterion:** `test_jitter_depth`'s bounded-depth invariant holds; zero allocations per **Exit criterion:** `test_jitter_depth`'s bounded-depth invariant holds; zero allocations per
mix cycle; a manual listen test on Windows and macOS with no audible glitching over 10 mix cycle; real multi-human calls on Windows and macOS have no reproducible audio-quality
minutes. defects once each client is feature-complete.
**Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every **Checkpoint (2026-09-16):** `VoiceCat.Audio` now owns local Opus streams, reframing at every
protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream protocol frame size, VAD/PTT/DTX, DRED → FEC → PLC receive recovery, bounded jitter, per-stream
controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero controls, RNNoise and stereo mixing. Its normal encode/decode/NR/mix cycle allocates zero
managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation managed bytes. Capture inputs use bounded non-waiting PCM rings. The Windows implementation
uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device uses direct C# WASAPI capture, loopback and playback instead of the proposed miniaudio device
shim; the codec/DSP shim remains the only native component. A real device smoke passed, but shim; the codec/DSP shim remains the only native component. A real device smoke passed;
the required ten-minute Windows/macOS listen test is still a manual release gate. final listen validation is performed with real multi-human calls after feature completion,
not a synthetic ten-minute sine-wave gate.
--- ---
@@ -850,7 +851,7 @@ Per §8.2. AppKit port, ScreenCaptureKit per-app audio selection, VoiceOver pari
**Exit criterion:** feature parity with `VoiceCatMac`, VoiceOver smoke-tested, notarized **Exit criterion:** feature parity with `VoiceCatMac`, VoiceOver smoke-tested, notarized
build produced. build produced.
**Checkpoint (2026-09-16):** `clients/apple/dotnet/VoiceCat.Mac` is a separate .NET 10 **Checkpoint (2026-09-18):** `clients/apple/dotnet/VoiceCat.Mac` is a separate .NET 10
AppKit application that consumes `VoiceCat.Core` directly. It implements guest and account AppKit application that consumes `VoiceCat.Core` directly. It implements guest and account
connection, validated saved server profiles whose JSON never contains passwords, connection, validated saved server profiles whose JSON never contains passwords,
interactive TOFU approval, protected-channel password prompts, channel browsing, roster, interactive TOFU approval, protected-channel password prompts, channel browsing, roster,
@@ -862,11 +863,17 @@ Core Audio capture is resampled
and converted through `AVAudioConverter` before entering the managed audio engine. Stereo and converted through `AVAudioConverter` before entering the managed audio engine. Stereo
playback uses `AVAudioSourceNode` and the shared bounded PCM ring; its render callback does playback uses `AVAudioSourceNode` and the shared bounded PCM ring; its render callback does
not allocate, lock or block. Voice stream lifetime follows subscription, disconnect and not allocate, lock or block. Voice stream lifetime follows subscription, disconnect and
channel changes. Current Apple API calls compile warning-free against Microsoft's channel changes. The app now targets Microsoft's Xcode 27 preview bindings from workload
26.4.10259 macOS reference assembly, and a macOS CI job builds the native codec/DSP shim and set 10.0.401. On Apple Silicon it selects a native Homebrew `protoc`, deduplicates the
Apple solution. A real macOS device run and listen test remain required. The existing Swift transitive media dylib before AppKit bundling, uses the macOS 27 error-returning Core Audio
app remains the release client until selectable devices and the moderation/settings overloads, and keeps hardened runtime Release-only so local ad-hoc Debug builds can load
surface, ScreenCaptureKit, VoiceOver their runtime libraries. It enumerates and selects Core Audio input/output devices, requests
microphone permission explicitly, and renders through Core Audio's native planar Float32
layout. The arm64 app builds, signs, launches and displays on macOS 27. Live testing proved
physical-microphone transmission and clean peer playback; Lobby's expected DTX comfort noise
was distinguished from corruption by repeating the tone in the DTX-disabled Music Room. A
macOS CI job builds the native codec/DSP shim and Apple solution. The existing Swift app
remains the release client until the moderation/settings surface, ScreenCaptureKit, VoiceOver
validation, signing and notarization are complete. validation, signing and notarization are complete.
--- ---
+6
View File
@@ -174,3 +174,9 @@ Native `vccli --test-tone-ms 4000` runs finite external capture/playback, feeds
and fails without decoded remote audio. Tests start two CLI processes in mono/stereo and fails without decoded remote audio. Tests start two CLI processes in mono/stereo
channels and also verify channel text. Normal `--voice` now explicitly subscribes before channels and also verify channel text. Normal `--voice` now explicitly subscribes before
announcing its microphone stream. No C ABI or wire changes were needed. announcing its microphone stream. No C ABI or wire changes were needed.
The managed CLI also accepts `--test-tone-seconds N` (maximum 3600) for a finite, accurately
paced peer used during device checks. Use a channel with Opus DTX disabled, such as the seeded
Music Room: a steady sine is intentionally classified as non-speech by DTX and becomes comfort
noise after its hangover period in Lobby. The finite peer prints received voice energy and exits
normally when its requested duration completes.
+1 -1
View File
@@ -1,3 +1,3 @@
{ {
"sdk": { "version": "10.0.203", "rollForward": "latestFeature" } "sdk": { "version": "10.0.401", "rollForward": "latestFeature" }
} }
+35 -11
View File
@@ -1,3 +1,4 @@
using System.Diagnostics;
using System.Text.Json; using System.Text.Json;
using VoiceCat.Audio; using VoiceCat.Audio;
using VoiceCat.Core; using VoiceCat.Core;
@@ -43,8 +44,14 @@ public static class CliCommand
if (options.Delay > TimeSpan.Zero) await Task.Delay(options.Delay, stopped.Token); if (options.Delay > TimeSpan.Zero) await Task.Delay(options.Delay, stopped.Token);
if (options.SendText is not null) if (options.SendText is not null)
client.Send(new() { TextMessage = new() { Scope = TextScope.TextChannel, TargetId = options.Channel, Body = options.SendText, ClientMsgId = Guid.NewGuid().ToString("N") } }); client.Send(new() { TextMessage = new() { Scope = TextScope.TextChannel, TargetId = options.Channel, Body = options.SendText, ClientMsgId = Guid.NewGuid().ToString("N") } });
Task? tone = options.Voice ? SendToneAsync(client, stopped.Token) : null; Task? tone = options.Voice ? SendToneAsync(client, options.ToneDuration, stopped.Token) : null;
if (options.OneShot) if (options.ToneDuration > TimeSpan.Zero)
{
await tone!;
Print("complete", options.Nickname, new { voiceEnergy = Interlocked.Read(ref energy) });
stopped.Cancel();
}
else if (options.OneShot)
{ {
using var deadline = CancellationTokenSource.CreateLinkedTokenSource(stopped.Token); using var deadline = CancellationTokenSource.CreateLinkedTokenSource(stopped.Token);
deadline.CancelAfter(options.Timeout); deadline.CancelAfter(options.Timeout);
@@ -75,16 +82,31 @@ public static class CliCommand
if (message.UserEvent is { } user) Print("user", user.Kind.ToString(), new { userId = user.User?.Id ?? user.LeftId }); if (message.UserEvent is { } user) Print("user", user.Kind.ToString(), new { userId = user.User?.Id ?? user.LeftId });
} }
} }
private static async Task SendToneAsync(VoiceCatClient client, CancellationToken token) private static async Task SendToneAsync(VoiceCatClient client, TimeSpan requestedDuration, CancellationToken token)
{ {
TimeSpan duration = requestedDuration > TimeSpan.Zero ? requestedDuration : TimeSpan.FromSeconds(3);
int frames = checked((int)Math.Ceiling(duration.TotalSeconds * 50));
const int leadFrames = 8;
long started = Stopwatch.GetTimestamp();
short[] pcm = new short[960]; short[] pcm = new short[960];
for (int frame = 0; frame < 150 && !token.IsCancellationRequested; frame++) for (int frame = 0; frame < frames && !token.IsCancellationRequested; frame++)
{ {
for (int i = 0; i < pcm.Length; i++) pcm[i] = (short)(Math.Sin((frame * 960 + i) * Math.PI * 880 / 48000) * 8000); if (frame >= leadFrames)
foreach (StreamInfo stream in client.LocalStreams) client.Audio.FeedPcm(stream.StreamId, pcm, 1); {
await Task.Delay(20, token); long target = started + (frame - leadFrames) * Stopwatch.Frequency / 50;
while (true)
{
double remainingMilliseconds = (target - Stopwatch.GetTimestamp()) * 1000.0 / Stopwatch.Frequency;
if (remainingMilliseconds <= 1) break;
await Task.Delay(TimeSpan.FromMilliseconds(remainingMilliseconds - 0.5), token);
} }
} }
for (int i = 0; i < pcm.Length; i++) pcm[i] = (short)(Math.Sin((frame * 960 + i) * Math.PI * 880 / 48000) * 8000);
foreach (StreamInfo stream in client.LocalStreams)
while (!client.Audio.FeedPcm(stream.StreamId, pcm, 1)) await Task.Delay(1, token);
}
await Task.Delay(TimeSpan.FromMilliseconds(leadFrames * 20), token);
}
private static async Task InteractiveAsync(VoiceCatClient client, CancellationToken token) private static async Task InteractiveAsync(VoiceCatClient client, CancellationToken token)
{ {
while (!token.IsCancellationRequested && await Console.In.ReadLineAsync(token) is { } line) while (!token.IsCancellationRequested && await Console.In.ReadLineAsync(token) is { } line)
@@ -96,20 +118,22 @@ public static class CliCommand
} }
private static void Print(string type, string value, object? extra = null) => Console.WriteLine(JsonSerializer.Serialize(new { type, value, extra })); private static void Print(string type, string value, object? extra = null) => Console.WriteLine(JsonSerializer.Serialize(new { type, value, extra }));
private sealed record Options(string Host, ushort Port, string Nickname, string Pins, uint Channel, bool TrustFirst, bool Voice, bool ExpectVoice, string? SendText, string? ExpectText, TimeSpan Delay, TimeSpan Linger, TimeSpan Timeout) private sealed record Options(string Host, ushort Port, string Nickname, string Pins, uint Channel, bool TrustFirst, bool Voice, bool ExpectVoice, string? SendText, string? ExpectText, TimeSpan Delay, TimeSpan Linger, TimeSpan Timeout, TimeSpan ToneDuration)
{ {
internal bool OneShot => SendText is not null || ExpectText is not null || ExpectVoice; internal bool OneShot => SendText is not null || ExpectText is not null || ExpectVoice;
internal static Options Parse(string[] args) internal static Options Parse(string[] args)
{ {
string Value(string name, string fallback) { int i = Array.IndexOf(args, name); return i >= 0 && i + 1 < args.Length ? args[i + 1] : fallback; } string Value(string name, string fallback) { int i = Array.IndexOf(args, name); return i >= 0 && i + 1 < args.Length ? args[i + 1] : fallback; }
bool Has(string name) => args.Contains(name, StringComparer.OrdinalIgnoreCase); bool Has(string name) => args.Contains(name, StringComparer.OrdinalIgnoreCase);
if (Has("--help")) { Console.WriteLine("VoiceCat.Cli --host HOST --port PORT --nickname NAME [--trust-first] [--channel ID] [--voice] [--send-text TEXT] [--expect-text TEXT] [--expect-voice] [--start-delay-ms N]"); Environment.Exit(0); } if (Has("--help")) { Console.WriteLine("VoiceCat.Cli --host HOST --port PORT --nickname NAME [--trust-first] [--channel ID] [--voice] [--test-tone-seconds N] [--send-text TEXT] [--expect-text TEXT] [--expect-voice] [--start-delay-ms N]"); Environment.Exit(0); }
TimeSpan toneDuration = TimeSpan.FromSeconds(int.Parse(Value("--test-tone-seconds", "0")));
if (toneDuration < TimeSpan.Zero || toneDuration > TimeSpan.FromHours(1)) throw new ArgumentOutOfRangeException("--test-tone-seconds", "Tone duration must be between 0 and 3600 seconds.");
return new(Value("--host", "127.0.0.1"), ushort.Parse(Value("--port", "8384")), Value("--nickname", Environment.UserName), return new(Value("--host", "127.0.0.1"), ushort.Parse(Value("--port", "8384")), Value("--nickname", Environment.UserName),
Value("--pins", Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "VoiceCat", "cli-tofu.txt")), uint.Parse(Value("--channel", "1")), Has("--trust-first"), Has("--voice") || Has("--expect-voice"), Has("--expect-voice"), Value("--pins", Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "VoiceCat", "cli-tofu.txt")), uint.Parse(Value("--channel", "1")), Has("--trust-first"), Has("--voice") || Has("--expect-voice") || toneDuration > TimeSpan.Zero, Has("--expect-voice"),
Array.IndexOf(args, "--send-text") is int send and >= 0 && send + 1 < args.Length ? args[send + 1] : null, Array.IndexOf(args, "--send-text") is int send and >= 0 && send + 1 < args.Length ? args[send + 1] : null,
Array.IndexOf(args, "--expect-text") is int expect and >= 0 && expect + 1 < args.Length ? args[expect + 1] : null, Array.IndexOf(args, "--expect-text") is int expect and >= 0 && expect + 1 < args.Length ? args[expect + 1] : null,
TimeSpan.FromMilliseconds(int.Parse(Value("--start-delay-ms", "0"))), TimeSpan.FromMilliseconds(int.Parse(Value("--linger-ms", "1000"))), TimeSpan.FromMilliseconds(int.Parse(Value("--start-delay-ms", "0"))), TimeSpan.FromMilliseconds(int.Parse(Value("--linger-ms", "1000"))),
TimeSpan.FromSeconds(int.Parse(Value("--timeout-seconds", "15")))); TimeSpan.FromSeconds(int.Parse(Value("--timeout-seconds", "15"))), toneDuration);
} }
} }
} }
@@ -5,6 +5,7 @@ internal static class PrivateFiles
public static void Write(string path, ReadOnlySpan<byte> data) public static void Write(string path, ReadOnlySpan<byte> data)
{ {
string destination = Path.GetFullPath(path); string destination = Path.GetFullPath(path);
Directory.CreateDirectory(Path.GetDirectoryName(destination)!);
string temporary = destination + "." + Guid.NewGuid().ToString("N") + ".tmp"; string temporary = destination + "." + Guid.NewGuid().ToString("N") + ".tmp";
try try
{ {
@@ -1,4 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup Condition="'$(Protobuf_ProtocFullPath)' == '' and Exists('/opt/homebrew/bin/protoc')">
<!-- Grpc.Tools currently ships an x64-only macOS protoc. Prefer Homebrew's native
compiler on Apple Silicon hosts that do not have Rosetta installed. -->
<Protobuf_ProtocFullPath>/opt/homebrew/bin/protoc</Protobuf_ProtocFullPath>
</PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Google.Protobuf" Version="3.36.1" /> <PackageReference Include="Google.Protobuf" Version="3.36.1" />
<PackageReference Include="Grpc.Tools" Version="2.83.0" PrivateAssets="all" /> <PackageReference Include="Grpc.Tools" Version="2.83.0" PrivateAssets="all" />
@@ -4,6 +4,22 @@ namespace VoiceCat.Tests;
public class TofuTlsTests public class TofuTlsTests
{ {
[Fact]
public void PinCreatesMissingParentDirectory()
{
string root = Path.Combine(Path.GetTempPath(), "voicecat-tofu-parent-" + Guid.NewGuid());
string path = Path.Combine(root, "nested", "pins.txt");
try
{
var store = new TofuStore(path);
store.Pin("localhost", 8384, new string('a', 64));
Assert.True(File.Exists(path));
Assert.Equal(TofuStatus.Matched, new TofuStore(path).Check("localhost", 8384, new string('A', 64)));
}
finally { if (Directory.Exists(root)) Directory.Delete(root, true); }
}
[Fact] [Fact]
public void RealHandshakesRequireAcceptanceAndRejectChangedCertificatesAfterRestart() public void RealHandshakesRequireAcceptanceAndRejectChangedCertificatesAfterRestart()
{ {
+2
View File
@@ -219,6 +219,7 @@ static void test_feed_pcm_round_trip(uint16_t port) {
CHECK(energy > 0); CHECK(energy > 0);
vc_stream_stop(clientA, a_sid); vc_stream_stop(clientA, a_sid);
CHECK(vc_set_pcm_sink(clientB, nullptr, nullptr) == VC_OK);
} }
cleanup_rt: cleanup_rt:
@@ -290,6 +291,7 @@ static void test_feed_pcm_stereo(uint16_t port) {
CHECK(R < L || L == 0); // L >= R (loud-L / quiet-R) CHECK(R < L || L == 0); // L >= R (loud-L / quiet-R)
vc_stream_stop(clientA, a_sid); vc_stream_stop(clientA, a_sid);
CHECK(vc_set_pcm_sink(clientB, nullptr, nullptr) == VC_OK);
} }
cleanup_st: cleanup_st:
+1
View File
@@ -225,6 +225,7 @@ static void run_case(uint16_t port, vc_client* admin, EventStore& evAdmin,
CHECK(sink.last_sample_rate == 48000); CHECK(sink.last_sample_rate == 48000);
vc_stream_stop(clientA, a_sid); vc_stream_stop(clientA, a_sid);
CHECK(vc_set_pcm_sink(clientB, nullptr, nullptr) == VC_OK);
} }
cleanup: cleanup: