diff --git a/CLAUDE.md b/CLAUDE.md index 3ad934d..9ae9318 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,6 +12,9 @@ and what's next* read [`PROGRESS.md`](PROGRESS.md); for *design* read [`docs/`]( > External PCM feed/tap API (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) shipped. > **Screen-audio sharing shipped on macOS (ScreenCaptureKit) and iOS (ReplayKit Broadcast > Upload Extension → host App Group ring → `vc_stream_feed_pcm`).** +> The managed C# AppKit replacement now implements the Swift macOS feature surface and has a +> validated ad-hoc Release bundle. Swift remains the release oracle until VoiceOver, live-call, +> and credentialed notarization gates pass; see `clients/apple/dotnet/README.md`. > **Noise suppression shipped (RNNoise, vendored at `third_party/rnnoise/`)** — both send-side > mic NR (`vc_set_input_noise_reduction`) and per-listener receive NR; client on/off toggles ship on all three clients (receive NR now denoises stereo mic streams too — fixed 2026-06-23). @@ -37,11 +40,10 @@ dotnet test dotnet/VoiceCat.slnx -c Release --no-build ``` See `dotnet/README.md` for C# conventions and required native voice/CLI conformance, -and `docs/api-dotnet.md` for managed interfaces. Phase 4 remains in progress; -media-aware reaping and server administration are implemented. Server deployment hardening -and the managed audio/client core plus Windows cutover are implemented. The Windows publish -contains only the permitted media shim. Accessibility/manual endurance validation and broad -server deployment work remain; Apple GUI rewrites follow Windows. +and `docs/api-dotnet.md` for managed interfaces. The managed server, CLI, audio/client core, +Windows cutover, and macOS functional surface are implemented. The Windows and macOS apps +contain only the permitted media shim. Accessibility/manual endurance validation, macOS +notarization, the iOS rewrite, and broad server deployment work remain. The default development preset is **`dev`** — it builds everything (server + tools + tests) with real vcpkg deps. The `skeleton` preset (no deps, stubs only) is a fast smoke check; see diff --git a/PROGRESS.md b/PROGRESS.md index 8c864ed..2dc06ba 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -10,6 +10,31 @@ up instantly. Newest status at the top. ## ▶ Where we left off / next action +- **In progress (2026-09-19): managed macOS functional-parity checkpoint.** Extended the + .NET AppKit client across the remaining Swift desktop surface: persistent audio and + notification settings; VAD, focus-scoped configurable PTT and always-on modes; input, + output and auxiliary-device selection; input/output/auxiliary gain; stereo microphone; + RNNoise; self mute/deafen; event sounds and speech; modeless private conversations; + speaking and mute state; moderation, permissions, channel configuration and account + administration. Added typed administration helpers to `VoiceCat.Core` and per-local-stream + gain to `VoiceCat.Audio`. ScreenCaptureKit now publishes desktop audio with entire-desktop, + application-only and application-exclusion scopes, including optional VoiceOver/speech + exclusion, without introducing a native protocol dependency. The managed client imports + Swift profiles, TOFU pins and Keychain passwords, preserving a one-time JSON backup when it + converts profile storage. Added a Release publishing script with an ad-hoc validation path + and Developer ID/notarization path; consistently re-signing the nested .NET runtime fixed a + hardened-runtime Team-ID launch failure. Signing now operates on a separate distribution + copy so it cannot corrupt MSBuild's incremental app bundle or signing cache. **Verified:** + the ad-hoc Release app passes strict deep signature verification and remains running after + launch; Debug and Release Apple builds + succeed; 190/198 managed tests pass with eight optional native/published-server oracles + skipped; native `dev` CTest passes 29/29. The administration protocol test exercises typed + result round trips and the profile test proves Swift import plus backup. **Next:** manually + exercise ScreenCaptureKit selection and a real multi-human call, complete the VoiceOver + navigation/announcement matrix, and run the credentialed Developer ID notarization path. + Keep the Swift macOS app as the release oracle until all three gates pass; only then remove + its replaceable app/UI code. + - **In progress (2026-09-18): managed macOS client and selectable audio path.** Added a separate .NET 10 AppKit solution with application/menu lifecycle, guest/account connection, explicit TOFU approval, channel selection, roster/chat, disconnect state, accessibility @@ -50,9 +75,8 @@ up instantly. Newest status at the top. clean sustained peer tone in the DTX-disabled Music Room. Lobby's later hiss was confirmed as expected Opus DTX comfort noise for a synthetic sine, not playback corruption. Per user direction, final listen quality will be exercised in real multi-human calls after feature - completion rather than an artificial ten-minute tone. **Next:** add moderation/settings, - ScreenCaptureKit, - VoiceOver verification, signing and notarization. + completion rather than an artificial ten-minute tone. This checkpoint is superseded by the + 2026-09-19 functional-parity entry above. - **Done (2026-09-16): Linux production packaging checkpoint.** Added a real TLS 1.3 `--health-check` with optional certificate pin verification. Linux x64 now has separate diff --git a/clients/apple/dotnet/README.md b/clients/apple/dotnet/README.md index 6faf7ef..a64af2b 100644 --- a/clients/apple/dotnet/README.md +++ b/clients/apple/dotnet/README.md @@ -2,7 +2,9 @@ `VoiceCat.Mac` is the native AppKit C# port. It targets `net10.0-macos27.0` and references the same `VoiceCat.Core` and `VoiceCat.Audio` assemblies used by the Windows client and managed CLI. The explicit platform version selects Microsoft's Xcode 27 preview bindings; the deployed app still supports macOS 14 and later. -The current checkpoint is a functional client: AppKit launch/menu lifecycle, saved server profiles, guest and account authentication, explicit TOFU approval (including changed-key warning), protected-channel password prompts, channel selection, user roster, channel and private text, voice subscription, managed microphone-stream lifecycle, selectable Core Audio input/output devices, disconnect reporting, native accessibility labels, sandbox/network/audio entitlements, and deterministic disposal. Profiles save the host, port, authentication mode, username or guest nickname. An account password can be remembered as a macOS Keychain generic-password item only after successful authentication; it is never written to profile JSON and is removed when remembering is disabled or the profile is deleted. Joining voice requests microphone authorization explicitly. Capture is converted to interleaved 48 kHz int16 PCM through `AVAudioConverter`; playback converts the shared bounded `PcmRing` into Core Audio's native planar Float32 layout inside an allocation-free, non-blocking `AVAudioSourceNode` callback. It does not yet replace the Swift release. Moderation sheets, settings, ScreenCaptureKit sharing, VoiceOver verification, signing and notarization remain. +The managed client now implements the Swift client's functional surface: profiles and Keychain authentication, TOFU, protected channels, hierarchical channel presentation and roster state, channel and modeless private text, microphone/auxiliary/screen-audio streams, selectable Core Audio devices, VAD/PTT/always-on input, stereo microphone, RNNoise, per-stream receive tuning, self/server mute and deafen, full channel configuration, moderation, permissions, account administration, event sounds and speech. It imports the legacy Swift profile, TOFU and Keychain state during cutover. Capture is converted to interleaved 48 kHz int16 PCM through `AVAudioConverter`; playback converts the shared bounded `PcmRing` into Core Audio's native planar Float32 layout inside an allocation-free, non-blocking `AVAudioSourceNode` callback. + +The Swift app remains the release oracle until the managed app passes the manual VoiceOver and real multi-human call matrix. Produce and validate an ad-hoc Release bundle with `zsh clients/apple/dotnet/publish-macos.sh --dry-run`. The script prefers `/usr/local/share/dotnet/dotnet`, where the pinned Apple workload is installed; set `VOICECAT_DOTNET` to override that host. Signing is performed on `bin/Release/distribution/VoiceCat.app`, leaving MSBuild's incremental app bundle untouched. For distribution, set `VOICECAT_CODESIGN_IDENTITY`; setting `APPLE_ID`, `APPLE_TEAM_ID`, and `APPLE_APP_PASSWORD` additionally submits, staples, and Gatekeeper-validates the notarized bundle. Build on Apple Silicon macOS 27 with Xcode 27, .NET SDK 10.0.401 and workload set 10.0.401. Homebrew `protobuf` supplies a native arm64 `protoc`; the current `Grpc.Tools` package contains only an x64 macOS compiler. diff --git a/clients/apple/dotnet/VoiceCat.Mac/AdministrationWindowController.cs b/clients/apple/dotnet/VoiceCat.Mac/AdministrationWindowController.cs new file mode 100644 index 0000000..50ffa59 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/AdministrationWindowController.cs @@ -0,0 +1,166 @@ +using AppKit; +using CoreGraphics; +using VoiceCat.Core; +using Voicecat.V1; +using Channel = Voicecat.V1.Channel; + +namespace VoiceCat.Mac; + +internal sealed class AdministrationWindowController : NSWindowController +{ + private readonly VoiceCatClient client; + private readonly uint selfId; + private readonly NSPopUpButton users = new(new CGRect(25, 335, 310, 28), false); + private readonly NSPopUpButton channels = new(new CGRect(25, 175, 310, 28), false); + private readonly NSTextField status = NSTextField.CreateLabel(""); + + internal AdministrationWindowController(VoiceCatClient client, uint selfId) : base(new NSWindow(new CGRect(0, 0, 560, 420), + NSWindowStyle.Titled | NSWindowStyle.Closable, NSBackingStore.Buffered, false)) + { + this.client = client; this.selfId = selfId; Window!.Title = "VoiceCat Administration"; Window.Center(); + NSView view = Window.ContentView!; + AddLabel(view, "User moderation", 25, 375); view.AddSubview(users); + AddButton(view, "Move…", 350, 333, Move); AddButton(view, "Kick…", 440, 333, Kick); + AddButton(view, "Ban…", 350, 293, Ban); AddButton(view, "Permissions…", 440, 293, Permissions); + AddButton(view, "Server mute", 350, 253, ServerMute); AddButton(view, "Server deafen", 440, 253, ServerDeafen); + AddButton(view, "Audio tuning…", 25, 253, Tune); + AddLabel(view, "Channel administration", 25, 215); view.AddSubview(channels); + AddButton(view, "Create…", 350, 173, CreateChannel); AddButton(view, "Edit…", 440, 173, EditChannel); + AddButton(view, "Delete", 350, 133, DeleteChannel); AddButton(view, "Accounts…", 440, 133, Accounts); + var refresh = new NSButton(new CGRect(25, 75, 100, 32)) { Title = "Refresh" }; refresh.Activated += (_, _) => Refresh(); view.AddSubview(refresh); + status.Frame = new CGRect(25, 25, 510, 35); status.AccessibilityLabel = "Administration result"; view.AddSubview(status); + ((INSAccessibility)users).AccessibilityLabel = "User to administer"; ((INSAccessibility)channels).AccessibilityLabel = "Channel to administer"; + Refresh(); ApplyPermissions(); + } + + private void Refresh() + { + uint selectedUser = Selected(users), selectedChannel = Selected(channels); + users.RemoveAllItems(); foreach (User user in client.Users.Where(user => user.Id != selfId).OrderBy(user => user.Nickname)) Add(users, user.Nickname, user.Id); + channels.RemoveAllItems(); foreach (Channel channel in client.Channels.OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) Add(channels, channel.Name, channel.Id); + Select(users, selectedUser); Select(channels, selectedChannel); + } + + private void ApplyPermissions() + { + Permissions permissions = client.Permissions; + status.StringValue = permissions.IsAdmin ? "Connected as administrator" : "Only actions allowed by your server permissions will succeed."; + } + + private async void Move(object? sender, EventArgs args) + { + uint userId = Selected(users); if (userId == 0) return; + string[] names = client.Channels.Select(channel => channel.Name).ToArray(); int index = PromptChoice("Move user", "Destination channel", names); if (index < 0) return; + Show(await client.MoveUserAsync(userId, client.Channels[index].Id)); + } + private async void Kick(object? sender, EventArgs args) { uint id = Selected(users); string? reason = PromptText("Kick user", "Reason", "Kicked by moderator"); if (id != 0 && reason is not null) Show(await client.KickUserAsync(id, reason)); } + private async void Ban(object? sender, EventArgs args) + { + uint id = Selected(users); if (id == 0) return; + string? reason = PromptText("Ban user", "Reason (append @1h, @1d, or @permanent)", "Banned by moderator @1d"); if (reason is null) return; + ulong expiry = 0; string lowered = reason.ToLowerInvariant(); + if (lowered.EndsWith("@1h")) expiry = checked((ulong)DateTimeOffset.UtcNow.AddHours(1).ToUnixTimeMilliseconds()); + else if (lowered.EndsWith("@1d")) expiry = checked((ulong)DateTimeOffset.UtcNow.AddDays(1).ToUnixTimeMilliseconds()); + Show(await client.BanUserAsync(id, reason.Replace("@1h", "", StringComparison.OrdinalIgnoreCase).Replace("@1d", "", StringComparison.OrdinalIgnoreCase).Replace("@permanent", "", StringComparison.OrdinalIgnoreCase).Trim(), expiry)); + } + private async void Permissions(object? sender, EventArgs args) + { + uint id = Selected(users); if (id == 0) return; + Permissions? permissions = PermissionEditor.Run(); if (permissions is not null) Show(await client.SetPermissionsAsync(id, permissions)); + } + private async void ServerMute(object? sender, EventArgs args) { User? user = User(); if (user is not null) Show(await client.SetServerMuteAsync(user.Id, !user.ServerMuted, user.ServerDeafened)); } + private async void ServerDeafen(object? sender, EventArgs args) { User? user = User(); if (user is not null) Show(await client.SetServerMuteAsync(user.Id, user.ServerMuted, !user.ServerDeafened)); } + private void Tune(object? sender, EventArgs args) { User? user = User(); if (user is not null) PerUserTuning.Run(client, user); } + private async void CreateChannel(object? sender, EventArgs args) { ChannelEdit? edit = ChannelEditor.Run(null, client.Channels); if (edit is not null) { Show(await client.CreateChannelAsync(edit.Channel, edit.Password)); Refresh(); } } + private async void EditChannel(object? sender, EventArgs args) { Channel? channel = Channel(); if (channel is null) return; ChannelEdit? edit = ChannelEditor.Run(channel, client.Channels); if (edit is not null) { Show(await client.EditChannelAsync(edit.Channel, edit.Password)); Refresh(); } } + private async void DeleteChannel(object? sender, EventArgs args) + { + Channel? channel = Channel(); if (channel is null || channel.Id == 1) return; + if (!Confirm("Delete channel?", channel.Name)) return; Show(await client.DeleteChannelAsync(channel.Id)); Refresh(); + } + private async void Accounts(object? sender, EventArgs args) => await AccountEditor.RunAsync(client); + + private User? User() => client.Users.FirstOrDefault(user => user.Id == Selected(users)); + private Channel? Channel() => client.Channels.FirstOrDefault(channel => channel.Id == Selected(channels)); + private void Show(GenericResult result) { status.StringValue = result.Ok ? result.Message.Length == 0 ? "Operation completed." : result.Message : result.Message; Refresh(); } + private static void Add(NSPopUpButton picker, string title, uint id) { picker.AddItem(title); picker.LastItem!.RepresentedObject = new Foundation.NSString(id.ToString()); } + private static uint Selected(NSPopUpButton picker) => uint.TryParse(picker.SelectedItem?.RepresentedObject?.ToString(), out uint id) ? id : 0; + private static void Select(NSPopUpButton picker, uint id) { for (nint i = 0; i < picker.ItemCount; i++) if (picker.ItemAtIndex(i)?.RepresentedObject?.ToString() == id.ToString()) { picker.SelectItem(i); break; } } + private static void AddLabel(NSView view, string value, double x, double y) { NSTextField label = NSTextField.CreateLabel(value); label.Frame = new CGRect(x, y, 300, 22); label.Font = NSFont.BoldSystemFontOfSize(13)!; view.AddSubview(label); } + private static void AddButton(NSView view, string title, double x, double y, EventHandler handler) { var button = new NSButton(new CGRect(x, y, 95, 32)) { Title = title }; button.Activated += handler; view.AddSubview(button); } + + internal static string? PromptText(string title, string prompt, string value = "", bool secure = false) + { + NSTextField field = secure ? new NSSecureTextField(new CGRect(0, 0, 340, 26)) : new NSTextField(new CGRect(0, 0, 340, 26)); field.StringValue = value; + var alert = new NSAlert { MessageText = title, InformativeText = prompt, AccessoryView = field }; alert.AddButton("OK"); alert.AddButton("Cancel"); + return alert.RunModal() == 1000 ? field.StringValue : null; + } + internal static int PromptChoice(string title, string prompt, string[] values) + { + var picker = new NSPopUpButton(new CGRect(0, 0, 340, 28), false); picker.AddItems(values); + var alert = new NSAlert { MessageText = title, InformativeText = prompt, AccessoryView = picker }; alert.AddButton("OK"); alert.AddButton("Cancel"); + return alert.RunModal() == 1000 ? checked((int)picker.IndexOfSelectedItem) : -1; + } + internal static bool Confirm(string title, string detail) { var alert = new NSAlert { MessageText = title, InformativeText = detail, AlertStyle = NSAlertStyle.Warning }; alert.AddButton("Continue"); alert.AddButton("Cancel"); return alert.RunModal() == 1000; } +} + +internal sealed record ChannelEdit(Channel Channel, string Password); + +internal static class PermissionEditor +{ + internal static Permissions? Run() + { + string[] labels = ["Create temporary channels", "Kick users", "Ban users", "Move users", "Manage accounts", "Full administrator"]; + var view = new NSView(new CGRect(0, 0, 330, 160)); var checks = new List(); + for (int i = 0; i < labels.Length; i++) { NSButton check = NSButton.CreateCheckbox(labels[i], () => { }); check.Frame = new CGRect(0, 135 - i * 25, 320, 22); view.AddSubview(check); checks.Add(check); } + var alert = new NSAlert { MessageText = "Set permissions", AccessoryView = view }; alert.AddButton("Save"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return null; + return new() { CanCreateTempChannel = On(0), CanKick = On(1), CanBan = On(2), CanMoveUsers = On(3), CanAdminAccounts = On(4), IsAdmin = On(5) }; + bool On(int i) => checks[i].State == NSCellStateValue.On; + } +} + +internal static class PerUserTuning +{ + internal static void Run(VoiceCatClient client, User user) + { + if (user.Streams.Count == 0) { AdministrationWindowController.PromptText("Audio tuning", "This user has no active streams."); return; } + var stream = new NSPopUpButton(new CGRect(0, 95, 340, 28), false); stream.AddItems(user.Streams.Select(value => $"{value.Label} ({value.Kind})").ToArray()); + var gain = new NSSlider(new CGRect(0, 60, 340, 24)) { MinValue = 0, MaxValue = 400, DoubleValue = 100 }; + var muted = NSButton.CreateCheckbox("Mute this stream", () => { }); muted.Frame = new CGRect(0, 30, 160, 24); + var noise = NSButton.CreateCheckbox("Receive noise reduction", () => { }); noise.Frame = new CGRect(170, 30, 170, 24); + var view = new NSView(new CGRect(0, 0, 340, 130)); view.AddSubview(stream); view.AddSubview(gain); view.AddSubview(muted); view.AddSubview(noise); + void LoadSelected() + { + if (stream.IndexOfSelectedItem < 0) return; + StreamInfo selected = user.Streams[checked((int)stream.IndexOfSelectedItem)]; + (float Gain, bool Muted, bool NoiseReduction)? state = client.Audio.GetRemotePlayback(user.Id, selected.StreamId); + gain.DoubleValue = (state?.Gain ?? 1) * 100; muted.State = state?.Muted == true ? NSCellStateValue.On : NSCellStateValue.Off; + noise.State = state?.NoiseReduction == true ? NSCellStateValue.On : NSCellStateValue.Off; + } + stream.Activated += (_, _) => LoadSelected(); LoadSelected(); + var alert = new NSAlert { MessageText = $"Audio — {user.Nickname}", InformativeText = "Stream, volume, mute, and receive noise reduction", AccessoryView = view }; + alert.AddButton("Apply"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return; + StreamInfo selected = user.Streams[checked((int)stream.IndexOfSelectedItem)]; client.Audio.SetRemotePlayback(user.Id, selected.StreamId, (float)gain.DoubleValue / 100, + muted.State == NSCellStateValue.On, noise.State == NSCellStateValue.On); + } +} + +internal static class AccountEditor +{ + internal static async Task RunAsync(VoiceCatClient client) + { + IReadOnlyList accounts = await client.ListAccountsAsync(); + string[] actions = ["Create account", "Reset password", "Delete account", "Close"]; + int action = AdministrationWindowController.PromptChoice("Server accounts", string.Join("\n", accounts.Select(a => $"{a.Username}{(a.IsAdmin ? " (admin)" : "")}")), actions); + if (action is < 0 or 3) return; + if (action == 0) + { + string? name = AdministrationWindowController.PromptText("Create account", "Username"); if (string.IsNullOrWhiteSpace(name)) return; + string? password = AdministrationWindowController.PromptText("Create account", "Password", secure: true); if (!string.IsNullOrEmpty(password)) await client.CreateAccountAsync(name.Trim(), password); + return; + } + int selected = AdministrationWindowController.PromptChoice(action == 1 ? "Reset password" : "Delete account", "Account", accounts.Select(a => a.Username).ToArray()); if (selected < 0) return; + if (action == 1) { string? password = AdministrationWindowController.PromptText("Reset password", "New password", secure: true); if (!string.IsNullOrEmpty(password)) await client.ResetPasswordAsync(accounts[selected].Username, password); } + else if (AdministrationWindowController.Confirm("Delete account?", accounts[selected].Username)) await client.DeleteAccountAsync(accounts[selected].Username); + } +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/ChannelEditor.cs b/clients/apple/dotnet/VoiceCat.Mac/ChannelEditor.cs new file mode 100644 index 0000000..c2eb07d --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/ChannelEditor.cs @@ -0,0 +1,58 @@ +using AppKit; +using CoreGraphics; +using Voicecat.V1; +using Channel = Voicecat.V1.Channel; + +namespace VoiceCat.Mac; + +internal static class ChannelEditor +{ + internal static ChannelEdit? Run(Channel? existing, IReadOnlyList channels) + { + var view = new NSView(new CGRect(0, 0, 520, 500)); + var name = Field(existing?.Name ?? "", 360); var topic = Field(existing?.Topic ?? "", 330); + var parent = Picker(channels.Select(channel => channel.Name).Prepend("No parent").ToArray(), 300); + int parentIndex = existing is null ? 0 : channels.ToList().FindIndex(channel => channel.Id == existing.ParentId) + 1; parent.SelectItem(Math.Max(0, parentIndex)); + var type = Picker(["Permanent", "Temporary"], 270); type.SelectItem(existing?.Type == ChannelType.ChannelTemporary ? 1 : 0); + var maximum = Field((existing?.MaxUsers ?? 0).ToString(), 240); var order = Field((existing?.Order ?? 0).ToString(), 210); + var password = new NSSecureTextField(new CGRect(155, 180, 345, 24)) { PlaceholderString = existing?.PasswordProtected == true ? "Leave blank to preserve" : "Optional" }; + var mode = Picker(["Mono", "Stereo"], 150); mode.SelectItem(existing?.Audio?.Mode == ChannelMode.ModeStereo ? 1 : 0); + var rate = Picker(["8000", "12000", "16000", "24000", "48000"], 120); Select(rate, (existing?.Audio?.SampleRate ?? 48000).ToString()); + var frame = Picker(["5", "10", "20", "40", "60"], 90); Select(frame, (existing?.Audio?.FrameMs ?? 20).ToString()); + var application = Picker(["VoIP", "Audio", "Low delay"], 60); application.SelectItem((int)(existing?.Audio?.Application ?? OpusApplication.OpusVoip)); + var bitrate = Field((existing?.Audio?.BitrateBps ?? 64000).ToString(), 30); + Add(view, "Name", name, 360); Add(view, "Topic", topic, 330); Add(view, "Parent", parent, 300); Add(view, "Type", type, 270); + Add(view, "Maximum users (0=none)", maximum, 240); Add(view, "Sort order", order, 210); Add(view, "Password", password, 180); + Add(view, "Mode", mode, 150); Add(view, "Sample rate", rate, 120); Add(view, "Frame ms", frame, 90); Add(view, "Application", application, 60); Add(view, "Bitrate", bitrate, 30); + + var advanced = new NSView(new CGRect(0, 0, 520, 105)); + var fec = Check("In-band FEC", existing?.Audio?.Fec ?? true, 0, 75); var dtx = Check("DTX", existing?.Audio?.Dtx ?? true, 130, 75); + var dred = Check("Deep redundancy", existing?.Audio?.Dred ?? false, 230, 75); + var loss = Field((existing?.Audio?.ExpectedPacketLoss ?? 5).ToString(), 40); loss.Frame = new CGRect(155, 40, 100, 24); + var complexity = Field((existing?.Audio?.Complexity ?? 10).ToString(), 10); complexity.Frame = new CGRect(155, 10, 100, 24); + advanced.AddSubview(fec); advanced.AddSubview(dtx); advanced.AddSubview(dred); Add(advanced, "Packet loss %", loss, 40); Add(advanced, "Complexity 0–10", complexity, 10); + var container = new NSView(new CGRect(0, 0, 520, 620)); view.Frame = new CGRect(0, 110, 520, 500); container.AddSubview(view); container.AddSubview(advanced); + var alert = new NSAlert { MessageText = existing is null ? "Create channel" : "Edit channel", AccessoryView = container }; + alert.AddButton(existing is null ? "Create" : "Save"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return null; + if (string.IsNullOrWhiteSpace(name.StringValue) || !uint.TryParse(maximum.StringValue, out uint max) || !int.TryParse(order.StringValue, out int sort) || + !uint.TryParse(rate.TitleOfSelectedItem, out uint sampleRate) || !uint.TryParse(frame.TitleOfSelectedItem, out uint frameMs) || + !uint.TryParse(bitrate.StringValue, out uint bitrateValue) || !uint.TryParse(loss.StringValue, out uint packetLoss) || + !uint.TryParse(complexity.StringValue, out uint complexityValue) || packetLoss > 100 || complexityValue > 10) return null; + uint parentId = parent.IndexOfSelectedItem <= 0 ? 0 : channels[checked((int)parent.IndexOfSelectedItem - 1)].Id; + return new(new Channel + { + Id = existing?.Id ?? 0, ParentId = parentId, Name = name.StringValue.Trim(), Topic = topic.StringValue.Trim(), MaxUsers = max, Order = sort, + Type = type.IndexOfSelectedItem == 1 ? ChannelType.ChannelTemporary : ChannelType.ChannelPermanent, + Audio = new AudioConfig { Codec = 0, Mode = mode.IndexOfSelectedItem == 1 ? ChannelMode.ModeStereo : ChannelMode.ModeMono, + SampleRate = sampleRate, FrameMs = frameMs, Application = (OpusApplication)(int)application.IndexOfSelectedItem, BitrateBps = bitrateValue, + Fec = On(fec), Dtx = On(dtx), Dred = On(dred), ExpectedPacketLoss = packetLoss, Complexity = complexityValue } + }, password.StringValue); + } + + private static NSTextField Field(string value, double y) => new(new CGRect(155, y, 345, 24)) { StringValue = value }; + private static NSPopUpButton Picker(string[] values, double y) { var picker = new NSPopUpButton(new CGRect(155, y, 345, 26), false); picker.AddItems(values); return picker; } + private static NSButton Check(string title, bool value, double x, double y) { NSButton check = NSButton.CreateCheckbox(title, () => { }); check.Frame = new CGRect(x, y, 125, 24); check.State = value ? NSCellStateValue.On : NSCellStateValue.Off; return check; } + private static bool On(NSButton value) => value.State == NSCellStateValue.On; + private static void Add(NSView view, string label, NSView control, double y) { var text = NSTextField.CreateLabel(label); text.Frame = new CGRect(0, y + 2, 145, 20); view.AddSubview(text); view.AddSubview(control); ((INSAccessibility)control).AccessibilityLabel = label; } + private static void Select(NSPopUpButton picker, string value) { for (nint i = 0; i < picker.ItemCount; i++) if (picker.ItemTitle(i) == value) { picker.SelectItem(i); return; } } +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/ConnectWindowController.cs b/clients/apple/dotnet/VoiceCat.Mac/ConnectWindowController.cs index fa7d27b..743bd30 100644 --- a/clients/apple/dotnet/VoiceCat.Mac/ConnectWindowController.cs +++ b/clients/apple/dotnet/VoiceCat.Mac/ConnectWindowController.cs @@ -80,7 +80,8 @@ internal sealed class ConnectWindowController : NSWindowController host.StringValue = FormatEndpoint(profile.Host, profile.Port); authentication.SelectItem(profile.Authentication == ServerAuthentication.Guest ? 0 : 1); identity.StringValue = profile.Authentication == ServerAuthentication.Guest ? profile.Nickname ?? Environment.UserName : profile.Username ?? ""; - string? storedPassword = profile.Authentication == ServerAuthentication.Account ? keychain.Load(profile.Id) : null; + string? storedPassword = profile.Authentication == ServerAuthentication.Account + ? keychain.Load(profile.Id) ?? keychain.LoadLegacy(profile.LegacyKeychainTag) : null; password.StringValue = storedPassword ?? ""; rememberPassword.State = storedPassword is null ? NSCellStateValue.Off : NSCellStateValue.On; UpdateAuthentication(); @@ -135,11 +136,12 @@ internal sealed class ConnectWindowController : NSWindowController { SetBusy(true); status.StringValue = "Connecting…"; ServerProfile profile = ReadProfile(); - string pins = Path.Combine(SupportDirectory, "tofu.txt"); + string pins = Path.Combine(SupportDirectory, "tofu_pins.txt"); client = new("VoiceCat macOS", "0.1.0", pins); await client.ConnectAsync(profile.Host, profile.Port, ConfirmIdentity); string accountPassword = password.StringValue; - if (profile.Authentication == ServerAuthentication.Account && accountPassword.Length == 0) accountPassword = keychain.Load(profile.Id) ?? ""; + if (profile.Authentication == ServerAuthentication.Account && accountPassword.Length == 0) + accountPassword = keychain.Load(profile.Id) ?? keychain.LoadLegacy(profile.LegacyKeychainTag) ?? ""; if (profile.Authentication == ServerAuthentication.Account && accountPassword.Length == 0) throw new ArgumentException("Enter the account password."); password.StringValue = ""; var result = profile.Authentication == ServerAuthentication.Guest diff --git a/clients/apple/dotnet/VoiceCat.Mac/EventFeedback.cs b/clients/apple/dotnet/VoiceCat.Mac/EventFeedback.cs new file mode 100644 index 0000000..e026527 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/EventFeedback.cs @@ -0,0 +1,49 @@ +using AVFoundation; +using Foundation; + +namespace VoiceCat.Mac; + +internal enum SoundEvent { ChannelJoin, ChannelLeave, ChannelReceived, ChannelSent, PrivateReceived, PrivateSent, Login, Logout, ConnectionLost, VoiceOn, VoiceOff, VoiceStart, VoiceStop, PushToTalk } + +internal sealed class EventFeedback : IDisposable +{ + private readonly MacSettings settings; + private readonly AVSpeechSynthesizer speech = new(); + private readonly Dictionary players = []; + + internal EventFeedback(MacSettings settings) => this.settings = settings; + + internal void Play(SoundEvent sound) + { + if (!settings.EventSounds || settings.EventVolume <= 0 || + (sound is SoundEvent.VoiceStart or SoundEvent.VoiceStop && !settings.SelfTalkSounds) || + (sound == SoundEvent.PushToTalk && !settings.PushToTalkSound)) return; + if (!players.TryGetValue(sound, out AVAudioPlayer? player)) + { + string path = Path.Combine(NSBundle.MainBundle.ResourcePath ?? "", "Sounds", FileName(sound) + ".wav"); + if (!File.Exists(path)) return; + player = AVAudioPlayer.FromUrl(NSUrl.FromFilename(path)); + if (player is null) return; + player.PrepareToPlay(); players[sound] = player; + } + player.Volume = settings.EventVolume; player.CurrentTime = 0; player.Play(); + } + + internal void Speak(string text) + { + if (!settings.SpokenEvents || string.IsNullOrWhiteSpace(text)) return; + speech.SpeakUtterance(new AVSpeechUtterance(text.Trim())); + } + + private static string FileName(SoundEvent value) => value switch + { + SoundEvent.ChannelJoin => "channel_join", SoundEvent.ChannelLeave => "channel_leave", + SoundEvent.ChannelReceived => "channel_recv", SoundEvent.ChannelSent => "channel_sent", + SoundEvent.PrivateReceived => "pm_recv", SoundEvent.PrivateSent => "pm_sent", + SoundEvent.Login => "login", SoundEvent.Logout => "logout", SoundEvent.ConnectionLost => "connection_lost", + SoundEvent.VoiceOn => "voice_on", SoundEvent.VoiceOff => "voice_off", SoundEvent.VoiceStart => "va_start", + SoundEvent.VoiceStop => "va_stop", _ => "ptt" + }; + + public void Dispose() { foreach (AVAudioPlayer player in players.Values) player.Dispose(); speech.Dispose(); } +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/MacKeychainPasswordStore.cs b/clients/apple/dotnet/VoiceCat.Mac/MacKeychainPasswordStore.cs index f09e13b..02eab54 100644 --- a/clients/apple/dotnet/VoiceCat.Mac/MacKeychainPasswordStore.cs +++ b/clients/apple/dotnet/VoiceCat.Mac/MacKeychainPasswordStore.cs @@ -19,6 +19,17 @@ internal sealed class MacKeychainPasswordStore finally { CryptographicOperations.ZeroMemory(encoded); } } + internal string? LoadLegacy(string? keychainTag) + { + if (string.IsNullOrWhiteSpace(keychainTag)) return null; + using var query = new SecRecord(SecKind.GenericPassword) { Service = "cat.voice.VoiceCatMac", Account = keychainTag }; + using SecRecord? result = SecKeyChain.QueryAsRecord(query, out SecStatusCode status); + if (status != SecStatusCode.Success || result?.ValueData is not { } data) return null; + byte[] encoded = data.ToArray(); + try { return Encoding.UTF8.GetString(encoded); } + finally { CryptographicOperations.ZeroMemory(encoded); } + } + internal void Save(Guid profileId, string password) { ArgumentException.ThrowIfNullOrEmpty(password); diff --git a/clients/apple/dotnet/VoiceCat.Mac/MacSettings.cs b/clients/apple/dotnet/VoiceCat.Mac/MacSettings.cs new file mode 100644 index 0000000..a255ee6 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/MacSettings.cs @@ -0,0 +1,69 @@ +using Foundation; +using VoiceCat.Audio; + +namespace VoiceCat.Mac; + +internal sealed class MacSettings +{ + private readonly NSUserDefaults values = NSUserDefaults.StandardUserDefaults; + + internal AudioInputMode InputMode { get; set; } = AudioInputMode.VoiceActivation; + internal float VadThreshold { get; set; } = 0.05f; + internal float InputGain { get; set; } = 1f; + internal float OutputGain { get; set; } = 0.8f; + internal bool InputNoiseReduction { get; set; } + internal bool StereoMicrophone { get; set; } + internal ushort PushToTalkKeyCode { get; set; } = 0x60; + internal string? InputDeviceId { get; set; } + internal string? OutputDeviceId { get; set; } + internal bool AuxiliaryEnabled { get; set; } + internal string? AuxiliaryDeviceId { get; set; } + internal float AuxiliaryGain { get; set; } = 1f; + internal bool EventSounds { get; set; } = true; + internal bool SpokenEvents { get; set; } + internal float EventVolume { get; set; } = 1f; + internal bool SelfTalkSounds { get; set; } + internal bool PushToTalkSound { get; set; } + + internal void Load() + { + NSString[] keys = [(NSString)"voice.inputMode", (NSString)"voice.vadThreshold", (NSString)"voice.inputGain", (NSString)"voice.outputGain", + (NSString)"voice.pttKeyCode", (NSString)"voice.auxGain", (NSString)"feedback.sounds", (NSString)"feedback.volume"]; + NSObject[] defaults = [NSNumber.FromInt32((int)AudioInputMode.VoiceActivation), NSNumber.FromFloat(0.05f), + NSNumber.FromFloat(1f), NSNumber.FromFloat(0.8f), NSNumber.FromInt32(0x60), NSNumber.FromFloat(1f), + NSNumber.FromBoolean(true), NSNumber.FromFloat(1f)]; + values.RegisterDefaults(new NSDictionary(keys, defaults)); + InputMode = Enum.IsDefined(typeof(AudioInputMode), (int)values.IntForKey("voice.inputMode")) + ? (AudioInputMode)(int)values.IntForKey("voice.inputMode") : AudioInputMode.VoiceActivation; + VadThreshold = Math.Clamp(values.FloatForKey("voice.vadThreshold"), 0f, 1f); + InputGain = Math.Clamp(values.FloatForKey("voice.inputGain"), 0f, 4f); + OutputGain = Math.Clamp(values.FloatForKey("voice.outputGain"), 0f, 1f); + InputNoiseReduction = values.BoolForKey("voice.inputNoiseReduction"); + StereoMicrophone = values.BoolForKey("voice.stereoMic"); + PushToTalkKeyCode = checked((ushort)Math.Clamp(values.IntForKey("voice.pttKeyCode"), 0, ushort.MaxValue)); + InputDeviceId = EmptyToNull(values.StringForKey("voice.inputDevice")); + OutputDeviceId = EmptyToNull(values.StringForKey("voice.outputDevice")); + AuxiliaryEnabled = values.BoolForKey("voice.auxEnabled"); + AuxiliaryDeviceId = EmptyToNull(values.StringForKey("voice.auxDeviceUID")); + AuxiliaryGain = Math.Clamp(values.FloatForKey("voice.auxGain"), 0f, 4f); + EventSounds = values.BoolForKey("feedback.sounds"); + SpokenEvents = values.BoolForKey("feedback.speech"); + EventVolume = Math.Clamp(values.FloatForKey("feedback.volume"), 0f, 1f); + SelfTalkSounds = values.BoolForKey("feedback.selfTalk"); + PushToTalkSound = values.BoolForKey("feedback.ptt"); + } + + internal void Save() + { + values.SetInt((int)InputMode, "voice.inputMode"); values.SetFloat(VadThreshold, "voice.vadThreshold"); + values.SetFloat(InputGain, "voice.inputGain"); values.SetFloat(OutputGain, "voice.outputGain"); + values.SetBool(InputNoiseReduction, "voice.inputNoiseReduction"); values.SetBool(StereoMicrophone, "voice.stereoMic"); + values.SetInt(PushToTalkKeyCode, "voice.pttKeyCode"); Set("voice.inputDevice", InputDeviceId); Set("voice.outputDevice", OutputDeviceId); + values.SetBool(AuxiliaryEnabled, "voice.auxEnabled"); Set("voice.auxDeviceUID", AuxiliaryDeviceId); values.SetFloat(AuxiliaryGain, "voice.auxGain"); + values.SetBool(EventSounds, "feedback.sounds"); values.SetBool(SpokenEvents, "feedback.speech"); values.SetFloat(EventVolume, "feedback.volume"); + values.SetBool(SelfTalkSounds, "feedback.selfTalk"); values.SetBool(PushToTalkSound, "feedback.ptt"); values.Synchronize(); + } + + private void Set(string key, string? value) { if (value is null) values.RemoveObject(key); else values.SetString(value, key); } + private static string? EmptyToNull(string? value) => string.IsNullOrWhiteSpace(value) ? null : value; +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/MainWindowController.cs b/clients/apple/dotnet/VoiceCat.Mac/MainWindowController.cs index 13a5cb1..3aab98e 100644 --- a/clients/apple/dotnet/VoiceCat.Mac/MainWindowController.cs +++ b/clients/apple/dotnet/VoiceCat.Mac/MainWindowController.cs @@ -22,20 +22,44 @@ internal sealed class MainWindowController : NSWindowController private readonly NSTextField compose = new(new CGRect(400, 55, 230, 28)) { PlaceholderString = "Message" }; private readonly NSButton send = new(new CGRect(640, 53, 90, 32)) { Title = "Send" }; private readonly NSButton voice = new(new CGRect(630, 510, 100, 32)) { Title = "Join voice" }; + private readonly NSButton settingsButton = new(new CGRect(740, 510, 90, 32)) { Title = "Settings" }; + private readonly NSButton screenAudioButton = new(new CGRect(840, 510, 130, 32)) { Title = "Share audio" }; + private readonly NSButton administrationButton = new(new CGRect(840, 20, 130, 28)) { Title = "Administration" }; + private readonly NSButton privateMessageButton = new(new CGRect(700, 20, 130, 28)) { Title = "Private chat" }; + private readonly NSButton muteButton = NSButton.CreateCheckbox("Mute", () => { }); + private readonly NSButton deafenButton = NSButton.CreateCheckbox("Deafen", () => { }); private readonly NSTextField status = NSTextField.CreateLabel("Connected"); + private readonly MacSettings settings = new(); + private readonly EventFeedback feedback; private readonly NSTimer timer; private uint currentChannel = 1; private uint microphoneStreamId; + private uint auxiliaryStreamId; + private uint screenAudioStreamId; private bool joinedVoice; private bool changingChannel; private IAudioCapture? microphone; + private IAudioCapture? auxiliary; private IAudioPlayback? playback; + private ScreenAudioCapture? screenAudio; + private ScreenAudioSelection screenAudioSelection = ScreenAudioSelection.Default; + private SettingsWindowController? settingsWindow; + private AdministrationWindowController? administrationWindow; + private readonly Dictionary privateWindows = []; + private readonly HashSet talkingUsers = []; + private bool lastTalking; + private string? activeInputDevice; + private string? activeOutputDevice; + private string? activeAuxiliaryDevice; + private bool activeStereo; + private NSObject? pushToTalkMonitor; + private bool pushToTalkEngaged; private long lastRemoteAudioTick; - internal MainWindowController(VoiceCatClient client, uint selfId, string nickname) : base(new NSWindow(new CGRect(0, 0, 760, 570), + internal MainWindowController(VoiceCatClient client, uint selfId, string nickname) : base(new NSWindow(new CGRect(0, 0, 1000, 570), NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable | NSWindowStyle.Miniaturizable, NSBackingStore.Buffered, false)) { - this.client = client; this.selfId = selfId; + this.client = client; this.selfId = selfId; settings.Load(); feedback = new(settings); Window!.Title = $"VoiceCat — {nickname}"; Window.Center(); Window.MinSize = new CGSize(680, 480); var content = Window.ContentView!; ((INSAccessibility)channels).AccessibilityLabel = "Channel"; channels.Activated += ChangeChannel; content.AddSubview(channels); @@ -43,26 +67,67 @@ internal sealed class MainWindowController : NSWindowController ((INSAccessibility)inputDevice).AccessibilityLabel = "Microphone input device"; inputDevice.Activated += ChangeAudioDevice; content.AddSubview(inputDevice); ((INSAccessibility)outputDevice).AccessibilityLabel = "Audio output device"; outputDevice.Activated += ChangeAudioDevice; content.AddSubview(outputDevice); ((INSAccessibility)voice).AccessibilityLabel = "Join or leave voice"; voice.Activated += ToggleVoice; content.AddSubview(voice); + ((INSAccessibility)settingsButton).AccessibilityLabel = "Open audio and notification settings"; settingsButton.Activated += OpenSettings; content.AddSubview(settingsButton); + ((INSAccessibility)screenAudioButton).AccessibilityLabel = "Start or stop sharing screen audio"; screenAudioButton.Activated += ToggleScreenAudio; content.AddSubview(screenAudioButton); + ((INSAccessibility)administrationButton).AccessibilityLabel = "Open server moderation and administration"; administrationButton.Activated += OpenAdministration; content.AddSubview(administrationButton); + ((INSAccessibility)privateMessageButton).AccessibilityLabel = "Open a private conversation with the selected message recipient"; privateMessageButton.Activated += OpenPrivateMessage; content.AddSubview(privateMessageButton); + muteButton.Frame = new CGRect(275, 490, 75, 22); deafenButton.Frame = new CGRect(355, 490, 85, 22); + ((INSAccessibility)muteButton).AccessibilityLabel = "Mute microphone"; ((INSAccessibility)deafenButton).AccessibilityLabel = "Deafen playback"; + muteButton.Activated += SelfAudioChanged; deafenButton.Activated += SelfAudioChanged; content.AddSubview(muteButton); content.AddSubview(deafenButton); var userScroll = new NSScrollView(new CGRect(20, 90, 190, 410)) { HasVerticalScroller = true, DocumentView = users }; userScroll.AccessibilityLabel = "Users in channel"; content.AddSubview(userScroll); var chatScroll = new NSScrollView(new CGRect(230, 90, 500, 410)) { HasVerticalScroller = true, DocumentView = chat }; chatScroll.AccessibilityLabel = "Channel messages"; content.AddSubview(chatScroll); ((INSAccessibility)messageTarget).AccessibilityLabel = "Message recipient"; content.AddSubview(messageTarget); ((INSAccessibility)compose).AccessibilityLabel = "Message"; compose.Activated += Send; content.AddSubview(compose); ((INSAccessibility)send).AccessibilityLabel = "Send message"; send.Activated += Send; content.AddSubview(send); - status.Frame = new CGRect(20, 22, 700, 22); status.AccessibilityLabel = "Connection status"; content.AddSubview(status); + status.Frame = new CGRect(20, 22, 660, 22); status.AccessibilityLabel = "Connection status"; content.AddSubview(status); timer = NSTimer.CreateRepeatingScheduledTimer(TimeSpan.FromMilliseconds(50), _ => Pump()); + ApplyEngineSettings(); feedback.Play(SoundEvent.Login); feedback.Speak("Connected"); + pushToTalkMonitor = NSEvent.AddLocalMonitorForEventsMatchingMask(NSEventMask.KeyDown | NSEventMask.KeyUp, value => HandlePushToTalk(value)!); RefreshState(); } + private NSEvent? HandlePushToTalk(NSEvent value) + { + if (settings.InputMode != AudioInputMode.PushToTalk || value.KeyCode != settings.PushToTalkKeyCode) return value; + bool engaged = value.Type == NSEventType.KeyDown; + client.Audio.PushToTalk = engaged; + if (engaged && !pushToTalkEngaged) feedback.Play(SoundEvent.PushToTalk); + pushToTalkEngaged = engaged; return null; + } private void Pump() { while (client.TryReadEvent(out Envelope? envelope)) { - if (envelope!.TextMessage is { } text) Append(FormatMessage(text)); + if (envelope!.TextMessage is { } text) + { + if (text.Scope == TextScope.TextPrivate) + { + uint other = text.SenderId == selfId ? text.TargetId : text.SenderId; + GetPrivateWindow(other).Append(text, Name(text.SenderId)); + feedback.Play(text.SenderId == selfId ? SoundEvent.PrivateSent : SoundEvent.PrivateReceived); + if (text.SenderId != selfId) feedback.Speak($"Private message from {Name(text.SenderId)}: {text.Body}"); + } + else { Append(FormatMessage(text)); feedback.Play(text.SenderId == selfId ? SoundEvent.ChannelSent : SoundEvent.ChannelReceived); } + } + if (envelope.StreamState is { } streamState) + { + if (streamState.Talking) talkingUsers.Add(streamState.UserId); else talkingUsers.Remove(streamState.UserId); + RefreshState(); + } + if (envelope.UserEvent is { } userEvent) + { + string name = userEvent.User?.Nickname ?? Name(userEvent.LeftId); + if (userEvent.Kind == UserEvent.Types.Kind.Joined && userEvent.User?.ChannelId == currentChannel && userEvent.User.Id != selfId) + { AppendActivity($"{name} joined the channel"); feedback.Play(SoundEvent.ChannelJoin); feedback.Speak($"{name} joined"); } + else if (userEvent.Kind == UserEvent.Types.Kind.Left) + { talkingUsers.Remove(userEvent.LeftId); AppendActivity($"{name} left the server"); feedback.Play(SoundEvent.ChannelLeave); } + } if (envelope.ServerState is not null || envelope.ChannelEvent is not null || envelope.UserEvent is not null) RefreshState(); if (envelope.Disconnect is { } disconnected) { StopAudioDevices(); joinedVoice = false; voice.Title = "Join voice"; status.StringValue = "Disconnected: " + disconnected.Reason; voice.Enabled = send.Enabled = messageTarget.Enabled = false; + feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost"); Announce(status.StringValue); } } if (client.Audio.Failure is { } failure) { status.StringValue = "Audio stopped: " + failure.Message; return; } @@ -70,22 +135,111 @@ internal sealed class MainWindowController : NSWindowController if (joinedVoice) { (float level, bool talking) = client.Audio.GetLocalLevel(microphoneStreamId); + if (talking != lastTalking) + { + lastTalking = talking; client.PublishStreamState(microphoneStreamId, talking); + feedback.Play(talking ? SoundEvent.VoiceStart : SoundEvent.VoiceStop); + } bool receiving = Environment.TickCount64 - Volatile.Read(ref lastRemoteAudioTick) < 1_000; status.StringValue = $"Voice connected · Mic {(talking ? "sending" : "idle")} {level:P0} · Remote audio {(receiving ? "active" : "idle")}"; } } + private void OpenSettings(object? sender, EventArgs args) + { + settingsWindow ??= new(this, settings, audioBackend); + settingsWindow.ShowWindow(this); settingsWindow.Window?.MakeKeyAndOrderFront(this); + } + private void OpenAdministration(object? sender, EventArgs args) + { + administrationWindow ??= new(client, selfId); administrationWindow.ShowWindow(this); administrationWindow.Window?.MakeKeyAndOrderFront(this); + } + private void OpenPrivateMessage(object? sender, EventArgs args) + { + uint target = uint.TryParse(messageTarget.SelectedItem?.RepresentedObject?.ToString(), out uint selected) ? selected : 0; + if (target == 0) { status.StringValue = "Choose a private message recipient first."; return; } + PrivateMessageWindowController window = GetPrivateWindow(target); window.ShowWindow(this); window.Window?.MakeKeyAndOrderFront(this); + } + private PrivateMessageWindowController GetPrivateWindow(uint userId) + { + if (privateWindows.TryGetValue(userId, out PrivateMessageWindowController? existing)) return existing; + var created = new PrivateMessageWindowController(client, userId, selfId, Name(userId)); privateWindows[userId] = created; return created; + } + + private void SelfAudioChanged(object? sender, EventArgs args) + { + bool muted = muteButton.State == NSCellStateValue.On; + bool deafened = deafenButton.State == NSCellStateValue.On; + client.SetSelfAudioState(muted, deafened); + status.StringValue = deafened ? "Deafened" : muted ? "Microphone muted" : "Voice active"; + } + + private async void ToggleScreenAudio(object? sender, EventArgs args) + { + try + { + screenAudioButton.Enabled = false; + if (screenAudioStreamId != 0) { StopScreenAudio(); return; } + if (!joinedVoice) throw new InvalidOperationException("Join voice before sharing screen audio."); + ScreenAudioSelection? selection = await ScreenAudioPicker.ChooseAsync(screenAudioSelection); + if (selection is null) return; + screenAudioSelection = selection; + StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamScreenAudio, "Desktop audio", 2); + screenAudioStreamId = stream.StreamId; + int channelCount = stream.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1; + var capture = new ScreenAudioCapture(channelCount, screenAudioSelection, + (pcm, channels) => { uint id = screenAudioStreamId; if (id != 0) client.Audio.FeedPcm(id, pcm, channels); }); + capture.Failed += exception => NSApplication.SharedApplication.InvokeOnMainThread(() => { status.StringValue = exception.Message; StopScreenAudio(); }); + screenAudio = capture; await capture.StartAsync(); screenAudioButton.Title = "Stop sharing"; + status.StringValue = "Sharing desktop audio"; + } + catch (Exception exception) { StopScreenAudio(); status.StringValue = exception.Message; } + finally { screenAudioButton.Enabled = true; } + } + + private void StopScreenAudio() + { + Interlocked.Exchange(ref screenAudio, null)?.Dispose(); + uint id = screenAudioStreamId; screenAudioStreamId = 0; + TryStopStream(id); + screenAudioButton.Title = "Share audio"; + } + + private void ApplyEngineSettings() + { + client.Audio.InputMode = settings.InputMode; client.Audio.VadThreshold = settings.VadThreshold; + if (settings.InputMode != AudioInputMode.PushToTalk) { client.Audio.PushToTalk = false; pushToTalkEngaged = false; } + client.Audio.InputGain = settings.InputGain; client.Audio.OutputGain = settings.OutputGain; + client.Audio.InputNoiseReduction = settings.InputNoiseReduction; + if (auxiliaryStreamId != 0) client.Audio.SetLocalGain(auxiliaryStreamId, settings.AuxiliaryGain); + } + + internal async Task ApplySettingsAsync() + { + ApplyEngineSettings(); + if (!joinedVoice) return; + bool restart = activeInputDevice != settings.InputDeviceId || activeOutputDevice != settings.OutputDeviceId || activeStereo != settings.StereoMicrophone; + if (restart) { await LeaveVoice(); await JoinVoice(); return; } + if (settings.AuxiliaryEnabled && auxiliaryStreamId == 0) await StartAuxiliaryAsync(); + else if (!settings.AuxiliaryEnabled && auxiliaryStreamId != 0) StopAuxiliary(); + else if (settings.AuxiliaryEnabled && activeAuxiliaryDevice != settings.AuxiliaryDeviceId) { StopAuxiliary(); await StartAuxiliaryAsync(); } + } private void RefreshState() { uint previousTarget = uint.TryParse(messageTarget.SelectedItem?.RepresentedObject?.ToString(), out uint selectedTarget) ? selectedTarget : 0; channels.RemoveAllItems(); - foreach (var channel in client.Channels.OrderBy(c => c.Order).ThenBy(c => c.Name)) + IReadOnlyList<(Channel Channel, int Depth)> hierarchy = FlattenChannels(client.Channels); + foreach ((Channel channel, int depth) in hierarchy) { - channels.AddItem(channel.Name + (channel.PasswordProtected ? " [password]" : "")); + int members = client.Users.Count(user => user.ChannelId == channel.Id); + channels.AddItem(new string(' ', depth * 3) + channel.Name + $" ({members})" + (channel.PasswordProtected ? " [password]" : "")); channels.LastItem!.RepresentedObject = new NSString(channel.Id.ToString()); + channels.LastItem!.ToolTip = channel.Topic; } currentChannel = client.Users.FirstOrDefault(u => u.Id == selfId)?.ChannelId ?? currentChannel; - int selectedIndex = client.Channels.ToList().FindIndex(c => c.Id == currentChannel); if (selectedIndex >= 0) channels.SelectItem(selectedIndex); - users.Value = string.Join("\n", client.Users.Where(u => u.ChannelId == currentChannel).OrderBy(u => u.Nickname).Select(u => (u.Id == selfId ? "You — " : "") + u.Nickname)); + int selectedIndex = hierarchy.ToList().FindIndex(item => item.Channel.Id == currentChannel); if (selectedIndex >= 0) channels.SelectItem(selectedIndex); + users.Value = string.Join("\n", client.Users.Where(u => u.ChannelId == currentChannel).OrderBy(u => u.Nickname).Select(u => + (talkingUsers.Contains(u.Id) ? "Speaking — " : "") + (u.Id == selfId ? "You — " : "") + u.Nickname + + (u.ServerDeafened ? " [server deafened]" : u.ServerMuted ? " [server muted]" : u.SelfDeafened ? " [deafened]" : u.SelfMicMuted ? " [muted]" : ""))); messageTarget.RemoveAllItems(); messageTarget.AddItem("Current channel"); messageTarget.LastItem!.RepresentedObject = new NSString("0"); foreach (var user in client.Users.Where(user => user.Id != selfId).OrderBy(user => user.Nickname)) { @@ -96,6 +250,18 @@ internal sealed class MainWindowController : NSWindowController messageTarget.SelectItem(Math.Max(0, targetIndex)); status.StringValue = $"Connected · {client.Users.Count} users"; } + private static IReadOnlyList<(Channel Channel, int Depth)> FlattenChannels(IReadOnlyList values) + { + var result = new List<(Channel, int)>(); var visited = new HashSet(); + void AddChildren(uint parent, int depth) + { + foreach (Channel channel in values.Where(channel => channel.ParentId == parent).OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) + if (visited.Add(channel.Id)) { result.Add((channel, depth)); AddChildren(channel.Id, depth + 1); } + } + AddChildren(0, 0); + foreach (Channel channel in values.Where(channel => !visited.Contains(channel.Id)).OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) result.Add((channel, 0)); + return result; + } private async void ChangeChannel(object? sender, EventArgs args) { if (changingChannel || !uint.TryParse(channels.SelectedItem?.RepresentedObject?.ToString(), out uint id) || id == currentChannel) return; @@ -196,12 +362,18 @@ internal sealed class MainWindowController : NSWindowController if (!result.Ok) throw new InvalidOperationException(result.Error); try { - playback = audioBackend.OpenPlayback(SelectedDevice(outputDevice)); + ApplyEngineSettings(); + activeInputDevice = settings.InputDeviceId ?? SelectedDevice(inputDevice); + activeOutputDevice = settings.OutputDeviceId ?? SelectedDevice(outputDevice); + activeStereo = settings.StereoMicrophone; + playback = audioBackend.OpenPlayback(activeOutputDevice); client.Audio.MixedPcm += PlayMixedPcm; - StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamMic, "Microphone"); + StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamMic, "Microphone", settings.StereoMicrophone ? 2 : 1); microphoneStreamId = stream.StreamId; - microphone = audioBackend.OpenCapture(SelectedDevice(inputDevice), false, FeedMicrophone); + microphone = audioBackend.OpenCapture(activeInputDevice, false, FeedMicrophone); + if (settings.AuxiliaryEnabled) await StartAuxiliaryAsync(); joinedVoice = true; voice.Title = "Leave voice"; status.StringValue = "Voice connected"; + feedback.Play(SoundEvent.VoiceOn); } catch { @@ -220,6 +392,29 @@ internal sealed class MainWindowController : NSWindowController if (streamId != 0) client.StopStream(streamId); await client.SubscribeVoiceAsync(false); status.StringValue = "Voice disconnected"; + feedback.Play(SoundEvent.VoiceOff); + } + + private async Task StartAuxiliaryAsync() + { + if (auxiliaryStreamId != 0) return; + StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamAuxDevice, "Aux device", 2); + auxiliaryStreamId = stream.StreamId; client.Audio.SetLocalGain(stream.StreamId, settings.AuxiliaryGain); + try { activeAuxiliaryDevice = settings.AuxiliaryDeviceId; auxiliary = audioBackend.OpenCapture(activeAuxiliaryDevice, false, FeedAuxiliary); } + catch { client.StopStream(auxiliaryStreamId); auxiliaryStreamId = 0; throw; } + } + + private void StopAuxiliary() + { + Interlocked.Exchange(ref auxiliary, null)?.Dispose(); + activeAuxiliaryDevice = null; + uint id = auxiliaryStreamId; auxiliaryStreamId = 0; + TryStopStream(id); + } + + private void FeedAuxiliary(ReadOnlySpan pcm, int channels) + { + uint id = auxiliaryStreamId; if (id != 0) client.Audio.FeedPcm(id, pcm, channels); } private void FeedMicrophone(ReadOnlySpan pcm, int channels) @@ -241,11 +436,21 @@ internal sealed class MainWindowController : NSWindowController private void StopAudioDevices() { + StopScreenAudio(); + StopAuxiliary(); microphoneStreamId = 0; Volatile.Write(ref lastRemoteAudioTick, 0); Interlocked.Exchange(ref microphone, null)?.Dispose(); client.Audio.MixedPcm -= PlayMixedPcm; Interlocked.Exchange(ref playback, null)?.Dispose(); + activeInputDevice = activeOutputDevice = null; lastTalking = false; + } + private void TryStopStream(uint id) + { + if (id == 0 || client.State != ClientConnectionState.Connected) return; + try { client.StopStream(id); } + catch (InvalidOperationException) { } + catch (IOException) { } } private string Name(uint id) => client.Users.FirstOrDefault(u => u.Id == id)?.Nickname ?? $"User {id}"; private string FormatMessage(TextMessage text) @@ -263,9 +468,16 @@ internal sealed class MainWindowController : NSWindowController chat.Value = chat.Value.Length == 0 ? line : chat.Value + "\n" + line; chat.ScrollRangeToVisible(new NSRange(chat.Value.Length, 0)); } + private void AppendActivity(string line) => Append($"[{DateTime.Now:t}] — {line}"); + private void Announce(string value) + { + NSString[] keys = [NSAccessibilityNotificationUserInfoKeys.AnnouncementKey, NSAccessibilityNotificationUserInfoKeys.PriorityKey]; + NSObject[] values = [(NSString)value, NSNumber.FromInt32(1)]; + NSAccessibility.PostNotification(status, NSView.AnnouncementRequestedNotification, new NSDictionary(keys, values)); + } protected override void Dispose(bool disposing) { - if (disposing) { timer.Invalidate(); StopAudioDevices(); client.DisposeAsync().AsTask().GetAwaiter().GetResult(); } + if (disposing) { timer.Invalidate(); if (pushToTalkMonitor is not null) NSEvent.RemoveMonitor(pushToTalkMonitor); client.Audio.PushToTalk = false; settingsWindow?.Close(); administrationWindow?.Close(); foreach (PrivateMessageWindowController window in privateWindows.Values) window.Close(); privateWindows.Clear(); StopAudioDevices(); feedback.Dispose(); client.DisposeAsync().AsTask().GetAwaiter().GetResult(); } base.Dispose(disposing); } } diff --git a/clients/apple/dotnet/VoiceCat.Mac/PrivateMessageWindowController.cs b/clients/apple/dotnet/VoiceCat.Mac/PrivateMessageWindowController.cs new file mode 100644 index 0000000..31e70a7 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/PrivateMessageWindowController.cs @@ -0,0 +1,43 @@ +using AppKit; +using CoreGraphics; +using VoiceCat.Core; +using Voicecat.V1; + +namespace VoiceCat.Mac; + +internal sealed class PrivateMessageWindowController : NSWindowController +{ + private readonly VoiceCatClient client; + private readonly uint otherUserId; + private readonly uint selfId; + private readonly NSTextView transcript = new(new CGRect(0, 0, 430, 300)) { Editable = false, Selectable = true }; + private readonly NSTextField compose = new(new CGRect(20, 20, 350, 28)) { PlaceholderString = "Private message" }; + + internal PrivateMessageWindowController(VoiceCatClient client, uint otherUserId, uint selfId, string nickname) : base(new NSWindow( + new CGRect(0, 0, 480, 400), NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable, + NSBackingStore.Buffered, false)) + { + this.client = client; this.otherUserId = otherUserId; this.selfId = selfId; + Window!.Title = $"Private message — {nickname}"; Window.Center(); + NSView view = Window.ContentView!; + var scroll = new NSScrollView(new CGRect(20, 65, 440, 315)) { DocumentView = transcript, HasVerticalScroller = true }; + scroll.AccessibilityLabel = $"Private conversation with {nickname}"; view.AddSubview(scroll); + ((INSAccessibility)compose).AccessibilityLabel = $"Message to {nickname}"; compose.Activated += Send; view.AddSubview(compose); + var button = new NSButton(new CGRect(380, 18, 80, 32)) { Title = "Send" }; button.Activated += Send; view.AddSubview(button); + } + + private void Send(object? sender, EventArgs args) + { + string value = compose.StringValue.Trim(); if (value.Length == 0) return; + client.Send(new() { TextMessage = new() { Scope = TextScope.TextPrivate, TargetId = otherUserId, Body = value, ClientMsgId = Guid.NewGuid().ToString("N") } }); + compose.StringValue = ""; + } + + internal void Append(TextMessage message, string sender) + { + string line = $"[{DateTime.Now:t}] {(message.SenderId == selfId ? "You" : sender)}: {message.Body}"; + transcript.Value = transcript.Value.Length == 0 ? line : transcript.Value + "\n" + line; + transcript.ScrollRangeToVisible(new Foundation.NSRange(transcript.Value.Length, 0)); + Window?.MakeKeyAndOrderFront(this); + } +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioCapture.cs b/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioCapture.cs new file mode 100644 index 0000000..a8d13a3 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioCapture.cs @@ -0,0 +1,156 @@ +using System.Runtime.InteropServices; +using CoreFoundation; +using CoreMedia; +using Foundation; +using ObjCRuntime; +using ScreenCaptureKit; +using VoiceCat.Audio; + +namespace VoiceCat.Mac; + +internal enum ScreenAudioScope { EntireDesktop, OnlyApplications, AllExceptApplications } +internal sealed record ScreenAudioSelection(ScreenAudioScope Scope, IReadOnlySet BundleIdentifiers, bool ExcludeScreenReader) +{ + internal static ScreenAudioSelection Default { get; } = new(ScreenAudioScope.EntireDesktop, new HashSet(), false); +} + +internal sealed class ScreenAudioCapture : NSObject, ISCStreamOutput +{ + private static readonly HashSet ScreenReaders = ["com.apple.VoiceOver", "com.apple.VoiceOver4", "com.apple.speech.speechsynthesisd"]; + private readonly int channels; + private readonly CapturePcmHandler onPcm; + private readonly ScreenAudioSelection selection; + private readonly DispatchQueue queue = new("net.iamtalon.voicecat.screen-audio"); + private readonly CaptureDelegate streamDelegate; + private SCStream? stream; + private int disposed; + + internal ScreenAudioCapture(int channels, ScreenAudioSelection selection, CapturePcmHandler onPcm) + { + this.channels = Math.Clamp(channels, 1, 2); this.selection = selection; this.onPcm = onPcm; + streamDelegate = new(error => Failed?.Invoke(error)); + } + + internal event Action? Failed; + + internal static async Task> GetApplicationsAsync(CancellationToken cancellationToken = default) + { + SCShareableContent content = await GetContentAsync(cancellationToken).ConfigureAwait(false); + return content.Applications.Where(application => !string.IsNullOrWhiteSpace(application.BundleIdentifier) && + !ScreenReaders.Contains(application.BundleIdentifier) && application.BundleIdentifier != NSBundle.MainBundle.BundleIdentifier) + .GroupBy(application => application.BundleIdentifier, StringComparer.Ordinal).Select(group => group.First()) + .Select(application => new ScreenApplication(string.IsNullOrWhiteSpace(application.ApplicationName) ? application.BundleIdentifier : application.ApplicationName, + application.BundleIdentifier)).OrderBy(application => application.Name, StringComparer.CurrentCultureIgnoreCase).ToArray(); + } + + internal async Task StartAsync(CancellationToken cancellationToken = default) + { + SCShareableContent content = await GetContentAsync(cancellationToken).ConfigureAwait(false); + SCDisplay display = content.Displays.FirstOrDefault() ?? throw new InvalidOperationException("No display is available for screen audio capture."); + SCRunningApplication[] matching = content.Applications.Where(application => Included(application.BundleIdentifier)).ToArray(); + SCContentFilterOption option = selection.Scope == ScreenAudioScope.OnlyApplications ? SCContentFilterOption.Include : SCContentFilterOption.Exclude; + var filter = new SCContentFilter(display, matching, [], option); + var configuration = new SCStreamConfiguration + { + CapturesAudio = true, ExcludesCurrentProcessAudio = true, SampleRate = 48_000, + ChannelCount = channels, Width = 2, Height = 2, QueueDepth = 6, + MinimumFrameInterval = new CMTime(1, 1) + }; + var created = new SCStream(filter, configuration, streamDelegate); + if (!created.AddStreamOutput(this, SCStreamOutputType.Audio, queue, out NSError? outputError)) + throw new InvalidOperationException(outputError?.LocalizedDescription ?? "Could not attach the screen-audio output."); + stream = created; + var started = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + created.StartCapture(error => { if (error is null) started.TrySetResult(); else started.TrySetException(new InvalidOperationException(error.LocalizedDescription)); }); + await started.Task.WaitAsync(TimeSpan.FromSeconds(15), cancellationToken).ConfigureAwait(false); + } + + private bool Included(string bundleIdentifier) + { + bool selected = selection.BundleIdentifiers.Contains(bundleIdentifier); + bool reader = selection.ExcludeScreenReader && ScreenReaders.Contains(bundleIdentifier); + return selection.Scope switch + { + ScreenAudioScope.OnlyApplications => selected, + ScreenAudioScope.AllExceptApplications => selected || reader, + _ => reader + }; + } + + [Export("stream:didOutputSampleBuffer:ofType:")] + public unsafe void DidOutputSampleBuffer(SCStream captureStream, CMSampleBuffer sampleBuffer, SCStreamOutputType type) + { + if (type != SCStreamOutputType.Audio || !sampleBuffer.DataIsReady || disposed != 0) return; + int frames = checked((int)sampleBuffer.NumSamples); + if (frames <= 0 || frames > 8192) return; + Span listStorage = stackalloc byte[40]; + fixed (byte* list = listStorage) + { + int status = CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer.Handle, out _, (IntPtr)list, + (nuint)listStorage.Length, IntPtr.Zero, IntPtr.Zero, 0, out IntPtr blockBuffer); + if (status != 0) return; + try + { + uint count = *(uint*)list; + if (count is 0 or > 2) return; + Span converted = stackalloc short[frames * channels]; + NativeAudioBuffer* first = (NativeAudioBuffer*)(list + 8); + bool planar = count > 1; + int sourceChannels = planar ? checked((int)count) : checked((int)Math.Max(1, first->Channels)); + for (int frame = 0; frame < frames; frame++) + { + float left = planar ? ((float*)first[0].Data)[frame] : ((float*)first[0].Data)[frame * sourceChannels]; + float right = sourceChannels > 1 + ? planar ? ((float*)first[1].Data)[frame] : ((float*)first[0].Data)[frame * sourceChannels + 1] + : left; + if (channels == 1) converted[frame] = ToInt16(sourceChannels > 1 ? (left + right) * 0.5f : left); + else { converted[frame * 2] = ToInt16(left); converted[frame * 2 + 1] = ToInt16(right); } + } + onPcm(converted, channels); + } + finally { if (blockBuffer != IntPtr.Zero) CFRelease(blockBuffer); } + } + } + + private static short ToInt16(float value) => (short)Math.Clamp((int)MathF.Round(Math.Clamp(value, -1, 1) * 32767), short.MinValue, short.MaxValue); + + private static Task GetContentAsync(CancellationToken cancellationToken) + { + var completion = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + SCShareableContent.GetShareableContent((content, error) => + { + if (error is not null) completion.TrySetException(new UnauthorizedAccessException(error.LocalizedDescription)); + else if (content is null) completion.TrySetException(new InvalidOperationException("ScreenCaptureKit returned no shareable content.")); + else completion.TrySetResult(content); + }); + cancellationToken.Register(() => completion.TrySetCanceled(cancellationToken)); + return completion.Task; + } + + protected override void Dispose(bool disposing) + { + if (disposing && Interlocked.Exchange(ref disposed, 1) == 0) + { + SCStream? previous = Interlocked.Exchange(ref stream, null); + if (previous is null) { streamDelegate.Dispose(); queue.Dispose(); } + else previous.StopCapture(_ => { previous.Dispose(); streamDelegate.Dispose(); queue.Dispose(); }); + } + base.Dispose(disposing); + } + + [StructLayout(LayoutKind.Sequential)] + private struct NativeAudioBuffer { internal uint Channels, DataByteSize; internal IntPtr Data; } + + private sealed class CaptureDelegate(Action failed) : SCStreamDelegate + { + public override void DidStop(SCStream stream, NSError error) => failed(new IOException(error.LocalizedDescription)); + } + + [DllImport("/System/Library/Frameworks/CoreMedia.framework/CoreMedia")] + private static extern int CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(IntPtr sampleBuffer, + out nuint bufferListSizeNeeded, IntPtr bufferList, nuint bufferListSize, IntPtr structureAllocator, + IntPtr blockAllocator, uint flags, out IntPtr blockBuffer); + + [DllImport("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation")] + private static extern void CFRelease(IntPtr value); +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioPicker.cs b/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioPicker.cs new file mode 100644 index 0000000..6a18050 --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/ScreenAudioPicker.cs @@ -0,0 +1,49 @@ +using AppKit; +using CoreGraphics; +using Foundation; +using ScreenCaptureKit; + +namespace VoiceCat.Mac; + +internal sealed record ScreenApplication(string Name, string BundleIdentifier); + +internal static class ScreenAudioPicker +{ + internal static async Task ChooseAsync(ScreenAudioSelection initial, CancellationToken cancellationToken = default) + { + IReadOnlyList applications = await ScreenAudioCapture.GetApplicationsAsync(cancellationToken); + var mode = new NSPopUpButton(new CGRect(0, 270, 390, 28), false); + mode.AddItems(["Everything", "Only selected applications", "All except selected applications"]); + mode.SelectItem((int)initial.Scope); + ((INSAccessibility)mode).AccessibilityLabel = "Screen audio sharing mode"; + + var source = new ApplicationSource(applications); + var table = new NSTableView(new CGRect(0, 0, 390, 225)) { DataSource = source, Delegate = source, AllowsMultipleSelection = true, HeaderView = null }; + table.AddColumn(new NSTableColumn("application") { Title = "Application", Width = 370 }); + ((INSAccessibility)table).AccessibilityLabel = "Applications to include or exclude"; + uint[] selected = applications.Select((application, index) => initial.BundleIdentifiers.Contains(application.BundleIdentifier) ? (uint?)index : null) + .Where(index => index.HasValue).Select(index => index!.Value).ToArray(); + if (selected.Length > 0) table.SelectRows(NSIndexSet.FromArray(selected), false); + var scroll = new NSScrollView(new CGRect(0, 38, 390, 225)) { DocumentView = table, HasVerticalScroller = true, BorderType = NSBorderType.BezelBorder }; + var exclude = NSButton.CreateCheckbox("Exclude VoiceOver and speech-synthesis audio", () => { }); + exclude.Frame = new CGRect(0, 5, 390, 24); exclude.State = initial.ExcludeScreenReader ? NSCellStateValue.On : NSCellStateValue.Off; + ((INSAccessibility)exclude).AccessibilityLabel = "Exclude screen reader audio"; + var accessory = new NSView(new CGRect(0, 0, 390, 305)); accessory.AddSubview(mode); accessory.AddSubview(scroll); accessory.AddSubview(exclude); + var alert = new NSAlert { MessageText = "Choose screen audio", InformativeText = "Select what other people will hear.", AccessoryView = accessory }; + alert.AddButton("Share"); alert.AddButton("Cancel"); + if (alert.RunModal() != 1000) return null; + var bundleIdentifiers = table.SelectedRows.ToArray().Select(index => applications[checked((int)index)].BundleIdentifier).ToHashSet(StringComparer.Ordinal); + return new((ScreenAudioScope)(int)mode.IndexOfSelectedItem, bundleIdentifiers, exclude.State == NSCellStateValue.On); + } + + private sealed class ApplicationSource(IReadOnlyList applications) : NSTableViewDataSource, INSTableViewDelegate + { + public override nint GetRowCount(NSTableView tableView) => applications.Count; + [Export("tableView:viewForTableColumn:row:")] + public NSView GetViewForItem(NSTableView tableView, NSTableColumn? tableColumn, nint row) + { + var value = NSTextField.CreateLabel(applications[checked((int)row)].Name); + value.LineBreakMode = AppKit.NSLineBreakMode.TruncatingTail; return value; + } + } +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/SettingsWindowController.cs b/clients/apple/dotnet/VoiceCat.Mac/SettingsWindowController.cs new file mode 100644 index 0000000..fe3c75d --- /dev/null +++ b/clients/apple/dotnet/VoiceCat.Mac/SettingsWindowController.cs @@ -0,0 +1,120 @@ +using AppKit; +using CoreGraphics; +using VoiceCat.Audio; + +namespace VoiceCat.Mac; + +internal sealed class SettingsWindowController : NSWindowController +{ + private readonly MainWindowController owner; + private readonly MacSettings settings; + private readonly MacAudioBackend audio; + private readonly NSPopUpButton mode = new(new CGRect(160, 455, 260, 28), false); + private readonly NSSlider vad = new(new CGRect(160, 415, 260, 24)) { MinValue = 0, MaxValue = 100 }; + private readonly NSSlider inputGain = new(new CGRect(160, 375, 260, 24)) { MinValue = 0, MaxValue = 400 }; + private readonly NSButton noiseReduction = NSButton.CreateCheckbox("Noise reduction (RNNoise)", () => { }); + private readonly NSButton stereo = NSButton.CreateCheckbox("Stereo microphone", () => { }); + private readonly NSPopUpButton input = new(new CGRect(160, 285, 260, 28), false); + private readonly NSPopUpButton output = new(new CGRect(160, 245, 260, 28), false); + private readonly NSSlider outputGain = new(new CGRect(160, 205, 260, 24)) { MinValue = 0, MaxValue = 100 }; + private readonly NSButton auxiliary = NSButton.CreateCheckbox("Aux input stream", () => { }); + private readonly NSPopUpButton auxiliaryDevice = new(new CGRect(160, 125, 260, 28), false); + private readonly NSSlider auxiliaryGain = new(new CGRect(160, 85, 260, 24)) { MinValue = 0, MaxValue = 400 }; + private readonly NSButton sounds = NSButton.CreateCheckbox("Event sounds", () => { }); + private readonly NSButton speech = NSButton.CreateCheckbox("Speak events", () => { }); + private readonly NSButton selfTalk = NSButton.CreateCheckbox("Own voice-activity sounds", () => { }); + private readonly NSButton pttSound = NSButton.CreateCheckbox("Push-to-talk cue", () => { }); + private readonly NSButton pttKey = new(new CGRect(220, 12, 200, 32)) { Title = "Change PTT key…" }; + private readonly NSSlider eventVolume = new(new CGRect(160, 505, 260, 24)) { MinValue = 0, MaxValue = 100 }; + + internal SettingsWindowController(MainWindowController owner, MacSettings settings, MacAudioBackend audio) : base( + new NSWindow(new CGRect(0, 0, 460, 620), NSWindowStyle.Titled | NSWindowStyle.Closable, + NSBackingStore.Buffered, false)) + { + this.owner = owner; this.settings = settings; this.audio = audio; + Window!.Title = "VoiceCat Settings"; Window.Center(); + NSView view = Window.ContentView!; + mode.AddItems(["Voice activation", "Push to talk", "Always on"]); + Add(view, "Sound volume", eventVolume, 515); + Add(view, "Input mode", mode, 465); Add(view, "VAD sensitivity", vad, 425); Add(view, "Microphone volume", inputGain, 385); + noiseReduction.Frame = new CGRect(160, 340, 260, 24); stereo.Frame = new CGRect(160, 315, 260, 24); + view.AddSubview(noiseReduction); view.AddSubview(stereo); + Add(view, "Input device", input, 295); Add(view, "Output device", output, 255); Add(view, "Output volume", outputGain, 215); + auxiliary.Frame = new CGRect(160, 165, 260, 24); view.AddSubview(auxiliary); + Add(view, "Aux device", auxiliaryDevice, 135); Add(view, "Aux volume", auxiliaryGain, 95); + sounds.Frame = new CGRect(25, 45, 110, 24); speech.Frame = new CGRect(145, 45, 110, 24); + selfTalk.Frame = new CGRect(265, 45, 180, 24); pttSound.Frame = new CGRect(25, 18, 180, 24); + view.AddSubview(sounds); view.AddSubview(speech); view.AddSubview(selfTalk); view.AddSubview(pttSound); view.AddSubview(pttKey); + foreach (NSControl control in new NSControl[] { mode, vad, inputGain, noiseReduction, stereo, input, output, + outputGain, auxiliary, auxiliaryDevice, auxiliaryGain, sounds, speech, selfTalk, pttSound, eventVolume }) control.Activated += Changed; + pttKey.Activated += CapturePushToTalkKey; + Populate(input, true, settings.InputDeviceId); Populate(output, false, settings.OutputDeviceId); Populate(auxiliaryDevice, true, settings.AuxiliaryDeviceId); + LoadValues(); SetAccessibility(); + } + + private static void Add(NSView view, string label, NSView control, double y) + { + NSTextField text = NSTextField.CreateLabel(label); text.Frame = new CGRect(25, y, 125, 22); view.AddSubview(text); view.AddSubview(control); + } + + private void Populate(NSPopUpButton picker, bool capture, string? selected) + { + picker.RemoveAllItems(); IReadOnlyList devices = audio.Enumerate(capture); + foreach (AudioDeviceInfo device in devices) { picker.AddItem(device.Name + (device.IsDefault ? " (default)" : "")); picker.LastItem!.RepresentedObject = new Foundation.NSString(device.Id); } + int index = devices.ToList().FindIndex(device => device.Id == selected); + if (index < 0) index = devices.ToList().FindIndex(device => device.IsDefault); + if (index >= 0) picker.SelectItem(index); picker.Enabled = devices.Count > 0; + } + + private void LoadValues() + { + mode.SelectItem((int)settings.InputMode); vad.DoubleValue = settings.VadThreshold * 100; + inputGain.DoubleValue = settings.InputGain * 100; outputGain.DoubleValue = settings.OutputGain * 100; + noiseReduction.State = State(settings.InputNoiseReduction); stereo.State = State(settings.StereoMicrophone); + auxiliary.State = State(settings.AuxiliaryEnabled); auxiliaryGain.DoubleValue = settings.AuxiliaryGain * 100; + sounds.State = State(settings.EventSounds); speech.State = State(settings.SpokenEvents); + eventVolume.DoubleValue = settings.EventVolume * 100; + selfTalk.State = State(settings.SelfTalkSounds); pttSound.State = State(settings.PushToTalkSound); + pttKey.Title = $"PTT key code: {settings.PushToTalkKeyCode}"; + UpdateEnabled(); + } + + private void CapturePushToTalkKey(object? sender, EventArgs args) + { + var alert = new NSAlert { MessageText = "Set push-to-talk key", InformativeText = "Press the key to use while VoiceCat is focused." }; + alert.AddButton("Cancel"); NSObject? monitor = null; + monitor = NSEvent.AddLocalMonitorForEventsMatchingMask(NSEventMask.KeyDown, value => + { + settings.PushToTalkKeyCode = value.KeyCode; settings.Save(); pttKey.Title = $"PTT key code: {value.KeyCode}"; + NSApplication.SharedApplication.AbortModal(); return null!; + }); + try { alert.RunModal(); } + finally { if (monitor is not null) NSEvent.RemoveMonitor(monitor); } + } + + private async void Changed(object? sender, EventArgs args) + { + settings.InputMode = (AudioInputMode)mode.IndexOfSelectedItem; settings.VadThreshold = (float)vad.DoubleValue / 100; + settings.InputGain = (float)inputGain.DoubleValue / 100; settings.OutputGain = (float)outputGain.DoubleValue / 100; + settings.InputNoiseReduction = On(noiseReduction); settings.StereoMicrophone = On(stereo); + settings.InputDeviceId = Selected(input); settings.OutputDeviceId = Selected(output); + settings.AuxiliaryEnabled = On(auxiliary); settings.AuxiliaryDeviceId = Selected(auxiliaryDevice); + settings.AuxiliaryGain = (float)auxiliaryGain.DoubleValue / 100; settings.EventSounds = On(sounds); + settings.EventVolume = (float)eventVolume.DoubleValue / 100; settings.SpokenEvents = On(speech); + settings.SelfTalkSounds = On(selfTalk); settings.PushToTalkSound = On(pttSound); + settings.Save(); UpdateEnabled(); await owner.ApplySettingsAsync(); + } + + private void UpdateEnabled() { vad.Enabled = settings.InputMode == AudioInputMode.VoiceActivation; auxiliaryDevice.Enabled = auxiliaryGain.Enabled = settings.AuxiliaryEnabled; } + private void SetAccessibility() + { + ((INSAccessibility)mode).AccessibilityLabel = "Microphone input mode"; ((INSAccessibility)vad).AccessibilityLabel = "Voice activation sensitivity"; + ((INSAccessibility)inputGain).AccessibilityLabel = "Microphone volume"; ((INSAccessibility)input).AccessibilityLabel = "Microphone device"; + ((INSAccessibility)output).AccessibilityLabel = "Output device"; ((INSAccessibility)outputGain).AccessibilityLabel = "Output volume"; + ((INSAccessibility)auxiliaryDevice).AccessibilityLabel = "Auxiliary input device"; ((INSAccessibility)auxiliaryGain).AccessibilityLabel = "Auxiliary input volume"; + ((INSAccessibility)eventVolume).AccessibilityLabel = "Event sound volume"; + } + private static bool On(NSButton button) => button.State == NSCellStateValue.On; + private static NSCellStateValue State(bool value) => value ? NSCellStateValue.On : NSCellStateValue.Off; + private static string? Selected(NSPopUpButton picker) => picker.SelectedItem?.RepresentedObject?.ToString(); +} diff --git a/clients/apple/dotnet/VoiceCat.Mac/VoiceCat.Mac.csproj b/clients/apple/dotnet/VoiceCat.Mac/VoiceCat.Mac.csproj index b5c79fe..5ac6eed 100644 --- a/clients/apple/dotnet/VoiceCat.Mac/VoiceCat.Mac.csproj +++ b/clients/apple/dotnet/VoiceCat.Mac/VoiceCat.Mac.csproj @@ -20,6 +20,7 @@ +