Lets a user transmit a second hardware input device (e.g. line-in / aux)
alongside the mic, with its own device picker and volume, from Audio Settings.
No core/ABI/proto changes: the aux is a VC_STREAM_AUX_DEVICE stream started
with external_feed=1 and fed via vc_stream_feed_pcm (the same external-feed
pipeline screen-audio uses). Per-kind local_streams_ already allows mic +
screen + one aux to coexist; volume is a client-side gain multiply (the core's
vc_set_input_gain is mic-only/global). Aux is always-on (core never gates
AUX_DEVICE on VAD/PTT) and is tied to the voice session.
Windows: new Audio/InputDeviceCapture.cs (WASAPI shared-mode capture from a
real input endpoint + capture-endpoint enumeration); aux section in
AudioSettingsForm.cs; lifecycle in MainForm.cs; persistence in VoiceSettings.cs.
macOS: new Audio/InputDeviceCapture.swift (AVAudioEngine input-node tap pinned
to the chosen Core Audio device + device enumeration by stable UID); aux section
in SettingsWindowController.swift; lifecycle + UserDefaults persistence in
MainWindowController.swift; file registered in project.pbxproj.
Windows verified (C# solution builds clean; aux confirmed working). macOS build
+ E2E pending a Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Let users choose what the SCREEN_AUDIO stream captures before sharing:
share everything, only selected apps, or all except selected apps, plus a
first-class "Exclude screen reader (VoiceOver) audio" toggle.
ScreenCaptureKit filters audio per application, so ScreenAudioCapture now
takes a ScreenAudioSelection and builds the matching SCContentFilter
(including:/excludingApplications:). New ScreenSharePickerSheet lists
running apps from SCShareableContent. iOS left untouched -- ReplayKit only
delivers the mixed system stream, so per-app filtering is impossible there.
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.
- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
(backward-compatible Codable), shown in Guest mode, wired into the guest auth
path -- guests could not set a display name on either before (only Windows)
Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
setVoiceJoinedState / setShareScreenButton updated only the inner NSButton's
title+image, but the text shown beneath a custom-view NSToolbarItem comes from
the item's label, not the button's title -- so the toolbar kept reading
"Join Voice" / "Share Screen Audio" after joining. Hold references to the
NSToolbarItems and update their label/paletteLabel alongside the button.
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
Mirrors the Windows client's UI overhaul (commit 97fa659 + 540ec13) adapted to
Mac-native conventions. The main window is now just toolbar + channels + users
+ chat; audio device settings moved to a modeless Settings window.
- NSToolbar: Join Voice, Share Screen Audio, Mute, Deafen (SF Symbol toggle
buttons) + Output Volume slider (NSSlider 0-100, default 80). Voice actions,
mute/deafen, and output volume moved out of the bottom panel into the toolbar
- Audio device settings (input mode, VAD sensitivity, PTT key, device picker,
level meter) moved to a new SettingsWindowController -- a modeless window
opened via the app menu's "Settings..." (Cmd+,) item. Source-of-truth for
audio state lives in MainWindowController so voice start applies settings even
before the window has been opened; SettingsWindowController reads from /
writes back to those properties and applies changes live when voice is active.
Level meter forwarded from handleLevel -> updateLevel(rms:)
- Unified log: chat NSTextView + activity NSTableView collapsed into a single
NSTextView -- activity events in secondaryLabelColor (gray), chat in default
- Private messaging: scope dropdown removed; compose always sends to the
current channel. Each PM conversation opens in its own modeless
PrivateMessageWindowController. Incoming .textMessage with .private scope
routed to the right window; outgoing PMs echoed by server arrive through the
same path. "Send Private Message..." added to user context menu.
- Messages menu: "New Private Message..." (Cmd+Shift+N) opens a UserPickerSheet
listing all server users so you can PM anyone on the server
- Channel tree now shows live user counts, e.g. "General (3)"; refreshChannelTree
called on .userJoined/.userLeft (was missing)
- Voice menu: Join Voice (Cmd+Shift+V), Share Screen Audio (Cmd+Shift+S),
Mute (Cmd+Shift+M), Deafen (Cmd+Shift+D) -- NSMenuItem key equivalents with
[.command, .shift] mask, dispatched by the responder chain
- setOutputVolume(_:) wrapper added to VoiceCatClient.swift (was missing -- the
C ABI + C# wrapper shipped in commit 97fa659 but the Swift wrapper was never
added); wired end-to-end: toolbar slider -> client.setOutputVolume(gain)
Part A -- fixed and verified the previously-uncompiled Swift from the external
PCM feed/tap commit (615d2a8):
- Rebuilt the macOS xcframework slice (regenerated the module map from current
voicecat.h, exposing vc_pcm_sink_cb / vc_stream_feed_pcm / vc_set_pcm_sink)
- Fixed feedPcm type bug: size_t imports as Int in Swift not UInt; the original
UInt(samplesPerChannel) was wrong
- Added VoiceCatPcmSinkCallback typealias -- a Swift-idiomatic public alias for
the C vc_pcm_sink_cb so consumers (tests, the macOS app) can declare a sink
callback without directly importing the VoiceCatC C module. Mirrors the C#
VcPcmSinkCallback delegate
- keyCodeName helper deduplicated (was in PttKeyCaptureSheet.swift +
MainWindowController.swift -- now shared)
Platform-specific adaptations (vs. Windows): NSToolbar instead of ToolStrip;
global menu bar + NSMenuItem key equivalents (Cmd not Ctrl, responder-chain
dispatched, no custom key monitor needed); PM windows as modeless NSWindows;
picker as Mac sheet; gray = secondaryLabelColor; SF Symbols for toolbar icons.
swift test 10/10 (4 ExternalPcmTests + 6 VoiceCatClientSmokeTests against a
live server); xcodebuild Debug + Release BUILD SUCCEEDED with 0 Swift warnings.
The main window's NSSplitView panels (channel list, user list, chat,
activity log) were collapsing to zero size because:
1. The scroll views inside the split views were missing
translatesAutoresizingMaskIntoConstraints = false, so Auto Layout
couldn't manage their sizes.
2. The inner split views were also missing it.
3. The voice panel had no height constraint, so it expanded to fill
all available space (539px of the 600px window), starving the
outer split view down to 1px tall.
4. The split views had no initial divider positions, so panels
collapsed to zero even when the split view had space.
Add translatesAutoresizingMaskIntoConstraints = false to all four
scroll views and both inner split views, give the voice panel a
96px height constraint, and set initial divider positions after the
window is on screen. The panels now get reasonable space, are
visible on screen, and are reachable by VoiceOver.
Setting accessibilityLabel on the NSScrollView wrappers turned them into
leaf elements, so VoiceOver never descended into the document views
(NSOutlineView/NSTableView/NSTextView) inside. Tab still worked because
the key-view loop is independent of the accessibility tree.
Also removed the redundant setAccessibilityRole calls on the document
views — they already default to those roles, and re-setting the same
role can interfere with the view's custom a11y implementation.
Matches the pattern already used in ConnectWindowController, whose
server table VoiceOver reaches correctly.
The main window controller was created as a local variable in
ConnectWindowController.authSucceeded and never retained — ARC
deallocated it immediately, which destroyed the VoiceCatClient
(connection silently dropped), nil'd every button's weak target
(clicks did nothing), and killed event delivery (channel list,
messages, voice never worked). Symptom: TOFU (in the retained
connect controller) worked, but everything in the main window
was a zombie shell.
Fix: store the MainWindowController in a new field on
ConnectWindowController (which AppDelegate retains for the app's
lifetime). Also add NSLog diagnostics in deinit/bootstrap/handleEvent
so lifecycle and event delivery are observable from the terminal
or Console.app.
The previous "shipped" claim was false — xcodebuild had never been run and
the macOS app source was never committed. This commit adds the 17 Swift
source files + xcodeproj and fixes three real defect classes so Debug and
Release both build clean:
1. MainWindowController.swift compile errors:
- NSAccessibility.post arg order (element:notification:userInfo:)
- NSAccessibilityPriorityMedium -> NSAccessibilityPriorityLevel.medium
- StreamSummary.streamId -> .id (Identifiable conformance)
- drop redundant VoiceCatResult.description extension
2. Linker: add -lc++ to OTHER_LDFLAGS (libvoicecat-fat.a is C++20; pure-Swift
app target has no .cpp sources so libc++ wasn't pulled in — swift test
passed because Package.swift testTarget has linkerSettings: c++).
3. Release config: add ONLY_ACTIVE_ARCH=YES (XCFramework only has arm64).
Verified: clean Debug + Release builds, otool -L shows libc++.1.dylib,
nm shows _vc_client_create/_vc_version_string, app launches and runs.
