Author SHA1 Message Date
Talon 186fe6dbb6 Fix iOS broadcast pump mapping churn
Build and test / test (macos-latest) (push) Waiting to run
Build and test / test (ubuntu-24.04) (push) Waiting to run
Build and test / test (windows-latest) (push) Waiting to run
Build and test / apple-client (push) Waiting to run
The screen-audio pump opened and disposed a memory mapping plus its container
lookup and path strings on every 5 ms tick - about 200 mapping pairs per
second of steady allocation that churned the GC under long calls. The producer
opens the ring with O_CREAT and never replaces it, so the pump now keeps one
mapping across ticks and rebuilds it only when the ring file disappears or a
drain fails.
2026-09-23 21:47:42 +02:00
Talon f348574bc1 Fix iOS background audio pacing
Build and test / test (macos-latest) (push) Waiting to run
Build and test / test (ubuntu-24.04) (push) Waiting to run
Build and test / test (windows-latest) (push) Waiting to run
Build and test / apple-client (push) Waiting to run
Sleep-paced threads stall for hundreds of milliseconds when iOS coalesces a
backgrounded app's wakeups, and the deadline resets that discarded the deficit
left the capture backlog queued until its ring overflowed: regular dropouts
that worsen the longer the app stays backgrounded. Pace both 20 ms hands-offs
from the AVAudioSourceNode render callback instead (mix via Audio.RunCycle
with deviceClockedAudio, capture handoff from the ring), wake the UDP sender
on a queue signal instead of a 1 ms poll, and let stalled consumers drop their
backlog to the buffer target instead of ratcheting it.
2026-09-23 21:37:24 +02:00
Talon 1487f2673b Fix iOS app icon and extension bundle version validation
Declare XSAppIconAssets in the app manifest so actool receives --app-icon
and the bundle carries CFBundleIconName, and always pass both Xcode version
placeholders to the ReplayKit extension so an empty build number cannot
drop CFBundleVersion and fail installd.
2026-09-23 21:36:34 +02:00
Talon ea76d5157a Improve iOS voice stability and user audio controls
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-22 22:07:00 +02:00
Talon 7b0c003ad4 Add adaptive packet loss handling
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-22 16:54:49 +02:00
Talon 7c4708c3de Persist channel DRED settings
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-22 14:11:36 +02:00
Talon 8471eac363 Fix Windows channel tree ordering
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 19:47:14 +02:00
Talon 4aa0aa4fbd Fix iOS stereo microphone capture
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 18:04:00 +02:00
Talon 0372baf589 Fix iOS voice capture and screen sharing
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 14:14:15 +02:00
Talon 35617e9708 Add signed iOS release workflow
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 04:05:40 +02:00
Talon cf808483e0 Fix physical iOS device deployment
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 03:10:25 +02:00
Talon 4fac7af140 Fix macOS publish bundle signing
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 02:53:27 +02:00
Talon 239e8e9013 Fix ad-hoc macOS publishing
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 02:14:24 +02:00
Talon 6de95ddaa1 Regenerate runtime publish locks
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 02:12:03 +02:00
Talon 9be33f357d Fix stereo capture paths 2026-09-21 02:11:33 +02:00
Talon 08e6c5930a Retire legacy implementations and flatten managed layout
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-21 00:11:32 +02:00
Talon dd811a0bb8 Fix audio clock drift and adaptive jitter buffering
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
2026-09-20 16:26:03 +02:00
Talon 3ee0df11a9 Fix managed iOS native media linking
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
2026-09-20 03:04:12 +02:00
Talon 57a63cc1cd Publish Windows and Linux servers together
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
2026-09-20 01:15:52 +02:00
Talon b79cbeff44 Fix managed package lock isolation
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
2026-09-19 22:55:48 +02:00
Talon c9ed832459 Retire legacy sources and verify managed iOS deployment
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
2026-09-19 22:40:48 +02:00
Talon 42e3bbe14c Keep managed iOS audio active in background
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-19 20:09:00 +02:00
Talon 9fc5598e8e Bring managed iOS client to feature parity
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-19 19:33:10 +02:00
Talon c6715028c1 Add managed UIKit iOS client
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-19 15:43:37 +02:00
Talon d0a72176ba Extend managed macOS client toward feature parity
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-19 00:39:04 +02:00
Talon 310c0f09dd Complete managed macOS platform bring-up
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-18 18:51:53 +02:00
Talon ab460ae12b Add private messaging to managed macOS client
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 22:24:52 +02:00
Talon 28f1dbb991 Store macOS account passwords in Keychain
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 22:22:03 +02:00
Talon 5a30018aeb Prompt for protected macOS channels
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 22:16:18 +02:00
Talon 52f7f51e59 Add managed server profiles and account login
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 22:13:34 +02:00
Talon edd7783a5c Add managed macOS Core Audio voice path
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 22:05:40 +02:00
Talon e5ff484cac Start managed macOS AppKit client
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 17:34:16 +02:00
Talon 7be93e81a1 Package managed server for Linux production
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 17:08:17 +02:00
Talon 48c754aed9 Add managed console client and interop scenarios
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 16:55:17 +02:00
Talon 82ad4c2811 Port managed client audio and Windows application
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-16 16:48:06 +02:00
Talon 5a226ba543 Harden managed server deployment and authentication 2026-09-15 23:24:11 +02:00
Talon 653131b876 Add managed channel administration and moderation
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-15 23:11:09 +02:00
Talon 274b85025c Add configurable media-aware managed session reaping 2026-09-15 22:58:16 +02:00
Talon 05eacb3092 Add encrypted managed UDP relay and native voice conformance 2026-09-15 22:53:54 +02:00
Talon 4067bab7c2 Add managed codec DSP and initial control server 2026-09-15 22:51:33 +02:00
Talon 2df79cdd4c Add managed TLS interoperability and persisted credentials
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-15 18:04:20 +02:00
Talon b76181d9fb Start .NET rewrite with wire and media crypto conformance
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s
2026-09-15 17:54:16 +02:00
TalonandClaude Opus 5 c6c003b8a7 docs: add the .NET/C# porting plan
Proposal for replacing the C++ core, C++ server, and Swift macOS/iOS
clients with a single .NET 10 / C# codebase.

Covers the dependency map (8 vcpkg deps + 1 vendored -> 3 native libs),
the real-time-audio design, per-client strategy, a test-porting plan for
all 29 ctest cases, doc-sync work, an 11-phase migration, and a risk
register.

Two findings drive the shape of the plan:

- SslStream has no RFC 5705 keying-material exporter, which the media
  AEAD key derivation depends on (docs/security.md 2). The API is an
  unapproved proposal and SChannel structurally cannot export secrets.
  Recommends BouncyCastle's managed TLS 1.3 stack, which does implement
  the exporter and keeps the wire format byte-compatible with the C++
  implementation -- so the existing tree stays usable as a conformance
  oracle throughout the port. Protocol-v3 in-band media keys documented
  as the fallback.

- The iOS ReplayKit broadcast upload extension stays in Swift: 50 MB
  jetsam cap plus an unsupported extension type in .NET for iOS, and it
  already doesn't link the core. Leaves one Swift file plus the shared
  App Group ring.

Indexed in docs/README.md. Nothing here is implemented yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 03:19:59 +02:00
Talon 4f71b784fe docs: condense implementation comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-23 13:37:05 +02:00
Talon 575e2907d0 Cleanup root cmakelists
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:59:12 +01:00
Talon f687370e47 Cleanup voicecat.cpp impl comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:56:14 +01:00
Talon 552ffceb66 Cleanup session/
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:55:17 +01:00
Talon a4b3838125 Cleanup protocol/ comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:52:46 +01:00
Talon eafa5eb90c Cleanup net/ comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:49:32 +01:00
Talon 158a2df062 Cleanup crypto comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:37:39 +01:00
Talon 826b3bfb86 Cleanup client and worker pool comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 15:29:09 +01:00
Talon cc81d19d02 Cleanup audio engine and codec comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 14:59:26 +01:00
Talon 8844325efa Cleanup audio engine comments
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 13:13:35 +01:00
Talon bd844e4710 Cleanup core header and proto code
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
2026-07-03 12:46:51 +01:00
TalonandClaude Sonnet 5 5c03e5f261 build: bundle vcpkg as a git submodule, pinned to the manifest baseline
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Adds vcpkg as a submodule at vcpkg/, pinned to the exact commit vcpkg.json
already declares as builtin-baseline, so the bundled checkout and the
manifest's resolved port versions can never drift apart.

cmake/voicecat-toolchain.cmake, scripts/common.sh, and
clients/apple/scripts/build-xcframework.sh now resolve vcpkg as:
VCPKG_ROOT env var (external checkout) > bundled submodule. Docs updated
to describe the new one-time setup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 10:42:42 +01:00
TalonandClaude Sonnet 5 bba605401d chore: comment cleanup pass ahead of open-sourcing
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Removes leftover debug scaffolding (stray Console.WriteLine/NSLog traces,
dead nick_buf_ptr, a no-op --print-config flag now implemented for real),
fixes stale/misleading comments (channel passwords are no longer a "future
M5+" feature, a wrong cross-reference, a stale TlsContext::close() mention,
an incomplete BanRecord::subject_type doc, and a smoke test pointing at a
build/m1-dev preset that no longer exists), strips internal M1-M5 milestone
jargon from comments now that the roadmap is done, trims comments that just
restated the following line, and consolidates a few "why" explanations that
were duplicated 2-3 times in the same file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 10:20:18 +01:00
Talon bda37ec27b fix(ios): add AppIcon asset catalog and fix launch screen for TestFlight
Adds Assets.xcassets with a placeholder 1024x1024 AppIcon so Xcode
compiles CFBundleIconName and the required icon sizes into the bundle.
Replaces UILaunchStoryboardName (missing storyboard) with UILaunchScreen
dict, valid for iOS 14+ (min target is iOS 18). Fixes all four App Store
Connect validation errors blocking TestFlight upload.
2026-06-30 13:22:14 +01:00
Talon 9612b0af89 chore(ios): update bundle ID to me.iamtalon.voicecat for App Store
Changes main app bundle ID from cat.voice.VoiceCatiOS, broadcast extension
from cat.voice.VoiceCatiOS.broadcast, and App Group from group.cat.voice.VoiceCat
to match the registered App Store identifier.
2026-06-30 11:48:02 +01:00
TalonandClaude Sonnet 4.6 2c8178fa02 chore: remove skeleton build mode and stub #ifdef scaffolding
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Drop the M0 no-deps skeleton preset and all VOICECAT_HAS_NET/AUDIO/OPUS/NS
guards that it required. Every subsystem is fully implemented; the stub
#else paths were dead code that added noise to every header and source file.

- CMakePresets.json: remove skeleton configure/build/test entries
- CMakeLists.txt (root/core/tests): remove VOICECAT_USE_VCPKG_DEPS option
  and guards; all targets now build unconditionally
- 17 C++ source files: unwrap HAS_* guards, delete stub #else blocks
- apm_processor.cpp: delete ApmPassthrough no-op class; create() always
  returns RnnoiseProcessor
- 18 test files: remove HAS_* guards and stub int main() skip bodies
- docs/building.md: remove skeleton from preset table and prose

VOICECAT_HAS_LOOPBACK (Windows WASAPI loopback platform gate) unchanged.
29/29 ctest green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 11:32:22 +01:00
Talon 9a20953c08 fix(ios): break AirPods-disconnect reinitialize loop on A2DP presets
Recovery from the 99937c9 audio-device-change commit broadened the
route-change recovery set to "everything except categoryChange /
routeConfigurationChange", which added .override. But .override is
fired by our own applyA2dpSpeakerFallback() -> overrideOutputAudioPort,
which recoverAudio() calls on every recovery. On an A2DP preset
(Stereo Mic / Mono Mic), disconnecting AirPods ping-ponged:

  oldDeviceUnavailable -> recoverAudio -> applyA2dpSpeakerFallback
  -> overrideOutputAudioPort(.speaker) -> .override routeChange
  -> recoverAudio -> applyConfiguration (setCategory resets override)
  -> applyA2dpSpeakerFallback -> overrideOutputAudioPort -> .override -> ...

Each iteration also rebuilt the AVAudioEngine via reconfigure() ->
rebuild() -- the audible reinitialize loop + CPU spin. Voice Chat and
Built-in Mic + Speaker were unaffected (applyA2dpSpeakerFallback
early-returns for non-A2DP modes, so no overrideOutputAudioPort call).

Two-part fix (pure Swift iOS-app target; no C ABI / proto / docs changes):
1. AudioSessionManager.handleRouteChange: added .override to the skip
   list alongside .categoryChange / .routeConfigurationChange. .override
   is only ever fired by our own overrideOutputAudioPort call, so
   treating it as a recovery reason is the loop by definition. The
   AVAudioEngineConfigurationChange observer in IOSVoiceProcessingEngine
   remains as the backstop if an override ever actually stops the engine.
2. IOSAudioRouter.applyA2dpSpeakerFallback: made idempotent via a
   lastAppliedOutputOverride tracker that skips the redundant
   overrideOutputAudioPort call when the desired state (.none for
   external output present, .speaker otherwise) already matches. Reset
   to nil at the top of applyConfiguration() (setCategory can reset the
   override) and on a failed call. Defense-in-depth on top of fix 1.

Build: xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj
-scheme VoiceCatiOS -destination 'generic/platform=iOS' build green
(Xcode 26.5 / iOS 18.0).
2026-06-25 16:18:21 +02:00
Talon 99937c9446 feat(ios): auto-reconnect + audio-device-change recovery
Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired
headphones, AirPods) used to leave the iOS client in a dead/zombie state:
the engine went silent, no reconnect was attempted, and a live-session
disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout.

Reconnect (AppState.swift, SessionState.swift):
- Two-layer reconcile. Once SessionState overwrites client.onEvent at auth
  success, AppState.handleConnectEvent no longer sees live-session events.
  Added a weak SessionState.appState; SessionState.handleEvent .disconnected
  calls appState.onLiveSessionDisconnected after the cue -- the single path
  AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect
  snapshots lastSession, stops audio, releases session/VoiceCatClient (io-
  thread join via vc_client_destroy), resets the backoff, and arms
  scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth
  success via existing TOFU_MATCHED auto-confirm + idempotent join_channel).
- NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi
  <-> cellular interface change or path .unsatisfied it calls
  proactiveReconnect: tearing the session down BEFORE the C core notices the
  dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff
  tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature.
  While mid-reconnect a .satisfied path resets the backoff for a fast retry.
- User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect
  and cancel all reconnect state (task + monitor + lastSession + connectedServer).

Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift):
- Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/
  .newDeviceAvailable route-change guard; fires on every externally-initiated
  route change reason except the ones we cause ourselves (.categoryChange/
  .routeConfigurationChange) to avoid a notification loop. Interruption-end
  now always recovers instead of only when .shouldResume is set.
- Added AVAudioEngineConfigurationChange observer on the engine so a system
  self-stop after our route-change handler wins the race is caught.
- IOSAudioEngine.rebuild() does a one-shot reactivation-retry on
  engine.start() failure (iOS sometimes refuses until the session is
  re-reactivated -- the silent-death case).

No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green
via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
2026-06-25 14:57:13 +02:00
Talon 44a336cc89 fix(ios): cast audio complexity UInt32 to Int in ChannelEditView 2026-06-24 16:49:37 +02:00
Talon 6fe7bf0158 feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):

1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
   Previously the button only toggled the local mic — receiving was always on
   (gated by channel membership alone). Added a protocol-level voice subscription
   concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
   proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
   functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
   by the SFU relay recipient filter, and core-client gating of remote-stream
   decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
   on Leave. Text chat works regardless of voice subscription.

2. Channel edit dialog now shows the channel's actual current settings. The read
   struct vc_channel was missing sort_order and audio fields — only the write
   struct vc_channel_info had them. Extended vc_channel with both (additive, no
   ABI break), updated the session model and list_channels marshaling to populate
   them, and updated all three clients' edit callers to use actual channel info
   instead of hardcoded defaults.

3. Channel parameter updates now automatically restart everyone's streams.
   Previously editing a channel's audio config persisted and broadcast a
   ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
   frozen at announce time. handle_channel_event now detects audio-config changes
   on the user's current channel and stop->starts each active local stream. The
   server reads the updated config on re-announce; peers wire up fresh decoders
   at the new ssrc.

All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
TalonandClaude Opus 4.8 2baefddbe4 feat(windows): system-wide push-to-talk via Raw Input
PTT was focus-scoped (WinForms KeyDown/KeyUp) so it died the moment the
window lost focus. Add an optional system-wide path using the Raw Input
API (RegisterRawInputDevices + WM_INPUT with RIDEV_INPUTSINK) instead of
a WH_KEYBOARD_LL low-level hook -- the latter is the keylogger pattern AV
heuristics flag, which is worse for our unsigned MinGW binary. Raw Input
involves no DLL injection or global hook and passes keys through.

- New VoiceCat.App/Native/RawInput.cs: P/Invoke + structs; register the
  keyboard sink, decode WM_INPUT to vkey/up-down, GetAsyncKeyState helper.
- MainForm overrides OnHandleCreated/OnHandleDestroyed/WndProc to manage
  the sink and route WM_INPUT to PTT; gates the focus-scoped KeyDown/KeyUp
  off when system-wide is on; makes the Deactivate force-release
  conditional; adds a GetAsyncKeyState watchdog on the pump timer so a
  missed key-up (RDP/lock-screen) cannot leave PTT stuck.
- VoiceSettings.SystemWidePtt (default ON) + system-wide checkbox in the
  Audio settings PTT section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 13:14:01 +02:00
TalonandClaude Opus 4.8 65736df464 fix(windows): polish hotkeys, user list, titlebar, and PM window
- Suppress the system ding on global hotkeys/PTT and the user-list Enter
  key by setting SuppressKeyPress (Handled alone leaves WM_CHAR to beep).
- Preserve the user-list keyboard selection across talking/mute refreshes
  instead of resetting it on every Items.Clear().
- Include the connected server name in the main window titlebar.
- Close the private-message window on Escape.
- Show the PM window without an owner so focus is no longer trapped to it
  and the main window can be worked in while a PM is open.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 12:30:48 +02:00
TalonandClaude Opus 4.8 8bb2ba933c feat(clients): unify volume boost cap to 4x across all clients
Mic input gain (was 3x) and per-user receive gain (was 2x) had
asymmetric boost ceilings. Raise both, plus the desktop aux input
gain (was 3x), to a uniform 4x (400%) on macOS, iOS, and Windows.

The master Output volume slider is unchanged (still 1x). No core
changes needed: the C ABI only clamps negatives, so the ceilings
live entirely in the client UI sliders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 12:18:10 +02:00
TalonandClaude Opus 4.8 7ef560ba8a fix(windows): statically link MinGW runtime into all binaries
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The static-MinGW recipe (-static-libgcc -static-libstdc++ -static) was
only applied inside the VOICECAT_BUILD_SHARED branch in core/CMakeLists,
so it covered voicecat.dll but not voicecat-server.exe / vccli. With the
x64-mingw-static triplet only vcpkg's own deps are static; the GCC/MinGW
runtime stays dynamic, so the server failed to start on clean Windows
machines with missing libgcc_s_seh-1.dll / libwinpthread-1.dll /
libstdc++-6.dll.

Move the recipe to a global add_link_options (WIN32 AND MINGW) so every
produced binary embeds the runtime. Verified with objdump -p: only
Windows system DLLs remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 01:28:29 +02:00
TalonandClaude Opus 4.8 47124b15a2 fix(windows): list all user-session apps in the app-audio picker
The picker gated its list on visible-window processes plus audio sessions
on only the default render device. That both showed non-audio apps (any
window) and missed real ones (windowless or routed to a secondary device).
Process loopback targets a PID and its child tree regardless of whether the
app is currently playing, so the gate fought the capture layer.

Now enumerate every process in the user's interactive session (windowed or
not), deduped by executable with the windowed tree-root as the capture PID;
scan all active render endpoints to flag currently-playing apps with a > and
sort them first. Adds a filter box and persists checks across filtering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 00:24:20 +02:00
TalonandClaude Sonnet 4.6 04bdb70d47 fix(codec): scale DRED duration to frame size instead of hardcoding 2
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
OPUS_SET_DRED_DURATION was hardcoded to 2 (20 ms), meaning DRED only
covered 1/3 of a lost 60 ms frame and was useless above 20 ms channels.
Now computed as max(2, ceil(frame_ms/10)) so DRED always embeds enough
redundancy to reconstruct one full previous frame regardless of frame size.
The floor of 2 preserves two-frame burst-loss coverage at 10 ms channels.
Decoder side and server are unaffected (server relays payloads verbatim).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 23:44:00 +02:00
TalonandClaude Opus 4.8 b44a200b95 fix(audio): apply receive-side NR to stereo mic streams
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The per-listener noise-reduction toggle (vc_set_remote_stream) did nothing
on Windows/macOS/iOS. The decode loop gated the RNNoise pass on
dec_channels == 1 as a proxy for "this stream is voice" (assuming
stereo => screen-share). The stereo-mic capture commit broke that: a stereo
mic with send-side NR off transmits stereo Opus, so the receiver decoded
two channels and skipped NR entirely. gain/mute have no channel guard, which
is why only NR appeared broken.

Thread the stream kind through init_recv_stream into RemoteStream::is_voice
(set from si.kind() == STREAM_MIC), gate receive NR on is_voice instead of
channel count, and fold a stereo voice frame to mono -> denoise -> duplicate
back across both channels in place (symmetric with the send-side downmix;
RNNoise is mono-only). Screen-audio shares are never denoised.

New test test_recv_noise_reduction drives AudioEngine and asserts a stereo
voice stream's noise floor collapses with NR on (RMS 1046 -> 0.1) while a
screen-audio share stays unchanged. ctest --preset dev green 29/29.
Docs: voice.md section 10. Shared-core fix; clients need only a rebuild.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 21:11:03 +02:00
TalonandClaude Opus 4.8 f72219ddf3 feat(audio): stereo mic capture on Windows & macOS desktop clients
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Both desktop mics were hard-mono: the core defaults capture_channels=1 and
neither client ever called vc_set_capture_channels (only iOS did). Add a
persisted "Stereo microphone" toggle to each client's Audio settings, applied
when the mic stream starts and live via vc_set_capture_channels + vc_audio_restart.
Expose both ABI calls in the Windows interop; the macOS wrapper already had them.

Core fix: encode_and_send_frame now folds a stereo mic frame to mono on a mono
channel - previously the channels==2 branch encoded interleaved L/R directly even
on a mono channel, feeding a mono opus_encode 2x its samples (wrong pitch/garbage).
Real stereo still only reaches the wire on a stereo channel; on a mono channel the
mic is cleanly downmixed.

Test: test_stereo_mic_mono_channel. ctest --preset dev green (28/28). Docs: voice.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 20:48:26 +02:00
Talon b14cf2a4e8 fix(ios): stop core opening a second mic device — dual capture/crackle
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
On iOS the remote end heard the mic twice and crackly (BT headset + internal
mic in Voice Chat; mono + stereo copies of the internal mic in Stereo Mic).

Root cause is an io-thread ordering race. `ensure_audio_running()` only set
`external_capture` when a MIC stream already existed, but it also runs from
`sync_remote_streams` on the post-auth `ServerStateSnapshot` — before the user
joins voice. With `external_playback_` still false and no MIC stream,
`AudioEngine::start()` opened a real miniaudio capture device that stayed open
all session (later calls early-return on running()), racing the AVAudioEngine
input tap fed via `vc_stream_feed_pcm`. `on_capture_frame` then encoded+sent
both paths — the mic transmitted twice, the two unsynchronized capture clocks
producing the crackle.

- core (ensure_audio_running): force `external_capture = true` whenever
  `external_playback_` is set, so iOS unified mode never opens a hardware
  capture device. No-op on desktop.
- ios (AppState): move `setExternalPlayback(true)` before `connect()`, so the
  flag is set before the io thread processes any message — closing the race.

Verified on device: remote end hears the iOS mic once and clean in both Voice
Chat (+ BT) and Stereo Mic.
2026-06-23 17:47:49 +02:00
Talon 19c2fb6ec9 fix(ios): pace mic feed with a prebuffer cushion to stop flutter/crackle
The iOS mic was unusable — a consistent ~40-60ms flutter + volume fade
('slow fan') on every preset. The core sends each captured frame
synchronously (no send pacer), so packet cadence == capture cadence, and
the receiver's playout keeps near-zero buffering by design (its jitter
estimate keys off the regular sender timestamp, so it's blind to arrival
jitter). That's smooth only for a steady sender (desktop miniaudio =
steady 20ms); the iOS AVAudioEngine tap delivers ~2 frames per ~40ms
callback -> bursty -> receiver underruns -> PLC fade.

Fix (iOS-only): the mic tap writes converted 48kHz int16 to an SPSC ring;
a 20ms feed pump drains it and calls feedPcm at a steady cadence. The pump
primes a small prebuffer cushion (3 frames ~60ms, self-healing up to
~120ms on underrun) before releasing, so the tap's bursts can't drain it
to empty. Never reads a partial frame (read consumes what it returns ->
partials were the crackle), and rebuilds with the current channel count
each rebuild() (a frozen count fed mono-as-stereo = octave-up on a
Stereo->Voice Chat switch).

Trade-off: ~60-120ms added mic-send latency, unavoidable when de-bursting
for a near-zero-buffer receiver. PROGRESS.md notes the proper follow-up:
make the jitter buffer measure real RFC-3550 arrival jitter so the
receiver absorbs bursts itself.

Verified: xcodebuild Debug BUILD SUCCEEDED (iOS Simulator, arm64).
2026-06-23 15:40:54 +02:00
Talon cd9c08a47a fix(apple): merge vendored librnnoise.a into xcframework fat static lib
Both VoiceCatMac and VoiceCatiOS failed to link with 'Undefined symbols
for architecture arm64: _rnnoise_create/_rnnoise_destroy/_rnnoise_process_frame'.
Root cause: build-xcframework.sh merged vcpkg deps into the fat static lib but
NOT the locally-built vendored librnnoise.a (a CMake target from
third_party/rnnoise/ linked privately into voicecat via VOICECAT_HAS_NS — not a
vcpkg dep). The xcframework had been rebuilt after the RNNoise commit but still
omitted the symbols, so every slice's libvoicecat-fat.a referenced _rnnoise_*
with no defining object. The iOS slices were also stale (pre-rnnoise) and
absent from the xcframework entirely.

Fix: build-xcframework.sh now also collects .a files from build/<preset>/lib/
(excluding libvoicecat*) so vendored CMake-target static libs like librnnoise.a
get merged in. Future-proof: any new vendored static-lib target landing in
build/<preset>/lib/ is picked up automatically. README 'Fat static library'
section updated.

Verify: rebuilt VoiceCatCore.xcframework --all → all 3 slices (macos-arm64,
ios-arm64, ios-arm64-simulator) now carry the 10 _rnnoise_* symbols; fat lib
~30 MB → ~33 MB. xcodebuild Debug BUILD SUCCEEDED for VoiceCatMac, VoiceCatiOS
(iphonesimulator arm64), and VoiceCatiOS (iphoneos arm64). No core/ABI/proto
changes — xcframework artifact + build script only.
2026-06-23 14:25:30 +02:00
Talon 2e0e0caccb feat(clients): wire RNNoise mic noise reduction into Windows, macOS, and iOS
Expose the existing send-side vc_set_input_noise_reduction C ABI (MIC-only,
mono, LOCAL — denoises captured mic PCM before input gain and VAD/PTT gate)
as a persisted global toggle in each client's audio settings, applied live
and re-applied on Join Voice. Mirrors the existing mic-gain wiring pattern.

- Shared Swift (VoiceCatCore): add setInputNoiseReduction(_:) wrapper
- Windows: P/Invoke + SetInputNoiseReduction wrapper, MicNoiseReduction in
  VoiceSettings, new checkbox in AudioSettingsForm (layout shifted +28px),
  apply on Join Voice; also fix stale 'planned - currently passthrough'
  label on the receive-side per-user NR checkbox (RNNoise now backs it)
- macOS: inputNoiseReduction state + UserDefaults in MainWindowController,
  NR checkbox + nrChanged action in SettingsWindowController
- iOS: inputNoiseReduction in VoiceState + setter + restore in SessionState,
  NR Toggle in SettingsView Voice section

Aux/screen are out of scope by design (core's NR guards kind == MIC). Apple
builds require a rebuilt VoiceCatCore.xcframework with VOICECAT_HAS_NS.
2026-06-23 14:11:18 +02:00
TalonandClaude Opus 4.8 bad9c7533a feat(audio): real noise suppression via vendored RNNoise (send + receive)
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener
vc_set_remote_stream noise_reduction toggle) was wired but inert:
ApmProcessor::create() returned a no-op passthrough, because the
originally-planned webrtc-audio-processing has no working Windows/macOS
build. Drop in RNNoise as the real backend behind the same ApmProcessor
interface, lighting up both NR paths.

- Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port
  is !windows !arm, so it can't cover our primary targets. Shrunk int8
  model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a
  standalone C static lib with no RTCD (portable scalar path on x86,
  auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in
  (rnnoise_create(NULL)); no runtime file.
- New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by
  ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample;
  our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so
  no resampling. RT-safe: allocates at construction, lock-free in the
  capture/playback callbacks.
- Receive-side: lit up via the factory; gated to mono streams (a stereo
  stream is a screen-audio share, not voice).
- Send-side (new): vc_set_input_noise_reduction(client, enable) ABI +
  vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A
  stereo mic is downmixed to mono ONLY when NR is on — with NR off a
  stereo mic keeps full stereo (never collapse mic quality unasked).
- Enable C as a project language for the vendored lib.
- New noise_suppression test: white noise through ApmProcessor::create()
  drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL
  builds clean with vc_set_input_noise_reduction exported, system-only deps.
- Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md,
  vcpkg.json note, PROGRESS.md, CLAUDE.md.

Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 13:30:54 +02:00
TalonandClaude Opus 4.8 7249a8fd30 feat(clients): add aux outgoing stream (mic + second input device) on Windows + macOS
Lets a user transmit a second hardware input device (e.g. line-in / aux)
alongside the mic, with its own device picker and volume, from Audio Settings.

No core/ABI/proto changes: the aux is a VC_STREAM_AUX_DEVICE stream started
with external_feed=1 and fed via vc_stream_feed_pcm (the same external-feed
pipeline screen-audio uses). Per-kind local_streams_ already allows mic +
screen + one aux to coexist; volume is a client-side gain multiply (the core's
vc_set_input_gain is mic-only/global). Aux is always-on (core never gates
AUX_DEVICE on VAD/PTT) and is tied to the voice session.

Windows: new Audio/InputDeviceCapture.cs (WASAPI shared-mode capture from a
real input endpoint + capture-endpoint enumeration); aux section in
AudioSettingsForm.cs; lifecycle in MainForm.cs; persistence in VoiceSettings.cs.

macOS: new Audio/InputDeviceCapture.swift (AVAudioEngine input-node tap pinned
to the chosen Core Audio device + device enumeration by stable UID); aux section
in SettingsWindowController.swift; lifecycle + UserDefaults persistence in
MainWindowController.swift; file registered in project.pbxproj.

Windows verified (C# solution builds clean; aux confirmed working). macOS build
+ E2E pending a Mac.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 12:56:09 +02:00
TalonandClaude Sonnet 4.6 a48b47d4ca feat(windows): move audio settings to dedicated dialog, fix device ComboBox accessibility
Audio input settings (device picker, VAD/PTT mode, VAD sensitivity, mic gain,
PTT key) move from the always-visible bottom panel into Settings > Audio...,
matching the macOS/iOS pattern.

The device ComboBox now uses DataSource + DisplayMember="Name" instead of
Items.Add() with no DisplayMember — this fixes both the display bug (was
showing the full DeviceInfo record ToString()) and the NVDA silence on
dropdown open (DataSource binding exposes proper MSAA text per item).

Changes apply live for immediate feedback; Cancel reverts. VoiceSettings
gains InputDeviceId to persist the chosen device across sessions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 12:15:18 +02:00
Talon 95f1fb70b0 feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
  - iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
  - macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
    (settings window also restores the VAD slider from the stored threshold)
  - Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
    mirrors FeedbackSettings) loaded/applied in MainForm

Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.

Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.

Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).

Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
Talon d30c4ee2f5 fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.

Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.

- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
  playback, conditional mic tap, VPIO/AGC; one rebuild() backing
  startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
  Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
  IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
  AppState wires external playback + listening at connect, stop at
  disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.

No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
TalonandClaude Opus 4.8 7547b8e140 fix(audio): wire in-band FEC into the decoder loss path
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The encoder set OPUS_SET_INBAND_FEC, but the decoder never invoked FEC --
the loss path went DRED -> PLC, so FEC redundancy was emitted (and paid for
in bitrate) yet never consumed.

Wire FEC recovery into AudioEngine::on_playback between DRED and PLC: copy
the next buffered packet once, try DRED, else (if the stream negotiated FEC)
decode(next_pkt, ..., fec=true), else PLC. Recovery priority is now
DRED -> FEC -> PLC. Add per-stream RemoteStream::fec_enabled_, captured from
OpusParams in init_recv_stream. Docs (voice.md) updated to match.

ctest --preset dev: 27/27 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 20:18:29 +02:00
TalonandClaude Opus 4.8 ce2035f271 fix(audio): bound playout depth to stop voice latency ratcheting up
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Latency between speakers grew to multiple seconds and only reset on
rejoining voice. Root cause was the receiver playout logic, not the
codec settings: the playout clock free-ran in real time while the
sender omitted silence from its timestamps (and set no header flags),
and the only correction snapped the clock to the *oldest* buffered
frame — which could only ever add standing latency. target_depth_ms_
was computed but never enforced, so latency could only grow or reset.

Fix: bound playout against the stream's leading edge (newest frame).
(Re)seed to the leading edge on start/marker/starve (no prebuffer, so
latency stays low), and frame-skip catch-up trims any backlog beyond
target+hysteresis — the missing downward force.

Hardening: sender now stamps kFlagMarker (talkspurt start) and kFlagDtx,
consumed on recv for clean resync; adaptive late-drop window; EWMA
outlier rejection so silence gaps/stragglers don't poison the estimate;
duplicate counting and ring-underrun diagnostics.

New test_jitter_depth asserts depth stays bounded (<200ms) while
arrivals outrun playback. ctest --preset dev green (27/27).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 20:02:20 +02:00
TalonandClaude Opus 4.8 e155e342f4 feat(audio): channel sample_rate caps Opus bandwidth (narrowband/wideband)
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The per-channel sample_rate field was inert after pinning the codec to
48 kHz. Make it meaningful without changing the 48 kHz clock: carry it as
OpusParams::max_bandwidth_hz and apply OPUS_SET_MAX_BANDWIDTH in
OpusEncoder::init (8000->narrowband, 16000->wideband, 24000->super-wideband,
48000->full). A low-bitrate room can now shed out-of-band content while
every endpoint keeps a single 48 kHz clock.

Make sample_rate channel-authoritative on the server: conn_session no
longer overrides effective sample_rate with the client's always-48000
request (it now behaves like frame_ms/mode). vc_get_stream_audio_config
reports the channel's configured rate for own streams too.

New ctest channel_samplerate: a 7 kHz tone is attenuated ~1000x on an
8 kHz (narrowband) channel vs a 48 kHz (full-band) channel, proving the cap
is in effect. ctest --preset dev 26/26.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:56:49 +02:00
TalonandClaude Opus 4.8 a460009a2f fix(audio): reframe send path to channel frame_ms; pin codec to 48 kHz
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
The AudioEngine capture clock is fixed at 48 kHz / 20 ms (960-sample
frames), but a channel may set any Opus frame_ms (2.5..60 ms, voice.md
§3) and the server enforces it unclamped. on_capture_frame handed the
engine's 960-sample frame straight to an encoder configured for the
channel's window: frame_ms > 20 was silently ignored, and frame_ms < 20
broke entirely (receiver sized its decode buffer too small ->
OPUS_BUFFER_TOO_SMALL -> dead audio). Affected the hardware mic and
vc_stream_feed_pcm alike.

Reframe each captured/fed block to ls.frame_samples via a per-LocalStream
accumulator (pre-sized at announce, no RT-thread alloc) before
encode_and_send_frame; the 20 ms case stays a zero-copy fast path. Also
pin the codec to 48 kHz in opus_params_from_audio_config — it was honoring
a non-48k effective sample_rate against a 48 kHz PCM clock.

New ctest frame_ms_reframe covers 40 ms (accumulate) and 10 ms (split)
feed->encode->relay->decode->sink round trips. ctest --preset dev 25/25.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 16:45:02 +02:00
TalonandClaude Opus 4.8 50416c33a2 feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.

TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.

Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.

macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).

No core/server code touched; ctest --preset dev unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:31 +02:00
TalonandClaude Opus 4.8 725bd8e925 fix(windows): screen-reader accessibility for context menus and channel tree
- ContextMenuStrip: select first item on Opened so keyboard-invoked menus
  (Shift+F10 / Apps key) raise the UIA focus event immediately instead of
  staying silent until the first arrow key.
- Name the SplitContainer/SplitterPanel containers so screen readers announce
  orientation instead of a stack of anonymous pane nodes.
- Take the resize splitters out of the Tab cycle (TabStop = false) so focus
  moves control-to-control.
- RefreshChannelTree: restore keyboard focus and re-announce the current node
  after a Nodes.Clear()/rebuild, so the tree no longer loses focus when
  channels/users change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 13:34:26 +02:00
TalonandClaude Opus 4.8 483f889910 fix(server): bind UDP media to the TCP port so self-host needs one forward rule
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
media_port defaulted to 0 (OS-assigned) and --port only set the TCP bind_port,
so the UDP relay bound a random high port and advertised it to clients in HELLO.
Self-hosters forwarding only 8384/udp saw connect-OK-but-no-voice, contradicting
docs/deployment.md (control and media share one port). Media now follows
bind_port when media_port is unset; 0=OS-assigned survives when bind_port is also
0 so ephemeral-port tests are unaffected. Banner now reads TCP :8384 UDP :8384.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 12:48:13 +02:00
TalonandClaude Opus 4.8 5e18dfa1c9 fix(windows): real native exclude + self-echo removal for screen audio
"All apps except selected" previously captured the complement of a frozen
app snapshot in INCLUDE mode (missed late-launched apps and system sounds,
wasted captures on silent windows). It now opens a single ProcessLoopbackCapture
in EXCLUDE mode (AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE)
of the one chosen app — true system-mix-minus-one, dynamic so apps launched
after sharing starts are included. The picker enforces single-selection in
exclude mode (the activation params take one target PID).

Adds an "Exclude VoiceCat's own audio (prevents echo)" checkbox (default on,
entire-desktop only) that routes the desktop capture through the same EXCLUDE
path targeting our own process id, killing the whole-device self-echo loop.

No C++/ABI changes. Updates voice.md and PROGRESS.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 12:28:53 +02:00
Talon fb73b694d0 fix(ios): VPIO silent playback + quiet speaker in stereo/studio presets
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Two on-device bugs in the native iOS Voice-Processing path (Swift-only;
no core/ABI change).

1. Voice Chat (VPIO) silent playback: doStartMicStream() called
   audioRestart() BEFORE startStream, so when the engine was already
   running (a remote stream had started it) it reopened with
   external_capture=false and opened a hardware miniaudio capture device.
   The announce-result restart then early-returns (engine already running)
   so that device was never dropped and fought the AVAudioEngine VPIO unit,
   silencing playback. Now: setExternalPlayback first, then startStream
   (stores external_feed synchronously), THEN audioRestart() — the core
   reopens in full external mode (no hardware devices). Added VPIO
   diagnostics: graph/route formats at start, ring written/read totals at
   teardown.

2. Stereo Mic / Studio quiet earpiece: the .builtInMicBtA2dp presets omit
   .defaultToSpeaker (it breaks A2DP) and skip forceSpeaker, so with no
   Bluetooth connected output pinned to the quiet receiver. New
   IOSAudioRouter.applyA2dpSpeakerFallback() overrides to the built-in
   speaker when no external (A2DP/wired/AirPlay) output is present and
   clears the override when one is — called after activation and on
   device-change route changes.
2026-06-22 03:43:00 +02:00
Talon 6c17881cc0 feat(ios): real echo cancellation/NR via native Voice-Processing engine
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.

Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
  AudioEngine opens no hardware playback device; a mixer-timer thread drives
  on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
  sink. start() also skips the hardware capture device when the MIC stream is
  external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).

iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
  setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
  tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
  the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
  presets; SessionState join/leave + reconcileVoicePath() switch paths;
  Voice Chat defaults to speaker; Settings surfaces AEC/NS state.

Known: pending on-device verification; a few bugs to fix afterward.
2026-06-22 02:38:01 +02:00
TalonandClaude Opus 4.8 e806b698ec feat(windows): per-app screen-audio sharing (include/exclude)
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Adds "only selected apps" and "all apps except selected" screen-audio
modes to the Windows client, alongside the existing entire-desktop path.

Per-app capture uses WASAPI process loopback (AUDCLNT_ACTIVATIONTYPE_
PROCESS_LOOPBACK) via ProcessLoopbackCapture, mixed by ProcessAudioMixer
and fed to the core through vc_stream_feed_pcm (external_feed=1 so the
core skips its own loopback device).

Init must pass AUDCLNT_STREAMFLAGS_LOOPBACK | EVENTCALLBACK |
AUTOCONVERTPCM; the LOOPBACK flag is what makes the virtual endpoint
deliver rendered audio (without it every buffer is flagged SILENT) and
AUTOCONVERTPCM resamples the app's native format to 48k s16.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 00:19:38 +02:00
TalonandClaude Sonnet 4.6 4f89d2d32d feat(deploy): Docker + Linux server deployment
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Multi-stage Dockerfile (builder → export → runtime) producing a 149 MB
Ubuntu 24.04 image, verified booting end-to-end on Docker Desktop.  vcpkg
fetched via shallow git fetch at the pinned baseline, release-only overlay
triplets (x64-linux, arm64-linux) to halve intermediate disk usage, and
buildtrees deleted within the RUN layer so they never land in the image or
the BuildKit cache.  Binary cache mount (VCPKG_BINARY_SOURCES) makes
subsequent rebuilds restore pre-built packages instead of recompiling.

Also adds:
- docker-compose.yml for one-command local deploy
- .dockerignore (excludes clients/, build/, .git/)
- .github/workflows/build-linux.yml — CI cross-build for amd64 + arm64
  with downloadable artifacts (primary path for building from Windows)
- scripts/build-linux-binaries.sh — local Docker binary extraction fallback
- deploy/linux/voicecat.service — hardened systemd unit for bare-metal
- cmake/voicecat-toolchain.cmake now auto-wires VCPKG_OVERLAY_TRIPLETS

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 19:51:43 +02:00
Talon 6071c8e238 fix(media): stop permanent voice loss after bad-network blip (protocol v2)
A bad UDP packet on a flaky link could permanently wedge the voice path,
unrecoverable even across app restarts. Three defects:

1. Anti-replay window was advanced from the UNAUTHENTICATED header seq
   before the AEAD tag was checked, and not rolled back on failure. One
   corrupted/forged frame shoved recv_highest_ far ahead, after which every
   legitimate frame was rejected as "too old" forever. Reorder to
   replay-check -> authenticate -> update (RFC 3711 3.3); the window now
   moves only after a successful tag check.

2. The wire seq was only the low 16 bits of the nonce counter (zero-extended
   on receive). After 65,536 frames the nonce desynced and all frames failed
   auth. Widen the voice frame seq u16 -> u64 (header 14 -> 20 bytes). The
   core owns all UDP framing, so Swift/C# clients need only a rebuild. This
   is a versioned wire change: VOICECAT_PROTOCOL_VERSION 1 -> 2, handshake
   rejects on mismatch.

3. Server leaked per-session UDP state on disconnect; unregister_session now
   frees udp_endpoints_/udp_tokens_/ssrc_to_session_.

Also add rate-limited dropped-frame logging to MediaRelay so a wedged media
path is observable. New regression tests in test_media_aead.cpp cover the
poison (fails on old code) and the 16-bit wrap. ctest --preset dev
-E external_pcm: 22/22 pass (external_pcm aborts on a pre-existing CoreAudio
shutdown race, unrelated).
2026-06-21 17:45:28 +02:00
Talon 5be6d8430d feat(ios): user-toggleable speakerphone output
Add a "speaker output" override so users can route audio to the built-in
speaker instead of the earpiece when no headphones/Bluetooth are connected.
Previously the receiver was the only fallback on the default Voice Chat preset.

The toggle inserts .defaultToSpeaker into the AVAudioSession category options
(skipped for the A2DP mode, where it would break Bluetooth routing). It yields
to connected BT/wired output and is orthogonal to preset matching. Exposed both
as a call-bar button in VoiceControlsView and a persisted Settings toggle.
2026-06-21 15:47:20 +02:00
Talon 0c6b1a36cf feat(ios): ad-hoc distribution scripts for pre-TestFlight testing
Add scripts/dist-ios-adhoc.sh and scripts/asc_api.py to build an ad-hoc
signed IPA and the OTA web-install files (manifest.plist + index.html)
for sharing the iOS client with registered devices before TestFlight.

- asc_api.py: App Store Connect API helper (ES256 JWT via cryptography,
  urllib) to register device UDIDs and list registered devices.
- dist-ios-adhoc.sh: registers UDIDs, builds the iOS device xcframework
  slice, archives + exports with method=release-testing using
  -allowProvisioningUpdates, and stages the install files into
  dist/ios-adhoc/.
- Document the workflow in clients/apple/README.md; ignore __pycache__.
2026-06-21 15:28:26 +02:00
Talon c1e6f4f7ff feat(macos): per-app audio selection for screen sharing
Let users choose what the SCREEN_AUDIO stream captures before sharing:
share everything, only selected apps, or all except selected apps, plus a
first-class "Exclude screen reader (VoiceOver) audio" toggle.

ScreenCaptureKit filters audio per application, so ScreenAudioCapture now
takes a ScreenAudioSelection and builds the matching SCContentFilter
(including:/excludingApplications:). New ScreenSharePickerSheet lists
running apps from SCShareableContent. iOS left untouched -- ReplayKit only
delivers the mixed system stream, so per-app filtering is impossible there.
2026-06-21 13:35:01 +02:00
Talon 6b7f06a282 feat(clients): expose all channel codec params + guest nickname everywhere
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.

- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
  C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
  complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
  SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
  (backward-compatible Codable), shown in Guest mode, wired into the guest auth
  path -- guests could not set a display name on either before (only Windows)

Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
2026-06-21 04:03:50 +02:00
Talon b07362e525 feat(ios): iPhone channel drill-down, fix chat compose box, unify chat+activity
- Channels tab is now a drill-down on iPhone: ChannelBrowserView lists top-level
  channels; ChannelDetailView shows the people in a channel, its sub-channels, and
  an explicit Join button (with password prompt). iPad split view unchanged.
- Extract self-contained UserRow (context menu + sheets) from UserListView so admin
  actions are reused in the drill-down.
- Fix off-screen chat compose box: pin VoiceControlsView via per-tab
  .safeAreaInset(edge: .bottom) instead of a floating overlay, so it reserves layout
  space above the tab bar (keeping the compose box visible, cooperating with keyboard
  avoidance) without covering the tab bar buttons.
- Collapse Activity into Chat like macOS/Windows: ChatView renders a merged,
  time-sorted timeline of messages + activity (activity rows in gray); remove the
  Activity tab and ActivityLogView.
- Label the RPSystemBroadcastPickerView inner UIButton for VoiceOver
  ("Share/Stop sharing screen audio") instead of relying on an outer SwiftUI label.
2026-06-21 03:02:58 +02:00
Talon c5ad080692 fix(macos): toggle Join Voice / Share Screen toolbar item labels
setVoiceJoinedState / setShareScreenButton updated only the inner NSButton's
title+image, but the text shown beneath a custom-view NSToolbarItem comes from
the item's label, not the button's title -- so the toolbar kept reading
"Join Voice" / "Share Screen Audio" after joining. Hold references to the
NSToolbarItems and update their label/paletteLabel alongside the button.
2026-06-21 02:09:08 +02:00
Talon ef88af14c2 fix(ios): show screen-audio broadcast extension in picker
The broadcast upload extension never appeared in the iOS broadcast picker
(or Control Center screen-recording list) because its Info.plist set
RPBroadcastProcessMode to the invalid string "BroadcastUpload". iOS only
recognises "RPBroadcastProcessModeSampleBuffer" for a sample-buffer upload
extension and silently rejects any other value, hiding the extension.

Also sign VoiceCatCore.xcframework in build-xcframework.sh: Xcode 15+ rejects
unsigned binary XCFrameworks consumed as SwiftPM binary targets. Signs with
CODESIGN_IDENTITY, else the first Apple Development identity, else ad-hoc.
2026-06-21 02:08:57 +02:00
Talon 8c90e250f0 feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.

macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
  (excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
  mono/stereo mode, and calls feedPcm. Capture starts on the self
  .streamStarted event (effective config known then). Wired into
  MainWindowController.screenAudioClicked().

iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
  .audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
  writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
  not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
  Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
  to mono when the channel is mono. Screen audio appears as a second stream
  of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
  View in VoiceControlsView. Removes the speculative BroadcastCredentials.

Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
Talon 6ab78fa792 feat(macos): UI overhaul -- toolbar, unified log, PM windows, settings window, hotkeys
Mirrors the Windows client's UI overhaul (commit 97fa659 + 540ec13) adapted to
Mac-native conventions. The main window is now just toolbar + channels + users
+ chat; audio device settings moved to a modeless Settings window.

- NSToolbar: Join Voice, Share Screen Audio, Mute, Deafen (SF Symbol toggle
  buttons) + Output Volume slider (NSSlider 0-100, default 80). Voice actions,
  mute/deafen, and output volume moved out of the bottom panel into the toolbar
- Audio device settings (input mode, VAD sensitivity, PTT key, device picker,
  level meter) moved to a new SettingsWindowController -- a modeless window
  opened via the app menu's "Settings..." (Cmd+,) item. Source-of-truth for
  audio state lives in MainWindowController so voice start applies settings even
  before the window has been opened; SettingsWindowController reads from /
  writes back to those properties and applies changes live when voice is active.
  Level meter forwarded from handleLevel -> updateLevel(rms:)
- Unified log: chat NSTextView + activity NSTableView collapsed into a single
  NSTextView -- activity events in secondaryLabelColor (gray), chat in default
- Private messaging: scope dropdown removed; compose always sends to the
  current channel. Each PM conversation opens in its own modeless
  PrivateMessageWindowController. Incoming .textMessage with .private scope
  routed to the right window; outgoing PMs echoed by server arrive through the
  same path. "Send Private Message..." added to user context menu.
- Messages menu: "New Private Message..." (Cmd+Shift+N) opens a UserPickerSheet
  listing all server users so you can PM anyone on the server
- Channel tree now shows live user counts, e.g. "General (3)"; refreshChannelTree
  called on .userJoined/.userLeft (was missing)
- Voice menu: Join Voice (Cmd+Shift+V), Share Screen Audio (Cmd+Shift+S),
  Mute (Cmd+Shift+M), Deafen (Cmd+Shift+D) -- NSMenuItem key equivalents with
  [.command, .shift] mask, dispatched by the responder chain
- setOutputVolume(_:) wrapper added to VoiceCatClient.swift (was missing -- the
  C ABI + C# wrapper shipped in commit 97fa659 but the Swift wrapper was never
  added); wired end-to-end: toolbar slider -> client.setOutputVolume(gain)

Part A -- fixed and verified the previously-uncompiled Swift from the external
PCM feed/tap commit (615d2a8):
- Rebuilt the macOS xcframework slice (regenerated the module map from current
  voicecat.h, exposing vc_pcm_sink_cb / vc_stream_feed_pcm / vc_set_pcm_sink)
- Fixed feedPcm type bug: size_t imports as Int in Swift not UInt; the original
  UInt(samplesPerChannel) was wrong
- Added VoiceCatPcmSinkCallback typealias -- a Swift-idiomatic public alias for
  the C vc_pcm_sink_cb so consumers (tests, the macOS app) can declare a sink
  callback without directly importing the VoiceCatC C module. Mirrors the C#
  VcPcmSinkCallback delegate
- keyCodeName helper deduplicated (was in PttKeyCaptureSheet.swift +
  MainWindowController.swift -- now shared)

Platform-specific adaptations (vs. Windows): NSToolbar instead of ToolStrip;
global menu bar + NSMenuItem key equivalents (Cmd not Ctrl, responder-chain
dispatched, no custom key monitor needed); PM windows as modeless NSWindows;
picker as Mac sheet; gray = secondaryLabelColor; SF Symbols for toolbar icons.

swift test 10/10 (4 ExternalPcmTests + 6 VoiceCatClientSmokeTests against a
live server); xcodebuild Debug + Release BUILD SUCCEEDED with 0 Swift warnings.
2026-06-20 23:30:52 +02:00
TalonandClaude Sonnet 4.6 615d2a8e5f feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink)
Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public,
stereo-capable production API and adds a symmetric PCM tap on the
receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots,
soundboards, and custom clients — all without a hardware audio device.

Core C++:
- voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef,
  vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias
- audio_engine: stereo-aware inject_capture (channels param + ring
  reset on channel-count change); atomic pcm_sink_ fired per decoded
  frame in on_playback; RemoteStream carries user_id/stream_id for
  RT-safe sink metadata; init_recv_stream takes user_id+stream_id
- client.cpp: stream_feed_pcm / set_pcm_sink implementations;
  sync_remote_streams passes user_id/stream_id to init_recv_stream
- voicecat.cpp: trampolines + channels=1/2 validation

Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip,
stereo feed L≠R, sink metadata+disable). ctest 23/23.

Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest
smoke tests (ExternalPcmTests.swift).

C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs
(vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate,
vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs).

Docs: architecture.md §4 new subsection, voice.md §9 updated
(macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit
no-protocol-change note, roadmap.md M5 entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
TalonandClaude Sonnet 4.6 540ec13a63 feat(windows): add Ctrl+Shift hotkeys for voice, screen share, mute, deafen
Ctrl+Shift+V — join/leave voice
Ctrl+Shift+S — screen share toggle
Ctrl+Shift+M — mute/unmute mic
Ctrl+Shift+D — deafen/undeafen

Focus-scoped (same as PTT). Shortcuts display in the Voice menu for
the two menu-backed actions. Hotkeys are suppressed when a text box
has focus.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 15:16:17 +02:00
TalonandClaude Sonnet 4.6 97fa659422 feat(windows): UI overhaul -- toolbar, unified log, PM windows, channel counts, output volume
- Voice actions (Join Voice, Share Screen Audio) moved to a ToolStrip toolbar and
  a new Voice menu in the menu bar; removed from the bottom voice panel
- Activity log and chat log collapsed into a single RichTextBox (rtbLog); activity
  events appear in gray, chat messages in default color
- Private messaging reworked: each conversation opens in its own modeless
  PrivateMessageForm instead of sharing the main chat log via a scope dropdown;
  cboScope removed; main compose bar always sends to the current channel
- New "Messages -> New Private Message..." menu item (Ctrl+P) opens a UserPickerDialog
  listing all connected server users (not just the current channel) so you can PM
  anyone on the server
- Channel tree now shows live user counts, e.g. "General (3)" -- counts sourced from
  the existing _users dictionary which already tracks all server users with channel IDs
- Global output volume slider (TrackBar, 0-100, default 80) added to the right panel;
  wired to new vc_set_output_volume C ABI function that applies a master gain multiplier
  in the audio engine playback callback after mixing all streams
- vc_set_output_volume added end-to-end: voicecat.h, audio_engine.h/.cpp,
  client.h/.cpp, voicecat.cpp, NativeMethods.cs, VoiceCatClient.cs
- Documented Windows PowerShell ctest requirement in AGENTS.md and CLAUDE.md:
  MinGW binaries exit 0xc0000139 in Git Bash; always run ctest/.exe via PowerShell

22/22 ctest green (PowerShell); dotnet build 0 warnings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 14:24:54 +02:00
TalonandClaude Sonnet 4.6 fdcd8d1427 feat: DRED (Deep REDundancy) per-channel toggle
Adds Opus 1.6 DRED support end-to-end: encoder embeds 20 ms of ML
redundancy in every packet when enabled; decoder recovers lost frames
from the next buffered packet's DRED extension rather than falling back
to PLC comfort noise.

Protocol: bool dred = 11 added to AudioConfig (backward-compatible,
defaults false). C ABI: int dred added to vc_audio_config. Encoder:
OPUS_SET_DRED_DURATION(2) when dred=true. Decoder: OpusDREDDecoder +
per-stream OpusDRED scratch pre-allocated off the RT thread;
JitterBuffer::try_copy_front_payload peeks at the next packet without
popping on every PLC step; opus_decoder_dred_decode reconstructs the
lost frame if DRED data is present, otherwise falls back to PLC.

New test: test_dred_toggle (22/22 ctest green).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 13:40:47 +02:00
TalonandClaude Sonnet 4.6 de2c253199 docs: condense PROGRESS.md — trim completed history, keep pending items
1398 → ~340 lines. All "Done" journal entries replaced by a compact
"Recent completed work" summary; M0–M4 sections condensed to scan-able
paragraphs. Pending items (iOS on-device verification, external PCM
feed/tap API plan) kept at full length. M5 task list updated to mark
macOS/iOS UI done and list remaining open items.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 13:14:39 +02:00
Talon f3c1172a3e docs(ios): drop stale debug comments + TeamTalk refs from audio router
After the stereo-mic/A2DP debugging settled, the iOS audio code carried
leftover TeamTalk5 comparison notes, source-line citations, TEMP DIAGNOSTIC
markers, and "this was the bug" narratives that no longer help. Reworded
those to state the current rules; kept the comments that document real
constraints (the setPreferredInputNumberOfChannels(2) trap, the re-entrancy
guard, the ma_context no-session-management config).

Comment-only — no behavior change. core builds, ctest --preset dev 21/21.
2026-06-20 03:03:34 +02:00
TalonandClaude Opus 4.8 f1e1ef59ed fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death)
The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.

Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.

Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.

Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:34:07 +02:00
TalonandClaude Sonnet 4.6 dcb7e6eeca fix(ios): fix stereo mic + A2DP output silence
Three coordinated fixes for the bug where enabling stereo mic capture
caused all audio output (A2DP, speaker, wired) to go silent:

1. audio_engine.cpp — open playback before capture
   On iOS, starting the stereo capture AudioUnit can trigger an audio
   route reconfiguration that drops A2DP before the playback device has
   a chance to claim the route. Opening and starting the playback device
   first commits the output route (A2DP), so iOS is less likely to drop
   it when stereo capture activates afterward.

2. client.cpp — decouple set_capture_channels from engine restart
   Previously vc_set_capture_channels() stopped and restarted the engine
   immediately, which opened capture first (old ordering) and raced
   against the settling AVAudioSession route. Now it only stores the
   channel count; the caller (Swift via vc_audio_restart) controls when
   the engine restarts, after the route has settled.

3. IOSAudioRouter.swift — call audioRestart() after channel config
   selectCaptureChannels() and applyPreset() now call audioRestart()
   after applyConfiguration() + setCaptureChannels(). This is the
   vc_audio_restart() path that was added to the ABI in fdcc84f but
   never wired up in the Swift layer. The restart sees the stored
   channel count and reopens devices in the correct order (playback
   first, capture second).

The doStartMicStream path is unaffected: setCaptureChannels is called
before the server acknowledges the stream (engine not yet running), so
ensure_audio_running() picks up capture_channels=2 directly when the
stream is confirmed and opens with the right count from the start.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 17:39:23 +02:00
Talon fdcc84fb42 fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:

1. configureStereoCapture now calls setPreferredInput +
   setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
   Previously omitted based on incorrect diagnosis that
   setPreferredInput collapsed A2DP — the real culprit was
   setPreferredInputNumberOfChannels(2), which neither project uses.

2. New C ABI: vc_audio_restart (full stop + re-init, unlike
   suspend/resume which only stop/start). Swift wrapper added.
   The withAudioSuspend wrapper that used it was removed after
   on-device testing showed it killed all audio (including
   VoiceOver) when switching presets — the core's
   set_capture_channels handles engine restart internally.

3. Bluetooth options: Voice Chat preset now includes BOTH
   .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
   UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.

4. Capture channels now reset when switching stereo→mono via
   selectCaptureChannels/applyPreset. AudioSessionManager tracks
   activeMicStreamId (set by SessionState on join/leave voice).

5. Docs synced: voice.md, tech-stack.md, architecture.md,
   PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
   references.

Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
2026-06-19 16:58:21 +02:00
Talon 1a1c8a1dfe feat(ios): rework audio presets — always-available + device-specific
Presets reorganized to give users choice at every level:

Always available (work with any output route):
- Voice Chat: AEC/AGC/HPF on, mono, system picks best route (BT HFP,
  wired, or speaker). The standard iOS VoIP experience.
- Stereo Mic: Stereo built-in mic (front+back capsules), A2DP output
  if BT connected else speaker/wired. Standard processing.
- Studio (No Processing): Stereo mic, no AEC/AGC/HPF (raw mode).
  Maximum fidelity. Echo risk on speaker.

When Bluetooth connected:
- Bluetooth Headset (HFP): BT mic + BT output, AEC on, mono.
- BT Headphones + Mono Mic: A2DP output + built-in mic, mono, no AEC.
- BT Headphones + Stereo Mic: A2DP output + stereo built-in mic.

When wired headset/earpods connected:
- Wired Headset: Wired output + wired/built-in mic, AEC on, mono.

Always:
- Custom: shown when advanced settings don't match any preset.

Key changes from previous version:
- Stereo Mic is no longer gated behind Bluetooth — it's always available
  and uses A2DP output if BT is connected, else speaker/wired.
- Wired headset detection (headphones/headsetMic/usbAudio port types)
  with a dedicated preset.
- Voice Chat preset always available with AEC — the safe default.
- activePreset checks device-specific presets first so e.g. when BT is
  connected and settings match 'Bluetooth Headset', it returns that
  instead of the equivalent 'Voice Chat'.
- detectAudioDevices() replaces detectBluetooth(), detects both BT and
  wired devices from currentRoute + availableInputs.
2026-06-19 14:09:08 +02:00
Talon d6352627e9 feat(ios): audio presets + advanced settings disclosure
Replaces the flat list of audio settings with a preset picker that shows
context-appropriate options based on whether a Bluetooth device is connected.

Presets:
- Default (Phone Speaker): built-in mic + speaker, standard, mono
- Bluetooth Headset (HFP): BT mic + BT output, standard, mono — only shown
  when a BT device is connected
- BT Headphones + Phone Mic: A2DP stereo output + built-in mic, standard,
  mono — only shown when BT connected
- BT Headphones + Stereo Mic: A2DP stereo output + built-in mic stereo
  (front+back capsules), standard, stereo — only shown when BT connected
- Custom: shown when advanced settings don't match any preset

When no Bluetooth device is connected, only 'Default' and 'Custom' appear,
with a hint to connect Bluetooth headphones for more options.

All granular controls (input port, orientation, polar pattern, mic mode,
channels, bluetooth mode, output route, AirPlay) are now under an
'Advanced Audio' disclosure group, collapsed by default.

IOSAudioRouter gains:
- AudioPreset enum with bluetoothMode/captureChannels/micMode/usesBuiltInMic
- hasBluetoothDevice detection (checks currentRoute + availableInputs for
  bluetoothA2DP/bluetoothHFP port types)
- availablePresets (filtered by BT connection state)
- activePreset (computed from current settings)
- applyPreset() (sets all individual settings + finds built-in mic port UID)
2026-06-19 14:05:19 +02:00
Talon 3e80af2f3f fix(ios): output muted with A2DP, session lifecycle, mic input issues
Three bugs causing no audio output and no mic input:

1. .voiceChat mode + A2DP = output muted. The .voiceChat mode uses hardware
   AEC/AGC/HPF but requires HFP-compatible routes. A2DP is NOT HFP — iOS
   mutes the output because it can't set up the voice processing pipeline on
   an A2DP route. Fix: use .default mode for Standard+A2DP (no hardware AEC,
   but audio routes correctly). .voiceChat kept for HFP and speaker modes.
   Added info warning in Settings UI for A2DP no-AEC.

2. Session lifecycle broken. stopMicStream() called deactivateAfterStreaming()
   which deactivated the AVAudioSession — but the AudioEngine keeps running for
   remote audio playback, so leaving voice killed all remote audio. And the
   session was never activated when a remote user started talking (only on
   Join Voice), so you couldn't hear anyone before joining voice. Fix:
   - ensureSessionActive() replaces activateForStreaming() — idempotent, called
     on Join Voice AND on .streamStarted (remote user starts talking).
   - stopMicStream() no longer deactivates the session.
   - deactivateSession() called only on disconnect from server.
   - isSessionActive flag tracks state, updated by interruption handler.

3. setPreferredInputNumberOfChannels(1) called for mono — unnecessary (1 is
   the default) and may put the session in a bad state on some devices. Fix:
   only call it when stereo is explicitly selected. Also handle empty input
   port ID (selecting 'Default' in the picker) correctly.

Added comprehensive route logging — after activation, logs the current output
and input route names so issues can be diagnosed from Console.app.
2026-06-19 13:46:20 +02:00
Talon ab973940df fix(ios): break audio session route-change feedback loop
handleRouteChange called applyConfiguration() unconditionally, which called
setCategory/setPreferredInput/etc., which triggered another route-change
notification, which called applyConfiguration() again — an infinite loop that
burned CPU (phone slowdown) and repeatedly tore down/rebuilt the audio session
(audio cycling on/off, VoiceOver glitching).

Two fixes:
1. handleRouteChange now only re-applies config on external device changes
   (.oldDeviceUnavailable / .newDeviceAvailable), not on .categoryChange /
   .routeConfigurationChange which are triggered by our own setCategory calls.
2. IOSAudioRouter.applyConfiguration() gained a re-entrancy guard
   (isApplyingConfiguration) for synchronous route-change notifications.

Also added os.Logger logging to both files (subsystem cat.voice.VoiceCatiOS)
so future issues can be debugged from Console.app on the Mac.
2026-06-19 13:26:23 +02:00
Talon 9fc51cffc4 feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:

1. Channel-id sync bug (mic button permanently dimmed): SessionState never
   synced currentChannelId from the self user's channelId on connect, so the
   mic button (gated on currentChannelId == 0) stayed dimmed. Added
   syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
   called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
   Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.

2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
   text button (parity with macOS). Mute/deafen disable when not in voice.

3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
   port selection, built-in mic orientation/polar patterns, Bluetooth
   HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
   UserDefaults persistence. AudioSessionManager delegates to it.

4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
   lets the core open the mic device in stereo (2-ch interleaved). LocalStream
   gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
   capture_accum_ + on_capture updated to channel-aware accumulation. Test
   test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
   VoiceCatClient.setCaptureChannels.

5. Settings UI rework: AVAudioSession-derived input/output tree replaces
   miniaudio device picker.

6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
   swift-tools-version 6.0 with swiftLanguageModes .v5.

Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.

Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
Talon a10a18aebe docs(progress): plan external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) 2026-06-19 03:51:14 +02:00
Talon cd530db024 docs(progress): plan iOS audio overhaul + Join/Leave Voice + channel-id sync fix 2026-06-19 03:39:12 +02:00
Talon e2616b60b4 fix(scripts): run-ios-simulator.sh exits silently when no Booted sim found
When no simulator is booted, grep -oE finds no UUID and exits 1. With
set -euo pipefail, that non-zero exit propagates through the command
substitution and kills the script at the UDID= assignment before it can
fall through to the boot-one branch.

Fix: add || true to both find_sim and sim_name pipelines so they always
return 0 regardless of whether a match was found.

Also replace sleep 2 with xcrun simctl bootstatus <UDID> -b, which blocks
until the simulator is fully booted — more reliable than a fixed delay
and faster when the simulator starts quickly.

Update docs/building.md §9 to show the bootstatus command.
2026-06-19 02:33:14 +02:00
Talon bf37fe8f0f docs+scripts: add iOS build and simulator run commands
scripts/build-ios-client.sh: builds VoiceCatiOS.app for iOS simulator
  - calls build-xcframework.sh --all (all 3 slices required for iOS sim slice)
  - xcodebuild -target VoiceCatiOS with SYMROOT=OBJROOT to co-locate SPM
    and app build products (fixes "unable to resolve module dependency" error)
  - stages result to dist/ios-client/

scripts/run-ios-simulator.sh: installs and launches on an iPhone simulator
  - finds a booted simulator or boots the first available iPhone sim
  - opens Simulator.app, installs via simctl install, launches via simctl launch
  - --build flag to build first; --log to stream app logs; --device / --udid to
    pin a specific simulator

scripts/build-all.sh: add --ios-client opt-in flag (macOS only)

docs/building.md:
  - add iOS entry to quick-nav table
  - update §6 (apple platform): remove "scaffolding" caveat now that iOS slices
    are validated; trim to essentials + pointer to build-xcframework.sh
  - add §9 (iOS client SwiftUI): XCFramework, xcodebuild command with rationale
    for -target/-SYMROOT flags, run script usage, full simctl commands, Xcode UI
2026-06-19 02:19:38 +02:00
Talon e26e7db5b1 feat(ios): ship iOS SwiftUI client (VoiceCatiOS)
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
  AudioSessionManager (AVAudioSession owner + interruption/route handling),
  ServerListStore/SavedServer (App Group container + Keychain sharing),
  and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)

C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.

XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.

Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
2026-06-19 02:10:25 +02:00
Talon dbf732ca91 Build WinExe so windows client does not show console 2026-06-18 21:29:47 +02:00
Talon 06a68b441a build(scripts): add per-artifact build scripts staging into dist/
One script per main artifact, all staging into dist/ (overridable via
--dist or VOICECAT_DIST_DIR):

- build-server.sh        server-release preset -> dist/server/
- build-windows-client.sh  windows-client DLL + dotnet publish -> dist/windows-client/
- build-macos-client.sh    XCFramework + xcodebuild VoiceCatMac.app -> dist/macos-client/
- build-libs.sh           libvoicecat per platform -> dist/lib/{macos,ios*,windows}/
                         + dist/lib/VoiceCatCore.xcframework/
- build-all.sh            orchestrator (host-applicable artifacts only)
- common.sh               shared helpers: dist/VCPKG_ROOT resolution, platform
                         guards, logging, --help from header comments

Host-platform guards prevent running the wrong script on the wrong OS.
build-libs.sh and build-macos-client.sh drive the existing validated
clients/apple/scripts/build-xcframework.sh rather than duplicating it.
dist/ added to .gitignore.
2026-06-18 18:52:54 +02:00
Talon 56a6e4fab5 docs: add Windows + macOS client build commands to building.md
Add §7 (Windows client: dotnet build) and §8 (macOS client: XCFramework
+ xcodebuild) with the full build/run commands for each client platform.
Also add a quick-navigation table at the top of the doc.
2026-06-18 17:36:47 +02:00
Talon 75c2782860 fix(macos): fix split view layout so panels are visible and VoiceOver-reachable
The main window's NSSplitView panels (channel list, user list, chat,
activity log) were collapsing to zero size because:

1. The scroll views inside the split views were missing
   translatesAutoresizingMaskIntoConstraints = false, so Auto Layout
   couldn't manage their sizes.
2. The inner split views were also missing it.
3. The voice panel had no height constraint, so it expanded to fill
   all available space (539px of the 600px window), starving the
   outer split view down to 1px tall.
4. The split views had no initial divider positions, so panels
   collapsed to zero even when the split view had space.

Add translatesAutoresizingMaskIntoConstraints = false to all four
scroll views and both inner split views, give the voice panel a
96px height constraint, and set initial divider positions after the
window is on screen. The panels now get reasonable space, are
visible on screen, and are reachable by VoiceOver.
2026-06-18 17:35:14 +02:00
Talon c684824b10 fix(macos): make channel/user/chat/activity views reachable by VoiceOver
Setting accessibilityLabel on the NSScrollView wrappers turned them into
leaf elements, so VoiceOver never descended into the document views
(NSOutlineView/NSTableView/NSTextView) inside. Tab still worked because
the key-view loop is independent of the accessibility tree.

Also removed the redundant setAccessibilityRole calls on the document
views — they already default to those roles, and re-setting the same
role can interfere with the view's custom a11y implementation.

Matches the pattern already used in ConnectWindowController, whose
server table VoiceOver reaches correctly.
2026-06-18 16:08:34 +02:00
Talon 69cd7d80ad fix(macos): retain MainWindowController so the client stays alive
The main window controller was created as a local variable in
ConnectWindowController.authSucceeded and never retained — ARC
deallocated it immediately, which destroyed the VoiceCatClient
(connection silently dropped), nil'd every button's weak target
(clicks did nothing), and killed event delivery (channel list,
messages, voice never worked). Symptom: TOFU (in the retained
connect controller) worked, but everything in the main window
was a zombie shell.

Fix: store the MainWindowController in a new field on
ConnectWindowController (which AppDelegate retains for the app's
lifetime). Also add NSLog diagnostics in deinit/bootstrap/handleEvent
so lifecycle and event delivery are observable from the terminal
or Console.app.
2026-06-18 15:52:13 +02:00
Talon 33169b01fa feat(macos): VoiceCatMac AppKit client + fix xcodebuild
The previous "shipped" claim was false — xcodebuild had never been run and
the macOS app source was never committed. This commit adds the 17 Swift
source files + xcodeproj and fixes three real defect classes so Debug and
Release both build clean:

1. MainWindowController.swift compile errors:
   - NSAccessibility.post arg order (element:notification:userInfo:)
   - NSAccessibilityPriorityMedium -> NSAccessibilityPriorityLevel.medium
   - StreamSummary.streamId -> .id (Identifiable conformance)
   - drop redundant VoiceCatResult.description extension
2. Linker: add -lc++ to OTHER_LDFLAGS (libvoicecat-fat.a is C++20; pure-Swift
   app target has no .cpp sources so libc++ wasn't pulled in — swift test
   passed because Package.swift testTarget has linkerSettings: c++).
3. Release config: add ONLY_ACTIVE_ARCH=YES (XCFramework only has arm64).

Verified: clean Debug + Release builds, otool -L shows libc++.1.dylib,
nm shows _vc_client_create/_vc_version_string, app launches and runs.
2026-06-18 15:26:47 +02:00
Talon b4766d2f24 feat(apple): VoiceCatCore Swift package + XCFramework build for macOS/iOS clients
Lays the groundwork for the macOS (AppKit) and iOS (SwiftUI) clients with a shared
Swift core wrapping the C ABI, mirroring the proven Windows VoiceCat.Interop layer.

Architecture decision: macOS UI = AppKit (not SwiftUI) for the most mature VoiceOver
accessibility story — same rationale as the Windows client's WinForms-over-WinUI-3
decision. iOS stays SwiftUI. Recorded in docs/roadmap.md §2.

Build infrastructure (Phase 0):
- clients/apple/scripts/build-xcframework.sh: runs cmake --preset apple-dev, merges
  libvoicecat.a + 107 vcpkg static deps into a single ~30 MB fat static library
  (libvoicecat-fat.a) via libtool -static (SPM binary targets link one .a per slice),
  stages voicecat.h + a generated module.modulemap (module VoiceCatC) into the headers,
  runs xcodebuild -create-xcframework -> clients/apple/VoiceCatCore.xcframework.

VoiceCatCore Swift Package (Phase 1):
- Package.swift: binary target (VoiceCatCoreXCF) + library (VoiceCatCore) + test target.
- Sources/VoiceCatCore/: 7 files mirroring the C# VoiceCat.Interop patterns adapted to
  Swift native C interop — Enums (9 Swift mirrors of C enums, UInt32-backed), Config,
  Event (copies ev.text to String inside the callback — the #1 lifetime rule), Models
  (10 Swift value types), Marshaling (C arrays -> Swift + immediate vc_free_*),
  Callbacks (@convention(c) + Unmanaged.passUnretained, the Swift analog of C#'s
  [UnmanagedCallersOnly] + GCHandle), VoiceCatClient (owns vc_client* as OpaquePointer,
  all 38 C ABI functions, deinit -> vc_client_destroy then frees config CStrings, event
  delivery on main queue via coalesced DispatchQueue.main drain).

Tests — 6/6 green (swift test against a real voicecat-server):
- testConnectTofuAuthListChannelsRoundTrips, testAdminChannelCrudAccountCrudRoundTrips,
  testScreenAudioStreamStartsAndStops, testPerStreamRecvControlsRoundTrip, plus two
  static smoke tests. Catches Swift-specific interop bugs (@convention(c) callback
  lifetime, Unmanaged pointer resolution, CString memory management, enum raw-value
  bridging, struct layout) that C++ ctest cannot. C++ suite still 21/21 green.

Docs updated (house rule): tech-stack.md §2, architecture.md §4, roadmap.md M4 + §2,
clients/apple/README.md (full rewrite), PROGRESS.md, .gitignore.
2026-06-18 14:20:38 +02:00
Talon b2af1a3001 feat(macos): validate dev + apple-dev presets on macOS, fix 3 cross-platform bugs
macOS port groundwork — core, server, tools, and tests now build and run on
macOS 26.5 / Apple Silicon. ctest --preset dev green 21/21 (2 consecutive runs).
apple-dev produces valid arm64 libvoicecat.a + XCFramework for the Swift Package.

Three real cross-platform bugs found and fixed (all latent on Windows/Linux):

1. test_m2_voice.cpp POSIX branch missing <netdb.h> — Linux glibc transitively
   includes it, macOS doesn't. Would fail on any strict POSIX system.

2. SIGPIPE killing processes on macOS — writing to a closed TCP socket raises
   SIGPIPE by default (doesn't exist on Windows, benign on Linux). Fixed by
   ignoring SIGPIPE in both core client init and server startup (POSIX-only,
   #ifndef _WIN32). Production fix, not just tests.

3. Use-after-free of Asio's kqueue reactor on server shutdown — the
   deterministic test_tofu_flow segfault. TcpServerConn's tls_read_loop runs on
   a blocking-I/O thread; when Server::run() returned, io_context was destroyed
   while those threads were still running. On macOS kqueue the reactor pointer
   is null'd immediately -> segfault in socket.close(). Latent on Windows IOCP
   and Linux epoll. Fix: TcpAcceptor now tracks connections; new shutdown()
   closes all + joins threads before io is destroyed; Server::stop() now closes
   acceptor + media_relay too (was just io.stop()).

Verified: dev + apple-dev presets build green, 21/21 tests pass, server starts
+ two vccli text chat over TLS (M1 on Mac), vccli --voice starts MIC stream via
CoreAudio (M2 protocol-level), vccli --list-devices enumerates CoreAudio
devices, xcodebuild -create-xcframework produces valid VoiceCatCore.xcframework.

No ABI or proto changes. Docs updated: building.md, clients/apple/README.md,
PROGRESS.md, CLAUDE.md status line.
2026-06-18 13:24:42 +02:00
Talon bcb7ae8ccb build(cmake): clean up presets, add release/apple presets, cross-platform triplets
Rationalize the preset set to match the project's actual state (past M5):
- Rename dev->skeleton (no-deps stub smoke), m1-dev->dev (default dev preset)
- Drop m2-dev (cache-identical to m1-dev)
- Add release preset (optimized + tests on, symbols kept)
- Strip server-release binaries (-s linker flag)
- Add apple-dev/apple-ios/apple-ios-sim scaffolding presets for XCFramework

Add cmake/voicecat-toolchain.cmake wrapper that auto-resolves the vcpkg
triplet from the host platform (x64-mingw-static/x64-linux/arm64-osx) so
the main presets work on Windows/Linux/macOS without per-OS variants.

Update all docs (building.md, CLAUDE.md, README.md, AGENTS.md, deployment.md,
tech-stack.md, client READMEs) and stale preset-name references in code
comments. No C++ behavior changes — the core was already portable.
2026-06-18 03:16:01 +02:00
Talon d397731db9 feat(audio): per-stream mix controls in Windows client + vc_get_remote_stream getter
PerUserTuningDialog previously broadcast one gain/mute/NR set to *all* of a
user's streams, even though the core mixer (AudioEngine::RemoteStream) and
the C ABI (vc_set_remote_stream) were already per-stream. The UI had no
per-mix controls anywhere.

Reworks the dialog to enumerate ListUserStreams on open and render one row
per stream (kind + label + Gain + Mute + NR), each wiring only to its own
stream_id. Adds a read-back ABI counterpart, vc_get_remote_stream, so the
dialog opens at the listener's actual current per-stream settings (defaults
1.0/unmuted/NR-off) rather than always 100%. Additive ABI change only; no
existing symbols touched.

Tests: test_m3_multistream extended with getter round-trip assertions; new
C# smoke test exercises the full P/Invoke marshaling path with two clients.
Docs: voice.md §10 notes the getter. NR checkbox keeps its honest
'passthrough' label (NS DSP still unbuilt per §8).
2026-06-18 02:06:44 +02:00
Talon 1dfe3c95ed chore: gitignore voicecat_tofu_pins.txt test artifact
The TOFU pin store is written by vc_client during headless tests (the
relative fallback path in core/src/core/client.cpp when tofu_store_path
is unset). Ignore it so it doesn't show up as untracked after test runs.
2026-06-18 01:22:11 +02:00
Talon 487a561963 fix(net): broadcast LEFT on disconnect, add keepalive/reaper, cap PLC hiss
Three reported bugs traced to one root cause plus two missing designed features:

1. Stale users + eternal PLC hiss (root cause): ConnSession::close() silently
   erased dropped users without broadcasting UserEvent::LEFT, so peers never
   learned the user left and their audio engines never called remove_stream —
   Opus PLC synthesized comfort noise forever. Fix: broadcast_left() helper
   + close() broadcasts LEFT before erasing.

2. PLC cap (defense-in-depth): on_playback now caps pure PLC at ~2s, then
   emits digital silence so a stale stream can never hiss forever even if
   remove_stream is skipped. Resets automatically on fresh packets.

3. No timeout / no ping: client never sent Ping, server had no last_seen /
   reaper, so half-open connections (NAT timeout, wifi loss, sleep) left
   ghost users forever. Fix: client Ping every 15s with RTT measurement,
   ConnSession::last_seen bumped on every inbound TCP/UDP frame, steady_timer
   reaper sweeps every 15s and drops sessions older than 45s (configurable
   via server::Config).

4. UDP KEEPALIVE: client sends plaintext kFrameKeepalive every 5s; server
   bumps last_seen + echoes back. Keeps NAT bindings alive and lets media
   activity defer the reaper independently of TCP.

5. Graceful client disconnect: vc_disconnect() sends Disconnect{code=0} via
   a flag-based io-thread exit (no double-close race); server handles
   client-sent Disconnect with immediate close() + LEFT broadcast.

3 new tests: disconnect_left, plc_cap, reaper_timeout. 21/21 ctest green.

Docs: protocol.md §6/§7, voice.md §6, architecture.md §5, PROGRESS.md.
2026-06-18 01:18:33 +02:00
Talon cccf085a87 fix(audio): stereo screen-audio loopback capture on Windows
start_loopback_capture hardcoded channels=1, forcing miniaudio to downmix the system's stereo mix to mono before the encoder saw it -- on_capture_frame then upmixed L=R to produce fake stereo. Now the loopback device opens in the channel's mode (stereo when the channel is stereo), CaptureCallback carries an explicit channels param so the encoder gets real interleaved L/R, and a mono fallback covers unusual render endpoints. New test_loopback_stereo_capture asserts L!=R end-to-end; 18/18 ctest green.
2026-06-17 23:27:59 +02:00
Talon a88656f2fa feat(windows): wire screen-audio sharing into the WinForms client
The core already supported SCREEN_AUDIO capture on Windows (post-M3 WASAPI
loopback via VOICECAT_HAS_LOOPBACK) and the C# Interop layer was complete
(VcStreamKind.ScreenAudio, StartStream/StopStream/SetRemoteStream). Only the
UI was missing -- no core, proto, or C ABI changes needed.

Adds a 'Share Screen Audio' toggle to the voice panel, independent of mic
voice (can share without joining voice). Disconnect/teardown now stops the
screen stream cleanly. New smoke test exercises the full StartStream ->
StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
2026-06-17 22:47:28 +02:00
TalonandClaude Opus 4.8 2185d9d15c fix(media): rewrite relay re-seal seq so multi-user audio decrypts
The media AEAD nonce is an implicit per-direction monotonic counter;
open() reconstructs it from the 14-byte header seq field (the AAD), so
the contract is header.seq == the counter seal() used. The SFU relay
decrypted inbound frames with the sender key, re-sealed with the
recipient send_crypto (its own counter), but forwarded the sender
header verbatim -- so seq carried the wrong counter and the recipient
rebuilt the wrong nonce, silently dropping every relayed frame. It only
worked for a single first-ever sender into a fresh recipient, which is
why reverse/3rd-party audio failed.

Rewrite the outgoing header seq to the recipient peek_send_counter()
before re-sealing so each server->client direction is one contiguous
monotonic counter and the nonce always matches. Safe: the jitter buffer
orders by timestamp, not seq. No wire-format/proto/ABI change.

Adds test_relay_interleaved_reseal regression coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 21:38:27 +02:00
TalonandClaude Opus 4.8 118ca5129f fix(protocol): deliver self-initiated state changes to the actor too
A connected Windows client would randomly snap from its joined channel
back to Lobby. Root cause was a state-sync inconsistency, not a drop:
the server delivered self-initiated state changes (channel join/leave,
stream announce/stop) only as a private *Result to the actor and
broadcast the authoritative UserEvent::UPDATED to everyone else. The
core never applied the result to its SessionModel, so vc_list_users()
kept self in the old channel; the Windows HandleUserUpdated rebuilds
_currentChannelId from vc_list_users() on any user's UPDATED event, so
the next unrelated event surfaced the stale self-channel.

Fix, per the response-vs-broadcast contract now documented in
docs/protocol.md §6: the *Result is pure ack/correlation/actor-private
payload; the resulting state change is broadcast to every client
INCLUDING the actor, and clients apply it to their local model rather
than re-deriving own state from a *Result.

- server: join/leave/stream announce+stop broadcast with exclude=0
- server: text fan-out includes the sender (channel + private echo)
- core: response handlers no longer mutate session_model_
- windows: drop optimistic text echo; render own message via the relay
- docs/protocol.md §6: document the response-vs-broadcast contract

Registry-level admin broadcasts (move/mute/kick/channel CRUD) already
used exclude=0 and were correct. ctest build/m1-dev 18/18 green;
VoiceCat.App builds 0 warnings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 20:48:50 +02:00
Talon 9b321d0d4f M5: Windows client moderation UI; add C ABI getters for account list, user mute/deafen, channel topic 2026-06-17 16:31:29 +02:00
Talon 3990f63f0f M5: moderation, permissions, channel CRUD, in-app account management
- Server-side moderation & permissions (kick/ban/move/server-mute, channel CRUD).

- Database schema v2: channels, bans; BLAKE2b channel passwords, Argon2id accounts.

- C ABI additions and client-side handling (vc_kick_user, vc_ban_user, vc_set_permission, vc_set_server_mute, vc_move_user, vc_create/edit/delete_channel, vc_create/reset/delete/list_account).

- vccli flags for all M5 operations plus --username/--password auth.

- Four new tests covering permissions, kick/ban/move/mute, admin accounts, channel CRUD.

- Docs: protocol.md envelope updates, security.md channel-password hashing, PROGRESS.md.
2026-06-17 15:08:05 +02:00
TalonandClaude Opus 4.8 a2f159e971 fix(audio): seed/re-sync playout clock so VAD/PTT gaps don't silence playback
RemoteStream::playout_ts was seeded to 0 and only advanced inside the
decode loop (including on every PLC iteration), so it free-ran at ~1x
wall-clock regardless of whether the sender was transmitting. The
sender's frame timestamps only advance while it actually sends (the
VAD/PTT gate returns before ls.timestamp += samples). Across a late join
or any VAD/PTT silence gap the two clocks diverged without bound; once
past the jitter buffer's 500 ms late-drop window every real frame was
dropped-as-late (clock ahead) or never-due (clock behind) -> permanent
silence, while the talk indicator (driven by push_recv_frame, independent
of the jitter buffer) stayed lit.

Add JitterBuffer::peek_front_ts() (try-lock, RT-safe) and seed/re-sync
playout_ts to the earliest buffered frame on the first frame and whenever
it has drifted past +/-200/500 ms. This seeds startup and recovers after
every silence gap.

New regression test test_playout_resync free-runs the clock ~2 s past the
drop window, pushes a ts=0 frame, and asserts audible output: fails
(energy=0) without the fix, passes with it. ctest --preset m1-dev: 14/14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:24:39 +02:00
TalonandClaude Sonnet 4.6 7da0a02b3a fix(audio): buffer capture frames to Opus encoder's fixed frame size
AudioEngine::on_capture() was passing miniaudio's hardware callback period
(commonly 480 samples / 10 ms on WASAPI shared mode) directly to opus_encode(),
which requires exactly frame_samples_ (960 for 20 ms @ 48 kHz). The mismatch
returned OPUS_BAD_ARG and silently dropped every real mic frame, while screen
share and injected test frames happened to be correctly sized and worked fine.

Fix: accumulate PCM in a pre-allocated CaptureAccum buffer (mirroring the
existing RemoteStream::ring fix on the playback side) and only call capture_cb_
when a full frame_samples_ chunk is ready. Same pattern applied to on_loopback().

Add test_capture_frame_accumulation() to verify the accumulator fires exactly the
right number of callbacks for misaligned chunk sizes (480, 240+720, 1920 samples).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 01:51:50 +02:00
TalonandClaude Sonnet 4.6 45d87bde67 docs(M4): expand Windows client README with build guide and known limitations
Replaces the M4-placeholder stub with a real build/run guide covering prerequisites,
build order (server → DLL → dotnet build), DLL dependency verification, manual test
runbook, and the known limitations (focus-scoped PTT, NR passthrough, no admin UI,
TOFU-pins-leaf-cert vs Ed25519).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:52:57 +02:00
Talon 2f643e4293 Update docs 2026-06-17 00:52:02 +02:00
TalonandClaude Sonnet 4.6 63b241cc2e feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
  for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
  until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
  it atomically so the audio RT path reads without a lock

C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
  password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
  after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)

Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
  Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
  Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
  ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
  per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
  Activity log ListBox as durable screen-reader record, AutomationNotification for
  curated live announcements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
TalonandClaude Sonnet 4.6 5be869c61a fix(audio): decouple Opus decode cadence from playback callback period
on_playback() was passing miniaudio's hardware playback-callback frame
count to opus_decode()'s max_samples, instead of the decoder's fixed
frame size (960 samples @ 20ms/48kHz). Since real packets decode to
more samples than the (often smaller, e.g. ~480 on default low-latency
WASAPI) hardware period, opus_decode returned OPUS_BUFFER_TOO_SMALL on
nearly every callback -- packets were received/decrypted/jitter-buffered
correctly but never decoded into audible PCM. Result: control-plane
events and VAD worked, but zero audio in headphones.

mix_for_test()'s white-box test masked this since it always called
on_playback with frames == frame_samples, the one case where the bug
is invisible.

Fix: RemoteStream gained a small ring buffer (init_ring/push_ring/
pop_ring) that decouples decode cadence from playback-callback cadence.
on_playback now tops the ring up by decoding whole Opus frames (always
decoder.frame_samples(), never the hardware frame count) and drains
exactly what the callback asks for, silence-padding (PLC) on underrun.

Side effect: also fixes playout_ts, which was advancing by the wrong
unit (hardware frames instead of decoded samples) -- it now tracks
correctly against jitter-buffer timestamps.

ctest --test-dir build/m1-dev: 12/12 green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 17:39:49 +02:00
Talon 845f995826 docs: add build/manual-testing guide, fix stale M0 stub claims in headers
- docs/building.md: explains what each CMake preset (dev, m1-dev, m2-dev,
  server-release) is actually for, and how to build voicecat-server + vccli
  for manual testing. Linked from CLAUDE.md's doc index.
- core/include/voicecat.h, core/src/voicecat.cpp, core/src/protocol/protocol.h,
  server/src/main.cpp: doc-header comments still claimed M0-skeleton/stub
  behavior (VC_ERR_NOT_IMPLEMENTED everywhere, "prints what it would do",
  protobuf codegen "commented") that M1-M3 made real. Updated to describe
  current behavior, with the dev-preset stub fallback noted explicitly where
  it still applies.
2026-06-16 16:30:07 +02:00
TalonandClaude Sonnet 4.6 5f6c223526 feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:

- Device enumeration (vc_list_devices) + input device selection
  (vc_set_input_device), backed by AudioEngine::enumerate_devices() via
  miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
  ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
  webrtc-audio-processing (the originally-planned APM) has no working
  Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
  abseil-cpp dependency), so VAD is a new lightweight, dependency-free
  energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
  interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
  stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
  stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
  miniaudio's loopback device type), replacing test-only injection as the
  production capture path.

Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).

Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.

Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.

Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
369 changed files with 404455 additions and 11235 deletions
-15
View File
@@ -1,15 +0,0 @@
# VoiceCat C++ style. Keep formatting boring and consistent.
BasedOnStyle: Google
Language: Cpp
Standard: c++20
ColumnLimit: 100
IndentWidth: 4
TabWidth: 4
UseTab: Never
AccessModifierOffset: -2
PointerAlignment: Left
DerivePointerAlignment: false
AllowShortFunctionsOnASingleLine: Inline
AllowShortIfStatementsOnASingleLine: false
SortIncludes: CaseInsensitive
IncludeBlocks: Regroup
+26
View File
@@ -0,0 +1,26 @@
# Git history — large and never needed inside the build context
.git/
# Previous build outputs
build/
artifacts/
**/bin/
**/obj/
# Native GUI client code (Swift/Xcode, C#/WinForms) — server build doesn't need these
clients/
# Documentation and prose — not compiled
docs/
*.md
AGENTS.md
PROGRESS.md
CLAUDE.md
# Editor / tooling config
.vscode/
.idea/
# OS noise
.DS_Store
Thumbs.db
+7
View File
@@ -0,0 +1,7 @@
root = true
[*.cs]
indent_style = space
indent_size = 4
csharp_style_namespace_declarations = file_scoped:warning
dotnet_sort_system_directives_first = true
+52
View File
@@ -0,0 +1,52 @@
name: Build and test
on:
push:
paths: ['src/**', 'tests/**', 'clients/**', 'native/**', 'proto/**', 'assets/**', 'scripts/**', '*.slnx', '*.props', '*.targets', 'global.json', 'Dockerfile', '.github/workflows/dotnet.yml']
pull_request:
paths: ['src/**', 'tests/**', 'clients/**', 'native/**', 'proto/**', 'assets/**', 'scripts/**', '*.slnx', '*.props', '*.targets', 'global.json', 'Dockerfile', '.github/workflows/dotnet.yml']
workflow_dispatch:
jobs:
test:
strategy:
fail-fast: false
matrix:
os: [windows-latest, ubuntu-24.04, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
global-json-file: global.json
cache: true
cache-dependency-path: '**/packages.lock.json'
- name: Build and stage native codec/DSP
shell: pwsh
run: ./scripts/build-native.ps1
- run: dotnet restore VoiceCat.slnx --locked-mode
- run: dotnet build VoiceCat.slnx -c Release --no-restore
- run: dotnet test VoiceCat.slnx -c Release --no-build
- shell: pwsh
run: ./scripts/check-licenses.ps1
apple-client:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
global-json-file: global.json
cache: true
cache-dependency-path: '**/packages.lock.json'
- name: Install Apple workloads
run: dotnet workload install macos ios --version 10.0.401
- name: Build and stage native codec/DSP
shell: pwsh
run: ./scripts/build-native.ps1
- name: Build and stage static iOS codec/DSP
run: ./scripts/build-native-ios.sh
- name: Restore managed Apple clients
run: dotnet restore clients/apple/VoiceCat.Apple.slnx
- name: Build managed AppKit and UIKit clients
run: dotnet build clients/apple/VoiceCat.Apple.slnx -c Debug --no-restore
+19 -4
View File
@@ -1,6 +1,13 @@
# Build output # Build output
**/bin/
**/obj/
**/TestResults/
/artifacts/
/build/ /build/
/out/ /out/
# Staged distribution artifacts (scripts/build-*.sh output)
/dist/
*.o *.o
*.obj *.obj
*.a *.a
@@ -11,9 +18,7 @@
*.exe *.exe
*.pdb *.pdb
# vcpkg # vcpkg (bundled as a submodule at /vcpkg — see docs/building.md §2)
/vcpkg_installed/
/vcpkg/
# Generated protobuf # Generated protobuf
*.pb.cc *.pb.cc
@@ -33,8 +38,18 @@
.DS_Store .DS_Store
Thumbs.db Thumbs.db
# Apple / Windows client build artifacts (added in M4) # Apple / Windows client build artifacts
clients/apple/**/build/ clients/apple/**/build/
clients/apple/**/bin/
clients/apple/**/obj/
clients/apple/**/*.xcodeproj/xcuserdata/ clients/apple/**/*.xcodeproj/xcuserdata/
clients/apple/**/*.xcodeproj/project.xcworkspace/
clients/apple/**/*.xcframework/
clients/windows/**/bin/ clients/windows/**/bin/
clients/windows/**/obj/ clients/windows/**/obj/
# SwiftPM build artifacts
# Test artifacts: TOFU pin store written by vc_client during headless tests
voicecat_tofu_pins.txt
# Python bytecode cache (e.g. scripts/asc_api.py)
+38 -80
View File
@@ -1,94 +1,52 @@
# AGENTS.md — working method # VoiceCat working method
This file is the **working method** for a developer or AI agent picking up VoiceCat. Companion Start with `CLAUDE.md` for commands and architecture and `PROGRESS.md` for the current handoff.
files: The supported product is the .NET 10 implementation.
- [`CLAUDE.md`](CLAUDE.md) — the hub: build/test commands, architecture at a glance, doc map. ## Definition of done
- [`PROGRESS.md`](PROGRESS.md) — living tracker: what's done, what's next. **Update it as you work.**
- [`docs/`](docs/) — the **source of truth** for all design.
Read those, then use the method below. A clean compile is the floor. Changes are done when their observable behavior is covered by a
test or, for hardware/accessibility/release behavior, by a clearly recorded manual gate.
## What this repo is right now For implementation work:
A complete **design** ([`docs/`](docs/)) plus an **M0 skeleton**: it compiles and links, but 1. Read the relevant code and current contract document.
`libvoicecat`'s subsystems are stubs that return `VC_ERR_NOT_IMPLEMENTED`. Your job is to turn 2. Add or adjust the smallest behavior test that demonstrates the requested result.
the design into working software, one milestone at a time. 3. Implement the change without weakening encryption, real-time safety, or accessibility.
4. Build and run the affected tests, then the full managed test suite when practical.
5. Update concise current documentation and `PROGRESS.md` if the handoff changed.
## The working method (important) ## Supported layout
**A clean compile is the floor, not the goal.** Do not treat "make the compiler errors go | Path | Purpose |
away" as done. Each milestone in [`docs/roadmap.md`](docs/roadmap.md) has an **exit |---|---|
criterion stated as observable behavior** — that is what "done" means. Examples: | `proto/voicecat.proto` | Control-plane wire schema |
| `src/` | Managed server, core, protocol, crypto, audio, codec/DSP, and CLI |
| `tests/VoiceCat.Tests/` | Managed behavior tests |
| `clients/windows/` | Supported WinForms client |
| `clients/apple/` | Supported AppKit and UIKit clients |
| `native/media/` | Required narrow Opus/RNNoise C ABI |
| `native/rnnoise/` | Vendored RNNoise source/model |
| `native/apple/broadcast/` | Required ReplayKit extension and shared-memory producer |
- M1 done = *two `vccli` instances actually chat through a real server over TLS*, not "it builds". ## Core verification
- M2 done = *you can talk between two clients in a channel and hear loss concealment work*.
So the loop is:
1. Pick the current milestone in `docs/roadmap.md`. Read the relevant design doc section.
2. Write the smallest test (CTest, or a `vccli` interaction) that encodes the exit behavior.
3. Implement the subsystem until that test passes — not just until it compiles.
4. Keep the build green and the existing tests passing on every commit.
Every commit must compile and pass `ctest`. Behavior tests are how you know you're actually
making progress.
## Build
Skeleton (no third-party deps — works immediately):
```bash ```bash
cmake --preset dev ./scripts/build-native.ps1
cmake --build --preset dev dotnet restore VoiceCat.slnx --locked-mode
ctest --preset dev dotnet build VoiceCat.slnx -c Release --no-restore
dotnet test VoiceCat.slnx -c Release --no-build
./scripts/check-licenses.ps1
``` ```
When a subsystem needs real libraries, turn on vcpkg deps: Apple builds additionally use `./scripts/build-native-ios.sh` and
`clients/apple/VoiceCat.Apple.slnx` on macOS.
```bash ## Hard rules
export VCPKG_ROOT=/path/to/vcpkg # bootstrap vcpkg first; cross-platform
cmake --preset server-release # installs deps pinned in vcpkg.json
cmake --build --preset server-release
```
`vcpkg.json` currently has a placeholder `builtin-baseline` — set it to a real vcpkg commit - Encryption is mandatory: TLS control plus authenticated encrypted UDP media.
SHA the first time you enable `VOICECAT_USE_VCPKG_DEPS`. - Never introduce GPL/LGPL dependencies.
- Real-time audio callbacks never allocate, lock, block, or perform I/O.
## Where each subsystem lives (and its doc) - Preserve accessible names, keyboard operation, and curated screen-reader announcements.
- Wire, database, and shared-ring changes are deliberate versioned changes.
| Path | Subsystem | Design | - Keep `PROGRESS.md` short. Use Git history rather than accumulating completed-work prose.
|------|-----------|--------|
| `core/include/voicecat.h` | The C ABI every client/server calls | docs/architecture.md §4 |
| `core/proto/voicecat.proto` | Control-plane wire format | docs/protocol.md |
| `core/src/net/` | Asio TCP/UDP transport, framing | docs/architecture.md, docs/protocol.md §1 |
| `core/src/crypto/` | TLS 1.3 (mbedTLS), media AEAD (libsodium), anti-replay | docs/security.md |
| `core/src/codec/` | Opus encode/decode, FEC/DTX | docs/voice.md §34 |
| `core/src/protocol/` | Envelope (de)serialize, state machine, routing | docs/protocol.md |
| `core/src/session/` | Channels, users, streams, permissions, text | docs/protocol.md §5 |
| `core/src/audio/` | Capture/playback (miniaudio), APM DSP, jitter buffer, mixer | docs/voice.md §811 |
| `server/` | Connection mgr, session registry, SFU relay, SQLite | docs/architecture.md §5 |
| `tools/vccli/` | Headless client to drive/verify the protocol | — |
## Suggested first steps (M1 spine)
1. **Wire protobuf + framing** (`net/` + `protocol/`): build, then generate C++ from
`voicecat.proto`, implement the `[u32 length][Envelope]` framing over a plain TCP socket
(TLS can come right after). Test: round-trip an `Envelope` through the framer.
2. **TLS 1.3 via mbedTLS** (`crypto/`): wrap the TCP channel. Test: `vccli` completes a TLS
handshake against `voicecat-server` and exchanges a `ClientHello`/`ServerHello`.
3. **Auth + state** (`session/`): guest + admin-provisioned accounts (Argon2id/SQLite),
channel tree snapshot/deltas, ephemeral text relay. Test: two `vccli` chat.
Then proceed to M2 (UDP media) per the roadmap.
## House rules
- **No GPL/LGPL dependencies, ever** (closed-source redistribution is a goal). See
docs/tech-stack.md §5. CI should fail on a copyleft transitive dep.
- **Encryption is mandatory** — never add a plaintext transport path. docs/security.md.
- **Real-time audio threads** never allocate, lock, or block. docs/architecture.md §3.
- Keep `docs/` and code in sync. If you change a wire format or the C ABI, update the doc in
the same commit.
- The C ABI is the contract for the Swift/C# clients — treat changes to `voicecat.h` and
`voicecat.proto` as deliberate, versioned events (docs/protocol.md §8).
+35 -129
View File
@@ -1,143 +1,49 @@
# CLAUDE.md — agent hub for VoiceCat # VoiceCat developer guide
Auto-loaded each session. This is the **map**: build commands, architecture at a glance, and VoiceCat is a self-hosted encrypted voice and text chat system. The supported implementation
where everything is. For the *working method* read [`AGENTS.md`](AGENTS.md); for *what's done is .NET 10. Read `AGENTS.md` for working rules and `PROGRESS.md` for the current short status.
and what's next* read [`PROGRESS.md`](PROGRESS.md); for *design* read [`docs/`](docs/).
> **One-line status:** M2 voice/media plane is complete and verified through the real client ## Build and test
> library, not just a raw-socket harness (`ctest --test-dir build/m1-dev` green — 10/10 tests,
> including `test_voice_client_abi` driving two real `vc_client`s end-to-end, and `vccli
> --voice` manually verified live). Next up is **M3** (multi-stream, per-channel tuning,
> listener-side NR). See [`PROGRESS.md`](PROGRESS.md).
VoiceCat = self-hosted native voice & text chat (TeamSpeak/Mumble-style). Plain TCP (control)
+ UDP (media), no WebRTC, encrypted by default. A shared C++ core (`libvoicecat`) drives
native clients (Swift on macOS/iOS, C# on Windows) and the server.
---
## Build & test commands
The **M0 skeleton builds with no third-party dependencies** — just CMake + Ninja + a C++20
compiler. Deps (vcpkg) are off until a subsystem needs them.
```bash ```bash
# Configure + build the skeleton (default; no vcpkg needed) ./scripts/build-native.ps1
cmake --preset dev dotnet restore VoiceCat.slnx --locked-mode
cmake --build --preset dev dotnet build VoiceCat.slnx -c Release --no-restore
dotnet test VoiceCat.slnx -c Release --no-build
# Run the tests (behavior smoke test today; grows per milestone) ./scripts/check-licenses.ps1
ctest --preset dev # or: ctest --test-dir build/dev --output-on-failure
# Run the binaries (Windows adds .exe; Linux/macOS no extension)
./build/dev/bin/vccli # headless test client
./build/dev/bin/voicecat-server --help
./build/dev/bin/voicecat-server --name "My Server"
# Build a single target / be verbose
cmake --build --preset dev --target vccli
cmake --build --preset dev --verbose
# Clean
rm -rf build/dev # nuke; or:
cmake --build --preset dev --target clean
``` ```
When you start a subsystem that needs real libraries (mbedTLS, libsodium, opus, protobuf, …), Apple client builds require macOS, Xcode, and the .NET macOS/iOS workloads:
turn vcpkg deps on:
```bash ```bash
# one-time: git clone https://github.com/microsoft/vcpkg && ./vcpkg/bootstrap-vcpkg.sh (.bat on Windows) ./scripts/build-native-ios.sh
export VCPKG_ROOT=/path/to/vcpkg # works on Linux / macOS / Windows dotnet restore clients/apple/VoiceCat.Apple.slnx
cmake --preset server-release # auto-installs deps pinned in vcpkg.json dotnet build clients/apple/VoiceCat.Apple.slnx -c Debug --no-restore
cmake --build --preset server-release
``` ```
Other useful toggles (pass with `-D` at configure time): Windows publishing uses `clients/windows/publish-client.ps1`. Server publishing uses
`scripts/publish-server.ps1`.
```bash ## Current architecture
cmake --preset dev -DVOICECAT_BUILD_SHARED=ON # build libvoicecat as a .dll/.so/.dylib (for the C# client)
cmake --preset dev -DVOICECAT_BUILD_SERVER=OFF # core + tools only
cmake --preset dev -DVOICECAT_BUILD_TESTS=OFF
```
Formatting: `clang-format` config is `.clang-format` (Google base, 100 cols, 4-space). - `proto/voicecat.proto` is the control-plane wire schema.
- `src/VoiceCat.Protocol` owns protobuf framing and generated types.
- `src/VoiceCat.Crypto` owns TLS, TOFU, media AEAD, identity, and password hashing.
- `src/VoiceCat.Server` owns the TLS/UDP server and SQLite state.
- `src/VoiceCat.Core` owns client connection and protocol state.
- `src/VoiceCat.Audio`, `.Codec`, and `.Dsp` own voice processing.
- `src/VoiceCat.Cli` is the supported headless client.
- `clients/windows` is the WinForms client.
- `clients/apple` contains the AppKit and UIKit clients.
- `native/media` and `native/rnnoise` are the required Opus/RNNoise native boundary.
- `native/apple/broadcast` is the required Swift ReplayKit extension.
```bash ## Invariants
git ls-files '*.cpp' '*.h' | xargs clang-format -i
```
--- - TLS control and encrypted UDP media are mandatory; do not add plaintext transports.
- No GPL or LGPL dependencies.
## Architecture at a glance - Audio callbacks must not allocate, lock, block, or perform network I/O.
- Treat `proto/voicecat.proto`, persisted database formats, and the ReplayKit ring layout as
Full detail: [`docs/architecture.md`](docs/architecture.md). The short version: explicit versioned contracts.
- A passing build is not enough: add or update behavior tests for observable changes.
``` - Keep `PROGRESS.md` concise; do not append a historical changelog.
Swift (macOS/iOS) ─┐ ┌─ C# (Windows)
├──▶ libvoicecat (C ABI: voicecat.h) ◀──┤
voicecat-server ───┘ net · crypto · codec · protocol · └─ all UIs are thin
(links core) session · audio the core owns audio
```
- **One core, many faces.** Protocol, Opus, crypto, networking, jitter buffer, and mixing
live once in C++. Clients call the **C ABI** (`core/include/voicecat.h`); the server links
the same core, so framing/crypto never drift between ends.
- **Two transports.** TCP + **TLS 1.3** (control, protobuf `Envelope`) and UDP + **exported-key
ChaCha20-Poly1305 AEAD** (media, fixed binary voice frame). Encryption is mandatory.
- **Threading.** Real-time audio threads never allocate/lock/block; they exchange data with
the net thread via lock-free ring buffers; a worker pool absorbs blocking work.
### Subsystem map (code ↔ design doc)
| Path | Subsystem | Design |
|------|-----------|--------|
| `core/include/voicecat.h` | The C ABI (client/server contract) | architecture.md §4 |
| `core/proto/voicecat.proto` | Control-plane wire format (source of truth) | protocol.md |
| `core/src/net/` | Asio TCP/UDP transport, `[u32 len][payload]` framing | protocol.md §1, voice.md §2 |
| `core/src/crypto/` | TLS 1.3 (mbedTLS), media AEAD (libsodium), anti-replay | security.md |
| `core/src/codec/` | Opus encode/decode, FEC/DTX | voice.md §34 |
| `core/src/protocol/` | Envelope (de)serialize, request/response, dispatch | protocol.md |
| `core/src/session/` | Channels, users, streams, permissions, ephemeral text | protocol.md §5 |
| `core/src/audio/` | miniaudio I/O, APM DSP, jitter buffer, mixer | voice.md §811 |
| `core/src/core/` | `vc_client` — the handle behind the C ABI | architecture.md §4 |
| `server/` | Connection mgr, session registry, SFU relay, SQLite | architecture.md §5 |
| `tools/vccli/` | Headless client that drives/verifies the protocol | — |
| `clients/apple/`, `clients/windows/` | Native GUIs (M4) | architecture.md §4 |
---
## Documentation index (source of truth)
Read [`docs/`](docs/) before changing behavior. Order:
1. [docs/README.md](docs/README.md) — overview, locked decisions, glossary
2. [docs/architecture.md](docs/architecture.md) — core, C ABI, threading, server
3. [docs/protocol.md](docs/protocol.md) — control plane, Envelope, message catalog
4. [docs/voice.md](docs/voice.md) — UDP media, Opus, multi-stream, two-sided NR, VAD/PTT
5. [docs/security.md](docs/security.md) — mandatory encryption, TLS+AEAD, accounts, threat model
6. [docs/tech-stack.md](docs/tech-stack.md) — libraries, permissive-license rule, tooling
7. [docs/deployment.md](docs/deployment.md) — zero-config self-host (Docker / binary / source)
8. [docs/roadmap.md](docs/roadmap.md) — milestones + resolved decisions
---
## Keeping track of progress
**[`PROGRESS.md`](PROGRESS.md) is the living status file.** When you finish a task, check it
off there and note the next step, so the next agent can pick up instantly. Treat it as part of
the work, not an afterthought — update it in the same commit as the code.
---
## House rules (hard constraints)
- **A clean compile is the floor, not the goal.** "Done" = the milestone's observable exit
criterion in [`docs/roadmap.md`](docs/roadmap.md) passes (e.g. M1 = two `vccli` actually chat
over TLS). Encode it as a test. See [`AGENTS.md`](AGENTS.md).
- **No GPL/LGPL dependencies, ever** (closed-source redistribution is a goal). docs/tech-stack.md §5.
- **Encryption is mandatory** — never add a plaintext transport path. docs/security.md.
- **Real-time audio threads** never allocate, lock, or block. docs/architecture.md §3.
- **Keep docs + code in sync.** Changing a wire format (`voicecat.proto`) or the C ABI
(`voicecat.h`) is a deliberate, versioned act — update the doc in the same commit (protocol.md §8).
- Every commit must build (`cmake --build --preset dev`) and pass `ctest --preset dev`.
-53
View File
@@ -1,53 +0,0 @@
cmake_minimum_required(VERSION 3.25)
project(voicecat
VERSION 0.0.1
DESCRIPTION "Self-hosted native voice & text chat (see docs/)"
LANGUAGES CXX)
# ── Options ───────────────────────────────────────────────────────────────────
# The M0 skeleton compiles with NO third-party dependencies: every subsystem is a
# stub that returns VC_ERR_NOT_IMPLEMENTED. As each subsystem is built out, flip
# VOICECAT_USE_VCPKG_DEPS=ON so CMake pulls the real libraries (mbedTLS, libsodium,
# opus, protobuf, ...) via the vcpkg toolchain (see vcpkg.json / docs/tech-stack.md).
option(VOICECAT_USE_VCPKG_DEPS "Link real third-party deps via vcpkg" OFF)
option(VOICECAT_BUILD_SERVER "Build voicecat-server" ON)
option(VOICECAT_BUILD_TOOLS "Build the vccli headless test client" ON)
option(VOICECAT_BUILD_TESTS "Build tests" ON)
option(VOICECAT_BUILD_SHARED "Build libvoicecat as a shared library" OFF)
# ── Global settings ───────────────────────────────────────────────────────────
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
set(CMAKE_BUILD_TYPE Debug CACHE STRING "" FORCE)
endif()
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
# ── Targets ───────────────────────────────────────────────────────────────────
add_subdirectory(core)
if(VOICECAT_BUILD_SERVER)
add_subdirectory(server)
endif()
if(VOICECAT_BUILD_TOOLS)
add_subdirectory(tools/vccli)
if(VOICECAT_USE_VCPKG_DEPS)
add_subdirectory(tools/voicecat-admin)
endif()
endif()
if(VOICECAT_BUILD_TESTS)
enable_testing()
add_subdirectory(tests)
endif()
message(STATUS "VoiceCat ${PROJECT_VERSION} configured "
"(vcpkg deps: ${VOICECAT_USE_VCPKG_DEPS}, "
"server: ${VOICECAT_BUILD_SERVER}, tools: ${VOICECAT_BUILD_TOOLS})")
-77
View File
@@ -1,77 +0,0 @@
{
"version": 6,
"cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 },
"configurePresets": [
{
"name": "dev",
"displayName": "Dev (skeleton, no third-party deps)",
"description": "Builds the stub skeleton with just a compiler. Works out of the box; no vcpkg required.",
"generator": "Ninja",
"binaryDir": "${sourceDir}/build/dev",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Debug",
"VOICECAT_USE_VCPKG_DEPS": "OFF"
}
},
{
"name": "vcpkg-base",
"hidden": true,
"description": "Shared base for presets that link real deps via vcpkg. Requires VCPKG_ROOT in the environment.",
"generator": "Ninja",
"toolchainFile": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake",
"cacheVariables": { "VOICECAT_USE_VCPKG_DEPS": "ON" }
},
{
"name": "m1-dev",
"inherits": "vcpkg-base",
"displayName": "M1 Dev (TLS control plane, deps via vcpkg)",
"description": "Active development preset for M1+. Requires VCPKG_ROOT env var pointing to a bootstrapped vcpkg. Set VCPKG_ROOT=D:\\code\\nvgt\\vcpkg\\bin (or wherever your vcpkg is).",
"binaryDir": "${sourceDir}/build/m1-dev",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Debug",
"VOICECAT_USE_VCPKG_DEPS": "ON",
"VOICECAT_BUILD_TOOLS": "ON",
"VOICECAT_BUILD_TESTS": "ON",
"VCPKG_TARGET_TRIPLET": "x64-mingw-static",
"VCPKG_HOST_TRIPLET": "x64-mingw-static"
}
},
{
"name": "m2-dev",
"inherits": "vcpkg-base",
"displayName": "M2 Dev (voice + media, deps via vcpkg)",
"description": "Active development preset for M2+. Requires VCPKG_ROOT env var pointing to a bootstrapped vcpkg. Set VCPKG_ROOT=D:\\code\\nvgt\\vcpkg\\bin (or wherever your vcpkg is).",
"binaryDir": "${sourceDir}/build/m2-dev",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Debug",
"VOICECAT_USE_VCPKG_DEPS": "ON",
"VOICECAT_BUILD_TOOLS": "ON",
"VOICECAT_BUILD_TESTS": "ON",
"VCPKG_TARGET_TRIPLET": "x64-mingw-static",
"VCPKG_HOST_TRIPLET": "x64-mingw-static"
}
},
{
"name": "server-release",
"inherits": "vcpkg-base",
"displayName": "Server (release, real deps)",
"binaryDir": "${sourceDir}/build/server-release",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Release",
"VOICECAT_BUILD_TOOLS": "ON",
"VCPKG_TARGET_TRIPLET": "x64-mingw-static"
}
}
],
"buildPresets": [
{ "name": "dev", "configurePreset": "dev" },
{ "name": "m1-dev", "configurePreset": "m1-dev" },
{ "name": "m2-dev", "configurePreset": "m2-dev" },
{ "name": "server-release", "configurePreset": "server-release" }
],
"testPresets": [
{ "name": "dev", "configurePreset": "dev", "output": { "outputOnFailure": true } },
{ "name": "m1-dev", "configurePreset": "m1-dev", "output": { "outputOnFailure": true } },
{ "name": "m2-dev", "configurePreset": "m2-dev", "output": { "outputOnFailure": true } }
]
}
+14
View File
@@ -0,0 +1,14 @@
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<AnalysisLevel>latest</AnalysisLevel>
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
</PropertyGroup>
<PropertyGroup Condition="'$(VoiceCatIosStatic)' == 'true'">
<RuntimeIdentifiers>ios-arm64;iossimulator-arm64</RuntimeIdentifiers>
<NuGetLockFilePath>packages.ios.lock.json</NuGetLockFilePath>
</PropertyGroup>
</Project>
+13
View File
@@ -0,0 +1,13 @@
<Project>
<PropertyGroup>
<VoiceCatNativeRid Condition="'$(VoiceCatNativeRid)' == '' and '$(RuntimeIdentifier)' != ''">$(RuntimeIdentifier)</VoiceCatNativeRid>
<VoiceCatNativeRid Condition="'$(VoiceCatNativeRid)' == ''">$(NETCoreSdkRuntimeIdentifier)</VoiceCatNativeRid>
<VoiceCatNativeDirectory Condition="'$(VoiceCatNativeDirectory)' == ''">$(MSBuildThisFileDirectory)artifacts/native/runtimes/$(VoiceCatNativeRid)/native</VoiceCatNativeDirectory>
</PropertyGroup>
<ItemGroup Condition="'$(MSBuildProjectName)' != 'VoiceCat.Server' and '$(MSBuildProjectName)' != 'VoiceCat.Crypto' and '$(MSBuildProjectName)' != 'VoiceCat.Protocol'">
<None Include="$(MSBuildThisFileDirectory)artifacts/native/licenses/*.txt" Link="licenses/%(Filename)%(Extension)" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="$(VoiceCatNativeDirectory)/voicecat_media.dll" Condition="Exists('$(VoiceCatNativeDirectory)/voicecat_media.dll')" Link="%(Filename)%(Extension)" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="$(VoiceCatNativeDirectory)/libvoicecat_media.so" Condition="Exists('$(VoiceCatNativeDirectory)/libvoicecat_media.so')" Link="%(Filename)%(Extension)" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="$(VoiceCatNativeDirectory)/libvoicecat_media.dylib" Condition="Exists('$(VoiceCatNativeDirectory)/libvoicecat_media.dylib')" Link="%(Filename)%(Extension)" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>
</Project>
+27
View File
@@ -0,0 +1,27 @@
# syntax=docker/dockerfile:1.7
FROM mcr.microsoft.com/dotnet/sdk:10.0.203-noble AS build
WORKDIR /src
COPY global.json Directory.Build.props Directory.Build.targets ./
COPY proto/voicecat.proto proto/voicecat.proto
COPY src/ src/
RUN dotnet restore src/VoiceCat.Server/VoiceCat.Server.csproj -r linux-x64 --locked-mode -p:NuGetLockFilePath=packages.publish.linux-x64.lock.json
RUN dotnet publish src/VoiceCat.Server/VoiceCat.Server.csproj -c Release -r linux-x64 --self-contained true --no-restore \
-p:PublishSingleFile=true -p:PublishTrimmed=false -p:IncludeNativeLibrariesForSelfExtract=true -o /out \
&& mkdir /empty-data
FROM scratch AS export
COPY --from=build /out/VoiceCat.Server /VoiceCat.Server
FROM mcr.microsoft.com/dotnet/runtime-deps:10.0.10-noble-chiseled-extra AS runtime
WORKDIR /app
COPY --from=build --chown=1654:1654 /out/ ./
COPY --from=build --chown=1654:1654 /empty-data/ /data/
USER 1654:1654
VOLUME ["/data"]
EXPOSE 8384/tcp
EXPOSE 8384/udp
ENV VOICECAT_DATA_DIR=/data \
VOICECAT_BIND_ADDRESS=0.0.0.0 \
VOICECAT_BIND_PORT=8384
HEALTHCHECK --interval=30s --timeout=6s --start-period=10s --retries=3 CMD ["/app/VoiceCat.Server", "--health-check", "127.0.0.1:8384"]
ENTRYPOINT ["/app/VoiceCat.Server"]
+45 -238
View File
@@ -1,250 +1,57 @@
# PROGRESS — VoiceCat # VoiceCat status
Living status. **Update this file in the same commit as your work** so the next agent picks Updated: 2026-09-22
up instantly. Newest status at the top.
- **Date convention:** ISO (YYYY-MM-DD). ## Current state
- Statuses: `[ ]` not started · `[~]` in progress · `[x]` done.
--- VoiceCat's supported implementation is .NET 10. The managed protocol, crypto, TLS, server,
CLI, client state, audio engine, Windows client, macOS client, and iOS client are implemented.
Retired implementations and compatibility projects have been removed; this tree contains only
the supported product and its required native media boundaries.
## ▶ Where we left off / next action The source-of-truth layout is:
- **Done:** **M3 — multi-stream & per-channel tuning** ✓ complete (2026-06-16). See the M3 - `proto/voicecat.proto` — wire schema.
section below for the full file-by-file change list. `ctest --test-dir build/m1-dev` - `src/` — protocol, crypto, codec/DSP bindings, server, client core, audio, and CLI.
**11/11 tests** green (3 consecutive full-suite runs), including the new - `tests/VoiceCat.Tests/` — managed behavior and integration tests.
`test_m3_multistream` (real `vc_client`s, not raw sockets — same lesson as M2: ABI-level - `clients/windows/` — WinForms application over the managed core.
coverage is what proves the client library, not just the wire protocol). - `clients/apple/` — AppKit and UIKit applications over the managed core.
**Two items intentionally still open** (carried forward, not silently dropped): - `native/media/` and `native/rnnoise/` — required Opus/RNNoise shim and vendored RNNoise.
- `vc_set_input_device`/`vc_set_input_mode`/`vc_set_push_to_talk`/`vc_list_devices` - `native/apple/broadcast/` — required ReplayKit upload extension and shared ring producer.
(device enumeration + VAD/PTT input gate) remain `VC_ERR_NOT_IMPLEMENTED` — explicitly
scoped out of this M3 pass; revisit in a future milestone.
- Stereo Opus is now wire-correct end-to-end (a channel configured `MODE_STEREO` really
encodes/decodes 2-channel Opus packets), but `AudioEngine`'s playback mixer/output device
stays mono internally — stereo streams are downmixed (avg L/R) immediately after decode,
before mixing. True stereo *playback output* is a follow-up, not part of M3.
- **Next:** **M4 — native clients** (Windows C#, macOS/iOS Swift). See `docs/roadmap.md §M4`.
--- The physical-device iOS voice path now supports ReplayKit fallback, stable Apple VPIO voice-chat
capture using a paced 20 ms handoff, and true built-in stereo microphone capture. Stereo was
verified on an iPhone 16 Pro Max with a two-channel AVAudioEngine input and distinct left/right
samples; the managed Apple binding requires native use of its otherwise-unmapped stereo polar
pattern constant.
## Milestones (see [docs/roadmap.md](docs/roadmap.md) for full detail) The iOS user list now opens a remote-user detail view with independent tuning for each active
audio stream. Private messages are grouped into per-user conversations with direct access to the
same user and audio controls.
- [x] **M0 — Scaffolding** ✓ complete SQLite schema v4 persists DRED and the channel packet-loss mode. Manual loss remains the default;
- [x] **M1 — Control plane** ✓ complete (2026-06-15) automatic Fast/Balanced/Stable modes measure each sender's authenticated UDP uplink at the server,
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16) cap the applied Opus hint at 30%, and feed it back over TLS.
- [x] **M3 — Multi-stream & per-channel tuning** ✓ complete (2026-06-16)
- [ ] **M4 — Native clients** (Windows C#, macOS/iOS Swift) ← next
- [ ] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)
--- ## Release gates
## M0 — Scaffolding ✓ (completed) - Run real multi-person calls on Windows, macOS, and physical iOS hardware, including adaptive
20/40/60 ms buffering, duration-aware DRED/FEC, automatic packet-loss feedback, and mismatched
input/output endpoints.
- Complete NVDA and VoiceOver navigation/announcement passes.
- Verify iOS remote-user tuning and private-conversation navigation with VoiceOver, including
multiple streams, users without active streams, and users who disconnect while a view is open.
- Exercise iOS background/lock, interruption, Bluetooth, route-change, ReplayKit, and iOS 27
ScreenCaptureKit paths on devices. The background/lock gate keeps a call active for 15+ minutes
backgrounded and screen-locked with no periodic glitches and flat `VC_AUDIO` `feedDrops`/`starved`
counters (the render callback now paces the mix and the 20 ms capture handoff). Complete a
30-minute iOS call and Wi-Fi/cellular switching with voice restoration, plus extended
mono/stereo/voice-chat switching while joined. Verify Windows desktop/per-app stereo sharing.
- Complete Developer ID signing/notarization. The iOS host and ReplayKit extension have been
distribution-signed and packaged locally; upload the IPA for Apple's server-side validation.
- Run the published Linux container and a 30-minute-or-longer server soak.
- [x] Repo layout (`core/ server/ tools/ clients/ tests/`), CMake + presets, vcpkg manifest. ## Working rule
- [x] C ABI header `core/include/voicecat.h` (full surface, stubbed).
- [x] Protocol source-of-truth `core/proto/voicecat.proto` (matches docs/protocol.md).
- [x] Core stubs for all six subsystems (net/crypto/codec/protocol/session/audio) + `vc_client`.
- [x] `voicecat-server` (arg parsing, config, stub run) and `vccli` (drives the C ABI).
- [x] CTest **smoke test** asserting the C ABI contract (not just "it compiles").
- [x] `.gitattributes` (LF), `.gitignore`, `.clang-format`, onboarding docs.
- **Verified:** `cmake --preset dev && cmake --build --preset dev && ctest --preset dev` → green.
--- Keep this file short. It records only current state and open release gates. Git history is the
implementation diary.
## M1 — Control plane ✓ (completed 2026-06-15)
**Exit criterion:**`test_m1_integration` — two clients authenticate over TLS 1.3 (guest
+ Argon2id password), exchange channel and private text messages. Passes in ~1 s.
- [x] vcpkg baseline + `m1-dev` preset; `find_package` for protobuf/mbedTLS/libsodium/asio/sqlite3.
- [x] `FrameCodec` feed + emit; `encode_envelope` / `decode_envelope`.
- [x] Asio TCP acceptor + `TcpServerConn` (TLS path: blocking handshake thread + `tls_read_loop`).
- [x] `TlsContext` (mbedTLS 1.3, server cert/identity, ECDSA-P256 self-signed, TOFU on client).
- [x] `WorkerPool` (3 threads, used for Argon2id).
- [x] `Database` — SQLite, Argon2id via libsodium, `create_account` / `authenticate` / bootstrap admin.
- [x] `voicecat-admin` — account add/reset/del/list against live DB file.
- [x] `ServerIdentityManager` — generate/persist Ed25519 key + cert; fingerprint display.
- [x] `ConnSession` — WaitingHello → WaitingAuth → Authenticated state machine; full protocol relay.
- [x] `SessionRegistry` — channel tree, user map, broadcast, text routing.
- [x] `vc_client` (`client.cpp`) — full M1 C ABI: connect/TLS/ClientHello/AuthRequest/text/disconnect.
- [x] `Server::run()` — io_context, acceptor, worker pool, signal handling, `on_ready` callback.
- [x] `test_m1_integration` — M1 exit criterion. Verified green 2026-06-15.
**Key bug fixed:** double-framing in `ConnSession::send_envelope``encode_envelope` was
pre-framing the protobuf, then `TcpServerConn::send_frame` re-framed it. Fixed by serializing
raw protobuf bytes directly and letting `send_frame` add the single `[4-byte len]` prefix.
---
---
## M2 — Voice, single stream ✓ (completed 2026-06-16)
**Exit criterion:**`test_m2_voice` — two headless clients authenticate over TLS, bind UDP,
announce a MIC stream, send 50 encrypted Opus frames; server SFU relay re-encrypts + forwards
to the second client; B receives ≥ 25 frames and all decrypt correctly. Passes in ~4 s.
- [x] `m2-dev` preset (inherits `vcpkg-base`, binaryDir `build/m2-dev`); `m1-dev` also builds all M2 code.
- [x] `core/CMakeLists.txt``find_package(Opus)`, `find_path(MINIAUDIO_INCLUDE_DIR)`.
- [x] `core/src/net/voice_frame.h` — 14-byte UDP header (type/flags/codec/ssrc/seq/ts), serialize/parse, `make_udp_binding_packet`.
- [x] `SodiumMediaCrypto` — ChaCha20-Poly1305 AEAD; counter-nonce; 64-bit sliding-window anti-replay; `derive_send/recv` from TLS RFC 5705 exporter.
- [x] `OpusEncoder` / `OpusDecoder` — libopus 1.6, FEC, DTX, PLC (free; nullptr → decoder extrapolates).
- [x] `UdpMediaChannel` — async UDP socket (asio); thread-safe `send_to`; async recv loop.
- [x] `JitterBuffer` — per-ssrc, EWMA jitter estimation, adaptive depth 20200 ms, late-drop at 500 ms.
- [x] `AudioEngine` — miniaudio capture+playback; `inject_capture()` bypass for headless tests; per-ssrc RemoteStream with OpusDecoder + JitterBuffer.
- [x] `ApmProcessor``ApmPassthrough` stub (VAD always open); WebRTC APM deferred until M3.
- [x] `on_tls_ready` callback in `TcpChannelCallbacks` — server derives and stores media AEAD keys immediately after TLS handshake.
- [x] `ConnSession` M2 — `udp_token` generated at construction; included in `AuthResult`; `handle_udp_binding` (verifies token, TCP ack); `handle_stream_announce` (assigns SSRC via registry); `udp_media_port` in `ServerHello`.
- [x] `SessionRegistry` M2 — `register_udp_token`, `find_by_udp_token`, `register_udp_endpoint`, `find_by_udp_endpoint`, `assign_ssrc`, `find_channel_sessions`, `user_channel`.
- [x] `MediaRelay` — SFU UDP relay; `kFrameUdpBinding` → endpoint binding; `kFrameVoice` → decrypt/re-encrypt/forward to channel members.
- [x] `Server::run()` — creates and binds `MediaRelay`; passes media port to `ConnSession`; wires `on_tls_ready` to derive per-connection media AEAD keys.
- [x] `test_voice_frame` — header round-trip, big-endian layout, binding packet format.
- [x] `test_media_aead` — seal/open round-trip, anti-replay, tamper detection, multi-packet sequence.
- [x] `test_opus_codec` — encode/decode round-trip energy check (within 3 dB), PLC, frame-samples helper.
- [x] `test_m2_voice` — M2 exit criterion (raw-socket harness). Verified green 2026-06-16.
**Follow-up (same day):** the above made `test_m2_voice` pass, but `vc_client`'s public voice
methods were still stubs — the *actual* M2 exit criterion ("two vccli/early-GUI clients talk")
wasn't met. Closed the gap:
- [x] `core/src/core/client.cpp` — real `stream_start`/`stream_stop`/`set_self_mute`/
`set_remote_stream`; UDP-binding handshake (`start_udp_binding`/`handle_udp_binding_ack`/
`finish_udp_binding`); media key derivation from `tls_` (RFC 5705 exporter); `run_udp_recv`
(AEAD-open → `JitterBuffer::Frame``audio_engine_.push_recv_frame`); `on_capture_frame`
(encode → seal → `sendto`); `sync_remote_streams` (diffs a `User` proto's `streams` against
`remote_streams_`, wiring up `OpusDecoder`s and emitting `STREAM_STARTED`/`STOPPED`).
`set_input_device`/`set_input_mode`/`set_push_to_talk`/`list_devices` remain
`VC_ERR_NOT_IMPLEMENTED` — no device-enumeration backend yet; scoped to M3 (VAD/PTT).
- [x] `core/src/session/session.cpp/h``SessionModel::find_user`, `find_user_by_ssrc`,
`Stream{stream_id, ssrc, kind, label, sample_rate, frame_ms}`.
- [x] `server/src/conn_session.cpp/h``handle_stream_announce`/`handle_stream_stop` now
broadcast via `SessionRegistry::set_user_stream`/`clear_user_stream``UserEvent::UPDATED`.
- [x] `server/src/session_registry.cpp/h``set_user_stream`/`clear_user_stream` (mutate a
user's `StreamInfo` list, return the updated `User` proto for broadcast).
- [x] `tests/test_voice_client_abi.cpp` — drives two real `vc_client` instances through
`vc_connect`/`vc_authenticate_guest`/`vc_stream_start`/`vc_stream_stop`; asserts client B
observes client A's `STREAM_STARTED`/`STOPPED` events. Verified green 2026-06-16.
- [x] `tools/vccli/src/main.cpp` — argv parsing (`--host/--port/--nick/--channel/--voice/
--mute/--text`); `--voice` starts a MIC stream and blocks on SIGINT, printing `on_event`
callbacks live (unbuffered stdout — MinGW/MSVCRT treat `_IOLBF` as full buffering for
non-console streams). Dropped the originally-planned `--voice-loopback` and the
`tx=N rx=M lost=K jitter=J` stats line: `voicecat.h` exposes no PCM-injection hook or
jitter/loss stats getter publicly, only `on_event` + `on_level` (RMS). Manually verified:
two `vccli --voice` instances see each other's stream start in real time.
---
## M3 — Multi-stream & per-channel tuning ✓ (completed 2026-06-16)
**Exit criterion:** ✓ `test_m3_multistream` — a real `vc_client` (A) runs two concurrent local
streams (MIC + SCREEN_AUDIO) with distinct stream ids; a second client (B) sees both as
separate `STREAM_STARTED` events and a `VC_EVENT_TALK_STATE` talking edge for A's MIC stream;
B independently sets gain/mute/noise-reduction on each of A's streams without one call
affecting the other; A then joins "Music Room" (channel 2: stereo/128kbps/`OPUS_AUDIO`/no
DTX) and announces a fresh MIC stream there, while B stays in "Lobby" (channel 1: mono/24kbps/
`OPUS_VOIP`/DTX on) — `vc_get_stream_audio_config` shows their effective Opus config differs
exactly as the server enforces per channel. Passes in ~2.4s; verified across 8 consecutive
standalone runs + 3 consecutive full-suite runs with no flakes.
Exploration before implementing turned up several bugs/gaps where the wire format already
supported this milestone but the client/server logic didn't — these were fixed as part of M3,
not treated as pre-existing-and-out-of-scope:
- [x] **Server `stream_id` bug** — `handle_stream_announce` always wrote `stream_id=1`, so a
second stream from the same user silently overwrote the first in
`SessionRegistry::set_user_stream`'s replace-by-id logic. Fixed with a per-session counter
(`ConnSession::next_stream_id_`) + `announced_stream_ids_` (also now validated in
`handle_stream_stop`, rejecting stops for ids the session never announced).
- [x] **Per-channel `AudioConfig` was modeled but never populated/enforced.**
`SessionRegistry::init_default_channels()` now seeds Lobby (id=1: mono, 24kbps, `OPUS_VOIP`,
FEC+DTX on) and a new "Music Room" (id=2: stereo, 128kbps, `OPUS_AUDIO`, FEC+DTX off) with
real `AudioConfig`s; new `SessionRegistry::channel_audio_config(channel_id)` accessor (there
was no per-id channel getter before, only `channel_snapshot()`). `handle_stream_announce`
now treats the channel's config as authoritative (mode/frame_ms/application/fec/dtx/
complexity), clamping (not overriding) `bitrate_bps` to the channel's ceiling.
- [x] **Client silently dropped `mode`/`dtx`/`complexity`/`application` from `effective_audio`**
even for the single M2 stream — `handle_stream_announce_result` and `sync_remote_streams`
only copied `sample_rate`/`bitrate_bps`/`frame_ms`/`fec` into `OpusParams`. New shared
`opus_params_from_audio_config()` helper (`client.cpp`) fixes both the send and receive
paths.
- [x] `core/src/codec/opus_codec.h/.cpp` — new `OpusApplication` enum + `OpusParams::application`
field; `OpusEncoder::init` now honors it instead of hardcoding `OPUS_APPLICATION_VOIP`.
- [x] `core/src/session/session.h/.cpp` — `Stream` struct extended with the full `AudioConfig`
(mode/bitrate_bps/application/fec/expected_packet_loss/dtx/complexity), not just
sample_rate/frame_ms; `copy_streams()` now copies all of it.
- [x] `core/src/core/client.h/.cpp` — local-stream state is now a `std::unordered_map<int,
LocalStream>` keyed by `vc_stream_kind` (one active stream per kind — MIC/SCREEN_AUDIO/
AUX_DEVICE are each singletons for a client), replacing the M2 single-stream fields.
`StreamAnnounce`/`StreamAnnounceResult` round-trips are now correlated by `request_id`
(already round-tripped on the wire; just wasn't read) via `pending_announce_kind_`, so
multiple concurrent announces from one client resolve to the right `LocalStream`.
`on_capture_frame` takes a `kind` parameter and upmixes mono capture to stereo (duplicate
L=R) when a stream's channel config calls for it. `vc_set_self_mute`'s `mic_muted` only
gates the `MIC` kind — a concurrent `SCREEN_AUDIO` share keeps playing while muted.
`set_remote_stream` now actually wires `noise_reduction` through (previously parsed and
discarded). New `run_talk_timer()` (a small dedicated thread, started alongside the UDP
media path, never the miniaudio callback thread) polls both remote talk-state edges
(`AudioEngine::poll_talk_transitions()`) and local capture-activity edges, emitting
`VC_EVENT_TALK_STATE`.
- [x] **Fixed a thread-join race in `teardown_voice()`** — it's called both from `run_io()`'s
own cleanup and from `disconnect()`, on different threads; without serialization both could
see `udp_thread_`/`talk_timer_thread_` as `joinable()` simultaneously and race to `join()`
the same `std::thread` (UB; surfaced as an intermittent `std::system_error: No such process`
under `ctest`). Added a `teardown_mu_` guard around the whole function. This pre-existed for
`udp_thread_` alone (likely the same root cause as the `test_m1_integration`/`test_m2_voice`
cleanup-path flake noted in the M2 section above) — adding `talk_timer_thread_`'s join just
made it surface more often, so it was fixed properly here rather than carried forward again.
- [x] `core/src/audio/audio_engine.h/.cpp` — `CaptureCallback` gained a `kind` parameter
(the real miniaudio capture device is always tagged `kind=0`/MIC; a second concurrent local
stream is fed via its own `inject_capture(kind, ...)` ring buffer — `inject_taps_`, keyed by
kind — since there is only one real hardware capture device in M3). Fixed a buffer-sizing
bug in `on_playback`'s per-stream decode (`opus_decode`'s `frame_size` parameter is
samples-*per-channel*, not total samples — the old code passed `frames * params_.channels`,
which would have overflowed the decode buffer for any stereo stream). Stereo decoder output
is downmixed (avg L/R) into the engine's mono mix accumulator immediately after decode.
`RemoteStream` gained `recv_ns`/`noise_reduction_enabled` (lazy `ApmProcessor` instantiation
— freed on disable, so no separate instance cap is needed per the roadmap's guidance) and
`last_voice_ms`/`talking` (talk-indicator edge state, updated in `push_recv_frame`); new
`set_stream_noise_reduction()` and `poll_talk_transitions()`. Note: until `VOICECAT_HAS_APM`
is wired to a real WebRTC APM build, the NS toggle is plumbed end-to-end but behaviorally a
passthrough no-op (`ApmPassthrough` doesn't touch PCM) — same situation send-side APM has
been in since M2; M3's job was the plumbing, not the DSP backend.
- [x] **New C ABI surface** (`core/include/voicecat.h`, additive only):
`vc_audio_config` struct + `vc_get_stream_audio_config(c, user_id, stream_id, out)` — the
effective Opus config for a stream you own or a peer's, reading from the (now richer)
`LocalStream`/`session::Stream`. `vc_test_inject_capture(c, stream_id, pcm, samples)` —
clearly-marked **test-only**, forwards to `AudioEngine::inject_capture`, so
`test_m3_multistream` can drive two concurrent synthetic-audio streams through the real ABI
without a microphone.
- [x] `tests/test_m3_multistream.cpp` — the M3 exit criterion (ABI-level, mirrors
`test_voice_client_abi.cpp`'s approach per the M2 lesson). Registered in `tests/CMakeLists.txt`.
**Explicitly out of scope for this pass** (confirmed with the user before implementing):
- `vc_set_input_device`/`vc_set_input_mode`/`vc_set_push_to_talk`/`vc_list_devices` (device
enumeration + VAD/PTT input gate) — still `VC_ERR_NOT_IMPLEMENTED`. These were mentioned as
"scoped to M3" in the M2 follow-up notes above, but docs/roadmap.md's M3 bullets never
actually listed them — deferred again, now tracked explicitly rather than implicitly.
- Real WASAPI desktop-audio loopback capture for `SCREEN_AUDIO` — the engine now supports
feeding a second concurrent local stream via `inject_capture`, but only synthetic PCM is
wired up; a real loopback capture device is a follow-up.
- True stereo *playback output* — `AudioEngine`'s mixer/output device stays mono; stereo
streams are downmixed after decode (see above). The Opus wire format itself is fully
stereo-correct.
---
## Decisions log
All architecture/scope decisions are settled and recorded in
[docs/roadmap.md §2 "Resolved decisions"](docs/roadmap.md) and reflected across `docs/`.
If you make a *new* decision, record it there and link it here.
---
## How to update this file
1. Check off tasks as you complete them; flip a milestone to `[x]` only when its **exit
criterion test** passes.
2. Keep the **"Where we left off / next action"** block at the top accurate — it's the first
thing the next agent reads.
3. When you start a milestone, copy its task list from `docs/roadmap.md` into a section here.
+28 -44
View File
@@ -1,58 +1,42 @@
# VoiceCat # VoiceCat
Self-hosted, native voice & text chat in the spirit of classic TeamSpeak / Mumble — VoiceCat is a self-hosted, channel-based voice and text chat system built on .NET 10. It uses
channel-based voice, channel + private text, one server you run yourself. Plain **TCP** TLS 1.3 for protobuf control traffic and authenticated encrypted UDP for Opus media. There is
(control) and **UDP** (media), no WebRTC. Encrypted by default. A shared **C++ core** no WebRTC, central directory, or plaintext mode.
(`libvoicecat`) drives native clients (Swift on macOS/iOS, C# on Windows) and the server.
> **Status: pre-implementation.** The design is complete in [`docs/`](docs/). The code is an The repository contains a managed server, CLI, shared client/audio core, and native Windows,
> M0 **skeleton** — it compiles and links, but every subsystem is a stub. See macOS, and iOS user interfaces. A small C library supplies Opus/RNNoise, and a small Swift iOS
> [`AGENTS.md`](AGENTS.md) to start building, and [`docs/roadmap.md`](docs/roadmap.md) for the extension captures ReplayKit application audio.
> milestones.
## Read the design first ## Build
The [`docs/`](docs/) folder is the source of truth. Start at [`docs/README.md`](docs/README.md),
then `architecture``protocol``voice``security``tech-stack``deployment`
`roadmap`.
## Build the skeleton (no dependencies needed yet)
The M0 skeleton builds with just a C++20 compiler + CMake + Ninja — **no vcpkg, no
third-party libraries**, because every subsystem is currently a stub.
```bash ```bash
cmake --preset dev ./scripts/build-native.ps1
cmake --build --preset dev dotnet restore VoiceCat.slnx --locked-mode
ctest --preset dev # runs the smoke test (links the core, calls the C ABI) dotnet build VoiceCat.slnx -c Release --no-restore
dotnet test VoiceCat.slnx -c Release --no-build
``` ```
Artifacts land in `build/dev/bin/` (`voicecat-server`, `vccli`). See [CLAUDE.md](CLAUDE.md) for the developer map, [docs/README.md](docs/README.md) for current
contracts, and [PROGRESS.md](PROGRESS.md) for the short release handoff.
When you start implementing a subsystem that needs real libraries, build with vcpkg deps:
```bash
# one-time: git clone https://github.com/microsoft/vcpkg && ./vcpkg/bootstrap-vcpkg.sh
export VCPKG_ROOT=/path/to/vcpkg # set VCPKG_ROOT (works on Linux/macOS/Windows)
cmake --preset server-release # auto-installs deps from vcpkg.json
cmake --build --preset server-release
```
## Layout ## Layout
``` ```text
docs/ design spec (read this) proto/ protobuf wire schema
core/ libvoicecat — the shared C++ core src/ managed protocol, crypto, server, client, audio, and CLI
include/ voicecat.h (the C ABI all clients call) tests/ managed behavior and integration tests
proto/ voicecat.proto (control-plane wire format, source of truth) clients/windows/ WinForms client
src/ net/ crypto/ codec/ protocol/ session/ audio/ (stubs today) clients/apple/ AppKit and UIKit clients
server/ voicecat-server (headless; links the core) native/media/ narrow Opus/RNNoise C shim
tools/vccli/ headless test client — drives the protocol from M1 on native/rnnoise/ vendored RNNoise source and model
clients/ apple/ (Swift, M4) windows/ (C#, M4) — placeholders for now native/apple/broadcast/ ReplayKit broadcast extension
tests/ CTest targets docs/ current contracts and operating documentation
``` ```
## License ## Non-negotiable constraints
Permissive-only dependencies (no GPL/LGPL) so the project can be redistributed freely, - Encryption is mandatory.
including closed-source. Project license: TBD (see [`docs/tech-stack.md`](docs/tech-stack.md) §5). - No GPL or LGPL dependencies.
- Real-time audio callbacks never allocate, lock, block, or perform I/O.
- Accessibility is a release requirement on every client platform.
+18
View File
@@ -0,0 +1,18 @@
<Solution>
<Folder Name="/src/">
<Project Path="src/VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
<Project Path="src/VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
<Project Path="src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
<Project Path="src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
<Project Path="src/VoiceCat.Server/VoiceCat.Server.csproj" />
<Project Path="src/VoiceCat.Core/VoiceCat.Core.csproj" />
<Project Path="src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
<Project Path="src/VoiceCat.Cli/VoiceCat.Cli.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/VoiceCat.Tests/VoiceCat.Tests.csproj" />
</Folder>
<Folder Name="/clients/">
<Project Path="clients/windows/VoiceCat.Windows/VoiceCat.Windows.csproj" />
</Folder>
</Solution>
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+41 -15
View File
@@ -1,19 +1,45 @@
# Apple client (macOS + iOS) — placeholder # Apple clients
Built in **M4** (see [`docs/roadmap.md`](../../docs/roadmap.md)). Swift + SwiftUI, consuming `VoiceCat.Mac` and `VoiceCat.iOS` are .NET 10 AppKit and UIKit clients over the shared managed
`libvoicecat` through the C ABI ([`core/include/voicecat.h`](../../core/include/voicecat.h)). core. The ReplayKit upload extension under `native/apple/broadcast` remains Swift because it
runs under the extension memory limit and writes the versioned shared audio ring.
Planned shape (see [`docs/architecture.md`](../../docs/architecture.md) §4 and Build on macOS with Xcode and the pinned .NET workloads:
[`docs/tech-stack.md`](../../docs/tech-stack.md) §2):
- A Swift Package wrapping the core as an **XCFramework** (macOS + iOS device + simulator). ```bash
- A module map exposing `voicecat.h` to Swift (Swift can also use C++ interop directly, but ./scripts/build-native.ps1
the C ABI is the stable contract). ./scripts/build-native-ios.sh
- SwiftUI app target for macOS and iOS. dotnet restore clients/apple/VoiceCat.Apple.slnx
- **iOS audio:** app owns `AVAudioSession` (`.playAndRecord` / `.voiceChat`), mic permission, dotnet build clients/apple/VoiceCat.Apple.slnx -c Debug --no-restore
interruption/route handling, calling `vc_audio_*` hooks on the core. ```
- **iOS screen/system audio (`SCREEN_AUDIO`):** a **ReplayKit Broadcast Upload Extension**
capturing `RPSampleBufferType.audioApp`, linking a minimal core slice, sharing session
state via an **App Group** ([`docs/voice.md`](../../docs/voice.md) §9).
Nothing here yet — the core must reach M2 (working voice) before the GUI is worth building. Use `publish-macos.sh --dry-run` to validate a local ad-hoc macOS bundle. The dry-run build does
not enable hardened runtime because ad-hoc signatures have no Team ID and cannot satisfy macOS
library validation. The script normalizes nested signatures and installs the verified bundle at
both `VoiceCat.Mac/bin/Release/net10.0-macos27.0/osx-arm64/VoiceCat.app` and
`VoiceCat.Mac/bin/Release/distribution/VoiceCat.app`. Distribution builds remain hardened and
require `VOICECAT_CODESIGN_IDENTITY`; optional notarization uses `APPLE_ID`, `APPLE_TEAM_ID`, and
`APPLE_APP_PASSWORD`.
For a physical iOS device, use `build-ios-device.sh` and `deploy-ios-device.sh`. The host and
ReplayKit extension require signing profiles with App Group `group.me.iamtalon.voicecat`.
Hardware validation must cover VoiceOver, background and lock behavior, interruptions, route
changes, Bluetooth, ReplayKit, and iOS 27 ScreenCaptureKit audio.
For iOS voice stability, leave a call joined with the microphone active for at least 30 minutes
and confirm speech stays clear and `VC_AUDIO` reports no growing `feedDrops`. While still joined,
toggle Wi-Fi off and on, switch between Wi-Fi and cellular, and confirm the app stays open,
reconnects, and restores the voice session. Repeat with mono, stereo, and voice processing.
The iOS remote-user manual gate must also cover Users → user detail → independent microphone and
screen-audio gain/mute controls, microphone receive noise reduction, the no-active-stream state,
and Private Chats → conversation → User/audio settings. Repeat the navigation with VoiceOver and
disconnect the remote user while its detail and conversation views are open.
App Store builds use the same device builder with `--configuration Release`. Set
`VOICECAT_BUILD_NUMBER`, `VOICECAT_DISPLAY_VERSION`, `VOICECAT_DEVELOPMENT_TEAM`, the host
`VOICECAT_CODESIGN_KEY`/`VOICECAT_CODESIGN_PROVISION` pair, and the extension
`VOICECAT_BROADCAST_CODESIGN_KEY`/`VOICECAT_BROADCAST_CODESIGN_PROVISION` pair. The host and
extension profiles must both be App Store Connect profiles and include the shared App Group.
The complete TestFlight workflow is documented in `docs/apple-ios-release.md`. The shared ring
contract is documented in `docs/broadcast-ring-format.md`.
+14
View File
@@ -0,0 +1,14 @@
<Solution>
<Folder Name="/apps/">
<Project Path="VoiceCat.Mac/VoiceCat.Mac.csproj" />
<Project Path="VoiceCat.iOS/VoiceCat.iOS.csproj" />
</Folder>
<Folder Name="/managed/">
<Project Path="../../src/VoiceCat.Protocol/VoiceCat.Protocol.csproj" />
<Project Path="../../src/VoiceCat.Crypto/VoiceCat.Crypto.csproj" />
<Project Path="../../src/VoiceCat.Codec/VoiceCat.Codec.csproj" />
<Project Path="../../src/VoiceCat.Dsp/VoiceCat.Dsp.csproj" />
<Project Path="../../src/VoiceCat.Audio/VoiceCat.Audio.csproj" />
<Project Path="../../src/VoiceCat.Core/VoiceCat.Core.csproj" />
</Folder>
</Solution>
@@ -0,0 +1,166 @@
using AppKit;
using CoreGraphics;
using VoiceCat.Core;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.Mac;
internal sealed class AdministrationWindowController : NSWindowController
{
private readonly VoiceCatClient client;
private readonly uint selfId;
private readonly NSPopUpButton users = new(new CGRect(25, 335, 310, 28), false);
private readonly NSPopUpButton channels = new(new CGRect(25, 175, 310, 28), false);
private readonly NSTextField status = NSTextField.CreateLabel("");
internal AdministrationWindowController(VoiceCatClient client, uint selfId) : base(new NSWindow(new CGRect(0, 0, 560, 420),
NSWindowStyle.Titled | NSWindowStyle.Closable, NSBackingStore.Buffered, false))
{
this.client = client; this.selfId = selfId; Window!.Title = "VoiceCat Administration"; Window.Center();
NSView view = Window.ContentView!;
AddLabel(view, "User moderation", 25, 375); view.AddSubview(users);
AddButton(view, "Move…", 350, 333, Move); AddButton(view, "Kick…", 440, 333, Kick);
AddButton(view, "Ban…", 350, 293, Ban); AddButton(view, "Permissions…", 440, 293, Permissions);
AddButton(view, "Server mute", 350, 253, ServerMute); AddButton(view, "Server deafen", 440, 253, ServerDeafen);
AddButton(view, "Audio tuning…", 25, 253, Tune);
AddLabel(view, "Channel administration", 25, 215); view.AddSubview(channels);
AddButton(view, "Create…", 350, 173, CreateChannel); AddButton(view, "Edit…", 440, 173, EditChannel);
AddButton(view, "Delete", 350, 133, DeleteChannel); AddButton(view, "Accounts…", 440, 133, Accounts);
var refresh = new NSButton(new CGRect(25, 75, 100, 32)) { Title = "Refresh" }; refresh.Activated += (_, _) => Refresh(); view.AddSubview(refresh);
status.Frame = new CGRect(25, 25, 510, 35); status.AccessibilityLabel = "Administration result"; view.AddSubview(status);
((INSAccessibility)users).AccessibilityLabel = "User to administer"; ((INSAccessibility)channels).AccessibilityLabel = "Channel to administer";
Refresh(); ApplyPermissions();
}
private void Refresh()
{
uint selectedUser = Selected(users), selectedChannel = Selected(channels);
users.RemoveAllItems(); foreach (User user in client.Users.Where(user => user.Id != selfId).OrderBy(user => user.Nickname)) Add(users, user.Nickname, user.Id);
channels.RemoveAllItems(); foreach (Channel channel in client.Channels.OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) Add(channels, channel.Name, channel.Id);
Select(users, selectedUser); Select(channels, selectedChannel);
}
private void ApplyPermissions()
{
Permissions permissions = client.Permissions;
status.StringValue = permissions.IsAdmin ? "Connected as administrator" : "Only actions allowed by your server permissions will succeed.";
}
private async void Move(object? sender, EventArgs args)
{
uint userId = Selected(users); if (userId == 0) return;
string[] names = client.Channels.Select(channel => channel.Name).ToArray(); int index = PromptChoice("Move user", "Destination channel", names); if (index < 0) return;
Show(await client.MoveUserAsync(userId, client.Channels[index].Id));
}
private async void Kick(object? sender, EventArgs args) { uint id = Selected(users); string? reason = PromptText("Kick user", "Reason", "Kicked by moderator"); if (id != 0 && reason is not null) Show(await client.KickUserAsync(id, reason)); }
private async void Ban(object? sender, EventArgs args)
{
uint id = Selected(users); if (id == 0) return;
string? reason = PromptText("Ban user", "Reason (append @1h, @1d, or @permanent)", "Banned by moderator @1d"); if (reason is null) return;
ulong expiry = 0; string lowered = reason.ToLowerInvariant();
if (lowered.EndsWith("@1h")) expiry = checked((ulong)DateTimeOffset.UtcNow.AddHours(1).ToUnixTimeMilliseconds());
else if (lowered.EndsWith("@1d")) expiry = checked((ulong)DateTimeOffset.UtcNow.AddDays(1).ToUnixTimeMilliseconds());
Show(await client.BanUserAsync(id, reason.Replace("@1h", "", StringComparison.OrdinalIgnoreCase).Replace("@1d", "", StringComparison.OrdinalIgnoreCase).Replace("@permanent", "", StringComparison.OrdinalIgnoreCase).Trim(), expiry));
}
private async void Permissions(object? sender, EventArgs args)
{
uint id = Selected(users); if (id == 0) return;
Permissions? permissions = PermissionEditor.Run(); if (permissions is not null) Show(await client.SetPermissionsAsync(id, permissions));
}
private async void ServerMute(object? sender, EventArgs args) { User? user = User(); if (user is not null) Show(await client.SetServerMuteAsync(user.Id, !user.ServerMuted, user.ServerDeafened)); }
private async void ServerDeafen(object? sender, EventArgs args) { User? user = User(); if (user is not null) Show(await client.SetServerMuteAsync(user.Id, user.ServerMuted, !user.ServerDeafened)); }
private void Tune(object? sender, EventArgs args) { User? user = User(); if (user is not null) PerUserTuning.Run(client, user); }
private async void CreateChannel(object? sender, EventArgs args) { ChannelEdit? edit = ChannelEditor.Run(null, client.Channels, client.SupportsAdaptivePacketLoss); if (edit is not null) { Show(await client.CreateChannelAsync(edit.Channel, edit.Password)); Refresh(); } }
private async void EditChannel(object? sender, EventArgs args) { Channel? channel = Channel(); if (channel is null) return; ChannelEdit? edit = ChannelEditor.Run(channel, client.Channels, client.SupportsAdaptivePacketLoss); if (edit is not null) { Show(await client.EditChannelAsync(edit.Channel, edit.Password)); Refresh(); } }
private async void DeleteChannel(object? sender, EventArgs args)
{
Channel? channel = Channel(); if (channel is null || channel.Id == 1) return;
if (!Confirm("Delete channel?", channel.Name)) return; Show(await client.DeleteChannelAsync(channel.Id)); Refresh();
}
private async void Accounts(object? sender, EventArgs args) => await AccountEditor.RunAsync(client);
private User? User() => client.Users.FirstOrDefault(user => user.Id == Selected(users));
private Channel? Channel() => client.Channels.FirstOrDefault(channel => channel.Id == Selected(channels));
private void Show(GenericResult result) { status.StringValue = result.Ok ? result.Message.Length == 0 ? "Operation completed." : result.Message : result.Message; Refresh(); }
private static void Add(NSPopUpButton picker, string title, uint id) { picker.AddItem(title); picker.LastItem!.RepresentedObject = new Foundation.NSString(id.ToString()); }
private static uint Selected(NSPopUpButton picker) => uint.TryParse(picker.SelectedItem?.RepresentedObject?.ToString(), out uint id) ? id : 0;
private static void Select(NSPopUpButton picker, uint id) { for (nint i = 0; i < picker.ItemCount; i++) if (picker.ItemAtIndex(i)?.RepresentedObject?.ToString() == id.ToString()) { picker.SelectItem(i); break; } }
private static void AddLabel(NSView view, string value, double x, double y) { NSTextField label = NSTextField.CreateLabel(value); label.Frame = new CGRect(x, y, 300, 22); label.Font = NSFont.BoldSystemFontOfSize(13)!; view.AddSubview(label); }
private static void AddButton(NSView view, string title, double x, double y, EventHandler handler) { var button = new NSButton(new CGRect(x, y, 95, 32)) { Title = title }; button.Activated += handler; view.AddSubview(button); }
internal static string? PromptText(string title, string prompt, string value = "", bool secure = false)
{
NSTextField field = secure ? new NSSecureTextField(new CGRect(0, 0, 340, 26)) : new NSTextField(new CGRect(0, 0, 340, 26)); field.StringValue = value;
var alert = new NSAlert { MessageText = title, InformativeText = prompt, AccessoryView = field }; alert.AddButton("OK"); alert.AddButton("Cancel");
return alert.RunModal() == 1000 ? field.StringValue : null;
}
internal static int PromptChoice(string title, string prompt, string[] values)
{
var picker = new NSPopUpButton(new CGRect(0, 0, 340, 28), false); picker.AddItems(values);
var alert = new NSAlert { MessageText = title, InformativeText = prompt, AccessoryView = picker }; alert.AddButton("OK"); alert.AddButton("Cancel");
return alert.RunModal() == 1000 ? checked((int)picker.IndexOfSelectedItem) : -1;
}
internal static bool Confirm(string title, string detail) { var alert = new NSAlert { MessageText = title, InformativeText = detail, AlertStyle = NSAlertStyle.Warning }; alert.AddButton("Continue"); alert.AddButton("Cancel"); return alert.RunModal() == 1000; }
}
internal sealed record ChannelEdit(Channel Channel, string Password);
internal static class PermissionEditor
{
internal static Permissions? Run()
{
string[] labels = ["Create temporary channels", "Kick users", "Ban users", "Move users", "Manage accounts", "Full administrator"];
var view = new NSView(new CGRect(0, 0, 330, 160)); var checks = new List<NSButton>();
for (int i = 0; i < labels.Length; i++) { NSButton check = NSButton.CreateCheckbox(labels[i], () => { }); check.Frame = new CGRect(0, 135 - i * 25, 320, 22); view.AddSubview(check); checks.Add(check); }
var alert = new NSAlert { MessageText = "Set permissions", AccessoryView = view }; alert.AddButton("Save"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return null;
return new() { CanCreateTempChannel = On(0), CanKick = On(1), CanBan = On(2), CanMoveUsers = On(3), CanAdminAccounts = On(4), IsAdmin = On(5) };
bool On(int i) => checks[i].State == NSCellStateValue.On;
}
}
internal static class PerUserTuning
{
internal static void Run(VoiceCatClient client, User user)
{
if (user.Streams.Count == 0) { AdministrationWindowController.PromptText("Audio tuning", "This user has no active streams."); return; }
var stream = new NSPopUpButton(new CGRect(0, 95, 340, 28), false); stream.AddItems(user.Streams.Select(value => $"{value.Label} ({value.Kind})").ToArray());
var gain = new NSSlider(new CGRect(0, 60, 340, 24)) { MinValue = 0, MaxValue = 400, DoubleValue = 100 };
var muted = NSButton.CreateCheckbox("Mute this stream", () => { }); muted.Frame = new CGRect(0, 30, 160, 24);
var noise = NSButton.CreateCheckbox("Receive noise reduction", () => { }); noise.Frame = new CGRect(170, 30, 170, 24);
var view = new NSView(new CGRect(0, 0, 340, 130)); view.AddSubview(stream); view.AddSubview(gain); view.AddSubview(muted); view.AddSubview(noise);
void LoadSelected()
{
if (stream.IndexOfSelectedItem < 0) return;
StreamInfo selected = user.Streams[checked((int)stream.IndexOfSelectedItem)];
(float Gain, bool Muted, bool NoiseReduction)? state = client.Audio.GetRemotePlayback(user.Id, selected.StreamId);
gain.DoubleValue = (state?.Gain ?? 1) * 100; muted.State = state?.Muted == true ? NSCellStateValue.On : NSCellStateValue.Off;
noise.State = state?.NoiseReduction == true ? NSCellStateValue.On : NSCellStateValue.Off;
}
stream.Activated += (_, _) => LoadSelected(); LoadSelected();
var alert = new NSAlert { MessageText = $"Audio — {user.Nickname}", InformativeText = "Stream, volume, mute, and receive noise reduction", AccessoryView = view };
alert.AddButton("Apply"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return;
StreamInfo selected = user.Streams[checked((int)stream.IndexOfSelectedItem)]; client.Audio.SetRemotePlayback(user.Id, selected.StreamId, (float)gain.DoubleValue / 100,
muted.State == NSCellStateValue.On, noise.State == NSCellStateValue.On);
}
}
internal static class AccountEditor
{
internal static async Task RunAsync(VoiceCatClient client)
{
IReadOnlyList<AccountEntry> accounts = await client.ListAccountsAsync();
string[] actions = ["Create account", "Reset password", "Delete account", "Close"];
int action = AdministrationWindowController.PromptChoice("Server accounts", string.Join("\n", accounts.Select(a => $"{a.Username}{(a.IsAdmin ? " (admin)" : "")}")), actions);
if (action is < 0 or 3) return;
if (action == 0)
{
string? name = AdministrationWindowController.PromptText("Create account", "Username"); if (string.IsNullOrWhiteSpace(name)) return;
string? password = AdministrationWindowController.PromptText("Create account", "Password", secure: true); if (!string.IsNullOrEmpty(password)) await client.CreateAccountAsync(name.Trim(), password);
return;
}
int selected = AdministrationWindowController.PromptChoice(action == 1 ? "Reset password" : "Delete account", "Account", accounts.Select(a => a.Username).ToArray()); if (selected < 0) return;
if (action == 1) { string? password = AdministrationWindowController.PromptText("Reset password", "New password", secure: true); if (!string.IsNullOrEmpty(password)) await client.ResetPasswordAsync(accounts[selected].Username, password); }
else if (AdministrationWindowController.Confirm("Delete account?", accounts[selected].Username)) await client.DeleteAccountAsync(accounts[selected].Username);
}
}
+22
View File
@@ -0,0 +1,22 @@
using AppKit;
using Foundation;
namespace VoiceCat.Mac;
internal sealed class AppDelegate : NSApplicationDelegate
{
private ConnectWindowController? connect;
public override void DidFinishLaunching(NSNotification notification)
{
NSApplication.SharedApplication.ActivationPolicy = NSApplicationActivationPolicy.Regular;
BuildMenu();
connect = new(); connect.ShowWindow(this);
}
public override bool ApplicationShouldTerminateAfterLastWindowClosed(NSApplication sender) => true;
private static void BuildMenu()
{
var menu = new NSMenu(); var root = new NSMenuItem(); menu.AddItem(root);
var application = new NSMenu(); application.AddItem(new NSMenuItem("Quit VoiceCat", "q", (_, _) => NSApplication.SharedApplication.Terminate(null)));
root.Submenu = application; NSApplication.SharedApplication.MainMenu = menu;
}
}
@@ -0,0 +1,68 @@
using AppKit;
using CoreGraphics;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.Mac;
internal static class ChannelEditor
{
internal static ChannelEdit? Run(Channel? existing, IReadOnlyList<Channel> channels, bool supportsAdaptivePacketLoss)
{
var view = new NSView(new CGRect(0, 0, 520, 500));
var name = Field(existing?.Name ?? "", 360); var topic = Field(existing?.Topic ?? "", 330);
var parent = Picker(channels.Select(channel => channel.Name).Prepend("No parent").ToArray(), 300);
int parentIndex = existing is null ? 0 : channels.ToList().FindIndex(channel => channel.Id == existing.ParentId) + 1; parent.SelectItem(Math.Max(0, parentIndex));
var type = Picker(["Permanent", "Temporary"], 270); type.SelectItem(existing?.Type == ChannelType.ChannelTemporary ? 1 : 0);
var maximum = Field((existing?.MaxUsers ?? 0).ToString(), 240); var order = Field((existing?.Order ?? 0).ToString(), 210);
var password = new NSSecureTextField(new CGRect(155, 180, 345, 24)) { PlaceholderString = existing?.PasswordProtected == true ? "Leave blank to preserve" : "Optional" };
var mode = Picker(["Mono", "Stereo"], 150); mode.SelectItem(existing?.Audio?.Mode == ChannelMode.ModeStereo ? 1 : 0);
var rate = Picker(["8000", "12000", "16000", "24000", "48000"], 120); Select(rate, (existing?.Audio?.SampleRate ?? 48000).ToString());
var frame = Picker(["5", "10", "20", "40", "60"], 90); Select(frame, (existing?.Audio?.FrameMs ?? 20).ToString());
var application = Picker(["VoIP", "Audio", "Low delay"], 60); application.SelectItem((int)(existing?.Audio?.Application ?? OpusApplication.OpusVoip));
var bitrate = Field((existing?.Audio?.BitrateBps ?? 64000).ToString(), 30);
Add(view, "Name", name, 360); Add(view, "Topic", topic, 330); Add(view, "Parent", parent, 300); Add(view, "Type", type, 270);
Add(view, "Maximum users (0=none)", maximum, 240); Add(view, "Sort order", order, 210); Add(view, "Password", password, 180);
Add(view, "Mode", mode, 150); Add(view, "Sample rate", rate, 120); Add(view, "Frame ms", frame, 90); Add(view, "Application", application, 60); Add(view, "Bitrate", bitrate, 30);
var advanced = new NSView(new CGRect(0, 0, 520, 175));
var automaticLoss = Check("Automatic packet loss", existing?.Audio?.PacketLossMode != PacketLossMode.PacketLossManual, 155, 140);
automaticLoss.Frame = new CGRect(155, 140, 200, 24);
automaticLoss.Enabled = supportsAdaptivePacketLoss;
var lossSpeed = Picker(["Fast", "Balanced", "Stable"], 105);
lossSpeed.SelectItem(existing?.Audio?.PacketLossMode switch { PacketLossMode.PacketLossAutoFast => 0, PacketLossMode.PacketLossAutoStable => 2, _ => 1 });
var loss = Field((existing?.Audio?.ExpectedPacketLoss ?? 5).ToString(), 70); loss.Frame = new CGRect(155, 70, 100, 24);
var complexity = Field((existing?.Audio?.Complexity ?? 10).ToString(), 40); complexity.Frame = new CGRect(155, 40, 100, 24);
var fec = Check("In-band FEC", existing?.Audio?.Fec ?? true, 0, 10); var dtx = Check("DTX", existing?.Audio?.Dtx ?? true, 130, 10);
var dred = Check("Deep redundancy", existing?.Audio?.Dred ?? false, 230, 10);
void UpdateLossControls() { loss.Enabled = !On(automaticLoss); lossSpeed.Enabled = supportsAdaptivePacketLoss && On(automaticLoss); }
automaticLoss.Activated += (_, _) => UpdateLossControls(); UpdateLossControls();
advanced.AddSubview(automaticLoss); Add(advanced, "Adaptation speed", lossSpeed, 105); Add(advanced, "Packet loss %", loss, 70); Add(advanced, "Complexity 010", complexity, 40);
advanced.AddSubview(fec); advanced.AddSubview(dtx); advanced.AddSubview(dred);
var container = new NSView(new CGRect(0, 0, 520, 690)); view.Frame = new CGRect(0, 180, 520, 500); container.AddSubview(view); container.AddSubview(advanced);
var alert = new NSAlert { MessageText = existing is null ? "Create channel" : "Edit channel", AccessoryView = container };
alert.AddButton(existing is null ? "Create" : "Save"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return null;
if (string.IsNullOrWhiteSpace(name.StringValue) || !uint.TryParse(maximum.StringValue, out uint max) || !int.TryParse(order.StringValue, out int sort) ||
!uint.TryParse(rate.TitleOfSelectedItem, out uint sampleRate) || !uint.TryParse(frame.TitleOfSelectedItem, out uint frameMs) ||
!uint.TryParse(bitrate.StringValue, out uint bitrateValue) || !uint.TryParse(loss.StringValue, out uint packetLoss) ||
!uint.TryParse(complexity.StringValue, out uint complexityValue) || packetLoss > 100 || complexityValue > 10) return null;
uint parentId = parent.IndexOfSelectedItem <= 0 ? 0 : channels[checked((int)parent.IndexOfSelectedItem - 1)].Id;
return new(new Channel
{
Id = existing?.Id ?? 0, ParentId = parentId, Name = name.StringValue.Trim(), Topic = topic.StringValue.Trim(), MaxUsers = max, Order = sort,
Type = type.IndexOfSelectedItem == 1 ? ChannelType.ChannelTemporary : ChannelType.ChannelPermanent,
Audio = new AudioConfig { Codec = 0, Mode = mode.IndexOfSelectedItem == 1 ? ChannelMode.ModeStereo : ChannelMode.ModeMono,
SampleRate = sampleRate, FrameMs = frameMs, Application = (OpusApplication)(int)application.IndexOfSelectedItem, BitrateBps = bitrateValue,
Fec = On(fec), Dtx = On(dtx), Dred = On(dred), ExpectedPacketLoss = packetLoss, Complexity = complexityValue,
PacketLossMode = On(automaticLoss) ? lossSpeed.IndexOfSelectedItem switch
{ 0 => PacketLossMode.PacketLossAutoFast, 2 => PacketLossMode.PacketLossAutoStable, _ => PacketLossMode.PacketLossAutoBalanced } : PacketLossMode.PacketLossManual }
}, password.StringValue);
}
private static NSTextField Field(string value, double y) => new(new CGRect(155, y, 345, 24)) { StringValue = value };
private static NSPopUpButton Picker(string[] values, double y) { var picker = new NSPopUpButton(new CGRect(155, y, 345, 26), false); picker.AddItems(values); return picker; }
private static NSButton Check(string title, bool value, double x, double y) { NSButton check = NSButton.CreateCheckbox(title, () => { }); check.Frame = new CGRect(x, y, 125, 24); check.State = value ? NSCellStateValue.On : NSCellStateValue.Off; return check; }
private static bool On(NSButton value) => value.State == NSCellStateValue.On;
private static void Add(NSView view, string label, NSView control, double y) { var text = NSTextField.CreateLabel(label); text.Frame = new CGRect(0, y + 2, 145, 20); view.AddSubview(text); view.AddSubview(control); ((INSAccessibility)control).AccessibilityLabel = label; }
private static void Select(NSPopUpButton picker, string value) { for (nint i = 0; i < picker.ItemCount; i++) if (picker.ItemTitle(i) == value) { picker.SelectItem(i); return; } }
}
@@ -0,0 +1,217 @@
using AppKit;
using CoreGraphics;
using VoiceCat.Core;
using VoiceCat.Crypto;
namespace VoiceCat.Mac;
internal sealed class ConnectWindowController : NSWindowController
{
private static readonly string SupportDirectory = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "VoiceCat");
private readonly ServerProfileStore profileStore = new(Path.Combine(SupportDirectory, "servers.json"));
private readonly MacKeychainPasswordStore keychain = new();
private readonly NSPopUpButton profiles = new(new CGRect(120, 315, 330, 28), false);
private readonly NSTextField host = new(new CGRect(120, 270, 330, 26)) { StringValue = "127.0.0.1:8384", PlaceholderString = "Host:port" };
private readonly NSPopUpButton authentication = new(new CGRect(120, 225, 180, 28), false);
private readonly NSTextField identity = new(new CGRect(120, 180, 330, 26)) { StringValue = Environment.UserName };
private readonly NSTextField identityLabel;
private readonly NSSecureTextField password = new(new CGRect(120, 135, 330, 26)) { PlaceholderString = "Password" };
private readonly NSButton rememberPassword = NSButton.CreateCheckbox("Remember in Keychain", () => { });
private readonly NSTextField status = NSTextField.CreateLabel("Ready to connect");
private readonly NSButton save = new(new CGRect(120, 75, 90, 32)) { Title = "Save" };
private readonly NSButton remove = new(new CGRect(220, 75, 90, 32)) { Title = "Remove" };
private readonly NSButton connect = new(new CGRect(360, 75, 90, 32)) { Title = "Connect", BezelStyle = NSBezelStyle.Rounded };
private List<ServerProfile> saved = [];
private Guid? editingId;
private VoiceCatClient? client;
private MainWindowController? main;
internal ConnectWindowController() : base(new NSWindow(new CGRect(0, 0, 570, 390), NSWindowStyle.Titled | NSWindowStyle.Closable,
NSBackingStore.Buffered, false))
{
Window!.Title = "Connect to VoiceCat"; Window.Center();
var view = Window.ContentView!;
AddLabel(view, "Saved server", 25, 320); AddLabel(view, "Server", 25, 275); AddLabel(view, "Authentication", 25, 230);
identityLabel = AddLabel(view, "Nickname", 25, 185); AddLabel(view, "Password", 25, 140);
authentication.AddItem("Guest"); authentication.AddItem("Account"); authentication.Activated += AuthenticationChanged;
profiles.Activated += ProfileChanged;
((INSAccessibility)profiles).AccessibilityLabel = "Saved servers";
((INSAccessibility)host).AccessibilityLabel = "Server hostname and port";
((INSAccessibility)authentication).AccessibilityLabel = "Authentication mode";
((INSAccessibility)identity).AccessibilityLabel = "Guest nickname";
((INSAccessibility)password).AccessibilityLabel = "Account password";
rememberPassword.Frame = new CGRect(310, 225, 180, 24);
((INSAccessibility)rememberPassword).AccessibilityLabel = "Remember account password in Keychain";
view.AddSubview(profiles); view.AddSubview(host); view.AddSubview(authentication); view.AddSubview(identity); view.AddSubview(password); view.AddSubview(rememberPassword);
status.Frame = new CGRect(25, 30, 520, 24); status.AccessibilityLabel = "Connection status"; view.AddSubview(status);
((INSAccessibility)save).AccessibilityLabel = "Save server profile"; save.Activated += SaveProfile; view.AddSubview(save);
((INSAccessibility)remove).AccessibilityLabel = "Remove server profile"; remove.Activated += RemoveProfile; view.AddSubview(remove);
((INSAccessibility)connect).AccessibilityLabel = "Connect to server"; connect.Activated += Connect; view.AddSubview(connect);
Window.DefaultButtonCell = connect.Cell;
ReloadProfiles(); UpdateAuthentication();
}
private static NSTextField AddLabel(NSView view, string text, double x, double y)
{
var label = NSTextField.CreateLabel(text); label.Frame = new CGRect(x, y, 90, 20); view.AddSubview(label); return label;
}
private void ReloadProfiles(Guid? select = null)
{
saved = profileStore.Load().ToList();
profiles.RemoveAllItems(); profiles.AddItem("New server…");
foreach (ServerProfile profile in saved) profiles.AddItem(profile.DisplayName);
int index = select is null ? (saved.Count == 0 ? 0 : 1) : saved.FindIndex(item => item.Id == select) + 1;
profiles.SelectItem(Math.Max(0, index));
ApplySelectedProfile();
}
private void ProfileChanged(object? sender, EventArgs args) => ApplySelectedProfile();
private void ApplySelectedProfile()
{
int index = checked((int)profiles.IndexOfSelectedItem) - 1;
if (index < 0 || index >= saved.Count)
{
editingId = null; remove.Enabled = false; password.StringValue = "";
rememberPassword.State = NSCellStateValue.Off; UpdateAuthentication(); return;
}
ServerProfile profile = saved[index]; editingId = profile.Id; remove.Enabled = true;
host.StringValue = FormatEndpoint(profile.Host, profile.Port);
authentication.SelectItem(profile.Authentication == ServerAuthentication.Guest ? 0 : 1);
identity.StringValue = profile.Authentication == ServerAuthentication.Guest ? profile.Nickname ?? Environment.UserName : profile.Username ?? "";
string? storedPassword = profile.Authentication == ServerAuthentication.Account
? keychain.Load(profile.Id) ?? keychain.LoadLegacy(profile.LegacyKeychainTag) : null;
password.StringValue = storedPassword ?? "";
rememberPassword.State = storedPassword is null ? NSCellStateValue.Off : NSCellStateValue.On;
UpdateAuthentication();
}
private void AuthenticationChanged(object? sender, EventArgs args) => UpdateAuthentication();
private void UpdateAuthentication()
{
bool account = authentication.IndexOfSelectedItem == 1;
identityLabel.StringValue = account ? "Username" : "Nickname";
identity.PlaceholderString = account ? "Username" : "Nickname";
((INSAccessibility)identity).AccessibilityLabel = account ? "Account username" : "Guest nickname";
password.Enabled = account;
rememberPassword.Enabled = account && editingId is not null;
}
private void SaveProfile(object? sender, EventArgs args)
{
try
{
ServerProfile profile = ReadProfile();
string pendingPassword = password.StringValue;
bool shouldRemember = rememberPassword.State == NSCellStateValue.On;
int index = saved.FindIndex(item => item.Id == profile.Id);
if (index < 0) saved.Add(profile); else saved[index] = profile;
if (profile.Authentication == ServerAuthentication.Guest || !shouldRemember) keychain.Remove(profile.Id);
profileStore.Save(saved); ReloadProfiles(profile.Id);
if (profile.Authentication == ServerAuthentication.Account && shouldRemember)
{
password.StringValue = pendingPassword;
rememberPassword.State = NSCellStateValue.On;
}
status.StringValue = "Server profile saved. Passwords are stored only after successful login.";
}
catch (Exception exception) { status.StringValue = exception.Message; }
}
private void RemoveProfile(object? sender, EventArgs args)
{
if (editingId is not { } id) return;
ServerProfile? profile = saved.FirstOrDefault(item => item.Id == id); if (profile is null) return;
var alert = new NSAlert { MessageText = "Remove server?", InformativeText = profile.DisplayName, AlertStyle = NSAlertStyle.Warning };
alert.AddButton("Remove"); alert.AddButton("Cancel"); if (alert.RunModal() != 1000) return;
keychain.Remove(profile.Id); saved.Remove(profile); profileStore.Save(saved); editingId = null; ReloadProfiles(); status.StringValue = "Server profile removed.";
}
private async void Connect(object? sender, EventArgs args)
{
if (client is not null) return;
try
{
SetBusy(true); status.StringValue = "Connecting…";
ServerProfile profile = ReadProfile();
string pins = Path.Combine(SupportDirectory, "tofu_pins.txt");
client = new("VoiceCat macOS", "0.1.0", pins);
await client.ConnectAsync(profile.Host, profile.Port, ConfirmIdentity);
string accountPassword = password.StringValue;
if (profile.Authentication == ServerAuthentication.Account && accountPassword.Length == 0)
accountPassword = keychain.Load(profile.Id) ?? keychain.LoadLegacy(profile.LegacyKeychainTag) ?? "";
if (profile.Authentication == ServerAuthentication.Account && accountPassword.Length == 0) throw new ArgumentException("Enter the account password.");
password.StringValue = "";
var result = profile.Authentication == ServerAuthentication.Guest
? await client.AuthenticateGuestAsync(profile.Nickname ?? Environment.UserName)
: await client.AuthenticateUserAsync(profile.Username!, accountPassword);
if (!result.Ok) throw new InvalidOperationException(result.Error);
try
{
if (profile.Authentication == ServerAuthentication.Account && rememberPassword.State == NSCellStateValue.On) keychain.Save(profile.Id, accountPassword);
else keychain.Remove(profile.Id);
}
catch (Exception exception) { ShowKeychainWarning(exception.Message); }
main = new(client, result.Self.Id, result.Self.Nickname);
client = null; main.ShowWindow(this); Window.Close();
}
catch (Exception exception)
{
if (client is not null) await client.DisposeAsync(); client = null;
status.StringValue = exception.Message; SetBusy(false);
}
}
private ServerProfile ReadProfile()
{
(string serverHost, ushort port) = ParseEndpoint(host.StringValue);
bool account = authentication.IndexOfSelectedItem == 1;
string name = identity.StringValue.Trim();
return ServerProfile.Create(serverHost, port, account ? ServerAuthentication.Account : ServerAuthentication.Guest,
username: account ? name : null, nickname: account ? null : name, id: editingId);
}
private void SetBusy(bool busy)
{
profiles.Enabled = host.Enabled = authentication.Enabled = identity.Enabled = save.Enabled = connect.Enabled = !busy;
remove.Enabled = !busy && editingId is not null;
password.Enabled = !busy && authentication.IndexOfSelectedItem == 1;
rememberPassword.Enabled = !busy && authentication.IndexOfSelectedItem == 1 && editingId is not null;
}
private ValueTask<bool> ConfirmIdentity(ServerIdentityChallenge challenge, CancellationToken cancellationToken)
{
var completion = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
NSApplication.SharedApplication.InvokeOnMainThread(() =>
{
var alert = new NSAlert { MessageText = challenge.Status == TofuStatus.FirstConnect ? "Trust this VoiceCat server?" : "Server identity changed",
InformativeText = $"{challenge.Host}:{challenge.Port}\n\nCertificate SHA-256:\n{challenge.CertificateFingerprint}", AlertStyle = challenge.Status == TofuStatus.Mismatch ? NSAlertStyle.Critical : NSAlertStyle.Informational };
alert.AddButton("Trust and connect"); alert.AddButton("Cancel"); completion.TrySetResult(alert.RunModal() == 1000);
});
return new(completion.Task.WaitAsync(cancellationToken));
}
private static void ShowKeychainWarning(string message)
{
var alert = new NSAlert
{
MessageText = "Password storage failed",
InformativeText = message + " You are still signed in, but the Keychain setting was not changed.",
AlertStyle = NSAlertStyle.Warning
};
alert.RunModal();
}
private static (string Host, ushort Port) ParseEndpoint(string value)
{
int separator = value.LastIndexOf(':');
if (separator <= 0 || !ushort.TryParse(value[(separator + 1)..], out ushort port) || port == 0) throw new ArgumentException("Enter a server as host:port.");
string parsedHost = value[..separator].Trim().Trim('[', ']');
if (parsedHost.Length == 0) throw new ArgumentException("Enter a server as host:port.");
return (parsedHost, port);
}
private static string FormatEndpoint(string serverHost, ushort port) => serverHost.Contains(':') ? $"[{serverHost}]:{port}" : $"{serverHost}:{port}";
}
@@ -0,0 +1,113 @@
using System.Runtime.InteropServices;
using Foundation;
using ObjCRuntime;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal static class CoreAudioDevices
{
private const string CoreAudio = "/System/Library/Frameworks/CoreAudio.framework/CoreAudio";
private const uint SystemObject = 1;
private const uint Devices = 0x64657623; // 'dev#'
private const uint DefaultInput = 0x64496e20; // 'dIn '
private const uint DefaultOutput = 0x644f7574; // 'dOut'
private const uint StreamConfiguration = 0x736c6179; // 'slay'
private const uint ObjectName = 0x6c6e616d; // 'lnam'
private const uint ScopeGlobal = 0x676c6f62; // 'glob'
private const uint ScopeInput = 0x696e7074; // 'inpt'
private const uint ScopeOutput = 0x6f757470; // 'outp'
[StructLayout(LayoutKind.Sequential)]
private struct PropertyAddress(uint selector, uint scope)
{
public uint Selector = selector;
public uint Scope = scope;
public uint Element;
}
[DllImport(CoreAudio)]
private static extern int AudioObjectGetPropertyDataSize(uint objectId, ref PropertyAddress address,
uint qualifierDataSize, nint qualifierData, out uint dataSize);
[DllImport(CoreAudio)]
private static extern int AudioObjectGetPropertyData(uint objectId, ref PropertyAddress address,
uint qualifierDataSize, nint qualifierData, ref uint dataSize, nint data);
internal static IReadOnlyList<AudioDeviceInfo> List(bool input)
{
uint defaultId = ReadUInt(SystemObject, input ? DefaultInput : DefaultOutput, ScopeGlobal);
var devices = new List<AudioDeviceInfo>();
foreach (uint id in ReadUIntArray(SystemObject, Devices, ScopeGlobal))
{
if (ChannelCount(id, input ? ScopeInput : ScopeOutput) == 0) continue;
string name = ReadString(id, ObjectName, ScopeGlobal) ?? $"Core Audio device {id}";
devices.Add(new(id.ToString(System.Globalization.CultureInfo.InvariantCulture), name, id == defaultId));
}
return devices.OrderByDescending(device => device.IsDefault).ThenBy(device => device.Name, StringComparer.CurrentCultureIgnoreCase).ToArray();
}
internal static uint ParseId(string? value)
{
if (!uint.TryParse(value, System.Globalization.NumberStyles.None, System.Globalization.CultureInfo.InvariantCulture, out uint id) || id == 0)
throw new ArgumentException("The selected Core Audio device is no longer available.", nameof(value));
return id;
}
private static uint ReadUInt(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
uint size = sizeof(uint);
nint memory = Marshal.AllocHGlobal(sizeof(uint));
try { return AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) == 0 ? unchecked((uint)Marshal.ReadInt32(memory)) : 0; }
finally { Marshal.FreeHGlobal(memory); }
}
private static uint[] ReadUIntArray(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
if (AudioObjectGetPropertyDataSize(objectId, ref address, 0, 0, out uint size) != 0 || size < sizeof(uint)) return [];
nint memory = Marshal.AllocHGlobal(checked((int)size));
try
{
if (AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) != 0) return [];
var result = new uint[size / sizeof(uint)];
for (int i = 0; i < result.Length; i++) result[i] = unchecked((uint)Marshal.ReadInt32(memory, i * sizeof(uint)));
return result;
}
finally { Marshal.FreeHGlobal(memory); }
}
private static int ChannelCount(uint deviceId, uint scope)
{
var address = new PropertyAddress(StreamConfiguration, scope);
if (AudioObjectGetPropertyDataSize(deviceId, ref address, 0, 0, out uint size) != 0 || size < 8) return 0;
nint memory = Marshal.AllocHGlobal(checked((int)size));
try
{
if (AudioObjectGetPropertyData(deviceId, ref address, 0, 0, ref size, memory) != 0) return 0;
int count = Marshal.ReadInt32(memory);
int first = IntPtr.Size == 8 ? 8 : 4;
int stride = IntPtr.Size == 8 ? 16 : 12;
int channels = 0;
for (int i = 0; i < count && first + i * stride + sizeof(uint) <= size; i++)
channels += Marshal.ReadInt32(memory, first + i * stride);
return channels;
}
finally { Marshal.FreeHGlobal(memory); }
}
private static string? ReadString(uint objectId, uint selector, uint scope)
{
var address = new PropertyAddress(selector, scope);
uint size = checked((uint)IntPtr.Size);
nint memory = Marshal.AllocHGlobal(IntPtr.Size);
try
{
if (AudioObjectGetPropertyData(objectId, ref address, 0, 0, ref size, memory) != 0) return null;
nint handle = Marshal.ReadIntPtr(memory);
return handle == 0 ? null : Runtime.GetNSObject<NSString>(handle, owns: false)?.ToString();
}
finally { Marshal.FreeHGlobal(memory); }
}
}
@@ -0,0 +1,49 @@
using AVFoundation;
using Foundation;
namespace VoiceCat.Mac;
internal enum SoundEvent { ChannelJoin, ChannelLeave, ChannelReceived, ChannelSent, PrivateReceived, PrivateSent, Login, Logout, ConnectionLost, VoiceOn, VoiceOff, VoiceStart, VoiceStop, PushToTalk }
internal sealed class EventFeedback : IDisposable
{
private readonly MacSettings settings;
private readonly AVSpeechSynthesizer speech = new();
private readonly Dictionary<SoundEvent, AVAudioPlayer> players = [];
internal EventFeedback(MacSettings settings) => this.settings = settings;
internal void Play(SoundEvent sound)
{
if (!settings.EventSounds || settings.EventVolume <= 0 ||
(sound is SoundEvent.VoiceStart or SoundEvent.VoiceStop && !settings.SelfTalkSounds) ||
(sound == SoundEvent.PushToTalk && !settings.PushToTalkSound)) return;
if (!players.TryGetValue(sound, out AVAudioPlayer? player))
{
string path = Path.Combine(NSBundle.MainBundle.ResourcePath ?? "", "Sounds", FileName(sound) + ".wav");
if (!File.Exists(path)) return;
player = AVAudioPlayer.FromUrl(NSUrl.FromFilename(path));
if (player is null) return;
player.PrepareToPlay(); players[sound] = player;
}
player.Volume = settings.EventVolume; player.CurrentTime = 0; player.Play();
}
internal void Speak(string text)
{
if (!settings.SpokenEvents || string.IsNullOrWhiteSpace(text)) return;
speech.SpeakUtterance(new AVSpeechUtterance(text.Trim()));
}
private static string FileName(SoundEvent value) => value switch
{
SoundEvent.ChannelJoin => "channel_join", SoundEvent.ChannelLeave => "channel_leave",
SoundEvent.ChannelReceived => "channel_recv", SoundEvent.ChannelSent => "channel_sent",
SoundEvent.PrivateReceived => "pm_recv", SoundEvent.PrivateSent => "pm_sent",
SoundEvent.Login => "login", SoundEvent.Logout => "logout", SoundEvent.ConnectionLost => "connection_lost",
SoundEvent.VoiceOn => "voice_on", SoundEvent.VoiceOff => "voice_off", SoundEvent.VoiceStart => "va_start",
SoundEvent.VoiceStop => "va_stop", _ => "ptt"
};
public void Dispose() { foreach (AVAudioPlayer player in players.Values) player.Dispose(); speech.Dispose(); }
}
+13
View File
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>CFBundleDisplayName</key><string>VoiceCat</string>
<key>CFBundleIdentifier</key><string>net.iamtalon.voicecat</string>
<key>CFBundleName</key><string>VoiceCat</string>
<key>CFBundleVersion</key><string>1</string>
<key>CFBundleShortVersionString</key><string>0.1.0</string>
<key>LSMinimumSystemVersion</key><string>14.0</string>
<key>NSMicrophoneUsageDescription</key><string>VoiceCat uses the microphone when you join voice and enable a microphone stream.</string>
<key>NSScreenCaptureUsageDescription</key><string>VoiceCat uses ScreenCaptureKit only when you share desktop or application audio.</string>
<key>NSPrincipalClass</key><string>NSApplication</string>
</dict></plist>
@@ -0,0 +1,243 @@
using System.Runtime.InteropServices;
using AudioUnit;
using AVFoundation;
using Foundation;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal sealed class MacAudioBackend : IAudioDeviceBackend
{
private Exception? failure;
internal Exception? Failure => Volatile.Read(ref failure);
public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => CoreAudioDevices.List(input);
public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler pcm)
{
if (loopback) throw new NotSupportedException("Screen audio uses ScreenCaptureKit, not microphone capture.");
return new Capture(deviceId, pcm, ReportFailure);
}
public IAudioPlayback OpenPlayback(string? deviceId = null)
{
return new Playback(deviceId, ReportFailure);
}
private void ReportFailure(Exception exception) => Interlocked.CompareExchange(ref failure, exception, null);
private sealed class Capture : IAudioCapture
{
private readonly CapturePcmHandler handler;
private readonly AVAudioEngine engine = new();
private readonly AVAudioFormat outputFormat;
private readonly AVAudioConverter converter;
private readonly AVAudioPcmBuffer converted;
private readonly AVAudioConverterInputHandler inputProvider;
private readonly Action<Exception> reportFailure;
private AVAudioPcmBuffer? pendingInput;
private bool inputProvided;
private int disposed;
internal Capture(string? deviceId, CapturePcmHandler handler, Action<Exception> reportFailure)
{
this.handler = handler;
this.reportFailure = reportFailure;
AVAudioInputNode input = engine.InputNode;
if (!string.IsNullOrEmpty(deviceId))
{
AudioUnitStatus status = input.AudioUnit!.SetCurrentDevice(CoreAudioDevices.ParseId(deviceId), AudioUnitScopeType.Global, 0);
if (status != AudioUnitStatus.NoError) throw new InvalidOperationException($"Core Audio input selection failed ({status}).");
}
AVAudioFormat inputFormat = input.GetBusOutputFormat(0);
uint channels = Math.Clamp(inputFormat.ChannelCount, 1u, 2u);
outputFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, channels, true);
converter = new(inputFormat, outputFormat);
uint outputCapacity = checked((uint)Math.Ceiling(4_096 * 48_000 / inputFormat.SampleRate) + 64);
converted = new(outputFormat, outputCapacity);
inputProvider = ProvideInput;
NSError? tapError = null;
if (OperatingSystem.IsMacOSVersionAtLeast(27))
input.InstallTapOnBus(0, 960, inputFormat, out tapError, Convert);
else
input.InstallTapOnBus(0, 960, inputFormat, Convert);
if (tapError is not null)
{
converted.Dispose();
converter.Dispose();
outputFormat.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio capture tap could not be installed: " + tapError.LocalizedDescription);
}
engine.Prepare();
if (!engine.StartAndReturnError(out var error))
{
input.RemoveTapOnBus(0);
converted.Dispose();
converter.Dispose();
outputFormat.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio capture could not start: " + error.LocalizedDescription);
}
}
private unsafe void Convert(AVAudioPcmBuffer input, AVAudioTime _time)
{
try
{
pendingInput = input;
inputProvided = false;
converted.FrameLength = 0;
AVAudioConverterOutputStatus result = converter.ConvertToBuffer(converted, out NSError? error, inputProvider);
if (result == AVAudioConverterOutputStatus.Error)
{
reportFailure(new InvalidOperationException("Core Audio input conversion failed: " + error?.LocalizedDescription));
return;
}
if (converted.FrameLength == 0) return;
nint samples = Marshal.ReadIntPtr(converted.Int16ChannelData);
if (samples == 0) return;
int channels = checked((int)outputFormat.ChannelCount);
int remainingFrames = checked((int)converted.FrameLength);
var pcm = new ReadOnlySpan<short>((void*)samples, checked(remainingFrames * channels));
while (remainingFrames > 0)
{
int frames = Math.Min(960, remainingFrames);
handler(pcm[..(frames * channels)], channels);
pcm = pcm[(frames * channels)..];
remainingFrames -= frames;
}
}
catch (Exception exception) { reportFailure(exception); }
finally { pendingInput = null; }
}
private AVAudioBuffer ProvideInput(uint _, out AVAudioConverterInputStatus status)
{
if (!inputProvided && pendingInput is { } input)
{
inputProvided = true;
status = AVAudioConverterInputStatus.HaveData;
return input;
}
status = AVAudioConverterInputStatus.NoDataNow;
return null!;
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
engine.InputNode.RemoveTapOnBus(0);
engine.Stop();
converted.Dispose();
converter.Dispose();
outputFormat.Dispose();
engine.Dispose();
}
}
private sealed class Playback : IAudioPlayback
{
private readonly AdaptivePcmBuffer pcm = new(2);
private readonly short[] renderScratch = new short[8_192];
private readonly AVAudioEngine engine = new();
private readonly AVAudioFormat format;
private readonly AVAudioSourceNode source;
private readonly Action<Exception> reportFailure;
private int callbackFailed;
private int disposed;
internal Playback(string? deviceId, Action<Exception> reportFailure)
{
this.reportFailure = reportFailure;
if (!string.IsNullOrEmpty(deviceId))
{
AudioUnitStatus status = engine.OutputNode.AudioUnit!.SetCurrentDevice(CoreAudioDevices.ParseId(deviceId), AudioUnitScopeType.Global, 0);
if (status != AudioUnitStatus.NoError) throw new InvalidOperationException($"Core Audio output selection failed ({status}).");
}
// AVAudioEngine's native mixer path is planar Float32. Convert from the managed
// Int16 ring directly in this allocation-free callback, avoiding an extra graph
// converter and matching the established Swift/VPIO renderer.
format = new(AVAudioCommonFormat.PCMFloat32, 48_000, 2, false);
source = new(format, Render);
engine.AttachNode(source);
NSError? connectionError = null;
if (OperatingSystem.IsMacOSVersionAtLeast(27))
engine.Connect(source, engine.MainMixerNode, format, out connectionError);
else
engine.Connect(source, engine.MainMixerNode, format);
if (connectionError is not null)
{
engine.DetachNode(source);
source.Dispose();
format.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio playback could not be connected: " + connectionError.LocalizedDescription);
}
engine.Prepare();
if (!engine.StartAndReturnError(out var error))
{
engine.DetachNode(source);
source.Dispose();
format.Dispose();
engine.Dispose();
throw new InvalidOperationException("Core Audio playback could not start: " + error.LocalizedDescription);
}
}
public void Write(ReadOnlySpan<short> stereoPcm) => pcm.TryWrite(stereoPcm);
public int BufferMilliseconds { get => pcm.BufferMilliseconds; set => pcm.BufferMilliseconds = value; }
private unsafe int Render(IntPtr isSilence, IntPtr _, uint frameCount, IntPtr outputData)
{
try
{
int bufferCount = Marshal.ReadInt32(outputData);
int firstBuffer = IntPtr.Size == 8 ? 8 : 4;
const int bufferStride64 = 16;
int stride = IntPtr.Size == 8 ? bufferStride64 : 12;
int frames = checked((int)frameCount);
int requested = checked(frames * 2);
if (bufferCount != 2 || requested > renderScratch.Length) return Fail("Core Audio returned an invalid planar stereo playback buffer.");
Span<short> source = renderScratch.AsSpan(0, requested);
int read = pcm.Read(source);
source[read..].Clear();
for (int channel = 0; channel < 2; channel++)
{
int offset = firstBuffer + channel * stride;
int channels = Marshal.ReadInt32(outputData, offset);
int byteCount = Marshal.ReadInt32(outputData, offset + 4);
nint data = Marshal.ReadIntPtr(outputData, offset + 8);
if (channels != 1 || data == 0 || byteCount < frames * sizeof(float)) return Fail("Core Audio returned an invalid Float32 playback channel.");
var output = new Span<float>((void*)data, frames);
for (int frame = 0; frame < frames; frame++) output[frame] = source[frame * 2 + channel] * (1.0f / 32_768.0f);
}
if (isSilence != IntPtr.Zero) Marshal.WriteByte(isSilence, read == 0 ? (byte)1 : (byte)0);
return 0;
}
catch (Exception exception)
{
if (Interlocked.Exchange(ref callbackFailed, 1) == 0) reportFailure(exception);
return -1;
}
}
private int Fail(string message)
{
if (Interlocked.Exchange(ref callbackFailed, 1) == 0) reportFailure(new InvalidOperationException(message));
return -1;
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
engine.Stop();
engine.DetachNode(source);
source.Dispose();
format.Dispose();
engine.Dispose();
}
}
}
@@ -0,0 +1,72 @@
using System.Security.Cryptography;
using System.Text;
using Foundation;
using Security;
namespace VoiceCat.Mac;
internal sealed class MacKeychainPasswordStore
{
private const string Service = "net.iamtalon.voicecat";
internal string? Load(Guid profileId)
{
using var query = Query(profileId);
using SecRecord? result = SecKeyChain.QueryAsRecord(query, out SecStatusCode status);
if (status != SecStatusCode.Success || result?.ValueData is not { } data) return null;
byte[] encoded = data.ToArray();
try { return Encoding.UTF8.GetString(encoded); }
finally { CryptographicOperations.ZeroMemory(encoded); }
}
internal string? LoadLegacy(string? keychainTag)
{
if (string.IsNullOrWhiteSpace(keychainTag)) return null;
using var query = new SecRecord(SecKind.GenericPassword) { Service = "cat.voice.VoiceCatMac", Account = keychainTag };
using SecRecord? result = SecKeyChain.QueryAsRecord(query, out SecStatusCode status);
if (status != SecStatusCode.Success || result?.ValueData is not { } data) return null;
byte[] encoded = data.ToArray();
try { return Encoding.UTF8.GetString(encoded); }
finally { CryptographicOperations.ZeroMemory(encoded); }
}
internal void Save(Guid profileId, string password)
{
ArgumentException.ThrowIfNullOrEmpty(password);
byte[] encoded = Encoding.UTF8.GetBytes(password);
try
{
using var data = NSData.FromArray(encoded);
using var query = Query(profileId);
using var attributes = new SecRecord
{
Label = "VoiceCat server password",
Accessible = SecAccessible.WhenUnlocked,
ValueData = data
};
SecStatusCode status = SecKeyChain.Update(query, attributes);
if (status == SecStatusCode.ItemNotFound)
{
using var record = Query(profileId);
record.Label = attributes.Label; record.Accessible = attributes.Accessible; record.ValueData = data;
status = SecKeyChain.Add(record);
}
if (status != SecStatusCode.Success) throw new InvalidOperationException($"Keychain save failed ({status}).");
}
finally { CryptographicOperations.ZeroMemory(encoded); }
}
internal void Remove(Guid profileId)
{
using var query = Query(profileId);
SecStatusCode status = SecKeyChain.Remove(query);
if (status is not (SecStatusCode.Success or SecStatusCode.ItemNotFound))
throw new InvalidOperationException($"Keychain removal failed ({status}).");
}
private static SecRecord Query(Guid profileId) => new(SecKind.GenericPassword)
{
Service = Service,
Account = profileId.ToString("D")
};
}
+71
View File
@@ -0,0 +1,71 @@
using Foundation;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal sealed class MacSettings
{
private readonly NSUserDefaults values = NSUserDefaults.StandardUserDefaults;
internal AudioInputMode InputMode { get; set; } = AudioInputMode.VoiceActivation;
internal float VadThreshold { get; set; } = 0.05f;
internal float InputGain { get; set; } = 1f;
internal float OutputGain { get; set; } = 0.8f;
internal bool InputNoiseReduction { get; set; }
internal bool StereoMicrophone { get; set; }
internal ushort PushToTalkKeyCode { get; set; } = 0x60;
internal string? InputDeviceId { get; set; }
internal string? OutputDeviceId { get; set; }
internal bool AuxiliaryEnabled { get; set; }
internal string? AuxiliaryDeviceId { get; set; }
internal float AuxiliaryGain { get; set; } = 1f;
internal int AudioBufferMilliseconds { get; set; } = 40;
internal bool EventSounds { get; set; } = true;
internal bool SpokenEvents { get; set; }
internal float EventVolume { get; set; } = 1f;
internal bool SelfTalkSounds { get; set; }
internal bool PushToTalkSound { get; set; }
internal void Load()
{
NSString[] keys = [(NSString)"voice.inputMode", (NSString)"voice.vadThreshold", (NSString)"voice.inputGain", (NSString)"voice.outputGain",
(NSString)"voice.pttKeyCode", (NSString)"voice.auxGain", (NSString)"voice.audioBufferMs", (NSString)"feedback.sounds", (NSString)"feedback.volume"];
NSObject[] defaults = [NSNumber.FromInt32((int)AudioInputMode.VoiceActivation), NSNumber.FromFloat(0.05f),
NSNumber.FromFloat(1f), NSNumber.FromFloat(0.8f), NSNumber.FromInt32(0x60), NSNumber.FromFloat(1f), NSNumber.FromInt32(40),
NSNumber.FromBoolean(true), NSNumber.FromFloat(1f)];
values.RegisterDefaults(new NSDictionary<NSString, NSObject>(keys, defaults));
InputMode = Enum.IsDefined(typeof(AudioInputMode), (int)values.IntForKey("voice.inputMode"))
? (AudioInputMode)(int)values.IntForKey("voice.inputMode") : AudioInputMode.VoiceActivation;
VadThreshold = Math.Clamp(values.FloatForKey("voice.vadThreshold"), 0f, 1f);
InputGain = Math.Clamp(values.FloatForKey("voice.inputGain"), 0f, 4f);
OutputGain = Math.Clamp(values.FloatForKey("voice.outputGain"), 0f, 1f);
InputNoiseReduction = values.BoolForKey("voice.inputNoiseReduction");
StereoMicrophone = values.BoolForKey("voice.stereoMic");
PushToTalkKeyCode = checked((ushort)Math.Clamp(values.IntForKey("voice.pttKeyCode"), 0, ushort.MaxValue));
InputDeviceId = EmptyToNull(values.StringForKey("voice.inputDevice"));
OutputDeviceId = EmptyToNull(values.StringForKey("voice.outputDevice"));
AuxiliaryEnabled = values.BoolForKey("voice.auxEnabled");
AuxiliaryDeviceId = EmptyToNull(values.StringForKey("voice.auxDeviceUID"));
AuxiliaryGain = Math.Clamp(values.FloatForKey("voice.auxGain"), 0f, 4f);
int audioBuffer = checked((int)values.IntForKey("voice.audioBufferMs")); AudioBufferMilliseconds = audioBuffer is 20 or 40 or 60 ? audioBuffer : 40;
EventSounds = values.BoolForKey("feedback.sounds");
SpokenEvents = values.BoolForKey("feedback.speech");
EventVolume = Math.Clamp(values.FloatForKey("feedback.volume"), 0f, 1f);
SelfTalkSounds = values.BoolForKey("feedback.selfTalk");
PushToTalkSound = values.BoolForKey("feedback.ptt");
}
internal void Save()
{
values.SetInt((int)InputMode, "voice.inputMode"); values.SetFloat(VadThreshold, "voice.vadThreshold");
values.SetFloat(InputGain, "voice.inputGain"); values.SetFloat(OutputGain, "voice.outputGain");
values.SetBool(InputNoiseReduction, "voice.inputNoiseReduction"); values.SetBool(StereoMicrophone, "voice.stereoMic");
values.SetInt(PushToTalkKeyCode, "voice.pttKeyCode"); Set("voice.inputDevice", InputDeviceId); Set("voice.outputDevice", OutputDeviceId);
values.SetBool(AuxiliaryEnabled, "voice.auxEnabled"); Set("voice.auxDeviceUID", AuxiliaryDeviceId); values.SetFloat(AuxiliaryGain, "voice.auxGain"); values.SetInt(AudioBufferMilliseconds, "voice.audioBufferMs");
values.SetBool(EventSounds, "feedback.sounds"); values.SetBool(SpokenEvents, "feedback.speech"); values.SetFloat(EventVolume, "feedback.volume");
values.SetBool(SelfTalkSounds, "feedback.selfTalk"); values.SetBool(PushToTalkSound, "feedback.ptt"); values.Synchronize();
}
private void Set(string key, string? value) { if (value is null) values.RemoveObject(key); else values.SetString(value, key); }
private static string? EmptyToNull(string? value) => string.IsNullOrWhiteSpace(value) ? null : value;
}
@@ -0,0 +1,486 @@
using AppKit;
using AVFoundation;
using CoreGraphics;
using Foundation;
using VoiceCat.Audio;
using VoiceCat.Core;
using Voicecat.V1;
namespace VoiceCat.Mac;
internal sealed class MainWindowController : NSWindowController
{
private readonly VoiceCatClient client;
private readonly MacAudioBackend audioBackend = new();
private readonly uint selfId;
private readonly NSPopUpButton channels = new(new CGRect(20, 515, 245, 28), false);
private readonly NSPopUpButton inputDevice = new(new CGRect(275, 515, 165, 28), false);
private readonly NSPopUpButton outputDevice = new(new CGRect(450, 515, 165, 28), false);
private readonly NSTextView users = new(new CGRect(0, 0, 190, 430)) { Editable = false, Selectable = true };
private readonly NSTextView chat = new(new CGRect(0, 0, 510, 390)) { Editable = false, Selectable = true };
private readonly NSPopUpButton messageTarget = new(new CGRect(230, 55, 160, 28), false);
private readonly NSTextField compose = new(new CGRect(400, 55, 230, 28)) { PlaceholderString = "Message" };
private readonly NSButton send = new(new CGRect(640, 53, 90, 32)) { Title = "Send" };
private readonly NSButton voice = new(new CGRect(630, 510, 100, 32)) { Title = "Join voice" };
private readonly NSButton settingsButton = new(new CGRect(740, 510, 90, 32)) { Title = "Settings" };
private readonly NSButton screenAudioButton = new(new CGRect(840, 510, 130, 32)) { Title = "Share audio" };
private readonly NSButton administrationButton = new(new CGRect(840, 20, 130, 28)) { Title = "Administration" };
private readonly NSButton privateMessageButton = new(new CGRect(700, 20, 130, 28)) { Title = "Private chat" };
private readonly NSButton muteButton = NSButton.CreateCheckbox("Mute", () => { });
private readonly NSButton deafenButton = NSButton.CreateCheckbox("Deafen", () => { });
private readonly NSTextField status = NSTextField.CreateLabel("Connected");
private readonly MacSettings settings = new();
private readonly EventFeedback feedback;
private readonly NSTimer timer;
private uint currentChannel = 1;
private uint microphoneStreamId;
private uint auxiliaryStreamId;
private uint screenAudioStreamId;
private bool joinedVoice;
private bool changingChannel;
private IAudioCapture? microphone;
private IAudioCapture? auxiliary;
private IAudioPlayback? playback;
private ScreenAudioCapture? screenAudio;
private ScreenAudioSelection screenAudioSelection = ScreenAudioSelection.Default;
private SettingsWindowController? settingsWindow;
private AdministrationWindowController? administrationWindow;
private readonly Dictionary<uint, PrivateMessageWindowController> privateWindows = [];
private readonly HashSet<uint> talkingUsers = [];
private bool lastTalking;
private string? activeInputDevice;
private string? activeOutputDevice;
private string? activeAuxiliaryDevice;
private bool activeStereo;
private NSObject? pushToTalkMonitor;
private bool pushToTalkEngaged;
private long lastRemoteAudioTick;
internal MainWindowController(VoiceCatClient client, uint selfId, string nickname) : base(new NSWindow(new CGRect(0, 0, 1000, 570),
NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable | NSWindowStyle.Miniaturizable, NSBackingStore.Buffered, false))
{
this.client = client; this.selfId = selfId; settings.Load(); feedback = new(settings);
Window!.Title = $"VoiceCat — {nickname}"; Window.Center(); Window.MinSize = new CGSize(680, 480);
var content = Window.ContentView!;
((INSAccessibility)channels).AccessibilityLabel = "Channel"; channels.Activated += ChangeChannel; content.AddSubview(channels);
PopulateAudioDevices(inputDevice, true); PopulateAudioDevices(outputDevice, false);
((INSAccessibility)inputDevice).AccessibilityLabel = "Microphone input device"; inputDevice.Activated += ChangeAudioDevice; content.AddSubview(inputDevice);
((INSAccessibility)outputDevice).AccessibilityLabel = "Audio output device"; outputDevice.Activated += ChangeAudioDevice; content.AddSubview(outputDevice);
((INSAccessibility)voice).AccessibilityLabel = "Join or leave voice"; voice.Activated += ToggleVoice; content.AddSubview(voice);
((INSAccessibility)settingsButton).AccessibilityLabel = "Open audio and notification settings"; settingsButton.Activated += OpenSettings; content.AddSubview(settingsButton);
((INSAccessibility)screenAudioButton).AccessibilityLabel = "Start or stop sharing screen audio"; screenAudioButton.Activated += ToggleScreenAudio; content.AddSubview(screenAudioButton);
((INSAccessibility)administrationButton).AccessibilityLabel = "Open server moderation and administration"; administrationButton.Activated += OpenAdministration; content.AddSubview(administrationButton);
((INSAccessibility)privateMessageButton).AccessibilityLabel = "Open a private conversation with the selected message recipient"; privateMessageButton.Activated += OpenPrivateMessage; content.AddSubview(privateMessageButton);
muteButton.Frame = new CGRect(275, 490, 75, 22); deafenButton.Frame = new CGRect(355, 490, 85, 22);
((INSAccessibility)muteButton).AccessibilityLabel = "Mute microphone"; ((INSAccessibility)deafenButton).AccessibilityLabel = "Deafen playback";
muteButton.Activated += SelfAudioChanged; deafenButton.Activated += SelfAudioChanged; content.AddSubview(muteButton); content.AddSubview(deafenButton);
var userScroll = new NSScrollView(new CGRect(20, 90, 190, 410)) { HasVerticalScroller = true, DocumentView = users }; userScroll.AccessibilityLabel = "Users in channel"; content.AddSubview(userScroll);
var chatScroll = new NSScrollView(new CGRect(230, 90, 500, 410)) { HasVerticalScroller = true, DocumentView = chat }; chatScroll.AccessibilityLabel = "Channel messages"; content.AddSubview(chatScroll);
((INSAccessibility)messageTarget).AccessibilityLabel = "Message recipient"; content.AddSubview(messageTarget);
((INSAccessibility)compose).AccessibilityLabel = "Message"; compose.Activated += Send; content.AddSubview(compose);
((INSAccessibility)send).AccessibilityLabel = "Send message"; send.Activated += Send; content.AddSubview(send);
status.Frame = new CGRect(20, 22, 660, 22); status.AccessibilityLabel = "Connection status"; content.AddSubview(status);
timer = NSTimer.CreateRepeatingScheduledTimer(TimeSpan.FromMilliseconds(50), _ => Pump());
ApplyEngineSettings(); feedback.Play(SoundEvent.Login); feedback.Speak("Connected");
pushToTalkMonitor = NSEvent.AddLocalMonitorForEventsMatchingMask(NSEventMask.KeyDown | NSEventMask.KeyUp, value => HandlePushToTalk(value)!);
RefreshState();
}
private NSEvent? HandlePushToTalk(NSEvent value)
{
if (settings.InputMode != AudioInputMode.PushToTalk || value.KeyCode != settings.PushToTalkKeyCode) return value;
bool engaged = value.Type == NSEventType.KeyDown;
client.Audio.PushToTalk = engaged;
if (engaged && !pushToTalkEngaged) feedback.Play(SoundEvent.PushToTalk);
pushToTalkEngaged = engaged; return null;
}
private void Pump()
{
while (client.TryReadEvent(out Envelope? envelope))
{
if (envelope!.TextMessage is { } text)
{
if (text.Scope == TextScope.TextPrivate)
{
uint other = text.SenderId == selfId ? text.TargetId : text.SenderId;
GetPrivateWindow(other).Append(text, Name(text.SenderId));
feedback.Play(text.SenderId == selfId ? SoundEvent.PrivateSent : SoundEvent.PrivateReceived);
if (text.SenderId != selfId) feedback.Speak($"Private message from {Name(text.SenderId)}: {text.Body}");
}
else { Append(FormatMessage(text)); feedback.Play(text.SenderId == selfId ? SoundEvent.ChannelSent : SoundEvent.ChannelReceived); }
}
if (envelope.StreamState is { } streamState)
{
if (streamState.Talking) talkingUsers.Add(streamState.UserId); else talkingUsers.Remove(streamState.UserId);
RefreshState();
}
if (envelope.UserEvent is { } userEvent)
{
string name = userEvent.User?.Nickname ?? Name(userEvent.LeftId);
if (userEvent.Kind == UserEvent.Types.Kind.Joined && userEvent.User?.ChannelId == currentChannel && userEvent.User.Id != selfId)
{ AppendActivity($"{name} joined the channel"); feedback.Play(SoundEvent.ChannelJoin); feedback.Speak($"{name} joined"); }
else if (userEvent.Kind == UserEvent.Types.Kind.Left)
{ talkingUsers.Remove(userEvent.LeftId); AppendActivity($"{name} left the server"); feedback.Play(SoundEvent.ChannelLeave); }
}
if (envelope.ServerState is not null || envelope.ChannelEvent is not null || envelope.UserEvent is not null) RefreshState();
if (envelope.Disconnect is { } disconnected)
{
StopAudioDevices(); joinedVoice = false; voice.Title = "Join voice";
status.StringValue = "Disconnected: " + disconnected.Reason; voice.Enabled = send.Enabled = messageTarget.Enabled = false;
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost"); Announce(status.StringValue);
}
}
if (client.Audio.Failure is { } failure) { status.StringValue = "Audio stopped: " + failure.Message; return; }
if (audioBackend.Failure is { } deviceFailure) { status.StringValue = "Audio device stopped: " + deviceFailure.Message; return; }
if (joinedVoice)
{
(float level, bool talking) = client.Audio.GetLocalLevel(microphoneStreamId);
if (talking != lastTalking)
{
lastTalking = talking; client.PublishStreamState(microphoneStreamId, talking);
feedback.Play(talking ? SoundEvent.VoiceStart : SoundEvent.VoiceStop);
}
bool receiving = Environment.TickCount64 - Volatile.Read(ref lastRemoteAudioTick) < 1_000;
status.StringValue = $"Voice connected · Mic {(talking ? "sending" : "idle")} {level:P0} · Remote audio {(receiving ? "active" : "idle")}";
}
}
private void OpenSettings(object? sender, EventArgs args)
{
settingsWindow ??= new(this, settings, audioBackend);
settingsWindow.ShowWindow(this); settingsWindow.Window?.MakeKeyAndOrderFront(this);
}
private void OpenAdministration(object? sender, EventArgs args)
{
administrationWindow ??= new(client, selfId); administrationWindow.ShowWindow(this); administrationWindow.Window?.MakeKeyAndOrderFront(this);
}
private void OpenPrivateMessage(object? sender, EventArgs args)
{
uint target = uint.TryParse(messageTarget.SelectedItem?.RepresentedObject?.ToString(), out uint selected) ? selected : 0;
if (target == 0) { status.StringValue = "Choose a private message recipient first."; return; }
PrivateMessageWindowController window = GetPrivateWindow(target); window.ShowWindow(this); window.Window?.MakeKeyAndOrderFront(this);
}
private PrivateMessageWindowController GetPrivateWindow(uint userId)
{
if (privateWindows.TryGetValue(userId, out PrivateMessageWindowController? existing)) return existing;
var created = new PrivateMessageWindowController(client, userId, selfId, Name(userId)); privateWindows[userId] = created; return created;
}
private void SelfAudioChanged(object? sender, EventArgs args)
{
bool muted = muteButton.State == NSCellStateValue.On;
bool deafened = deafenButton.State == NSCellStateValue.On;
client.SetSelfAudioState(muted, deafened);
status.StringValue = deafened ? "Deafened" : muted ? "Microphone muted" : "Voice active";
}
private async void ToggleScreenAudio(object? sender, EventArgs args)
{
try
{
screenAudioButton.Enabled = false;
if (screenAudioStreamId != 0) { StopScreenAudio(); return; }
if (!joinedVoice) throw new InvalidOperationException("Join voice before sharing screen audio.");
ScreenAudioSelection? selection = await ScreenAudioPicker.ChooseAsync(screenAudioSelection);
if (selection is null) return;
screenAudioSelection = selection;
StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamScreenAudio, "Desktop audio", 2);
screenAudioStreamId = stream.StreamId;
int channelCount = stream.Audio.Mode == ChannelMode.ModeStereo ? 2 : 1;
var capture = new ScreenAudioCapture(channelCount, screenAudioSelection,
(pcm, channels) => { uint id = screenAudioStreamId; if (id != 0) client.Audio.FeedPcm(id, pcm, channels); });
capture.Failed += exception => NSApplication.SharedApplication.InvokeOnMainThread(() => { status.StringValue = exception.Message; StopScreenAudio(); });
screenAudio = capture; await capture.StartAsync(); screenAudioButton.Title = "Stop sharing";
status.StringValue = "Sharing desktop audio";
}
catch (Exception exception) { StopScreenAudio(); status.StringValue = exception.Message; }
finally { screenAudioButton.Enabled = true; }
}
private void StopScreenAudio()
{
Interlocked.Exchange(ref screenAudio, null)?.Dispose();
uint id = screenAudioStreamId; screenAudioStreamId = 0;
TryStopStream(id);
screenAudioButton.Title = "Share audio";
}
private void ApplyEngineSettings()
{
client.Audio.InputMode = settings.InputMode; client.Audio.VadThreshold = settings.VadThreshold;
if (settings.InputMode != AudioInputMode.PushToTalk) { client.Audio.PushToTalk = false; pushToTalkEngaged = false; }
client.Audio.InputGain = settings.InputGain; client.Audio.OutputGain = settings.OutputGain;
client.Audio.InputNoiseReduction = settings.InputNoiseReduction;
client.Audio.DeviceBufferMilliseconds = settings.AudioBufferMilliseconds;
if (playback is not null) playback.BufferMilliseconds = settings.AudioBufferMilliseconds;
if (auxiliaryStreamId != 0) client.Audio.SetLocalGain(auxiliaryStreamId, settings.AuxiliaryGain);
}
internal async Task ApplySettingsAsync()
{
ApplyEngineSettings();
if (!joinedVoice) return;
bool restart = activeInputDevice != settings.InputDeviceId || activeOutputDevice != settings.OutputDeviceId || activeStereo != settings.StereoMicrophone;
if (restart) { await LeaveVoice(); await JoinVoice(); return; }
if (settings.AuxiliaryEnabled && auxiliaryStreamId == 0) await StartAuxiliaryAsync();
else if (!settings.AuxiliaryEnabled && auxiliaryStreamId != 0) StopAuxiliary();
else if (settings.AuxiliaryEnabled && activeAuxiliaryDevice != settings.AuxiliaryDeviceId) { StopAuxiliary(); await StartAuxiliaryAsync(); }
}
private void RefreshState()
{
uint previousTarget = uint.TryParse(messageTarget.SelectedItem?.RepresentedObject?.ToString(), out uint selectedTarget) ? selectedTarget : 0;
channels.RemoveAllItems();
IReadOnlyList<(Channel Channel, int Depth)> hierarchy = FlattenChannels(client.Channels);
foreach ((Channel channel, int depth) in hierarchy)
{
int members = client.Users.Count(user => user.ChannelId == channel.Id);
channels.AddItem(new string(' ', depth * 3) + channel.Name + $" ({members})" + (channel.PasswordProtected ? " [password]" : ""));
channels.LastItem!.RepresentedObject = new NSString(channel.Id.ToString());
channels.LastItem!.ToolTip = channel.Topic;
}
currentChannel = client.Users.FirstOrDefault(u => u.Id == selfId)?.ChannelId ?? currentChannel;
int selectedIndex = hierarchy.ToList().FindIndex(item => item.Channel.Id == currentChannel); if (selectedIndex >= 0) channels.SelectItem(selectedIndex);
users.Value = string.Join("\n", client.Users.Where(u => u.ChannelId == currentChannel).OrderBy(u => u.Nickname).Select(u =>
(talkingUsers.Contains(u.Id) ? "Speaking — " : "") + (u.Id == selfId ? "You — " : "") + u.Nickname +
(u.ServerDeafened ? " [server deafened]" : u.ServerMuted ? " [server muted]" : u.SelfDeafened ? " [deafened]" : u.SelfMicMuted ? " [muted]" : "")));
messageTarget.RemoveAllItems(); messageTarget.AddItem("Current channel"); messageTarget.LastItem!.RepresentedObject = new NSString("0");
foreach (var user in client.Users.Where(user => user.Id != selfId).OrderBy(user => user.Nickname))
{
messageTarget.AddItem("Private: " + user.Nickname);
messageTarget.LastItem!.RepresentedObject = new NSString(user.Id.ToString());
}
int targetIndex = client.Users.Where(user => user.Id != selfId).OrderBy(user => user.Nickname).ToList().FindIndex(user => user.Id == previousTarget) + 1;
messageTarget.SelectItem(Math.Max(0, targetIndex));
status.StringValue = $"Connected · {client.Users.Count} users";
}
private static IReadOnlyList<(Channel Channel, int Depth)> FlattenChannels(IReadOnlyList<Channel> values)
{
var result = new List<(Channel, int)>(); var visited = new HashSet<uint>();
void AddChildren(uint parent, int depth)
{
foreach (Channel channel in values.Where(channel => channel.ParentId == parent).OrderBy(channel => channel.Order).ThenBy(channel => channel.Name))
if (visited.Add(channel.Id)) { result.Add((channel, depth)); AddChildren(channel.Id, depth + 1); }
}
AddChildren(0, 0);
foreach (Channel channel in values.Where(channel => !visited.Contains(channel.Id)).OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) result.Add((channel, 0));
return result;
}
private async void ChangeChannel(object? sender, EventArgs args)
{
if (changingChannel || !uint.TryParse(channels.SelectedItem?.RepresentedObject?.ToString(), out uint id) || id == currentChannel) return;
var channel = client.Channels.FirstOrDefault(item => item.Id == id);
if (channel is null) { RefreshState(); return; }
string? password = channel.PasswordProtected ? PromptForChannelPassword(channel.Name) : "";
if (password is null) { RefreshState(); return; }
bool resumeVoice = joinedVoice;
try
{
changingChannel = true; channels.Enabled = false;
if (resumeVoice) await LeaveVoice();
var result = (await client.RequestAsync(new() { JoinChannel = new() { ChannelId = id, Password = password } })).JoinChannelResult;
if (result.Ok) currentChannel = id;
if (resumeVoice) await JoinVoice();
RefreshState();
status.StringValue = result.Ok ? $"Joined {channel.Name}" : result.Error;
}
catch (Exception exception)
{
string message = exception.Message;
if (resumeVoice && !joinedVoice)
{
try { await JoinVoice(); }
catch (Exception restoreException) { message += " Voice could not be restored: " + restoreException.Message; }
}
RefreshState(); status.StringValue = message;
}
finally { changingChannel = false; channels.Enabled = true; }
}
private string? PromptForChannelPassword(string channelName)
{
var field = new NSSecureTextField(new CGRect(0, 0, 320, 26)) { PlaceholderString = "Channel password" };
((INSAccessibility)field).AccessibilityLabel = $"Password for {channelName}";
var alert = new NSAlert
{
MessageText = $"Join {channelName}",
InformativeText = "Enter the channel password.",
AlertStyle = NSAlertStyle.Informational,
AccessoryView = field
};
alert.AddButton("Join"); alert.AddButton("Cancel");
if (alert.RunModal() != 1000) { field.StringValue = ""; return null; }
string value = field.StringValue; field.StringValue = ""; return value;
}
private void Send(object? sender, EventArgs args)
{
string body = compose.StringValue.Trim(); if (body.Length == 0) return;
uint target = uint.TryParse(messageTarget.SelectedItem?.RepresentedObject?.ToString(), out uint selectedTarget) ? selectedTarget : 0;
try { client.Send(new() { TextMessage = new() { Scope = target == 0 ? TextScope.TextChannel : TextScope.TextPrivate, TargetId = target == 0 ? currentChannel : target, Body = body, ClientMsgId = Guid.NewGuid().ToString("N") } }); compose.StringValue = ""; }
catch (Exception exception) { status.StringValue = exception.Message; }
}
private async void ToggleVoice(object? sender, EventArgs args)
{
try
{
voice.Enabled = false;
if (!joinedVoice) await JoinVoice();
else await LeaveVoice();
}
catch (Exception exception) { status.StringValue = exception.Message; }
finally { voice.Enabled = true; }
}
private void PopulateAudioDevices(NSPopUpButton menu, bool input)
{
IReadOnlyList<AudioDeviceInfo> devices = audioBackend.Enumerate(input);
foreach (AudioDeviceInfo device in devices)
{
menu.AddItem(device.Name + (device.IsDefault ? " (default)" : ""));
menu.LastItem!.RepresentedObject = new NSString(device.Id);
}
int selected = devices.ToList().FindIndex(device => device.IsDefault);
if (selected >= 0) menu.SelectItem(selected);
menu.Enabled = devices.Count > 0;
}
private async void ChangeAudioDevice(object? sender, EventArgs args)
{
if (!joinedVoice) return;
try { await LeaveVoice(); await JoinVoice(); }
catch (Exception exception) { status.StringValue = exception.Message; }
}
private async Task JoinVoice()
{
AVAuthorizationStatus permission = AVCaptureDevice.GetAuthorizationStatus(AVAuthorizationMediaType.Audio);
if (permission == AVAuthorizationStatus.NotDetermined)
{
bool granted = await AVCaptureDevice.RequestAccessForMediaTypeAsync(AVAuthorizationMediaType.Audio);
permission = granted ? AVAuthorizationStatus.Authorized : AVAuthorizationStatus.Denied;
}
if (permission != AVAuthorizationStatus.Authorized)
throw new UnauthorizedAccessException("Microphone access is disabled. Enable VoiceCat in System Settings → Privacy & Security → Microphone, then join voice again.");
var result = await client.SubscribeVoiceAsync();
if (!result.Ok) throw new InvalidOperationException(result.Error);
try
{
ApplyEngineSettings();
activeInputDevice = settings.InputDeviceId ?? SelectedDevice(inputDevice);
activeOutputDevice = settings.OutputDeviceId ?? SelectedDevice(outputDevice);
activeStereo = settings.StereoMicrophone;
playback = audioBackend.OpenPlayback(activeOutputDevice);
playback.BufferMilliseconds = settings.AudioBufferMilliseconds;
client.Audio.MixedPcm += PlayMixedPcm;
StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamMic, "Microphone", settings.StereoMicrophone ? 2 : 1);
microphoneStreamId = stream.StreamId;
microphone = audioBackend.OpenCapture(activeInputDevice, false, FeedMicrophone);
if (settings.AuxiliaryEnabled) await StartAuxiliaryAsync();
joinedVoice = true; voice.Title = "Leave voice"; status.StringValue = "Voice connected";
feedback.Play(SoundEvent.VoiceOn);
}
catch
{
try { if (microphoneStreamId != 0) client.StopStream(microphoneStreamId); } catch { }
StopAudioDevices();
try { await client.SubscribeVoiceAsync(false); } catch { }
throw;
}
}
private async Task LeaveVoice()
{
uint streamId = microphoneStreamId;
joinedVoice = false; voice.Title = "Join voice";
StopAudioDevices();
if (streamId != 0) client.StopStream(streamId);
await client.SubscribeVoiceAsync(false);
status.StringValue = "Voice disconnected";
feedback.Play(SoundEvent.VoiceOff);
}
private async Task StartAuxiliaryAsync()
{
if (auxiliaryStreamId != 0) return;
StreamInfo stream = await client.StartStreamAsync(StreamKind.StreamAuxDevice, "Aux device", 2);
auxiliaryStreamId = stream.StreamId; client.Audio.SetLocalGain(stream.StreamId, settings.AuxiliaryGain);
try { activeAuxiliaryDevice = settings.AuxiliaryDeviceId; auxiliary = audioBackend.OpenCapture(activeAuxiliaryDevice, false, FeedAuxiliary); }
catch { client.StopStream(auxiliaryStreamId); auxiliaryStreamId = 0; throw; }
}
private void StopAuxiliary()
{
Interlocked.Exchange(ref auxiliary, null)?.Dispose();
activeAuxiliaryDevice = null;
uint id = auxiliaryStreamId; auxiliaryStreamId = 0;
TryStopStream(id);
}
private void FeedAuxiliary(ReadOnlySpan<short> pcm, int channels)
{
uint id = auxiliaryStreamId; if (id != 0) client.Audio.FeedPcm(id, pcm, channels);
}
private void FeedMicrophone(ReadOnlySpan<short> pcm, int channels)
{
uint streamId = microphoneStreamId;
if (streamId != 0) client.Audio.FeedPcm(streamId, pcm, channels);
}
private static string? SelectedDevice(NSPopUpButton menu) => menu.SelectedItem?.RepresentedObject?.ToString();
private void PlayMixedPcm(ReadOnlySpan<short> pcm)
{
bool signal = false;
foreach (short sample in pcm)
if (sample is > 64 or < -64) { signal = true; break; }
if (signal) Volatile.Write(ref lastRemoteAudioTick, Environment.TickCount64);
Volatile.Read(ref playback)?.Write(pcm);
}
private void StopAudioDevices()
{
StopScreenAudio();
StopAuxiliary();
microphoneStreamId = 0;
Volatile.Write(ref lastRemoteAudioTick, 0);
Interlocked.Exchange(ref microphone, null)?.Dispose();
client.Audio.MixedPcm -= PlayMixedPcm;
Interlocked.Exchange(ref playback, null)?.Dispose();
activeInputDevice = activeOutputDevice = null; lastTalking = false;
}
private void TryStopStream(uint id)
{
if (id == 0 || client.State != ClientConnectionState.Connected) return;
try { client.StopStream(id); }
catch (InvalidOperationException) { }
catch (IOException) { }
}
private string Name(uint id) => client.Users.FirstOrDefault(u => u.Id == id)?.Nickname ?? $"User {id}";
private string FormatMessage(TextMessage text)
{
string prefix = text.Scope switch
{
TextScope.TextPrivate => $"[private: {Name(text.SenderId == selfId ? text.TargetId : text.SenderId)}] ",
TextScope.TextServer => "[server] ",
_ => ""
};
return $"[{DateTime.Now:t}] {prefix}{Name(text.SenderId)}: {text.Body}";
}
private void Append(string line)
{
chat.Value = chat.Value.Length == 0 ? line : chat.Value + "\n" + line;
chat.ScrollRangeToVisible(new NSRange(chat.Value.Length, 0));
}
private void AppendActivity(string line) => Append($"[{DateTime.Now:t}] — {line}");
private void Announce(string value)
{
NSString[] keys = [NSAccessibilityNotificationUserInfoKeys.AnnouncementKey, NSAccessibilityNotificationUserInfoKeys.PriorityKey];
NSObject[] values = [(NSString)value, NSNumber.FromInt32(1)];
NSAccessibility.PostNotification(status, NSView.AnnouncementRequestedNotification, new NSDictionary<NSString, NSObject>(keys, values));
}
protected override void Dispose(bool disposing)
{
if (disposing) { timer.Invalidate(); if (pushToTalkMonitor is not null) NSEvent.RemoveMonitor(pushToTalkMonitor); client.Audio.PushToTalk = false; settingsWindow?.Close(); administrationWindow?.Close(); foreach (PrivateMessageWindowController window in privateWindows.Values) window.Close(); privateWindows.Clear(); StopAudioDevices(); feedback.Dispose(); client.DisposeAsync().AsTask().GetAwaiter().GetResult(); }
base.Dispose(disposing);
}
}
@@ -0,0 +1,43 @@
using AppKit;
using CoreGraphics;
using VoiceCat.Core;
using Voicecat.V1;
namespace VoiceCat.Mac;
internal sealed class PrivateMessageWindowController : NSWindowController
{
private readonly VoiceCatClient client;
private readonly uint otherUserId;
private readonly uint selfId;
private readonly NSTextView transcript = new(new CGRect(0, 0, 430, 300)) { Editable = false, Selectable = true };
private readonly NSTextField compose = new(new CGRect(20, 20, 350, 28)) { PlaceholderString = "Private message" };
internal PrivateMessageWindowController(VoiceCatClient client, uint otherUserId, uint selfId, string nickname) : base(new NSWindow(
new CGRect(0, 0, 480, 400), NSWindowStyle.Titled | NSWindowStyle.Closable | NSWindowStyle.Resizable,
NSBackingStore.Buffered, false))
{
this.client = client; this.otherUserId = otherUserId; this.selfId = selfId;
Window!.Title = $"Private message — {nickname}"; Window.Center();
NSView view = Window.ContentView!;
var scroll = new NSScrollView(new CGRect(20, 65, 440, 315)) { DocumentView = transcript, HasVerticalScroller = true };
scroll.AccessibilityLabel = $"Private conversation with {nickname}"; view.AddSubview(scroll);
((INSAccessibility)compose).AccessibilityLabel = $"Message to {nickname}"; compose.Activated += Send; view.AddSubview(compose);
var button = new NSButton(new CGRect(380, 18, 80, 32)) { Title = "Send" }; button.Activated += Send; view.AddSubview(button);
}
private void Send(object? sender, EventArgs args)
{
string value = compose.StringValue.Trim(); if (value.Length == 0) return;
client.Send(new() { TextMessage = new() { Scope = TextScope.TextPrivate, TargetId = otherUserId, Body = value, ClientMsgId = Guid.NewGuid().ToString("N") } });
compose.StringValue = "";
}
internal void Append(TextMessage message, string sender)
{
string line = $"[{DateTime.Now:t}] {(message.SenderId == selfId ? "You" : sender)}: {message.Body}";
transcript.Value = transcript.Value.Length == 0 ? line : transcript.Value + "\n" + line;
transcript.ScrollRangeToVisible(new Foundation.NSRange(transcript.Value.Length, 0));
Window?.MakeKeyAndOrderFront(this);
}
}
+5
View File
@@ -0,0 +1,5 @@
using AppKit;
NSApplication.Init();
NSApplication.SharedApplication.Delegate = new VoiceCat.Mac.AppDelegate();
NSApplication.SharedApplication.Run();
@@ -0,0 +1,156 @@
using System.Runtime.InteropServices;
using CoreFoundation;
using CoreMedia;
using Foundation;
using ObjCRuntime;
using ScreenCaptureKit;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal enum ScreenAudioScope { EntireDesktop, OnlyApplications, AllExceptApplications }
internal sealed record ScreenAudioSelection(ScreenAudioScope Scope, IReadOnlySet<string> BundleIdentifiers, bool ExcludeScreenReader)
{
internal static ScreenAudioSelection Default { get; } = new(ScreenAudioScope.EntireDesktop, new HashSet<string>(), false);
}
internal sealed class ScreenAudioCapture : NSObject, ISCStreamOutput
{
private static readonly HashSet<string> ScreenReaders = ["com.apple.VoiceOver", "com.apple.VoiceOver4", "com.apple.speech.speechsynthesisd"];
private readonly int channels;
private readonly CapturePcmHandler onPcm;
private readonly ScreenAudioSelection selection;
private readonly DispatchQueue queue = new("net.iamtalon.voicecat.screen-audio");
private readonly CaptureDelegate streamDelegate;
private SCStream? stream;
private int disposed;
internal ScreenAudioCapture(int channels, ScreenAudioSelection selection, CapturePcmHandler onPcm)
{
this.channels = Math.Clamp(channels, 1, 2); this.selection = selection; this.onPcm = onPcm;
streamDelegate = new(error => Failed?.Invoke(error));
}
internal event Action<Exception>? Failed;
internal static async Task<IReadOnlyList<ScreenApplication>> GetApplicationsAsync(CancellationToken cancellationToken = default)
{
SCShareableContent content = await GetContentAsync(cancellationToken).ConfigureAwait(false);
return content.Applications.Where(application => !string.IsNullOrWhiteSpace(application.BundleIdentifier) &&
!ScreenReaders.Contains(application.BundleIdentifier) && application.BundleIdentifier != NSBundle.MainBundle.BundleIdentifier)
.GroupBy(application => application.BundleIdentifier, StringComparer.Ordinal).Select(group => group.First())
.Select(application => new ScreenApplication(string.IsNullOrWhiteSpace(application.ApplicationName) ? application.BundleIdentifier : application.ApplicationName,
application.BundleIdentifier)).OrderBy(application => application.Name, StringComparer.CurrentCultureIgnoreCase).ToArray();
}
internal async Task StartAsync(CancellationToken cancellationToken = default)
{
SCShareableContent content = await GetContentAsync(cancellationToken).ConfigureAwait(false);
SCDisplay display = content.Displays.FirstOrDefault() ?? throw new InvalidOperationException("No display is available for screen audio capture.");
SCRunningApplication[] matching = content.Applications.Where(application => Included(application.BundleIdentifier)).ToArray();
SCContentFilterOption option = selection.Scope == ScreenAudioScope.OnlyApplications ? SCContentFilterOption.Include : SCContentFilterOption.Exclude;
var filter = new SCContentFilter(display, matching, [], option);
var configuration = new SCStreamConfiguration
{
CapturesAudio = true, ExcludesCurrentProcessAudio = true, SampleRate = 48_000,
ChannelCount = channels, Width = 2, Height = 2, QueueDepth = 6,
MinimumFrameInterval = new CMTime(1, 1)
};
var created = new SCStream(filter, configuration, streamDelegate);
if (!created.AddStreamOutput(this, SCStreamOutputType.Audio, queue, out NSError? outputError))
throw new InvalidOperationException(outputError?.LocalizedDescription ?? "Could not attach the screen-audio output.");
stream = created;
var started = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
created.StartCapture(error => { if (error is null) started.TrySetResult(); else started.TrySetException(new InvalidOperationException(error.LocalizedDescription)); });
await started.Task.WaitAsync(TimeSpan.FromSeconds(15), cancellationToken).ConfigureAwait(false);
}
private bool Included(string bundleIdentifier)
{
bool selected = selection.BundleIdentifiers.Contains(bundleIdentifier);
bool reader = selection.ExcludeScreenReader && ScreenReaders.Contains(bundleIdentifier);
return selection.Scope switch
{
ScreenAudioScope.OnlyApplications => selected,
ScreenAudioScope.AllExceptApplications => selected || reader,
_ => reader
};
}
[Export("stream:didOutputSampleBuffer:ofType:")]
public unsafe void DidOutputSampleBuffer(SCStream captureStream, CMSampleBuffer sampleBuffer, SCStreamOutputType type)
{
if (type != SCStreamOutputType.Audio || !sampleBuffer.DataIsReady || disposed != 0) return;
int frames = checked((int)sampleBuffer.NumSamples);
if (frames <= 0 || frames > 8192) return;
Span<byte> listStorage = stackalloc byte[40];
fixed (byte* list = listStorage)
{
int status = CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer.Handle, out _, (IntPtr)list,
(nuint)listStorage.Length, IntPtr.Zero, IntPtr.Zero, 0, out IntPtr blockBuffer);
if (status != 0) return;
try
{
uint count = *(uint*)list;
if (count is 0 or > 2) return;
Span<short> converted = stackalloc short[frames * channels];
NativeAudioBuffer* first = (NativeAudioBuffer*)(list + 8);
bool planar = count > 1;
int sourceChannels = planar ? checked((int)count) : checked((int)Math.Max(1, first->Channels));
for (int frame = 0; frame < frames; frame++)
{
float left = planar ? ((float*)first[0].Data)[frame] : ((float*)first[0].Data)[frame * sourceChannels];
float right = sourceChannels > 1
? planar ? ((float*)first[1].Data)[frame] : ((float*)first[0].Data)[frame * sourceChannels + 1]
: left;
if (channels == 1) converted[frame] = ToInt16(sourceChannels > 1 ? (left + right) * 0.5f : left);
else { converted[frame * 2] = ToInt16(left); converted[frame * 2 + 1] = ToInt16(right); }
}
onPcm(converted, channels);
}
finally { if (blockBuffer != IntPtr.Zero) CFRelease(blockBuffer); }
}
}
private static short ToInt16(float value) => (short)Math.Clamp((int)MathF.Round(Math.Clamp(value, -1, 1) * 32767), short.MinValue, short.MaxValue);
private static Task<SCShareableContent> GetContentAsync(CancellationToken cancellationToken)
{
var completion = new TaskCompletionSource<SCShareableContent>(TaskCreationOptions.RunContinuationsAsynchronously);
SCShareableContent.GetShareableContent((content, error) =>
{
if (error is not null) completion.TrySetException(new UnauthorizedAccessException(error.LocalizedDescription));
else if (content is null) completion.TrySetException(new InvalidOperationException("ScreenCaptureKit returned no shareable content."));
else completion.TrySetResult(content);
});
cancellationToken.Register(() => completion.TrySetCanceled(cancellationToken));
return completion.Task;
}
protected override void Dispose(bool disposing)
{
if (disposing && Interlocked.Exchange(ref disposed, 1) == 0)
{
SCStream? previous = Interlocked.Exchange(ref stream, null);
if (previous is null) { streamDelegate.Dispose(); queue.Dispose(); }
else previous.StopCapture(_ => { previous.Dispose(); streamDelegate.Dispose(); queue.Dispose(); });
}
base.Dispose(disposing);
}
[StructLayout(LayoutKind.Sequential)]
private struct NativeAudioBuffer { internal uint Channels, DataByteSize; internal IntPtr Data; }
private sealed class CaptureDelegate(Action<Exception> failed) : SCStreamDelegate
{
public override void DidStop(SCStream stream, NSError error) => failed(new IOException(error.LocalizedDescription));
}
[DllImport("/System/Library/Frameworks/CoreMedia.framework/CoreMedia")]
private static extern int CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(IntPtr sampleBuffer,
out nuint bufferListSizeNeeded, IntPtr bufferList, nuint bufferListSize, IntPtr structureAllocator,
IntPtr blockAllocator, uint flags, out IntPtr blockBuffer);
[DllImport("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation")]
private static extern void CFRelease(IntPtr value);
}
@@ -0,0 +1,49 @@
using AppKit;
using CoreGraphics;
using Foundation;
using ScreenCaptureKit;
namespace VoiceCat.Mac;
internal sealed record ScreenApplication(string Name, string BundleIdentifier);
internal static class ScreenAudioPicker
{
internal static async Task<ScreenAudioSelection?> ChooseAsync(ScreenAudioSelection initial, CancellationToken cancellationToken = default)
{
IReadOnlyList<ScreenApplication> applications = await ScreenAudioCapture.GetApplicationsAsync(cancellationToken);
var mode = new NSPopUpButton(new CGRect(0, 270, 390, 28), false);
mode.AddItems(["Everything", "Only selected applications", "All except selected applications"]);
mode.SelectItem((int)initial.Scope);
((INSAccessibility)mode).AccessibilityLabel = "Screen audio sharing mode";
var source = new ApplicationSource(applications);
var table = new NSTableView(new CGRect(0, 0, 390, 225)) { DataSource = source, Delegate = source, AllowsMultipleSelection = true, HeaderView = null };
table.AddColumn(new NSTableColumn("application") { Title = "Application", Width = 370 });
((INSAccessibility)table).AccessibilityLabel = "Applications to include or exclude";
uint[] selected = applications.Select((application, index) => initial.BundleIdentifiers.Contains(application.BundleIdentifier) ? (uint?)index : null)
.Where(index => index.HasValue).Select(index => index!.Value).ToArray();
if (selected.Length > 0) table.SelectRows(NSIndexSet.FromArray(selected), false);
var scroll = new NSScrollView(new CGRect(0, 38, 390, 225)) { DocumentView = table, HasVerticalScroller = true, BorderType = NSBorderType.BezelBorder };
var exclude = NSButton.CreateCheckbox("Exclude VoiceOver and speech-synthesis audio", () => { });
exclude.Frame = new CGRect(0, 5, 390, 24); exclude.State = initial.ExcludeScreenReader ? NSCellStateValue.On : NSCellStateValue.Off;
((INSAccessibility)exclude).AccessibilityLabel = "Exclude screen reader audio";
var accessory = new NSView(new CGRect(0, 0, 390, 305)); accessory.AddSubview(mode); accessory.AddSubview(scroll); accessory.AddSubview(exclude);
var alert = new NSAlert { MessageText = "Choose screen audio", InformativeText = "Select what other people will hear.", AccessoryView = accessory };
alert.AddButton("Share"); alert.AddButton("Cancel");
if (alert.RunModal() != 1000) return null;
var bundleIdentifiers = table.SelectedRows.ToArray().Select(index => applications[checked((int)index)].BundleIdentifier).ToHashSet(StringComparer.Ordinal);
return new((ScreenAudioScope)(int)mode.IndexOfSelectedItem, bundleIdentifiers, exclude.State == NSCellStateValue.On);
}
private sealed class ApplicationSource(IReadOnlyList<ScreenApplication> applications) : NSTableViewDataSource, INSTableViewDelegate
{
public override nint GetRowCount(NSTableView tableView) => applications.Count;
[Export("tableView:viewForTableColumn:row:")]
public NSView GetViewForItem(NSTableView tableView, NSTableColumn? tableColumn, nint row)
{
var value = NSTextField.CreateLabel(applications[checked((int)row)].Name);
value.LineBreakMode = AppKit.NSLineBreakMode.TruncatingTail; return value;
}
}
}
@@ -0,0 +1,126 @@
using AppKit;
using CoreGraphics;
using VoiceCat.Audio;
namespace VoiceCat.Mac;
internal sealed class SettingsWindowController : NSWindowController
{
private readonly MainWindowController owner;
private readonly MacSettings settings;
private readonly MacAudioBackend audio;
private readonly NSPopUpButton mode = new(new CGRect(160, 455, 260, 28), false);
private readonly NSSlider vad = new(new CGRect(160, 415, 260, 24)) { MinValue = 0, MaxValue = 100 };
private readonly NSSlider inputGain = new(new CGRect(160, 375, 260, 24)) { MinValue = 0, MaxValue = 400 };
private readonly NSButton noiseReduction = NSButton.CreateCheckbox("Noise reduction (RNNoise)", () => { });
private readonly NSButton stereo = NSButton.CreateCheckbox("Stereo microphone", () => { });
private readonly NSPopUpButton input = new(new CGRect(160, 285, 260, 28), false);
private readonly NSPopUpButton output = new(new CGRect(160, 245, 260, 28), false);
private readonly NSSlider outputGain = new(new CGRect(160, 205, 260, 24)) { MinValue = 0, MaxValue = 100 };
private readonly NSButton auxiliary = NSButton.CreateCheckbox("Aux input stream", () => { });
private readonly NSPopUpButton auxiliaryDevice = new(new CGRect(160, 125, 260, 28), false);
private readonly NSSlider auxiliaryGain = new(new CGRect(160, 85, 260, 24)) { MinValue = 0, MaxValue = 400 };
private readonly NSButton sounds = NSButton.CreateCheckbox("Event sounds", () => { });
private readonly NSButton speech = NSButton.CreateCheckbox("Speak events", () => { });
private readonly NSButton selfTalk = NSButton.CreateCheckbox("Own voice-activity sounds", () => { });
private readonly NSButton pttSound = NSButton.CreateCheckbox("Push-to-talk cue", () => { });
private readonly NSButton pttKey = new(new CGRect(220, 12, 200, 32)) { Title = "Change PTT key…" };
private readonly NSSlider eventVolume = new(new CGRect(160, 505, 260, 24)) { MinValue = 0, MaxValue = 100 };
private readonly NSPopUpButton audioBuffer = new(new CGRect(160, 545, 260, 28), false);
internal SettingsWindowController(MainWindowController owner, MacSettings settings, MacAudioBackend audio) : base(
new NSWindow(new CGRect(0, 0, 460, 660), NSWindowStyle.Titled | NSWindowStyle.Closable,
NSBackingStore.Buffered, false))
{
this.owner = owner; this.settings = settings; this.audio = audio;
Window!.Title = "VoiceCat Settings"; Window.Center();
NSView view = Window.ContentView!;
mode.AddItems(["Voice activation", "Push to talk", "Always on"]);
audioBuffer.AddItems(["Low latency (20 ms)", "Balanced (40 ms)", "Stable (60 ms)"]);
Add(view, "Audio buffering", audioBuffer, 555);
Add(view, "Sound volume", eventVolume, 515);
Add(view, "Input mode", mode, 465); Add(view, "VAD sensitivity", vad, 425); Add(view, "Microphone volume", inputGain, 385);
noiseReduction.Frame = new CGRect(160, 340, 260, 24); stereo.Frame = new CGRect(160, 315, 260, 24);
view.AddSubview(noiseReduction); view.AddSubview(stereo);
Add(view, "Input device", input, 295); Add(view, "Output device", output, 255); Add(view, "Output volume", outputGain, 215);
auxiliary.Frame = new CGRect(160, 165, 260, 24); view.AddSubview(auxiliary);
Add(view, "Aux device", auxiliaryDevice, 135); Add(view, "Aux volume", auxiliaryGain, 95);
sounds.Frame = new CGRect(25, 45, 110, 24); speech.Frame = new CGRect(145, 45, 110, 24);
selfTalk.Frame = new CGRect(265, 45, 180, 24); pttSound.Frame = new CGRect(25, 18, 180, 24);
view.AddSubview(sounds); view.AddSubview(speech); view.AddSubview(selfTalk); view.AddSubview(pttSound); view.AddSubview(pttKey);
foreach (NSControl control in new NSControl[] { mode, vad, inputGain, noiseReduction, stereo, input, output,
outputGain, auxiliary, auxiliaryDevice, auxiliaryGain, sounds, speech, selfTalk, pttSound, eventVolume, audioBuffer }) control.Activated += Changed;
pttKey.Activated += CapturePushToTalkKey;
Populate(input, true, settings.InputDeviceId); Populate(output, false, settings.OutputDeviceId); Populate(auxiliaryDevice, true, settings.AuxiliaryDeviceId);
LoadValues(); SetAccessibility();
}
private static void Add(NSView view, string label, NSView control, double y)
{
NSTextField text = NSTextField.CreateLabel(label); text.Frame = new CGRect(25, y, 125, 22); view.AddSubview(text); view.AddSubview(control);
}
private void Populate(NSPopUpButton picker, bool capture, string? selected)
{
picker.RemoveAllItems(); IReadOnlyList<AudioDeviceInfo> devices = audio.Enumerate(capture);
foreach (AudioDeviceInfo device in devices) { picker.AddItem(device.Name + (device.IsDefault ? " (default)" : "")); picker.LastItem!.RepresentedObject = new Foundation.NSString(device.Id); }
int index = devices.ToList().FindIndex(device => device.Id == selected);
if (index < 0) index = devices.ToList().FindIndex(device => device.IsDefault);
if (index >= 0) picker.SelectItem(index); picker.Enabled = devices.Count > 0;
}
private void LoadValues()
{
mode.SelectItem((int)settings.InputMode); vad.DoubleValue = settings.VadThreshold * 100;
inputGain.DoubleValue = settings.InputGain * 100; outputGain.DoubleValue = settings.OutputGain * 100;
noiseReduction.State = State(settings.InputNoiseReduction); stereo.State = State(settings.StereoMicrophone);
auxiliary.State = State(settings.AuxiliaryEnabled); auxiliaryGain.DoubleValue = settings.AuxiliaryGain * 100;
sounds.State = State(settings.EventSounds); speech.State = State(settings.SpokenEvents);
eventVolume.DoubleValue = settings.EventVolume * 100;
selfTalk.State = State(settings.SelfTalkSounds); pttSound.State = State(settings.PushToTalkSound);
pttKey.Title = $"PTT key code: {settings.PushToTalkKeyCode}";
audioBuffer.SelectItem(settings.AudioBufferMilliseconds switch { 20 => 0, 60 => 2, _ => 1 });
UpdateEnabled();
}
private void CapturePushToTalkKey(object? sender, EventArgs args)
{
var alert = new NSAlert { MessageText = "Set push-to-talk key", InformativeText = "Press the key to use while VoiceCat is focused." };
alert.AddButton("Cancel"); NSObject? monitor = null;
monitor = NSEvent.AddLocalMonitorForEventsMatchingMask(NSEventMask.KeyDown, value =>
{
settings.PushToTalkKeyCode = value.KeyCode; settings.Save(); pttKey.Title = $"PTT key code: {value.KeyCode}";
NSApplication.SharedApplication.AbortModal(); return null!;
});
try { alert.RunModal(); }
finally { if (monitor is not null) NSEvent.RemoveMonitor(monitor); }
}
private async void Changed(object? sender, EventArgs args)
{
settings.InputMode = (AudioInputMode)mode.IndexOfSelectedItem; settings.VadThreshold = (float)vad.DoubleValue / 100;
settings.InputGain = (float)inputGain.DoubleValue / 100; settings.OutputGain = (float)outputGain.DoubleValue / 100;
settings.InputNoiseReduction = On(noiseReduction); settings.StereoMicrophone = On(stereo);
settings.InputDeviceId = Selected(input); settings.OutputDeviceId = Selected(output);
settings.AuxiliaryEnabled = On(auxiliary); settings.AuxiliaryDeviceId = Selected(auxiliaryDevice);
settings.AuxiliaryGain = (float)auxiliaryGain.DoubleValue / 100; settings.EventSounds = On(sounds);
settings.EventVolume = (float)eventVolume.DoubleValue / 100; settings.SpokenEvents = On(speech);
settings.SelfTalkSounds = On(selfTalk); settings.PushToTalkSound = On(pttSound);
settings.AudioBufferMilliseconds = audioBuffer.IndexOfSelectedItem switch { 0 => 20, 2 => 60, _ => 40 };
settings.Save(); UpdateEnabled(); await owner.ApplySettingsAsync();
}
private void UpdateEnabled() { vad.Enabled = settings.InputMode == AudioInputMode.VoiceActivation; auxiliaryDevice.Enabled = auxiliaryGain.Enabled = settings.AuxiliaryEnabled; }
private void SetAccessibility()
{
((INSAccessibility)mode).AccessibilityLabel = "Microphone input mode"; ((INSAccessibility)vad).AccessibilityLabel = "Voice activation sensitivity";
((INSAccessibility)inputGain).AccessibilityLabel = "Microphone volume"; ((INSAccessibility)input).AccessibilityLabel = "Microphone device";
((INSAccessibility)output).AccessibilityLabel = "Output device"; ((INSAccessibility)outputGain).AccessibilityLabel = "Output volume";
((INSAccessibility)auxiliaryDevice).AccessibilityLabel = "Auxiliary input device"; ((INSAccessibility)auxiliaryGain).AccessibilityLabel = "Auxiliary input volume";
((INSAccessibility)eventVolume).AccessibilityLabel = "Event sound volume";
((INSAccessibility)audioBuffer).AccessibilityLabel = "Audio buffering";
}
private static bool On(NSButton button) => button.State == NSCellStateValue.On;
private static NSCellStateValue State(bool value) => value ? NSCellStateValue.On : NSCellStateValue.Off;
private static string? Selected(NSPopUpButton picker) => picker.SelectedItem?.RepresentedObject?.ToString();
}
@@ -0,0 +1,46 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0-macos27.0</TargetFramework>
<RuntimeIdentifier>osx-arm64</RuntimeIdentifier>
<SupportedOSPlatformVersion>14.0</SupportedOSPlatformVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<ApplicationTitle>VoiceCat</ApplicationTitle>
<ApplicationId>net.iamtalon.voicecat</ApplicationId>
<UseHardenedRuntime Condition="'$(Configuration)' == 'Release'">true</UseHardenedRuntime>
<UseHardenedRuntime Condition="'$(Configuration)' != 'Release'">false</UseHardenedRuntime>
<ApplicationManifest>Info.plist</ApplicationManifest>
<CodesignEntitlements>VoiceCat.Mac.entitlements</CodesignEntitlements>
<NoWarn>$(NoWarn);XCODE_27_0_PREVIEW</NoWarn>
<VoiceCatNativeMediaPath>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../../artifacts/native/runtimes/osx-arm64/native/libvoicecat_media.dylib'))</VoiceCatNativeMediaPath>
<VoiceCatNativeLicenseDirectory>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../../artifacts/native/licenses'))</VoiceCatNativeLicenseDirectory>
<_ComputePublishLocationDependsOn>VoiceCatPrepareNativeAssets;$(_ComputePublishLocationDependsOn)</_ComputePublishLocationDependsOn>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../../src/VoiceCat.Core/VoiceCat.Core.csproj" />
<BundleResource Include="../../../assets/sounds/*.wav" Link="Sounds/%(Filename)%(Extension)" />
</ItemGroup>
<!-- Every managed media project stages the same dylib for ordinary .NET consumers.
The macOS bundler preserves those transitive items separately and otherwise runs
install_name_tool against the same temporary file concurrently. Collapse them to
the single native reference the application actually ships. -->
<Target Name="VoiceCatPrepareNativeAssets">
<ItemGroup>
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'libvoicecat_media.dylib'" />
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'NOTICE.txt' or '%(Filename)%(Extension)' == 'Opus.txt' or '%(Filename)%(Extension)' == 'RNNoise.txt'" />
<ResolvedFileToPublish Include="$(VoiceCatNativeMediaPath)" Condition="Exists('$(VoiceCatNativeMediaPath)')">
<RelativePath>libvoicecat_media.dylib</RelativePath>
<PublishFolderType>DynamicLibrary</PublishFolderType>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</ResolvedFileToPublish>
<ResolvedFileToPublish Include="$(VoiceCatNativeLicenseDirectory)/*.txt">
<RelativePath>licenses/%(Filename)%(Extension)</RelativePath>
<PublishFolderType>Assembly</PublishFolderType>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</ResolvedFileToPublish>
</ItemGroup>
</Target>
</Project>
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>com.apple.security.app-sandbox</key><true/>
<key>com.apple.security.cs.allow-jit</key><true/>
<key>com.apple.security.network.client</key><true/>
<key>com.apple.security.device.audio-input</key><true/>
<key>com.apple.security.files.user-selected.read-write</key><true/>
</dict></plist>
@@ -0,0 +1,58 @@
{
"version": 1,
"dependencies": {
"net10.0-macos27.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.12, )",
"resolved": "10.0.12",
"contentHash": "xi+BDjFpW+Sb+MHFHaH6Y/gV9I8BluFwRXc1QyCdoZbIK26eNiBeFuMTe/FMwc33G1wdHCyDg7CVTmb8OdQrMQ=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0-macos27.0/osx-arm64": {}
}
}
@@ -0,0 +1,150 @@
using UIKit;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.iOS;
internal sealed class UsersController : UITableViewController
{
private readonly AppModel model; private IReadOnlyList<User> Visible => model.CurrentChannelId == 0 ? model.Users : model.Users.Where(user => user.ChannelId == model.CurrentChannelId).ToArray();
internal UsersController(AppModel model) { this.model = model; Title = "Users"; model.Changed += () => TableView.ReloadData(); }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "user"); }
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
User user = Visible[indexPath.Row]; UITableViewCell cell = tableView.DequeueReusableCell("user", indexPath); var content = cell.DefaultContentConfiguration;
content.Text = user.Nickname + (user.Id == model.SelfUserId ? " (you)" : "");
content.SecondaryText = user.ServerDeafened ? "server deafened" : user.ServerMuted ? "server muted" : user.SelfDeafened ? "deafened" : user.SelfMicMuted ? "muted" : user.IsGuest ? "guest" : "account";
content.Image = UIImage.GetSystemImage(user.ServerMuted || user.SelfMicMuted ? "mic.slash.fill" : "mic.fill"); cell.ContentConfiguration = content; cell.AccessibilityLabel = $"{content.Text}, {content.SecondaryText}";
cell.Accessory = user.Id == model.SelfUserId ? UITableViewCellAccessory.None : UITableViewCellAccessory.DisclosureIndicator;
if (user.Id != model.SelfUserId) cell.AccessibilityCustomActions =
[
new("Open user details", (Func<UIAccessibilityCustomAction, bool>)(_ => { Open(user.Id); return true; })),
new("Private message", (Func<UIAccessibilityCustomAction, bool>)(_ => { NavigationController?.PushViewController(new PrivateConversationController(model, user.Id, user.Nickname), true); return true; }))
];
return cell;
}
public override void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
User user = Visible[indexPath.Row]; tableView.DeselectRow(indexPath, true); if (user.Id == model.SelfUserId) return;
Open(user.Id);
}
private void Open(uint userId) => NavigationController?.PushViewController(new UserDetailController(model, userId), true);
}
internal sealed class UserDetailController : FormController
{
private readonly AppModel model; private readonly uint userId;
private User? User => model.Users.FirstOrDefault(value => value.Id == userId);
private IReadOnlyList<(string Title, Action Run, bool Destructive)> AdminActions
{
get
{
User? user = User; if (user is null) return [];
Permissions permissions = model.Client?.Permissions ?? new(); var result = new List<(string, Action, bool)>();
if (permissions.CanKick || permissions.IsAdmin) result.Add(("Kick", () => PromptReason(user, false), true));
if (permissions.CanBan || permissions.IsAdmin) result.Add(("Ban", () => NavigationController?.PushViewController(new BanUserController(model, user), true), true));
if (permissions.CanMoveUsers || permissions.IsAdmin) result.Add(("Move to channel", () => NavigationController?.PushViewController(new MoveUserController(model, user), true), false));
if (permissions.IsAdmin)
{
result.Add((user.ServerMuted ? "Server unmute" : "Server mute", () => Run(() => model.Client!.SetServerMuteAsync(user.Id, !user.ServerMuted, user.ServerDeafened)), false));
result.Add((user.ServerDeafened ? "Server undeafen" : "Server deafen", () => Run(() => model.Client!.SetServerMuteAsync(user.Id, user.ServerMuted, !user.ServerDeafened)), false));
result.Add(("Permissions", () => NavigationController?.PushViewController(new PermissionsController(model, user), true), false));
}
return result;
}
}
internal UserDetailController(AppModel model, uint userId) : base(model.Users.FirstOrDefault(value => value.Id == userId)?.Nickname ?? $"User {userId}")
{ this.model = model; this.userId = userId; model.Changed += Reload; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "detail"); Reload(); }
public override nint NumberOfSections(UITableView tableView) => AdminActions.Count == 0 ? 2 : 3;
public override nint RowsInSection(UITableView tableView, nint section) => section switch
{ 0 => Math.Max(User?.Streams.Count ?? 0, 1), 1 => 1, _ => AdminActions.Count };
public override string? TitleForHeader(UITableView tableView, nint section) => section switch { 0 => "Audio streams", 1 => "Conversation", _ => "Administration" };
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
User? user = User;
if (indexPath.Section == 0)
{
if (user is null || user.Streams.Count == 0)
{
UITableViewCell empty = TextCell(tableView, indexPath, "detail", user is null ? "User is offline" : "No active audio streams");
empty.SelectionStyle = UITableViewCellSelectionStyle.None; empty.AccessibilityTraits |= UIAccessibilityTrait.NotEnabled; return empty;
}
StreamInfo stream = user.Streams[indexPath.Row]; string kind = stream.Kind == StreamKind.StreamMic ? "Microphone" : "Screen audio";
(float Gain, bool Muted, bool NoiseReduction)? state = model.Client?.Audio.GetRemotePlayback(userId, stream.StreamId);
string status = state is null ? kind : $"{kind} · {state.Value.Gain:P0}{(state.Value.Muted ? " · muted" : "")}{(state.Value.NoiseReduction && stream.Kind == StreamKind.StreamMic ? " · noise reduction" : "")}";
string displayName = string.IsNullOrWhiteSpace(stream.Label) ? kind : stream.Label; UITableViewCell cell = TextCell(tableView, indexPath, "detail", displayName, status);
cell.Accessory = UITableViewCellAccessory.DisclosureIndicator; cell.AccessibilityLabel = $"{displayName}, {status}"; return cell;
}
if (indexPath.Section == 1)
{
UITableViewCell cell = TextCell(tableView, indexPath, "detail", "Private message", user is null ? "User is offline" : null);
cell.Accessory = UITableViewCellAccessory.DisclosureIndicator; cell.UserInteractionEnabled = user is not null; if (user is null) cell.AccessibilityTraits |= UIAccessibilityTrait.NotEnabled; return cell;
}
(string title, _, bool destructive) = AdminActions[indexPath.Row]; UITableViewCell action = TextCell(tableView, indexPath, "detail", title);
if (destructive) { UIListContentConfiguration content = action.DefaultContentConfiguration; content.TextProperties.Color = UIColor.SystemRed; action.ContentConfiguration = content; } return action;
}
public override void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
tableView.DeselectRow(indexPath, true); User? user = User; if (user is null) return;
if (indexPath.Section == 0 && user.Streams.Count > indexPath.Row)
NavigationController?.PushViewController(new PerUserTuningController(model, user.Id, user.Streams[indexPath.Row].StreamId), true);
else if (indexPath.Section == 1) NavigationController?.PushViewController(new PrivateConversationController(model, user.Id, user.Nickname), true);
else if (indexPath.Section == 2) AdminActions[indexPath.Row].Run();
}
private void Reload() { Title = User?.Nickname ?? $"User {userId}"; if (IsViewLoaded) TableView.ReloadData(); }
private void PromptReason(User user, bool ban) { UIAlertController prompt = UIAlertController.Create(ban ? "Ban user" : "Kick user", "Reason (optional)", UIAlertControllerStyle.Alert); prompt.AddTextField(field => field.AccessibilityLabel = "Reason"); prompt.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); prompt.AddAction(UIAlertAction.Create(ban ? "Ban" : "Kick", UIAlertActionStyle.Destructive, _ => Run(() => model.Client!.KickUserAsync(user.Id, prompt.TextFields?[0].Text ?? "")))); PresentViewController(prompt, true, null); }
private async void Run(Func<Task<GenericResult>> command) { try { GenericResult result = await model.RunAdminAsync(_ => command()); if (!result.Ok) throw new InvalidOperationException(result.Message); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
internal abstract class FormController : UITableViewController
{
protected FormController(string title) : base(UITableViewStyle.InsetGrouped) { Title = title; }
protected static UITableViewCell TextCell(UITableView table, NSIndexPath path, string id, string text, string? detail = null)
{ UITableViewCell cell = table.DequeueReusableCell(id, path); var content = cell.DefaultContentConfiguration; content.Text = text; content.SecondaryText = detail; cell.ContentConfiguration = content; return cell; }
protected static UISwitch Switch(bool value, string label, EventHandler handler) { var toggle = new UISwitch { On = value, AccessibilityLabel = label }; toggle.ValueChanged += handler; return toggle; }
}
internal sealed class PermissionsController : FormController
{
private readonly AppModel model; private readonly User user; private readonly string[] names = ["Create temporary channels", "Kick users", "Ban users", "Move users", "Manage accounts", "Administrator"];
private readonly bool[] values = new bool[6];
internal PermissionsController(AppModel model, User user) : base($"Permissions — {user.Nickname}") { this.model = model; this.user = user; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "permission"); NavigationItem.RightBarButtonItem = new("Save", UIBarButtonItemStyle.Done, async (_, _) => await Save()); }
public override nint RowsInSection(UITableView tableView, nint section) => names.Length;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath) { UITableViewCell cell = TextCell(tableView, indexPath, "permission", names[indexPath.Row]); int row = indexPath.Row; cell.AccessoryView = Switch(values[row], names[row], (_, _) => values[row] = ((UISwitch)cell.AccessoryView!).On); return cell; }
private async Task Save() { try { var permissions = new Permissions { CanCreateTempChannel = values[0], CanKick = values[1], CanBan = values[2], CanMoveUsers = values[3], CanAdminAccounts = values[4], IsAdmin = values[5] }; GenericResult result = await model.RunAdminAsync(client => client.SetPermissionsAsync(user.Id, permissions)); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
internal sealed class MoveUserController : FormController
{
private readonly AppModel model; private readonly User user; private uint selected;
internal MoveUserController(AppModel model, User user) : base($"Move {user.Nickname}") { this.model = model; this.user = user; selected = user.ChannelId; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel"); NavigationItem.RightBarButtonItem = new("Move", UIBarButtonItemStyle.Done, async (_, _) => await Move()); }
public override nint RowsInSection(UITableView tableView, nint section) => model.Channels.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath) { Channel channel = model.Channels[indexPath.Row]; UITableViewCell cell = TextCell(tableView, indexPath, "channel", channel.Name); cell.Accessory = channel.Id == selected ? UITableViewCellAccessory.Checkmark : UITableViewCellAccessory.None; cell.AccessibilityLabel = channel.Name + (channel.Id == selected ? ", selected" : ""); return cell; }
public override void RowSelected(UITableView tableView, NSIndexPath indexPath) { selected = model.Channels[indexPath.Row].Id; tableView.ReloadData(); }
private async Task Move() { try { GenericResult result = await model.RunAdminAsync(client => client.MoveUserAsync(user.Id, selected)); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
internal sealed class BanUserController : UIViewController
{
private readonly AppModel model; private readonly User user; private readonly UITextField reason = UiHelpers.Field("Reason (optional)"); private readonly UISegmentedControl duration = new(["Permanent", "1 hour", "1 day", "1 week"]);
internal BanUserController(AppModel model, User user) { this.model = model; this.user = user; Title = $"Ban {user.Nickname}"; }
public override void ViewDidLoad() { base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemGroupedBackground; duration.SelectedSegment = 0; duration.AccessibilityLabel = "Ban duration"; UIStackView stack = new([reason, duration]) { Axis = UILayoutConstraintAxis.Vertical, Spacing = 16, TranslatesAutoresizingMaskIntoConstraints = false }; View.AddSubview(stack); NSLayoutConstraint.ActivateConstraints([stack.TopAnchor.ConstraintEqualTo(View.SafeAreaLayoutGuide.TopAnchor, 24), stack.LeadingAnchor.ConstraintEqualTo(View.LayoutMarginsGuide.LeadingAnchor), stack.TrailingAnchor.ConstraintEqualTo(View.LayoutMarginsGuide.TrailingAnchor)]); NavigationItem.RightBarButtonItem = new("Ban", UIBarButtonItemStyle.Done, async (_, _) => await Ban()); }
private async Task Ban() { try { ulong expires = duration.SelectedSegment switch { 1 => (ulong)DateTimeOffset.UtcNow.AddHours(1).ToUnixTimeMilliseconds(), 2 => (ulong)DateTimeOffset.UtcNow.AddDays(1).ToUnixTimeMilliseconds(), 3 => (ulong)DateTimeOffset.UtcNow.AddDays(7).ToUnixTimeMilliseconds(), _ => 0 }; GenericResult result = await model.RunAdminAsync(client => client.BanUserAsync(user.Id, reason.Text ?? "", expires)); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
internal sealed class PerUserTuningController : FormController
{
private readonly AppModel model; private readonly uint userId, streamId; private readonly UISlider gain = new() { MinValue = 0, MaxValue = 4, Value = 1 };
private bool muted, noise;
private User? User => model.Users.FirstOrDefault(value => value.Id == userId);
private StreamInfo? Stream => User?.Streams.FirstOrDefault(value => value.StreamId == streamId);
internal PerUserTuningController(AppModel model, uint userId, uint streamId) : base("Audio tuning") { this.model = model; this.userId = userId; this.streamId = streamId; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "tuning"); StreamInfo? stream = Stream; Title = stream?.Label ?? "Audio tuning"; gain.AccessibilityLabel = $"Volume gain for {User?.Nickname ?? $"User {userId}"}, {stream?.Label ?? "stream"}"; gain.ValueChanged += (_, _) => Apply(); if (model.Client?.Audio.GetRemotePlayback(userId, streamId) is { } state) { gain.Value = state.Gain; muted = state.Muted; noise = state.NoiseReduction; } }
public override nint RowsInSection(UITableView tableView, nint section) => Stream?.Kind == StreamKind.StreamMic ? 3 : 2;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath) { string title = indexPath.Row switch { 0 => "Gain", 1 => "Mute", _ => "Noise reduction" }; UITableViewCell cell = TextCell(tableView, indexPath, "tuning", title); if (indexPath.Row == 0) cell.AccessoryView = gain; else { int row = indexPath.Row; cell.AccessoryView = Switch(row == 1 ? muted : noise, title, (_, _) => { bool value = ((UISwitch)cell.AccessoryView!).On; if (row == 1) muted = value; else noise = value; Apply(); }); } return cell; }
private void Apply() { if (model.Client is not { } client || Stream is null) return; client.Audio.SetRemotePlayback(userId, streamId, gain.Value, muted, noise); }
}
+18
View File
@@ -0,0 +1,18 @@
using Foundation;
using UIKit;
namespace VoiceCat.iOS;
[Register("AppDelegate")]
internal sealed class AppDelegate : UIApplicationDelegate
{
public override bool FinishedLaunching(UIApplication application, NSDictionary? launchOptions)
{
AppModel.Shared.Load();
return true;
}
public override UISceneConfiguration GetConfiguration(UIApplication application, UISceneSession connectingSceneSession,
UISceneConnectionOptions options) => new("Default Configuration", connectingSceneSession.Role)
{ DelegateType = typeof(SceneDelegate) };
}
+375
View File
@@ -0,0 +1,375 @@
using UIKit;
using VoiceCat.Audio;
using VoiceCat.Core;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.iOS;
internal sealed record ChatEntry(DateTime Timestamp, uint SenderId, uint PeerUserId, string Sender, string Peer, string Text, bool Private);
internal sealed record ActivityEntry(DateTime Timestamp, string Text);
internal sealed class AppModel
{
internal static AppModel Shared { get; } = new();
private readonly IosStorage storage = new();
private readonly List<ServerProfile> profiles = [];
private readonly List<ChatEntry> messages = [];
private readonly List<ActivityEntry> activity = [];
private readonly IosSettings settings = new();
private readonly EventFeedback feedback;
private CancellationTokenSource? lifetime;
private VoiceCatClient? client;
private TaskCompletionSource<bool>? identityDecision;
private ServerProfile? connectedProfile;
private int reconnectAttempt;
private bool explicitDisconnect;
private uint microphoneStream;
private BroadcastAudioPump? broadcast;
private Timer? levelTimer;
private bool lastTalking;
private bool restoringVoice;
private uint restoreChannel;
private bool restoreMuted;
private bool restoreDeafened;
private bool backgrounded;
private int diagnosticPolls;
internal event Action? Changed;
internal event Action<ServerIdentityChallenge>? IdentityRequested;
internal IReadOnlyList<ServerProfile> Profiles => profiles;
internal IReadOnlyList<ChatEntry> Messages => messages;
internal IReadOnlyList<ActivityEntry> Activity => activity;
internal IosSettings Settings => settings;
internal VoiceCatClient? Client => client;
internal bool IsConnected => client?.State == ClientConnectionState.Connected;
internal bool SupportsAdaptivePacketLoss => client?.SupportsAdaptivePacketLoss == true;
internal bool IsConnecting { get; private set; }
internal bool VoiceJoined => microphoneStream != 0;
internal bool ScreenSharing => broadcast?.IsActive == true;
internal bool IsBackgrounded => backgrounded;
internal float MicrophoneLevel { get; private set; }
internal string Status { get; private set; } = "Not connected";
internal uint CurrentChannelId { get; private set; }
internal uint SelfUserId => client?.Authentication?.Self.Id ?? 0;
internal IReadOnlyList<Channel> Channels => client?.Channels ?? [];
internal IReadOnlyList<User> Users => client?.Users ?? [];
private AppModel() { feedback = new(settings); }
internal void Load() { profiles.Clear(); profiles.AddRange(storage.LoadProfiles()); settings.Load(); IosAudioRouter.Shared.Load(); Notify(); }
internal void Save() { storage.SaveProfiles(profiles); settings.Save(); }
internal void DidEnterBackground()
{
backgrounded = true; Save();
// Do not stop or rebuild AVAudioEngine here. With the `audio` background mode and an
// active PlayAndRecord session, capture, playback, media UDP, and screen-ring draining
// remain live while the scene is backgrounded or the device is locked.
}
internal void WillEnterForeground()
{
backgrounded = false;
IosAudioRouter.Shared.Recover("foreground");
}
internal void DidBecomeActive()
{
backgrounded = false;
IosAudioRouter.Shared.EnsureAudio("active scene");
Notify();
}
internal void UpsertProfile(ServerProfile profile, string? password)
{
int index = profiles.FindIndex(item => item.Id == profile.Id);
if (index < 0) profiles.Add(profile); else profiles[index] = profile;
if (!string.IsNullOrEmpty(password)) storage.SavePassword(profile.Id, password);
Save(); Notify();
}
internal void RemoveProfile(ServerProfile profile)
{
storage.RemovePassword(profile.Id); profiles.RemoveAll(item => item.Id == profile.Id); Save(); Notify();
}
internal async Task ConnectAsync(ServerProfile profile, string? suppliedPassword = null, bool restoring = false)
{
if (IsConnecting || IsConnected) return;
explicitDisconnect = false; IsConnecting = true; connectedProfile = profile;
Status = restoring ? "Reconnecting…" : "Connecting…"; Notify();
lifetime?.Cancel(); lifetime?.Dispose(); lifetime = new();
// The AVAudioSourceNode render callback drives Audio.RunCycle and the 20 ms capture
// handoff; a sleep-paced audio worker stalls when iOS coalesces backgrounded wakeups and
// the call glitches after several minutes in the background.
VoiceCatClient next = new("VoiceCat-iOS", "0.0.1", storage.TofuPath, deviceClockedAudio: true);
next.ConnectionStateChanged += state =>
{
if (state == ClientConnectionState.Disconnected && next.ConnectionFailure is { } failure)
Console.Error.WriteLine($"VoiceCat control connection failed: {failure}");
if (state == ClientConnectionState.Disconnected)
UIApplication.SharedApplication.BeginInvokeOnMainThread(async () =>
{
try { await HandleConnectionLostAsync(next); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Reconnect cleanup failed: {exception}"); }
});
};
client = next;
try
{
await next.ConnectAsync(profile.Host, profile.Port, ConfirmIdentityAsync, lifetime.Token);
AuthResult auth = profile.Authentication == ServerAuthentication.Guest
? await next.AuthenticateGuestAsync(profile.Nickname ?? "iOS User", lifetime.Token)
: await next.AuthenticateUserAsync(profile.Username!, suppliedPassword ?? storage.LoadPassword(profile) ?? "", lifetime.Token);
if (!auth.Ok) throw new InvalidOperationException(auth.Error);
CurrentChannelId = auth.Self.ChannelId; reconnectAttempt = 0; IsConnecting = false; Status = "Connected";
ApplyAudioSettings(next);
IosAudioEngine.Shared.StartListening(next);
broadcast = new(); broadcast.Changed += BroadcastChanged; broadcast.Start(next);
_ = PumpEventsAsync(next, lifetime.Token);
levelTimer?.Dispose(); levelTimer = new(_ =>
UIApplication.SharedApplication.BeginInvokeOnMainThread(PollAudio), null, 50, 50);
feedback.Play(SoundEvent.Login); feedback.Speak(restoring ? "Reconnected" : "Connected");
if (restoring && restoreChannel != 0) await RestoreSessionAsync(next);
Notify();
}
catch (Exception exception)
{
System.Diagnostics.Debug.WriteLine($"Connection failed: {exception}");
IsConnecting = false; Status = exception.Message;
if (ReferenceEquals(client, next)) { client = null; await StopSessionResourcesAsync(); }
await next.DisposeAsync();
Notify();
if (restoring && !explicitDisconnect) ScheduleReconnect();
else throw;
}
}
private ValueTask<bool> ConfirmIdentityAsync(ServerIdentityChallenge challenge, CancellationToken token)
{
identityDecision = new(TaskCreationOptions.RunContinuationsAsynchronously);
token.Register(() => identityDecision.TrySetCanceled(token));
UIApplication.SharedApplication.BeginInvokeOnMainThread(() => IdentityRequested?.Invoke(challenge));
return new(identityDecision.Task);
}
internal void ResolveIdentity(bool accepted) { identityDecision?.TrySetResult(accepted); identityDecision = null; }
private async Task PumpEventsAsync(VoiceCatClient owner, CancellationToken token)
{
try
{
await foreach (Envelope envelope in owner.ReadEventsAsync(token))
{
if (envelope.JoinChannelResult?.Ok == true) CurrentChannelId = envelope.JoinChannelResult.ChannelId;
if (envelope.JoinChannelResult is { Ok: false } joinFailure) AddActivity("Join failed: " + joinFailure.Error);
if (envelope.TextMessage is { } text)
{
User? sender = owner.Users.FirstOrDefault(user => user.Id == text.SenderId);
bool privateMessage = text.Scope == TextScope.TextPrivate;
uint peerUserId = privateMessage ? text.SenderId == SelfUserId ? text.TargetId : text.SenderId : 0;
User? peer = privateMessage ? owner.Users.FirstOrDefault(user => user.Id == peerUserId) : null;
messages.Add(new(DateTime.Now, text.SenderId, peerUserId, sender?.Nickname ?? $"User {text.SenderId}", peer?.Nickname ?? $"User {peerUserId}", text.Body, privateMessage));
if (messages.Count > 500) messages.RemoveAt(0);
bool self = text.SenderId == SelfUserId;
feedback.Play(text.Scope == TextScope.TextPrivate
? self ? SoundEvent.PrivateSent : SoundEvent.PrivateReceived
: self ? SoundEvent.ChannelSent : SoundEvent.ChannelReceived);
if (!self) feedback.Speak(text.Scope == TextScope.TextPrivate ? $"Private message from {sender?.Nickname}: {text.Body}" : $"{sender?.Nickname}: {text.Body}");
}
if (envelope.UserEvent is { } userEvent) HandleUserEvent(owner, userEvent);
if (envelope.StreamState is { } streamState) AddActivity($"{Name(owner, streamState.UserId)} {(streamState.Talking ? "started" : "stopped")} talking");
if (envelope.StreamAnnounceResult?.Ok == false) AddActivity("Stream failed: " + envelope.StreamAnnounceResult.Error);
if (envelope.GenericResult is { Ok: false } result) AddActivity("Operation failed: " + result.Message);
if (envelope.Disconnect is { } disconnected) AddActivity("Disconnected: " + disconnected.Reason);
UIApplication.SharedApplication.BeginInvokeOnMainThread(Notify);
}
}
catch (OperationCanceledException) { }
finally
{
await HandleConnectionLostAsync(owner);
}
}
private async Task HandleConnectionLostAsync(VoiceCatClient owner)
{
if (!ReferenceEquals(client, owner) || IsConnecting || explicitDisconnect) return;
// A failed TLS reader changes State but leaves the event channel open until disposal.
// Handle that state change directly so a Wi-Fi transition cannot strand this session.
CaptureRestoreState(owner);
client = null; Status = "Connection lost";
try { await StopSessionResourcesAsync(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio cleanup failed: {exception}"); }
try { await owner.DisposeAsync(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Connection cleanup failed: {exception}"); }
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting"); Notify();
ScheduleReconnect();
}
internal async Task JoinChannelAsync(uint channelId, string password = "")
{
VoiceCatClient active = client ?? throw new InvalidOperationException("Not connected.");
Envelope response = await active.RequestAsync(new() { JoinChannel = new() { ChannelId = channelId, Password = password } });
if (response.JoinChannelResult?.Ok != true) throw new InvalidOperationException(response.JoinChannelResult?.Error ?? "Join failed.");
CurrentChannelId = channelId; Notify();
}
internal void SendText(string body, uint targetUser = 0)
{
if (string.IsNullOrWhiteSpace(body)) return;
(client ?? throw new InvalidOperationException("Not connected.")).Send(new()
{
TextMessage = new() { Scope = targetUser == 0 ? TextScope.TextChannel : TextScope.TextPrivate,
TargetId = targetUser == 0 ? CurrentChannelId : targetUser, Body = body.Trim(), ClientMsgId = Guid.NewGuid().ToString("N") }
});
}
internal async Task ToggleVoiceAsync()
{
VoiceCatClient active = client ?? throw new InvalidOperationException("Not connected.");
if (microphoneStream != 0)
{
IosAudioEngine.Shared.StopMicrophone(); active.StopStream(microphoneStream); microphoneStream = 0;
await active.SubscribeVoiceAsync(false); MicrophoneLevel = 0; feedback.Play(SoundEvent.VoiceOff); Notify(); return;
}
if (AVFoundation.AVCaptureDevice.GetAuthorizationStatus(AVFoundation.AVAuthorizationMediaType.Audio) == AVFoundation.AVAuthorizationStatus.NotDetermined)
await AVFoundation.AVCaptureDevice.RequestAccessForMediaTypeAsync(AVFoundation.AVAuthorizationMediaType.Audio);
VoiceSubscriptionResult subscribed = await active.SubscribeVoiceAsync();
if (!subscribed.Ok) throw new InvalidOperationException(subscribed.Error);
int channels = IosAudioRouter.Shared.CaptureChannels;
StreamInfo stream = await active.StartStreamAsync(StreamKind.StreamMic, "Microphone", channels);
microphoneStream = stream.StreamId; IosAudioEngine.Shared.StartMicrophone(stream.StreamId, channels); feedback.Play(SoundEvent.VoiceOn); Notify();
}
internal void ToggleScreenAudio()
{
if (!OperatingSystem.IsIOSVersionAtLeast(27)) throw new PlatformNotSupportedException("Use the ReplayKit broadcast picker on this iOS version.");
if (ScreenSharing) { IosScreenCapture.Stop(); broadcast?.RequestStop(); }
else
{
if (!IsConnected || CurrentChannelId == 0) throw new InvalidOperationException("Connect and join a channel before sharing screen audio.");
IosScreenCapture.Present();
}
}
internal void SetSelfAudio(bool muted, bool deafened) { client?.SetSelfAudioState(muted, deafened); Notify(); }
internal void SetPushToTalk(bool active)
{
if (client is null) return; client.Audio.PushToTalk = active;
if (active) feedback.Play(SoundEvent.PushToTalk); Notify();
}
internal void ApplyVoiceSettings()
{
if (client is { } active) ApplyAudioSettings(active); settings.Save(); Notify();
}
internal async Task<GenericResult> RunAdminAsync(Func<VoiceCatClient, Task<GenericResult>> action)
{
GenericResult result = await action(client ?? throw new InvalidOperationException("Not connected."));
AddActivity(result.Ok ? (string.IsNullOrEmpty(result.Message) ? "Operation completed." : result.Message) : "Operation failed: " + result.Message);
Notify(); return result;
}
internal async Task DisconnectAsync()
{
explicitDisconnect = true; lifetime?.Cancel(); await StopSessionResourcesAsync();
VoiceCatClient? old = client; client = null; microphoneStream = 0; IsConnecting = false; Status = "Not connected"; Notify();
if (old is not null) await old.DisposeAsync(); feedback.Play(SoundEvent.Logout); feedback.Speak("Disconnected");
}
private void ScheduleReconnect()
{
ServerProfile? profile = connectedProfile; if (profile is null || explicitDisconnect) return;
int delay = Math.Min(1 << Math.Min(reconnectAttempt++, 5), 30);
CancellationToken token = lifetime?.Token ?? default;
_ = Task.Run(async () =>
{
try { await Task.Delay(TimeSpan.FromSeconds(delay), token); }
catch (OperationCanceledException) { return; }
UIApplication.SharedApplication.BeginInvokeOnMainThread(async () =>
{
if (token.IsCancellationRequested || explicitDisconnect) return;
try { await ConnectAsync(profile, restoring: true); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Reconnect failed: {exception}"); }
});
});
}
private void Notify() => Changed?.Invoke();
private void ApplyAudioSettings(VoiceCatClient owner)
{
owner.Audio.InputMode = settings.InputMode; owner.Audio.VadThreshold = settings.VadThreshold;
owner.Audio.InputGain = settings.InputGain; owner.Audio.OutputGain = settings.OutputGain;
owner.Audio.InputNoiseReduction = settings.InputNoiseReduction;
owner.Audio.DeviceBufferMilliseconds = settings.AudioBufferMilliseconds;
IosAudioEngine.Shared.BufferMilliseconds = settings.AudioBufferMilliseconds;
}
private void PollAudio()
{
VoiceCatClient? owner = client; uint stream = microphoneStream;
if (owner is null || owner.State != ClientConnectionState.Connected || stream == 0) return;
(float level, bool talking) = owner.Audio.GetLocalLevel(stream); MicrophoneLevel = level;
if (++diagnosticPolls >= 20)
{
diagnosticPolls = 0;
LocalAudioDiagnostics audio = owner.Audio.GetLocalDiagnostics(stream);
Console.Error.WriteLine($"VC_AUDIO {IosAudioEngine.Shared.CaptureDiagnostics()} cycles={audio.Cycles} starved={audio.StarvedCycles} " +
$"encoded={audio.EncodedPackets} packetDrops={audio.RejectedPackets} queued={audio.BufferedFrames}");
}
if (talking != lastTalking)
{
try { owner.PublishStreamState(stream, talking); }
catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { return; }
lastTalking = talking; feedback.Play(talking ? SoundEvent.VoiceStart : SoundEvent.VoiceStop);
}
Notify();
}
private void HandleUserEvent(VoiceCatClient owner, UserEvent value)
{
string name = value.User?.Nickname ?? Name(owner, value.LeftId);
if (value.Kind == UserEvent.Types.Kind.Joined && value.User?.ChannelId == CurrentChannelId && value.User.Id != SelfUserId)
{ AddActivity($"{name} joined"); feedback.Play(SoundEvent.ChannelJoin); feedback.Speak($"{name} joined"); }
else if (value.Kind == UserEvent.Types.Kind.Left)
{ AddActivity($"{name} left"); feedback.Play(SoundEvent.ChannelLeave); feedback.Speak($"{name} left"); }
User? self = owner.Users.FirstOrDefault(user => user.Id == SelfUserId); if (self is not null) CurrentChannelId = self.ChannelId;
}
private void AddActivity(string text) { activity.Add(new(DateTime.Now, text)); if (activity.Count > 500) activity.RemoveAt(0); }
private static string Name(VoiceCatClient owner, uint id) => owner.Users.FirstOrDefault(user => user.Id == id)?.Nickname ?? $"User {id}";
private void BroadcastChanged()
{
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
// Presenting either system picker can replace or interrupt the app's audio session.
// Rebuild after the producer becomes active so playback and an existing mic tap are
// attached to the session that will remain in use for the broadcast.
if (ScreenSharing) IosAudioRouter.Shared.Recover("screen sharing started");
Notify();
});
}
private void CaptureRestoreState(VoiceCatClient owner)
{
restoreChannel = CurrentChannelId; restoringVoice = microphoneStream != 0;
restoreMuted = owner.Audio.MicMuted; restoreDeafened = owner.Audio.Deafened;
}
private async Task RestoreSessionAsync(VoiceCatClient owner)
{
Envelope joined = await owner.RequestAsync(new() { JoinChannel = new() { ChannelId = restoreChannel } });
if (joined.JoinChannelResult?.Ok != true) { AddActivity("Could not restore the previous channel."); return; }
CurrentChannelId = restoreChannel;
if (restoringVoice) await ToggleVoiceAsync();
owner.SetSelfAudioState(restoreMuted, restoreDeafened); AddActivity($"Restored to channel {restoreChannel}{(restoringVoice ? " with voice" : "")}");
}
private async Task StopSessionResourcesAsync()
{
levelTimer?.Dispose(); levelTimer = null; IosAudioEngine.Shared.Stop(); microphoneStream = 0; MicrophoneLevel = 0; lastTalking = false;
if (broadcast is { } pump) { broadcast = null; pump.Changed -= BroadcastChanged; await pump.DisposeAsync(); }
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

@@ -0,0 +1,14 @@
{
"images" : [
{
"filename" : "AppIcon-1024.png",
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,136 @@
using System.IO.MemoryMappedFiles;
using VoiceCat.Core;
using Voicecat.V1;
namespace VoiceCat.iOS;
internal sealed class BroadcastAudioPump : IAsyncDisposable
{
private const uint Magic = 0x56434252, Version = 1;
private const int Header = 64, Capacity = 96_000, Frame = 960;
private readonly CancellationTokenSource stop = new();
private readonly short[] scratch = new short[Frame * 2];
private Thread? worker;
private VoiceCatClient? client;
private MemoryMappedFile? map;
private MemoryMappedViewAccessor? view;
private string? path;
private uint streamId;
private bool active;
private int generation;
internal event Action? Changed;
internal bool IsActive => active;
internal void Start(VoiceCatClient owner)
{
client = owner;
worker = new Thread(Run) { IsBackground = true, Name = "VoiceCat iOS screen audio", Priority = ThreadPriority.AboveNormal };
worker.Start();
}
private void Run()
{
CancellationToken token = stop.Token;
try
{
while (!token.IsCancellationRequested)
{
try { DrainAsync(token).GetAwaiter().GetResult(); }
catch (Exception exception) when (exception is IOException or UnauthorizedAccessException or InvalidDataException or OperationCanceledException) { CloseMapping(); }
if (!token.IsCancellationRequested) Thread.Sleep(5);
}
}
finally { CloseMapping(); }
}
private async Task DrainAsync(CancellationToken token)
{
if (!OpenMapping()) return;
MemoryMappedViewAccessor pages = view!;
bool active = pages.ReadUInt32(16) != 0;
if (!active) { StopStream(); SetActive(false); return; }
VoiceCatClient owner = client ?? throw new IOException("Client disconnected.");
if (streamId == 0)
{
int startGeneration = Volatile.Read(ref generation);
StreamInfo stream = await owner.StartStreamAsync(StreamKind.StreamScreenAudio, "Screen audio", 2, token).ConfigureAwait(false);
if (startGeneration != Volatile.Read(ref generation) || pages.ReadUInt32(16) == 0 || !ReferenceEquals(client, owner))
{
try { owner.StopStream(stream.StreamId); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
return;
}
streamId = stream.StreamId; pages.Write(32, pages.ReadUInt64(24)); SetActive(true);
}
ulong write = pages.ReadUInt64(24), read = pages.ReadUInt64(32);
if (write - read > Capacity) read = write - Capacity;
while (write - read >= Frame * 2)
{
for (int sample = 0; sample < scratch.Length; sample++)
{
ulong index = (read + (ulong)sample) % Capacity;
scratch[sample] = pages.ReadInt16(Header + checked((long)index * sizeof(short)));
}
if (!owner.Audio.FeedPcm(streamId, scratch, 2)) break;
read += (ulong)scratch.Length;
pages.Write(32, read);
}
}
// The producer opens the ring with O_CREAT and never replaces it, so one mapping covers the
// pump's whole life. Creating and disposing a mapping (plus its container lookup and path
// strings) every 5 ms tick allocated steadily at 200 Hz and churned the GC under long calls;
// the mapping is now built on the ring's transitions and reused until the file disappears or
// a drain fails.
private bool OpenMapping()
{
path ??= ResolvePath();
if (path is null) return false;
bool present = File.Exists(path);
if (view is not null)
{
if (present) return true;
CloseMapping(); return false;
}
if (!present) return false;
MemoryMappedFile candidate = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
try
{
MemoryMappedViewAccessor pages = candidate.CreateViewAccessor(0, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
if (pages.ReadUInt32(0) != Magic || pages.ReadUInt32(4) != Version)
{
pages.Dispose(); throw new InvalidDataException("Unsupported broadcast ring.");
}
map = candidate; view = pages; return true;
}
catch { candidate.Dispose(); throw; }
}
private static string? ResolvePath()
{
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
return root?.Path is null ? null : Path.Combine(root.Path, "voicecat", "broadcast_audio.ring");
}
private void CloseMapping()
{
view?.Dispose(); view = null;
map?.Dispose(); map = null;
}
private void StopStream()
{
uint id = streamId; streamId = 0; if (id == 0 || client?.State != ClientConnectionState.Connected) return;
try { client.StopStream(id); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
}
internal void RequestStop() { Interlocked.Increment(ref generation); SetActive(false); }
private void SetActive(bool value) { if (active == value) return; active = value; Changed?.Invoke(); }
public ValueTask DisposeAsync()
{
stop.Cancel(); Interlocked.Increment(ref generation); worker?.Join(); worker = null;
StopStream(); SetActive(false); stop.Dispose(); return ValueTask.CompletedTask;
}
}
@@ -0,0 +1,57 @@
using UIKit;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.iOS;
internal sealed class ChannelEditorController : UIViewController
{
private readonly AppModel model; private readonly Channel? existing;
private readonly UITextField name = UiHelpers.Field("Channel name"), topic = UiHelpers.Field("Topic (optional)"), maximum = UiHelpers.Field("Maximum users, 0 is unlimited"), order = UiHelpers.Field("Sort order"), password = UiHelpers.Field("Password; blank preserves existing", true), bitrate = UiHelpers.Field("Bitrate in bits per second"), loss = UiHelpers.Field("Expected packet loss percent"), complexity = UiHelpers.Field("Complexity 0 through 10");
private readonly UISegmentedControl type = new(["Permanent", "Temporary"]), mode = new(["Mono", "Stereo"]), application = new(["VoIP", "Audio", "Low delay"]);
private readonly UIButton parent = UIButton.FromType(UIButtonType.System), sampleRate = UIButton.FromType(UIButtonType.System), frame = UIButton.FromType(UIButtonType.System);
private readonly UIButton lossSpeed = UIButton.FromType(UIButtonType.System);
private readonly UISwitch automaticLoss = new(), fec = new(), dtx = new(), dred = new(); private uint parentId;
internal ChannelEditorController(AppModel model, Channel? existing) { this.model = model; this.existing = existing; Title = existing is null ? "New Channel" : "Edit Channel"; }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemGroupedBackground; UIScrollView scroll = new() { TranslatesAutoresizingMaskIntoConstraints = false }; UIStackView stack = new() { Axis = UILayoutConstraintAxis.Vertical, Spacing = 12, TranslatesAutoresizingMaskIntoConstraints = false };
View.AddSubview(scroll); scroll.AddSubview(stack); NSLayoutConstraint.ActivateConstraints([scroll.TopAnchor.ConstraintEqualTo(View.SafeAreaLayoutGuide.TopAnchor), scroll.BottomAnchor.ConstraintEqualTo(View.BottomAnchor), scroll.LeadingAnchor.ConstraintEqualTo(View.LeadingAnchor), scroll.TrailingAnchor.ConstraintEqualTo(View.TrailingAnchor), stack.TopAnchor.ConstraintEqualTo(scroll.ContentLayoutGuide.TopAnchor, 16), stack.BottomAnchor.ConstraintEqualTo(scroll.ContentLayoutGuide.BottomAnchor, -24), stack.LeadingAnchor.ConstraintEqualTo(scroll.FrameLayoutGuide.LeadingAnchor, 20), stack.TrailingAnchor.ConstraintEqualTo(scroll.FrameLayoutGuide.TrailingAnchor, -20)]);
foreach (UIView row in new UIView[] { name, topic, PickerRow("Parent channel", parent), PickerRow("Channel type", type), maximum, order, password, PickerRow("Channel mode", mode), PickerRow("Sample rate", sampleRate), PickerRow("Frame duration", frame), PickerRow("Opus application", application), bitrate, SwitchRow("Automatic packet loss", automaticLoss), PickerRow("Adaptation speed", lossSpeed), loss, complexity, SwitchRow("Forward error correction", fec), SwitchRow("Discontinuous transmission", dtx), SwitchRow("Deep redundancy", dred) }) stack.AddArrangedSubview(row);
maximum.KeyboardType = order.KeyboardType = bitrate.KeyboardType = loss.KeyboardType = complexity.KeyboardType = UIKeyboardType.NumberPad;
parent.Menu = UIMenu.Create(model.Channels.Where(channel => channel.Id != existing?.Id).OrderBy(channel => channel.Name).Select(channel => UIAction.Create(channel.Name, null, null, _ => { parentId = channel.Id; parent.SetTitle(channel.Name, UIControlState.Normal); })).Prepend(UIAction.Create("Root", null, null, _ => { parentId = 0; parent.SetTitle("Root", UIControlState.Normal); })).ToArray()); parent.ShowsMenuAsPrimaryAction = true;
sampleRate.Menu = Choice(sampleRate, ["48000"]); sampleRate.ShowsMenuAsPrimaryAction = true; frame.Menu = Choice(frame, ["5", "10", "20", "40", "60"]); frame.ShowsMenuAsPrimaryAction = true;
lossSpeed.Menu = Choice(lossSpeed, ["Fast", "Balanced", "Stable"]); lossSpeed.ShowsMenuAsPrimaryAction = true;
automaticLoss.Enabled = model.SupportsAdaptivePacketLoss;
automaticLoss.ValueChanged += (_, _) => UpdateLossControls();
Load(); NavigationItem.RightBarButtonItem = new("Save", UIBarButtonItemStyle.Done, async (_, _) => await Save());
}
private void Load()
{
Channel channel = existing ?? new Channel { Audio = new AudioConfig { SampleRate = 48000, BitrateBps = 64000, FrameMs = 20, Fec = true, Complexity = 10, ExpectedPacketLoss = 5 } };
name.Text = channel.Name; topic.Text = channel.Topic; maximum.Text = channel.MaxUsers.ToString(); order.Text = channel.Order.ToString(); parentId = channel.ParentId;
parent.SetTitle(model.Channels.FirstOrDefault(value => value.Id == parentId)?.Name ?? "Root", UIControlState.Normal); type.SelectedSegment = channel.Type == ChannelType.ChannelTemporary ? 1 : 0; mode.SelectedSegment = channel.Audio?.Mode == ChannelMode.ModeStereo ? 1 : 0;
sampleRate.SetTitle((channel.Audio?.SampleRate ?? 48000).ToString(), UIControlState.Normal); frame.SetTitle((channel.Audio?.FrameMs ?? 20).ToString(), UIControlState.Normal); application.SelectedSegment = (nint)(channel.Audio?.Application ?? OpusApplication.OpusVoip);
bitrate.Text = (channel.Audio?.BitrateBps ?? 64000).ToString(); loss.Text = (channel.Audio?.ExpectedPacketLoss ?? 5).ToString(); complexity.Text = (channel.Audio?.Complexity ?? 10).ToString(); fec.On = channel.Audio?.Fec ?? true; dtx.On = channel.Audio?.Dtx ?? false; dred.On = channel.Audio?.Dred ?? false;
automaticLoss.On = channel.Audio?.PacketLossMode != PacketLossMode.PacketLossManual;
lossSpeed.SetTitle(channel.Audio?.PacketLossMode switch { PacketLossMode.PacketLossAutoFast => "Fast", PacketLossMode.PacketLossAutoStable => "Stable", _ => "Balanced" }, UIControlState.Normal);
UpdateLossControls();
}
private async Task Save()
{
try
{
if (string.IsNullOrWhiteSpace(name.Text) || !uint.TryParse(maximum.Text, out uint max) || !int.TryParse(order.Text, out int sort) || !uint.TryParse(sampleRate.Title(UIControlState.Normal), out uint rate) || !uint.TryParse(frame.Title(UIControlState.Normal), out uint frameMs) || !uint.TryParse(bitrate.Text, out uint bits) || !uint.TryParse(loss.Text, out uint packetLoss) || !uint.TryParse(complexity.Text, out uint cpu) || packetLoss > 100 || cpu > 10) throw new ArgumentException("Enter valid channel and Opus settings.");
var channel = new Channel { Id = existing?.Id ?? 0, ParentId = parentId, Name = name.Text.Trim(), Topic = topic.Text?.Trim() ?? "", MaxUsers = max, Order = sort, Type = type.SelectedSegment == 1 ? ChannelType.ChannelTemporary : ChannelType.ChannelPermanent,
Audio = new AudioConfig { Codec = 0, Mode = mode.SelectedSegment == 1 ? ChannelMode.ModeStereo : ChannelMode.ModeMono, SampleRate = rate, FrameMs = frameMs, Application = (OpusApplication)(int)application.SelectedSegment, BitrateBps = bits, ExpectedPacketLoss = packetLoss, Complexity = cpu, Fec = fec.On, Dtx = dtx.On, Dred = dred.On,
PacketLossMode = automaticLoss.On ? lossSpeed.Title(UIControlState.Normal) switch { "Fast" => PacketLossMode.PacketLossAutoFast, "Stable" => PacketLossMode.PacketLossAutoStable, _ => PacketLossMode.PacketLossAutoBalanced } : PacketLossMode.PacketLossManual } };
GenericResult result = await model.RunAdminAsync(client => existing is null ? client.CreateChannelAsync(channel, password.Text ?? "") : client.EditChannelAsync(channel, password.Text ?? "")); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true);
}
catch (Exception exception) { UiHelpers.ShowError(this, exception); }
}
private void UpdateLossControls() { loss.Enabled = !automaticLoss.On; lossSpeed.Enabled = model.SupportsAdaptivePacketLoss && automaticLoss.On; }
private static UIView PickerRow(string label, UIView control) { var row = new UIStackView { Axis = UILayoutConstraintAxis.Horizontal, Distribution = UIStackViewDistribution.FillEqually, Spacing = 8 }; var text = new UILabel { Text = label }; control.AccessibilityLabel = label; row.AddArrangedSubview(text); row.AddArrangedSubview(control); return row; }
private static UIView SwitchRow(string label, UISwitch toggle) { var row = new UIStackView { Axis = UILayoutConstraintAxis.Horizontal, Distribution = UIStackViewDistribution.EqualSpacing }; row.AddArrangedSubview(new UILabel { Text = label }); toggle.AccessibilityLabel = label; row.AddArrangedSubview(toggle); return row; }
private static UIMenu Choice(UIButton button, string[] values) => UIMenu.Create(values.Select(value => UIAction.Create(value, null, null, _ => button.SetTitle(value, UIControlState.Normal))).ToArray());
}
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>com.apple.security.application-groups</key><array><string>group.me.iamtalon.voicecat</string></array>
</dict></plist>
@@ -0,0 +1,49 @@
using AVFoundation;
using Foundation;
namespace VoiceCat.iOS;
internal enum SoundEvent { ChannelJoin, ChannelLeave, ChannelReceived, ChannelSent, PrivateReceived, PrivateSent, Login, Logout, ConnectionLost, VoiceOn, VoiceOff, VoiceStart, VoiceStop, PushToTalk }
internal sealed class EventFeedback : IDisposable
{
private readonly IosSettings settings;
private readonly AVSpeechSynthesizer speech = new();
private readonly Dictionary<SoundEvent, AVAudioPlayer> players = [];
internal EventFeedback(IosSettings settings) => this.settings = settings;
internal void Play(SoundEvent sound)
{
if (!settings.EventSounds || settings.EventVolume <= 0 ||
(sound is SoundEvent.VoiceStart or SoundEvent.VoiceStop && !settings.SelfTalkSounds) ||
(sound == SoundEvent.PushToTalk && !settings.PushToTalkSound)) return;
if (!players.TryGetValue(sound, out AVAudioPlayer? player))
{
string path = Path.Combine(NSBundle.MainBundle.ResourcePath ?? "", "Sounds", FileName(sound) + ".wav");
if (!File.Exists(path)) return;
player = AVAudioPlayer.FromUrl(NSUrl.FromFilename(path));
if (player is null) return;
player.PrepareToPlay(); players[sound] = player;
}
player.Volume = settings.EventVolume; player.CurrentTime = 0; player.Play();
}
internal void Speak(string text)
{
if (!settings.SpokenEvents || string.IsNullOrWhiteSpace(text)) return;
speech.SpeakUtterance(new AVSpeechUtterance(text.Trim()));
}
private static string FileName(SoundEvent value) => value switch
{
SoundEvent.ChannelJoin => "channel_join", SoundEvent.ChannelLeave => "channel_leave",
SoundEvent.ChannelReceived => "channel_recv", SoundEvent.ChannelSent => "channel_sent",
SoundEvent.PrivateReceived => "pm_recv", SoundEvent.PrivateSent => "pm_sent",
SoundEvent.Login => "login", SoundEvent.Logout => "logout", SoundEvent.ConnectionLost => "connection_lost",
SoundEvent.VoiceOn => "voice_on", SoundEvent.VoiceOff => "voice_off", SoundEvent.VoiceStart => "va_start",
SoundEvent.VoiceStop => "va_stop", _ => "ptt"
};
public void Dispose() { foreach (AVAudioPlayer player in players.Values) player.Dispose(); speech.Dispose(); }
}
+22
View File
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>CFBundleDisplayName</key><string>VoiceCat</string>
<key>CFBundleIdentifier</key><string>me.iamtalon.voicecat</string>
<key>XSAppIconAssets</key><string>Assets.xcassets/AppIcon.appiconset</string>
<key>LSRequiresIPhoneOS</key><true/>
<key>NSMicrophoneUsageDescription</key><string>VoiceCat needs microphone access to transmit your voice in channels.</string>
<key>UIBackgroundModes</key><array><string>audio</string><string>screen-capture</string></array>
<key>UIRequiresFullScreen</key><false/>
<key>UIDeviceFamily</key><array><integer>1</integer><integer>2</integer></array>
<key>UILaunchScreen</key><dict/>
<key>UISupportedInterfaceOrientations</key><array><string>UIInterfaceOrientationPortrait</string><string>UIInterfaceOrientationLandscapeLeft</string><string>UIInterfaceOrientationLandscapeRight</string></array>
<key>UISupportedInterfaceOrientations~ipad</key><array><string>UIInterfaceOrientationPortrait</string><string>UIInterfaceOrientationPortraitUpsideDown</string><string>UIInterfaceOrientationLandscapeLeft</string><string>UIInterfaceOrientationLandscapeRight</string></array>
<key>UIApplicationSceneManifest</key><dict>
<key>UIApplicationSupportsMultipleScenes</key><false/>
<key>UISceneConfigurations</key><dict><key>UIWindowSceneSessionRoleApplication</key><array><dict>
<key>UISceneConfigurationName</key><string>Default Configuration</string>
<key>UISceneDelegateClassName</key><string>VoiceCat.iOS.SceneDelegate</string>
</dict></array></dict>
</dict>
</dict></plist>
@@ -0,0 +1,254 @@
using System.Runtime.InteropServices;
using AVFoundation;
using UIKit;
using VoiceCat.Audio;
using VoiceCat.Core;
namespace VoiceCat.iOS;
internal sealed class IosAudioEngine
{
// InstallTapOnBus bufferSize is only a request. Physical iOS hardware can deliver 4,800
// frames per callback (observed with the voice-processing graph), so conversion storage
// must cover substantially more than the requested 960 frames without allocating in Capture.
private const int MaximumCaptureCallbackFrames = 16_384;
internal static IosAudioEngine Shared { get; } = new();
private readonly AdaptivePcmBuffer playbackRing = new(2, capacityFrames: 65_536);
private readonly short[] renderScratch = new short[16_384];
private AVAudioEngine? engine;
private Foundation.NSObject? engineConfigurationObserver;
private AVAudioSourceNode? source;
private AVAudioFormat? outputFormat;
private VoiceCatClient? client;
private sealed class MicrophoneRoute(uint streamId, int channels)
{
internal readonly uint StreamId = streamId;
internal readonly int Channels = channels;
// Capture hardware and the managed 20 ms sender have independent clocks. Correct
// their small rate difference before the queue eventually reaches its hard edge.
// RemoteIO can deliver capture in 100 ms bursts, so retain one burst of headroom.
internal readonly AdaptivePcmBuffer Ring = new(channels, 120, 65_536);
}
private MicrophoneRoute? microphone;
private readonly short[] microphoneFrame = new short[960 * 2];
private int microphoneCredit;
private AVAudioFormat? microphoneFormat;
private AVAudioConverter? microphoneConverter;
private AVAudioPcmBuffer? convertedMicrophone;
private AVAudioPcmBuffer? pendingInput;
private AVAudioConverterInputHandler? inputProvider;
private bool inputProvided;
private bool tapInstalled;
private long captureCallbacks, capturedFrames, convertedFrames, rejectedFeeds, converterFailures, stereoFrames, stereoDifferentFrames;
internal bool IsConnected { get; private set; }
internal int BufferMilliseconds { get => playbackRing.BufferMilliseconds; set => playbackRing.BufferMilliseconds = value; }
internal void StartListening(VoiceCatClient owner)
{
Stop(); client = owner; IsConnected = true; owner.Audio.MixedPcm += ReceiveMixedPcm; Rebuild();
}
internal void StartMicrophone(uint streamId, int channels)
{
Volatile.Write(ref microphone, CreateMicrophoneRoute(streamId, channels)); Rebuild();
}
internal void StopMicrophone() { Volatile.Write(ref microphone, null); Rebuild(); }
internal void Reconfigure()
{
if (!IsConnected) return;
MicrophoneRoute? current = Volatile.Read(ref microphone);
int channels = IosAudioRouter.Shared.CaptureChannels;
if (current is not null)
{
// Stop the old tap before publishing a route with a different sample width. Otherwise
// an in-flight callback could interpret its old converter buffer using the new width.
DestroyGraph();
if (current.Channels != channels) client!.Audio.SetCaptureChannels(current.StreamId, channels);
Volatile.Write(ref microphone, CreateMicrophoneRoute(current.StreamId, channels));
}
Rebuild();
}
private static MicrophoneRoute CreateMicrophoneRoute(uint streamId, int channels)
{
return new MicrophoneRoute(streamId, Math.Clamp(channels, 1, 2));
}
internal bool EnsureRunning()
{
if (!IsConnected || engine?.Running == true) return true;
Rebuild(); return engine?.Running == true;
}
private void Rebuild()
{
DestroyGraph(); MicrophoneRoute? route = Volatile.Read(ref microphone); IosAudioRouter.Shared.Apply(route is not null);
var next = new AVAudioEngine();
AVAudioInputNode? input = null;
if (route is not null)
{
// Enabling voice processing rebuilds both sides of AVAudioEngine. Do it before any
// formats are queried or nodes are connected so the graph is built from the final IO.
input = next.InputNode;
if (!input.SetVoiceProcessingEnabled(IosAudioRouter.Shared.UsesVoiceProcessing, out NSError? processingError))
throw new InvalidOperationException(processingError.LocalizedDescription);
if (IosAudioRouter.Shared.UsesVoiceProcessing) input.VoiceProcessingAgcEnabled = IosAudioRouter.Shared.AutomaticGainControl;
}
outputFormat = new(AVAudioCommonFormat.PCMFloat32, 48_000, 2, false);
source = new(outputFormat, Render);
next.AttachNode(source);
NSError? connectionError = null;
if (OperatingSystem.IsIOSVersionAtLeast(27)) next.Connect(source, next.MainMixerNode, outputFormat, out connectionError);
else next.Connect(source, next.MainMixerNode, outputFormat);
if (connectionError is not null) throw new InvalidOperationException(connectionError.LocalizedDescription);
if (route is not null)
{
AVAudioFormat inputFormat = input!.GetBusOutputFormat(0);
Console.Error.WriteLine($"VC_GRAPH vpio={IosAudioRouter.Shared.UsesVoiceProcessing} requestedCh={route.Channels} " +
$"inputCh={inputFormat.ChannelCount} inputRate={inputFormat.SampleRate}");
microphoneFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, (uint)route.Channels, true);
microphoneConverter = new(inputFormat, microphoneFormat);
uint capacity = checked((uint)Math.Ceiling(MaximumCaptureCallbackFrames * 48_000 / inputFormat.SampleRate) + 64);
convertedMicrophone = new(microphoneFormat, capacity);
inputProvider = ProvideInput;
NSError? tapError = null;
if (OperatingSystem.IsIOSVersionAtLeast(27)) input.InstallTapOnBus(0, 960, inputFormat, out tapError, Capture);
else input.InstallTapOnBus(0, 960, inputFormat, Capture);
if (tapError is not null) throw new InvalidOperationException(tapError.LocalizedDescription);
tapInstalled = true;
}
next.Prepare();
if (!next.StartAndReturnError(out NSError? error)) { next.Dispose(); throw new InvalidOperationException(error.LocalizedDescription); }
engine = next;
engineConfigurationObserver = Foundation.NSNotificationCenter.DefaultCenter.AddObserver(
AVAudioEngine.ConfigurationChangeNotification, notification =>
{
if (!ReferenceEquals(notification.Object, next)) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
if (IsConnected && ReferenceEquals(engine, next) && !next.Running) Rebuild();
});
}, next);
}
private unsafe void Capture(AVAudioPcmBuffer buffer, AVAudioTime time)
{
Interlocked.Increment(ref captureCallbacks);
Interlocked.Add(ref capturedFrames, buffer.FrameLength);
VoiceCatClient? owner = client; MicrophoneRoute? route = Volatile.Read(ref microphone);
if (owner is null || route is null || buffer.FrameLength == 0) return;
AVAudioConverter? converter = microphoneConverter;
AVAudioPcmBuffer? converted = convertedMicrophone;
AVAudioConverterInputHandler? provider = inputProvider;
if (converter is null || converted is null || provider is null) return;
pendingInput = buffer; inputProvided = false; converted.FrameLength = 0;
converter.ConvertToBuffer(converted, out NSError? conversionError, provider);
if (conversionError is not null) Interlocked.Increment(ref converterFailures);
if (converted.FrameLength == 0) return;
Interlocked.Add(ref convertedFrames, converted.FrameLength);
nint samples = Marshal.ReadIntPtr(converted.Int16ChannelData);
if (samples != 0 && ReferenceEquals(route, Volatile.Read(ref microphone)))
{
var pcm = new ReadOnlySpan<short>((void*)samples, checked((int)converted.FrameLength * route.Channels));
if (route.Channels == 2)
{
Interlocked.Add(ref stereoFrames, converted.FrameLength);
long different = 0;
for (int frame = 0; frame < converted.FrameLength; frame++)
if (pcm[frame * 2] != pcm[frame * 2 + 1]) different++;
Interlocked.Add(ref stereoDifferentFrames, different);
}
if (!route.Ring.TryWrite(pcm)) Interlocked.Increment(ref rejectedFeeds);
}
pendingInput = null;
}
internal string CaptureDiagnostics()
{
return $"callbacks={Interlocked.Exchange(ref captureCallbacks, 0)} input={Interlocked.Exchange(ref capturedFrames, 0)} " +
$"converted={Interlocked.Exchange(ref convertedFrames, 0)} feedDrops={Interlocked.Exchange(ref rejectedFeeds, 0)} " +
$"converterErrors={Interlocked.Exchange(ref converterFailures, 0)} stereo={Interlocked.Exchange(ref stereoDifferentFrames, 0)}/" +
$"{Interlocked.Exchange(ref stereoFrames, 0)}";
}
private AVAudioBuffer ProvideInput(uint _, out AVAudioConverterInputStatus status)
{
if (!inputProvided && pendingInput is { } input) { inputProvided = true; status = AVAudioConverterInputStatus.HaveData; return input; }
status = AVAudioConverterInputStatus.NoDataNow; return null!;
}
// One paced 20 ms handoff from the capture ring into the encoder. The render callback calls
// this at its demand rate: the ring absorbs RemoteIO's burst pattern and the small capture vs
// output clock difference, so the sender sees a steady 20 ms feed without a sleep-paced pacer
// thread to stall when iOS coalesces a backgrounded app's wakeups.
private void PumpMicrophoneChunk(VoiceCatClient owner)
{
MicrophoneRoute? route = Volatile.Read(ref microphone);
if (route is null) return;
int required = 960 * route.Channels;
if (route.Ring.Read(microphoneFrame.AsSpan(0, required)) == required &&
ReferenceEquals(route, Volatile.Read(ref microphone)) &&
!owner.Audio.FeedPcm(route.StreamId, microphoneFrame.AsSpan(0, required), route.Channels))
Interlocked.Increment(ref rejectedFeeds);
}
private void ReceiveMixedPcm(ReadOnlySpan<short> pcm) => playbackRing.TryWrite(pcm);
private unsafe int Render(IntPtr isSilence, IntPtr timestamp, uint frameCount, IntPtr outputData)
{
int frames = checked((int)frameCount), requested = checked(frames * 2);
if (requested > renderScratch.Length) return -1;
// This callback is the cadence iOS keeps exact while the app is backgrounded or the device
// is locked, so it owns both managed 20 ms hands-offs: capture into the sender and one mix
// cycle per 20 ms of render demand. Both run allocation-free and without locks or I/O.
VoiceCatClient? owner = Volatile.Read(ref client);
if (owner is null || !IsConnected) microphoneCredit = 0;
else
{
microphoneCredit += frames;
while (microphoneCredit >= 960) { microphoneCredit -= 960; PumpMicrophoneChunk(owner); }
// Top the mix ring up to cover this callback plus its configured target so the read
// below never starves and the buffer keeps its chosen buffering latency.
int deficit = frames + playbackRing.TargetFrames - playbackRing.CountFrames;
for (int produced = 0; produced < deficit; produced += 960) owner.Audio.RunCycle();
}
Span<short> input = renderScratch.AsSpan(0, requested);
int read = playbackRing.Read(input); input[read..].Clear();
int count = Marshal.ReadInt32(outputData), first = IntPtr.Size == 8 ? 8 : 4, stride = IntPtr.Size == 8 ? 16 : 12;
if (count != 2) return -1;
for (int channel = 0; channel < 2; channel++)
{
nint data = Marshal.ReadIntPtr(outputData, first + channel * stride + 8);
var output = new Span<float>((void*)data, frames);
for (int frame = 0; frame < frames; frame++) output[frame] = input[frame * 2 + channel] / 32768f;
}
if (isSilence != IntPtr.Zero) Marshal.WriteByte(isSilence, read == 0 ? (byte)1 : (byte)0);
return 0;
}
internal void Stop()
{
IsConnected = false; Volatile.Write(ref microphone, null);
if (client is { } owner) owner.Audio.MixedPcm -= ReceiveMixedPcm;
DestroyGraph(); client = null; IosAudioRouter.Shared.Deactivate();
}
private void DestroyGraph()
{
if (engineConfigurationObserver is { } observer)
{
Foundation.NSNotificationCenter.DefaultCenter.RemoveObserver(observer);
observer.Dispose(); engineConfigurationObserver = null;
}
if (engine is { } old)
{
if (tapInstalled) old.InputNode.RemoveTapOnBus(0);
old.Stop(); if (source is not null) old.DetachNode(source); old.Dispose();
}
tapInstalled = false; pendingInput = null; inputProvider = null;
convertedMicrophone?.Dispose(); convertedMicrophone = null;
microphoneConverter?.Dispose(); microphoneConverter = null;
microphoneFormat?.Dispose(); microphoneFormat = null;
source?.Dispose(); source = null; outputFormat?.Dispose(); outputFormat = null; engine = null;
}
}
@@ -0,0 +1,248 @@
using AVFoundation;
using Foundation;
using ObjCRuntime;
using UIKit;
namespace VoiceCat.iOS;
internal enum IosAudioPreset { VoiceChat, StereoMicrophone, MonoMicrophone, Advanced }
internal enum IosBluetoothMode { HfpVoice, BuiltInMicA2dp, BuiltInMicSpeaker }
internal enum IosMicMode { Standard, Raw }
internal sealed record IosAudioPort(string Id, string Name, string Type);
internal sealed record IosAudioDataSource(string Id, string Name, IReadOnlyList<AVAudioDataSourcePolarPattern> Patterns);
internal sealed class IosAudioRouter
{
private static readonly NativeHandle SupportedPolarPatternsSelector = Selector.GetHandle("supportedPolarPatterns");
private static readonly NativeHandle SetPreferredPolarPatternSelector = Selector.GetHandle("setPreferredPolarPattern:error:");
internal static IosAudioRouter Shared { get; } = new();
private readonly NSUserDefaults defaults = NSUserDefaults.StandardUserDefaults;
private bool applying;
internal event Action? Changed;
internal IosAudioPreset Preset { get; private set; } = IosAudioPreset.VoiceChat;
internal IosBluetoothMode BluetoothMode { get; private set; } = IosBluetoothMode.HfpVoice;
internal IosMicMode MicMode { get; private set; } = IosMicMode.Standard;
internal bool ForceSpeaker { get; private set; }
internal bool VoiceProcessing { get; private set; } = true;
internal bool AutomaticGainControl { get; private set; } = true;
internal int CaptureChannels { get; private set; } = 1;
internal string? SelectedInputId { get; private set; }
internal string? SelectedDataSourceId { get; private set; }
internal AVAudioDataSourcePolarPattern SelectedPolarPattern { get; private set; } = AVAudioDataSourcePolarPattern.Unknown;
internal IReadOnlyList<IosAudioPort> Inputs { get; private set; } = [];
internal IReadOnlyList<IosAudioPort> Outputs { get; private set; } = [];
internal bool VoiceProcessingAvailable => CaptureChannels == 1 && MicMode == IosMicMode.Standard && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp;
internal bool UsesVoiceProcessing => VoiceProcessing && VoiceProcessingAvailable;
private IosAudioRouter()
{
NSNotificationCenter.DefaultCenter.AddObserver(AVAudioSession.RouteChangeNotification, HandleRouteChange);
NSNotificationCenter.DefaultCenter.AddObserver(AVAudioSession.InterruptionNotification, HandleInterruption);
NSNotificationCenter.DefaultCenter.AddObserver(AVAudioSession.MediaServicesWereResetNotification, _ => Recover("media services reset"));
}
internal void Load()
{
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.preset"), true, out IosAudioPreset preset)) Preset = preset;
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.bluetoothMode"), true, out IosBluetoothMode bluetooth)) BluetoothMode = bluetooth;
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.micMode"), true, out IosMicMode mic)) MicMode = mic;
ForceSpeaker = defaults.BoolForKey("cat.voice.audio.forceSpeaker");
VoiceProcessing = defaults.ValueForKey(new NSString("cat.voice.audio.voiceProcessing")) is null || defaults.BoolForKey("cat.voice.audio.voiceProcessing");
AutomaticGainControl = defaults.ValueForKey(new NSString("cat.voice.audio.agc")) is null || defaults.BoolForKey("cat.voice.audio.agc");
CaptureChannels = defaults.IntForKey("cat.voice.audio.captureChannels") == 2 ? 2 : Preset == IosAudioPreset.StereoMicrophone ? 2 : 1;
SelectedInputId = defaults.StringForKey("cat.voice.audio.inputPortId"); SelectedDataSourceId = defaults.StringForKey("cat.voice.audio.dataSourceId");
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.polarPattern"), true, out AVAudioDataSourcePolarPattern pattern)) SelectedPolarPattern = pattern;
RefreshRoutes();
}
internal void SelectPreset(IosAudioPreset preset)
{
Preset = preset;
if (preset != IosAudioPreset.Advanced)
{
CaptureChannels = preset == IosAudioPreset.StereoMicrophone ? 2 : 1; MicMode = IosMicMode.Standard;
BluetoothMode = preset == IosAudioPreset.VoiceChat ? IosBluetoothMode.HfpVoice : IosBluetoothMode.BuiltInMicA2dp;
if (preset is IosAudioPreset.StereoMicrophone or IosAudioPreset.MonoMicrophone)
SelectedInputId = AVAudioSession.SharedInstance().AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic)?.UID;
else SelectedInputId = null;
// Named presets never carry an Advanced capsule selection across transitions.
// Stereo derives its data source below; mono/voice chat must clear a stale stereo one.
SelectedDataSourceId = null; SelectedPolarPattern = AVAudioDataSourcePolarPattern.Unknown;
if (preset == IosAudioPreset.VoiceChat) ForceSpeaker = true;
}
SaveAndReconfigure();
}
internal void SetForceSpeaker(bool value) { ForceSpeaker = value; SaveAndReconfigure(); }
internal void SetVoiceProcessing(bool value) { VoiceProcessing = value; SaveAndReconfigure(); }
internal void SetAutomaticGainControl(bool value) { AutomaticGainControl = value; SaveAndReconfigure(); }
internal void SetCaptureChannels(int value) { CaptureChannels = value == 2 ? 2 : 1; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal void SetBluetoothMode(IosBluetoothMode value) { BluetoothMode = value; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal void SetMicMode(IosMicMode value) { MicMode = value; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal void SelectInput(string? id) { SelectedInputId = string.IsNullOrEmpty(id) ? null : id; SelectedDataSourceId = null; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal void SelectDataSource(string? id) { SelectedDataSourceId = string.IsNullOrEmpty(id) ? null : id; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal void SelectPolarPattern(AVAudioDataSourcePolarPattern pattern) { SelectedPolarPattern = pattern; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
internal IReadOnlyList<IosAudioDataSource> DataSources()
{
AVAudioSessionPortDescription? port = AVAudioSession.SharedInstance().AvailableInputs?.FirstOrDefault(value => value.UID == SelectedInputId);
return port?.DataSources?.Select(value => new IosAudioDataSource(value.DataSourceID.ToString(), value.DataSourceName,
value.SupportedPolarPatterns?.ToArray() ?? [])).ToArray() ?? [];
}
internal void RefreshRoutes()
{
AVAudioSession session = AVAudioSession.SharedInstance();
Inputs = session.AvailableInputs?.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray() ?? [];
Outputs = session.CurrentRoute.Outputs.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray();
SelectedInputId ??= session.PreferredInput?.UID; Changed?.Invoke();
}
internal void Apply(bool configureInput)
{
if (applying) return; applying = true;
try
{
AVAudioSession session = AVAudioSession.SharedInstance(); AVAudioSessionCategoryOptions options = AVAudioSessionCategoryOptions.MixWithOthers;
if (BluetoothMode == IosBluetoothMode.HfpVoice) options |= AVAudioSessionCategoryOptions.AllowBluetooth | AVAudioSessionCategoryOptions.AllowBluetoothA2DP | AVAudioSessionCategoryOptions.AllowAirPlay;
if (BluetoothMode == IosBluetoothMode.BuiltInMicA2dp) options |= AVAudioSessionCategoryOptions.AllowBluetoothA2DP | AVAudioSessionCategoryOptions.AllowAirPlay;
if (BluetoothMode == IosBluetoothMode.BuiltInMicSpeaker || ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp) options |= AVAudioSessionCategoryOptions.DefaultToSpeaker;
AVAudioSessionMode mode = CaptureChannels == 2 ? AVAudioSessionMode.Default : MicMode == IosMicMode.Raw ? AVAudioSessionMode.Measurement
: BluetoothMode == IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionMode.VideoRecording : AVAudioSessionMode.VoiceChat;
if (!session.SetCategory(AVAudioSessionCategory.PlayAndRecord, mode, options, out NSError? categoryError)) throw new InvalidOperationException(categoryError.LocalizedDescription);
session.SetPreferredSampleRate(48_000, out _); session.SetPreferredIOBufferDuration(0.02, out _);
if (configureInput) ApplyInputSelection(session);
if (!session.SetActive(true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? activeError)) throw new InvalidOperationException(activeError.LocalizedDescription);
Console.Error.WriteLine($"VC_ROUTE preset={Preset} requestedCh={CaptureChannels} sessionCh={session.InputNumberOfChannels} " +
$"preferred={session.PreferredInput?.PortName ?? "default"} dataSource={session.InputDataSource?.DataSourceName ?? "default"} " +
$"pattern={session.InputDataSource?.SelectedPolarPattern.ToString() ?? "default"}");
session.OverrideOutputAudioPort(ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _); RefreshRoutes();
}
finally { applying = false; }
}
private void ApplyInputSelection(AVAudioSession session)
{
AVAudioSessionPortDescription? port = session.AvailableInputs?.FirstOrDefault(value => value.UID == SelectedInputId);
if (CaptureChannels == 2) port ??= session.AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic);
if (port is null) { if (CaptureChannels == 1) ClearStereoPolarPattern(session); return; }
if (CaptureChannels == 2)
{
// Polar-pattern discovery alone is insufficient on current iPhones: until a stereo
// orientation is requested, the built-in port can expose only its mono Bottom source
// and AVAudioEngine consequently binds a one-channel input node. This is Apple's
// dedicated switch for built-in stereo recording. Portrait matches this portrait UI;
// it also gives Core Audio an unambiguous left/right mapping before graph creation.
if (!session.SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait, out NSError? orientationError))
throw new InvalidOperationException(orientationError?.LocalizedDescription ?? "Could not set the stereo microphone orientation.");
}
AVAudioSessionDataSourceDescription? source = CaptureChannels == 2
? port.DataSources?.FirstOrDefault(SupportsStereoPolarPattern)
: port.DataSources?.FirstOrDefault(value => value.DataSourceID.ToString() == SelectedDataSourceId);
if (CaptureChannels == 1 && source is null) ClearStereoPolarPattern(session);
if (source is not null)
{
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError.LocalizedDescription);
if (CaptureChannels == 2) SetStereoPolarPattern(source);
else if (SelectedPolarPattern != AVAudioDataSourcePolarPattern.Unknown &&
!source.SetPreferredPolarPattern(SelectedPolarPattern, out NSError? patternError))
throw new InvalidOperationException(patternError.LocalizedDescription);
}
if (!session.SetPreferredInput(port, out NSError? inputError)) throw new InvalidOperationException(inputError.LocalizedDescription);
// The working Swift client deliberately does not call
// SetPreferredInputNumberOfChannels: doing so disrupts stereo + A2DP routing. The stereo
// capsule and polar pattern above cause the input node to expose its two-channel format.
if (CaptureChannels == 2 && source is not null &&
!session.SetInputDataSource(source, out NSError? sourceError))
throw new InvalidOperationException(sourceError.LocalizedDescription);
Console.Error.WriteLine($"VC_ROUTE_SELECT port={port.PortName} source={source?.DataSourceName ?? "none"} " +
$"stereoPattern={source is not null && SupportsStereoPolarPattern(source)} " +
$"patterns={string.Join(',', source?.SupportedPolarPatterns?.Select(value => value.ToString()) ?? [])}");
}
// Microsoft.iOS 27 exposes AVAudioSessionPolarPatternStereo as an NSString constant but its
// AVAudioDataSourcePolarPattern smart enum still has no Stereo member. Reading the native
// NSArray and setting the native NSString avoids silently mapping Stereo to Unknown.
private static bool SupportsStereoPolarPattern(AVAudioSessionDataSourceDescription source)
{
NativeHandle handle = NativeMethods.GetObject(source.Handle, SupportedPolarPatternsSelector);
NSArray? patterns = Runtime.GetNSObject<NSArray>(handle);
if (patterns is null) return false;
for (nuint index = 0; index < patterns.Count; index++)
if (patterns.GetItem<NSString>(index)?.IsEqualTo(AVAudioSession.PolarPatternStereo.Handle) == true) return true;
return false;
}
private static void SetStereoPolarPattern(AVAudioSessionDataSourceDescription source)
{
NativeHandle error = NativeHandle.Zero;
if (NativeMethods.SetObject(source.Handle, SetPreferredPolarPatternSelector,
AVAudioSession.PolarPatternStereo.Handle, ref error) == 0)
throw new InvalidOperationException(Runtime.GetNSObject<NSError>(error)?.LocalizedDescription ?? "Could not enable stereo microphone capture.");
}
private static class NativeMethods
{
[System.Runtime.InteropServices.DllImport("/usr/lib/libobjc.dylib", EntryPoint = "objc_msgSend")]
internal static extern NativeHandle GetObject(NativeHandle receiver, NativeHandle selector);
[System.Runtime.InteropServices.DllImport("/usr/lib/libobjc.dylib", EntryPoint = "objc_msgSend")]
internal static extern byte SetObject(NativeHandle receiver, NativeHandle selector, NativeHandle value, ref NativeHandle error);
}
private static void ClearStereoPolarPattern(AVAudioSession session)
{
session.SetPreferredInputOrientation(AVAudioStereoOrientation.None, out _);
AVAudioSessionPortDescription? builtIn = session.AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic);
if (builtIn is null) return;
foreach (AVAudioSessionDataSourceDescription source in builtIn.DataSources ?? [])
if (source.SelectedPolarPattern == AVAudioDataSourcePolarPattern.Unknown && SupportsStereoPolarPattern(source))
source.SetPreferredPolarPattern(AVAudioDataSourcePolarPattern.Unknown, out _);
}
private void SaveAndReconfigure()
{
defaults.SetString(Preset.ToString(), "cat.voice.audio.preset"); defaults.SetString(BluetoothMode.ToString(), "cat.voice.audio.bluetoothMode");
defaults.SetString(MicMode.ToString(), "cat.voice.audio.micMode"); defaults.SetBool(ForceSpeaker, "cat.voice.audio.forceSpeaker");
defaults.SetBool(VoiceProcessing, "cat.voice.audio.voiceProcessing"); defaults.SetBool(AutomaticGainControl, "cat.voice.audio.agc"); defaults.SetInt(CaptureChannels, "cat.voice.audio.captureChannels");
Set("cat.voice.audio.inputPortId", SelectedInputId); Set("cat.voice.audio.dataSourceId", SelectedDataSourceId); defaults.SetString(SelectedPolarPattern.ToString(), "cat.voice.audio.polarPattern"); defaults.Synchronize();
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(); Changed?.Invoke();
}
private void Set(string key, string? value) { if (value is null) defaults.RemoveObject(key); else defaults.SetString(value, key); }
internal void Deactivate() => AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out _);
internal void EnsureAudio(string reason)
{
if (!IosAudioEngine.Shared.IsConnected) return;
try { IosAudioEngine.Shared.EnsureRunning(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
}
internal void Recover(string reason)
{
RefreshRoutes();
if (!IosAudioEngine.Shared.IsConnected) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
try { IosAudioEngine.Shared.Reconfigure(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
});
}
private void HandleRouteChange(NSNotification note)
{
NSNumber? value = note.UserInfo?[new NSString("AVAudioSessionRouteChangeReasonKey")] as NSNumber;
AVAudioSessionRouteChangeReason reason = (AVAudioSessionRouteChangeReason)(value?.UInt32Value ?? 0);
RefreshRoutes();
if (reason is AVAudioSessionRouteChangeReason.CategoryChange
or AVAudioSessionRouteChangeReason.Override
or AVAudioSessionRouteChangeReason.RouteConfigurationChange)
return;
Recover($"route change ({reason})");
}
private void HandleInterruption(NSNotification note)
{
NSNumber? type = note.UserInfo?[new NSString("AVAudioSessionInterruptionTypeKey")] as NSNumber;
AVAudioSessionInterruptionType interruption = (AVAudioSessionInterruptionType)(type?.UInt32Value ?? 0);
if (interruption == AVAudioSessionInterruptionType.Ended) Recover("interruption ended");
}
}
@@ -0,0 +1,8 @@
namespace VoiceCat.iOS;
internal static class IosConstants
{
internal const string AppGroup = "group.me.iamtalon.voicecat";
internal const string BroadcastExtension = "me.iamtalon.voicecat.broadcast";
internal const string PasswordService = "me.iamtalon.voicecat.ios";
}
@@ -0,0 +1,25 @@
using System.Runtime.InteropServices;
using Foundation;
namespace VoiceCat.iOS;
internal static partial class IosScreenCapture
{
internal static void Present()
{
if (!OperatingSystem.IsIOSVersionAtLeast(27)) return;
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
if (root?.Path is null) throw new InvalidOperationException("The VoiceCat App Group is unavailable.");
string path = Path.Combine(root.Path, "voicecat", "broadcast_audio.ring"); Directory.CreateDirectory(Path.GetDirectoryName(path)!);
if (!IsAvailable) throw new InvalidOperationException("Screen audio sharing is unavailable on this device."); PresentNative(path);
}
internal static bool IsAvailable => OperatingSystem.IsIOSVersionAtLeast(27) && Available() != 0;
[LibraryImport("__Internal", EntryPoint = "vc_ios_screen_capture_available")]
private static partial int Available();
[LibraryImport("__Internal", EntryPoint = "vc_ios_screen_capture_present", StringMarshalling = StringMarshalling.Utf8)]
private static partial void PresentNative(string ringPath);
[LibraryImport("__Internal", EntryPoint = "vc_ios_screen_capture_stop")]
internal static partial void Stop();
}
+52
View File
@@ -0,0 +1,52 @@
using Foundation;
using VoiceCat.Audio;
namespace VoiceCat.iOS;
internal sealed class IosSettings
{
private readonly NSUserDefaults values = NSUserDefaults.StandardUserDefaults;
internal AudioInputMode InputMode { get; set; } = AudioInputMode.VoiceActivation;
internal float VadThreshold { get; set; } = 0.025f;
internal float InputGain { get; set; } = 1f;
internal float OutputGain { get; set; } = 1f;
internal bool InputNoiseReduction { get; set; }
internal bool EventSounds { get; set; } = true;
internal bool SpokenEvents { get; set; }
internal float EventVolume { get; set; } = 1f;
internal bool SelfTalkSounds { get; set; }
internal bool PushToTalkSound { get; set; }
internal int AudioBufferMilliseconds { get; set; } = 40;
internal void Load()
{
NSString[] keys = [(NSString)"voice.inputMode", (NSString)"voice.vadThreshold", (NSString)"voice.inputGain",
(NSString)"voice.outputGain", (NSString)"voice.audioBufferMs", (NSString)"feedback.sounds", (NSString)"feedback.volume"];
NSObject[] defaults = [NSNumber.FromInt32((int)AudioInputMode.VoiceActivation), NSNumber.FromFloat(0.025f),
NSNumber.FromFloat(1f), NSNumber.FromFloat(1f), NSNumber.FromInt32(40), NSNumber.FromBoolean(true), NSNumber.FromFloat(1f)];
values.RegisterDefaults(new NSDictionary<NSString, NSObject>(keys, defaults));
int inputMode = checked((int)values.IntForKey("voice.inputMode"));
InputMode = Enum.IsDefined(typeof(AudioInputMode), inputMode) ? (AudioInputMode)inputMode : AudioInputMode.VoiceActivation;
VadThreshold = Math.Clamp(values.FloatForKey("voice.vadThreshold"), 0.001f, 0.1f);
InputGain = Math.Clamp(values.FloatForKey("voice.inputGain"), 0f, 4f);
OutputGain = Math.Clamp(values.FloatForKey("voice.outputGain"), 0f, 1f);
InputNoiseReduction = values.BoolForKey("voice.inputNoiseReduction");
int audioBuffer = checked((int)values.IntForKey("voice.audioBufferMs")); AudioBufferMilliseconds = audioBuffer is 20 or 40 or 60 ? audioBuffer : 40;
EventSounds = values.BoolForKey("feedback.sounds");
SpokenEvents = values.BoolForKey("feedback.speech");
EventVolume = Math.Clamp(values.FloatForKey("feedback.volume"), 0f, 1f);
SelfTalkSounds = values.BoolForKey("feedback.selfTalk");
PushToTalkSound = values.BoolForKey("feedback.ptt");
}
internal void Save()
{
values.SetInt((int)InputMode, "voice.inputMode"); values.SetFloat(VadThreshold, "voice.vadThreshold");
values.SetFloat(InputGain, "voice.inputGain"); values.SetFloat(OutputGain, "voice.outputGain");
values.SetInt(AudioBufferMilliseconds, "voice.audioBufferMs");
values.SetBool(InputNoiseReduction, "voice.inputNoiseReduction"); values.SetBool(EventSounds, "feedback.sounds");
values.SetBool(SpokenEvents, "feedback.speech"); values.SetFloat(EventVolume, "feedback.volume");
values.SetBool(SelfTalkSounds, "feedback.selfTalk"); values.SetBool(PushToTalkSound, "feedback.ptt"); values.Synchronize();
}
}
+90
View File
@@ -0,0 +1,90 @@
using System.Security.Cryptography;
using System.Text;
using Foundation;
using Security;
using VoiceCat.Core;
namespace VoiceCat.iOS;
internal sealed class IosStorage
{
private readonly string directory;
private readonly ServerProfileStore profiles;
internal IosStorage()
{
NSUrl? group = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
string root = group?.Path ?? Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);
directory = Path.Combine(root, "voicecat");
profiles = new(Path.Combine(directory, "servers.json"));
}
internal string TofuPath => Path.Combine(directory, "tofu_pins.txt");
internal IReadOnlyList<ServerProfile> LoadProfiles()
{
MigrateLegacyFiles();
return profiles.Load();
}
internal void SaveProfiles(IEnumerable<ServerProfile> values) => profiles.Save(values);
private void MigrateLegacyFiles()
{
Directory.CreateDirectory(directory);
string legacy = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "voicecat");
if (Path.GetFullPath(legacy) == Path.GetFullPath(directory) || !Directory.Exists(legacy)) return;
foreach (string name in new[] { "servers.json", "tofu_pins.txt" })
{
string source = Path.Combine(legacy, name), destination = Path.Combine(directory, name);
if (File.Exists(source) && !File.Exists(destination)) File.Copy(source, destination);
}
}
internal string? LoadPassword(ServerProfile profile)
{
string? value = ReadPassword(IosConstants.PasswordService, profile.Id.ToString("D"), IosConstants.AppGroup);
return value ?? (profile.LegacyKeychainTag is { Length: > 0 } tag
? ReadPassword("cat.voice.VoiceCatiOS", tag, IosConstants.AppGroup) ?? ReadPassword("cat.voice.VoiceCatiOS", tag, null)
: null);
}
internal void SavePassword(Guid id, string password)
{
byte[] encoded = Encoding.UTF8.GetBytes(password);
try
{
using var data = NSData.FromArray(encoded);
using var query = PasswordQuery(IosConstants.PasswordService, id.ToString("D"), IosConstants.AppGroup);
using var attributes = new SecRecord { ValueData = data, Label = "VoiceCat server password", Accessible = SecAccessible.AfterFirstUnlock };
SecStatusCode status = SecKeyChain.Update(query, attributes);
if (status == SecStatusCode.ItemNotFound)
{
using var record = PasswordQuery(IosConstants.PasswordService, id.ToString("D"), IosConstants.AppGroup);
record.ValueData = data; record.Label = attributes.Label; record.Accessible = attributes.Accessible;
status = SecKeyChain.Add(record);
}
if (status != SecStatusCode.Success) throw new InvalidOperationException($"Keychain save failed ({status}).");
}
finally { CryptographicOperations.ZeroMemory(encoded); }
}
internal void RemovePassword(Guid id)
{
using var query = PasswordQuery(IosConstants.PasswordService, id.ToString("D"), IosConstants.AppGroup);
SecStatusCode status = SecKeyChain.Remove(query);
if (status is not (SecStatusCode.Success or SecStatusCode.ItemNotFound))
throw new InvalidOperationException($"Keychain removal failed ({status}).");
}
private static string? ReadPassword(string service, string account, string? group)
{
using var query = PasswordQuery(service, account, group);
using SecRecord? result = SecKeyChain.QueryAsRecord(query, out SecStatusCode status);
if (status != SecStatusCode.Success || result?.ValueData is not { } data) return null;
byte[] bytes = data.ToArray();
try { return Encoding.UTF8.GetString(bytes); }
finally { CryptographicOperations.ZeroMemory(bytes); }
}
private static SecRecord PasswordQuery(string service, string account, string? group) => new(SecKind.GenericPassword)
{ Service = service, Account = account, AccessGroup = group };
}
+3
View File
@@ -0,0 +1,3 @@
using UIKit;
UIApplication.Main(args, null, typeof(VoiceCat.iOS.AppDelegate));
@@ -0,0 +1,189 @@
using UIKit;
using VoiceCat.Audio;
using Voicecat.V1;
using Channel = Voicecat.V1.Channel;
namespace VoiceCat.iOS;
internal sealed class MainTabController : UITabBarController
{
internal MainTabController(AppModel model)
{
UIViewController channels = Wrap(new ChannelsController(model), "Channels", "list.bullet.indent", 0);
UIViewController chat = Wrap(new ChatController(model), "Chat", "message", 1);
UIViewController settings = Wrap(new SettingsController(model), "Settings", "gear", 2);
ViewControllers = [channels, chat, settings];
}
private static UIViewController Wrap(UIViewController content, string title, string image, nint tag)
{
var root = new UIViewController(); UIView rootView = root.View!; rootView.BackgroundColor = UIColor.SystemBackground;
var navigation = new UINavigationController(content); UIView navigationView = navigation.View!; navigationView.TranslatesAutoresizingMaskIntoConstraints = false;
root.AddChildViewController(navigation); rootView.AddSubview(navigationView); navigation.DidMoveToParentViewController(root);
var voice = new VoiceControlsView(AppModel.Shared) { TranslatesAutoresizingMaskIntoConstraints = false }; rootView.AddSubview(voice);
NSLayoutConstraint.ActivateConstraints([navigationView.TopAnchor.ConstraintEqualTo(rootView.TopAnchor), navigationView.LeadingAnchor.ConstraintEqualTo(rootView.LeadingAnchor),
navigationView.TrailingAnchor.ConstraintEqualTo(rootView.TrailingAnchor), navigationView.BottomAnchor.ConstraintEqualTo(voice.TopAnchor),
voice.LeadingAnchor.ConstraintEqualTo(rootView.LeadingAnchor), voice.TrailingAnchor.ConstraintEqualTo(rootView.TrailingAnchor),
voice.BottomAnchor.ConstraintEqualTo(rootView.SafeAreaLayoutGuide.BottomAnchor), voice.HeightAnchor.ConstraintEqualTo(58)]);
root.TabBarItem = new(title, UIImage.GetSystemImage(image), tag); return root;
}
}
internal sealed class VoiceControlsView : UIView
{
private readonly AppModel model; private readonly UIButton join = UIButton.FromType(UIButtonType.System);
private readonly UIButton ptt = UIButton.FromType(UIButtonType.System); private readonly UIButton mute = UIButton.FromType(UIButtonType.System);
private readonly UIButton deafen = UIButton.FromType(UIButtonType.System); private readonly UIProgressView level = new(UIProgressViewStyle.Default);
internal VoiceControlsView(AppModel model) { this.model = model; model.Changed += Refresh; Build(); }
private void Build()
{
BackgroundColor = UIColor.SecondarySystemBackground; join.TouchUpInside += async (_, _) => await Run(model.ToggleVoiceAsync);
ptt.TouchDown += (_, _) => model.SetPushToTalk(true); ptt.TouchUpInside += (_, _) => model.SetPushToTalk(false);
ptt.TouchUpOutside += (_, _) => model.SetPushToTalk(false); ptt.TouchCancel += (_, _) => model.SetPushToTalk(false);
mute.TouchUpInside += (_, _) => model.SetSelfAudio(!(model.Client?.Audio.MicMuted ?? false), model.Client?.Audio.Deafened ?? false);
deafen.TouchUpInside += (_, _) => model.SetSelfAudio(model.Client?.Audio.MicMuted ?? false, !(model.Client?.Audio.Deafened ?? false));
UIStackView stack = new([join, ptt, level, mute, deafen]) { Axis = UILayoutConstraintAxis.Horizontal, Alignment = UIStackViewAlignment.Center,
Distribution = UIStackViewDistribution.Fill, Spacing = 12, TranslatesAutoresizingMaskIntoConstraints = false };
AddSubview(stack); level.WidthAnchor.ConstraintEqualTo(65).Active = true;
NSLayoutConstraint.ActivateConstraints([stack.LeadingAnchor.ConstraintEqualTo(LeadingAnchor, 12), stack.TrailingAnchor.ConstraintEqualTo(TrailingAnchor, -12), stack.CenterYAnchor.ConstraintEqualTo(CenterYAnchor)]); Refresh();
}
private void Refresh()
{
join.SetTitle(model.VoiceJoined ? "Leave Voice" : "Join Voice", UIControlState.Normal); join.AccessibilityLabel = join.Title(UIControlState.Normal); join.Enabled = model.CurrentChannelId != 0;
ptt.SetTitle("Hold to Talk", UIControlState.Normal); ptt.AccessibilityLabel = "Push to talk, hold to transmit"; ptt.Hidden = model.Settings.InputMode != AudioInputMode.PushToTalk; ptt.Enabled = model.VoiceJoined;
bool muted = model.Client?.Audio.MicMuted == true, deafened = model.Client?.Audio.Deafened == true;
mute.SetImage(UIImage.GetSystemImage(muted ? "mic.slash.fill" : "mic.fill"), UIControlState.Normal); mute.AccessibilityLabel = muted ? "Unmute microphone" : "Mute microphone"; mute.Enabled = model.VoiceJoined;
deafen.SetImage(UIImage.GetSystemImage(deafened ? "headphones.slash" : "headphones"), UIControlState.Normal); deafen.AccessibilityLabel = deafened ? "Undeafen" : "Deafen"; deafen.Enabled = model.VoiceJoined;
level.Progress = Math.Clamp(model.MicrophoneLevel * 10, 0, 1); level.AccessibilityLabel = "Microphone level"; level.AccessibilityValue = $"{level.Progress:P0}";
}
private async Task Run(Func<Task> operation) { try { await operation(); } catch (Exception exception) { if (Window?.RootViewController is { } owner) UiHelpers.ShowError(owner, exception); } }
}
internal sealed class ChannelsController : UITableViewController
{
private readonly AppModel model; private IReadOnlyList<(Channel Channel, int Depth)> Visible => Flatten();
internal ChannelsController(AppModel model) { this.model = model; Title = "Channels"; model.Changed += Reload; }
public override void ViewDidLoad()
{
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel");
NavigationItem.RightBarButtonItems = [new("Users", UIBarButtonItemStyle.Plain, (_, _) => NavigationController?.PushViewController(new UsersController(model), true)),
new(UIBarButtonSystemItem.Add, (_, _) => NavigationController?.PushViewController(new ChannelEditorController(model, null), true))]; Reload();
}
private void Reload() { TableView.ReloadData(); NavigationItem.RightBarButtonItems![1].Enabled = model.Client?.Permissions is { } p && (p.IsAdmin || p.CanCreateTempChannel); }
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
(Channel channel, int depth) = Visible[indexPath.Row]; UITableViewCell cell = tableView.DequeueReusableCell("channel", indexPath);
int count = model.Users.Count(user => user.ChannelId == channel.Id); var content = cell.DefaultContentConfiguration;
content.Text = new string(' ', depth * 3) + channel.Name; content.SecondaryText = string.IsNullOrEmpty(channel.Topic) ? $"{count} users" : $"{channel.Topic} • {count} users";
content.Image = UIImage.GetSystemImage(channel.Id == model.CurrentChannelId ? "checkmark.circle.fill" : channel.PasswordProtected ? "lock.fill" : "bubble.left"); cell.ContentConfiguration = content;
cell.AccessibilityLabel = $"{channel.Name}{(channel.Id == model.CurrentChannelId ? ", current" : "")}{(channel.PasswordProtected ? ", password protected" : "")}, {count} users"; return cell;
}
public override async void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
Channel channel = Visible[indexPath.Row].Channel; tableView.DeselectRow(indexPath, true);
if (channel.PasswordProtected) { PromptPassword(channel); return; }
try { await model.JoinChannelAsync(channel.Id); } catch (Exception exception) { UiHelpers.ShowError(this, exception); }
}
public override UISwipeActionsConfiguration? GetTrailingSwipeActionsConfiguration(UITableView tableView, NSIndexPath indexPath)
{
if (model.Client?.Permissions.IsAdmin != true) return null; Channel channel = Visible[indexPath.Row].Channel;
UIContextualAction edit = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Normal, "Edit", (_, _, done) => { NavigationController?.PushViewController(new ChannelEditorController(model, channel), true); done(true); });
UIContextualAction delete = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Destructive, "Delete", (_, _, done) => { ConfirmDelete(channel); done(true); });
delete.BackgroundColor = UIColor.SystemRed; return UISwipeActionsConfiguration.FromActions(channel.Id == 1 ? [edit] : [delete, edit]);
}
private void PromptPassword(Channel channel)
{
UIAlertController prompt = UIAlertController.Create("Channel Password", channel.Name, UIAlertControllerStyle.Alert); prompt.AddTextField(field => { field.SecureTextEntry = true; field.AccessibilityLabel = "Channel password"; });
prompt.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); prompt.AddAction(UIAlertAction.Create("Join", UIAlertActionStyle.Default, async _ => { try { await model.JoinChannelAsync(channel.Id, prompt.TextFields?[0].Text ?? ""); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } })); PresentViewController(prompt, true, null);
}
private void ConfirmDelete(Channel channel)
{
UIAlertController alert = UIAlertController.Create("Delete channel?", channel.Name, UIAlertControllerStyle.Alert); alert.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null));
alert.AddAction(UIAlertAction.Create("Delete", UIAlertActionStyle.Destructive, async _ => { try { GenericResult result = await model.RunAdminAsync(client => client.DeleteChannelAsync(channel.Id)); if (!result.Ok) throw new InvalidOperationException(result.Message); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } })); PresentViewController(alert, true, null);
}
private IReadOnlyList<(Channel, int)> Flatten()
{
var result = new List<(Channel, int)>(); void Add(uint parent, int depth) { foreach (Channel value in model.Channels.Where(channel => channel.ParentId == parent).OrderBy(channel => channel.Order).ThenBy(channel => channel.Name)) { result.Add((value, depth)); Add(value.Id, depth + 1); } }
Add(0, 0); return result;
}
}
internal sealed class ChatController : UIViewController
{
private readonly AppModel model; private readonly UITextView log = new(); private readonly UITextField compose = UiHelpers.Field("Message");
internal ChatController(AppModel model) { this.model = model; Title = "Chat"; model.Changed += Refresh; }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; log.Editable = false; log.Font = UIFont.PreferredBody; log.AccessibilityLabel = "Chat and activity timeline"; log.TranslatesAutoresizingMaskIntoConstraints = false;
NavigationItem.RightBarButtonItem = new("Private Chats", UIBarButtonItemStyle.Plain, (_, _) => NavigationController?.PushViewController(new PrivateChatsController(model), true));
UIButton send = UIButton.FromType(UIButtonType.System); send.SetTitle("Send", UIControlState.Normal); send.AccessibilityLabel = "Send message"; send.TranslatesAutoresizingMaskIntoConstraints = false; send.TouchUpInside += (_, _) => { model.SendText(compose.Text ?? ""); compose.Text = ""; };
compose.TranslatesAutoresizingMaskIntoConstraints = false; View.AddSubviews(log, compose, send);
NSLayoutConstraint.ActivateConstraints([log.TopAnchor.ConstraintEqualTo(View.SafeAreaLayoutGuide.TopAnchor), log.LeadingAnchor.ConstraintEqualTo(View.LeadingAnchor, 12), log.TrailingAnchor.ConstraintEqualTo(View.TrailingAnchor, -12), compose.TopAnchor.ConstraintEqualTo(log.BottomAnchor, 8), compose.LeadingAnchor.ConstraintEqualTo(View.LeadingAnchor, 12), compose.BottomAnchor.ConstraintEqualTo(View.KeyboardLayoutGuide.TopAnchor, -8), send.LeadingAnchor.ConstraintEqualTo(compose.TrailingAnchor, 8), send.TrailingAnchor.ConstraintEqualTo(View.TrailingAnchor, -12), send.CenterYAnchor.ConstraintEqualTo(compose.CenterYAnchor), compose.WidthAnchor.ConstraintGreaterThanOrEqualTo(120)]); Refresh();
}
private void Refresh()
{
IEnumerable<(DateTime Time, string Text)> chat = model.Messages.Where(message => !message.Private).Select(message => (message.Timestamp, $"{message.Sender}: {message.Text}"));
IEnumerable<(DateTime Time, string Text)> activity = model.Activity.Select(value => (value.Timestamp, $"• {value.Text}"));
log.Text = string.Join("\n", chat.Concat(activity).OrderBy(value => value.Time).Select(value => $"[{value.Time:t}] {value.Text}")); if (log.Text.Length > 0) log.ScrollRangeToVisible(new(log.Text.Length - 1, 1));
}
}
internal sealed class PrivateChatsController : UITableViewController
{
private readonly AppModel model;
private IReadOnlyList<(uint Id, string Name, ChatEntry? Latest, bool Online)> Peers
{
get
{
var online = model.Users.Where(user => user.Id != model.SelfUserId).ToDictionary(user => user.Id);
var history = model.Messages.Where(message => message.Private).GroupBy(message => message.PeerUserId).ToDictionary(group => group.Key, group => group.OrderByDescending(message => message.Timestamp).First());
return online.Keys.Concat(history.Keys).Distinct().Select(id =>
{
online.TryGetValue(id, out User? user); history.TryGetValue(id, out ChatEntry? latest);
string name = user?.Nickname ?? latest?.Peer ?? $"User {id}"; return (Id: id, Name: name, Latest: latest, Online: user is not null);
}).OrderByDescending(peer => peer.Latest?.Timestamp ?? DateTime.MinValue).ThenBy(peer => peer.Name).ToArray();
}
}
internal PrivateChatsController(AppModel model) { this.model = model; Title = "Private Chats"; model.Changed += Reload; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "private-peer"); Reload(); }
public override nint RowsInSection(UITableView tableView, nint section) => Peers.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
(uint _, string name, ChatEntry? latest, bool online) = Peers[indexPath.Row]; UITableViewCell cell = tableView.DequeueReusableCell("private-peer", indexPath); var content = cell.DefaultContentConfiguration;
content.Text = name; content.SecondaryText = latest?.Text ?? "Start a conversation"; content.Image = UIImage.GetSystemImage(online ? "person.crop.circle.fill" : "person.crop.circle.badge.xmark"); cell.ContentConfiguration = content;
cell.Accessory = UITableViewCellAccessory.DisclosureIndicator; cell.AccessibilityLabel = $"{name}, {(online ? "online" : "offline")}, {content.SecondaryText}"; return cell;
}
public override void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
(uint id, string name, _, _) = Peers[indexPath.Row]; tableView.DeselectRow(indexPath, true); NavigationController?.PushViewController(new PrivateConversationController(model, id, name), true);
}
private void Reload() { if (IsViewLoaded) { TableView.ReloadData(); TableView.BackgroundView = Peers.Count == 0 ? new UILabel { Text = "No other users or private conversations", TextAlignment = UITextAlignment.Center, AccessibilityLabel = "No other users or private conversations" } : null; } }
}
internal sealed class PrivateConversationController : UIViewController
{
private readonly AppModel model; private readonly uint peerUserId; private readonly string fallbackName;
private readonly UITextView transcript = new(); private readonly UITextField compose = UiHelpers.Field("Private message"); private readonly UIButton send = UIButton.FromType(UIButtonType.System);
private User? Peer => model.Users.FirstOrDefault(user => user.Id == peerUserId);
internal PrivateConversationController(AppModel model, uint peerUserId, string fallbackName)
{ this.model = model; this.peerUserId = peerUserId; this.fallbackName = fallbackName; Title = fallbackName; model.Changed += Refresh; }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; transcript.Editable = false; transcript.Font = UIFont.PreferredBody; transcript.AccessibilityLabel = $"Private conversation with {fallbackName}"; transcript.TranslatesAutoresizingMaskIntoConstraints = false;
compose.AccessibilityLabel = $"Message to {fallbackName}"; compose.TranslatesAutoresizingMaskIntoConstraints = false;
send.SetTitle("Send", UIControlState.Normal); send.AccessibilityLabel = $"Send message to {fallbackName}"; send.TranslatesAutoresizingMaskIntoConstraints = false; send.TouchUpInside += (_, _) => Send();
NavigationItem.RightBarButtonItem = new("User", UIBarButtonItemStyle.Plain, (_, _) => NavigationController?.PushViewController(new UserDetailController(model, peerUserId), true)) { AccessibilityLabel = $"Audio and user settings for {fallbackName}" };
View.AddSubviews(transcript, compose, send);
NSLayoutConstraint.ActivateConstraints([transcript.TopAnchor.ConstraintEqualTo(View.SafeAreaLayoutGuide.TopAnchor), transcript.LeadingAnchor.ConstraintEqualTo(View.LeadingAnchor, 12), transcript.TrailingAnchor.ConstraintEqualTo(View.TrailingAnchor, -12), compose.TopAnchor.ConstraintEqualTo(transcript.BottomAnchor, 8), compose.LeadingAnchor.ConstraintEqualTo(View.LeadingAnchor, 12), compose.BottomAnchor.ConstraintEqualTo(View.KeyboardLayoutGuide.TopAnchor, -8), send.LeadingAnchor.ConstraintEqualTo(compose.TrailingAnchor, 8), send.TrailingAnchor.ConstraintEqualTo(View.TrailingAnchor, -12), send.CenterYAnchor.ConstraintEqualTo(compose.CenterYAnchor), compose.WidthAnchor.ConstraintGreaterThanOrEqualTo(120)]); Refresh();
}
private void Send() { string text = compose.Text ?? ""; if (string.IsNullOrWhiteSpace(text) || Peer is null) return; model.SendText(text, peerUserId); compose.Text = ""; }
private void Refresh()
{
if (!IsViewLoaded) return; string name = Peer?.Nickname ?? fallbackName; Title = name;
IEnumerable<ChatEntry> messages = model.Messages.Where(message => message.Private && message.PeerUserId == peerUserId).OrderBy(message => message.Timestamp);
transcript.Text = string.Join("\n", messages.Select(message => $"[{message.Timestamp:t}] {(message.SenderId == model.SelfUserId ? "You" : message.Sender)}: {message.Text}"));
if (transcript.Text.Length > 0) transcript.ScrollRangeToVisible(new(transcript.Text.Length - 1, 1));
bool online = Peer is not null; compose.Enabled = online; send.Enabled = online; NavigationItem.RightBarButtonItem!.Enabled = online;
}
}
@@ -0,0 +1,31 @@
using VoiceCat.Core;
using UIKit;
namespace VoiceCat.iOS;
internal sealed class RootViewController : UIViewController
{
private readonly AppModel model;
private UIViewController? current;
internal RootViewController(AppModel model) { this.model = model; model.Changed += Refresh; model.IdentityRequested += ShowIdentity; }
public override void ViewDidLoad() { base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; Refresh(); }
private void Refresh()
{
bool main = model.IsConnected;
if (current is MainTabController && main || current is UINavigationController && !main) return;
UIViewController next = main ? new MainTabController(model) : new UINavigationController(new ServerListController(model));
if (current is not null) { current.WillMoveToParentViewController(null); current.View!.RemoveFromSuperview(); current.RemoveFromParentViewController(); }
UIView rootView = View!, nextView = next.View!;
AddChildViewController(next); nextView.Frame = rootView.Bounds; nextView.AutoresizingMask = UIViewAutoresizing.All; rootView.AddSubview(nextView); next.DidMoveToParentViewController(this); current = next;
}
private void ShowIdentity(ServerIdentityChallenge challenge)
{
UIAlertController alert = UIAlertController.Create(challenge.Status == VoiceCat.Crypto.TofuStatus.Mismatch ? "Server Identity Changed" : "New Server Identity",
$"{challenge.Host}:{challenge.Port}\n\nSHA-256\n{challenge.CertificateFingerprint}", UIAlertControllerStyle.Alert);
alert.AddAction(UIAlertAction.Create("Reject", UIAlertActionStyle.Destructive, _ => model.ResolveIdentity(false)));
alert.AddAction(UIAlertAction.Create("Trust", UIAlertActionStyle.Default, _ => model.ResolveIdentity(true)));
PresentViewController(alert, true, null);
}
}
@@ -0,0 +1,28 @@
using Foundation;
using UIKit;
namespace VoiceCat.iOS;
[Register("SceneDelegate")]
internal sealed class SceneDelegate : UIResponder, IUIWindowSceneDelegate
{
[Export("window")]
public UIWindow? Window { get; set; }
[Export("scene:willConnectToSession:options:")]
public void WillConnect(UIScene scene, UISceneSession session, UISceneConnectionOptions options)
{
if (scene is not UIWindowScene windowScene) return;
Window = new(windowScene) { RootViewController = new RootViewController(AppModel.Shared) };
Window.MakeKeyAndVisible();
}
[Export("sceneDidEnterBackground:")]
public void DidEnterBackground(UIScene scene) => AppModel.Shared.DidEnterBackground();
[Export("sceneWillEnterForeground:")]
public void WillEnterForeground(UIScene scene) => AppModel.Shared.WillEnterForeground();
[Export("sceneDidBecomeActive:")]
public void DidBecomeActive(UIScene scene) => AppModel.Shared.DidBecomeActive();
}
@@ -0,0 +1,67 @@
using VoiceCat.Core;
using UIKit;
namespace VoiceCat.iOS;
internal sealed class ServerListController : UITableViewController
{
private readonly AppModel model;
internal ServerListController(AppModel model) { this.model = model; Title = "Servers"; TabBarItem = new(UITabBarSystemItem.Favorites, 0); }
public override void ViewDidLoad()
{
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "server");
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PresentEditor(null)); model.Changed += Reload;
}
public override nint RowsInSection(UITableView tableView, nint section) => model.Profiles.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
UITableViewCell cell = tableView.DequeueReusableCell("server", indexPath);
ServerProfile p = model.Profiles[indexPath.Row];
var content = cell.DefaultContentConfiguration; content.Text = p.DisplayName; content.SecondaryText = p.Authentication.ToString(); cell.ContentConfiguration = content;
cell.Accessory = UITableViewCellAccessory.DisclosureIndicator; return cell;
}
public override async void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
tableView.DeselectRow(indexPath, true); try { await model.ConnectAsync(model.Profiles[indexPath.Row]); } catch (Exception e) { UiHelpers.ShowError(this, e); }
}
public override UISwipeActionsConfiguration GetTrailingSwipeActionsConfiguration(UITableView tableView, NSIndexPath indexPath)
{
ServerProfile profile = model.Profiles[indexPath.Row];
UIContextualAction edit = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Normal, "Edit", (_, _, done) => { PresentEditor(profile); done(true); });
UIContextualAction delete = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Destructive, "Delete", (_, _, done) => { model.RemoveProfile(profile); done(true); });
return UISwipeActionsConfiguration.FromActions([delete, edit]);
}
private void PresentEditor(ServerProfile? profile) => PresentViewController(new UINavigationController(new ServerEditorController(model, profile)), true, null);
private void Reload() { TableView.ReloadData(); NavigationItem.Prompt = model.IsConnecting ? model.Status : null; }
}
internal sealed class ServerEditorController : UIViewController
{
private readonly AppModel model; private readonly ServerProfile? existing;
private readonly UITextField host = UiHelpers.Field("Hostname or IP address");
private readonly UITextField port = UiHelpers.Field("Port");
private readonly UISegmentedControl mode = new(["Guest", "Account"]);
private readonly UITextField name = UiHelpers.Field("Nickname or username");
private readonly UITextField password = UiHelpers.Field("Password (optional)", true);
internal ServerEditorController(AppModel model, ServerProfile? existing) { this.model = model; this.existing = existing; Title = existing is null ? "Add Server" : "Edit Server"; }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemGroupedBackground; port.KeyboardType = UIKeyboardType.NumberPad;
UIStackView stack = new([host, port, mode, name, password]) { Axis = UILayoutConstraintAxis.Vertical, Spacing = 12, TranslatesAutoresizingMaskIntoConstraints = false };
View.AddSubview(stack); NSLayoutConstraint.ActivateConstraints([stack.TopAnchor.ConstraintEqualTo(View.SafeAreaLayoutGuide.TopAnchor, 24), stack.LeadingAnchor.ConstraintEqualTo(View.LayoutMarginsGuide.LeadingAnchor), stack.TrailingAnchor.ConstraintEqualTo(View.LayoutMarginsGuide.TrailingAnchor)]);
mode.SelectedSegment = existing?.Authentication == ServerAuthentication.Account ? 1 : 0; host.Text = existing?.Host; port.Text = (existing?.Port ?? 8384).ToString(); name.Text = existing?.Username ?? existing?.Nickname;
NavigationItem.LeftBarButtonItem = new(UIBarButtonSystemItem.Cancel, (_, _) => DismissViewController(true, null));
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Save, (_, _) => Save());
}
private void Save()
{
try
{
if (!ushort.TryParse(port.Text, out ushort number)) throw new ArgumentException("Enter a valid port.");
ServerAuthentication auth = mode.SelectedSegment == 1 ? ServerAuthentication.Account : ServerAuthentication.Guest;
ServerProfile p = ServerProfile.Create(host.Text ?? "", number, auth, auth == ServerAuthentication.Account ? name.Text : null, auth == ServerAuthentication.Guest ? name.Text : null, existing?.Id);
model.UpsertProfile(p, password.Text); DismissViewController(true, null);
}
catch (Exception e) { UiHelpers.ShowError(this, e); }
}
}
@@ -0,0 +1,99 @@
using AVFoundation;
using AVKit;
using ReplayKit;
using UIKit;
using VoiceCat.Audio;
using Voicecat.V1;
namespace VoiceCat.iOS;
internal sealed class SettingsController : UITableViewController
{
private readonly AppModel model;
internal SettingsController(AppModel model) : base(UITableViewStyle.InsetGrouped) { this.model = model; Title = "Settings"; model.Changed += Reload; IosAudioRouter.Shared.Changed += Reload; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "setting"); }
private void Reload() => TableView.ReloadData();
public override nint NumberOfSections(UITableView tableView) => 5;
public override nint RowsInSection(UITableView tableView, nint section) => section switch { 0 => 5, 1 => 4, 2 => 5, 3 => model.Client?.Permissions is { } p && (p.IsAdmin || p.CanAdminAccounts) ? 1 : 0, _ => 2 };
public override string? TitleForHeader(UITableView tableView, nint section) => section switch { 0 => "Audio", 1 => "Voice", 2 => "Notifications", 3 => "Administration", _ => "Server" };
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath path)
{
UITableViewCell cell = tableView.DequeueReusableCell("setting", path); cell.AccessoryView = null; cell.Accessory = UITableViewCellAccessory.None;
string title = path.Section switch
{
0 => path.Row switch { 0 => "Audio preset", 1 => "Speaker output", 2 => "Advanced audio", 3 => "Audio buffering", _ => model.ScreenSharing ? "Stop sharing screen audio" : "Share screen audio" },
1 => path.Row switch { 0 => "Input mode", 1 => $"VAD threshold: {model.Settings.VadThreshold:0.000}", 2 => $"Microphone volume: {model.Settings.InputGain:P0}", _ => "Microphone noise reduction" },
2 => path.Row switch { 0 => "Event sounds", 1 => $"Sound volume: {model.Settings.EventVolume:P0}", 2 => "Speak events", 3 => "Own voice activity sounds", _ => "Push-to-talk cue" },
3 => "Manage accounts", _ => path.Row == 0 ? "Disconnect" : "VoiceCat 0.0.1"
};
var content = cell.DefaultContentConfiguration; content.Text = title; content.SecondaryText = path.Section == 0 && path.Row == 0 ? IosAudioRouter.Shared.Preset.ToString() : path.Section == 1 && path.Row == 0 ? model.Settings.InputMode.ToString() : path.Section == 0 && path.Row == 3 ? $"{model.Settings.AudioBufferMilliseconds} ms" : path.Section == 0 && path.Row == 4 && model.ScreenSharing ? "Sharing" : null; cell.ContentConfiguration = content; cell.AccessibilityLabel = title + (content.SecondaryText is null ? "" : ", " + content.SecondaryText);
if (path.Section == 0 && path.Row == 1) cell.AccessoryView = Toggle(IosAudioRouter.Shared.ForceSpeaker, "Speaker output", (_, _) => IosAudioRouter.Shared.SetForceSpeaker(((UISwitch)cell.AccessoryView!).On));
else if (path.Section == 1 && path.Row == 3) cell.AccessoryView = Toggle(model.Settings.InputNoiseReduction, title, (_, _) => { model.Settings.InputNoiseReduction = ((UISwitch)cell.AccessoryView!).On; model.ApplyVoiceSettings(); });
else if (path.Section == 2 && path.Row is 0 or 2 or 3 or 4) { bool value = path.Row switch { 0 => model.Settings.EventSounds, 2 => model.Settings.SpokenEvents, 3 => model.Settings.SelfTalkSounds, _ => model.Settings.PushToTalkSound }; int row = path.Row; cell.AccessoryView = Toggle(value, title, (_, _) => { bool on = ((UISwitch)cell.AccessoryView!).On; if (row == 0) model.Settings.EventSounds = on; else if (row == 2) model.Settings.SpokenEvents = on; else if (row == 3) model.Settings.SelfTalkSounds = on; else model.Settings.PushToTalkSound = on; model.Save(); }); }
else if (path.Section == 0 && path.Row is 2 or 3 || path.Section == 3) cell.Accessory = UITableViewCellAccessory.DisclosureIndicator;
return cell;
}
public override async void RowSelected(UITableView tableView, NSIndexPath path)
{
tableView.DeselectRow(path, true); try
{
if (path.Section == 0 && path.Row == 0) Choice("Audio preset", Enum.GetValues<IosAudioPreset>().Select(value => value.ToString()).ToArray(), index => IosAudioRouter.Shared.SelectPreset(Enum.GetValues<IosAudioPreset>()[index]));
else if (path.Section == 0 && path.Row == 2) NavigationController?.PushViewController(new AdvancedAudioController(), true);
else if (path.Section == 0 && path.Row == 3) Choice("Audio buffering", ["Low latency (20 ms)", "Balanced (40 ms)", "Stable (60 ms)"], index => { model.Settings.AudioBufferMilliseconds = index switch { 0 => 20, 2 => 60, _ => 40 }; model.ApplyVoiceSettings(); });
else if (path.Section == 0 && path.Row == 4) ShowScreenSharing();
else if (path.Section == 1 && path.Row == 0) Choice("Input mode", ["Voice activation", "Push to talk", "Always on"], index => { model.Settings.InputMode = (AudioInputMode)index; model.ApplyVoiceSettings(); });
else if (path.Section == 1 && path.Row == 1) Slider("Voice activation threshold", 0.001f, 0.1f, model.Settings.VadThreshold, value => { model.Settings.VadThreshold = value; model.ApplyVoiceSettings(); });
else if (path.Section == 1 && path.Row == 2) Slider("Microphone volume", 0, 4, model.Settings.InputGain, value => { model.Settings.InputGain = value; model.ApplyVoiceSettings(); });
else if (path.Section == 2 && path.Row == 1) Slider("Sound volume", 0, 1, model.Settings.EventVolume, value => { model.Settings.EventVolume = value; model.Save(); });
else if (path.Section == 3) NavigationController?.PushViewController(new AccountsController(model), true);
else if (path.Section == 4 && path.Row == 0) await model.DisconnectAsync();
}
catch (Exception exception) { UiHelpers.ShowError(this, exception); }
}
private static UISwitch Toggle(bool value, string label, EventHandler changed) { var toggle = new UISwitch { On = value, AccessibilityLabel = label }; toggle.ValueChanged += changed; return toggle; }
private void Choice(string title, string[] values, Action<int> selected) { UIAlertController alert = UIAlertController.Create(title, null, UIAlertControllerStyle.ActionSheet); for (int i = 0; i < values.Length; i++) { int index = i; alert.AddAction(UIAlertAction.Create(values[i], UIAlertActionStyle.Default, _ => selected(index))); } alert.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); alert.PopoverPresentationController!.SourceView = View!; PresentViewController(alert, true, null); }
private void Slider(string title, float minimum, float maximum, float current, Action<float> changed) { var slider = new UISlider(new CoreGraphics.CGRect(16, 48, 238, 28)) { MinValue = minimum, MaxValue = maximum, Value = current, AccessibilityLabel = title }; UIAlertController alert = UIAlertController.Create(title, null, UIAlertControllerStyle.Alert); alert.View!.AddSubview(slider); alert.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); alert.AddAction(UIAlertAction.Create("Apply", UIAlertActionStyle.Default, _ => changed(slider.Value))); PresentViewController(alert, true, null); }
private void ShowScreenSharing()
{
// ScreenCaptureKit is the preferred iOS 27 path, but `IsAvailable` also reflects
// policy restrictions and transient OS state. Keep the bundled ReplayKit provider as
// a compatibility path instead of reporting that capable hardware is unsupported.
if (IosScreenCapture.IsAvailable) { model.ToggleScreenAudio(); return; }
#pragma warning disable CA1422
var picker = new RPSystemBroadcastPickerView(new CoreGraphics.CGRect(0, 0, 60, 60)) { PreferredExtension = IosConstants.BroadcastExtension, ShowsMicrophoneButton = false };
#pragma warning restore CA1422
foreach (UIView view in picker.Subviews) view.AccessibilityLabel = model.ScreenSharing ? "Stop sharing screen audio" : "Share screen audio";
UIAlertController alert = UIAlertController.Create("Screen Audio", "Tap the broadcast button, then choose Start Broadcast.", UIAlertControllerStyle.Alert); UIView alertView = alert.View!; alertView.AddSubview(picker); picker.Center = new(alertView.Bounds.GetMidX(), 110); alert.AddAction(UIAlertAction.Create("Done", UIAlertActionStyle.Cancel, null)); PresentViewController(alert, true, null);
}
}
internal sealed class AdvancedAudioController : UITableViewController
{
private readonly IosAudioRouter router = IosAudioRouter.Shared;
internal AdvancedAudioController() : base(UITableViewStyle.InsetGrouped) { Title = "Advanced Audio"; router.Changed += () => TableView.ReloadData(); }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "audio"); router.RefreshRoutes(); }
public override nint RowsInSection(UITableView tableView, nint section) => 8;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath path)
{
string title = path.Row switch { 0 => "Input port", 1 => "Microphone orientation", 2 => "Polar pattern", 3 => "Microphone mode", 4 => "Capture channels", 5 => "Bluetooth mode", 6 => "Voice processing and AGC", _ => "Current outputs" };
UITableViewCell cell = tableView.DequeueReusableCell("audio", path); var content = cell.DefaultContentConfiguration; content.Text = title; content.SecondaryText = path.Row switch { 0 => router.Inputs.FirstOrDefault(value => value.Id == router.SelectedInputId)?.Name ?? "Default", 1 => router.DataSources().FirstOrDefault(value => value.Id == router.SelectedDataSourceId)?.Name ?? "Default", 2 => router.SelectedPolarPattern.ToString(), 3 => router.MicMode.ToString(), 4 => router.CaptureChannels == 2 ? "Stereo" : "Mono", 5 => router.BluetoothMode.ToString(), 6 => router.UsesVoiceProcessing ? "AEC/NS on" + (router.AutomaticGainControl ? ", AGC on" : ", AGC off") : "Unavailable or off", _ => string.Join(", ", router.Outputs.Select(value => value.Name)) }; cell.ContentConfiguration = content; cell.AccessibilityLabel = title + ", " + content.SecondaryText; cell.Accessory = path.Row < 7 ? UITableViewCellAccessory.DisclosureIndicator : UITableViewCellAccessory.None; return cell;
}
public override void RowSelected(UITableView tableView, NSIndexPath path) { tableView.DeselectRow(path, true); if (path.Row == 0) Menu("Input port", router.Inputs.Select(value => value.Name).Prepend("Default").ToArray(), index => router.SelectInput(index == 0 ? null : router.Inputs[index - 1].Id)); else if (path.Row == 1) { var values = router.DataSources(); Menu("Microphone orientation", values.Select(value => value.Name).Prepend("Default").ToArray(), index => router.SelectDataSource(index == 0 ? null : values[index - 1].Id)); } else if (path.Row == 2) Menu("Polar pattern", Enum.GetValues<AVAudioDataSourcePolarPattern>().Select(value => value.ToString()).ToArray(), index => router.SelectPolarPattern(Enum.GetValues<AVAudioDataSourcePolarPattern>()[index])); else if (path.Row == 3) Menu("Microphone mode", Enum.GetValues<IosMicMode>().Select(value => value.ToString()).ToArray(), index => router.SetMicMode(Enum.GetValues<IosMicMode>()[index])); else if (path.Row == 4) Menu("Capture channels", ["Mono", "Stereo"], index => router.SetCaptureChannels(index + 1)); else if (path.Row == 5) Menu("Bluetooth mode", Enum.GetValues<IosBluetoothMode>().Select(value => value.ToString()).ToArray(), index => router.SetBluetoothMode(Enum.GetValues<IosBluetoothMode>()[index])); else if (path.Row == 6) Menu("Voice processing", ["Off", "On without AGC", "On with AGC"], index => { router.SetVoiceProcessing(index != 0); router.SetAutomaticGainControl(index == 2); }); }
private void Menu(string title, string[] values, Action<int> selected) { UIAlertController alert = UIAlertController.Create(title, null, UIAlertControllerStyle.ActionSheet); for (int i = 0; i < values.Length; i++) { int index = i; alert.AddAction(UIAlertAction.Create(values[i], UIAlertActionStyle.Default, _ => selected(index))); } alert.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); alert.PopoverPresentationController!.SourceView = View!; PresentViewController(alert, true, null); }
}
internal sealed class AccountsController : UITableViewController
{
private readonly AppModel model; private IReadOnlyList<AccountEntry> accounts = [];
internal AccountsController(AppModel model) { this.model = model; Title = "Accounts"; }
public override async void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "account"); NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PromptCreate()); await Refresh(); }
public override nint RowsInSection(UITableView tableView, nint section) => accounts.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath path) { AccountEntry account = accounts[path.Row]; UITableViewCell cell = tableView.DequeueReusableCell("account", path); var content = cell.DefaultContentConfiguration; content.Text = account.Username; content.SecondaryText = $"{(account.IsAdmin ? "Administrator" : "Account")} • created {DateTimeOffset.FromUnixTimeMilliseconds((long)account.CreatedAtUnixMs):d}"; cell.ContentConfiguration = content; cell.AccessibilityLabel = content.Text + ", " + content.SecondaryText; return cell; }
public override UISwipeActionsConfiguration GetTrailingSwipeActionsConfiguration(UITableView tableView, NSIndexPath path) { AccountEntry account = accounts[path.Row]; UIContextualAction reset = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Normal, "Reset password", (_, _, done) => { PromptReset(account); done(true); }); UIContextualAction delete = UIContextualAction.FromContextualActionStyle(UIContextualActionStyle.Destructive, "Delete", (_, _, done) => { ConfirmDelete(account); done(true); }); return UISwipeActionsConfiguration.FromActions([delete, reset]); }
private async Task Refresh() { try { accounts = await model.Client!.ListAccountsAsync(); TableView.ReloadData(); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
private void PromptCreate() => Credentials("Create Account", "Create", async (name, password) => await Execute(client => client.CreateAccountAsync(name, password)));
private void PromptReset(AccountEntry account) => Credentials($"Reset {account.Username}", "Reset", async (_, password) => await Execute(client => client.ResetPasswordAsync(account.Username, password)), account.Username, false);
private void ConfirmDelete(AccountEntry account) { UIAlertController alert = UIAlertController.Create("Delete account?", account.Username, UIAlertControllerStyle.Alert); alert.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); alert.AddAction(UIAlertAction.Create("Delete", UIAlertActionStyle.Destructive, async _ => await Execute(client => client.DeleteAccountAsync(account.Username)))); PresentViewController(alert, true, null); }
private void Credentials(string title, string action, Func<string, string, Task> run, string username = "", bool editName = true) { UIAlertController prompt = UIAlertController.Create(title, null, UIAlertControllerStyle.Alert); prompt.AddTextField(field => { field.Placeholder = "Username"; field.Text = username; field.Enabled = editName; field.AccessibilityLabel = "Username"; }); prompt.AddTextField(field => { field.Placeholder = "Password"; field.SecureTextEntry = true; field.AccessibilityLabel = "New password"; }); prompt.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); prompt.AddAction(UIAlertAction.Create(action, UIAlertActionStyle.Default, async _ => await run(prompt.TextFields?[0].Text ?? "", prompt.TextFields?[1].Text ?? ""))); PresentViewController(prompt, true, null); }
private async Task Execute(Func<VoiceCat.Core.VoiceCatClient, Task<GenericResult>> operation) { try { GenericResult result = await model.RunAdminAsync(operation); if (!result.Ok) throw new InvalidOperationException(result.Message); await Refresh(); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
+20
View File
@@ -0,0 +1,20 @@
using UIKit;
namespace VoiceCat.iOS;
internal static class UiHelpers
{
internal static void ShowError(UIViewController owner, Exception exception) => ShowMessage(owner, "VoiceCat", exception.Message);
internal static void ShowMessage(UIViewController owner, string title, string message)
{
UIAlertController alert = UIAlertController.Create(title, message, UIAlertControllerStyle.Alert);
alert.AddAction(UIAlertAction.Create("OK", UIAlertActionStyle.Default, null)); owner.PresentViewController(alert, true, null);
}
internal static UITextField Field(string placeholder, bool secure = false)
{
var field = new UITextField { Placeholder = placeholder, BorderStyle = UITextBorderStyle.RoundedRect,
SecureTextEntry = secure, TranslatesAutoresizingMaskIntoConstraints = false };
field.AccessibilityLabel = placeholder; return field;
}
}
@@ -0,0 +1,65 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0-ios27.0</TargetFramework>
<RuntimeIdentifier Condition="'$(RuntimeIdentifier)' == ''">iossimulator-arm64</RuntimeIdentifier>
<SupportedOSPlatformVersion>18.0</SupportedOSPlatformVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<ApplicationTitle>VoiceCat</ApplicationTitle>
<ApplicationId>me.iamtalon.voicecat</ApplicationId>
<ApplicationVersion>1</ApplicationVersion>
<ApplicationDisplayVersion>0.0.1</ApplicationDisplayVersion>
<CodesignEntitlements>Entitlements.plist</CodesignEntitlements>
<ApplicationManifest>Info.plist</ApplicationManifest>
<TrimMode Condition="'$(Configuration)' == 'Release'">full</TrimMode>
<NoWarn>$(NoWarn);XCODE_27_0_PREVIEW</NoWarn>
<WarningsNotAsErrors>$(WarningsNotAsErrors);IL2104</WarningsNotAsErrors>
<VoiceCatIosStatic>true</VoiceCatIosStatic>
<VoiceCatNativeRid Condition="$([System.String]::Copy('$(RuntimeIdentifier)').StartsWith('iossimulator'))">iossimulator-arm64</VoiceCatNativeRid>
<VoiceCatNativeRid Condition="'$(VoiceCatNativeRid)' == ''">ios-arm64</VoiceCatNativeRid>
<VoiceCatNativeMediaPath>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../../artifacts/native/runtimes/$(VoiceCatNativeRid)/native/libvoicecat_media.a'))</VoiceCatNativeMediaPath>
<VoiceCatBroadcastSdk Condition="$([System.String]::Copy('$(RuntimeIdentifier)').StartsWith('iossimulator'))">iphonesimulator</VoiceCatBroadcastSdk>
<VoiceCatBroadcastSdk Condition="'$(VoiceCatBroadcastSdk)' == ''">iphoneos</VoiceCatBroadcastSdk>
<VoiceCatBroadcastArch>arm64</VoiceCatBroadcastArch>
<VoiceCatBroadcastOutput>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)obj/$(Configuration)/$(TargetFramework)/$(RuntimeIdentifier)/broadcast'))</VoiceCatBroadcastOutput>
<VoiceCatIosCaptureOutput>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)obj/$(Configuration)/$(TargetFramework)/$(RuntimeIdentifier)/capture'))</VoiceCatIosCaptureOutput>
<_ComputePublishLocationDependsOn>VoiceCatPrepareIosNativeAssets;$(_ComputePublishLocationDependsOn)</_ComputePublishLocationDependsOn>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../../src/VoiceCat.Core/VoiceCat.Core.csproj" AdditionalProperties="VoiceCatIosStatic=true" />
<NativeReference Include="$(VoiceCatNativeMediaPath)" Condition="Exists('$(VoiceCatNativeMediaPath)')">
<Kind>Static</Kind>
<ForceLoad>true</ForceLoad>
</NativeReference>
<LinkerArgument Include="-Wl,-force_load,$(VoiceCatNativeMediaPath)" Condition="Exists('$(VoiceCatNativeMediaPath)')" />
<ReferenceNativeSymbol Include="vcm_opus_version;vcm_opus_error;vcm_encoder_create;vcm_encoder_destroy;vcm_encoder_set;vcm_encoder_get_dred;vcm_encode;vcm_decoder_create;vcm_decoder_destroy;vcm_decode;vcm_dred_decoder_create;vcm_dred_decoder_destroy;vcm_dred_create;vcm_dred_destroy;vcm_dred_parse;vcm_dred_decode;vcm_rnnoise_create;vcm_rnnoise_destroy;vcm_rnnoise_process">
<SymbolType>Function</SymbolType>
</ReferenceNativeSymbol>
<NativeReference Include="$(VoiceCatIosCaptureOutput)/libvoicecat_ios_capture.a">
<Kind>Static</Kind>
<ForceLoad>true</ForceLoad>
<Frameworks Condition="!$([System.String]::Copy('$(RuntimeIdentifier)').StartsWith('iossimulator'))">AVFoundation CoreMedia</Frameworks>
</NativeReference>
<AdditionalAppExtensions Include="$(VoiceCatBroadcastOutput)">
<Name>VoiceCatBroadcast</Name>
<BuildOutput>.</BuildOutput>
<CodesignEntitlements>$(VoiceCatBroadcastOutput)/VoiceCatBroadcast.xcent</CodesignEntitlements>
</AdditionalAppExtensions>
<BundleResource Include="../../../assets/sounds/*.wav" Link="Sounds/%(Filename)%(Extension)" />
<ImageAsset Include="Assets.xcassets/**" Link="Assets.xcassets/%(RecursiveDir)%(Filename)%(Extension)" />
</ItemGroup>
<Target Name="VoiceCatBuildBroadcastExtension" BeforeTargets="_ResolveAppExtensionReferences">
<Exec Command="&quot;$(MSBuildThisFileDirectory)build-broadcast-extension.sh&quot; &quot;$(Configuration)&quot; &quot;$(VoiceCatBroadcastSdk)&quot; &quot;$(VoiceCatBroadcastArch)&quot; &quot;$(VoiceCatBroadcastOutput)&quot;" />
</Target>
<Target Name="VoiceCatBuildIosCaptureBridge" BeforeTargets="PrepareForBuild">
<Exec Command="&quot;$(MSBuildThisFileDirectory)build-screen-capture-bridge.sh&quot; &quot;$(VoiceCatBroadcastSdk)&quot; &quot;$(VoiceCatBroadcastArch)&quot; &quot;$(VoiceCatIosCaptureOutput)&quot;" />
</Target>
<Target Name="VoiceCatPrepareIosNativeAssets">
<ItemGroup>
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'libvoicecat_media.dylib' or '%(Filename)%(Extension)' == 'voicecat_media.dll' or '%(Filename)%(Extension)' == 'libvoicecat_media.so'" />
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)" Condition="'%(Filename)%(Extension)' == 'NOTICE.txt' or '%(Filename)%(Extension)' == 'Opus.txt' or '%(Filename)%(Extension)' == 'RNNoise.txt'" />
</ItemGroup>
</Target>
</Project>
+53
View File
@@ -0,0 +1,53 @@
#!/bin/zsh
set -euo pipefail
configuration="${1:?configuration is required}"
sdk="${2:?sdk is required}"
architecture="${3:?architecture is required}"
output="${4:?output directory is required}"
script_dir="${0:A:h}"
project="$script_dir/../../../native/apple/broadcast/VoiceCatBroadcast.xcodeproj"
mkdir -p "$output"
signing=()
# Both keys must always be set. The extension manifest expands these placeholders, and an
# empty CURRENT_PROJECT_VERSION drops CFBundleVersion, which installd rejects. The defaults
# match ApplicationDisplayVersion/ApplicationVersion in VoiceCat.iOS.csproj so that the host
# and extension versions agree when no release build number is supplied.
versioning=(
MARKETING_VERSION="${VOICECAT_DISPLAY_VERSION:-0.0.1}"
CURRENT_PROJECT_VERSION="${VOICECAT_BUILD_NUMBER:-1}"
)
if [[ "$sdk" == "iphonesimulator" ]]; then
signing+=(CODE_SIGNING_ALLOWED=NO)
elif [[ -n "${VOICECAT_BROADCAST_CODESIGN_KEY:-}" && -n "${VOICECAT_BROADCAST_CODESIGN_PROVISION:-}" ]]; then
signing+=(
DEVELOPMENT_TEAM="${VOICECAT_DEVELOPMENT_TEAM:?VOICECAT_DEVELOPMENT_TEAM is required for distribution signing}"
CODE_SIGN_STYLE=Manual
CODE_SIGN_IDENTITY="$VOICECAT_BROADCAST_CODESIGN_KEY"
PROVISIONING_PROFILE_SPECIFIER="$VOICECAT_BROADCAST_CODESIGN_PROVISION"
)
elif [[ -n "${VOICECAT_DEVELOPMENT_TEAM:-}" ]]; then
signing+=(DEVELOPMENT_TEAM="$VOICECAT_DEVELOPMENT_TEAM" CODE_SIGN_STYLE=Automatic)
if [[ "${VOICECAT_ALLOW_PROVISIONING_UPDATES:-}" == "1" ]]; then
signing=(-allowProvisioningUpdates "${signing[@]}")
fi
fi
xcodebuild \
-project "$project" \
-scheme VoiceCatBroadcast \
-configuration "$configuration" \
-sdk "$sdk" \
-arch "$architecture" \
-derivedDataPath "$output/derived" \
CONFIGURATION_BUILD_DIR="$output" \
"${versioning[@]}" \
"${signing[@]}" \
build
xcent=$(find "$output/derived" -name 'VoiceCatBroadcast.appex.xcent' -print -quit)
if [[ -f "$xcent" ]]; then
cp "$xcent" "$output/VoiceCatBroadcast.xcent"
else
cp "$script_dir/../../../native/apple/broadcast/VoiceCatBroadcast.entitlements" "$output/VoiceCatBroadcast.xcent"
fi
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
sdk="$1"
arch="$2"
output="$3"
root="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
mkdir -p "$output"
sdk_path="$(xcrun --sdk "$sdk" --show-sdk-path)"
target="arm64-apple-ios18.0"
[ "$sdk" = "iphonesimulator" ] && target="arm64-apple-ios18.0-simulator"
source="$root/native/ios_screen_capture.m"
flags="-fobjc-arc -fmodules -fmodules-cache-path=$output/module-cache -Wno-unguarded-availability-new"
if [ "$sdk" = "iphonesimulator" ]; then source="$root/native/ios_screen_capture_stub.c"; flags=""; fi
# shellcheck disable=SC2086
xcrun --sdk "$sdk" clang -target "$target" -isysroot "$sdk_path" $flags -Werror -c "$source" -o "$output/ios_screen_capture.o"
xcrun --sdk "$sdk" ar rcs "$output/libvoicecat_ios_capture.a" "$output/ios_screen_capture.o"
@@ -0,0 +1,124 @@
#import <AVFoundation/AVFoundation.h>
#import <CoreMedia/CoreMedia.h>
#import <Foundation/Foundation.h>
#import <ScreenCaptureKit/ScreenCaptureKit.h>
#import <fcntl.h>
#import <stdatomic.h>
#import <sys/mman.h>
#import <sys/stat.h>
#import <unistd.h>
enum { VCRingHeader = 64, VCRingCapacity = 96000 };
@interface VCScreenCaptureBridge : NSObject <SCContentSharingPickerObserver, SCStreamOutput, SCStreamDelegate>
@property(nonatomic) SCStream *stream;
@property(nonatomic) AVAudioConverter *converter;
@property(nonatomic) AVAudioFormat *inputFormat;
@property(nonatomic) AVAudioFormat *outputFormat;
@property(nonatomic) NSString *ringPath;
@property(nonatomic) int ringFd;
@property(nonatomic) void *ringMap;
@end
@implementation VCScreenCaptureBridge
static SCContentSharingPicker *VCSharedPicker(void) {
Class pickerClass = NSClassFromString(@"SCContentSharingPicker");
return pickerClass ? [pickerClass performSelector:@selector(sharedPicker)] : nil;
}
- (instancetype)init {
if ((self = [super init])) {
_ringFd = -1;
_outputFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatInt16 sampleRate:48000 channels:2 interleaved:YES];
}
return self;
}
- (void)openRing {
if (_ringMap || !_ringPath) return;
[[NSFileManager defaultManager] createDirectoryAtPath:[_ringPath stringByDeletingLastPathComponent] withIntermediateDirectories:YES attributes:nil error:nil];
_ringFd = open(_ringPath.fileSystemRepresentation, O_RDWR | O_CREAT, 0644);
size_t size = VCRingHeader + VCRingCapacity * sizeof(int16_t);
if (_ringFd < 0 || ftruncate(_ringFd, (off_t)size) != 0) return;
_ringMap = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, _ringFd, 0);
if (_ringMap == MAP_FAILED) _ringMap = NULL;
if (_ringMap && *(uint32_t *)_ringMap != 0x56434252) {
memset(_ringMap, 0, VCRingHeader); *(uint32_t *)((uint8_t *)_ringMap + 4) = 1; atomic_thread_fence(memory_order_seq_cst); *(uint32_t *)_ringMap = 0x56434252;
}
}
- (void)setRingActive:(BOOL)active {
[self openRing]; if (!_ringMap) return;
if (active) { *(uint32_t *)((uint8_t *)_ringMap + 8) = 2; *(uint32_t *)((uint8_t *)_ringMap + 12) = 48000; }
atomic_thread_fence(memory_order_seq_cst); *(uint32_t *)((uint8_t *)_ringMap + 16) = active ? 1 : 0;
}
- (void)push:(const int16_t *)samples count:(NSUInteger)count {
if (!_ringMap || count == 0 || count > VCRingCapacity) return;
uint8_t *base = _ringMap; uint64_t write = *(uint64_t *)(base + 24); atomic_thread_fence(memory_order_seq_cst); uint64_t read = *(uint64_t *)(base + 32);
if (VCRingCapacity - (write - read) < count) return;
int16_t *data = (int16_t *)(base + VCRingHeader); NSUInteger index = write % VCRingCapacity;
NSUInteger first = MIN(count, VCRingCapacity - index); memcpy(data + index, samples, first * sizeof(int16_t));
if (first < count) memcpy(data, samples + first, (count - first) * sizeof(int16_t));
atomic_thread_fence(memory_order_seq_cst); *(uint64_t *)(base + 24) = write + count;
}
- (void)present:(NSString *)path API_AVAILABLE(ios(27.0)) {
self.ringPath = path; [self openRing]; SCContentSharingPicker *picker = VCSharedPicker();
Class configurationClass = NSClassFromString(@"SCContentSharingPickerConfiguration");
SCContentSharingPickerConfiguration *configuration = [configurationClass new];
if (!picker || !configuration) return;
configuration.showsMicrophoneControl = NO; configuration.showsCameraControl = NO; picker.defaultConfiguration = configuration;
[picker removeObserver:self]; [picker addObserver:self]; picker.active = YES; [picker presentPickerUsingContentStyle:SCShareableContentStyleDisplay];
}
- (void)stop {
[self.stream stopCaptureWithCompletionHandler:^(__unused NSError *error) {}]; self.stream = nil;
SCContentSharingPicker *picker = VCSharedPicker(); [picker removeObserver:self]; picker.active = NO; [self setRingActive:NO];
}
- (void)contentSharingPicker:(SCContentSharingPicker *)picker didUpdateWithFilter:(SCContentFilter *)filter forStream:(SCStream *)stream API_AVAILABLE(ios(27.0)) {
[self stop]; Class configurationClass = NSClassFromString(@"SCStreamConfiguration"); Class streamClass = NSClassFromString(@"SCStream");
SCStreamConfiguration *configuration = [configurationClass new]; if (!configuration || !streamClass) return; configuration.capturesAudio = YES;
configuration.excludesCurrentProcessAudio = YES; configuration.sampleRate = 48000; configuration.channelCount = 2;
configuration.width = 2; configuration.height = 2;
self.stream = [[streamClass alloc] initWithFilter:filter configuration:configuration delegate:self]; NSError *error = nil;
if (![self.stream addStreamOutput:self type:SCStreamOutputTypeAudio sampleHandlerQueue:dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0) error:&error]) { self.stream = nil; return; }
[self.stream startCaptureWithCompletionHandler:^(NSError *captureError) { [self setRingActive:captureError == nil]; }];
}
- (void)contentSharingPicker:(SCContentSharingPicker *)picker didCancelForStream:(SCStream *)stream API_AVAILABLE(ios(27.0)) { [self stop]; }
- (void)contentSharingPickerStartDidFailWithError:(NSError *)error API_AVAILABLE(ios(27.0)) { [self stop]; }
- (void)stream:(SCStream *)stream didStopWithError:(NSError *)error { [self stop]; }
- (void)stream:(SCStream *)stream didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer ofType:(SCStreamOutputType)type {
if (type != SCStreamOutputTypeAudio || !CMSampleBufferDataIsReady(sampleBuffer)) return;
CMAudioFormatDescriptionRef description = CMSampleBufferGetFormatDescription(sampleBuffer); const AudioStreamBasicDescription *asbd = CMAudioFormatDescriptionGetStreamBasicDescription(description);
if (!asbd) return; AVAudioFormat *format = [[AVAudioFormat alloc] initWithStreamDescription:asbd]; AVAudioFrameCount frames = (AVAudioFrameCount)CMSampleBufferGetNumSamples(sampleBuffer);
AVAudioPCMBuffer *input = [[AVAudioPCMBuffer alloc] initWithPCMFormat:format frameCapacity:frames]; input.frameLength = frames;
if (CMSampleBufferCopyPCMDataIntoAudioBufferList(sampleBuffer, 0, frames, input.mutableAudioBufferList) != noErr) return;
if (!self.converter || ![self.inputFormat isEqual:format]) { self.inputFormat = format; self.converter = [[AVAudioConverter alloc] initFromFormat:format toFormat:self.outputFormat]; }
AVAudioFrameCount capacity = (AVAudioFrameCount)ceil(frames * 48000.0 / format.sampleRate) + 64; AVAudioPCMBuffer *output = [[AVAudioPCMBuffer alloc] initWithPCMFormat:self.outputFormat frameCapacity:capacity];
__block BOOL supplied = NO; NSError *error = nil; AVAudioConverterOutputStatus status = [self.converter convertToBuffer:output error:&error withInputFromBlock:^AVAudioBuffer *(AVAudioPacketCount count, AVAudioConverterInputStatus *inputStatus) { if (supplied) { *inputStatus = AVAudioConverterInputStatus_NoDataNow; return nil; } supplied = YES; *inputStatus = AVAudioConverterInputStatus_HaveData; return input; }];
if (status == AVAudioConverterOutputStatus_Error || output.frameLength == 0 || !output.int16ChannelData) return;
[self push:output.int16ChannelData[0] count:output.frameLength * 2];
}
- (void)dealloc {
[self setRingActive:NO]; if (_ringMap) munmap(_ringMap, VCRingHeader + VCRingCapacity * sizeof(int16_t)); if (_ringFd >= 0) close(_ringFd);
}
@end
static VCScreenCaptureBridge *bridge;
int vc_ios_screen_capture_available(void) {
if (@available(iOS 27.0, *)) return VCSharedPicker().available ? 1 : 0;
return 0;
}
void vc_ios_screen_capture_present(const char *ring_path) {
if (@available(iOS 27.0, *)) { if (!bridge) bridge = [VCScreenCaptureBridge new]; [bridge present:[NSString stringWithUTF8String:ring_path]]; }
}
void vc_ios_screen_capture_stop(void) { [bridge stop]; }
@@ -0,0 +1,3 @@
int vc_ios_screen_capture_available(void) { return 0; }
void vc_ios_screen_capture_present(const char *ring_path) { (void)ring_path; }
void vc_ios_screen_capture_stop(void) {}
@@ -0,0 +1,59 @@
{
"version": 1,
"dependencies": {
"net10.0-ios27.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.12, )",
"resolved": "10.0.12",
"contentHash": "xi+BDjFpW+Sb+MHFHaH6Y/gV9I8BluFwRXc1QyCdoZbIK26eNiBeFuMTe/FMwc33G1wdHCyDg7CVTmb8OdQrMQ=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0-ios27.0/ios-arm64": {},
"net10.0-ios27.0/iossimulator-arm64": {}
}
}
@@ -0,0 +1,58 @@
{
"version": 1,
"dependencies": {
"net10.0-ios27.0": {
"Microsoft.NET.ILLink.Tasks": {
"type": "Direct",
"requested": "[10.0.12, )",
"resolved": "10.0.12",
"contentHash": "xi+BDjFpW+Sb+MHFHaH6Y/gV9I8BluFwRXc1QyCdoZbIK26eNiBeFuMTe/FMwc33G1wdHCyDg7CVTmb8OdQrMQ=="
},
"BouncyCastle.Cryptography": {
"type": "Transitive",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"Google.Protobuf": {
"type": "Transitive",
"resolved": "3.36.1",
"contentHash": "77AqPEoaY1ODE+syYBHti0jXiwQq0J/fUr/fRyYhNlc9oKtH5dZZEr/OLKtdKNVG83PRnCYB2r8B80ZrObzOGQ=="
},
"voicecat.audio": {
"type": "Project",
"dependencies": {
"VoiceCat.Codec": "[1.0.0, )",
"VoiceCat.Dsp": "[1.0.0, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.codec": {
"type": "Project"
},
"voicecat.core": {
"type": "Project",
"dependencies": {
"VoiceCat.Audio": "[1.0.0, )",
"VoiceCat.Crypto": "[1.0.0, )"
}
},
"voicecat.crypto": {
"type": "Project",
"dependencies": {
"BouncyCastle.Cryptography": "[2.6.2, )",
"VoiceCat.Protocol": "[1.0.0, )"
}
},
"voicecat.dsp": {
"type": "Project"
},
"voicecat.protocol": {
"type": "Project",
"dependencies": {
"Google.Protobuf": "[3.36.1, )"
}
}
},
"net10.0-ios27.0/iossimulator-arm64": {}
}
}
+53
View File
@@ -0,0 +1,53 @@
#!/bin/sh
set -eu
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
configuration="${CONFIGURATION:-Debug}"
output="$root/dist/ios-managed-device"
dotnet_host="${VOICECAT_DOTNET:-/usr/local/share/dotnet/dotnet}"
while [ "$#" -gt 0 ]; do
case "$1" in
--configuration) configuration="$2"; shift 2 ;;
--output) output="$2"; shift 2 ;;
-h|--help)
echo "usage: $0 [--configuration Debug|Release] [--output DIR]"
echo "optional env: VOICECAT_DOTNET, VOICECAT_BUILD_NUMBER, VOICECAT_DISPLAY_VERSION, VOICECAT_CODESIGN_KEY, VOICECAT_CODESIGN_PROVISION"
echo "distribution extension env: VOICECAT_DEVELOPMENT_TEAM, VOICECAT_BROADCAST_CODESIGN_KEY, VOICECAT_BROADCAST_CODESIGN_PROVISION"
exit 0 ;;
*) echo "unknown argument: $1" >&2; exit 2 ;;
esac
done
project="$root/clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj"
"$root/scripts/build-native-ios.sh"
"$dotnet_host" restore "$project" --locked-mode -p:VoiceCatIosStatic=true
device_lock="obj/packages.device.lock.json"
"$dotnet_host" restore "$project" -r ios-arm64 --no-dependencies \
-p:RestorePackagesWithLockFile=true -p:RestoreLockedMode=false -p:NuGetLockFilePath="$device_lock"
set -- build "$project" -c "$configuration" -r ios-arm64 --no-restore -m:1 \
-p:NuGetLockFilePath="$device_lock"
if [ -n "${VOICECAT_CODESIGN_KEY:-}" ]; then set -- "$@" -p:CodesignKey="$VOICECAT_CODESIGN_KEY"; fi
if [ -n "${VOICECAT_CODESIGN_PROVISION:-}" ]; then set -- "$@" -p:CodesignProvision="$VOICECAT_CODESIGN_PROVISION"; fi
if [ -n "${VOICECAT_BUILD_NUMBER:-}" ]; then set -- "$@" -p:ApplicationVersion="$VOICECAT_BUILD_NUMBER"; fi
"$dotnet_host" "$@"
app="$root/clients/apple/VoiceCat.iOS/bin/$configuration/net10.0-ios27.0/ios-arm64/VoiceCat.iOS.app"
[ -d "$app" ] || { echo "device app was not produced at $app" >&2; exit 1; }
/usr/bin/strings "$app/VoiceCat.Codec.dll" | /usr/bin/grep -q GetMainProgramHandle || {
echo "device codec does not contain the iOS static-library resolver" >&2
exit 1
}
/usr/bin/nm -gU "$app/VoiceCat.iOS" | /usr/bin/grep -q _vcm_decoder_create || {
echo "device executable does not export the statically linked media codec" >&2
exit 1
}
/usr/bin/nm -gU "$app/VoiceCat.iOS" | /usr/bin/grep -q _vcm_rnnoise_create || {
echo "device executable does not export the statically linked RNNoise processor" >&2
exit 1
}
mkdir -p "$output"
/usr/bin/ditto "$app" "$output/VoiceCat.iOS.app"
/usr/bin/codesign --verify --deep --strict "$output/VoiceCat.iOS.app"
echo "$output/VoiceCat.iOS.app"
+35
View File
@@ -0,0 +1,35 @@
#!/bin/sh
set -eu
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
device="${VOICECAT_IOS_DEVICE:-}"
configuration="${CONFIGURATION:-Debug}"
build=true
console=false
while [ "$#" -gt 0 ]; do
case "$1" in
--device) device="$2"; shift 2 ;;
--configuration) configuration="$2"; shift 2 ;;
--no-build) build=false; shift ;;
--console) console=true; shift ;;
--list) xcrun devicectl list devices; exit 0 ;;
-h|--help)
echo "usage: $0 --device NAME-OR-UDID [--configuration Debug|Release] [--no-build] [--console]"
echo "use --list to show devices, or set VOICECAT_IOS_DEVICE"
exit 0 ;;
*) echo "unknown argument: $1" >&2; exit 2 ;;
esac
done
[ -n "$device" ] || { echo "pass --device NAME-OR-UDID or set VOICECAT_IOS_DEVICE" >&2; exit 2; }
app="$root/dist/ios-managed-device/VoiceCat.iOS.app"
if $build; then "$root/clients/apple/build-ios-device.sh" --configuration "$configuration"; fi
[ -d "$app" ] || { echo "managed device app not found at $app; run build-ios-device.sh first" >&2; exit 1; }
xcrun devicectl device install app --device "$device" "$app"
if $console; then
xcrun devicectl device process launch --device "$device" --terminate-existing --console me.iamtalon.voicecat
else
xcrun devicectl device process launch --device "$device" --terminate-existing me.iamtalon.voicecat
fi
+112
View File
@@ -0,0 +1,112 @@
#!/bin/zsh
set -euo pipefail
script_dir=${0:A:h}
project="$script_dir/VoiceCat.Mac/VoiceCat.Mac.csproj"
configuration=Release
dry_run=0
dotnet_host=${VOICECAT_DOTNET:-}
if [[ -z "$dotnet_host" ]]; then
if [[ -x /usr/local/share/dotnet/dotnet ]]; then
dotnet_host=/usr/local/share/dotnet/dotnet
else
dotnet_host=$(command -v dotnet)
fi
fi
if [[ ${1:-} == "--dry-run" ]]; then
dry_run=1
fi
publish_properties=(-p:ArchiveOnBuild=false)
if (( dry_run )); then
# Hardened-runtime library validation requires the executable and every bundled dylib to
# have the same real Team ID. Ad-hoc signatures have no Team ID, so a hardened ad-hoc app
# passes codesign verification but is rejected by dyld at launch. Keep local validation
# ad-hoc and reserve the hardened runtime for the Developer ID distribution path below.
publish_properties+=(-p:UseHardenedRuntime=false)
fi
"$dotnet_host" publish "$project" -c "$configuration" --no-restore "${publish_properties[@]}"
built_app="$script_dir/VoiceCat.Mac/bin/Release/net10.0-macos27.0/osx-arm64/VoiceCat.app"
if [[ ! -d "$built_app" ]]; then
print -u2 "VoiceCat.app was not produced at $built_app"
exit 1
fi
# Signing mutates every nested runtime binary. Work on a disposable distribution copy so a
# validation run can never poison MSBuild's incremental output or its signing caches.
distribution="$script_dir/VoiceCat.Mac/bin/Release/distribution"
app="$distribution/VoiceCat.app"
if [[ -e "$app" ]]; then
rm -rf "$app"
fi
mkdir -p "$distribution"
ditto "$built_app" "$app"
entitlements="$script_dir/VoiceCat.Mac/VoiceCat.Mac.entitlements"
sign_nested_code() {
local identity=$1
local timestamp_option=${2:-}
local runtime_option=${3:-}
local nested
local sign_options=(--force)
[[ -n "$timestamp_option" ]] && sign_options+=("$timestamp_option")
[[ -n "$runtime_option" ]] && sign_options+=("$runtime_option")
# Do not use codesign --deep for signing. It is intended as an emergency repair operation
# and can apply the app's entitlements to nested code. Sign each bundled Mach-O library first,
# then sign the outer app once with its own entitlements.
while IFS= read -r -d '' nested; do
codesign "${sign_options[@]}" --sign "$identity" "$nested"
done < <(find "$app/Contents" -type f \( -name '*.dylib' -o -name '*.so' \) -print0)
}
install_verified_primary_bundle() {
# dotnet publish must create the workload bundle before it can be normalized. Replace that
# misleading intermediate with the verified result so both documented output paths work.
rm -rf "$built_app"
ditto "$app" "$built_app"
}
if (( dry_run )); then
sign_nested_code -
codesign --force --entitlements "$entitlements" --sign - "$app"
codesign --verify --deep --strict "$app"
spctl --assess --type execute "$app" 2>/dev/null || true
install_verified_primary_bundle
print "Validated ad-hoc build: $app"
exit 0
fi
identity=${VOICECAT_CODESIGN_IDENTITY:-}
if [[ -z "$identity" ]]; then
print -u2 "Set VOICECAT_CODESIGN_IDENTITY to a Developer ID Application identity."
exit 2
fi
sign_nested_code "$identity" --timestamp --options=runtime
codesign --force --timestamp --options runtime \
--entitlements "$entitlements" \
--sign "$identity" "$app"
codesign --verify --deep --strict --verbose=2 "$app"
install_verified_primary_bundle
archive="$distribution/VoiceCat-macOS-arm64.zip"
ditto -c -k --keepParent "$app" "$archive"
if [[ -n ${APPLE_ID:-} && -n ${APPLE_TEAM_ID:-} && -n ${APPLE_APP_PASSWORD:-} ]]; then
xcrun notarytool submit "$archive" --wait \
--apple-id "$APPLE_ID" --team-id "$APPLE_TEAM_ID" --password "$APPLE_APP_PASSWORD"
xcrun stapler staple "$app"
xcrun stapler validate "$app"
spctl --assess --type execute --verbose=2 "$app"
ditto -c -k --keepParent "$app" "$archive"
print "Signed, notarized, and stapled: $archive"
else
print "Signed archive produced without notarization: $archive"
print "Set APPLE_ID, APPLE_TEAM_ID, and APPLE_APP_PASSWORD to submit it."
fi
+19 -14
View File
@@ -1,18 +1,23 @@
# Windows client — placeholder # Windows client
Built in **M4** (see [`docs/roadmap.md`](../../docs/roadmap.md)). C# / .NET 8+, consuming The .NET 10 WinForms client uses `VoiceCat.Core` and `VoiceCat.Audio`. Its only native runtime
`libvoicecat` through the C ABI ([`core/include/voicecat.h`](../../core/include/voicecat.h)). component is `voicecat_media.dll`, the narrow Opus/RNNoise shim.
Planned shape (see [`docs/architecture.md`](../../docs/architecture.md) §4 and ```powershell
[`docs/tech-stack.md`](../../docs/tech-stack.md) §2): ./scripts/build-native.ps1
dotnet restore clients/windows/VoiceCat.slnx --locked-mode
dotnet build clients/windows/VoiceCat.slnx -c Release --no-restore
./clients/windows/publish-client.ps1
```
- A .NET solution with a P/Invoke interop layer over the C ABI using **`LibraryImport`** Publishing produces `artifacts/client/win-x64`. Runtime-specific restore graphs are generated
(source-generated, .NET 7+). Build `libvoicecat` as a **shared library** under each project's ignored `obj` directory; they are intentionally not committed because they
(`-DVOICECAT_BUILD_SHARED=ON`) so the DLL sits beside the app. include SDK-injected packages that change with servicing updates. The noninteractive form and
- The `on_event` callback marshaled as a function pointer (`[UnmanagedCallersOnly]`) to avoid WASAPI device gate is:
delegate-lifetime issues; keep the interface "chunky" to minimize managed↔native crossings.
- UI in **WinUI 3** (most native) or **Avalonia** (if a single C# desktop UI is wanted later).
- Audio (capture/playback, WASAPI loopback for `SCREEN_AUDIO`) is handled inside the core; the
C# layer only drives device selection, meters, and the VAD/PTT + per-user NR controls.
Nothing here yet — the core must reach M2 (working voice) before the GUI is worth building. ```powershell
./artifacts/client/win-x64/VoiceCat.App.exe --smoke-test --audio
```
Manual release validation includes keyboard operation, NVDA navigation, curated announcements,
and a sustained real call.
@@ -0,0 +1,275 @@
using System.Diagnostics;
using System.Runtime.InteropServices;
namespace VoiceCat.App.Audio;
// ── Scope types (mirror macOS ScreenAudioScope / ScreenAudioSelection) ────────
public abstract record AppAudioScope;
// ExcludeSelf=true captures the whole system render mix minus VoiceCat's own process tree
// (kills the self-echo loop). Routed through the external-feed mixer as a single EXCLUDE
// capture; ExcludeSelf=false keeps the core's whole-device loopback path.
public sealed record EntireDesktop(bool ExcludeSelf = false) : AppAudioScope;
public sealed record OnlyApps(IReadOnlyList<int> Pids, IReadOnlyList<string> Names) : AppAudioScope;
public sealed record AllExceptApps(IReadOnlyList<int> Pids, IReadOnlyList<string> Names) : AppAudioScope;
// IsPlaying = a process in this app's executable group currently has an *active* audio
// session on some render endpoint. Purely informational for the picker — capture works
// on any PID regardless (process-loopback yields silence until the app plays).
public sealed record AudioAppInfo(int Pid, string DisplayName, bool IsPlaying);
// ── Enumerator ─────────────────────────────────────────────────────────────────
public static class AudioSessionEnumerator
{
// Returns every app in the user's interactive session — windowed or not — so any of
// them can be picked for capture even before it starts producing audio. Process
// loopback (see ProcessLoopbackCapture) targets a PID and its child tree, so a silent
// selection simply starts working the moment that app plays.
//
// Apps are deduped by executable (multiple PIDs of the same program collapse to one
// row whose PID is the process-tree root). Session-0 services and VoiceCat itself are
// excluded. Apps currently producing audio are flagged IsPlaying and sorted first.
public static IReadOnlyList<AudioAppInfo> GetAudioApps()
{
int selfPid = Environment.ProcessId;
int sessionId = SafeCurrentSessionId();
var playingPids = GetActiveAudioPids();
// Group by executable name; keep the best representative PID per group.
var groups = new Dictionary<string, AppGroup>(StringComparer.OrdinalIgnoreCase);
foreach (var proc in Process.GetProcesses())
{
try
{
if (proc.Id == selfPid) continue;
if (proc.SessionId != sessionId) continue; // drop session-0 services
string name = proc.ProcessName;
if (string.IsNullOrEmpty(name)) continue;
bool hasWindow = proc.MainWindowHandle != IntPtr.Zero;
bool playing = playingPids.Contains(proc.Id);
string title = hasWindow ? SafeWindowTitle(proc) : "";
string display = title.Length > 0 ? $"{name} — {title}" : name;
if (groups.TryGetValue(name, out var g))
{
g.AnyPlaying |= playing;
// Prefer a windowed PID (the process-tree root) as the capture target;
// among non-windowed, prefer a currently-playing PID.
bool better = (hasWindow && !g.HasWindow)
|| (!g.HasWindow && !hasWindow && playing && !g.RepPlaying);
if (better)
{
g.Pid = proc.Id;
g.Display = display;
g.HasWindow = hasWindow;
g.RepPlaying = playing;
}
}
else
{
groups[name] = new AppGroup
{
Pid = proc.Id,
Display = display,
HasWindow = hasWindow,
RepPlaying = playing,
AnyPlaying = playing,
};
}
}
catch { /* protected/exited process — skip */ }
finally { proc.Dispose(); }
}
return groups.Values
.Select(g => new AudioAppInfo(g.Pid, g.Display, g.AnyPlaying))
.OrderByDescending(a => a.IsPlaying)
.ThenBy(a => a.DisplayName, StringComparer.OrdinalIgnoreCase)
.ToList();
}
private sealed class AppGroup
{
public int Pid;
public string Display = "";
public bool HasWindow;
public bool RepPlaying; // representative PID is playing
public bool AnyPlaying; // any PID in the group is playing
}
private static int SafeCurrentSessionId()
{
try { using var me = Process.GetCurrentProcess(); return me.SessionId; }
catch { return 1; } // typical interactive session fallback
}
private static string SafeWindowTitle(Process proc)
{
try { return proc.MainWindowTitle; }
catch { return ""; }
}
// ── WASAPI: PIDs with an active render session (any endpoint) ─────────────
// Scans ALL active render endpoints, not just the default — an app routed to a
// secondary device still counts as "playing now".
private static HashSet<int> GetActiveAudioPids()
{
var pids = new HashSet<int>();
IMMDeviceEnumerator? enumerator = null;
IMMDeviceCollection? devices = null;
try
{
enumerator = (IMMDeviceEnumerator)Activator.CreateInstance(
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
if (enumerator.EnumAudioEndpoints(0 /*eRender*/, 0x1 /*DEVICE_STATE_ACTIVE*/,
out devices) < 0 || devices == null)
return pids;
devices.GetCount(out int devCount);
for (int d = 0; d < devCount; d++)
CollectActivePids(devices, d, pids);
}
catch { /* no audio device or WASAPI unavailable — ignore */ }
finally
{
if (devices != null) Marshal.ReleaseComObject(devices);
if (enumerator != null) Marshal.ReleaseComObject(enumerator);
}
return pids;
}
private static void CollectActivePids(IMMDeviceCollection devices, int index, HashSet<int> pids)
{
IMMDevice? device = null;
IAudioSessionManager2? manager = null;
IAudioSessionEnumerator? sessions = null;
try
{
if (devices.Item(index, out device) < 0 || device == null) return;
var mgr2Iid = new Guid("77AA99A0-1BD6-484F-8BC7-2C654C9A9B6F");
if (device.Activate(ref mgr2Iid, 0x17 /*CLSCTX_ALL*/, IntPtr.Zero, out object mgr) < 0)
return;
manager = (IAudioSessionManager2)mgr;
if (manager.GetSessionEnumerator(out sessions) < 0 || sessions == null) return;
sessions.GetCount(out int count);
for (int i = 0; i < count; i++)
{
IAudioSessionControl? ctrl = null;
try
{
if (sessions.GetSession(i, out ctrl) < 0 || ctrl == null) continue;
ctrl.GetState(out int state);
if (state != 1 /*AudioSessionStateActive*/) continue;
var ctrl2 = (IAudioSessionControl2)ctrl;
ctrl2.GetProcessId(out uint pid);
if (pid != 0) pids.Add((int)pid);
}
catch { /* stale session */ }
finally { if (ctrl != null) Marshal.ReleaseComObject(ctrl); }
}
}
catch { /* device went away */ }
finally
{
if (sessions != null) Marshal.ReleaseComObject(sessions);
if (manager != null) Marshal.ReleaseComObject(manager);
if (device != null) Marshal.ReleaseComObject(device);
}
}
// ── COM interface declarations ─────────────────────────────────────────────
[ComImport, Guid("A95664D2-9614-4F35-A746-DE8DB63617E6"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDeviceEnumerator
{
[PreserveSig] int EnumAudioEndpoints(int dataFlow, int stateMask, out IMMDeviceCollection devices);
[PreserveSig] int GetDefaultAudioEndpoint(int dataFlow, int role, out IMMDevice endpoint);
[PreserveSig] int GetDevice([MarshalAs(UnmanagedType.LPWStr)] string id, out IMMDevice device);
[PreserveSig] int RegisterEndpointNotificationCallback(IntPtr client);
[PreserveSig] int UnregisterEndpointNotificationCallback(IntPtr client);
}
[ComImport, Guid("0BD7A1BE-7A1A-44DB-8397-CC5392387B5E"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDeviceCollection
{
[PreserveSig] int GetCount(out int count);
[PreserveSig] int Item(int index, out IMMDevice device);
}
[ComImport, Guid("D666063F-1587-4E43-81F1-B948E807363F"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDevice
{
[PreserveSig] int Activate(ref Guid iid, int clsCtx, IntPtr activationParams,
[MarshalAs(UnmanagedType.IUnknown)] out object ppInterface);
[PreserveSig] int OpenPropertyStore(int stgmAccess, out IntPtr propStore);
[PreserveSig] int GetId([MarshalAs(UnmanagedType.LPWStr)] out string id);
[PreserveSig] int GetState(out int state);
}
[ComImport, Guid("77AA99A0-1BD6-484F-8BC7-2C654C9A9B6F"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioSessionManager2
{
[PreserveSig] int GetAudioSessionControl(ref Guid audioSessionGuid, int streamFlags,
out IAudioSessionControl session);
[PreserveSig] int GetSimpleAudioVolume(ref Guid audioSessionGuid, int streamFlags,
out IntPtr audioVolume);
[PreserveSig] int GetSessionEnumerator(out IAudioSessionEnumerator sessionEnum);
[PreserveSig] int RegisterSessionNotification(IntPtr notification);
[PreserveSig] int UnregisterSessionNotification(IntPtr notification);
[PreserveSig] int RegisterDuckNotification([MarshalAs(UnmanagedType.LPWStr)] string sessionID,
IntPtr notification);
[PreserveSig] int UnregisterDuckNotification(IntPtr notification);
}
[ComImport, Guid("E2F5BB11-0570-40CA-ACDD-3AA01277DEE8"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioSessionEnumerator
{
[PreserveSig] int GetCount(out int sessionCount);
[PreserveSig] int GetSession(int sessionIndex, out IAudioSessionControl session);
}
[ComImport, Guid("F4B1A599-7266-4319-A8CA-E70ACB11E8CD"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioSessionControl
{
[PreserveSig] int GetState(out int state);
[PreserveSig] int GetDisplayName([MarshalAs(UnmanagedType.LPWStr)] out string name);
[PreserveSig] int SetDisplayName([MarshalAs(UnmanagedType.LPWStr)] string name,
ref Guid eventContext);
[PreserveSig] int GetIconPath([MarshalAs(UnmanagedType.LPWStr)] out string iconPath);
[PreserveSig] int SetIconPath([MarshalAs(UnmanagedType.LPWStr)] string iconPath,
ref Guid eventContext);
[PreserveSig] int GetGroupingParam(out Guid groupingParam);
[PreserveSig] int SetGroupingParam(ref Guid groupingParam, ref Guid eventContext);
[PreserveSig] int RegisterAudioSessionNotification(IntPtr notification);
[PreserveSig] int UnregisterAudioSessionNotification(IntPtr notification);
}
[ComImport, Guid("BFB7FF88-7239-4FC9-8FA2-07C950BE9C6D"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioSessionControl2 : IAudioSessionControl
{
[PreserveSig] int GetSessionIdentifier([MarshalAs(UnmanagedType.LPWStr)] out string id);
[PreserveSig] int GetSessionInstanceIdentifier([MarshalAs(UnmanagedType.LPWStr)] out string id);
[PreserveSig] int GetProcessId(out uint pid);
[PreserveSig] int IsSystemSoundsSession();
[PreserveSig] int SetDuckingPreference(bool optOut);
}
}
@@ -0,0 +1,450 @@
using System.Runtime.InteropServices;
// Captures a second hardware input for AUX_DEVICE and feeds it through vc_stream_feed_pcm.
// Endpoint identifiers are WASAPI-specific and cannot be exchanged with the core's miniaudio ids.
namespace VoiceCat.App.Audio;
/// <summary>An audio input (capture) endpoint for the aux-stream device picker. <see cref="Id"/>
/// is a WASAPI endpoint id (round-trip only; never construct by hand) — pass null to capture the
/// system default. <see cref="ToString"/> returns the friendly name for ComboBox display.</summary>
public sealed record InputDeviceInfo(string Id, string Name, bool IsDefault)
{
public override string ToString() => Name;
}
/// <summary>Enumerates WASAPI capture endpoints. Separate from the core's vc_list_devices because
/// the aux device is opened client-side and needs a WASAPI id, not a miniaudio one.</summary>
public static class InputDeviceEnumerator
{
public static IReadOnlyList<InputDeviceInfo> List(bool input = true)
{
var result = new List<InputDeviceInfo>();
InputDeviceCapture.IMMDeviceEnumerator? enumerator = null;
IntPtr collectionPtr = IntPtr.Zero;
string? defaultId = null;
try
{
enumerator = (InputDeviceCapture.IMMDeviceEnumerator)Activator.CreateInstance(
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
// Resolve the default capture endpoint id so the picker can flag it.
if (enumerator.GetDefaultAudioEndpoint(input ? 1 : 0, 0 /*eConsole*/,
out var defDev) == 0 && defDev != null)
{
try { if (defDev.GetId(out string id) == 0) defaultId = id; }
finally { Marshal.ReleaseComObject(defDev); }
}
// DEVICE_STATE_ACTIVE = 0x1 — only currently-usable endpoints.
if (enumerator.EnumAudioEndpoints(input ? 1 : 0, 0x1, out collectionPtr) != 0
|| collectionPtr == IntPtr.Zero)
return result;
var collection = (InputDeviceCapture.IMMDeviceCollection)
Marshal.GetObjectForIUnknown(collectionPtr);
collection.GetCount(out int count);
for (int i = 0; i < count; i++)
{
if (collection.Item(i, out var dev) != 0 || dev == null) continue;
try
{
if (dev.GetId(out string id) != 0) continue;
string name = ReadFriendlyName(dev) ?? "Unknown input device";
result.Add(new InputDeviceInfo(id, name, id == defaultId));
}
finally { Marshal.ReleaseComObject(dev); }
}
}
catch { /* no audio subsystem / WASAPI unavailable — return what we have */ }
finally
{
if (collectionPtr != IntPtr.Zero) Marshal.Release(collectionPtr);
if (enumerator != null) Marshal.ReleaseComObject(enumerator);
}
return result.OrderBy(d => d.Name, StringComparer.OrdinalIgnoreCase).ToList();
}
private static string? ReadFriendlyName(InputDeviceCapture.IMMDevice dev)
{
if (dev.OpenPropertyStore(0 /*STGM_READ*/, out IntPtr storePtr) != 0
|| storePtr == IntPtr.Zero)
return null;
try
{
var store = (InputDeviceCapture.IPropertyStore)Marshal.GetObjectForIUnknown(storePtr);
// PKEY_Device_FriendlyName = {a45c254e-df1c-4efd-8020-67d146a850e0}, pid 14.
var key = new InputDeviceCapture.PropertyKey
{
fmtid = new Guid("a45c254e-df1c-4efd-8020-67d146a850e0"),
pid = 14,
};
if (store.GetValue(ref key, out var pv) != 0) return null;
try
{
// VT_LPWSTR = 31.
return pv.vt == 31 ? Marshal.PtrToStringUni(pv.pointerValue) : null;
}
finally { InputDeviceCapture.PropVariantClear(ref pv); }
}
finally { Marshal.Release(storePtr); }
}
}
/// <summary>Captures a single hardware input device in WASAPI shared mode and raises a 20 ms
/// (960 samples/channel @ 48 kHz, interleaved s16) frame event. The caller feeds these to the
/// core's external-feed stream. Threading mirrors ProcessLoopbackCapture: all WASAPI work runs on
/// a dedicated background (MTA) thread; the event fires on that thread.</summary>
public sealed class InputDeviceCapture : IDisposable
{
/// <summary>Fired on the capture thread every 20 ms: (interleaved s16 PCM, samplesPerChannel
/// = 960, channels).</summary>
public event Action<short[], int /*samplesPerChannel*/, int /*channels*/>? PcmFrameReady;
private const int SampleRate = 48000;
private const int FrameSamples = 960; // 20 ms
private readonly string? _deviceId; // null = system default capture endpoint
private readonly bool _loopback;
private IAudioClient? _audioClient;
private IAudioCaptureClient? _captureClient;
private AutoResetEvent? _bufferEvent;
private Thread? _captureThread;
private volatile bool _running;
private int _channels;
// Accumulator: assembles driver-callback-sized fragments into FrameSamples chunks.
private short[] _accumBuf = [];
private int _accumCount;
// Init-done signal: Set() by the capture thread after activation completes.
private readonly ManualResetEventSlim _initDone = new(false);
private bool _initOk;
private int _disposed;
public InputDeviceCapture(string? deviceId, bool loopback = false) { _deviceId = deviceId; _loopback = loopback; }
/// <summary>Starts capture. Blocks until WASAPI activation completes (typically &lt;100 ms).
/// Returns false if the device cannot be opened.</summary>
public bool Start()
{
if (_running) return false;
_running = true;
_captureThread = new Thread(CaptureThreadProc)
{
IsBackground = true,
Name = "AuxInputCapture",
};
_captureThread.Start();
bool ok = _initDone.Wait(5000) && _initOk;
if (!ok) _running = false;
return ok;
}
public void Stop()
{
_running = false;
_bufferEvent?.Set();
if (_captureThread is not null && !_captureThread.Join(5000)) throw new TimeoutException("Capture did not stop.");
}
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
Stop();
_bufferEvent?.Dispose();
_initDone.Dispose();
}
// ── Capture thread (MTA) ──────────────────────────────────────────────────
private void CaptureThreadProc()
{
try
{
_initOk = ActivateAndStart();
_initDone.Set();
if (_initOk && _running) CaptureLoop();
}
catch { _initOk = false; _initDone.Set(); }
finally
{
try { _audioClient?.Stop(); } catch { }
if (_captureClient != null) { Marshal.ReleaseComObject(_captureClient); _captureClient = null; }
if (_audioClient != null) { Marshal.ReleaseComObject(_audioClient); _audioClient = null; }
}
}
private bool ActivateAndStart()
{
if (!ActivateClient()) return false;
_bufferEvent = new AutoResetEvent(false);
if (_audioClient!.SetEventHandle(_bufferEvent.SafeWaitHandle.DangerousGetHandle()) < 0)
return false;
return _audioClient.Start() >= 0;
}
private bool ActivateClient()
{
IMMDeviceEnumerator? enumerator = null;
IMMDevice? device = null;
try
{
enumerator = (IMMDeviceEnumerator)Activator.CreateInstance(
Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
int hr = _deviceId is null
? enumerator.GetDefaultAudioEndpoint(_loopback ? 0 : 1, 0 /*eConsole*/, out device)
: enumerator.GetDevice(_deviceId, out device);
if (hr != 0 || device == null) return false;
var iidAudioClient = new Guid("1CB9AD4C-DBFA-4c32-B178-C2F568A703B2");
if (device.Activate(ref iidAudioClient, 0x17 /*CLSCTX_ALL*/, IntPtr.Zero,
out object acObj) != 0 || acObj is not IAudioClient ac)
return false;
_audioClient = ac;
return InitializeStream();
}
catch { return false; }
finally
{
if (device != null) Marshal.ReleaseComObject(device);
if (enumerator != null) Marshal.ReleaseComObject(enumerator);
}
}
private bool InitializeStream()
{
// Try s16 stereo first; fall back to s16 mono. AUTOCONVERTPCM lets the audio engine
// resample/convert the device's native format to our requested 48 kHz s16; EVENTCALLBACK
// drives the buffer-ready event. Shared mode (0), no LOOPBACK (this is a capture device).
// AUDCLNT_STREAMFLAGS_EVENTCALLBACK = 0x00040000
// AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM = 0x80000000
// AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY = 0x08000000
uint streamFlags = 0x00040000u | 0x80000000u | 0x08000000u | (_loopback ? 0x00020000u : 0u);
foreach (int ch in new[] { 2, 1 })
{
var fmt = new WaveFormatEx
{
wFormatTag = 1, // WAVE_FORMAT_PCM
nChannels = (ushort)ch,
nSamplesPerSec = SampleRate,
wBitsPerSample = 16,
nBlockAlign = (ushort)(ch * 2),
nAvgBytesPerSec = (uint)(SampleRate * ch * 2),
cbSize = 0,
};
IntPtr pFmt = Marshal.AllocHGlobal(Marshal.SizeOf<WaveFormatEx>());
try
{
Marshal.StructureToPtr(fmt, pFmt, false);
int hr = _audioClient!.Initialize(0 /*AUDCLNT_SHAREMODE_SHARED*/, streamFlags,
2_000_000 /*200 ms hns*/, 0, pFmt, IntPtr.Zero);
if (hr >= 0)
{
_channels = ch;
_accumBuf = new short[FrameSamples * ch];
_accumCount = 0;
var iidCapture = new Guid("C8ADBD64-E71E-48a0-A4DE-185C395CD317");
if (_audioClient.GetService(ref iidCapture, out object ccObj) != 0
|| ccObj is not IAudioCaptureClient cc)
return false;
_captureClient = cc;
return true;
}
if (ch == 1) return false;
}
finally { Marshal.FreeHGlobal(pFmt); }
}
return false;
}
// ── Capture loop ─────────────────────────────────────────────────────────
private void CaptureLoop()
{
while (_running)
{
_bufferEvent!.WaitOne(100);
if (!_running) break;
while (_running)
{
if (_captureClient!.GetNextPacketSize(out uint packetSize) < 0 || packetSize == 0)
break;
if (_captureClient.GetBuffer(out IntPtr dataPtr, out uint framesAvailable,
out uint flags, out _, out _) < 0)
break;
bool silent = (flags & 2) != 0; // AUDCLNT_BUFFERFLAGS_SILENT
if (framesAvailable > 0)
{
if (silent) AccumulateSilence((int)framesAvailable);
else AccumulatePcm(dataPtr, (int)framesAvailable);
}
_captureClient.ReleaseBuffer(framesAvailable);
}
}
}
private unsafe void AccumulatePcm(IntPtr data, int frames)
{
var src = (short*)data.ToPointer();
int total = frames * _channels;
int idx = 0;
while (idx < total)
{
int space = _accumBuf.Length - _accumCount;
int copy = Math.Min(total - idx, space);
fixed (short* dst = _accumBuf)
Buffer.MemoryCopy(src + idx, dst + _accumCount, copy * 2L, copy * 2L);
_accumCount += copy;
idx += copy;
if (_accumCount == _accumBuf.Length)
FlushFrame();
}
}
private void AccumulateSilence(int frames)
{
int total = frames * _channels;
int idx = 0;
while (idx < total)
{
int space = _accumBuf.Length - _accumCount;
int fill = Math.Min(total - idx, space);
Array.Clear(_accumBuf, _accumCount, fill);
_accumCount += fill;
idx += fill;
if (_accumCount == _accumBuf.Length)
FlushFrame();
}
}
private void FlushFrame()
{
PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels); // Borrowed until callback returns.
_accumCount = 0;
}
// ── COM declarations ───────────────────────────────────────────────────────
//
// Declared internal here so InputDeviceEnumerator can share them. These are standard MMDevice
// / WASAPI interfaces; a normal capture endpoint honours QueryInterface, so RCW marshalling is
// safe (unlike ProcessLoopbackCapture's process-loopback objects, which need raw vtable calls).
[ComImport, Guid("A95664D2-9614-4F35-A746-DE8DB63617E6"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDeviceEnumerator
{
[PreserveSig] int EnumAudioEndpoints(int dataFlow, int stateMask, out IntPtr devices);
[PreserveSig] int GetDefaultAudioEndpoint(int dataFlow, int role, out IMMDevice endpoint);
[PreserveSig] int GetDevice([MarshalAs(UnmanagedType.LPWStr)] string id, out IMMDevice device);
[PreserveSig] int RegisterEndpointNotificationCallback(IntPtr client);
[PreserveSig] int UnregisterEndpointNotificationCallback(IntPtr client);
}
[ComImport, Guid("0BD7A1BE-7A1A-44DB-8397-CC5392387B5E"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDeviceCollection
{
[PreserveSig] int GetCount(out int count);
[PreserveSig] int Item(int index, out IMMDevice device);
}
[ComImport, Guid("D666063F-1587-4E43-81F1-B948E807363F"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IMMDevice
{
[PreserveSig] int Activate(ref Guid iid, int clsCtx, IntPtr activationParams,
[MarshalAs(UnmanagedType.IUnknown)] out object ppInterface);
[PreserveSig] int OpenPropertyStore(int stgmAccess, out IntPtr propStore);
[PreserveSig] int GetId([MarshalAs(UnmanagedType.LPWStr)] out string id);
[PreserveSig] int GetState(out int state);
}
[ComImport, Guid("886D8EEB-8CF2-4446-8D02-CDBA1DBDCF99"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IPropertyStore
{
[PreserveSig] int GetCount(out int count);
[PreserveSig] int GetAt(int index, out PropertyKey key);
[PreserveSig] int GetValue(ref PropertyKey key, out PropVariant value);
[PreserveSig] int SetValue(ref PropertyKey key, ref PropVariant value);
[PreserveSig] int Commit();
}
[ComImport, Guid("1CB9AD4C-DBFA-4C32-B178-C2F568A703B2"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioClient
{
[PreserveSig] int Initialize(int shareMode, uint streamFlags, long hnsBufferDuration,
long hnsPeriodicity, IntPtr pFormat, IntPtr audioSessionGuid);
[PreserveSig] int GetBufferSize(out uint numBufferFrames);
[PreserveSig] int GetStreamLatency(out long latency);
[PreserveSig] int GetCurrentPadding(out uint numPaddingFrames);
[PreserveSig] int IsFormatSupported(int shareMode, IntPtr pFormat, out IntPtr closestMatch);
[PreserveSig] int GetMixFormat(out IntPtr deviceFormat);
[PreserveSig] int GetDevicePeriod(out long defaultDevicePeriod, out long minimumDevicePeriod);
[PreserveSig] int Start();
[PreserveSig] int Stop();
[PreserveSig] int Reset();
[PreserveSig] int SetEventHandle(IntPtr eventHandle);
[PreserveSig] int GetService(ref Guid riid,
[MarshalAs(UnmanagedType.IUnknown)] out object ppv);
}
[ComImport, Guid("C8ADBD64-E71E-48A0-A4DE-185C395CD317"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IAudioCaptureClient
{
[PreserveSig] int GetBuffer(out IntPtr data, out uint numFramesToRead, out uint flags,
out ulong devicePosition, out ulong qpcPosition);
[PreserveSig] int ReleaseBuffer(uint numFramesRead);
[PreserveSig] int GetNextPacketSize(out uint numFramesInNextPacket);
}
[StructLayout(LayoutKind.Sequential, Pack = 2)]
private struct WaveFormatEx
{
public ushort wFormatTag;
public ushort nChannels;
public uint nSamplesPerSec;
public uint nAvgBytesPerSec;
public ushort nBlockAlign;
public ushort wBitsPerSample;
public ushort cbSize;
}
[StructLayout(LayoutKind.Sequential)]
internal struct PropertyKey
{
public Guid fmtid;
public int pid;
}
// Minimal PROPVARIANT: we only ever read VT_LPWSTR (friendly name). x64 layout — the value
// union starts at offset 8 after vt(2)+reserved(6).
[StructLayout(LayoutKind.Explicit)]
internal struct PropVariant
{
[FieldOffset(0)] public ushort vt;
[FieldOffset(8)] public IntPtr pointerValue;
}
[DllImport("ole32.dll")]
internal static extern int PropVariantClear(ref PropVariant pvar);
}
@@ -0,0 +1,85 @@
using VoiceCat.Audio;
using VoiceCat.Windows;
namespace VoiceCat.App.Audio;
// Capture threads own their ring producers; the mix thread alone consumes them.
public sealed class ProcessAudioMixer : IDisposable
{
private sealed class Input
{
internal readonly PcmRing Ring = new(16384);
private readonly short[] stereo = new short[1920];
internal void Feed(short[] pcm, int channels)
{
if (channels == 2) { Ring.TryWrite(pcm); return; }
if (channels != 1 || pcm.Length > 960) return;
for (int i = 0; i < pcm.Length; i++) stereo[2 * i] = stereo[2 * i + 1] = pcm[i];
Ring.TryWrite(stereo.AsSpan(0, pcm.Length * 2));
}
}
private readonly List<ProcessLoopbackCapture> captures = [];
private Input[] inputs = [];
private Thread? thread;
private volatile bool running;
private VoiceCatClient? client;
private uint streamId;
public void Start(AppAudioScope scope, VoiceCatClient client, uint streamId)
{
if (running) return;
this.client = client; this.streamId = streamId;
var specs = ResolveCaptures(scope);
inputs = specs.Select(_ => new Input()).ToArray();
try
{
for (int i = 0; i < specs.Count; i++)
{
Input input = inputs[i]; var (pid, mode) = specs[i];
var capture = new ProcessLoopbackCapture(pid, mode);
capture.PcmFrameReady += (pcm, _, channels) => input.Feed(pcm, channels);
captures.Add(capture);
if (!capture.Start()) throw new InvalidOperationException("Process audio capture could not start.");
}
if (inputs.Length == 0) return;
running = true;
thread = new Thread(MixLoop) { IsBackground = true, Name = "ProcessAudioMixer" }; thread.Start();
}
catch { Stop(); throw; }
}
public void Stop()
{
running = false;
if (thread is not null && !thread.Join(5000)) throw new TimeoutException("Process audio mixer did not stop.");
foreach (var capture in captures) capture.Dispose();
captures.Clear(); inputs = []; thread = null;
}
public void Dispose() => Stop();
private void MixLoop()
{
var frame = new short[1920]; var mix = new short[1920]; var sums = new int[1920];
long deadline = System.Diagnostics.Stopwatch.GetTimestamp();
while (running)
{
sums.AsSpan().Clear();
foreach (Input input in inputs)
{
while (input.Ring.Count > 11520) input.Ring.Read(frame);
frame.AsSpan().Clear(); input.Ring.Read(frame);
for (int i = 0; i < frame.Length; i++) sums[i] += frame[i];
}
for (int i = 0; i < mix.Length; i++) mix[i] = (short)Math.Clamp(sums[i], short.MinValue, short.MaxValue);
client?.StreamFeedPcm(streamId, mix, 960, 2);
deadline += System.Diagnostics.Stopwatch.Frequency / 50;
double wait = (deadline - System.Diagnostics.Stopwatch.GetTimestamp()) * 1000.0 / System.Diagnostics.Stopwatch.Frequency;
if (wait > 0) Thread.Sleep((int)Math.Ceiling(wait));
else if (wait < -100) deadline = System.Diagnostics.Stopwatch.GetTimestamp();
}
}
private static List<(int pid, ProcessLoopbackCapture.Mode mode)> ResolveCaptures(AppAudioScope scope) => scope switch
{
OnlyApps o => o.Pids.Select(p => (p, ProcessLoopbackCapture.Mode.Include)).ToList(),
AllExceptApps a when a.Pids.Count > 0 => [(a.Pids[0], ProcessLoopbackCapture.Mode.Exclude)],
EntireDesktop { ExcludeSelf: true } => [(Environment.ProcessId, ProcessLoopbackCapture.Mode.Exclude)],
_ => [],
};
}
@@ -0,0 +1,438 @@
using System.Runtime.InteropServices;
// Process loopback requires MTA activation; blocking activation from the WinForms STA
// deadlocks COM completion. The returned interfaces also reject RCW QueryInterface, so audio
// calls use explicitly owned raw pointers and vtable dispatch.
namespace VoiceCat.App.Audio;
public sealed class ProcessLoopbackCapture : IDisposable
{
public enum Mode { Include, Exclude }
// Fired on the capture thread every 20 ms (960 samples/channel @ 48 kHz, interleaved s16).
public event Action<short[], int /*samplesPerChannel*/, int /*channels*/>? PcmFrameReady;
private const int SampleRate = 48000;
private const int FrameSamples = 960; // 20 ms
private const string LoopbackDevicePath = "VAD\\Process_Loopback";
private readonly int _pid;
private readonly Mode _mode;
// Raw COM pointers — managed via explicit AddRef/Release, no RCW wrapping.
private IntPtr _audioClientPtr; // IAudioClient*
private IntPtr _captureClientPtr; // IAudioCaptureClient*
private AutoResetEvent? _bufferEvent;
private Thread? _captureThread;
private volatile bool _running;
private int _channels;
// Accumulator: assembles driver-callback-sized fragments into FrameSamples chunks.
private short[] _accumBuf = [];
private int _accumCount;
// Init-done signal: Set() by the capture thread after ActivateClient() completes.
private readonly ManualResetEventSlim _initDone = new(false);
private bool _initOk;
public ProcessLoopbackCapture(int pid, Mode mode)
{
_pid = pid;
_mode = mode;
}
/// <summary>Starts capture. Blocks until WASAPI activation completes (typically &lt;100 ms).
/// Returns false if the process cannot be captured.</summary>
public bool Start()
{
if (_running) return false;
_running = true;
_captureThread = new Thread(CaptureThreadProc)
{
IsBackground = true,
Name = $"ProcLoopback:{_pid}",
};
_captureThread.Start();
bool ok = _initDone.Wait(5000) && _initOk;
if (!ok) _running = false;
return ok;
}
public void Stop()
{
_running = false;
_bufferEvent?.Set();
if (_captureThread is not null && !_captureThread.Join(5000))
throw new TimeoutException("Process capture did not stop.");
}
private int _disposed;
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0) return;
Stop();
_bufferEvent?.Dispose();
_initDone.Dispose();
}
// ── Capture thread (MTA) ──────────────────────────────────────────────────
private void CaptureThreadProc()
{
try
{
_initOk = ActivateAndStart();
_initDone.Set();
if (_initOk && _running) CaptureLoop();
}
catch
{
_initOk = false;
_initDone.Set();
}
finally
{
if (_audioClientPtr != IntPtr.Zero) AC_Stop(_audioClientPtr);
ComRelease(ref _captureClientPtr);
ComRelease(ref _audioClientPtr);
}
}
private bool ActivateAndStart()
{
if (!ActivateClient()) return false;
_bufferEvent = new AutoResetEvent(false);
if (AC_SetEventHandle(_audioClientPtr, _bufferEvent.SafeWaitHandle.DangerousGetHandle()) < 0)
return false;
return AC_Start(_audioClientPtr) >= 0;
}
// ── Activation ───────────────────────────────────────────────────────────
private unsafe bool ActivateClient()
{
var activationParams = new AudioClientActivationParams
{
ActivationType = 1, // AUDCLNT_ACTIVATIONTYPE_PROCESS_LOOPBACK
TargetProcessId = (uint)_pid,
ProcessLoopbackMode = _mode == Mode.Include ? 0u : 1u,
};
var handler = new ActivationCompletionHandler();
var audioClientIid = new Guid("1CB9AD4C-DBFA-4c32-B178-C2F568A703B2");
IntPtr opPtr;
{
AudioClientActivationParams* pParams = &activationParams;
// PROPVARIANT (VT_BLOB) x64: vt(2)+res(6)+cbSize(4)+pad(4)+pBlobData(8) = 24 B.
var pv = stackalloc byte[24];
*(ushort*)(pv + 0) = 65;
*(uint*) (pv + 8) = (uint)sizeof(AudioClientActivationParams);
*(nint*) (pv + 16) = (nint)pParams;
int hr = ActivateAudioInterfaceAsync(
LoopbackDevicePath, ref audioClientIid,
(IntPtr)pv, handler, out opPtr);
if (hr < 0) return false;
}
if (!handler.CompletionEvent.Wait(3000))
{
ComRelease(ref opPtr);
return false;
}
// Call GetActivateResult via vtable (slot 3) — avoids QI on the async-op object.
if (!Vtable_GetActivateResult(handler.OperationPtr, out int activateHr, out IntPtr activatedPtr))
{
handler.ReleaseOp();
ComRelease(ref opPtr);
return false;
}
handler.ReleaseOp();
ComRelease(ref opPtr);
if (activateHr < 0 || activatedPtr == IntPtr.Zero) return false;
// Store the raw IAudioClient* — do NOT create an RCW; use vtable dispatch instead.
_audioClientPtr = activatedPtr;
// activatedPtr already has ref count from GetActivateResult; don't double-release.
return InitializeStream();
}
// IActivateAudioInterfaceAsyncOperation vtable slot 3: GetActivateResult(HRESULT*, IUnknown**)
private static unsafe bool Vtable_GetActivateResult(IntPtr op,
out int activateHr, out IntPtr activatedPtr)
{
activateHr = unchecked((int)0x80004005);
activatedPtr = IntPtr.Zero;
if (op == IntPtr.Zero) return false;
void** vtable = *(void***)op.ToPointer();
var fn = (delegate* unmanaged[Stdcall]<IntPtr, int*, IntPtr*, int>)vtable[3];
fixed (int* pHr = &activateHr)
fixed (IntPtr* pPtr = &activatedPtr)
return fn(op, pHr, pPtr) >= 0;
}
private unsafe bool InitializeStream()
{
// Try s16 stereo first; fall back to s16 mono.
foreach (int ch in new[] { 2, 1 })
{
var fmt = new WaveFormatEx
{
wFormatTag = 1, // WAVE_FORMAT_PCM
nChannels = (ushort)ch,
nSamplesPerSec = SampleRate,
wBitsPerSample = 16,
nBlockAlign = (ushort)(ch * 2),
nAvgBytesPerSec = (uint)(SampleRate * ch * 2),
cbSize = 0,
};
// Process-loopback requires LOOPBACK (deliver rendered audio) + EVENTCALLBACK +
// AUTOCONVERTPCM (resample the app's native format to our requested s16/48k). Without
// LOOPBACK every buffer comes back AUDCLNT_BUFFERFLAGS_SILENT; without AUTOCONVERTPCM
// the requested format is rejected. Matches the MS ApplicationLoopback sample.
// AUDCLNT_SHAREMODE_SHARED = 0
// AUDCLNT_STREAMFLAGS_LOOPBACK = 0x00020000
// AUDCLNT_STREAMFLAGS_EVENTCALLBACK = 0x00040000
// AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM = 0x80000000
const uint streamFlags = 0x00020000u | 0x00040000u | 0x80000000u;
int hr = AC_Initialize(_audioClientPtr, 0, streamFlags,
2_000_000 /*200ms hns*/, 0, &fmt, null);
if (hr >= 0)
{
_channels = ch;
_accumBuf = new short[FrameSamples * ch];
_accumCount = 0;
break;
}
if (ch == 1) return false;
}
var captureIid = new Guid("C8ADBD64-E71E-48a0-A4DE-185C395CD317");
int getHr = AC_GetService(_audioClientPtr, ref captureIid, out _captureClientPtr);
return getHr >= 0 && _captureClientPtr != IntPtr.Zero;
}
// ── Capture loop ─────────────────────────────────────────────────────────
private void CaptureLoop()
{
while (_running)
{
_bufferEvent!.WaitOne(100);
if (!_running) break;
while (_running)
{
int hr = CC_GetNextPacketSize(_captureClientPtr, out uint packetSize);
if (hr < 0 || packetSize == 0) break;
hr = CC_GetBuffer(_captureClientPtr, out IntPtr dataPtr, out uint framesAvailable,
out uint flags);
if (hr < 0) break;
bool silent = (flags & 2) != 0; // AUDCLNT_BUFFERFLAGS_SILENT
if (framesAvailable > 0)
{
if (silent) AccumulateSilence((int)framesAvailable);
else AccumulatePcm(dataPtr, (int)framesAvailable);
}
CC_ReleaseBuffer(_captureClientPtr, framesAvailable);
}
}
}
private unsafe void AccumulatePcm(IntPtr data, int frames)
{
var src = (short*)data.ToPointer();
int total = frames * _channels;
int idx = 0;
while (idx < total)
{
int space = _accumBuf.Length - _accumCount;
int copy = Math.Min(total - idx, space);
fixed (short* dst = _accumBuf)
Buffer.MemoryCopy(src + idx, dst + _accumCount, copy * 2L, copy * 2L);
_accumCount += copy;
idx += copy;
if (_accumCount == _accumBuf.Length)
FlushFrame();
}
}
private void AccumulateSilence(int frames)
{
int total = frames * _channels;
int idx = 0;
while (idx < total)
{
int space = _accumBuf.Length - _accumCount;
int fill = Math.Min(total - idx, space);
Array.Clear(_accumBuf, _accumCount, fill);
_accumCount += fill;
idx += fill;
if (_accumCount == _accumBuf.Length)
FlushFrame();
}
}
private void FlushFrame()
{
// The callback borrows this buffer until it returns. This capture owner cannot
// overwrite it concurrently, which avoids one allocation every 20 ms.
PcmFrameReady?.Invoke(_accumBuf, FrameSamples, _channels);
_accumCount = 0;
}
// ── IAudioClient vtable helpers (raw dispatch, no RCW / no QI) ───────────
//
// Vtable layout (IUnknown base: 0=QI 1=AddRef 2=Release; then IAudioClient methods):
// 3=Initialize 4=GetBufferSize 5=GetStreamLatency 6=GetCurrentPadding
// 7=IsFormatSupported 8=GetMixFormat 9=GetDevicePeriod
// 10=Start 11=Stop 12=Reset 13=SetEventHandle 14=GetService
private static unsafe int AC_Initialize(IntPtr ac, int shareMode, uint streamFlags,
long hnsBufferDuration, long hnsPeriodicity, WaveFormatEx* pFormat, Guid* pSession)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, int, uint, long, long, WaveFormatEx*, Guid*, int>)
(*(void***)ac)[3];
return fn(ac, shareMode, streamFlags, hnsBufferDuration, hnsPeriodicity, pFormat, pSession);
}
private static unsafe int AC_Start(IntPtr ac)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, int>)(*(void***)ac)[10];
return fn(ac);
}
private static unsafe int AC_Stop(IntPtr ac)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, int>)(*(void***)ac)[11];
return fn(ac);
}
private static unsafe int AC_SetEventHandle(IntPtr ac, IntPtr eventHandle)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, IntPtr, int>)(*(void***)ac)[13];
return fn(ac, eventHandle);
}
private static unsafe int AC_GetService(IntPtr ac, ref Guid riid, out IntPtr ppv)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, Guid*, IntPtr*, int>)(*(void***)ac)[14];
fixed (Guid* pIid = &riid)
fixed (IntPtr* pPpv = &ppv)
return fn(ac, pIid, pPpv);
}
// ── IAudioCaptureClient vtable helpers ────────────────────────────────────
//
// Vtable (IUnknown: 0-2; then): 3=GetBuffer 4=ReleaseBuffer 5=GetNextPacketSize
private static unsafe int CC_GetBuffer(IntPtr cc, out IntPtr ppData,
out uint pNumFrames, out uint pdwFlags)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, IntPtr*, uint*, uint*, ulong*, ulong*, int>)
(*(void***)cc)[3];
ulong devPos = 0, qpcPos = 0;
fixed (IntPtr* p0 = &ppData)
fixed (uint* p1 = &pNumFrames)
fixed (uint* p2 = &pdwFlags)
return fn(cc, p0, p1, p2, &devPos, &qpcPos);
}
private static unsafe int CC_ReleaseBuffer(IntPtr cc, uint numFrames)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, uint, int>)(*(void***)cc)[4];
return fn(cc, numFrames);
}
private static unsafe int CC_GetNextPacketSize(IntPtr cc, out uint pNumFrames)
{
var fn = (delegate* unmanaged[Stdcall]<IntPtr, uint*, int>)(*(void***)cc)[5];
fixed (uint* p = &pNumFrames)
return fn(cc, p);
}
// ── COM utilities ─────────────────────────────────────────────────────────
private static unsafe void ComRelease(ref IntPtr ptr)
{
if (ptr == IntPtr.Zero) return;
var fn = (delegate* unmanaged[Stdcall]<IntPtr, uint>)(*(void***)ptr)[2]; // IUnknown::Release
fn(ptr);
ptr = IntPtr.Zero;
}
// ── P/Invoke & structs ────────────────────────────────────────────────────
[DllImport("Mmdevapi.dll", CharSet = CharSet.Unicode)]
private static extern int ActivateAudioInterfaceAsync(
string deviceInterfacePath,
ref Guid riid,
IntPtr activationParams,
[MarshalAs(UnmanagedType.Interface)] IActivateAudioInterfaceCompletionHandler completionHandler,
out IntPtr activationOperation);
[StructLayout(LayoutKind.Sequential)]
private struct AudioClientActivationParams
{
public int ActivationType; // AUDCLNT_ACTIVATIONTYPE_PROCESS_LOOPBACK = 1
public uint TargetProcessId;
public uint ProcessLoopbackMode; // INCLUDE=0, EXCLUDE=1
}
[StructLayout(LayoutKind.Sequential, Pack = 2)]
private struct WaveFormatEx
{
public ushort wFormatTag;
public ushort nChannels;
public uint nSamplesPerSec;
public uint nAvgBytesPerSec;
public ushort nBlockAlign;
public ushort wBitsPerSample;
public ushort cbSize;
}
// ── Completion handler CCW ────────────────────────────────────────────────
//
// Only this object still uses .NET COM interop (as a CCW). The activateOperation
// parameter is IntPtr to avoid QI on the incoming async-op pointer.
[ComVisible(true), ClassInterface(ClassInterfaceType.None)]
private sealed class ActivationCompletionHandler : IActivateAudioInterfaceCompletionHandler
{
public readonly ManualResetEventSlim CompletionEvent = new(false);
public IntPtr OperationPtr { get; private set; }
public void ActivateCompleted(IntPtr activateOperation)
{
OperationPtr = activateOperation;
if (OperationPtr != IntPtr.Zero) Marshal.AddRef(OperationPtr);
CompletionEvent.Set();
}
public void ReleaseOp()
{
if (OperationPtr == IntPtr.Zero) return;
Marshal.Release(OperationPtr);
OperationPtr = IntPtr.Zero;
}
}
[ComImport, Guid("41D949AB-9862-444A-80F6-C261334DA5EB"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IActivateAudioInterfaceCompletionHandler
{
void ActivateCompleted(IntPtr activateOperation);
}
}
@@ -0,0 +1,111 @@
using System.Runtime.InteropServices;
using VoiceCat.Audio;
using static VoiceCat.App.Audio.InputDeviceCapture;
namespace VoiceCat.App.Audio;
public sealed class WasapiAudioBackend : IAudioDeviceBackend
{
public IReadOnlyList<AudioDeviceInfo> Enumerate(bool input) => InputDeviceEnumerator.List(input).Select(d => new AudioDeviceInfo(d.Id, d.Name, d.IsDefault)).ToArray();
public IAudioCapture OpenCapture(string? deviceId, bool loopback, CapturePcmHandler handler)
{
var capture = new InputDeviceCapture(NormalizeDeviceId(deviceId), loopback);
capture.PcmFrameReady += (pcm, _, channels) => handler(pcm, channels);
if (!capture.Start()) { capture.Dispose(); throw new InvalidOperationException("WASAPI capture could not start."); }
return new Capture(capture);
}
public IAudioPlayback OpenPlayback(string? deviceId = null) => new Playback(deviceId);
// The old miniaudio ABI persisted its raw device union as hex. Its Windows member
// is a null-terminated UTF-16 WASAPI endpoint id; preserve saved selections.
internal static string? NormalizeDeviceId(string? id)
{
if (string.IsNullOrEmpty(id)) return null;
if (id.StartsWith('{')) return id;
try
{
byte[] bytes = Convert.FromHexString(id);
string decoded = System.Text.Encoding.Unicode.GetString(bytes).Split('\0')[0];
return decoded.StartsWith('{') ? decoded : null;
}
catch (FormatException) { return id; }
}
private sealed class Capture(InputDeviceCapture capture) : IAudioCapture { public void Dispose() => capture.Dispose(); }
private sealed class Playback : IAudioPlayback
{
private readonly string? deviceId;
private readonly AdaptivePcmBuffer pcm = new(2);
private readonly ManualResetEventSlim initialized = new(false);
private readonly Thread thread;
private volatile bool running = true;
private bool ready;
private int disposed;
internal Playback(string? deviceId)
{
this.deviceId = deviceId;
thread = new Thread(Work) { IsBackground = true, Name = "VoiceCat WASAPI playback" };
thread.Start();
if (!initialized.Wait(5000) || !ready) { Dispose(); throw new InvalidOperationException("WASAPI playback could not start."); }
}
public int BufferMilliseconds { get => pcm.BufferMilliseconds; set => pcm.BufferMilliseconds = value; }
public void Write(ReadOnlySpan<short> stereoPcm) => pcm.TryWrite(stereoPcm);
private unsafe void Work()
{
IMMDeviceEnumerator? enumerator = null; IMMDevice? device = null; IAudioClient? client = null; IAudioRenderClient? render = null;
using var bufferReady = new AutoResetEvent(false);
try
{
enumerator = (IMMDeviceEnumerator)Activator.CreateInstance(Type.GetTypeFromCLSID(new Guid("BCDE0395-E52F-467C-8E3D-C4579291692E"))!)!;
int result = deviceId is null ? enumerator.GetDefaultAudioEndpoint(0, 0, out device) : enumerator.GetDevice(deviceId, out device);
if (result < 0 || device is null) return;
Guid iid = typeof(IAudioClient).GUID;
if (device.Activate(ref iid, 0x17, 0, out object audio) < 0) return;
client = (IAudioClient)audio;
WaveFormat format = new() { Format = 1, Channels = 2, Samples = 48000, Bytes = 192000, Align = 4, Bits = 16 };
if (client.Initialize(0, 0x00040000u | 0x80000000u | 0x08000000u, 600000, 0, (nint)(&format), 0) < 0) return;
if (client.GetBufferSize(out uint capacity) < 0 || client.SetEventHandle(bufferReady.SafeWaitHandle.DangerousGetHandle()) < 0) return;
iid = typeof(IAudioRenderClient).GUID;
if (client.GetService(ref iid, out object output) < 0) return;
render = (IAudioRenderClient)output;
if (client.Start() < 0) return;
ready = true; initialized.Set();
while (running)
{
bufferReady.WaitOne(20); // Scheduling wait is outside the buffer-fill cycle.
if (!running || client.GetCurrentPadding(out uint padding) < 0) break;
uint frames = capacity - Math.Min(capacity, padding);
if (frames == 0) continue;
if (render.GetBuffer(frames, out nint buffer) < 0) break;
var destination = new Span<short>((void*)buffer, checked((int)frames * 2));
int count = pcm.Read(destination); destination[count..].Clear();
if (render.ReleaseBuffer(frames, 0) < 0) break;
}
}
catch { ready = false; }
finally
{
initialized.Set();
try { client?.Stop(); } catch { }
if (render is not null) Marshal.ReleaseComObject(render);
if (client is not null) Marshal.ReleaseComObject(client);
if (device is not null) Marshal.ReleaseComObject(device);
if (enumerator is not null) Marshal.ReleaseComObject(enumerator);
}
}
public void Dispose()
{
if (Interlocked.Exchange(ref disposed, 1) != 0) return;
running = false;
if (!thread.Join(5000)) throw new TimeoutException("WASAPI playback did not stop.");
initialized.Dispose();
}
}
[StructLayout(LayoutKind.Sequential, Pack = 2)]
private struct WaveFormat { internal ushort Format, Channels; internal uint Samples, Bytes; internal ushort Align, Bits, Extra; }
[ComImport, Guid("F294ACFC-3146-4483-A7BF-ADDCA7C260E2"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IAudioRenderClient
{
[PreserveSig] int GetBuffer(uint frames, out nint data);
[PreserveSig] int ReleaseBuffer(uint frames, uint flags);
}
}
@@ -0,0 +1,200 @@
using VoiceCat.Windows;
namespace VoiceCat.App.Forms;
/// <summary>
/// Admin account management: list, create, reset password, and delete server accounts.
/// </summary>
public sealed class AccountsDialog : Form
{
private readonly VoiceCatClient _client;
private readonly ListView _lvAccounts;
private readonly Button _btnRefresh;
private readonly Button _btnAdd;
private readonly Button _btnResetPassword;
private readonly Button _btnDelete;
public AccountsDialog(VoiceCatClient client)
{
_client = client;
Text = "Server accounts";
FormBorderStyle = FormBorderStyle.Sizable;
StartPosition = FormStartPosition.CenterParent;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(560, 360);
MinimumSize = new Size(420, 260);
_lvAccounts = new ListView
{
View = View.Details,
FullRowSelect = true,
GridLines = true,
Dock = DockStyle.Fill,
TabIndex = 0,
};
_lvAccounts.Columns.Add("Username", 160);
_lvAccounts.Columns.Add("Admin", 50);
_lvAccounts.Columns.Add("Created", 130);
_lvAccounts.Columns.Add("Last login", 130);
_lvAccounts.SelectedIndexChanged += (_, _) => UpdateButtons();
var pnlButtons = new Panel
{
Dock = DockStyle.Bottom,
Height = 44,
};
_btnRefresh = new Button
{
Text = "&Refresh",
Location = new Point(12, 9),
Size = new Size(75, 27),
TabIndex = 1,
};
_btnRefresh.Click += (_, _) => RefreshList();
_btnAdd = new Button
{
Text = "&Add...",
Location = new Point(100, 9),
Size = new Size(75, 27),
TabIndex = 2,
};
_btnAdd.Click += (_, _) => AddAccount();
_btnResetPassword = new Button
{
Text = "&Reset password...",
Location = new Point(188, 9),
Size = new Size(120, 27),
TabIndex = 3,
Enabled = false,
};
_btnResetPassword.Click += (_, _) => ResetPassword();
_btnDelete = new Button
{
Text = "&Delete",
Location = new Point(320, 9),
Size = new Size(75, 27),
TabIndex = 4,
Enabled = false,
};
_btnDelete.Click += (_, _) => DeleteAccount();
pnlButtons.Controls.AddRange([_btnRefresh, _btnAdd, _btnResetPassword, _btnDelete]);
Controls.Add(_lvAccounts);
Controls.Add(pnlButtons);
Load += (_, _) => RefreshList();
}
private void UpdateButtons()
{
bool selected = _lvAccounts.SelectedItems.Count > 0;
_btnResetPassword.Enabled = selected;
_btnDelete.Enabled = selected;
}
private void RefreshList()
{
_client.RequestAccountList();
// The event will come back asynchronously; poll briefly for the list.
var deadline = DateTime.UtcNow.AddSeconds(2);
List<AccountInfo>? accounts = null;
while (DateTime.UtcNow < deadline)
{
_client.PumpEvents();
accounts = _client.ListAccounts();
if (accounts.Count > 0) break;
Thread.Sleep(30);
}
accounts ??= _client.ListAccounts();
_lvAccounts.BeginUpdate();
_lvAccounts.Items.Clear();
foreach (var a in accounts.OrderBy(a => a.Username))
{
var item = new ListViewItem(a.Username);
item.SubItems.Add(a.IsAdmin ? "Yes" : "No");
item.SubItems.Add(FormatDate(a.CreatedAtUnixMs));
item.SubItems.Add(FormatDate(a.LastLoginUnixMs));
_lvAccounts.Items.Add(item);
}
_lvAccounts.EndUpdate();
UpdateButtons();
}
private static string FormatDate(ulong ms)
{
if (ms == 0) return "—";
try
{
return DateTimeOffset.FromUnixTimeMilliseconds((long)ms).LocalDateTime.ToString("g");
}
catch
{
return "—";
}
}
private void AddAccount()
{
using var userDlg = new InputDialog("Add account", "&Username:");
if (userDlg.ShowDialog(this) != DialogResult.OK || string.IsNullOrWhiteSpace(userDlg.TextValue))
return;
using var pwDlg = new PasswordPromptDialog($"Password for {userDlg.TextValue}:");
if (pwDlg.ShowDialog(this) != DialogResult.OK || string.IsNullOrEmpty(pwDlg.Password))
return;
var r = _client.CreateAccount(userDlg.TextValue, pwDlg.Password);
if (r != VcResult.Ok)
{
MessageBox.Show(this, $"Create account failed: {r}", "VoiceCat",
MessageBoxButtons.OK, MessageBoxIcon.Warning);
return;
}
RefreshList();
}
private void ResetPassword()
{
if (_lvAccounts.SelectedItems.Count == 0) return;
string username = _lvAccounts.SelectedItems[0].Text;
using var dlg = new PasswordPromptDialog($"New password for {username}:");
if (dlg.ShowDialog(this) != DialogResult.OK || string.IsNullOrEmpty(dlg.Password))
return;
var r = _client.ResetPassword(username, dlg.Password);
if (r != VcResult.Ok)
{
MessageBox.Show(this, $"Reset password failed: {r}", "VoiceCat",
MessageBoxButtons.OK, MessageBoxIcon.Warning);
return;
}
RefreshList();
}
private void DeleteAccount()
{
if (_lvAccounts.SelectedItems.Count == 0) return;
string username = _lvAccounts.SelectedItems[0].Text;
var confirm = MessageBox.Show(this, $"Delete account '{username}'?",
"VoiceCat", MessageBoxButtons.YesNo, MessageBoxIcon.Question);
if (confirm != DialogResult.Yes) return;
var r = _client.DeleteAccount(username);
if (r != VcResult.Ok)
{
MessageBox.Show(this, $"Delete account failed: {r}", "VoiceCat",
MessageBoxButtons.OK, MessageBoxIcon.Warning);
return;
}
RefreshList();
}
}
@@ -0,0 +1,187 @@
namespace VoiceCat.App.Forms;
partial class AddServerDialog
{
private System.ComponentModel.IContainer components = null!;
private Label lblDisplayName = null!;
private TextBox txtDisplayName = null!;
private Label lblHost = null!;
private TextBox txtHost = null!;
private Label lblPort = null!;
private NumericUpDown numPort = null!;
private GroupBox grpAuth = null!;
private RadioButton radioGuest = null!;
private RadioButton radioPassword = null!;
private Label lblNickname = null!;
private TextBox txtNickname = null!;
private Label lblUsername = null!;
private TextBox txtUsername = null!;
private Label lblPassword = null!;
private TextBox txtPassword = null!;
private CheckBox chkRememberPassword = null!;
private Button btnOk = null!;
private Button btnCancel = null!;
protected override void Dispose(bool disposing)
{
if (disposing) components?.Dispose();
base.Dispose(disposing);
}
private void InitializeComponent()
{
components = new System.ComponentModel.Container();
lblDisplayName = new Label();
txtDisplayName = new TextBox();
lblHost = new Label();
txtHost = new TextBox();
lblPort = new Label();
numPort = new NumericUpDown();
grpAuth = new GroupBox();
radioGuest = new RadioButton();
radioPassword = new RadioButton();
lblNickname = new Label();
txtNickname = new TextBox();
lblUsername = new Label();
txtUsername = new TextBox();
lblPassword = new Label();
txtPassword = new TextBox();
chkRememberPassword = new CheckBox();
btnOk = new Button();
btnCancel = new Button();
int y = 12;
const int rowH = 30;
lblDisplayName.Text = "&Display name:";
lblDisplayName.AutoSize = true;
lblDisplayName.Location = new Point(12, y + 3);
lblDisplayName.TabIndex = 0;
txtDisplayName.AccessibleName = "Display name";
txtDisplayName.Location = new Point(140, y);
txtDisplayName.Size = new Size(280, 23);
txtDisplayName.TabIndex = 1;
y += rowH;
lblHost.Text = "&Host:";
lblHost.AutoSize = true;
lblHost.Location = new Point(12, y + 3);
lblHost.TabIndex = 2;
txtHost.AccessibleName = "Host";
txtHost.Location = new Point(140, y);
txtHost.Size = new Size(200, 23);
txtHost.TabIndex = 3;
txtHost.PlaceholderText = "127.0.0.1";
y += rowH;
lblPort.Text = "&Port:";
lblPort.AutoSize = true;
lblPort.Location = new Point(12, y + 3);
lblPort.TabIndex = 4;
numPort.AccessibleName = "Port";
numPort.Location = new Point(140, y);
numPort.Size = new Size(100, 23);
numPort.Minimum = 1;
numPort.Maximum = 65535;
numPort.Value = 8384;
numPort.TabIndex = 5;
y += rowH;
grpAuth.Text = "Authentication";
grpAuth.Location = new Point(12, y);
grpAuth.Size = new Size(408, 140);
grpAuth.TabIndex = 6;
radioGuest.Text = "Connect as &guest";
radioGuest.AutoSize = true;
radioGuest.Location = new Point(12, 24);
radioGuest.Checked = true;
radioGuest.TabIndex = 0;
lblNickname.Text = "&Nickname:";
lblNickname.AutoSize = true;
lblNickname.Location = new Point(30, 50);
lblNickname.TabIndex = 1;
txtNickname.AccessibleName = "Guest nickname";
txtNickname.Location = new Point(140, 47);
txtNickname.Size = new Size(240, 23);
txtNickname.TabIndex = 2;
radioPassword.Text = "Use a &saved account";
radioPassword.AutoSize = true;
radioPassword.Location = new Point(12, 76);
radioPassword.TabIndex = 3;
lblUsername.Text = "&Username:";
lblUsername.AutoSize = true;
lblUsername.Location = new Point(30, 102);
lblUsername.TabIndex = 4;
txtUsername.AccessibleName = "Username";
txtUsername.Location = new Point(140, 99);
txtUsername.Size = new Size(240, 23);
txtUsername.TabIndex = 5;
lblPassword.Text = "Pass&word:";
lblPassword.AutoSize = true;
lblPassword.Location = new Point(30, 132);
lblPassword.TabIndex = 6;
txtPassword.AccessibleName = "Password";
txtPassword.Location = new Point(140, 129);
txtPassword.Size = new Size(240, 23);
txtPassword.UseSystemPasswordChar = true;
txtPassword.TabIndex = 7;
grpAuth.Controls.Add(radioGuest);
grpAuth.Controls.Add(lblNickname);
grpAuth.Controls.Add(txtNickname);
grpAuth.Controls.Add(radioPassword);
grpAuth.Controls.Add(lblUsername);
grpAuth.Controls.Add(txtUsername);
grpAuth.Controls.Add(lblPassword);
grpAuth.Controls.Add(txtPassword);
y += 148;
chkRememberPassword.Text = "&Remember my password on this computer";
chkRememberPassword.AutoSize = true;
chkRememberPassword.AccessibleDescription =
"Stores the password protected by Windows Data Protection (DPAPI), decryptable " +
"only by your Windows account on this machine. Leave unchecked to be prompted " +
"for the password every time.";
chkRememberPassword.Location = new Point(12, y);
chkRememberPassword.TabIndex = 7;
y += rowH;
btnOk.Text = "&OK";
btnOk.DialogResult = DialogResult.OK;
btnOk.Location = new Point(264, y);
btnOk.Size = new Size(75, 27);
btnOk.TabIndex = 8;
btnCancel.Text = "&Cancel";
btnCancel.DialogResult = DialogResult.Cancel;
btnCancel.Location = new Point(345, y);
btnCancel.Size = new Size(75, 27);
btnCancel.TabIndex = 9;
y += rowH + 12;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(432, y);
Controls.Add(lblDisplayName);
Controls.Add(txtDisplayName);
Controls.Add(lblHost);
Controls.Add(txtHost);
Controls.Add(lblPort);
Controls.Add(numPort);
Controls.Add(grpAuth);
Controls.Add(chkRememberPassword);
Controls.Add(btnOk);
Controls.Add(btnCancel);
AcceptButton = btnOk;
CancelButton = btnCancel;
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
Text = "Add server";
}
}
@@ -0,0 +1,80 @@
using VoiceCat.App.Models;
namespace VoiceCat.App.Forms;
/// <summary>Add or edit a saved server entry. DialogResult.OK -> read Result.</summary>
public partial class AddServerDialog : Form
{
private readonly SavedServer _editing;
private bool _passwordChanged;
public SavedServer? Result { get; private set; }
public AddServerDialog(SavedServer? existing = null)
{
InitializeComponent();
_editing = existing ?? new SavedServer();
txtDisplayName.Text = _editing.DisplayName;
txtHost.Text = _editing.Host;
numPort.Value = _editing.Port == 0 ? 8384 : _editing.Port;
txtNickname.Text = _editing.LastNickname;
radioGuest.Checked = _editing.AuthMode == AuthMode.Guest;
radioPassword.Checked = _editing.AuthMode == AuthMode.Password;
txtUsername.Text = _editing.SavedUsername ?? "";
chkRememberPassword.Checked = _editing.ProtectedPasswordBase64 is not null;
if (_editing.ProtectedPasswordBase64 is not null)
txtPassword.Text = "********"; // placeholder — never decrypt-and-show; re-typing replaces it
UpdateAuthFieldsEnabled();
radioGuest.CheckedChanged += (_, _) => UpdateAuthFieldsEnabled();
radioPassword.CheckedChanged += (_, _) => UpdateAuthFieldsEnabled();
txtPassword.TextChanged += (_, _) => _passwordChanged = true;
btnOk.Click += BtnOk_Click;
}
private void UpdateAuthFieldsEnabled()
{
bool password = radioPassword.Checked;
txtNickname.Enabled = !password;
txtUsername.Enabled = password;
txtPassword.Enabled = password;
chkRememberPassword.Enabled = password;
}
private void BtnOk_Click(object? sender, EventArgs e)
{
if (string.IsNullOrWhiteSpace(txtHost.Text))
{
MessageBox.Show(this, "Host is required.", "VoiceCat", MessageBoxButtons.OK, MessageBoxIcon.Warning);
DialogResult = DialogResult.None;
return;
}
_editing.DisplayName = txtDisplayName.Text.Trim();
_editing.Host = txtHost.Text.Trim();
_editing.Port = (ushort)numPort.Value;
_editing.AuthMode = radioPassword.Checked ? AuthMode.Password : AuthMode.Guest;
if (_editing.AuthMode == AuthMode.Guest)
{
_editing.LastNickname = txtNickname.Text.Trim();
_editing.SavedUsername = null;
_editing.ProtectedPasswordBase64 = null;
}
else
{
_editing.SavedUsername = txtUsername.Text.Trim();
if (chkRememberPassword.Checked && _passwordChanged && txtPassword.Text.Length > 0)
_editing.ProtectedPasswordBase64 = PasswordProtector.Protect(txtPassword.Text);
else if (!chkRememberPassword.Checked)
_editing.ProtectedPasswordBase64 = null;
// else: remember-password still checked and password box untouched (still shows
// the placeholder) — keep whatever was already protected/stored.
}
Result = _editing;
DialogResult = DialogResult.OK;
}
}
@@ -0,0 +1,263 @@
using VoiceCat.App.Audio;
namespace VoiceCat.App.Forms;
/// <summary>
/// Modal dialog for selecting which apps' audio to share.
/// Returns the chosen <see cref="AppAudioScope"/> via <see cref="ChosenScope"/>,
/// or <see cref="DialogResult.Cancel"/> if the user dismissed without sharing.
/// </summary>
public sealed class AppAudioPickerDialog : Form
{
private readonly RadioButton _rdoAll;
private readonly RadioButton _rdoOnly;
private readonly RadioButton _rdoExcept;
private readonly CheckBox _chkExcludeSelf;
private readonly TextBox _txtFilter;
private readonly ListView _appList;
private readonly Label _lblApps;
// Snapshot taken when the dialog opens (refresh on open, not on every check change).
private IReadOnlyList<AudioAppInfo> _apps = [];
// Checked apps survive filtering (a filtered-out row keeps its check here).
// pid → clean display name (used for the shared scope's Names).
private readonly Dictionary<int, string> _checked = new();
public AppAudioScope? ChosenScope { get; private set; }
public AppAudioPickerDialog()
{
// ── Radio buttons ───────────────────────────────────────────────────
_rdoAll = new RadioButton
{
Text = "&Entire desktop",
Checked = true,
Location = new Point(12, 12),
Size = new Size(360, 20),
TabIndex = 0,
};
_rdoOnly = new RadioButton
{
Text = "&Only selected apps",
Location = new Point(12, 36),
Size = new Size(360, 20),
TabIndex = 1,
};
_rdoExcept = new RadioButton
{
Text = "All apps e&xcept selected",
Location = new Point(12, 60),
Size = new Size(360, 20),
TabIndex = 2,
};
_rdoAll.CheckedChanged += OnModeChanged;
_rdoOnly.CheckedChanged += OnModeChanged;
_rdoExcept.CheckedChanged += OnModeChanged;
// ── Self-exclude (echo prevention) ───────────────────────────────────
// Captures the whole desktop minus VoiceCat's own playback, so the channel
// doesn't echo back the voices you're already hearing. Only applies to the
// entire-desktop mode (the per-app modes already exclude this app's tree).
_chkExcludeSelf = new CheckBox
{
Text = "E&xclude VoiceCat's own audio (prevents echo)",
Checked = true,
Location = new Point(28, 84),
Size = new Size(344, 20),
TabIndex = 3,
};
// ── App list ────────────────────────────────────────────────────────
_lblApps = new Label
{
Text = "Apps (▶ = currently playing):",
AutoSize = true,
Location = new Point(12, 112),
Visible = false,
TabIndex = 4,
};
_txtFilter = new TextBox
{
Location = new Point(12, 132),
Size = new Size(360, 23),
PlaceholderText = "Filter apps…",
Visible = false,
TabIndex = 5,
};
_txtFilter.TextChanged += (_, _) => PopulateList();
_appList = new ListView
{
Location = new Point(12, 160),
Size = new Size(360, 150),
CheckBoxes = true,
View = View.List,
Visible = false,
TabIndex = 6,
FullRowSelect = true,
};
// Exclude mode supports only one target process tree (WASAPI EXCLUDE takes a single
// PID), so enforce a single check while "All apps except selected" is active.
_appList.ItemCheck += OnAppItemCheck;
// ── Buttons ─────────────────────────────────────────────────────────
var btnShare = new Button
{
Text = "&Share",
DialogResult = DialogResult.OK,
Location = new Point(216, 320),
Size = new Size(75, 27),
TabIndex = 7,
};
var btnCancel = new Button
{
Text = "&Cancel",
DialogResult = DialogResult.Cancel,
Location = new Point(297, 320),
Size = new Size(75, 27),
TabIndex = 8,
};
btnShare.Click += OnShareClick;
AcceptButton = btnShare;
CancelButton = btnCancel;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(384, 360);
Controls.AddRange([_rdoAll, _rdoOnly, _rdoExcept, _chkExcludeSelf, _lblApps, _txtFilter,
_appList, btnShare, btnCancel]);
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
Text = "Share App Audio";
}
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
RefreshAppList();
}
private void RefreshAppList()
{
_apps = AudioSessionEnumerator.GetAudioApps();
PopulateList();
}
// Rebuild the visible rows from _apps + the current filter text, restoring check
// state from _checked so a selection persists while the user filters.
private void PopulateList()
{
string filter = _txtFilter.Text.Trim();
_suppressItemCheck = true;
_appList.BeginUpdate();
_appList.Items.Clear();
foreach (var app in _apps)
{
if (filter.Length > 0 &&
app.DisplayName.IndexOf(filter, StringComparison.OrdinalIgnoreCase) < 0)
continue;
string marker = app.IsPlaying ? "▶ " : "";
_appList.Items.Add(new ListViewItem($"{marker}{app.DisplayName} (PID {app.Pid})")
{
Tag = app,
Checked = _checked.ContainsKey(app.Pid),
});
}
_appList.EndUpdate();
_suppressItemCheck = false;
}
private void OnModeChanged(object? sender, EventArgs e)
{
bool showList = _rdoOnly.Checked || _rdoExcept.Checked;
_lblApps.Visible = showList;
_txtFilter.Visible = showList;
_appList.Visible = showList;
_lblApps.Text = _rdoExcept.Checked
? "App to exclude (everything else is shared):"
: "Apps (▶ = currently playing):";
// Self-exclude only applies to entire-desktop; the per-app modes already exclude
// this app's own tree (Include) or spend the single EXCLUDE slot on the chosen app.
_chkExcludeSelf.Enabled = _rdoAll.Checked;
// Switching into exclude mode: collapse any multi-selection down to a single item.
if (_rdoExcept.Checked)
TrimCheckedToOne();
if (showList && _appList.Items.Count == 0)
RefreshAppList();
else
PopulateList(); // re-render markers/labels and restore check state
}
// In exclude mode the list behaves like radio buttons: checking one item clears the rest.
private bool _suppressItemCheck;
private void OnAppItemCheck(object? sender, ItemCheckEventArgs e)
{
if (_suppressItemCheck) return;
if (_appList.Items[e.Index].Tag is not AudioAppInfo app) return;
if (e.NewValue == CheckState.Checked)
{
if (_rdoExcept.Checked)
{
// Radio behavior: clear every other visible check and the persisted set.
_suppressItemCheck = true;
foreach (ListViewItem item in _appList.Items)
if (item.Index != e.Index && item.Checked) item.Checked = false;
_suppressItemCheck = false;
_checked.Clear();
}
_checked[app.Pid] = app.DisplayName;
}
else
{
_checked.Remove(app.Pid);
}
}
// Reduce the persisted selection to a single (first) entry — used when entering
// exclude mode, whose single EXCLUDE slot can only target one process tree.
private void TrimCheckedToOne()
{
if (_checked.Count <= 1) return;
var first = _checked.First();
_checked.Clear();
_checked[first.Key] = first.Value;
}
private void OnShareClick(object? sender, EventArgs e)
{
if (_rdoAll.Checked)
{
ChosenScope = new EntireDesktop(_chkExcludeSelf.Checked);
return;
}
if (_checked.Count == 0)
{
MessageBox.Show(
"Select at least one app, or choose 'Entire desktop'.",
"No apps selected",
MessageBoxButtons.OK,
MessageBoxIcon.Warning);
DialogResult = DialogResult.None; // prevent close
return;
}
var pids = _checked.Keys.ToList();
var names = _checked.Values.ToList();
ChosenScope = _rdoOnly.Checked
? new OnlyApps(pids, names)
: new AllExceptApps(pids, names);
}
}
@@ -0,0 +1,595 @@
using System.ComponentModel;
using VoiceCat.App.Audio;
using VoiceCat.App.Models;
using VoiceCat.Windows;
namespace VoiceCat.App.Forms;
/// <summary>
/// Edits audio input settings: device selection, transmission mode (VAD/PTT/always-on),
/// VAD sensitivity, mic gain, and PTT key. Changes are applied live to the client for
/// immediate feedback; Cancel reverts them.
/// </summary>
public sealed class AudioSettingsForm : Form
{
private readonly VoiceCatClient _client;
private readonly VoiceSettings _settings;
private readonly uint _micStreamId;
// Aux-stream live-apply callbacks, supplied by MainForm (which owns the aux capture). Null
// when not connected — the controls still edit settings, just without live effect.
private readonly Action<bool>? _applyAuxEnabled;
private readonly Action<string?>? _applyAuxDevice;
private readonly Action<float>? _applyAuxGain;
// Snapshot of original values so Cancel can restore them
private readonly string? _origDeviceId;
private readonly VcInputMode _origMode;
private readonly int _origVadSlider;
private readonly int _origMicGain;
private readonly bool _origMicNoiseReduction;
private readonly bool _origStereoMic;
private readonly Keys _origPttKey;
private readonly bool _origSystemWidePtt;
private readonly bool _origAuxEnabled;
private readonly string? _origAuxDeviceId;
private readonly int _origAuxGain;
private readonly int _origAudioBufferMilliseconds;
private readonly ComboBox _cboDevice;
private readonly Button _btnRefresh;
private readonly RadioButton _radioVad;
private readonly RadioButton _radioPtt;
private readonly RadioButton _radioAlwaysOn;
private readonly Label _lblPttKey;
private readonly Button _btnChangePtt;
private readonly CheckBox _chkSystemWidePtt;
private readonly Label _lblSensitivity;
private readonly TrackBar _trkVad;
private readonly TrackBar _trkGain;
private readonly CheckBox _chkNoiseReduction;
private readonly CheckBox _chkStereoMic;
private readonly CheckBox _chkAux;
private readonly Label _lblAuxDevice;
private readonly ComboBox _cboAuxDevice;
private readonly Button _btnAuxRefresh;
private readonly Label _lblAuxGain;
private readonly TrackBar _trkAuxGain;
private readonly ComboBox _cboBuffer;
private Keys _pttKey;
public AudioSettingsForm(VoiceCatClient client, VoiceSettings settings, uint micStreamId,
Action<bool>? applyAuxEnabled = null, Action<string?>? applyAuxDevice = null,
Action<float>? applyAuxGain = null)
{
_client = client;
_settings = settings;
_micStreamId = micStreamId;
_applyAuxEnabled = applyAuxEnabled;
_applyAuxDevice = applyAuxDevice;
_applyAuxGain = applyAuxGain;
_pttKey = (Keys)settings.PttKey;
_origDeviceId = settings.InputDeviceId;
_origMode = (VcInputMode)settings.InputMode;
_origVadSlider = settings.VadThresholdSlider;
_origMicGain = settings.MicGain;
_origMicNoiseReduction = settings.MicNoiseReduction;
_origStereoMic = settings.StereoMic;
_origPttKey = _pttKey;
_origSystemWidePtt = settings.SystemWidePtt;
_origAuxEnabled = settings.AuxEnabled;
_origAuxDeviceId = settings.AuxDeviceId;
_origAuxGain = settings.AuxGain;
_origAudioBufferMilliseconds = settings.AudioBufferMilliseconds;
Text = "Audio settings";
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(420, 690);
// ── Device row ────────────────────────────────────────────────────────
var lblDevice = new Label
{
Text = "Input &device:",
Location = new Point(12, 16),
AutoSize = true,
};
_cboDevice = new ComboBox
{
Location = new Point(12, 36),
Width = 300,
DropDownStyle = ComboBoxStyle.DropDownList,
DisplayMember = "Name",
ValueMember = "Id",
AccessibleName = "Input device",
AccessibleDescription = "Select which microphone or audio input device to use.",
};
_cboDevice.SelectedIndexChanged += CboDevice_SelectedIndexChanged;
_btnRefresh = new Button
{
Text = "&Refresh",
Location = new Point(320, 34),
Size = new Size(80, 26),
};
_btnRefresh.Click += (_, _) => LoadDevices();
// ── Transmission mode ─────────────────────────────────────────────────
var lblMode = new Label
{
Text = "Transmission mode:",
Location = new Point(12, 76),
AutoSize = true,
};
_radioVad = new RadioButton
{
Text = "&Voice activation",
Location = new Point(20, 96),
AutoSize = true,
};
_radioVad.CheckedChanged += RadioMode_CheckedChanged;
_lblSensitivity = new Label
{
Text = "Sensitivity:",
Location = new Point(36, 122),
AutoSize = true,
};
_trkVad = new TrackBar
{
Location = new Point(36, 140),
Size = new Size(200, 45),
Minimum = 1,
Maximum = 100,
TickFrequency = 10,
SmallChange = 1,
LargeChange = 10,
Value = Math.Clamp(settings.VadThresholdSlider, 1, 100),
};
_trkVad.AccessibleName = "VAD sensitivity";
_trkVad.AccessibleDescription =
"Voice detection sensitivity. Higher = more sensitive. Range 1100.";
_trkVad.Scroll += TrkVad_Scroll;
_radioPtt = new RadioButton
{
Text = "&Push to talk",
Location = new Point(20, 192),
AutoSize = true,
};
_radioPtt.CheckedChanged += RadioMode_CheckedChanged;
_lblPttKey = new Label
{
Location = new Point(140, 194),
AutoSize = true,
};
_btnChangePtt = new Button
{
Text = "Change key...",
Location = new Point(200, 189),
Size = new Size(105, 26),
};
_btnChangePtt.Click += BtnChangePtt_Click;
// Indented PTT sub-option: observe the key system-wide (Raw Input) so it works while
// another app is focused. Visible only in PTT mode (RadioMode_CheckedChanged).
_chkSystemWidePtt = new CheckBox
{
Text = "Wor&ks in the background (system-wide)",
Location = new Point(36, 218),
AutoSize = true,
Checked = settings.SystemWidePtt,
AccessibleName = "System-wide push-to-talk",
AccessibleDescription =
"Let push-to-talk work while another application is focused. Uses the Windows Raw " +
"Input API, not a keyboard hook.",
};
_chkSystemWidePtt.CheckedChanged += ChkSystemWidePtt_CheckedChanged;
_radioAlwaysOn = new RadioButton
{
Text = "A&lways on",
Location = new Point(20, 252),
AutoSize = true,
};
_radioAlwaysOn.CheckedChanged += RadioMode_CheckedChanged;
// ── Mic gain ──────────────────────────────────────────────────────────
var lblGain = new Label
{
Text = "Microphone &volume:",
Location = new Point(12, 296),
AutoSize = true,
};
_trkGain = new TrackBar
{
Location = new Point(12, 316),
Size = new Size(200, 45),
Minimum = 0,
Maximum = 400,
TickFrequency = 25,
SmallChange = 5,
LargeChange = 25,
Value = Math.Clamp(settings.MicGain, 0, 400),
};
_trkGain.AccessibleName = "Microphone volume";
_trkGain.AccessibleDescription =
"Boost a quiet microphone. 100 is unity gain; range 0400 percent.";
_trkGain.Scroll += TrkGain_Scroll;
// ── Mic noise reduction ───────────────────────────────────────────────
// Send-side RNNoise denoise of the mic stream. MIC-only (the core's NR runs before
// the input gain and VAD/PTT gate; aux/screen are excluded). One pass for all listeners.
_chkNoiseReduction = new CheckBox
{
Text = "Noise &reduction (RNNoise)",
Location = new Point(12, 368),
AutoSize = true,
Checked = settings.MicNoiseReduction,
AccessibleName = "Microphone noise reduction",
AccessibleDescription =
"RNNoise denoising of your microphone. Cleans your signal for everyone listening.",
};
_chkNoiseReduction.CheckedChanged += ChkNoiseReduction_CheckedChanged;
// ── Stereo microphone ─────────────────────────────────────────────────
// Opens the mic capture device in stereo (interleaved L/R) instead of mono. Real stereo
// only reaches the wire on a stereo channel; the core folds a stereo mic to mono on a mono
// channel. Toggling while connected restarts the capture device (vc_audio_restart) so the
// new channel count takes effect immediately.
_chkStereoMic = new CheckBox
{
Text = "&Stereo microphone",
Location = new Point(12, 392),
AutoSize = true,
Checked = settings.StereoMic,
AccessibleName = "Stereo microphone",
AccessibleDescription =
"Capture your microphone in stereo. Only transmitted in stereo on a stereo channel.",
};
_chkStereoMic.CheckedChanged += ChkStereoMic_CheckedChanged;
// ── Aux input stream ──────────────────────────────────────────────────
// A second outgoing stream from another hardware input device (e.g. line-in / aux),
// captured client-side and fed to the core. Device + volume only — aux is always-on
// (the core never gates AUX_DEVICE on VAD/PTT).
_chkAux = new CheckBox
{
Text = "&Aux stream (second input device)",
Location = new Point(12, 432),
AutoSize = true,
Checked = settings.AuxEnabled,
AccessibleName = "Enable aux input stream",
AccessibleDescription =
"Transmit a second hardware input device alongside your microphone.",
};
_chkAux.CheckedChanged += ChkAux_CheckedChanged;
_lblAuxDevice = new Label
{
Text = "Aux d&evice:",
Location = new Point(12, 462),
AutoSize = true,
};
_cboAuxDevice = new ComboBox
{
Location = new Point(12, 482),
Width = 300,
DropDownStyle = ComboBoxStyle.DropDownList,
DisplayMember = "Name",
ValueMember = "Id",
AccessibleName = "Aux input device",
AccessibleDescription = "Select the second audio input device to transmit.",
};
_cboAuxDevice.SelectedIndexChanged += CboAuxDevice_SelectedIndexChanged;
_btnAuxRefresh = new Button
{
Text = "Re&fresh",
Location = new Point(320, 480),
Size = new Size(80, 26),
};
_btnAuxRefresh.Click += (_, _) => LoadAuxDevices();
_lblAuxGain = new Label
{
Text = "Aux vo&lume:",
Location = new Point(12, 518),
AutoSize = true,
};
_trkAuxGain = new TrackBar
{
Location = new Point(12, 538),
Size = new Size(200, 45),
Minimum = 0,
Maximum = 400,
TickFrequency = 25,
SmallChange = 5,
LargeChange = 25,
Value = Math.Clamp(settings.AuxGain, 0, 400),
};
_trkAuxGain.AccessibleName = "Aux volume";
_trkAuxGain.AccessibleDescription =
"Volume of the aux input stream. 100 is unity gain; range 0400 percent.";
_trkAuxGain.Scroll += TrkAuxGain_Scroll;
var lblBuffer = new Label { Text = "Audio &buffering:", Location = new Point(12, 590), AutoSize = true };
_cboBuffer = new ComboBox
{
Location = new Point(130, 586), Width = 190, DropDownStyle = ComboBoxStyle.DropDownList,
AccessibleName = "Audio buffering",
AccessibleDescription = "Local capture and playback buffering. Lower values reduce latency; higher values tolerate more scheduling jitter."
};
_cboBuffer.Items.AddRange(["Low latency (20 ms)", "Balanced (40 ms)", "Stable (60 ms)"]);
_cboBuffer.SelectedIndex = settings.AudioBufferMilliseconds switch { 20 => 0, 60 => 2, _ => 1 };
_cboBuffer.SelectedIndexChanged += (_, _) =>
{
_settings.AudioBufferMilliseconds = _cboBuffer.SelectedIndex switch { 0 => 20, 2 => 60, _ => 40 };
_client.SetAudioBufferMilliseconds(_settings.AudioBufferMilliseconds);
};
// ── OK / Cancel ───────────────────────────────────────────────────────
var btnOk = new Button
{
Text = "&OK",
DialogResult = DialogResult.OK,
Location = new Point(228, 642),
Size = new Size(80, 27),
};
var btnCancel = new Button
{
Text = "&Cancel",
DialogResult = DialogResult.Cancel,
Location = new Point(316, 642),
Size = new Size(80, 27),
};
AcceptButton = btnOk;
CancelButton = btnCancel;
btnOk.Click += (_, _) =>
{
_settings.Save();
};
FormClosing += AudioSettingsForm_FormClosing;
Controls.AddRange([
lblDevice, _cboDevice, _btnRefresh,
lblMode, _radioVad, _lblSensitivity, _trkVad,
_radioPtt, _lblPttKey, _btnChangePtt, _chkSystemWidePtt, _radioAlwaysOn,
lblGain, _trkGain, _chkNoiseReduction, _chkStereoMic,
_chkAux, _lblAuxDevice, _cboAuxDevice, _btnAuxRefresh, _lblAuxGain, _trkAuxGain,
lblBuffer, _cboBuffer,
btnOk, btnCancel,
]);
// Apply saved mode (fires RadioMode_CheckedChanged which sets visibility)
switch ((VcInputMode)settings.InputMode)
{
case VcInputMode.PushToTalk: _radioPtt.Checked = true; break;
case VcInputMode.AlwaysOn: _radioAlwaysOn.Checked = true; break;
default: _radioVad.Checked = true; break;
}
LoadDevices();
LoadAuxDevices();
UpdateAuxControlsEnabled();
}
private void LoadDevices()
{
string? currentId = (_cboDevice.SelectedItem as DeviceInfo)?.Id ?? _settings.InputDeviceId;
var devices = _client.ListDevices(VcDeviceKind.Input);
_cboDevice.SelectedIndexChanged -= CboDevice_SelectedIndexChanged;
_cboDevice.DataSource = new BindingList<DeviceInfo>(devices);
_cboDevice.SelectedIndexChanged += CboDevice_SelectedIndexChanged;
// Try to restore previous selection
int idx = -1;
if (currentId is not null)
idx = devices.FindIndex(d => d.Id == currentId);
if (idx < 0)
idx = devices.FindIndex(d => d.IsDefault);
_cboDevice.SelectedIndex = idx >= 0 ? idx : (devices.Count > 0 ? 0 : -1);
}
private void CboDevice_SelectedIndexChanged(object? sender, EventArgs e)
{
if (_cboDevice.SelectedItem is not DeviceInfo dev) return;
_settings.InputDeviceId = dev.IsDefault ? null : dev.Id;
if (_micStreamId != 0)
_client.SetInputDevice(_micStreamId, dev.IsDefault ? null : dev.Id);
}
// ── Aux input stream ──────────────────────────────────────────────────────
private void LoadAuxDevices()
{
string? currentId = (_cboAuxDevice.SelectedItem as InputDeviceInfo)?.Id
?? _settings.AuxDeviceId;
var devices = InputDeviceEnumerator.List().ToList();
// Set the data source AND restore the selection while unsubscribed so neither the initial
// populate nor a Refresh fires a spurious device-change (which would restart the capture).
_cboAuxDevice.SelectedIndexChanged -= CboAuxDevice_SelectedIndexChanged;
_cboAuxDevice.DataSource = new BindingList<InputDeviceInfo>(devices);
int idx = -1;
if (currentId is not null)
idx = devices.FindIndex(d => d.Id == currentId);
if (idx < 0)
idx = devices.FindIndex(d => d.IsDefault);
_cboAuxDevice.SelectedIndex = idx >= 0 ? idx : (devices.Count > 0 ? 0 : -1);
_cboAuxDevice.SelectedIndexChanged += CboAuxDevice_SelectedIndexChanged;
// Persist the resolved selection (null = default) so Join Voice opens the same device.
if (_cboAuxDevice.SelectedItem is InputDeviceInfo dev)
_settings.AuxDeviceId = dev.IsDefault ? null : dev.Id;
}
private void ChkAux_CheckedChanged(object? sender, EventArgs e)
{
_settings.AuxEnabled = _chkAux.Checked;
UpdateAuxControlsEnabled();
_applyAuxEnabled?.Invoke(_chkAux.Checked);
}
private void CboAuxDevice_SelectedIndexChanged(object? sender, EventArgs e)
{
if (_cboAuxDevice.SelectedItem is not InputDeviceInfo dev) return;
_settings.AuxDeviceId = dev.IsDefault ? null : dev.Id;
if (_chkAux.Checked)
_applyAuxDevice?.Invoke(dev.IsDefault ? null : dev.Id);
}
private void TrkAuxGain_Scroll(object? sender, EventArgs e)
{
_settings.AuxGain = _trkAuxGain.Value;
if (_chkAux.Checked)
_applyAuxGain?.Invoke(_trkAuxGain.Value / 100f);
}
private void UpdateAuxControlsEnabled()
{
bool on = _chkAux.Checked;
_lblAuxDevice.Enabled = on;
_cboAuxDevice.Enabled = on;
_btnAuxRefresh.Enabled = on;
_lblAuxGain.Enabled = on;
_trkAuxGain.Enabled = on;
}
private void RadioMode_CheckedChanged(object? sender, EventArgs e)
{
var mode = CurrentMode();
bool isVad = mode == VcInputMode.VoiceActivation;
bool isPtt = mode == VcInputMode.PushToTalk;
_lblSensitivity.Visible = isVad;
_trkVad.Visible = isVad;
_lblPttKey.Visible = isPtt;
_btnChangePtt.Visible = isPtt;
_chkSystemWidePtt.Visible = isPtt;
UpdatePttKeyLabel();
_settings.InputMode = (int)mode;
if (_micStreamId != 0)
{
_client.SetInputMode(mode);
if (isVad) _client.SetVadThreshold(VadThresholdFromSlider());
if (isPtt) _client.SetPushToTalk(false);
}
}
private void TrkVad_Scroll(object? sender, EventArgs e)
{
_settings.VadThresholdSlider = _trkVad.Value;
if (_micStreamId != 0 && CurrentMode() == VcInputMode.VoiceActivation)
_client.SetVadThreshold(VadThresholdFromSlider());
}
private void TrkGain_Scroll(object? sender, EventArgs e)
{
_settings.MicGain = _trkGain.Value;
if (_micStreamId != 0)
_client.SetInputGain(_trkGain.Value / 100f);
}
private void ChkNoiseReduction_CheckedChanged(object? sender, EventArgs e)
{
_settings.MicNoiseReduction = _chkNoiseReduction.Checked;
if (_micStreamId != 0)
_client.SetInputNoiseReduction(_chkNoiseReduction.Checked);
}
private void ChkStereoMic_CheckedChanged(object? sender, EventArgs e)
{
_settings.StereoMic = _chkStereoMic.Checked;
// Channel count only takes effect when the capture device (re)starts, so restart it live.
if (_micStreamId != 0)
{
_client.SetCaptureChannels(_micStreamId, _chkStereoMic.Checked ? 2u : 1u);
_client.AudioRestart();
}
}
private void ChkSystemWidePtt_CheckedChanged(object? sender, EventArgs e) =>
// No live effect here — MainForm reads SystemWidePtt after the dialog closes and
// registers/unregisters the Raw Input keyboard sink accordingly.
_settings.SystemWidePtt = _chkSystemWidePtt.Checked;
private void BtnChangePtt_Click(object? sender, EventArgs e)
{
using var dlg = new PttKeyCaptureDialog(_pttKey);
if (dlg.ShowDialog(this) == DialogResult.OK)
{
_pttKey = dlg.CapturedKey;
_settings.PttKey = (int)_pttKey;
UpdatePttKeyLabel();
}
}
private void AudioSettingsForm_FormClosing(object? sender, FormClosingEventArgs e)
{
if (DialogResult == DialogResult.OK) return;
// Cancel: restore originals to settings and live client
_settings.InputDeviceId = _origDeviceId;
_settings.InputMode = (int)_origMode;
_settings.VadThresholdSlider = _origVadSlider;
_settings.MicGain = _origMicGain;
_settings.MicNoiseReduction = _origMicNoiseReduction;
_settings.StereoMic = _origStereoMic;
_settings.PttKey = (int)_origPttKey;
_settings.SystemWidePtt = _origSystemWidePtt;
_settings.AudioBufferMilliseconds = _origAudioBufferMilliseconds;
_client.SetAudioBufferMilliseconds(_origAudioBufferMilliseconds);
if (_micStreamId != 0)
{
_client.SetInputDevice(_micStreamId, _origDeviceId);
_client.SetInputMode(_origMode);
if (_origMode == VcInputMode.VoiceActivation)
_client.SetVadThreshold(0.1f * (1f - (_origVadSlider - 1f) / 99f));
_client.SetInputGain(_origMicGain / 100f);
_client.SetInputNoiseReduction(_origMicNoiseReduction);
// Restore capture channel count; restart the device only if it actually changed.
if (_origStereoMic != _chkStereoMic.Checked)
{
_client.SetCaptureChannels(_micStreamId, _origStereoMic ? 2u : 1u);
_client.AudioRestart();
}
}
// Aux: restore originals to settings and re-apply live (order: device + gain first so a
// re-enable starts with the right configuration).
_settings.AuxEnabled = _origAuxEnabled;
_settings.AuxDeviceId = _origAuxDeviceId;
_settings.AuxGain = _origAuxGain;
_applyAuxDevice?.Invoke(_origAuxDeviceId);
_applyAuxGain?.Invoke(_origAuxGain / 100f);
_applyAuxEnabled?.Invoke(_origAuxEnabled);
}
private void UpdatePttKeyLabel() =>
_lblPttKey.Text = $"({_pttKey})";
private VcInputMode CurrentMode() =>
_radioPtt.Checked ? VcInputMode.PushToTalk :
_radioAlwaysOn.Checked ? VcInputMode.AlwaysOn :
VcInputMode.VoiceActivation;
private float VadThresholdFromSlider() =>
0.1f * (1f - (_trkVad.Value - 1f) / 99f);
}
@@ -0,0 +1,94 @@
namespace VoiceCat.App.Forms;
/// <summary>
/// Prompt for a ban reason and duration.
/// </summary>
public sealed class BanUserDialog : Form
{
private readonly TextBox _txtReason;
private readonly ComboBox _cboDuration;
public string Reason => _txtReason.Text.Trim();
public ulong ExpiresUnixMs { get; private set; }
public BanUserDialog(string nickname)
{
Text = $"Ban {nickname}";
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(384, 160);
var lblReason = new Label
{
Text = "&Reason:",
Location = new Point(12, 12),
AutoSize = true,
};
_txtReason = new TextBox
{
Location = new Point(12, 34),
Size = new Size(360, 23),
TabIndex = 1,
};
var lblDuration = new Label
{
Text = "&Duration:",
Location = new Point(12, 66),
AutoSize = true,
};
_cboDuration = new ComboBox
{
Location = new Point(12, 88),
Size = new Size(200, 23),
DropDownStyle = ComboBoxStyle.DropDownList,
TabIndex = 2,
};
_cboDuration.Items.AddRange(new object[]
{
new DurationItem("1 hour", TimeSpan.FromHours(1)),
new DurationItem("1 day", TimeSpan.FromDays(1)),
new DurationItem("1 week", TimeSpan.FromDays(7)),
new DurationItem("Permanent", TimeSpan.Zero),
});
_cboDuration.SelectedIndex = 1;
var btnOk = new Button
{
Text = "&Ban",
DialogResult = DialogResult.OK,
Location = new Point(216, 126),
Size = new Size(75, 27),
TabIndex = 3,
};
var btnCancel = new Button
{
Text = "&Cancel",
DialogResult = DialogResult.Cancel,
Location = new Point(297, 126),
Size = new Size(75, 27),
TabIndex = 4,
};
AcceptButton = btnOk;
CancelButton = btnCancel;
Controls.AddRange([lblReason, _txtReason, lblDuration, _cboDuration, btnOk, btnCancel]);
btnOk.Click += (_, _) =>
{
var selected = _cboDuration.SelectedItem as DurationItem;
ExpiresUnixMs = selected?.Duration == TimeSpan.Zero
? 0
: (ulong)DateTimeOffset.UtcNow.Add(selected!.Duration).ToUnixTimeMilliseconds();
};
}
private sealed class DurationItem(string label, TimeSpan duration)
{
public TimeSpan Duration { get; } = duration;
public override string ToString() => label;
}
}
@@ -0,0 +1,435 @@
using VoiceCat.Windows;
namespace VoiceCat.App.Forms;
/// <summary>
/// Create or edit a channel, including its full per-channel Opus audio config.
/// Returns DialogResult.OK with Result set.
/// </summary>
public sealed class ChannelEditDialog : Form
{
private readonly bool _isCreate;
private readonly uint _editingId;
private readonly List<ChannelInfo> _channels;
private TextBox _txtName = null!;
private TextBox _txtTopic = null!;
private ComboBox _cboParent = null!;
private NumericUpDown _numMaxUsers = null!;
private NumericUpDown _numSortOrder = null!;
private CheckBox _chkPassword = null!;
private TextBox _txtPassword = null!;
private ComboBox _cboMode = null!;
private NumericUpDown _numSampleRate = null!;
private NumericUpDown _numBitrate = null!;
private NumericUpDown _numFrameMs = null!;
private ComboBox _cboApplication = null!;
private CheckBox _chkFec = null!;
private CheckBox _chkAutomaticLoss = null!;
private ComboBox _cboLossSpeed = null!;
private NumericUpDown _numExpectedLoss = null!;
private CheckBox _chkDtx = null!;
private CheckBox _chkDred = null!;
private NumericUpDown _numComplexity = null!;
public ChannelEditInfo? Result { get; private set; }
private readonly bool _supportsAdaptivePacketLoss;
public ChannelEditDialog(IEnumerable<ChannelInfo> channels, ChannelEditInfo? existing = null, bool supportsAdaptivePacketLoss = true)
{
_isCreate = existing is null;
_editingId = existing?.Id ?? 0;
_channels = channels.Where(c => c.Id != _editingId).ToList();
_supportsAdaptivePacketLoss = supportsAdaptivePacketLoss;
Text = _isCreate ? "Create channel" : "Edit channel";
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(520, 520);
var tabs = new TabControl
{
Dock = DockStyle.Fill,
TabIndex = 0,
};
var pageGeneral = new TabPage("General");
BuildGeneralPage(pageGeneral, existing);
var pageAudio = new TabPage("Audio config");
BuildAudioPage(pageAudio, existing?.Audio);
tabs.TabPages.Add(pageGeneral);
tabs.TabPages.Add(pageAudio);
var btnOk = new Button
{
Text = "&OK",
DialogResult = DialogResult.OK,
Location = new Point(350, 486),
Size = new Size(75, 27),
TabIndex = 100,
};
var btnCancel = new Button
{
Text = "&Cancel",
DialogResult = DialogResult.Cancel,
Location = new Point(431, 486),
Size = new Size(75, 27),
TabIndex = 101,
};
AcceptButton = btnOk;
CancelButton = btnCancel;
Controls.Add(tabs);
Controls.Add(btnOk);
Controls.Add(btnCancel);
btnOk.Click += BtnOk_Click;
}
private void BuildGeneralPage(TabPage page, ChannelEditInfo? existing)
{
int y = 16;
int labelWidth = 110;
int inputX = 128;
AddLabel(page, "&Name:", 12, y, labelWidth);
_txtName = new TextBox
{
Location = new Point(inputX, y - 2),
Size = new Size(360, 23),
Text = existing?.Name ?? "",
TabIndex = 1,
};
page.Controls.Add(_txtName);
y += 36;
AddLabel(page, "&Topic:", 12, y, labelWidth);
_txtTopic = new TextBox
{
Location = new Point(inputX, y - 2),
Size = new Size(360, 23),
Text = existing?.Topic ?? "",
TabIndex = 2,
};
page.Controls.Add(_txtTopic);
y += 36;
AddLabel(page, "&Parent channel:", 12, y, labelWidth);
_cboParent = new ComboBox
{
Location = new Point(inputX, y - 2),
Size = new Size(360, 23),
DropDownStyle = ComboBoxStyle.DropDownList,
TabIndex = 3,
};
_cboParent.Items.Add(new ChannelItem("(root)", 0));
foreach (var ch in _channels.OrderBy(c => c.Name))
_cboParent.Items.Add(new ChannelItem(ch.Name, ch.Id));
SelectParent(existing?.ParentId ?? 0);
page.Controls.Add(_cboParent);
y += 36;
AddLabel(page, "&Max users:", 12, y, labelWidth);
_numMaxUsers = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 0,
Maximum = 10000,
Value = existing?.MaxUsers ?? 0,
TabIndex = 4,
};
page.Controls.Add(_numMaxUsers);
AddLabel(page, "(0 = unlimited)", inputX + 128, y, 100);
y += 36;
AddLabel(page, "&Sort order:", 12, y, labelWidth);
_numSortOrder = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = int.MinValue,
Maximum = int.MaxValue,
Value = existing?.SortOrder ?? 0,
TabIndex = 5,
};
page.Controls.Add(_numSortOrder);
y += 36;
_chkPassword = new CheckBox
{
Text = "&Password protected",
Location = new Point(inputX, y),
AutoSize = true,
Checked = existing?.PasswordProtected ?? false,
TabIndex = 6,
};
page.Controls.Add(_chkPassword);
y += 28;
_txtPassword = new TextBox
{
Location = new Point(inputX, y),
Size = new Size(360, 23),
UseSystemPasswordChar = true,
Enabled = _chkPassword.Checked,
Text = existing?.Password ?? "",
TabIndex = 7,
};
page.Controls.Add(_txtPassword);
_chkPassword.CheckedChanged += (_, _) => _txtPassword.Enabled = _chkPassword.Checked;
}
private void BuildAudioPage(TabPage page, AudioConfigInfo? audio)
{
audio ??= new AudioConfigInfo(0, false, 48000, 0, 20, 0, true, 0, false, 10, false);
int y = 16;
int labelWidth = 150;
int inputX = 168;
AddLabel(page, "Codec (0 = Opus):", 12, y, labelWidth);
var numCodec = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 0,
Maximum = 0,
Value = audio.Codec,
Enabled = false,
};
page.Controls.Add(numCodec);
y += 34;
AddLabel(page, "&Mode:", 12, y, labelWidth);
_cboMode = new ComboBox
{
Location = new Point(inputX, y - 2),
Size = new Size(160, 23),
DropDownStyle = ComboBoxStyle.DropDownList,
TabIndex = 10,
};
_cboMode.Items.AddRange(["Mono", "Stereo"]);
_cboMode.SelectedIndex = audio.Stereo ? 1 : 0;
page.Controls.Add(_cboMode);
y += 34;
AddLabel(page, "Sample &rate (Hz):", 12, y, labelWidth);
_numSampleRate = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 8000,
Maximum = 96000,
Value = audio.SampleRate,
TabIndex = 11,
};
page.Controls.Add(_numSampleRate);
y += 34;
AddLabel(page, "&Bitrate (bps, 0 = default):", 12, y, labelWidth);
_numBitrate = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 0,
Maximum = 512000,
Increment = 1000,
Value = audio.BitrateBps,
TabIndex = 12,
};
page.Controls.Add(_numBitrate);
y += 34;
AddLabel(page, "Frame &ms:", 12, y, labelWidth);
_numFrameMs = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 5,
Maximum = 120,
Value = audio.FrameMs,
TabIndex = 13,
};
page.Controls.Add(_numFrameMs);
y += 34;
AddLabel(page, "&Application:", 12, y, labelWidth);
_cboApplication = new ComboBox
{
Location = new Point(inputX, y - 2),
Size = new Size(160, 23),
DropDownStyle = ComboBoxStyle.DropDownList,
TabIndex = 14,
};
_cboApplication.Items.AddRange(["VoIP", "Audio", "Low delay"]);
_cboApplication.SelectedIndex = (int)Math.Min(2, audio.Application);
page.Controls.Add(_cboApplication);
y += 34;
_chkAutomaticLoss = new CheckBox
{
Text = "&Automatic packet loss",
Location = new Point(inputX, y),
AutoSize = true,
Checked = audio.PacketLossMode != VcPacketLossMode.Manual,
Enabled = _supportsAdaptivePacketLoss,
TabIndex = 15,
};
page.Controls.Add(_chkAutomaticLoss);
y += 30;
AddLabel(page, "Adaptation &speed:", 12, y, labelWidth);
_cboLossSpeed = new ComboBox
{
Location = new Point(inputX, y - 2),
Size = new Size(160, 23),
DropDownStyle = ComboBoxStyle.DropDownList,
TabIndex = 16,
};
_cboLossSpeed.Items.AddRange(["Fast", "Balanced", "Stable"]);
_cboLossSpeed.SelectedIndex = audio.PacketLossMode == VcPacketLossMode.Manual ? 1 : (int)audio.PacketLossMode - 1;
page.Controls.Add(_cboLossSpeed);
y += 34;
AddLabel(page, "Expected packet loss (%):", 12, y, labelWidth);
_numExpectedLoss = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 0,
Maximum = 100,
Value = audio.ExpectedPacketLoss,
TabIndex = 17,
};
page.Controls.Add(_numExpectedLoss);
void UpdateLossControls()
{
_numExpectedLoss.Enabled = !_chkAutomaticLoss.Checked;
_cboLossSpeed.Enabled = _supportsAdaptivePacketLoss && _chkAutomaticLoss.Checked;
}
_chkAutomaticLoss.CheckedChanged += (_, _) => UpdateLossControls();
UpdateLossControls();
y += 34;
AddLabel(page, "Com&plexity (010):", 12, y, labelWidth);
_numComplexity = new NumericUpDown
{
Location = new Point(inputX, y - 2),
Size = new Size(120, 23),
Minimum = 0,
Maximum = 10,
Value = audio.Complexity,
TabIndex = 18,
};
page.Controls.Add(_numComplexity);
y += 34;
_chkFec = new CheckBox
{
Text = "&FEC",
Location = new Point(inputX, y),
AutoSize = true,
Checked = audio.Fec,
TabIndex = 19,
};
page.Controls.Add(_chkFec);
y += 28;
_chkDtx = new CheckBox
{
Text = "&DTX",
Location = new Point(inputX, y),
AutoSize = true,
Checked = audio.Dtx,
TabIndex = 20,
};
page.Controls.Add(_chkDtx);
y += 28;
_chkDred = new CheckBox
{
Text = "D&RED (deep redundancy)",
Location = new Point(inputX, y),
AutoSize = true,
Checked = audio.Dred,
TabIndex = 21,
};
page.Controls.Add(_chkDred);
}
private static void AddLabel(Control parent, string text, int x, int y, int width)
{
parent.Controls.Add(new Label
{
Text = text,
Location = new Point(x, y),
Size = new Size(width, 17),
TextAlign = System.Drawing.ContentAlignment.MiddleLeft,
});
}
private void SelectParent(uint parentId)
{
for (int i = 0; i < _cboParent.Items.Count; i++)
{
if (_cboParent.Items[i] is ChannelItem item && item.Id == parentId)
{
_cboParent.SelectedIndex = i;
return;
}
}
_cboParent.SelectedIndex = 0;
}
private void BtnOk_Click(object? sender, EventArgs e)
{
string name = _txtName.Text.Trim();
if (string.IsNullOrEmpty(name))
{
MessageBox.Show(this, "Channel name is required.", "VoiceCat",
MessageBoxButtons.OK, MessageBoxIcon.Warning);
DialogResult = DialogResult.None;
return;
}
uint parentId = (_cboParent.SelectedItem as ChannelItem)?.Id ?? 0;
var audio = new AudioConfigInfo(
Codec: 0,
Stereo: _cboMode.SelectedIndex == 1,
SampleRate: (uint)_numSampleRate.Value,
BitrateBps: (uint)_numBitrate.Value,
FrameMs: (uint)_numFrameMs.Value,
Application: (uint)_cboApplication.SelectedIndex,
Fec: _chkFec.Checked,
ExpectedPacketLoss: (uint)_numExpectedLoss.Value,
Dtx: _chkDtx.Checked,
Complexity: (uint)_numComplexity.Value,
Dred: _chkDred.Checked,
PacketLossMode: _chkAutomaticLoss.Checked ? (VcPacketLossMode)(_cboLossSpeed.SelectedIndex + 1) : VcPacketLossMode.Manual);
Result = new ChannelEditInfo(
Id: _editingId,
ParentId: parentId,
Name: name,
Topic: _txtTopic.Text.Trim(),
PasswordProtected: _chkPassword.Checked,
Password: _chkPassword.Checked ? _txtPassword.Text : null,
MaxUsers: (uint)_numMaxUsers.Value,
SortOrder: (int)_numSortOrder.Value,
Audio: audio);
}
private sealed class ChannelItem(string name, uint id)
{
public uint Id { get; } = id;
public override string ToString() => name;
}
}
@@ -0,0 +1,83 @@
namespace VoiceCat.App.Forms;
partial class ConnectDialog
{
private System.ComponentModel.IContainer components = null!;
private Label lblServers = null!;
private ListBox lstServers = null!;
private Button btnConnect = null!;
private Button btnAddNew = null!;
private Button btnEdit = null!;
private Button btnRemove = null!;
private Label lblStatus = null!;
protected override void Dispose(bool disposing)
{
if (disposing) components?.Dispose();
base.Dispose(disposing);
}
private void InitializeComponent()
{
components = new System.ComponentModel.Container();
lblServers = new Label();
lstServers = new ListBox();
btnConnect = new Button();
btnAddNew = new Button();
btnEdit = new Button();
btnRemove = new Button();
lblStatus = new Label();
lblServers.Text = "&Saved servers:";
lblServers.AutoSize = true;
lblServers.Location = new Point(12, 12);
lblServers.TabIndex = 0;
lstServers.AccessibleName = "Saved servers";
lstServers.Location = new Point(12, 32);
lstServers.Size = new Size(360, 200);
lstServers.TabIndex = 1;
lstServers.SelectedIndexChanged += (_, _) => UpdateButtonsEnabled();
btnConnect.Text = "&Connect";
btnConnect.Location = new Point(384, 32);
btnConnect.Size = new Size(110, 27);
btnConnect.TabIndex = 2;
btnAddNew.Text = "&Add new...";
btnAddNew.Location = new Point(384, 65);
btnAddNew.Size = new Size(110, 27);
btnAddNew.TabIndex = 3;
btnEdit.Text = "&Edit...";
btnEdit.Location = new Point(384, 98);
btnEdit.Size = new Size(110, 27);
btnEdit.TabIndex = 4;
btnRemove.Text = "&Remove";
btnRemove.Location = new Point(384, 131);
btnRemove.Size = new Size(110, 27);
btnRemove.TabIndex = 5;
lblStatus.AccessibleName = "Connection status";
lblStatus.Location = new Point(12, 244);
lblStatus.Size = new Size(482, 23);
lblStatus.TabIndex = 6;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(506, 280);
Controls.Add(lblServers);
Controls.Add(lstServers);
Controls.Add(btnConnect);
Controls.Add(btnAddNew);
Controls.Add(btnEdit);
Controls.Add(btnRemove);
Controls.Add(lblStatus);
AcceptButton = btnConnect;
MinimizeBox = false;
MaximizeBox = false;
FormBorderStyle = FormBorderStyle.FixedDialog;
StartPosition = FormStartPosition.CenterScreen;
Text = "VoiceCat — Connect to a server";
}
}
@@ -0,0 +1,262 @@
using VoiceCat.App.Models;
using VoiceCat.Windows;
namespace VoiceCat.App.Forms;
/// <summary>
/// Saved-server list + connect flow. On success, ConnectedClient/SelfUserId/Nickname are set
/// and DialogResult == OK; the caller (Program.cs) takes ownership of ConnectedClient (does
/// NOT dispose it here — MainForm owns its lifetime from that point on).
/// </summary>
public partial class ConnectDialog : Form
{
private readonly List<SavedServer> _servers;
private readonly System.Windows.Forms.Timer _pumpTimer = new() { Interval = 30 };
private VoiceCatClient? _client;
private bool _identityDialogShown;
public VoiceCatClient? ConnectedClient { get; private set; }
public uint SelfUserId { get; private set; }
public string Nickname { get; private set; } = "";
public string ServerName { get; private set; } = "";
public ConnectDialog()
{
InitializeComponent();
_servers = ServerListStore.Load();
RefreshServerList();
_pumpTimer.Tick += (_, _) =>
{
try { _client?.PumpEvents(); }
catch (Exception ex) { Console.Error.WriteLine($"[ConnectDialog] EXCEPTION in PumpEvents/event handler: {ex}"); }
};
btnAddNew.Click += BtnAddNew_Click;
btnEdit.Click += BtnEdit_Click;
btnRemove.Click += BtnRemove_Click;
btnConnect.Click += BtnConnect_Click;
lstServers.DoubleClick += BtnConnect_Click;
}
private void RefreshServerList()
{
object? previouslySelected = lstServers.SelectedItem;
lstServers.Items.Clear();
foreach (var s in _servers) lstServers.Items.Add(s);
if (previouslySelected is not null && _servers.Contains(previouslySelected))
lstServers.SelectedItem = previouslySelected;
else if (lstServers.Items.Count > 0)
lstServers.SelectedIndex = 0;
UpdateButtonsEnabled();
}
private void UpdateButtonsEnabled()
{
bool hasSelection = lstServers.SelectedItem is not null;
btnConnect.Enabled = hasSelection;
btnEdit.Enabled = hasSelection;
btnRemove.Enabled = hasSelection;
}
private void BtnAddNew_Click(object? sender, EventArgs e)
{
using var dlg = new AddServerDialog();
if (dlg.ShowDialog(this) != DialogResult.OK || dlg.Result is null) return;
_servers.Add(dlg.Result);
ServerListStore.Save(_servers);
RefreshServerList();
lstServers.SelectedItem = dlg.Result;
}
private void BtnEdit_Click(object? sender, EventArgs e)
{
if (lstServers.SelectedItem is not SavedServer existing) return;
using var dlg = new AddServerDialog(existing);
if (dlg.ShowDialog(this) != DialogResult.OK) return;
ServerListStore.Save(_servers);
RefreshServerList();
}
private void BtnRemove_Click(object? sender, EventArgs e)
{
if (lstServers.SelectedItem is not SavedServer existing) return;
var confirm = MessageBox.Show(this, $"Remove '{existing}' from the saved-server list?",
"VoiceCat", MessageBoxButtons.YesNo, MessageBoxIcon.Question);
if (confirm != DialogResult.Yes) return;
_servers.Remove(existing);
ServerListStore.Save(_servers);
RefreshServerList();
}
private void BtnConnect_Click(object? sender, EventArgs e)
{
if (lstServers.SelectedItem is not SavedServer server)
return;
try
{
StartConnect(server);
}
catch (Exception ex)
{
Console.Error.WriteLine($"[ConnectDialog] EXCEPTION in StartConnect: {ex}");
lblStatus.Text = $"Internal error: {ex.Message}";
CleanupFailedAttempt();
}
}
private void StartConnect(SavedServer server)
{
SetBusy(true);
lblStatus.Text = "Connecting...";
ServerName = string.IsNullOrWhiteSpace(server.DisplayName)
? $"{server.Host}:{server.Port}"
: server.DisplayName;
string tofuDir = Path.GetDirectoryName(ServerListStore.TofuStorePath)!;
Directory.CreateDirectory(tofuDir);
_client = new VoiceCatClient("VoiceCat-Windows", VoiceCatClient.VersionString,
VcLogLevel.Info, ServerListStore.TofuStorePath, new VoiceCat.App.Audio.WasapiAudioBackend());
_client.EventReceived += OnEvent;
_identityDialogShown = false;
_pumpTimer.Start();
var connectResult = _client.Connect(server.Host, server.Port);
if (connectResult != VcResult.Ok)
{
lblStatus.Text = $"Connect failed: {connectResult}";
CleanupFailedAttempt();
return;
}
if (server.AuthMode == AuthMode.Guest)
{
Nickname = string.IsNullOrWhiteSpace(server.LastNickname) ? Environment.UserName : server.LastNickname;
_client.AuthenticateGuest(Nickname);
}
else
{
string password;
if (server.ProtectedPasswordBase64 is not null)
{
password = PasswordProtector.Unprotect(server.ProtectedPasswordBase64);
}
else
{
using var pwDlg = new PasswordPromptDialog($"Password for {server.SavedUsername}@{server.Host}:");
if (pwDlg.ShowDialog(this) != DialogResult.OK)
{
lblStatus.Text = "Cancelled.";
CleanupFailedAttempt();
return;
}
password = pwDlg.Password;
}
Nickname = server.SavedUsername ?? "";
_client.AuthenticateUser(server.SavedUsername ?? "", password);
}
}
private void OnEvent(VoiceCatEvent ev)
{
switch (ev.Type)
{
case VcEventType.ConnectionState:
lblStatus.Text = ev.ConnectionState switch
{
VcConnectionState.Connecting => "Connecting...",
VcConnectionState.TlsHandshake => "TLS handshake...",
VcConnectionState.VerifyingIdentity => "Verifying server identity...",
VcConnectionState.Authenticating => "Authenticating...",
VcConnectionState.Connected => "Connected.",
_ => lblStatus.Text,
};
break;
case VcEventType.ServerIdentity:
HandleServerIdentity((VcTofuStatus)ev.U32a, ev.Text ?? "");
break;
case VcEventType.AuthResult:
if (ev.Result == VcResult.Ok)
{
SelfUserId = ev.UserId;
ConnectedClient = _client;
_pumpTimer.Stop();
_client!.EventReceived -= OnEvent;
DialogResult = DialogResult.OK;
Close();
}
else
{
lblStatus.Text = $"Authentication failed: {ev.Text}";
CleanupFailedAttempt();
}
break;
case VcEventType.Disconnected:
if (ConnectedClient is null)
{
lblStatus.Text = string.IsNullOrEmpty(ev.Text) ? "Disconnected." : $"Disconnected: {ev.Text}";
CleanupFailedAttempt();
}
break;
}
}
private void HandleServerIdentity(VcTofuStatus status, string certFingerprintHex)
{
if (_identityDialogShown) return; // one decision per connect attempt
if (status == VcTofuStatus.Matched)
{
// Silent success path — no dialog. See ServerIdentityDialog's doc comment.
_client!.ConfirmServerIdentity(true);
return;
}
_identityDialogShown = true;
using var dlg = new ServerIdentityDialog(status, certFingerprintHex, _client!.GetServerIdentityDisplay());
bool accept = dlg.ShowDialog(this) == DialogResult.OK;
_client.ConfirmServerIdentity(accept);
if (!accept) lblStatus.Text = "Server identity rejected.";
}
private void CleanupFailedAttempt()
{
_pumpTimer.Stop();
if (_client is not null)
{
_client.EventReceived -= OnEvent;
_client.Dispose();
_client = null;
}
SetBusy(false);
}
private void SetBusy(bool busy)
{
lstServers.Enabled = !busy;
btnConnect.Enabled = !busy && lstServers.SelectedItem is not null;
btnAddNew.Enabled = !busy;
btnEdit.Enabled = !busy && lstServers.SelectedItem is not null;
btnRemove.Enabled = !busy && lstServers.SelectedItem is not null;
}
protected override void OnFormClosing(FormClosingEventArgs e)
{
if (DialogResult != DialogResult.OK)
{
_pumpTimer.Stop();
if (_client is not null)
{
_client.EventReceived -= OnEvent;
_client.Dispose();
_client = null;
}
}
base.OnFormClosing(e);
}
}
@@ -0,0 +1,60 @@
namespace VoiceCat.App.Forms;
/// <summary>
/// A small reusable modal that prompts for a single line of text (e.g., kick/ban reason).
/// Returns DialogResult.OK with Text set, or DialogResult.Cancel.
/// </summary>
public sealed class InputDialog : Form
{
private readonly TextBox _txtInput;
public string TextValue => _txtInput.Text.Trim();
public InputDialog(string caption, string label, string defaultText = "")
{
var lbl = new Label
{
Text = label,
AutoSize = true,
Location = new Point(12, 12),
TabIndex = 0,
};
_txtInput = new TextBox
{
Text = defaultText,
Location = new Point(12, 36),
Size = new Size(360, 23),
TabIndex = 1,
};
var btnOk = new Button
{
Text = "&OK",
DialogResult = DialogResult.OK,
Location = new Point(216, 72),
Size = new Size(75, 27),
TabIndex = 2,
};
var btnCancel = new Button
{
Text = "&Cancel",
DialogResult = DialogResult.Cancel,
Location = new Point(297, 72),
Size = new Size(75, 27),
TabIndex = 3,
};
AcceptButton = btnOk;
CancelButton = btnCancel;
AutoScaleMode = AutoScaleMode.Font;
ClientSize = new Size(384, 112);
Controls.AddRange([lbl, _txtInput, btnOk, btnCancel]);
FormBorderStyle = FormBorderStyle.FixedDialog;
MaximizeBox = false;
MinimizeBox = false;
StartPosition = FormStartPosition.CenterParent;
Text = caption;
}
}

Some files were not shown because too many files have changed in this diff Show More