Proposal for replacing the C++ core, C++ server, and Swift macOS/iOS
clients with a single .NET 10 / C# codebase.
Covers the dependency map (8 vcpkg deps + 1 vendored -> 3 native libs),
the real-time-audio design, per-client strategy, a test-porting plan for
all 29 ctest cases, doc-sync work, an 11-phase migration, and a risk
register.
Two findings drive the shape of the plan:
- SslStream has no RFC 5705 keying-material exporter, which the media
AEAD key derivation depends on (docs/security.md 2). The API is an
unapproved proposal and SChannel structurally cannot export secrets.
Recommends BouncyCastle's managed TLS 1.3 stack, which does implement
the exporter and keeps the wire format byte-compatible with the C++
implementation -- so the existing tree stays usable as a conformance
oracle throughout the port. Protocol-v3 in-band media keys documented
as the fallback.
- The iOS ReplayKit broadcast upload extension stays in Swift: 50 MB
jetsam cap plus an unsupported extension type in .NET for iOS, and it
already doesn't link the core. Leaves one Swift file plus the shared
App Group ring.
Indexed in docs/README.md. Nothing here is implemented yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds vcpkg as a submodule at vcpkg/, pinned to the exact commit vcpkg.json
already declares as builtin-baseline, so the bundled checkout and the
manifest's resolved port versions can never drift apart.
cmake/voicecat-toolchain.cmake, scripts/common.sh, and
clients/apple/scripts/build-xcframework.sh now resolve vcpkg as:
VCPKG_ROOT env var (external checkout) > bundled submodule. Docs updated
to describe the new one-time setup.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Removes leftover debug scaffolding (stray Console.WriteLine/NSLog traces,
dead nick_buf_ptr, a no-op --print-config flag now implemented for real),
fixes stale/misleading comments (channel passwords are no longer a "future
M5+" feature, a wrong cross-reference, a stale TlsContext::close() mention,
an incomplete BanRecord::subject_type doc, and a smoke test pointing at a
build/m1-dev preset that no longer exists), strips internal M1-M5 milestone
jargon from comments now that the roadmap is done, trims comments that just
restated the following line, and consolidates a few "why" explanations that
were duplicated 2-3 times in the same file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds Assets.xcassets with a placeholder 1024x1024 AppIcon so Xcode
compiles CFBundleIconName and the required icon sizes into the bundle.
Replaces UILaunchStoryboardName (missing storyboard) with UILaunchScreen
dict, valid for iOS 14+ (min target is iOS 18). Fixes all four App Store
Connect validation errors blocking TestFlight upload.
Changes main app bundle ID from cat.voice.VoiceCatiOS, broadcast extension
from cat.voice.VoiceCatiOS.broadcast, and App Group from group.cat.voice.VoiceCat
to match the registered App Store identifier.
Drop the M0 no-deps skeleton preset and all VOICECAT_HAS_NET/AUDIO/OPUS/NS
guards that it required. Every subsystem is fully implemented; the stub
#else paths were dead code that added noise to every header and source file.
- CMakePresets.json: remove skeleton configure/build/test entries
- CMakeLists.txt (root/core/tests): remove VOICECAT_USE_VCPKG_DEPS option
and guards; all targets now build unconditionally
- 17 C++ source files: unwrap HAS_* guards, delete stub #else blocks
- apm_processor.cpp: delete ApmPassthrough no-op class; create() always
returns RnnoiseProcessor
- 18 test files: remove HAS_* guards and stub int main() skip bodies
- docs/building.md: remove skeleton from preset table and prose
VOICECAT_HAS_LOOPBACK (Windows WASAPI loopback platform gate) unchanged.
29/29 ctest green.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Recovery from the 99937c9 audio-device-change commit broadened the
route-change recovery set to "everything except categoryChange /
routeConfigurationChange", which added .override. But .override is
fired by our own applyA2dpSpeakerFallback() -> overrideOutputAudioPort,
which recoverAudio() calls on every recovery. On an A2DP preset
(Stereo Mic / Mono Mic), disconnecting AirPods ping-ponged:
oldDeviceUnavailable -> recoverAudio -> applyA2dpSpeakerFallback
-> overrideOutputAudioPort(.speaker) -> .override routeChange
-> recoverAudio -> applyConfiguration (setCategory resets override)
-> applyA2dpSpeakerFallback -> overrideOutputAudioPort -> .override -> ...
Each iteration also rebuilt the AVAudioEngine via reconfigure() ->
rebuild() -- the audible reinitialize loop + CPU spin. Voice Chat and
Built-in Mic + Speaker were unaffected (applyA2dpSpeakerFallback
early-returns for non-A2DP modes, so no overrideOutputAudioPort call).
Two-part fix (pure Swift iOS-app target; no C ABI / proto / docs changes):
1. AudioSessionManager.handleRouteChange: added .override to the skip
list alongside .categoryChange / .routeConfigurationChange. .override
is only ever fired by our own overrideOutputAudioPort call, so
treating it as a recovery reason is the loop by definition. The
AVAudioEngineConfigurationChange observer in IOSVoiceProcessingEngine
remains as the backstop if an override ever actually stops the engine.
2. IOSAudioRouter.applyA2dpSpeakerFallback: made idempotent via a
lastAppliedOutputOverride tracker that skips the redundant
overrideOutputAudioPort call when the desired state (.none for
external output present, .speaker otherwise) already matches. Reset
to nil at the top of applyConfiguration() (setCategory can reset the
override) and on a failed call. Defense-in-depth on top of fix 1.
Build: xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj
-scheme VoiceCatiOS -destination 'generic/platform=iOS' build green
(Xcode 26.5 / iOS 18.0).
Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired
headphones, AirPods) used to leave the iOS client in a dead/zombie state:
the engine went silent, no reconnect was attempted, and a live-session
disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout.
Reconnect (AppState.swift, SessionState.swift):
- Two-layer reconcile. Once SessionState overwrites client.onEvent at auth
success, AppState.handleConnectEvent no longer sees live-session events.
Added a weak SessionState.appState; SessionState.handleEvent .disconnected
calls appState.onLiveSessionDisconnected after the cue -- the single path
AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect
snapshots lastSession, stops audio, releases session/VoiceCatClient (io-
thread join via vc_client_destroy), resets the backoff, and arms
scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth
success via existing TOFU_MATCHED auto-confirm + idempotent join_channel).
- NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi
<-> cellular interface change or path .unsatisfied it calls
proactiveReconnect: tearing the session down BEFORE the C core notices the
dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff
tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature.
While mid-reconnect a .satisfied path resets the backoff for a fast retry.
- User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect
and cancel all reconnect state (task + monitor + lastSession + connectedServer).
Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift):
- Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/
.newDeviceAvailable route-change guard; fires on every externally-initiated
route change reason except the ones we cause ourselves (.categoryChange/
.routeConfigurationChange) to avoid a notification loop. Interruption-end
now always recovers instead of only when .shouldResume is set.
- Added AVAudioEngineConfigurationChange observer on the engine so a system
self-stop after our route-change handler wins the race is caught.
- IOSAudioEngine.rebuild() does a one-shot reactivation-retry on
engine.start() failure (iOS sometimes refuses until the session is
re-reactivated -- the silent-death case).
No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green
via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
PTT was focus-scoped (WinForms KeyDown/KeyUp) so it died the moment the
window lost focus. Add an optional system-wide path using the Raw Input
API (RegisterRawInputDevices + WM_INPUT with RIDEV_INPUTSINK) instead of
a WH_KEYBOARD_LL low-level hook -- the latter is the keylogger pattern AV
heuristics flag, which is worse for our unsigned MinGW binary. Raw Input
involves no DLL injection or global hook and passes keys through.
- New VoiceCat.App/Native/RawInput.cs: P/Invoke + structs; register the
keyboard sink, decode WM_INPUT to vkey/up-down, GetAsyncKeyState helper.
- MainForm overrides OnHandleCreated/OnHandleDestroyed/WndProc to manage
the sink and route WM_INPUT to PTT; gates the focus-scoped KeyDown/KeyUp
off when system-wide is on; makes the Deactivate force-release
conditional; adds a GetAsyncKeyState watchdog on the pump timer so a
missed key-up (RDP/lock-screen) cannot leave PTT stuck.
- VoiceSettings.SystemWidePtt (default ON) + system-wide checkbox in the
Audio settings PTT section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Suppress the system ding on global hotkeys/PTT and the user-list Enter
key by setting SuppressKeyPress (Handled alone leaves WM_CHAR to beep).
- Preserve the user-list keyboard selection across talking/mute refreshes
instead of resetting it on every Items.Clear().
- Include the connected server name in the main window titlebar.
- Close the private-message window on Escape.
- Show the PM window without an owner so focus is no longer trapped to it
and the main window can be worked in while a PM is open.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mic input gain (was 3x) and per-user receive gain (was 2x) had
asymmetric boost ceilings. Raise both, plus the desktop aux input
gain (was 3x), to a uniform 4x (400%) on macOS, iOS, and Windows.
The master Output volume slider is unchanged (still 1x). No core
changes needed: the C ABI only clamps negatives, so the ceilings
live entirely in the client UI sliders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The static-MinGW recipe (-static-libgcc -static-libstdc++ -static) was
only applied inside the VOICECAT_BUILD_SHARED branch in core/CMakeLists,
so it covered voicecat.dll but not voicecat-server.exe / vccli. With the
x64-mingw-static triplet only vcpkg's own deps are static; the GCC/MinGW
runtime stays dynamic, so the server failed to start on clean Windows
machines with missing libgcc_s_seh-1.dll / libwinpthread-1.dll /
libstdc++-6.dll.
Move the recipe to a global add_link_options (WIN32 AND MINGW) so every
produced binary embeds the runtime. Verified with objdump -p: only
Windows system DLLs remain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The picker gated its list on visible-window processes plus audio sessions
on only the default render device. That both showed non-audio apps (any
window) and missed real ones (windowless or routed to a secondary device).
Process loopback targets a PID and its child tree regardless of whether the
app is currently playing, so the gate fought the capture layer.
Now enumerate every process in the user's interactive session (windowed or
not), deduped by executable with the windowed tree-root as the capture PID;
scan all active render endpoints to flag currently-playing apps with a > and
sort them first. Adds a filter box and persists checks across filtering.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OPUS_SET_DRED_DURATION was hardcoded to 2 (20 ms), meaning DRED only
covered 1/3 of a lost 60 ms frame and was useless above 20 ms channels.
Now computed as max(2, ceil(frame_ms/10)) so DRED always embeds enough
redundancy to reconstruct one full previous frame regardless of frame size.
The floor of 2 preserves two-frame burst-loss coverage at 10 ms channels.
Decoder side and server are unaffected (server relays payloads verbatim).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The per-listener noise-reduction toggle (vc_set_remote_stream) did nothing
on Windows/macOS/iOS. The decode loop gated the RNNoise pass on
dec_channels == 1 as a proxy for "this stream is voice" (assuming
stereo => screen-share). The stereo-mic capture commit broke that: a stereo
mic with send-side NR off transmits stereo Opus, so the receiver decoded
two channels and skipped NR entirely. gain/mute have no channel guard, which
is why only NR appeared broken.
Thread the stream kind through init_recv_stream into RemoteStream::is_voice
(set from si.kind() == STREAM_MIC), gate receive NR on is_voice instead of
channel count, and fold a stereo voice frame to mono -> denoise -> duplicate
back across both channels in place (symmetric with the send-side downmix;
RNNoise is mono-only). Screen-audio shares are never denoised.
New test test_recv_noise_reduction drives AudioEngine and asserts a stereo
voice stream's noise floor collapses with NR on (RMS 1046 -> 0.1) while a
screen-audio share stays unchanged. ctest --preset dev green 29/29.
Docs: voice.md section 10. Shared-core fix; clients need only a rebuild.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both desktop mics were hard-mono: the core defaults capture_channels=1 and
neither client ever called vc_set_capture_channels (only iOS did). Add a
persisted "Stereo microphone" toggle to each client's Audio settings, applied
when the mic stream starts and live via vc_set_capture_channels + vc_audio_restart.
Expose both ABI calls in the Windows interop; the macOS wrapper already had them.
Core fix: encode_and_send_frame now folds a stereo mic frame to mono on a mono
channel - previously the channels==2 branch encoded interleaved L/R directly even
on a mono channel, feeding a mono opus_encode 2x its samples (wrong pitch/garbage).
Real stereo still only reaches the wire on a stereo channel; on a mono channel the
mic is cleanly downmixed.
Test: test_stereo_mic_mono_channel. ctest --preset dev green (28/28). Docs: voice.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On iOS the remote end heard the mic twice and crackly (BT headset + internal
mic in Voice Chat; mono + stereo copies of the internal mic in Stereo Mic).
Root cause is an io-thread ordering race. `ensure_audio_running()` only set
`external_capture` when a MIC stream already existed, but it also runs from
`sync_remote_streams` on the post-auth `ServerStateSnapshot` — before the user
joins voice. With `external_playback_` still false and no MIC stream,
`AudioEngine::start()` opened a real miniaudio capture device that stayed open
all session (later calls early-return on running()), racing the AVAudioEngine
input tap fed via `vc_stream_feed_pcm`. `on_capture_frame` then encoded+sent
both paths — the mic transmitted twice, the two unsynchronized capture clocks
producing the crackle.
- core (ensure_audio_running): force `external_capture = true` whenever
`external_playback_` is set, so iOS unified mode never opens a hardware
capture device. No-op on desktop.
- ios (AppState): move `setExternalPlayback(true)` before `connect()`, so the
flag is set before the io thread processes any message — closing the race.
Verified on device: remote end hears the iOS mic once and clean in both Voice
Chat (+ BT) and Stereo Mic.
The iOS mic was unusable — a consistent ~40-60ms flutter + volume fade
('slow fan') on every preset. The core sends each captured frame
synchronously (no send pacer), so packet cadence == capture cadence, and
the receiver's playout keeps near-zero buffering by design (its jitter
estimate keys off the regular sender timestamp, so it's blind to arrival
jitter). That's smooth only for a steady sender (desktop miniaudio =
steady 20ms); the iOS AVAudioEngine tap delivers ~2 frames per ~40ms
callback -> bursty -> receiver underruns -> PLC fade.
Fix (iOS-only): the mic tap writes converted 48kHz int16 to an SPSC ring;
a 20ms feed pump drains it and calls feedPcm at a steady cadence. The pump
primes a small prebuffer cushion (3 frames ~60ms, self-healing up to
~120ms on underrun) before releasing, so the tap's bursts can't drain it
to empty. Never reads a partial frame (read consumes what it returns ->
partials were the crackle), and rebuilds with the current channel count
each rebuild() (a frozen count fed mono-as-stereo = octave-up on a
Stereo->Voice Chat switch).
Trade-off: ~60-120ms added mic-send latency, unavoidable when de-bursting
for a near-zero-buffer receiver. PROGRESS.md notes the proper follow-up:
make the jitter buffer measure real RFC-3550 arrival jitter so the
receiver absorbs bursts itself.
Verified: xcodebuild Debug BUILD SUCCEEDED (iOS Simulator, arm64).
Both VoiceCatMac and VoiceCatiOS failed to link with 'Undefined symbols
for architecture arm64: _rnnoise_create/_rnnoise_destroy/_rnnoise_process_frame'.
Root cause: build-xcframework.sh merged vcpkg deps into the fat static lib but
NOT the locally-built vendored librnnoise.a (a CMake target from
third_party/rnnoise/ linked privately into voicecat via VOICECAT_HAS_NS — not a
vcpkg dep). The xcframework had been rebuilt after the RNNoise commit but still
omitted the symbols, so every slice's libvoicecat-fat.a referenced _rnnoise_*
with no defining object. The iOS slices were also stale (pre-rnnoise) and
absent from the xcframework entirely.
Fix: build-xcframework.sh now also collects .a files from build/<preset>/lib/
(excluding libvoicecat*) so vendored CMake-target static libs like librnnoise.a
get merged in. Future-proof: any new vendored static-lib target landing in
build/<preset>/lib/ is picked up automatically. README 'Fat static library'
section updated.
Verify: rebuilt VoiceCatCore.xcframework --all → all 3 slices (macos-arm64,
ios-arm64, ios-arm64-simulator) now carry the 10 _rnnoise_* symbols; fat lib
~30 MB → ~33 MB. xcodebuild Debug BUILD SUCCEEDED for VoiceCatMac, VoiceCatiOS
(iphonesimulator arm64), and VoiceCatiOS (iphoneos arm64). No core/ABI/proto
changes — xcframework artifact + build script only.
Talon
