Adds vcpkg as a submodule at vcpkg/, pinned to the exact commit vcpkg.json
already declares as builtin-baseline, so the bundled checkout and the
manifest's resolved port versions can never drift apart.
cmake/voicecat-toolchain.cmake, scripts/common.sh, and
clients/apple/scripts/build-xcframework.sh now resolve vcpkg as:
VCPKG_ROOT env var (external checkout) > bundled submodule. Docs updated
to describe the new one-time setup.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Removes leftover debug scaffolding (stray Console.WriteLine/NSLog traces,
dead nick_buf_ptr, a no-op --print-config flag now implemented for real),
fixes stale/misleading comments (channel passwords are no longer a "future
M5+" feature, a wrong cross-reference, a stale TlsContext::close() mention,
an incomplete BanRecord::subject_type doc, and a smoke test pointing at a
build/m1-dev preset that no longer exists), strips internal M1-M5 milestone
jargon from comments now that the roadmap is done, trims comments that just
restated the following line, and consolidates a few "why" explanations that
were duplicated 2-3 times in the same file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds Assets.xcassets with a placeholder 1024x1024 AppIcon so Xcode
compiles CFBundleIconName and the required icon sizes into the bundle.
Replaces UILaunchStoryboardName (missing storyboard) with UILaunchScreen
dict, valid for iOS 14+ (min target is iOS 18). Fixes all four App Store
Connect validation errors blocking TestFlight upload.
Changes main app bundle ID from cat.voice.VoiceCatiOS, broadcast extension
from cat.voice.VoiceCatiOS.broadcast, and App Group from group.cat.voice.VoiceCat
to match the registered App Store identifier.
Recovery from the 99937c9 audio-device-change commit broadened the
route-change recovery set to "everything except categoryChange /
routeConfigurationChange", which added .override. But .override is
fired by our own applyA2dpSpeakerFallback() -> overrideOutputAudioPort,
which recoverAudio() calls on every recovery. On an A2DP preset
(Stereo Mic / Mono Mic), disconnecting AirPods ping-ponged:
oldDeviceUnavailable -> recoverAudio -> applyA2dpSpeakerFallback
-> overrideOutputAudioPort(.speaker) -> .override routeChange
-> recoverAudio -> applyConfiguration (setCategory resets override)
-> applyA2dpSpeakerFallback -> overrideOutputAudioPort -> .override -> ...
Each iteration also rebuilt the AVAudioEngine via reconfigure() ->
rebuild() -- the audible reinitialize loop + CPU spin. Voice Chat and
Built-in Mic + Speaker were unaffected (applyA2dpSpeakerFallback
early-returns for non-A2DP modes, so no overrideOutputAudioPort call).
Two-part fix (pure Swift iOS-app target; no C ABI / proto / docs changes):
1. AudioSessionManager.handleRouteChange: added .override to the skip
list alongside .categoryChange / .routeConfigurationChange. .override
is only ever fired by our own overrideOutputAudioPort call, so
treating it as a recovery reason is the loop by definition. The
AVAudioEngineConfigurationChange observer in IOSVoiceProcessingEngine
remains as the backstop if an override ever actually stops the engine.
2. IOSAudioRouter.applyA2dpSpeakerFallback: made idempotent via a
lastAppliedOutputOverride tracker that skips the redundant
overrideOutputAudioPort call when the desired state (.none for
external output present, .speaker otherwise) already matches. Reset
to nil at the top of applyConfiguration() (setCategory can reset the
override) and on a failed call. Defense-in-depth on top of fix 1.
Build: xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj
-scheme VoiceCatiOS -destination 'generic/platform=iOS' build green
(Xcode 26.5 / iOS 18.0).
Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired
headphones, AirPods) used to leave the iOS client in a dead/zombie state:
the engine went silent, no reconnect was attempted, and a live-session
disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout.
Reconnect (AppState.swift, SessionState.swift):
- Two-layer reconcile. Once SessionState overwrites client.onEvent at auth
success, AppState.handleConnectEvent no longer sees live-session events.
Added a weak SessionState.appState; SessionState.handleEvent .disconnected
calls appState.onLiveSessionDisconnected after the cue -- the single path
AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect
snapshots lastSession, stops audio, releases session/VoiceCatClient (io-
thread join via vc_client_destroy), resets the backoff, and arms
scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth
success via existing TOFU_MATCHED auto-confirm + idempotent join_channel).
- NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi
<-> cellular interface change or path .unsatisfied it calls
proactiveReconnect: tearing the session down BEFORE the C core notices the
dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff
tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature.
While mid-reconnect a .satisfied path resets the backoff for a fast retry.
- User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect
and cancel all reconnect state (task + monitor + lastSession + connectedServer).
Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift):
- Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/
.newDeviceAvailable route-change guard; fires on every externally-initiated
route change reason except the ones we cause ourselves (.categoryChange/
.routeConfigurationChange) to avoid a notification loop. Interruption-end
now always recovers instead of only when .shouldResume is set.
- Added AVAudioEngineConfigurationChange observer on the engine so a system
self-stop after our route-change handler wins the race is caught.
- IOSAudioEngine.rebuild() does a one-shot reactivation-retry on
engine.start() failure (iOS sometimes refuses until the session is
re-reactivated -- the silent-death case).
No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green
via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
PTT was focus-scoped (WinForms KeyDown/KeyUp) so it died the moment the
window lost focus. Add an optional system-wide path using the Raw Input
API (RegisterRawInputDevices + WM_INPUT with RIDEV_INPUTSINK) instead of
a WH_KEYBOARD_LL low-level hook -- the latter is the keylogger pattern AV
heuristics flag, which is worse for our unsigned MinGW binary. Raw Input
involves no DLL injection or global hook and passes keys through.
- New VoiceCat.App/Native/RawInput.cs: P/Invoke + structs; register the
keyboard sink, decode WM_INPUT to vkey/up-down, GetAsyncKeyState helper.
- MainForm overrides OnHandleCreated/OnHandleDestroyed/WndProc to manage
the sink and route WM_INPUT to PTT; gates the focus-scoped KeyDown/KeyUp
off when system-wide is on; makes the Deactivate force-release
conditional; adds a GetAsyncKeyState watchdog on the pump timer so a
missed key-up (RDP/lock-screen) cannot leave PTT stuck.
- VoiceSettings.SystemWidePtt (default ON) + system-wide checkbox in the
Audio settings PTT section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Suppress the system ding on global hotkeys/PTT and the user-list Enter
key by setting SuppressKeyPress (Handled alone leaves WM_CHAR to beep).
- Preserve the user-list keyboard selection across talking/mute refreshes
instead of resetting it on every Items.Clear().
- Include the connected server name in the main window titlebar.
- Close the private-message window on Escape.
- Show the PM window without an owner so focus is no longer trapped to it
and the main window can be worked in while a PM is open.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mic input gain (was 3x) and per-user receive gain (was 2x) had
asymmetric boost ceilings. Raise both, plus the desktop aux input
gain (was 3x), to a uniform 4x (400%) on macOS, iOS, and Windows.
The master Output volume slider is unchanged (still 1x). No core
changes needed: the C ABI only clamps negatives, so the ceilings
live entirely in the client UI sliders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The picker gated its list on visible-window processes plus audio sessions
on only the default render device. That both showed non-audio apps (any
window) and missed real ones (windowless or routed to a secondary device).
Process loopback targets a PID and its child tree regardless of whether the
app is currently playing, so the gate fought the capture layer.
Now enumerate every process in the user's interactive session (windowed or
not), deduped by executable with the windowed tree-root as the capture PID;
scan all active render endpoints to flag currently-playing apps with a > and
sort them first. Adds a filter box and persists checks across filtering.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both desktop mics were hard-mono: the core defaults capture_channels=1 and
neither client ever called vc_set_capture_channels (only iOS did). Add a
persisted "Stereo microphone" toggle to each client's Audio settings, applied
when the mic stream starts and live via vc_set_capture_channels + vc_audio_restart.
Expose both ABI calls in the Windows interop; the macOS wrapper already had them.
Core fix: encode_and_send_frame now folds a stereo mic frame to mono on a mono
channel - previously the channels==2 branch encoded interleaved L/R directly even
on a mono channel, feeding a mono opus_encode 2x its samples (wrong pitch/garbage).
Real stereo still only reaches the wire on a stereo channel; on a mono channel the
mic is cleanly downmixed.
Test: test_stereo_mic_mono_channel. ctest --preset dev green (28/28). Docs: voice.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On iOS the remote end heard the mic twice and crackly (BT headset + internal
mic in Voice Chat; mono + stereo copies of the internal mic in Stereo Mic).
Root cause is an io-thread ordering race. `ensure_audio_running()` only set
`external_capture` when a MIC stream already existed, but it also runs from
`sync_remote_streams` on the post-auth `ServerStateSnapshot` — before the user
joins voice. With `external_playback_` still false and no MIC stream,
`AudioEngine::start()` opened a real miniaudio capture device that stayed open
all session (later calls early-return on running()), racing the AVAudioEngine
input tap fed via `vc_stream_feed_pcm`. `on_capture_frame` then encoded+sent
both paths — the mic transmitted twice, the two unsynchronized capture clocks
producing the crackle.
- core (ensure_audio_running): force `external_capture = true` whenever
`external_playback_` is set, so iOS unified mode never opens a hardware
capture device. No-op on desktop.
- ios (AppState): move `setExternalPlayback(true)` before `connect()`, so the
flag is set before the io thread processes any message — closing the race.
Verified on device: remote end hears the iOS mic once and clean in both Voice
Chat (+ BT) and Stereo Mic.
The iOS mic was unusable — a consistent ~40-60ms flutter + volume fade
('slow fan') on every preset. The core sends each captured frame
synchronously (no send pacer), so packet cadence == capture cadence, and
the receiver's playout keeps near-zero buffering by design (its jitter
estimate keys off the regular sender timestamp, so it's blind to arrival
jitter). That's smooth only for a steady sender (desktop miniaudio =
steady 20ms); the iOS AVAudioEngine tap delivers ~2 frames per ~40ms
callback -> bursty -> receiver underruns -> PLC fade.
Fix (iOS-only): the mic tap writes converted 48kHz int16 to an SPSC ring;
a 20ms feed pump drains it and calls feedPcm at a steady cadence. The pump
primes a small prebuffer cushion (3 frames ~60ms, self-healing up to
~120ms on underrun) before releasing, so the tap's bursts can't drain it
to empty. Never reads a partial frame (read consumes what it returns ->
partials were the crackle), and rebuilds with the current channel count
each rebuild() (a frozen count fed mono-as-stereo = octave-up on a
Stereo->Voice Chat switch).
Trade-off: ~60-120ms added mic-send latency, unavoidable when de-bursting
for a near-zero-buffer receiver. PROGRESS.md notes the proper follow-up:
make the jitter buffer measure real RFC-3550 arrival jitter so the
receiver absorbs bursts itself.
Verified: xcodebuild Debug BUILD SUCCEEDED (iOS Simulator, arm64).
Both VoiceCatMac and VoiceCatiOS failed to link with 'Undefined symbols
for architecture arm64: _rnnoise_create/_rnnoise_destroy/_rnnoise_process_frame'.
Root cause: build-xcframework.sh merged vcpkg deps into the fat static lib but
NOT the locally-built vendored librnnoise.a (a CMake target from
third_party/rnnoise/ linked privately into voicecat via VOICECAT_HAS_NS — not a
vcpkg dep). The xcframework had been rebuilt after the RNNoise commit but still
omitted the symbols, so every slice's libvoicecat-fat.a referenced _rnnoise_*
with no defining object. The iOS slices were also stale (pre-rnnoise) and
absent from the xcframework entirely.
Fix: build-xcframework.sh now also collects .a files from build/<preset>/lib/
(excluding libvoicecat*) so vendored CMake-target static libs like librnnoise.a
get merged in. Future-proof: any new vendored static-lib target landing in
build/<preset>/lib/ is picked up automatically. README 'Fat static library'
section updated.
Verify: rebuilt VoiceCatCore.xcframework --all → all 3 slices (macos-arm64,
ios-arm64, ios-arm64-simulator) now carry the 10 _rnnoise_* symbols; fat lib
~30 MB → ~33 MB. xcodebuild Debug BUILD SUCCEEDED for VoiceCatMac, VoiceCatiOS
(iphonesimulator arm64), and VoiceCatiOS (iphoneos arm64). No core/ABI/proto
changes — xcframework artifact + build script only.
Expose the existing send-side vc_set_input_noise_reduction C ABI (MIC-only,
mono, LOCAL — denoises captured mic PCM before input gain and VAD/PTT gate)
as a persisted global toggle in each client's audio settings, applied live
and re-applied on Join Voice. Mirrors the existing mic-gain wiring pattern.
- Shared Swift (VoiceCatCore): add setInputNoiseReduction(_:) wrapper
- Windows: P/Invoke + SetInputNoiseReduction wrapper, MicNoiseReduction in
VoiceSettings, new checkbox in AudioSettingsForm (layout shifted +28px),
apply on Join Voice; also fix stale 'planned - currently passthrough'
label on the receive-side per-user NR checkbox (RNNoise now backs it)
- macOS: inputNoiseReduction state + UserDefaults in MainWindowController,
NR checkbox + nrChanged action in SettingsWindowController
- iOS: inputNoiseReduction in VoiceState + setter + restore in SessionState,
NR Toggle in SettingsView Voice section
Aux/screen are out of scope by design (core's NR guards kind == MIC). Apple
builds require a rebuilt VoiceCatCore.xcframework with VOICECAT_HAS_NS.
Lets a user transmit a second hardware input device (e.g. line-in / aux)
alongside the mic, with its own device picker and volume, from Audio Settings.
No core/ABI/proto changes: the aux is a VC_STREAM_AUX_DEVICE stream started
with external_feed=1 and fed via vc_stream_feed_pcm (the same external-feed
pipeline screen-audio uses). Per-kind local_streams_ already allows mic +
screen + one aux to coexist; volume is a client-side gain multiply (the core's
vc_set_input_gain is mic-only/global). Aux is always-on (core never gates
AUX_DEVICE on VAD/PTT) and is tied to the voice session.
Windows: new Audio/InputDeviceCapture.cs (WASAPI shared-mode capture from a
real input endpoint + capture-endpoint enumeration); aux section in
AudioSettingsForm.cs; lifecycle in MainForm.cs; persistence in VoiceSettings.cs.
macOS: new Audio/InputDeviceCapture.swift (AVAudioEngine input-node tap pinned
to the chosen Core Audio device + device enumeration by stable UID); aux section
in SettingsWindowController.swift; lifecycle + UserDefaults persistence in
MainWindowController.swift; file registered in project.pbxproj.
Windows verified (C# solution builds clean; aux confirmed working). macOS build
+ E2E pending a Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audio input settings (device picker, VAD/PTT mode, VAD sensitivity, mic gain,
PTT key) move from the always-visible bottom panel into Settings > Audio...,
matching the macOS/iOS pattern.
The device ComboBox now uses DataSource + DisplayMember="Name" instead of
Items.Add() with no DisplayMember — this fixes both the display bug (was
showing the full DeviceInfo record ToString()) and the NVDA silence on
dropdown open (DataSource binding exposes proper MSAA text per item).
Changes apply live for immediate feedback; Cancel reverts. VoiceSettings
gains InputDeviceId to persist the chosen device across sessions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- ContextMenuStrip: select first item on Opened so keyboard-invoked menus
(Shift+F10 / Apps key) raise the UIA focus event immediately instead of
staying silent until the first arrow key.
- Name the SplitContainer/SplitterPanel containers so screen readers announce
orientation instead of a stack of anonymous pane nodes.
- Take the resize splitters out of the Tab cycle (TabStop = false) so focus
moves control-to-control.
- RefreshChannelTree: restore keyboard focus and re-announce the current node
after a Nodes.Clear()/rebuild, so the tree no longer loses focus when
channels/users change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"All apps except selected" previously captured the complement of a frozen
app snapshot in INCLUDE mode (missed late-launched apps and system sounds,
wasted captures on silent windows). It now opens a single ProcessLoopbackCapture
in EXCLUDE mode (AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE)
of the one chosen app — true system-mix-minus-one, dynamic so apps launched
after sharing starts are included. The picker enforces single-selection in
exclude mode (the activation params take one target PID).
Adds an "Exclude VoiceCat's own audio (prevents echo)" checkbox (default on,
entire-desktop only) that routes the desktop capture through the same EXCLUDE
path targeting our own process id, killing the whole-device self-echo loop.
No C++/ABI changes. Updates voice.md and PROGRESS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two on-device bugs in the native iOS Voice-Processing path (Swift-only;
no core/ABI change).
1. Voice Chat (VPIO) silent playback: doStartMicStream() called
audioRestart() BEFORE startStream, so when the engine was already
running (a remote stream had started it) it reopened with
external_capture=false and opened a hardware miniaudio capture device.
The announce-result restart then early-returns (engine already running)
so that device was never dropped and fought the AVAudioEngine VPIO unit,
silencing playback. Now: setExternalPlayback first, then startStream
(stores external_feed synchronously), THEN audioRestart() — the core
reopens in full external mode (no hardware devices). Added VPIO
diagnostics: graph/route formats at start, ring written/read totals at
teardown.
2. Stereo Mic / Studio quiet earpiece: the .builtInMicBtA2dp presets omit
.defaultToSpeaker (it breaks A2DP) and skip forceSpeaker, so with no
Bluetooth connected output pinned to the quiet receiver. New
IOSAudioRouter.applyA2dpSpeakerFallback() overrides to the built-in
speaker when no external (A2DP/wired/AirPlay) output is present and
clears the override when one is — called after activation and on
device-change route changes.
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.
Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
AudioEngine opens no hardware playback device; a mixer-timer thread drives
on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
sink. start() also skips the hardware capture device when the MIC stream is
external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).
iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
presets; SessionState join/leave + reconcileVoicePath() switch paths;
Voice Chat defaults to speaker; Settings surfaces AEC/NS state.
Known: pending on-device verification; a few bugs to fix afterward.
Adds "only selected apps" and "all apps except selected" screen-audio
modes to the Windows client, alongside the existing entire-desktop path.
Per-app capture uses WASAPI process loopback (AUDCLNT_ACTIVATIONTYPE_
PROCESS_LOOPBACK) via ProcessLoopbackCapture, mixed by ProcessAudioMixer
and fed to the core through vc_stream_feed_pcm (external_feed=1 so the
core skips its own loopback device).
Init must pass AUDCLNT_STREAMFLAGS_LOOPBACK | EVENTCALLBACK |
AUTOCONVERTPCM; the LOOPBACK flag is what makes the virtual endpoint
deliver rendered audio (without it every buffer is flagged SILENT) and
AUTOCONVERTPCM resamples the app's native format to 48k s16.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a "speaker output" override so users can route audio to the built-in
speaker instead of the earpiece when no headphones/Bluetooth are connected.
Previously the receiver was the only fallback on the default Voice Chat preset.
The toggle inserts .defaultToSpeaker into the AVAudioSession category options
(skipped for the A2DP mode, where it would break Bluetooth routing). It yields
to connected BT/wired output and is orthogonal to preset matching. Exposed both
as a call-bar button in VoiceControlsView and a persisted Settings toggle.
Add scripts/dist-ios-adhoc.sh and scripts/asc_api.py to build an ad-hoc
signed IPA and the OTA web-install files (manifest.plist + index.html)
for sharing the iOS client with registered devices before TestFlight.
- asc_api.py: App Store Connect API helper (ES256 JWT via cryptography,
urllib) to register device UDIDs and list registered devices.
- dist-ios-adhoc.sh: registers UDIDs, builds the iOS device xcframework
slice, archives + exports with method=release-testing using
-allowProvisioningUpdates, and stages the install files into
dist/ios-adhoc/.
- Document the workflow in clients/apple/README.md; ignore __pycache__.
Let users choose what the SCREEN_AUDIO stream captures before sharing:
share everything, only selected apps, or all except selected apps, plus a
first-class "Exclude screen reader (VoiceOver) audio" toggle.
ScreenCaptureKit filters audio per application, so ScreenAudioCapture now
takes a ScreenAudioSelection and builds the matching SCContentFilter
(including:/excludingApplications:). New ScreenSharePickerSheet lists
running apps from SCShareableContent. iOS left untouched -- ReplayKit only
delivers the mixed system stream, so per-app filtering is impossible there.
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.
- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
(backward-compatible Codable), shown in Guest mode, wired into the guest auth
path -- guests could not set a display name on either before (only Windows)
Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
- Channels tab is now a drill-down on iPhone: ChannelBrowserView lists top-level
channels; ChannelDetailView shows the people in a channel, its sub-channels, and
an explicit Join button (with password prompt). iPad split view unchanged.
- Extract self-contained UserRow (context menu + sheets) from UserListView so admin
actions are reused in the drill-down.
- Fix off-screen chat compose box: pin VoiceControlsView via per-tab
.safeAreaInset(edge: .bottom) instead of a floating overlay, so it reserves layout
space above the tab bar (keeping the compose box visible, cooperating with keyboard
avoidance) without covering the tab bar buttons.
- Collapse Activity into Chat like macOS/Windows: ChatView renders a merged,
time-sorted timeline of messages + activity (activity rows in gray); remove the
Activity tab and ActivityLogView.
- Label the RPSystemBroadcastPickerView inner UIButton for VoiceOver
("Share/Stop sharing screen audio") instead of relying on an outer SwiftUI label.
Talon
