iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.
Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
AudioEngine opens no hardware playback device; a mixer-timer thread drives
on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
sink. start() also skips the hardware capture device when the MIC stream is
external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).
iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
presets; SessionState join/leave + reconcileVoicePath() switch paths;
Voice Chat defaults to speaker; Settings surfaces AEC/NS state.
Known: pending on-device verification; a few bugs to fix afterward.
- Channels tab is now a drill-down on iPhone: ChannelBrowserView lists top-level
channels; ChannelDetailView shows the people in a channel, its sub-channels, and
an explicit Join button (with password prompt). iPad split view unchanged.
- Extract self-contained UserRow (context menu + sheets) from UserListView so admin
actions are reused in the drill-down.
- Fix off-screen chat compose box: pin VoiceControlsView via per-tab
.safeAreaInset(edge: .bottom) instead of a floating overlay, so it reserves layout
space above the tab bar (keeping the compose box visible, cooperating with keyboard
avoidance) without covering the tab bar buttons.
- Collapse Activity into Chat like macOS/Windows: ChatView renders a merged,
time-sorted timeline of messages + activity (activity rows in gray); remove the
Activity tab and ActivityLogView.
- Label the RPSystemBroadcastPickerView inner UIButton for VoiceOver
("Share/Stop sharing screen audio") instead of relying on an outer SwiftUI label.
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
AudioSessionManager (AVAudioSession owner + interruption/route handling),
ServerListStore/SavedServer (App Group container + Keychain sharing),
and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)
C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.
XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.
Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
