Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:
1. configureStereoCapture now calls setPreferredInput +
setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
Previously omitted based on incorrect diagnosis that
setPreferredInput collapsed A2DP — the real culprit was
setPreferredInputNumberOfChannels(2), which neither project uses.
2. New C ABI: vc_audio_restart (full stop + re-init, unlike
suspend/resume which only stop/start). Swift wrapper added.
The withAudioSuspend wrapper that used it was removed after
on-device testing showed it killed all audio (including
VoiceOver) when switching presets — the core's
set_capture_channels handles engine restart internally.
3. Bluetooth options: Voice Chat preset now includes BOTH
.allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.
4. Capture channels now reset when switching stereo→mono via
selectCaptureChannels/applyPreset. AudioSessionManager tracks
activeMicStreamId (set by SessionState on join/leave voice).
5. Docs synced: voice.md, tech-stack.md, architecture.md,
PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
references.
Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
Full SwiftUI app at clients/apple/iOS/VoiceCatiOS.xcodeproj:
- 24 Swift source files: AppState + SessionState (@Observable @MainActor),
AudioSessionManager (AVAudioSession owner + interruption/route handling),
ServerListStore/SavedServer (App Group container + Keychain sharing),
and 14 SwiftUI views covering the full feature set
- NavigationSplitView on iPad, TabView on iPhone (horizontalSizeClass)
- Channel tree via OutlineGroup, user list with context menu admin actions
- PTT via DragGesture(minimumDistance: 0) + @GestureState
- onEvent closures hop to MainActor via Task { @MainActor in ... }
- App Group: group.cat.voice.VoiceCat (shared with future ReplayKit extension)
C ABI: add vc_audio_suspend / vc_audio_resume (AudioEngine::suspend/resume)
called by AudioSessionManager on AVAudioSession interruption events.
XCFramework: add ios-arm64 and ios-arm64-simulator slices to build-xcframework.sh;
Package.swift gains .iOS(.v17) platform; CMakePresets.json adds apple-ios /
apple-ios-sim presets with arm64-ios / arm64-ios-simulator vcpkg triplets.
Verified: xcodebuild -target VoiceCatiOS -sdk iphonesimulator26.5 BUILD SUCCEEDED.
Rationalize the preset set to match the project's actual state (past M5):
- Rename dev->skeleton (no-deps stub smoke), m1-dev->dev (default dev preset)
- Drop m2-dev (cache-identical to m1-dev)
- Add release preset (optimized + tests on, symbols kept)
- Strip server-release binaries (-s linker flag)
- Add apple-dev/apple-ios/apple-ios-sim scaffolding presets for XCFramework
Add cmake/voicecat-toolchain.cmake wrapper that auto-resolves the vcpkg
triplet from the host platform (x64-mingw-static/x64-linux/arm64-osx) so
the main presets work on Windows/Linux/macOS without per-OS variants.
Update all docs (building.md, CLAUDE.md, README.md, AGENTS.md, deployment.md,
tech-stack.md, client READMEs) and stale preset-name references in code
comments. No C++ behavior changes — the core was already portable.
PerUserTuningDialog previously broadcast one gain/mute/NR set to *all* of a
user's streams, even though the core mixer (AudioEngine::RemoteStream) and
the C ABI (vc_set_remote_stream) were already per-stream. The UI had no
per-mix controls anywhere.
Reworks the dialog to enumerate ListUserStreams on open and render one row
per stream (kind + label + Gain + Mute + NR), each wiring only to its own
stream_id. Adds a read-back ABI counterpart, vc_get_remote_stream, so the
dialog opens at the listener's actual current per-stream settings (defaults
1.0/unmuted/NR-off) rather than always 100%. Additive ABI change only; no
existing symbols touched.
Tests: test_m3_multistream extended with getter round-trip assertions; new
C# smoke test exercises the full P/Invoke marshaling path with two clients.
Docs: voice.md §10 notes the getter. NR checkbox keeps its honest
'passthrough' label (NS DSP still unbuilt per §8).
- docs/building.md: explains what each CMake preset (dev, m1-dev, m2-dev,
server-release) is actually for, and how to build voicecat-server + vccli
for manual testing. Linked from CLAUDE.md's doc index.
- core/include/voicecat.h, core/src/voicecat.cpp, core/src/protocol/protocol.h,
server/src/main.cpp: doc-header comments still claimed M0-skeleton/stub
behavior (VC_ERR_NOT_IMPLEMENTED everywhere, "prints what it would do",
protobuf codegen "commented") that M1-M3 made real. Updated to describe
current behavior, with the dev-preset stub fallback noted explicitly where
it still applies.
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Turn the design into a buildable, dependency-free M0 skeleton plus the
onboarding layer so a new agent can pick up instantly.
Build system:
- CMake + CMakePresets (dev = no deps; server-release = vcpkg) + vcpkg.json
- Skeleton builds with just a C++20 compiler; deps stay off until needed
- .gitattributes (LF), .gitignore, .clang-format
Core (libvoicecat):
- core/include/voicecat.h: full C ABI (the client/server contract), stubbed
- core/proto/voicecat.proto: control-plane wire format, matches docs/protocol.md
- src/{net,crypto,codec,protocol,session,audio,core}: subsystem stubs that
return VC_ERR_NOT_IMPLEMENTED, each pointing to its design doc
- server/ (voicecat-server) and tools/vccli/ link the core
- tests/: CTest smoke test asserting the C ABI contract (behavior, not just build)
- clients/{apple,windows}: M4 placeholders
Onboarding for agents:
- CLAUDE.md: hub — build/test commands, architecture at a glance, doc map, rules
- AGENTS.md: working method (behavior-driven; clean compile is the floor not the goal)
- PROGRESS.md: living tracker — M0 done, M1 task checklist, "where we left off"
Verified: cmake --preset dev && cmake --build --preset dev && ctest --preset dev → green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
