on_playback() was passing miniaudio's hardware playback-callback frame count to opus_decode()'s max_samples, instead of the decoder's fixed frame size (960 samples @ 20ms/48kHz). Since real packets decode to more samples than the (often smaller, e.g. ~480 on default low-latency WASAPI) hardware period, opus_decode returned OPUS_BUFFER_TOO_SMALL on nearly every callback -- packets were received/decrypted/jitter-buffered correctly but never decoded into audible PCM. Result: control-plane events and VAD worked, but zero audio in headphones. mix_for_test()'s white-box test masked this since it always called on_playback with frames == frame_samples, the one case where the bug is invisible. Fix: RemoteStream gained a small ring buffer (init_ring/push_ring/ pop_ring) that decouples decode cadence from playback-callback cadence. on_playback now tops the ring up by decoding whole Opus frames (always decoder.frame_samples(), never the hardware frame count) and drains exactly what the callback asks for, silence-padding (PLC) on underrun. Side effect: also fixes playout_ts, which was advancing by the wrong unit (hardware frames instead of decoded samples) -- it now tracks correctly against jitter-buffer timestamps. ctest --test-dir build/m1-dev: 12/12 green. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
27 KiB
PROGRESS — VoiceCat
Living status. Update this file in the same commit as your work so the next agent picks up instantly. Newest status at the top.
- Date convention: ISO (YYYY-MM-DD).
- Statuses:
[ ]not started ·[~]in progress ·[x]done.
▶ Where we left off / next action
- Done: Fixed silent-playback bug in
AudioEngine::on_playback(2026-06-16, found via live manual test: twovccli --voiceclients, control-plane events and VAD all correct, but zero audible output). Root cause:opus_decode()'smax_sampleswas being passed the hardware playback callback's frame count (miniaudio's own choice, frequently smaller than one Opus frame — e.g. ~480 samples on default low-latency WASAPI periods), instead of the decoder's fixed frame size (960 @ 20ms/48kHz). Since the real packet almost always decodes to more samples than that,opus_decodereturnedOPUS_BUFFER_TOO_SMALLon nearly every callback — frames were correctly received/decrypted/jitter-buffered, just never decoded into audible PCM.mix_for_test()'s white-box test masked this because it always calledon_playbackwithframes == frame_samples, the one case where the bug is invisible. Fix:RemoteStream(core/src/audio/audio_engine.h) gained a small ring buffer (init_ring/push_ring/pop_ring) that decouples decode cadence from playback-callback cadence —on_playback(core/src/audio/audio_engine.cpp) now tops the ring up by decoding whole Opus frames (decoder.frame_samples(), never the hardwareframes) and drains exactlyframessamples-per-channel from it each callback, silence-padding (PLC) on underrun. Also fixes a latentplayout_tsbug: it now advances by the actual decoded sample count per Opus frame, not by the hardware callback's (unrelated) frame count, which was the wrong unit for jitter-buffer timestamp comparisons.ctest --test-dir build/m1-dev— 12/12 green (run via PowerShell; Git Bash exec gotcha for these binaries, seedocs/building.md). Not yet confirmed audible by ear — pending the user re-running their live two-vcclitest. - Done: Post-M3 follow-up — device enumeration, VAD/PTT gate, stereo playback, WASAPI
loopback ✓ complete (2026-06-16). Closes all three items M3 explicitly carried forward as
out of scope (see the dated section below for the full file-by-file change list).
ctest --test-dir build/m1-dev— 12/12 tests green (3 consecutive full-suite runs), including the newtest_vad_ptt_devices(realvc_clients against a real server, plus a white-boxAudioEnginestereo-mix check — same ABI-level-coverage lesson as M2/M3). Manually verified live:vccli --list-devicesagainst real hardware, andvccli --voice --input-mode vadconnecting/streaming without incident. Still explicitly out of scope (carried forward, not silently dropped):- Real
webrtc-audio-processing/AEC — no working Windows/MSVC build upstream; v1 ships a lightweight energy/RMS VAD instead (see docs/roadmap.md §2, docs/voice.md §8/§11). There is no AEC, NS, or AGC implementation at all, not just a deferred VAD.vc_set_remote_stream(..., noise_reduction)'s per-stream NS toggle is unaffected by this pass and stays exactly as inert as it was after M3 (ApmPassthrough, no PCM modification). - macOS/iOS
SCREEN_AUDIOcapture (ScreenCaptureKit / ReplayKit) — this pass is Windows-only for real loopback capture; other platforms keepvc_test_inject_captureas the only way to feedSCREEN_AUDIO. - Process-specific WASAPI loopback — miniaudio's loopback mode captures the whole render endpoint (including this app's own incoming voice mix), not a single process.
- The pre-existing RT-thread rule violation in
on_capture_frame/AudioEngine::on_capture(mutex lock, heap allocation, blockingsendtoon the miniaudio real-time callback thread) — predates this work, documented but not fixed; fixing it needs the lock-free ring-buffer hand-offdocs/architecture.md §3specifies, a separate, larger refactor.
- Real
- Next: M4 — native clients (Windows C#, macOS/iOS Swift). See
docs/roadmap.md §M4.
Milestones (see docs/roadmap.md for full detail)
- M0 — Scaffolding ✓ complete
- M1 — Control plane ✓ complete (2026-06-15)
- M2 — Voice, single stream ✓ complete (2026-06-16)
- M3 — Multi-stream & per-channel tuning ✓ complete (2026-06-16)
- M4 — Native clients (Windows C#, macOS/iOS Swift) ← next
- M5 — Moderation, polish, beyond (perms, bans, DRED; then file transfer, E2EE, …)
M0 — Scaffolding ✓ (completed)
- Repo layout (
core/ server/ tools/ clients/ tests/), CMake + presets, vcpkg manifest. - C ABI header
core/include/voicecat.h(full surface, stubbed). - Protocol source-of-truth
core/proto/voicecat.proto(matches docs/protocol.md). - Core stubs for all six subsystems (net/crypto/codec/protocol/session/audio) +
vc_client. voicecat-server(arg parsing, config, stub run) andvccli(drives the C ABI).- CTest smoke test asserting the C ABI contract (not just "it compiles").
.gitattributes(LF),.gitignore,.clang-format, onboarding docs.- Verified:
cmake --preset dev && cmake --build --preset dev && ctest --preset dev→ green.
M1 — Control plane ✓ (completed 2026-06-15)
Exit criterion: ✓ test_m1_integration — two clients authenticate over TLS 1.3 (guest
- Argon2id password), exchange channel and private text messages. Passes in ~1 s.
- vcpkg baseline +
m1-devpreset;find_packagefor protobuf/mbedTLS/libsodium/asio/sqlite3. FrameCodecfeed + emit;encode_envelope/decode_envelope.- Asio TCP acceptor +
TcpServerConn(TLS path: blocking handshake thread +tls_read_loop). TlsContext(mbedTLS 1.3, server cert/identity, ECDSA-P256 self-signed, TOFU on client).WorkerPool(3 threads, used for Argon2id).Database— SQLite, Argon2id via libsodium,create_account/authenticate/ bootstrap admin.voicecat-admin— account add/reset/del/list against live DB file.ServerIdentityManager— generate/persist Ed25519 key + cert; fingerprint display.ConnSession— WaitingHello → WaitingAuth → Authenticated state machine; full protocol relay.SessionRegistry— channel tree, user map, broadcast, text routing.vc_client(client.cpp) — full M1 C ABI: connect/TLS/ClientHello/AuthRequest/text/disconnect.Server::run()— io_context, acceptor, worker pool, signal handling,on_readycallback.test_m1_integration— M1 exit criterion. Verified green 2026-06-15.
Key bug fixed: double-framing in ConnSession::send_envelope — encode_envelope was
pre-framing the protobuf, then TcpServerConn::send_frame re-framed it. Fixed by serializing
raw protobuf bytes directly and letting send_frame add the single [4-byte len] prefix.
M2 — Voice, single stream ✓ (completed 2026-06-16)
Exit criterion: ✓ test_m2_voice — two headless clients authenticate over TLS, bind UDP,
announce a MIC stream, send 50 encrypted Opus frames; server SFU relay re-encrypts + forwards
to the second client; B receives ≥ 25 frames and all decrypt correctly. Passes in ~4 s.
m2-devpreset (inheritsvcpkg-base, binaryDirbuild/m2-dev);m1-devalso builds all M2 code.core/CMakeLists.txt—find_package(Opus),find_path(MINIAUDIO_INCLUDE_DIR).core/src/net/voice_frame.h— 14-byte UDP header (type/flags/codec/ssrc/seq/ts), serialize/parse,make_udp_binding_packet.SodiumMediaCrypto— ChaCha20-Poly1305 AEAD; counter-nonce; 64-bit sliding-window anti-replay;derive_send/recvfrom TLS RFC 5705 exporter.OpusEncoder/OpusDecoder— libopus 1.6, FEC, DTX, PLC (free; nullptr → decoder extrapolates).UdpMediaChannel— async UDP socket (asio); thread-safesend_to; async recv loop.JitterBuffer— per-ssrc, EWMA jitter estimation, adaptive depth 20–200 ms, late-drop at 500 ms.AudioEngine— miniaudio capture+playback;inject_capture()bypass for headless tests; per-ssrc RemoteStream with OpusDecoder + JitterBuffer.ApmProcessor—ApmPassthroughstub (VAD always open); WebRTC APM deferred until M3.on_tls_readycallback inTcpChannelCallbacks— server derives and stores media AEAD keys immediately after TLS handshake.ConnSessionM2 —udp_tokengenerated at construction; included inAuthResult;handle_udp_binding(verifies token, TCP ack);handle_stream_announce(assigns SSRC via registry);udp_media_portinServerHello.SessionRegistryM2 —register_udp_token,find_by_udp_token,register_udp_endpoint,find_by_udp_endpoint,assign_ssrc,find_channel_sessions,user_channel.MediaRelay— SFU UDP relay;kFrameUdpBinding→ endpoint binding;kFrameVoice→ decrypt/re-encrypt/forward to channel members.Server::run()— creates and bindsMediaRelay; passes media port toConnSession; wireson_tls_readyto derive per-connection media AEAD keys.test_voice_frame— header round-trip, big-endian layout, binding packet format.test_media_aead— seal/open round-trip, anti-replay, tamper detection, multi-packet sequence.test_opus_codec— encode/decode round-trip energy check (within 3 dB), PLC, frame-samples helper.test_m2_voice— M2 exit criterion (raw-socket harness). Verified green 2026-06-16.
Follow-up (same day): the above made test_m2_voice pass, but vc_client's public voice
methods were still stubs — the actual M2 exit criterion ("two vccli/early-GUI clients talk")
wasn't met. Closed the gap:
core/src/core/client.cpp— realstream_start/stream_stop/set_self_mute/set_remote_stream; UDP-binding handshake (start_udp_binding/handle_udp_binding_ack/finish_udp_binding); media key derivation fromtls_(RFC 5705 exporter);run_udp_recv(AEAD-open →JitterBuffer::Frame→audio_engine_.push_recv_frame);on_capture_frame(encode → seal →sendto);sync_remote_streams(diffs aUserproto'sstreamsagainstremote_streams_, wiring upOpusDecoders and emittingSTREAM_STARTED/STOPPED).set_input_device/set_input_mode/set_push_to_talk/list_devicesremainVC_ERR_NOT_IMPLEMENTED— no device-enumeration backend yet; scoped to M3 (VAD/PTT).core/src/session/session.cpp/h—SessionModel::find_user,find_user_by_ssrc,Stream{stream_id, ssrc, kind, label, sample_rate, frame_ms}.server/src/conn_session.cpp/h—handle_stream_announce/handle_stream_stopnow broadcast viaSessionRegistry::set_user_stream/clear_user_stream→UserEvent::UPDATED.server/src/session_registry.cpp/h—set_user_stream/clear_user_stream(mutate a user'sStreamInfolist, return the updatedUserproto for broadcast).tests/test_voice_client_abi.cpp— drives two realvc_clientinstances throughvc_connect/vc_authenticate_guest/vc_stream_start/vc_stream_stop; asserts client B observes client A'sSTREAM_STARTED/STOPPEDevents. Verified green 2026-06-16.tools/vccli/src/main.cpp— argv parsing (--host/--port/--nick/--channel/--voice/ --mute/--text);--voicestarts a MIC stream and blocks on SIGINT, printingon_eventcallbacks live (unbuffered stdout — MinGW/MSVCRT treat_IOLBFas full buffering for non-console streams). Dropped the originally-planned--voice-loopbackand thetx=N rx=M lost=K jitter=Jstats line:voicecat.hexposes no PCM-injection hook or jitter/loss stats getter publicly, onlyon_event+on_level(RMS). Manually verified: twovccli --voiceinstances see each other's stream start in real time.
M3 — Multi-stream & per-channel tuning ✓ (completed 2026-06-16)
Exit criterion: ✓ test_m3_multistream — a real vc_client (A) runs two concurrent local
streams (MIC + SCREEN_AUDIO) with distinct stream ids; a second client (B) sees both as
separate STREAM_STARTED events and a VC_EVENT_TALK_STATE talking edge for A's MIC stream;
B independently sets gain/mute/noise-reduction on each of A's streams without one call
affecting the other; A then joins "Music Room" (channel 2: stereo/128kbps/OPUS_AUDIO/no
DTX) and announces a fresh MIC stream there, while B stays in "Lobby" (channel 1: mono/24kbps/
OPUS_VOIP/DTX on) — vc_get_stream_audio_config shows their effective Opus config differs
exactly as the server enforces per channel. Passes in ~2.4s; verified across 8 consecutive
standalone runs + 3 consecutive full-suite runs with no flakes.
Exploration before implementing turned up several bugs/gaps where the wire format already supported this milestone but the client/server logic didn't — these were fixed as part of M3, not treated as pre-existing-and-out-of-scope:
- Server
stream_idbug —handle_stream_announcealways wrotestream_id=1, so a second stream from the same user silently overwrote the first inSessionRegistry::set_user_stream's replace-by-id logic. Fixed with a per-session counter (ConnSession::next_stream_id_) +announced_stream_ids_(also now validated inhandle_stream_stop, rejecting stops for ids the session never announced). - Per-channel
AudioConfigwas modeled but never populated/enforced.SessionRegistry::init_default_channels()now seeds Lobby (id=1: mono, 24kbps,OPUS_VOIP, FEC+DTX on) and a new "Music Room" (id=2: stereo, 128kbps,OPUS_AUDIO, FEC+DTX off) with realAudioConfigs; newSessionRegistry::channel_audio_config(channel_id)accessor (there was no per-id channel getter before, onlychannel_snapshot()).handle_stream_announcenow treats the channel's config as authoritative (mode/frame_ms/application/fec/dtx/ complexity), clamping (not overriding)bitrate_bpsto the channel's ceiling. - Client silently dropped
mode/dtx/complexity/applicationfromeffective_audioeven for the single M2 stream —handle_stream_announce_resultandsync_remote_streamsonly copiedsample_rate/bitrate_bps/frame_ms/fecintoOpusParams. New sharedopus_params_from_audio_config()helper (client.cpp) fixes both the send and receive paths. core/src/codec/opus_codec.h/.cpp— newOpusApplicationenum +OpusParams::applicationfield;OpusEncoder::initnow honors it instead of hardcodingOPUS_APPLICATION_VOIP.core/src/session/session.h/.cpp—Streamstruct extended with the fullAudioConfig(mode/bitrate_bps/application/fec/expected_packet_loss/dtx/complexity), not just sample_rate/frame_ms;copy_streams()now copies all of it.core/src/core/client.h/.cpp— local-stream state is now astd::unordered_map<int, LocalStream>keyed byvc_stream_kind(one active stream per kind — MIC/SCREEN_AUDIO/ AUX_DEVICE are each singletons for a client), replacing the M2 single-stream fields.StreamAnnounce/StreamAnnounceResultround-trips are now correlated byrequest_id(already round-tripped on the wire; just wasn't read) viapending_announce_kind_, so multiple concurrent announces from one client resolve to the rightLocalStream.on_capture_frametakes akindparameter and upmixes mono capture to stereo (duplicate L=R) when a stream's channel config calls for it.vc_set_self_mute'smic_mutedonly gates theMICkind — a concurrentSCREEN_AUDIOshare keeps playing while muted.set_remote_streamnow actually wiresnoise_reductionthrough (previously parsed and discarded). Newrun_talk_timer()(a small dedicated thread, started alongside the UDP media path, never the miniaudio callback thread) polls both remote talk-state edges (AudioEngine::poll_talk_transitions()) and local capture-activity edges, emittingVC_EVENT_TALK_STATE.- Fixed a thread-join race in
teardown_voice()— it's called both fromrun_io()'s own cleanup and fromdisconnect(), on different threads; without serialization both could seeudp_thread_/talk_timer_thread_asjoinable()simultaneously and race tojoin()the samestd::thread(UB; surfaced as an intermittentstd::system_error: No such processunderctest). Added ateardown_mu_guard around the whole function. This pre-existed forudp_thread_alone (likely the same root cause as thetest_m1_integration/test_m2_voicecleanup-path flake noted in the M2 section above) — addingtalk_timer_thread_'s join just made it surface more often, so it was fixed properly here rather than carried forward again. core/src/audio/audio_engine.h/.cpp—CaptureCallbackgained akindparameter (the real miniaudio capture device is always taggedkind=0/MIC; a second concurrent local stream is fed via its owninject_capture(kind, ...)ring buffer —inject_taps_, keyed by kind — since there is only one real hardware capture device in M3). Fixed a buffer-sizing bug inon_playback's per-stream decode (opus_decode'sframe_sizeparameter is samples-per-channel, not total samples — the old code passedframes * params_.channels, which would have overflowed the decode buffer for any stereo stream). Stereo decoder output is downmixed (avg L/R) into the engine's mono mix accumulator immediately after decode.RemoteStreamgainedrecv_ns/noise_reduction_enabled(lazyApmProcessorinstantiation — freed on disable, so no separate instance cap is needed per the roadmap's guidance) andlast_voice_ms/talking(talk-indicator edge state, updated inpush_recv_frame); newset_stream_noise_reduction()andpoll_talk_transitions(). Note: untilVOICECAT_HAS_APMis wired to a real WebRTC APM build, the NS toggle is plumbed end-to-end but behaviorally a passthrough no-op (ApmPassthroughdoesn't touch PCM) — same situation send-side APM has been in since M2; M3's job was the plumbing, not the DSP backend.- New C ABI surface (
core/include/voicecat.h, additive only):vc_audio_configstruct +vc_get_stream_audio_config(c, user_id, stream_id, out)— the effective Opus config for a stream you own or a peer's, reading from the (now richer)LocalStream/session::Stream.vc_test_inject_capture(c, stream_id, pcm, samples)— clearly-marked test-only, forwards toAudioEngine::inject_capture, sotest_m3_multistreamcan drive two concurrent synthetic-audio streams through the real ABI without a microphone. tests/test_m3_multistream.cpp— the M3 exit criterion (ABI-level, mirrorstest_voice_client_abi.cpp's approach per the M2 lesson). Registered intests/CMakeLists.txt.
Explicitly out of scope for this pass (confirmed with the user before implementing):
vc_set_input_device/vc_set_input_mode/vc_set_push_to_talk/vc_list_devices(device enumeration + VAD/PTT input gate) — stillVC_ERR_NOT_IMPLEMENTED. These were mentioned as "scoped to M3" in the M2 follow-up notes above, but docs/roadmap.md's M3 bullets never actually listed them — deferred again, now tracked explicitly rather than implicitly.- Real WASAPI desktop-audio loopback capture for
SCREEN_AUDIO— the engine now supports feeding a second concurrent local stream viainject_capture, but only synthetic PCM is wired up; a real loopback capture device is a follow-up. - True stereo playback output —
AudioEngine's mixer/output device stays mono; stereo streams are downmixed after decode (see above). The Opus wire format itself is fully stereo-correct.
Post-M3 follow-up — device enumeration, VAD/PTT gate, stereo playback, WASAPI loopback ✓ (completed 2026-06-16)
Closes all three items the M3 section above explicitly carried forward as out of scope.
Exit verification: ctest --test-dir build/m1-dev — 12/12 tests green (3 consecutive
full-suite runs), including the new test_vad_ptt_devices (device enumeration + VAD/PTT gate
through real vc_clients against a real server, plus a white-box AudioEngine stereo-mix
check — no audio hardware needed for that last part). Also verified 5 consecutive standalone
runs of the new test alone, no flakes. Manually verified live on Windows: vccli --list-devices against real hardware (3 input / 4 output devices, correct is_default
flags), and vccli --voice --input-mode vad connecting + streaming without incident.
- Device enumeration (
vc_list_devices) —AudioEngine::enumerate_devices(bool capture)(static, works without a running engine — inits a throwawayma_contextviama_context_get_devices).device_id/vc_device.idis an opaque hex-encoded rawma_device_id(not the device name — names aren't guaranteed unique); documented as an internal contract callers must round-trip, never construct by hand.vc_client::list_devicesworks in any connection state (noVC_STATE_CONNECTEDgate) since device pickers need to populate pre-connect.vc_free_device_listis now a real free (was a no-op stub). - Input device selection (
vc_set_input_device) — stores the device id on the targetedLocalStream(new field); for the MIC stream, if the engine is already running, restarts it (stop()+ensure_audio_running()) to pick up the new device. Simplified: it restarts unconditionally rather than trying to detect whether the device id actually changed (AudioEnginehas no getter for "current device"). - VAD/PTT input gate (
vc_set_input_mode,vc_set_push_to_talk) — newEnergyVadProcessor(core/src/audio/apm_processor.cpp) implementing the existingApmProcessorinterface: energy/RMS threshold (default ~0.025 normalized) + hang-time (default 300 ms, matchingkTalkHangoverMs). New factoryApmProcessor::create_vad(), kept separate fromcreate()(which recv-side per-stream NS still uses, unaffected by this pass).vc_clientgainedcurrent_input_mode_/ptt_active_/mic_vad_; the gate is inserted inon_capture_frame, MIC-only —SCREEN_AUDIO/AUX_DEVICEalways bypass it (gating a desktop-audio share on the user's own voice activity would silently drop shared music/video audio).last_capture_ms(drives the talk indicator) is now updated after the gate check, not before, so a VAD/PTT-closed frame never shows as "talking".mic_vad_is constructed once the MIC stream'sStreamAnnounceResultlands (onio_thread_), not lazily inside the capture path. - True stereo playback —
AudioParams::channelssplit intocapture_channels(stays- and
playback_channels(now 2, unconditionally).AudioEngine::on_playbackno longer downmixes decoded stereo streams to mono before mixing — stereo decode output is mixed directly (L→L, R→R); mono decode output is upmixed (duplicated into both channels). Falls back to a 1-channel playback device once if the 2-channelma_device_initfails (unusual hardware). New test-onlyAudioEngine::mix_for_test()exposes the mixer for white-box testing without a realma_device.
- and
- WASAPI loopback capture for
SCREEN_AUDIO— newVOICECAT_HAS_LOOPBACKmacro (core/CMakeLists.txt, Windows-only).AudioEnginegained a separateloopback_device_(own lifecycle, decoupled from the mic capture/playback devices) withstart_loopback_capture()/stop_loopback_capture(), using miniaudio'sma_device_type_loopbackagainst the default render endpoint. Its callback feedscapture_cb_directly (same pattern as the real mic capture device), not throughinject_capture()'s test-only ring. Wired intovc_client::handle_stream_announce_result(start, alongsideensure_audio_running()) andstream_stop(stop) forVC_STREAM_SCREEN_AUDIO. Non-Windows builds keepvc_test_inject_captureas the only way to feedSCREEN_AUDIO. tools/vccli/src/main.cpp— new flags--list-devices,--input-device,--input-mode vad|ptt,--share-screen-audio; while--voiceis running, a background stdin-reader thread acceptsptt on/ptt off/mode vad/mode ptt(the most portable way to drive PTT interactively from a headless CLI — no SIGUSR1 equivalent on Windows). Also printsVC_EVENT_TALK_STATE. Known minor caveat: on Windows the stdin-reader thread is detached (not joined) on exit, sincestd::getlinecan't be interrupted from another thread — avc_client*use-after-free is theoretically possible if a command line arrives in the brief window between teardown and process exit; acceptable for a headless test/dev tool.tests/test_vad_ptt_devices.cpp— new test covering all four items above; registered intests/CMakeLists.txt.tests/test_smoke.cpp's device-list assertion is now conditional onVOICECAT_HAS_AUDIO(was a hardVC_ERR_NOT_IMPLEMENTEDassertion) —VC_OKonly, nevercount > 0(a headless CI build agent may legitimately report zero audio devices).
Still explicitly out of scope (carried forward, not silently dropped):
- Real
webrtc-audio-processing/AEC — no working Windows/MSVC build upstream (see docs/roadmap.md §2's superseding decision-log entry). There is no AEC, NS, or AGC implementation at all, not just a deferred VAD. The per-stream NS toggle (vc_set_remote_stream(..., noise_reduction)) is unaffected by this pass and stays exactly as inert as it was after M3 (ApmPassthrough, no PCM modification) — don't mistake this pass for having fixed it. - macOS/iOS
SCREEN_AUDIOcapture (ScreenCaptureKit / ReplayKit) — Windows-only loopback in this pass. - Process-specific WASAPI loopback — whole-device capture only; inherently captures this app's own incoming voice mix along with everything else playing.
- The pre-existing RT-thread rule violation in
on_capture_frame/AudioEngine::on_capture(mutex lock, heap allocation for the stereo-upmix path, blockingsendto, all on the miniaudio real-time callback thread) — predates this work (was already present in M2/M3); documented here explicitly rather than silently carried forward again. Fixing it properly needs the lock-free ring-buffer hand-offdocs/architecture.md §3specifies — a separate, larger refactor, out of scope for this pass.
Decisions log
All architecture/scope decisions are settled and recorded in
docs/roadmap.md §2 "Resolved decisions" and reflected across docs/.
If you make a new decision, record it there and link it here.
How to update this file
- Check off tasks as you complete them; flip a milestone to
[x]only when its exit criterion test passes. - Keep the "Where we left off / next action" block at the top accurate — it's the first thing the next agent reads.
- When you start a milestone, copy its task list from
docs/roadmap.mdinto a section here.