fix(audio): stereo screen-audio loopback capture on Windows

start_loopback_capture hardcoded channels=1, forcing miniaudio to downmix the system's stereo mix to mono before the encoder saw it -- on_capture_frame then upmixed L=R to produce fake stereo. Now the loopback device opens in the channel's mode (stereo when the channel is stereo), CaptureCallback carries an explicit channels param so the encoder gets real interleaved L/R, and a mono fallback covers unusual render endpoints. New test_loopback_stereo_capture asserts L!=R end-to-end; 18/18 ctest green.
This commit is contained in:
2026-06-17 23:27:59 +02:00
parent a88656f2fa
commit cccf085a87
7 changed files with 258 additions and 39 deletions

View File

@@ -10,6 +10,39 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action
- **Done:** **Stereo screen-audio loopback capture on Windows** (2026-06-17). The WASAPI
loopback path (`start_loopback_capture`) used to hardcode `cfg.capture.channels = 1`,
downmixing the system's stereo mix to mono before the encoder ever saw it — so even on a
stereo channel, `SCREEN_AUDIO` was effectively mono (the encoder then upmixed L=R to
produce a *fake* stereo bitstream). Now the loopback device opens in the channel's mode:
stereo (interleaved L/R) when the channel is configured stereo, mono when mono. Real stereo
flows end-to-end through loopback → Opus encode → decode → stereo playback mixer.
- `audio_engine.h``CaptureCallback` gained an `int channels` parameter (the encoder
needs to know whether the PCM is real stereo or mono to avoid upmixing real stereo).
`start_loopback_capture(int kind)``start_loopback_capture(int kind, int channels)`.
New `loopback_channels_` member; new `feed_loopback_for_test` test hook (self-contained,
works on headless CI where the real WASAPI device can't init).
- `audio_engine.cpp``on_loopback` accumulator is now channel-aware (sized to
`frame_samples_*loopback_channels_`); `start_loopback_capture` sizes the accumulator off
the RT thread before `ma_device_start`, opens the device with `channels`, and falls back
to mono if the render endpoint rejects stereo (mirrors the playback path's fallback).
`on_capture`/`inject_capture`/`feed_capture_for_test` forward `channels` through the
callback (mic path always 1; loopback path 1 or 2).
- `client.cpp``on_capture_frame` takes `channels`; for `channels==2` (real stereo
loopback PCM) it encodes directly with no upmix; for `channels==1` on a stereo channel it
keeps the existing L=R upmix (mic stays mono in v1). `handle_stream_announce_result` reads
`effective_params.stereo` under the lock and passes `2` or `1` to `start_loopback_capture`.
- `test_vad_ptt_devices.cpp` — new `test_loopback_stereo_capture` behavior test: feeds a
loud-L / silent-R stereo signal through `feed_loopback_for_test`, encodes (as
`on_capture_frame` now does for `channels==2`), decodes, mixes, and asserts L≠R across
the frame (total_diff ~8.2M, well above the 960k threshold). A mono-downmixed-then-
upmixed bitstream would have L==R. Existing test lambda updated for the 4-arg callback.
- `docs/voice.md §8/§9` — diagram + notes updated: mic stays mono; `SCREEN_AUDIO` loopback
captures stereo when the channel is stereo.
- `cmake --build --preset m2-dev` + `ctest --preset m2-dev --parallel 1`**18/18 green**.
The `vad_ptt_devices` VAD-gate sub-test has a pre-existing parallel-run timing flake
(passes serially and in isolation); unrelated to this change (VAD gate logic is unchanged
for `channels==1`, the only path the MIC uses).
- **Done:** **Screen-audio sharing wired into the Windows WinForms client** (2026-06-17).
The core already fully supported `SCREEN_AUDIO` capture on Windows (post-M3 WASAPI
loopback via `VOICECAT_HAS_LOOPBACK`, always on for the `windows-client` preset —
@@ -337,8 +370,9 @@ not treated as pre-existing-and-out-of-scope:
`StreamAnnounce`/`StreamAnnounceResult` round-trips are now correlated by `request_id`
(already round-tripped on the wire; just wasn't read) via `pending_announce_kind_`, so
multiple concurrent announces from one client resolve to the right `LocalStream`.
`on_capture_frame` takes a `kind` parameter and upmixes mono capture to stereo (duplicate
L=R) when a stream's channel config calls for it. `vc_set_self_mute`'s `mic_muted` only
`on_capture_frame` takes a `kind` and `channels` parameter; for mono capture (`channels==1`)
on a stereo channel it upmixes L=R, and for real stereo capture (`channels==2`, the
`SCREEN_AUDIO` loopback path on a stereo channel) it encodes directly with no upmix. `vc_set_self_mute`'s `mic_muted` only
gates the `MIC` kind — a concurrent `SCREEN_AUDIO` share keeps playing while muted.
`set_remote_stream` now actually wires `noise_reduction` through (previously parsed and
discarded). New `run_talk_timer()` (a small dedicated thread, started alongside the UDP