fix(audio): stereo screen-audio loopback capture on Windows
start_loopback_capture hardcoded channels=1, forcing miniaudio to downmix the system's stereo mix to mono before the encoder saw it -- on_capture_frame then upmixed L=R to produce fake stereo. Now the loopback device opens in the channel's mode (stereo when the channel is stereo), CaptureCallback carries an explicit channels param so the encoder gets real interleaved L/R, and a mono fallback covers unusual render endpoints. New test_loopback_stereo_capture asserts L!=R end-to-end; 18/18 ctest green.
This commit is contained in:
38
PROGRESS.md
38
PROGRESS.md
@@ -10,6 +10,39 @@ up instantly. Newest status at the top.
|
||||
|
||||
## ▶ Where we left off / next action
|
||||
|
||||
- **Done:** **Stereo screen-audio loopback capture on Windows** (2026-06-17). The WASAPI
|
||||
loopback path (`start_loopback_capture`) used to hardcode `cfg.capture.channels = 1`,
|
||||
downmixing the system's stereo mix to mono before the encoder ever saw it — so even on a
|
||||
stereo channel, `SCREEN_AUDIO` was effectively mono (the encoder then upmixed L=R to
|
||||
produce a *fake* stereo bitstream). Now the loopback device opens in the channel's mode:
|
||||
stereo (interleaved L/R) when the channel is configured stereo, mono when mono. Real stereo
|
||||
flows end-to-end through loopback → Opus encode → decode → stereo playback mixer.
|
||||
- `audio_engine.h` — `CaptureCallback` gained an `int channels` parameter (the encoder
|
||||
needs to know whether the PCM is real stereo or mono to avoid upmixing real stereo).
|
||||
`start_loopback_capture(int kind)` → `start_loopback_capture(int kind, int channels)`.
|
||||
New `loopback_channels_` member; new `feed_loopback_for_test` test hook (self-contained,
|
||||
works on headless CI where the real WASAPI device can't init).
|
||||
- `audio_engine.cpp` — `on_loopback` accumulator is now channel-aware (sized to
|
||||
`frame_samples_*loopback_channels_`); `start_loopback_capture` sizes the accumulator off
|
||||
the RT thread before `ma_device_start`, opens the device with `channels`, and falls back
|
||||
to mono if the render endpoint rejects stereo (mirrors the playback path's fallback).
|
||||
`on_capture`/`inject_capture`/`feed_capture_for_test` forward `channels` through the
|
||||
callback (mic path always 1; loopback path 1 or 2).
|
||||
- `client.cpp` — `on_capture_frame` takes `channels`; for `channels==2` (real stereo
|
||||
loopback PCM) it encodes directly with no upmix; for `channels==1` on a stereo channel it
|
||||
keeps the existing L=R upmix (mic stays mono in v1). `handle_stream_announce_result` reads
|
||||
`effective_params.stereo` under the lock and passes `2` or `1` to `start_loopback_capture`.
|
||||
- `test_vad_ptt_devices.cpp` — new `test_loopback_stereo_capture` behavior test: feeds a
|
||||
loud-L / silent-R stereo signal through `feed_loopback_for_test`, encodes (as
|
||||
`on_capture_frame` now does for `channels==2`), decodes, mixes, and asserts L≠R across
|
||||
the frame (total_diff ~8.2M, well above the 960k threshold). A mono-downmixed-then-
|
||||
upmixed bitstream would have L==R. Existing test lambda updated for the 4-arg callback.
|
||||
- `docs/voice.md §8/§9` — diagram + notes updated: mic stays mono; `SCREEN_AUDIO` loopback
|
||||
captures stereo when the channel is stereo.
|
||||
- `cmake --build --preset m2-dev` + `ctest --preset m2-dev --parallel 1` — **18/18 green**.
|
||||
The `vad_ptt_devices` VAD-gate sub-test has a pre-existing parallel-run timing flake
|
||||
(passes serially and in isolation); unrelated to this change (VAD gate logic is unchanged
|
||||
for `channels==1`, the only path the MIC uses).
|
||||
- **Done:** **Screen-audio sharing wired into the Windows WinForms client** (2026-06-17).
|
||||
The core already fully supported `SCREEN_AUDIO` capture on Windows (post-M3 WASAPI
|
||||
loopback via `VOICECAT_HAS_LOOPBACK`, always on for the `windows-client` preset —
|
||||
@@ -337,8 +370,9 @@ not treated as pre-existing-and-out-of-scope:
|
||||
`StreamAnnounce`/`StreamAnnounceResult` round-trips are now correlated by `request_id`
|
||||
(already round-tripped on the wire; just wasn't read) via `pending_announce_kind_`, so
|
||||
multiple concurrent announces from one client resolve to the right `LocalStream`.
|
||||
`on_capture_frame` takes a `kind` parameter and upmixes mono capture to stereo (duplicate
|
||||
L=R) when a stream's channel config calls for it. `vc_set_self_mute`'s `mic_muted` only
|
||||
`on_capture_frame` takes a `kind` and `channels` parameter; for mono capture (`channels==1`)
|
||||
on a stereo channel it upmixes L=R, and for real stereo capture (`channels==2`, the
|
||||
`SCREEN_AUDIO` loopback path on a stereo channel) it encodes directly with no upmix. `vc_set_self_mute`'s `mic_muted` only
|
||||
gates the `MIC` kind — a concurrent `SCREEN_AUDIO` share keeps playing while muted.
|
||||
`set_remote_stream` now actually wires `noise_reduction` through (previously parsed and
|
||||
discarded). New `run_talk_timer()` (a small dedicated thread, started alongside the UDP
|
||||
|
||||
Reference in New Issue
Block a user