Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(windows): real native exclude + self-echo removal for screen audio

Browse Source 
"All apps except selected" previously captured the complement of a frozen
app snapshot in INCLUDE mode (missed late-launched apps and system sounds,
wasted captures on silent windows). It now opens a single ProcessLoopbackCapture
in EXCLUDE mode (AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE)
of the one chosen app — true system-mix-minus-one, dynamic so apps launched
after sharing starts are included. The picker enforces single-selection in
exclude mode (the activation params take one target PID).

Adds an "Exclude VoiceCat's own audio (prevents echo)" checkbox (default on,
entire-desktop only) that routes the desktop capture through the same EXCLUDE
path targeting our own process id, killing the whole-device self-echo loop.

No C++/ABI changes. Updates voice.md and PROGRESS.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-22 12:28:53 +02:00
parent fb73b694d0
commit 5e18dfa1c9
6 changed files with 131 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
docs/voice.md
View File

@@ -301,7 +301,7 @@ normal stream; only the *source* is platform-specific.
| Platform | Mechanism | Notes |
|----------|-----------|-------|
| **Windows** | **WASAPI loopback** capture of the default render endpoint (via miniaudio's loopback mode) | **Implemented.** Captures in the channel's mode — stereo (interleaved L/R) when the channel is stereo, mono when the channel is mono — so a stereo music/screen-share channel gets genuine stereo end-to-end (no downmix). Whole-device capture, not process-specific — it inherently captures this app's own incoming voice mix along with everything else playing (an accepted self-echo-loop characteristic of desktop-audio capture, not a bug). Windows 10 2004+'s process-specific loopback (`AUDIOCLIENT_ACTIVATION_PARAMS`) would avoid this but miniaudio doesn't expose it — a future enhancement. |
| **Windows** | **WASAPI loopback** (whole-device, via miniaudio) **or WASAPI process loopback** (`AUDIOCLIENT_ACTIVATION_PARAMS`, Win10 2004+) for per-app / self-exclude | **Implemented.** Default *entire desktop* uses miniaudio's whole-device loopback in the channel's mode — stereo (interleaved L/R) when the channel is stereo, mono when mono — so a stereo channel gets genuine stereo end-to-end (no downmix). It inherently captures this app's own incoming voice mix (self-echo). The **per-app modes and the "exclude VoiceCat's own audio" option** instead drive `ProcessLoopbackCapture` (process-specific INCLUDE/EXCLUDE) through the external-feed mixer (`vc_stream_feed_pcm`, `external_feed=1`), which avoids self-echo and supports true "everything except". See below. |
| **macOS** | **ScreenCaptureKit** system-audio capture (macOS 13+) | **Implemented** (`clients/apple/macOS/VoiceCatMac/Audio/ScreenAudioCapture.swift`). OS requires screen-recording permission; capture happens in the main app. An `SCStream` with `capturesAudio` + `excludesCurrentProcessAudio` delivers audio `CMSampleBuffer`s; Swift converts Float32 → int16 (in the channel's mono/stereo mode) and calls `vc_stream_feed_pcm` — no miniaudio loopback device involved (`VOICECAT_HAS_LOOPBACK` is Windows-only). **Supports per-app audio selection** — see below. |
| **iOS** | **ReplayKit Broadcast Upload Extension** (the Discord mechanism) | **Implemented.** See below — separate process, App Group, ~50 MB cap (fine for audio-only). ReplayKit only ever delivers the *mixed* system stream as `.audioApp`, so **per-app filtering / VoiceOver exclusion is not possible on iOS** (it has no per-app granularity, unlike ScreenCaptureKit). |
@@ -321,6 +321,25 @@ the speech-synthesis daemon that actually renders the spoken audio). The chosen
`ScreenAudioSelection` is passed into `ScreenAudioCapture`, which builds the matching
`SCContentFilter`. iOS/ReplayKit has no equivalent control (see the table note above).
### Windows detail — per-app audio selection and self-echo
`AppAudioPickerDialog` (`clients/windows/VoiceCat.App/Forms/`) offers the same shape as macOS:
- **Entire desktop** — whole-device miniaudio loopback handled by the core (default path).
- **Only selected apps** — one `ProcessLoopbackCapture` in **INCLUDE** mode per ticked app,
mixed by `ProcessAudioMixer` and fed via `vc_stream_feed_pcm`.
- **All apps except selected** — a **single** `ProcessLoopbackCapture` in **EXCLUDE** mode of
the chosen process tree. WASAPI's `AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE`
captures the whole render mix minus that tree *dynamically* (apps launched after sharing
starts are included automatically). The activation params take a **single** target PID, so
exclude is restricted to **one** app — the picker enforces single-selection in this mode.
An **"Exclude VoiceCat's own audio (prevents echo)"** checkbox (default on, enabled for
*entire desktop*) routes the desktop capture through the same EXCLUDE path targeting
VoiceCat's **own** process id (`Environment.ProcessId`) — i.e. "entire desktop except this
app" — which removes the self-echo loop the whole-device path otherwise has. The per-app
INCLUDE modes already never capture this app's tree, so they have no self-echo to remove.
### iOS detail
The extension **captures**, the host app **sends**. Unlike a self-connecting extension, this