feat(windows): wire screen-audio sharing into the WinForms client

The core already supported SCREEN_AUDIO capture on Windows (post-M3 WASAPI
loopback via VOICECAT_HAS_LOOPBACK) and the C# Interop layer was complete
(VcStreamKind.ScreenAudio, StartStream/StopStream/SetRemoteStream). Only the
UI was missing -- no core, proto, or C ABI changes needed.

Adds a 'Share Screen Audio' toggle to the voice panel, independent of mic
voice (can share without joining voice). Disconnect/teardown now stops the
screen stream cleanly. New smoke test exercises the full StartStream ->
StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
This commit is contained in:
2026-06-17 22:47:28 +02:00
parent 2185d9d15c
commit a88656f2fa
4 changed files with 140 additions and 0 deletions

View File

@@ -10,6 +10,44 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action ## ▶ Where we left off / next action
- **Done:** **Screen-audio sharing wired into the Windows WinForms client** (2026-06-17).
The core already fully supported `SCREEN_AUDIO` capture on Windows (post-M3 WASAPI
loopback via `VOICECAT_HAS_LOOPBACK`, always on for the `windows-client` preset —
`core/CMakeLists.txt:85`; loopback start/stop at `client.cpp:1093`/`audio_engine.cpp:529`;
VAD/PTT/self-mute/server-mute correctly bypassed for non-MIC kinds at `client.cpp:862-879`)
and the C# Interop layer was already complete (`VcStreamKind.ScreenAudio`,
`StartStream`/`StopStream`/`SetRemoteStream`/`ListUserStreams` all generic). The gap was
purely UI wiring. **No core, proto, or C ABI changes were needed** — confirming the
"the core should support it already" assessment.
- `MainForm.Designer.cs` — new `btnScreenShareToggle` button in the voice panel top row
(`flpVoiceTop`), right after `btnMicToggle`, with full `AccessibleName`/
`AccessibleDescription` per the existing accessibility convention.
- `MainForm.cs` — new `_screenStreamId` field; `BtnScreenShareToggle_Click` handler
mirroring `BtnMicToggle_Click` but with no device picker / VAD / PTT / mode / mute
(screen audio bypasses all of those in the core). Independent of mic — can share
without joining voice and vice versa. `HandleDisconnected` now resets
`_screenStreamId` and disables the screen toggle. `OnFormClosed` now explicitly stops
both mic and screen streams before `Disconnect()` (clean `StreamStop` messages go out
before the control channel closes). `HandleStreamStarted` already labeled
`ScreenAudio => "screen audio"`; `PerUserTuningDialog.ApplySettings` already iterates
all of a peer's streams — both unchanged, peers can independently volume-tune a
screen-audio stream vs that user's mic.
- `VoiceCatClientSmokeTests.cs` — new `ScreenAudioStream_Starts_And_Stops` test:
connects + TOFU + guest auth, `StartStream(ScreenAudio)`, asserts
`VC_EVENT_STREAM_STARTED` arrives with matching `StreamId`, `StopStream`, asserts
`VC_EVENT_STREAM_STOPPED`. Passes headless (the `StreamAnnounce` succeeds regardless
of whether the loopback device initializes on a CI box).
- `dotnet build` — 0 warnings/errors. `dotnet test`**4/4 tests green**
(3 existing + 1 new). Event trace confirms the full
`StreamStarted → UserUpdated → StreamStopped` path through P/Invoke against a live
`voicecat-server.exe`.
- **Not yet confirmed audible by ear** — pending manual two-instance live test (one
shares screen audio while something plays on the default render endpoint, the other
hears it). This is the observable-behavior exit criterion per `AGENTS.md`.
- **Documented caveat** (`docs/voice.md §9`, unchanged): whole-device WASAPI loopback
inherently re-captures this app's own incoming voice mix (self-echo loop) — accepted
characteristic, not a bug. Process-specific loopback (Windows 10 2004+
`AUDIOCLIENT_ACTIVATION_PARAMS`) is a future enhancement; miniaudio doesn't expose it.
- **In progress:** **M5 — moderation & admin** (2026-06-17). Server-side and C ABI are - **In progress:** **M5 — moderation & admin** (2026-06-17). Server-side and C ABI are
implemented and tested: permissions, kick/ban/move/server-mute, channel CRUD, in-app account implemented and tested: permissions, kick/ban/move/server-mute, channel CRUD, in-app account
management. Four new tests pass: `test_m5_permissions`, `test_m5_kick_ban_move_mute`, management. Four new tests pass: `test_m5_permissions`, `test_m5_kick_ban_move_mute`,

View File

@@ -36,6 +36,7 @@ partial class MainForm
private FlowLayoutPanel flpVoiceTop = null!; private FlowLayoutPanel flpVoiceTop = null!;
private FlowLayoutPanel flpVoiceBottom = null!; private FlowLayoutPanel flpVoiceBottom = null!;
private Button btnMicToggle = null!; private Button btnMicToggle = null!;
private Button btnScreenShareToggle = null!;
private CheckBox chkMute = null!; private CheckBox chkMute = null!;
private CheckBox chkDeafen = null!; private CheckBox chkDeafen = null!;
private RadioButton radioVad = null!; private RadioButton radioVad = null!;
@@ -81,6 +82,7 @@ partial class MainForm
flpVoiceTop = new FlowLayoutPanel(); flpVoiceTop = new FlowLayoutPanel();
flpVoiceBottom = new FlowLayoutPanel(); flpVoiceBottom = new FlowLayoutPanel();
btnMicToggle = new Button(); btnMicToggle = new Button();
btnScreenShareToggle = new Button();
chkMute = new CheckBox(); chkMute = new CheckBox();
chkDeafen = new CheckBox(); chkDeafen = new CheckBox();
radioVad = new RadioButton(); radioVad = new RadioButton();
@@ -227,6 +229,20 @@ partial class MainForm
btnMicToggle.Margin = new Padding(0, 2, 6, 0); btnMicToggle.Margin = new Padding(0, 2, 6, 0);
btnMicToggle.TabIndex = 0; btnMicToggle.TabIndex = 0;
// Screen-audio share — independent of mic voice (can share without joining voice and
// vice versa). The core's WASAPI loopback path (VOICECAT_HAS_LOOPBACK, always on for
// the windows-client preset) captures the default render endpoint; see docs/voice.md
// §9. Whole-device loopback inherently re-captures this app's own incoming voice mix —
// an accepted self-echo characteristic, not a bug.
btnScreenShareToggle.Text = "Share Screen &Audio";
btnScreenShareToggle.AccessibleName = "Share screen audio";
btnScreenShareToggle.AccessibleDescription =
"Start or stop sharing your computer's audio (desktop/system audio) with the channel. " +
"Independent of the microphone. Captures everything playing through your default speakers.";
btnScreenShareToggle.AutoSize = true;
btnScreenShareToggle.Margin = new Padding(0, 2, 12, 0);
btnScreenShareToggle.TabIndex = 10;
chkMute.Text = "&Mute mic"; chkMute.Text = "&Mute mic";
chkMute.AutoSize = true; chkMute.AutoSize = true;
chkMute.Enabled = false; chkMute.Enabled = false;
@@ -276,6 +292,7 @@ partial class MainForm
flpVoiceTop.AutoSize = false; flpVoiceTop.AutoSize = false;
flpVoiceTop.Padding = new Padding(4, 2, 4, 0); flpVoiceTop.Padding = new Padding(4, 2, 4, 0);
flpVoiceTop.Controls.Add(btnMicToggle); flpVoiceTop.Controls.Add(btnMicToggle);
flpVoiceTop.Controls.Add(btnScreenShareToggle);
flpVoiceTop.Controls.Add(chkMute); flpVoiceTop.Controls.Add(chkMute);
flpVoiceTop.Controls.Add(chkDeafen); flpVoiceTop.Controls.Add(chkDeafen);
flpVoiceTop.Controls.Add(lblMode); flpVoiceTop.Controls.Add(lblMode);

View File

@@ -23,6 +23,7 @@ public partial class MainForm : Form
// Voice state // Voice state
private uint _micStreamId; // 0 = not started private uint _micStreamId; // 0 = not started
private uint _screenStreamId; // 0 = not sharing screen audio
private Keys _pttKey = Keys.F8; private Keys _pttKey = Keys.F8;
private bool _serverMuted; private bool _serverMuted;
private bool _serverDeafened; private bool _serverDeafened;
@@ -60,6 +61,7 @@ public partial class MainForm : Form
// Voice controls // Voice controls
btnMicToggle.Click += BtnMicToggle_Click; btnMicToggle.Click += BtnMicToggle_Click;
btnScreenShareToggle.Click += BtnScreenShareToggle_Click;
chkMute.CheckedChanged += (_, _) => ApplySelfMute(); chkMute.CheckedChanged += (_, _) => ApplySelfMute();
chkDeafen.CheckedChanged += (_, _) => ApplySelfMute(); chkDeafen.CheckedChanged += (_, _) => ApplySelfMute();
radioVad.CheckedChanged += RadioVad_CheckedChanged; radioVad.CheckedChanged += RadioVad_CheckedChanged;
@@ -404,9 +406,11 @@ public partial class MainForm : Form
_talkingUsers.Clear(); _talkingUsers.Clear();
_currentChannelId = 0; _currentChannelId = 0;
_micStreamId = 0; _micStreamId = 0;
_screenStreamId = 0;
txtCompose.Enabled = false; txtCompose.Enabled = false;
btnSend.Enabled = false; btnSend.Enabled = false;
btnMicToggle.Enabled = false; btnMicToggle.Enabled = false;
btnScreenShareToggle.Enabled = false;
} }
// ── Level meter ─────────────────────────────────────────────────────────── // ── Level meter ───────────────────────────────────────────────────────────
@@ -604,6 +608,35 @@ public partial class MainForm : Form
private void ApplySelfMute() => private void ApplySelfMute() =>
_client.SetSelfMute(chkMute.Checked, chkDeafen.Checked); _client.SetSelfMute(chkMute.Checked, chkDeafen.Checked);
// Screen-audio share — independent of mic voice. The core bypasses VAD/PTT/self-mute/
// server-mute for non-MIC kinds (client.cpp on_capture_frame), so no input-mode or mute
// state applies here. WASAPI loopback captures the default render endpoint (whole-device,
// not process-specific — docs/voice.md §9).
private void BtnScreenShareToggle_Click(object? sender, EventArgs e)
{
if (_screenStreamId == 0)
{
var (result, streamId) = _client.StartStream(VcStreamKind.ScreenAudio, "Desktop audio");
if (result == VcResult.Ok)
{
_screenStreamId = streamId;
btnScreenShareToggle.Text = "Stop Screen &Audio";
AddActivity("Started sharing screen audio");
}
else
{
AddActivity($"Failed to start screen audio: {result}");
}
}
else
{
_client.StopStream(_screenStreamId);
_screenStreamId = 0;
btnScreenShareToggle.Text = "Share Screen &Audio";
AddActivity("Stopped sharing screen audio");
}
}
private void RadioVad_CheckedChanged(object? sender, EventArgs e) private void RadioVad_CheckedChanged(object? sender, EventArgs e)
{ {
if (!radioVad.Checked) return; if (!radioVad.Checked) return;
@@ -916,6 +949,11 @@ public partial class MainForm : Form
_pumpTimer.Stop(); _pumpTimer.Stop();
_client.LevelChanged -= OnLevelChanged; _client.LevelChanged -= OnLevelChanged;
_client.EventReceived -= OnEvent; _client.EventReceived -= OnEvent;
// Stop any active local streams before tearing down — the core stops loopback in
// stream_stop/destroy, but explicit stops ensure clean StreamStop protocol messages
// go out before Disconnect closes the control channel.
if (_screenStreamId != 0) _client.StopStream(_screenStreamId);
if (_micStreamId != 0) _client.StopStream(_micStreamId);
_client.Disconnect(); _client.Disconnect();
_client.Dispose(); _client.Dispose();
base.OnFormClosed(e); base.OnFormClosed(e);

View File

@@ -246,4 +246,51 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
client.Disconnect(); client.Disconnect();
} }
/// <summary>
/// Screen-audio (SCREEN_AUDIO) stream start/stop through the P/Invoke layer. The core's
/// WASAPI loopback path (VOICECAT_HAS_LOOPBACK) captures the default render endpoint; the
/// StreamAnnounce succeeds regardless of whether the loopback device actually initializes
/// on a headless box, so this test passes in CI while still exercising the full
/// StartStream -> StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
/// See docs/voice.md §9 and MainForm's BtnScreenShareToggle_Click.
/// </summary>
[Fact]
public void ScreenAudioStream_Starts_And_Stops()
{
var events = new List<VoiceCatEvent>();
using var client = new VoiceCatClient("vc-csharp-screen", "0.1", VcLogLevel.Off,
tofuStorePath: Path.Combine(_tempDir, "tofu_pins_screen.txt"));
client.EventReceived += events.Add;
Assert.Equal(VcResult.Ok, client.Connect("127.0.0.1", _port));
Assert.Equal(VcResult.Ok, client.AuthenticateGuest("CSharpScreen"));
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ServerIdentity), 5000));
Assert.Equal(VcResult.Ok, client.ConfirmServerIdentity(accept: true));
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.AuthResult), 5000));
Assert.Equal(VcResult.Ok, events.First(e => e.Type == VcEventType.AuthResult).Result);
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ChannelList), 3000));
// Give the async UDP binding handshake a moment to land before announcing a stream
// (mirrors vccli's 500ms sleep after auth).
Thread.Sleep(500);
var (startResult, streamId) = client.StartStream(VcStreamKind.ScreenAudio, "Desktop audio");
Assert.Equal(VcResult.Ok, startResult);
Assert.True(streamId != 0, "StreamId should be non-zero on success");
// The core emits VC_EVENT_STREAM_STARTED for the local client too (client.cpp
// handle_stream_announce_result), so we see our own screen-audio stream start.
Assert.True(PumpUntil(client,
() => events.Any(e => e.Type == VcEventType.StreamStarted && e.StreamId == streamId), 5000),
"did not receive VC_EVENT_STREAM_STARTED for screen-audio stream");
Assert.Equal(VcResult.Ok, client.StopStream(streamId));
Assert.True(PumpUntil(client,
() => events.Any(e => e.Type == VcEventType.StreamStopped && e.StreamId == streamId), 5000),
"did not receive VC_EVENT_STREAM_STOPPED for screen-audio stream");
client.Disconnect();
}
} }