feat(windows): wire screen-audio sharing into the WinForms client
The core already supported SCREEN_AUDIO capture on Windows (post-M3 WASAPI loopback via VOICECAT_HAS_LOOPBACK) and the C# Interop layer was complete (VcStreamKind.ScreenAudio, StartStream/StopStream/SetRemoteStream). Only the UI was missing -- no core, proto, or C ABI changes needed. Adds a 'Share Screen Audio' toggle to the voice panel, independent of mic voice (can share without joining voice). Disconnect/teardown now stops the screen stream cleanly. New smoke test exercises the full StartStream -> StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
This commit is contained in:
38
PROGRESS.md
38
PROGRESS.md
@@ -10,6 +10,44 @@ up instantly. Newest status at the top.
|
|||||||
|
|
||||||
## ▶ Where we left off / next action
|
## ▶ Where we left off / next action
|
||||||
|
|
||||||
|
- **Done:** **Screen-audio sharing wired into the Windows WinForms client** (2026-06-17).
|
||||||
|
The core already fully supported `SCREEN_AUDIO` capture on Windows (post-M3 WASAPI
|
||||||
|
loopback via `VOICECAT_HAS_LOOPBACK`, always on for the `windows-client` preset —
|
||||||
|
`core/CMakeLists.txt:85`; loopback start/stop at `client.cpp:1093`/`audio_engine.cpp:529`;
|
||||||
|
VAD/PTT/self-mute/server-mute correctly bypassed for non-MIC kinds at `client.cpp:862-879`)
|
||||||
|
and the C# Interop layer was already complete (`VcStreamKind.ScreenAudio`,
|
||||||
|
`StartStream`/`StopStream`/`SetRemoteStream`/`ListUserStreams` all generic). The gap was
|
||||||
|
purely UI wiring. **No core, proto, or C ABI changes were needed** — confirming the
|
||||||
|
"the core should support it already" assessment.
|
||||||
|
- `MainForm.Designer.cs` — new `btnScreenShareToggle` button in the voice panel top row
|
||||||
|
(`flpVoiceTop`), right after `btnMicToggle`, with full `AccessibleName`/
|
||||||
|
`AccessibleDescription` per the existing accessibility convention.
|
||||||
|
- `MainForm.cs` — new `_screenStreamId` field; `BtnScreenShareToggle_Click` handler
|
||||||
|
mirroring `BtnMicToggle_Click` but with no device picker / VAD / PTT / mode / mute
|
||||||
|
(screen audio bypasses all of those in the core). Independent of mic — can share
|
||||||
|
without joining voice and vice versa. `HandleDisconnected` now resets
|
||||||
|
`_screenStreamId` and disables the screen toggle. `OnFormClosed` now explicitly stops
|
||||||
|
both mic and screen streams before `Disconnect()` (clean `StreamStop` messages go out
|
||||||
|
before the control channel closes). `HandleStreamStarted` already labeled
|
||||||
|
`ScreenAudio => "screen audio"`; `PerUserTuningDialog.ApplySettings` already iterates
|
||||||
|
all of a peer's streams — both unchanged, peers can independently volume-tune a
|
||||||
|
screen-audio stream vs that user's mic.
|
||||||
|
- `VoiceCatClientSmokeTests.cs` — new `ScreenAudioStream_Starts_And_Stops` test:
|
||||||
|
connects + TOFU + guest auth, `StartStream(ScreenAudio)`, asserts
|
||||||
|
`VC_EVENT_STREAM_STARTED` arrives with matching `StreamId`, `StopStream`, asserts
|
||||||
|
`VC_EVENT_STREAM_STOPPED`. Passes headless (the `StreamAnnounce` succeeds regardless
|
||||||
|
of whether the loopback device initializes on a CI box).
|
||||||
|
- `dotnet build` — 0 warnings/errors. `dotnet test` — **4/4 tests green**
|
||||||
|
(3 existing + 1 new). Event trace confirms the full
|
||||||
|
`StreamStarted → UserUpdated → StreamStopped` path through P/Invoke against a live
|
||||||
|
`voicecat-server.exe`.
|
||||||
|
- **Not yet confirmed audible by ear** — pending manual two-instance live test (one
|
||||||
|
shares screen audio while something plays on the default render endpoint, the other
|
||||||
|
hears it). This is the observable-behavior exit criterion per `AGENTS.md`.
|
||||||
|
- **Documented caveat** (`docs/voice.md §9`, unchanged): whole-device WASAPI loopback
|
||||||
|
inherently re-captures this app's own incoming voice mix (self-echo loop) — accepted
|
||||||
|
characteristic, not a bug. Process-specific loopback (Windows 10 2004+
|
||||||
|
`AUDIOCLIENT_ACTIVATION_PARAMS`) is a future enhancement; miniaudio doesn't expose it.
|
||||||
- **In progress:** **M5 — moderation & admin** (2026-06-17). Server-side and C ABI are
|
- **In progress:** **M5 — moderation & admin** (2026-06-17). Server-side and C ABI are
|
||||||
implemented and tested: permissions, kick/ban/move/server-mute, channel CRUD, in-app account
|
implemented and tested: permissions, kick/ban/move/server-mute, channel CRUD, in-app account
|
||||||
management. Four new tests pass: `test_m5_permissions`, `test_m5_kick_ban_move_mute`,
|
management. Four new tests pass: `test_m5_permissions`, `test_m5_kick_ban_move_mute`,
|
||||||
|
|||||||
@@ -36,6 +36,7 @@ partial class MainForm
|
|||||||
private FlowLayoutPanel flpVoiceTop = null!;
|
private FlowLayoutPanel flpVoiceTop = null!;
|
||||||
private FlowLayoutPanel flpVoiceBottom = null!;
|
private FlowLayoutPanel flpVoiceBottom = null!;
|
||||||
private Button btnMicToggle = null!;
|
private Button btnMicToggle = null!;
|
||||||
|
private Button btnScreenShareToggle = null!;
|
||||||
private CheckBox chkMute = null!;
|
private CheckBox chkMute = null!;
|
||||||
private CheckBox chkDeafen = null!;
|
private CheckBox chkDeafen = null!;
|
||||||
private RadioButton radioVad = null!;
|
private RadioButton radioVad = null!;
|
||||||
@@ -81,6 +82,7 @@ partial class MainForm
|
|||||||
flpVoiceTop = new FlowLayoutPanel();
|
flpVoiceTop = new FlowLayoutPanel();
|
||||||
flpVoiceBottom = new FlowLayoutPanel();
|
flpVoiceBottom = new FlowLayoutPanel();
|
||||||
btnMicToggle = new Button();
|
btnMicToggle = new Button();
|
||||||
|
btnScreenShareToggle = new Button();
|
||||||
chkMute = new CheckBox();
|
chkMute = new CheckBox();
|
||||||
chkDeafen = new CheckBox();
|
chkDeafen = new CheckBox();
|
||||||
radioVad = new RadioButton();
|
radioVad = new RadioButton();
|
||||||
@@ -227,6 +229,20 @@ partial class MainForm
|
|||||||
btnMicToggle.Margin = new Padding(0, 2, 6, 0);
|
btnMicToggle.Margin = new Padding(0, 2, 6, 0);
|
||||||
btnMicToggle.TabIndex = 0;
|
btnMicToggle.TabIndex = 0;
|
||||||
|
|
||||||
|
// Screen-audio share — independent of mic voice (can share without joining voice and
|
||||||
|
// vice versa). The core's WASAPI loopback path (VOICECAT_HAS_LOOPBACK, always on for
|
||||||
|
// the windows-client preset) captures the default render endpoint; see docs/voice.md
|
||||||
|
// §9. Whole-device loopback inherently re-captures this app's own incoming voice mix —
|
||||||
|
// an accepted self-echo characteristic, not a bug.
|
||||||
|
btnScreenShareToggle.Text = "Share Screen &Audio";
|
||||||
|
btnScreenShareToggle.AccessibleName = "Share screen audio";
|
||||||
|
btnScreenShareToggle.AccessibleDescription =
|
||||||
|
"Start or stop sharing your computer's audio (desktop/system audio) with the channel. " +
|
||||||
|
"Independent of the microphone. Captures everything playing through your default speakers.";
|
||||||
|
btnScreenShareToggle.AutoSize = true;
|
||||||
|
btnScreenShareToggle.Margin = new Padding(0, 2, 12, 0);
|
||||||
|
btnScreenShareToggle.TabIndex = 10;
|
||||||
|
|
||||||
chkMute.Text = "&Mute mic";
|
chkMute.Text = "&Mute mic";
|
||||||
chkMute.AutoSize = true;
|
chkMute.AutoSize = true;
|
||||||
chkMute.Enabled = false;
|
chkMute.Enabled = false;
|
||||||
@@ -276,6 +292,7 @@ partial class MainForm
|
|||||||
flpVoiceTop.AutoSize = false;
|
flpVoiceTop.AutoSize = false;
|
||||||
flpVoiceTop.Padding = new Padding(4, 2, 4, 0);
|
flpVoiceTop.Padding = new Padding(4, 2, 4, 0);
|
||||||
flpVoiceTop.Controls.Add(btnMicToggle);
|
flpVoiceTop.Controls.Add(btnMicToggle);
|
||||||
|
flpVoiceTop.Controls.Add(btnScreenShareToggle);
|
||||||
flpVoiceTop.Controls.Add(chkMute);
|
flpVoiceTop.Controls.Add(chkMute);
|
||||||
flpVoiceTop.Controls.Add(chkDeafen);
|
flpVoiceTop.Controls.Add(chkDeafen);
|
||||||
flpVoiceTop.Controls.Add(lblMode);
|
flpVoiceTop.Controls.Add(lblMode);
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ public partial class MainForm : Form
|
|||||||
|
|
||||||
// Voice state
|
// Voice state
|
||||||
private uint _micStreamId; // 0 = not started
|
private uint _micStreamId; // 0 = not started
|
||||||
|
private uint _screenStreamId; // 0 = not sharing screen audio
|
||||||
private Keys _pttKey = Keys.F8;
|
private Keys _pttKey = Keys.F8;
|
||||||
private bool _serverMuted;
|
private bool _serverMuted;
|
||||||
private bool _serverDeafened;
|
private bool _serverDeafened;
|
||||||
@@ -60,6 +61,7 @@ public partial class MainForm : Form
|
|||||||
|
|
||||||
// Voice controls
|
// Voice controls
|
||||||
btnMicToggle.Click += BtnMicToggle_Click;
|
btnMicToggle.Click += BtnMicToggle_Click;
|
||||||
|
btnScreenShareToggle.Click += BtnScreenShareToggle_Click;
|
||||||
chkMute.CheckedChanged += (_, _) => ApplySelfMute();
|
chkMute.CheckedChanged += (_, _) => ApplySelfMute();
|
||||||
chkDeafen.CheckedChanged += (_, _) => ApplySelfMute();
|
chkDeafen.CheckedChanged += (_, _) => ApplySelfMute();
|
||||||
radioVad.CheckedChanged += RadioVad_CheckedChanged;
|
radioVad.CheckedChanged += RadioVad_CheckedChanged;
|
||||||
@@ -404,9 +406,11 @@ public partial class MainForm : Form
|
|||||||
_talkingUsers.Clear();
|
_talkingUsers.Clear();
|
||||||
_currentChannelId = 0;
|
_currentChannelId = 0;
|
||||||
_micStreamId = 0;
|
_micStreamId = 0;
|
||||||
|
_screenStreamId = 0;
|
||||||
txtCompose.Enabled = false;
|
txtCompose.Enabled = false;
|
||||||
btnSend.Enabled = false;
|
btnSend.Enabled = false;
|
||||||
btnMicToggle.Enabled = false;
|
btnMicToggle.Enabled = false;
|
||||||
|
btnScreenShareToggle.Enabled = false;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Level meter ───────────────────────────────────────────────────────────
|
// ── Level meter ───────────────────────────────────────────────────────────
|
||||||
@@ -604,6 +608,35 @@ public partial class MainForm : Form
|
|||||||
private void ApplySelfMute() =>
|
private void ApplySelfMute() =>
|
||||||
_client.SetSelfMute(chkMute.Checked, chkDeafen.Checked);
|
_client.SetSelfMute(chkMute.Checked, chkDeafen.Checked);
|
||||||
|
|
||||||
|
// Screen-audio share — independent of mic voice. The core bypasses VAD/PTT/self-mute/
|
||||||
|
// server-mute for non-MIC kinds (client.cpp on_capture_frame), so no input-mode or mute
|
||||||
|
// state applies here. WASAPI loopback captures the default render endpoint (whole-device,
|
||||||
|
// not process-specific — docs/voice.md §9).
|
||||||
|
private void BtnScreenShareToggle_Click(object? sender, EventArgs e)
|
||||||
|
{
|
||||||
|
if (_screenStreamId == 0)
|
||||||
|
{
|
||||||
|
var (result, streamId) = _client.StartStream(VcStreamKind.ScreenAudio, "Desktop audio");
|
||||||
|
if (result == VcResult.Ok)
|
||||||
|
{
|
||||||
|
_screenStreamId = streamId;
|
||||||
|
btnScreenShareToggle.Text = "Stop Screen &Audio";
|
||||||
|
AddActivity("Started sharing screen audio");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
AddActivity($"Failed to start screen audio: {result}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
_client.StopStream(_screenStreamId);
|
||||||
|
_screenStreamId = 0;
|
||||||
|
btnScreenShareToggle.Text = "Share Screen &Audio";
|
||||||
|
AddActivity("Stopped sharing screen audio");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private void RadioVad_CheckedChanged(object? sender, EventArgs e)
|
private void RadioVad_CheckedChanged(object? sender, EventArgs e)
|
||||||
{
|
{
|
||||||
if (!radioVad.Checked) return;
|
if (!radioVad.Checked) return;
|
||||||
@@ -916,6 +949,11 @@ public partial class MainForm : Form
|
|||||||
_pumpTimer.Stop();
|
_pumpTimer.Stop();
|
||||||
_client.LevelChanged -= OnLevelChanged;
|
_client.LevelChanged -= OnLevelChanged;
|
||||||
_client.EventReceived -= OnEvent;
|
_client.EventReceived -= OnEvent;
|
||||||
|
// Stop any active local streams before tearing down — the core stops loopback in
|
||||||
|
// stream_stop/destroy, but explicit stops ensure clean StreamStop protocol messages
|
||||||
|
// go out before Disconnect closes the control channel.
|
||||||
|
if (_screenStreamId != 0) _client.StopStream(_screenStreamId);
|
||||||
|
if (_micStreamId != 0) _client.StopStream(_micStreamId);
|
||||||
_client.Disconnect();
|
_client.Disconnect();
|
||||||
_client.Dispose();
|
_client.Dispose();
|
||||||
base.OnFormClosed(e);
|
base.OnFormClosed(e);
|
||||||
|
|||||||
@@ -246,4 +246,51 @@ public sealed class VoiceCatClientSmokeTests : IDisposable
|
|||||||
|
|
||||||
client.Disconnect();
|
client.Disconnect();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Screen-audio (SCREEN_AUDIO) stream start/stop through the P/Invoke layer. The core's
|
||||||
|
/// WASAPI loopback path (VOICECAT_HAS_LOOPBACK) captures the default render endpoint; the
|
||||||
|
/// StreamAnnounce succeeds regardless of whether the loopback device actually initializes
|
||||||
|
/// on a headless box, so this test passes in CI while still exercising the full
|
||||||
|
/// StartStream -> StreamStarted -> StopStream -> StreamStopped path through P/Invoke.
|
||||||
|
/// See docs/voice.md §9 and MainForm's BtnScreenShareToggle_Click.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void ScreenAudioStream_Starts_And_Stops()
|
||||||
|
{
|
||||||
|
var events = new List<VoiceCatEvent>();
|
||||||
|
using var client = new VoiceCatClient("vc-csharp-screen", "0.1", VcLogLevel.Off,
|
||||||
|
tofuStorePath: Path.Combine(_tempDir, "tofu_pins_screen.txt"));
|
||||||
|
client.EventReceived += events.Add;
|
||||||
|
|
||||||
|
Assert.Equal(VcResult.Ok, client.Connect("127.0.0.1", _port));
|
||||||
|
Assert.Equal(VcResult.Ok, client.AuthenticateGuest("CSharpScreen"));
|
||||||
|
|
||||||
|
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ServerIdentity), 5000));
|
||||||
|
Assert.Equal(VcResult.Ok, client.ConfirmServerIdentity(accept: true));
|
||||||
|
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.AuthResult), 5000));
|
||||||
|
Assert.Equal(VcResult.Ok, events.First(e => e.Type == VcEventType.AuthResult).Result);
|
||||||
|
Assert.True(PumpUntil(client, () => events.Any(e => e.Type == VcEventType.ChannelList), 3000));
|
||||||
|
|
||||||
|
// Give the async UDP binding handshake a moment to land before announcing a stream
|
||||||
|
// (mirrors vccli's 500ms sleep after auth).
|
||||||
|
Thread.Sleep(500);
|
||||||
|
|
||||||
|
var (startResult, streamId) = client.StartStream(VcStreamKind.ScreenAudio, "Desktop audio");
|
||||||
|
Assert.Equal(VcResult.Ok, startResult);
|
||||||
|
Assert.True(streamId != 0, "StreamId should be non-zero on success");
|
||||||
|
|
||||||
|
// The core emits VC_EVENT_STREAM_STARTED for the local client too (client.cpp
|
||||||
|
// handle_stream_announce_result), so we see our own screen-audio stream start.
|
||||||
|
Assert.True(PumpUntil(client,
|
||||||
|
() => events.Any(e => e.Type == VcEventType.StreamStarted && e.StreamId == streamId), 5000),
|
||||||
|
"did not receive VC_EVENT_STREAM_STARTED for screen-audio stream");
|
||||||
|
|
||||||
|
Assert.Equal(VcResult.Ok, client.StopStream(streamId));
|
||||||
|
Assert.True(PumpUntil(client,
|
||||||
|
() => events.Any(e => e.Type == VcEventType.StreamStopped && e.StreamId == streamId), 5000),
|
||||||
|
"did not receive VC_EVENT_STREAM_STOPPED for screen-audio stream");
|
||||||
|
|
||||||
|
client.Disconnect();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user