Author SHA1 Message Date
Talon a918334e06 Record iOS TestFlight build 2026092701 2026-09-27 18:08:35 +02:00
Talon 4bad264605 Preserve iOS screen audio during stream startup
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-27 17:56:18 +02:00
Talon 3dbe763598 Record TestFlight IPA build 2026092602 2026-09-26 21:22:50 +02:00
Talon c1763c9a5d Activate iOS audio session before stereo input selection
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-26 21:02:54 +02:00
Talon 4cc13a27a0 Stabilize iOS reconnect and screen audio sessions
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-26 20:30:24 +02:00
Talon 5a2dddba89 Record build 2026092504 as the last verified release build 2026-09-25 20:59:14 +02:00
Talon 01bae734b8 fix(ios): keep the voice-processing graph up instead of rebuilding it
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
Joining voice on the voice-chat preset was unreliable: audio arrived after
several seconds of the route flipping back and forth, sometimes not at all, and
VoiceOver went quiet while it happened. Device logs show why. A graph with
voice processing enabled reports a successful start and is then torn down
within a second, roughly three times in four; every configuration without voice
processing — both microphone presets, and voice chat with processing off — comes
up first time and runs indefinitely.

With voice processing the input and output are one IO unit, and it only stays up
while the input is part of the render chain. The input node carried a tap and no
connection, which leaves it out of that chain. Route the input through a silent
mixer so it is genuinely rendered.

The rest of this is the amplifier rather than the cause, and each part of it
turned one failed start into a storm:

The stall watchdog rebuilt on every missed tick, without bound. That converted a
graph that could not start into endless session reconfiguration, which is what
the user heard and what hid the reason from the log. It now backs off after each
failed attempt and stops after four, logging VC_WATCHDOG exhausted, so a
transient freeze still recovers and a graph that will not start fails visibly.

Nothing waited for a graph to start before judging it dead. Enabling voice
processing rebuilds both halves of the IO, which posts a configuration change
and reads as not running for several hundred milliseconds, so the
configuration-change handler and the watchdog both tore down graphs that were
about to run. A settling window holds them off for two seconds.

A route change forced a full rebuild, and every rebuild moves the route, so one
notification produced the next. Route changes now take the non-forcing path,
which rebuilds a stopped graph and leaves a healthy one alone; the hardware test
it uses reads the input node's format, not AVAudioSession, whose reported rate
and channel count do not settle until after the graph has started.

The input side is built once per session instead of being added when voice is
joined, so joining and leaving voice set a stream id rather than replacing the
graph, and a mono voice-chat apply no longer clears a stereo capsule
configuration it never applied.

Every rebuild now logs its cause, and VC_START/VC_START_CHECK record whether the
graph survived its start. The first-attempt failure is not fixed and is recorded
in PROGRESS.md as a release gate: capture still comes up on a watchdog rebuild
rather than immediately.

The changed logic sits on AVAudioSession and AVAudioEngine, which the net10.0
test project cannot reference, so the behaviour is covered by the existing
source assertions; verification is on device.
2026-09-25 20:56:31 +02:00
Talon 1a0ff957ae fix(ios): stop the speaker toggle rebuilding the audio graph in a loop
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
Toggling speaker output flipped the route back and forth indefinitely. Two
loops, both of which made a rebuild produce the condition for the next one.

Reconfiguring the session moves the route, and moving the route is reported
back through RouteChangeNotification. Forcing the speaker takes a headset out
of the route, which arrives as OldDeviceUnavailable, and releasing it brings
the headset back as NewDeviceAvailable; neither is among the reasons the
handler filters, so each rebuild answered its own echo with another rebuild.
Nothing compared the reported route against the route the live graph was
actually built on.

Record that route at the end of Apply, once the session is configured, and
rebuild only when a reported change differs from it; notifications that arrive
while Apply is still running describe the change Apply is itself making and are
ignored outright. The decision is AudioRouteWatcher in VoiceCat.Core, which is
platform-agnostic and tested, following ControlPathWatcher; the route identity
it compares is supplied by the caller, on iOS the UIDs of the current route's
ports. A graph whose route is unchanged but broken is still the stall
watchdog's to catch.

The port override was also re-asserted on every Apply, so where the system
wanted to hand output back to a connected headset each rebuild forced it to the
speaker again and the resulting route change drove the next rebuild. It is now
the one-shot request it should always have been, issued by the toggle alone;
the DefaultToSpeaker category option is the part that persists across rebuilds.

Also updates the route test from 724f7e9, which asserted the voice-chat preset
clearing the speaker flag and the absence of the port override. Both were
deliberately removed when speaker output became orthogonal to the preset, and
the test should have been updated with them.
2026-09-25 19:05:14 +02:00
Talon 938b23d4c0 Record build 2026092503 as the last verified release build 2026-09-25 18:52:47 +02:00
Talon 7033053184 fix(ios): keep the audio preset when speaker output is toggled
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
Speaker output decides where audio goes, not how it is captured, which is why
it sits beside the preset row rather than inside Advanced audio. It was
nonetheless moving the preset to Advanced, and the voice-chat preset was
clearing it back, so the two controls overwrote each other and a user who
wanted the speaker lost the preset that describes their capture setup.

SetForceSpeaker no longer touches the preset, and Load and SelectPreset no
longer clear the flag. A speakerIsExplicit default records that the user
actually chose, so the automatic value older installs inherited from the
voice-chat preset is dropped once rather than pinning a headset user to the
speaker, and the toggle is honoured from then on.

Apply issues the speaker port override again when the flag is set. Without it
the toggle barely did anything with a headset attached, because
DefaultToSpeaker only decides the route when nothing else is connected. With
the toggle off it passes None, so a route left alone still follows an HFP
headset.

The changed logic sits on AVAudioSession, which the net10.0 test project
cannot reference, so this carries no tests; the toggle needs device
verification under the voice-chat preset with a Bluetooth headset connected.
2026-09-25 18:48:08 +02:00
Talon 69a9729c28 Record build 2026092502 as the last verified release build 2026-09-25 17:25:43 +02:00
Talon 0dad40c9d7 Reconnect on a real handover instead of waiting for a dead path
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
A Wi-Fi to cellular switch left the session visibly dropping: the media transport rebound
itself within a few seconds, but nothing noticed the blackholed control connection until an
unanswered keepalive proved it, and the teardown that followed announced a lost connection and
waited another second before dialling again.

Watch the system path on iOS and fail the control connection the moment the carrying interface
changes, which is the only path change TCP cannot survive. Roaming between access points and a
link that is merely unusable for a while keep the same interface and the same source address,
so ControlPathWatcher reports neither; an unsatisfied path holds the last signature rather than
reporting, so a reconnect is never started into a route that cannot carry it. Tighten the
keepalive window on the phone as the backstop for what the monitor cannot see, run the first
reconnect attempt immediately, and defer the lost-connection announcement until an attempt has
actually failed, so a sub-second handover is silent and only a real outage is announced.

A control reconnect still re-authenticates and rejoins: the media keys come from the TLS
exporter of the connection that was lost, so seamless handover needs control-plane session
resumption rather than a faster reconnect.
2026-09-25 17:20:52 +02:00
Talon 0b81b81c0c fix(ios): rebuild audio only when needed and stop VoiceOver list churn
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
Two iOS bugs with the same shape: unconditional rebuilds where a
conditional check belongs.

The audio graph was torn down on every unintentional disconnect and every
foreground transition. A lost connection ran the same teardown as an
explicit disconnect, deactivating the AVAudioSession and so dropping the
Bluetooth HFP link for a transport blip, and foregrounding always called
Reconfigure even though the `audio` background mode keeps the graph live.
Both cost seconds of dead audio on a headset.

Split "session ended" from "transport blipped". Detach unbinds the client
but keeps the session, graph, and route, so a reconnect rebinds to a live
HFP link; the route is parked on stream id 0 so capture cannot feed the
next connection a stream it never announced. StartListening reuses a
running graph, StartMicrophone reuses a running tap of the same width, and
Reconfigure gained a non-forcing mode that no-ops when tap presence,
channel width, and voice processing all still match. Foregrounding now
ensures the graph is running and only reconfigures if it actually stopped.
Route changes, media-services resets, and the stall watchdog still force a
full rebuild.

Every list also reloaded on a model event raised 20 times a second by the
microphone level timer. ReloadData recreates the accessibility element
tree, so VoiceOver explore mode re-announced the row under a dragging
finger and a double tap landed on an element that no longer existed. No
controller ever unsubscribed, so popped controllers kept reloading too.

Move the level to its own LevelChanged event, and reload lists through
ListRefresher, which subscribes only while on screen and only reloads when
the rendered content signature changed. The voice bar publishes its
accessibility value on 5% steps, MoveUserController reloads just its two
checkmark rows, and the chat transcripts skip reassigning identical text.

The changed logic sits on UIKit and AVFoundation types the net10.0 test
project cannot reference, so this carries no tests; the Bluetooth
reconnect and foreground paths need device verification.
2026-09-25 17:00:25 +02:00
Talon 724f7e912d fix(ios): keep voice chat on Bluetooth headset
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-25 12:27:27 +02:00
Talon f3ac779bf4 Survive changing networks and deepen the receive buffer
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
Media died silently whenever a client's source address changed. The relay bound a
peer's endpoint once and refused to move it, and the client stopped offering its
binding token after the first bind, so a Wi-Fi/cellular handover stranded the
session in both directions. Add an authenticated Rebind media frame: the binding
token travels in the clear for peer lookup only, and the AEAD tag over header and
token plus the peer's existing replay window are what authorize the move, so a
captured rebind cannot be replayed to redirect someone else's downlink. The client
rebuilds its UDP socket instead of retrying on one still pinned to a vanished
interface.

Nothing judged the control connection live: pings were sent and pongs ignored, so a
blackholed TCP path went unnoticed for minutes while the UI showed a live session.
Treat any server traffic as liveness and fail the connection when it stops, which
drives the existing reconnect.

The receive jitter buffer had lost its depth floor, so a channel without FEC or
DRED played out with no buffer at all and ordinary reordering became concealment.
Restore a one-frame floor, observe every arrival rather than only accepted ones —
a shallow buffer was rejecting the late arrivals that should have deepened it —
and allow playout to hold a frame so depth can follow a degrading link. A stalled
consumer now sheds the oldest queued packet instead of refusing the live talkspurt.

Add a deterministic network-impairment simulation covering bursty loss, jitter,
reordering, duplication, outages and a stalled consumer, a handover test against a
real relay, a replay test for the rebind path, and a blackholed control connection
driven through a freezable TCP proxy.
2026-09-24 19:15:16 +02:00
Talon fb740bcfb2 Try to fix background glitching over long periods of time
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
2026-09-24 13:31:16 +02:00
Talon 186fe6dbb6 Fix iOS broadcast pump mapping churn
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s
The screen-audio pump opened and disposed a memory mapping plus its container
lookup and path strings on every 5 ms tick - about 200 mapping pairs per
second of steady allocation that churned the GC under long calls. The producer
opens the ring with O_CREAT and never replaces it, so the pump now keeps one
mapping across ticks and rebuilds it only when the ring file disappears or a
drain fails.
2026-09-23 21:47:42 +02:00
31 changed files with 2001 additions and 178 deletions
+72 -6
View File
@@ -1,6 +1,6 @@
# VoiceCat status
Updated: 2026-09-22
Updated: 2026-09-25
## Current state
@@ -23,11 +23,67 @@ The physical-device iOS voice path now supports ReplayKit fallback, stable Apple
capture using a paced 20 ms handoff, and true built-in stereo microphone capture. Stereo was
verified on an iPhone 16 Pro Max with a two-channel AVAudioEngine input and distinct left/right
samples; the managed Apple binding requires native use of its otherwise-unmapped stereo polar
pattern constant.
pattern constant. The voice-chat preset now leaves speaker routing off so a connected Bluetooth
headset can supply both input and output; speaker routing remains an explicit Advanced setting.
Device-selected inputs are not saved as preferences during route refresh. Bluetooth switching
still needs device validation.
The iOS user list now opens a remote-user detail view with independent tuning for each active
audio stream. Private messages are grouped into per-user conversations with direct access to the
same user and audio controls.
same user and audio controls. Lists reload only when their rendered content actually changed and
only while on screen, and the 20 Hz microphone level is a separate signal from the general model
change, so VoiceOver explore mode no longer re-announces the row under a dragging finger or loses
the element a double tap was aimed at.
The iOS audio graph is rebuilt only when the audio configuration changed. A lost connection
unbinds the client but keeps the session, graph, and route alive, so a reconnect rebinds to a
live Bluetooth HFP link instead of renegotiating one, and foregrounding ensures the graph is
running rather than rebuilding it. The input side — input node, voice processing, and capture tap
— is built once per session rather than added when voice is joined, so joining and leaving voice
name a stream instead of replacing the graph. The input is rendered through a silent mixer,
because the voice-processing IO unit only stays up while the input is in the render chain. Route
changes no longer force a rebuild; media-services resets still do, and the stall watchdog now
backs off and stops after four failed attempts instead of rebuilding without end.
An explicit disconnect now pauses the prepared iOS graph and releases the audio session; the
next connection resumes that graph before attempting a rebuild. A signed iOS 26 simulator gate
verified guest-to-account editing, saved-password login after a process restart, two voice joins,
and a screen-audio stream started before the microphone. Screen sharing no longer forces a graph
rebuild, and its pump subscribes before announcing the stream. The pump retains up to 120 ms of
screen PCM captured during stream negotiation; the simulator gate now checks that this PCM
reaches the encoder and passed on iOS 26. Audible screen sharing was confirmed on an iPhone 16
Pro Max after deployment. An idle shared broadcast ring is no longer held mapped through suspension. A
TestFlight report from build 2026092504 was an iOS
`0xdead10cc` shared-file-lock termination; physical-device validation of that mitigation remains
open.
Stereo capture now selects the preferred input only after activating the audio session and no
longer calls `SetInputDataSource` with a source from a port that may not yet be current. A Debug
build installed on an iPhone 16 Pro Max connected with the StereoMicrophone preset and recorded
a two-channel input graph with distinct left/right PCM and encoded packets. The simulator's mono
input also completed stereo-preset login, voice join, disconnect, and reconnect.
Voice-chat capture is not yet reliable on the first attempt: a graph with voice processing still
sometimes fails to start and is only recovered by a watchdog rebuild, which costs seconds before
audio appears. Speaker output is an explicit choice that no longer changes the preset. Every
rebuild logs `VC_REBUILD cause=`, and `VC_START`/`VC_START_CHECK` record whether the graph
survived its start; that instrumentation is what the remaining investigation needs.
The media path now survives changing networks. A client whose source address changes proves
possession of its media key from the new address with an authenticated `Rebind` frame and the
relay moves its endpoint, instead of the session dying silently in both directions; the client
rebuilds its UDP socket rather than retrying on one pinned to a vanished interface. The control
connection is judged live by server traffic rather than assumed live, so a blackholed TCP path
is detected in 30 s instead of waiting minutes for the OS, and 12 s on iOS. iOS also watches the
system path and fails the control connection the moment the carrying interface changes, so a
handover reconnects in about a second instead of waiting out the silence timeout; access-point
roaming and an unusable-but-unchanged link are deliberately not handovers and are ridden out.
The first reconnect attempt is immediate, and a loss is only announced once an attempt has
actually failed, so a handover reads as a hiccup rather than a dropped call. A control reconnect
still re-authenticates and rejoins: seamless handover needs control-plane session resumption,
because the media keys come from the TLS exporter of the connection that was lost. The receive jitter buffer keeps a
one-frame depth floor, measures late and reordered arrivals, and can deepen mid-call, and a
stalled consumer now costs bounded audio rather than the live talkspurt.
SQLite schema v4 persists DRED and the channel packet-loss mode. Manual loss remains the default;
automatic Fast/Balanced/Stable modes measure each sender's authenticated UDP uplink at the server,
@@ -35,6 +91,12 @@ cap the applied Opus hint at 30%, and feed it back over TLS.
## Release gates
- Find why an iOS graph with voice processing intermittently starts and then stops within a
second, so voice-chat capture comes up on the first attempt rather than after a watchdog rebuild.
- On physical iOS 27 hardware, repeat account login after app restart, explicit disconnect and
reconnect, microphone permission denial, ReplayKit and ScreenCaptureKit sharing with and
without a joined microphone, and background/foreground transitions. Confirm the graph keeps
rendering and that `0xdead10cc` does not recur.
- Run real multi-person calls on Windows, macOS, and physical iOS hardware, including adaptive
20/40/60 ms buffering, duration-aware DRED/FEC, automatic packet-loss feedback, and mismatched
input/output endpoints.
@@ -44,11 +106,15 @@ cap the applied Opus hint at 30%, and feed it back over TLS.
- Exercise iOS background/lock, interruption, Bluetooth, route-change, ReplayKit, and iOS 27
ScreenCaptureKit paths on devices. The background/lock gate keeps a call active for 15+ minutes
backgrounded and screen-locked with no periodic glitches and flat `VC_AUDIO` `feedDrops`/`starved`
counters (the render callback now paces the mix and the 20 ms capture handoff). Complete a
30-minute iOS call and Wi-Fi/cellular switching with voice restoration, plus extended
counters (the render callback now paces the mix and the 20 ms capture handoff, and a watchdog
rebuilds a graph that stops calling back). Take a Siri or phone-call interruption while
backgrounded and confirm audio resumes without foregrounding. Complete a
30-minute iOS call and Wi-Fi/cellular switching with voice restoration (the switch is covered
by simulation in `NetworkImpairmentTests`; hardware confirms the real route change), plus extended
mono/stereo/voice-chat switching while joined. Verify Windows desktop/per-app stereo sharing.
- Complete Developer ID signing/notarization. The iOS host and ReplayKit extension have been
distribution-signed and packaged locally; upload the IPA for Apple's server-side validation.
distribution-signed and packaged as build `2026092701`; upload the IPA for Apple's server-side
validation.
- Run the published Linux container and a 30-minute-or-longer server soak.
## Working rule
+4 -1
View File
@@ -24,7 +24,10 @@ require `VOICECAT_CODESIGN_IDENTITY`; optional notarization uses `APPLE_ID`, `AP
For a physical iOS device, use `build-ios-device.sh` and `deploy-ios-device.sh`. The host and
ReplayKit extension require signing profiles with App Group `group.me.iamtalon.voicecat`.
Hardware validation must cover VoiceOver, background and lock behavior, interruptions, route
changes, Bluetooth, ReplayKit, and iOS 27 ScreenCaptureKit audio.
changes, Bluetooth, ReplayKit, and iOS 27 ScreenCaptureKit audio. The opt-in simulator share
smoke gate checks that ring PCM captured during stream startup reaches the encoder; audible
screen sharing was confirmed on an iPhone 16 Pro Max. The iOS 27 capture path still needs a
device check.
For iOS voice stability, leave a call joined with the microphone active for at least 30 minutes
and confirm speech stays clear and `VC_AUDIO` reports no growing `feedDrops`. While still joined,
toggle Wi-Fi off and on, switch between Wi-Fi and cellular, and confirm the app stays open,
@@ -6,9 +6,18 @@ namespace VoiceCat.iOS;
internal sealed class UsersController : UITableViewController
{
private readonly AppModel model; private IReadOnlyList<User> Visible => model.CurrentChannelId == 0 ? model.Users : model.Users.Where(user => user.ChannelId == model.CurrentChannelId).ToArray();
internal UsersController(AppModel model) { this.model = model; Title = "Users"; model.Changed += () => TableView.ReloadData(); }
private readonly AppModel model; private readonly ListRefresher refresher;
private IReadOnlyList<User> Visible => model.CurrentChannelId == 0 ? model.Users : model.Users.Where(user => user.ChannelId == model.CurrentChannelId).ToArray();
internal UsersController(AppModel model)
{
this.model = model; Title = "Users";
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
}
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "user"); }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature() => string.Join('\n', Visible.Select(user =>
$"{user.Id}|{user.Nickname}|{user.Id == model.SelfUserId}|{user.ServerDeafened}|{user.ServerMuted}|{user.SelfDeafened}|{user.SelfMicMuted}|{user.IsGuest}"));
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
@@ -34,7 +43,7 @@ internal sealed class UsersController : UITableViewController
internal sealed class UserDetailController : FormController
{
private readonly AppModel model; private readonly uint userId;
private readonly AppModel model; private readonly uint userId; private readonly ListRefresher refresher;
private User? User => model.Users.FirstOrDefault(value => value.Id == userId);
private IReadOnlyList<(string Title, Action Run, bool Destructive)> AdminActions
{
@@ -55,8 +64,20 @@ internal sealed class UserDetailController : FormController
}
}
internal UserDetailController(AppModel model, uint userId) : base(model.Users.FirstOrDefault(value => value.Id == userId)?.Nickname ?? $"User {userId}")
{ this.model = model; this.userId = userId; model.Changed += Reload; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "detail"); Reload(); }
{
this.model = model; this.userId = userId;
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
}
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "detail"); }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature()
{
User? user = User; Title = user?.Nickname ?? $"User {userId}";
string streams = user is null ? "offline" : string.Join(',', user.Streams.Select(stream =>
$"{stream.StreamId}:{stream.Kind}:{stream.Label}:{model.Client?.Audio.GetRemotePlayback(userId, stream.StreamId)}"));
return $"{Title}|{streams}|{string.Join(',', AdminActions.Select(action => action.Title))}";
}
public override nint NumberOfSections(UITableView tableView) => AdminActions.Count == 0 ? 2 : 3;
public override nint RowsInSection(UITableView tableView, nint section) => section switch
{ 0 => Math.Max(User?.Streams.Count ?? 0, 1), 1 => 1, _ => AdminActions.Count };
@@ -93,7 +114,6 @@ internal sealed class UserDetailController : FormController
else if (indexPath.Section == 1) NavigationController?.PushViewController(new PrivateConversationController(model, user.Id, user.Nickname), true);
else if (indexPath.Section == 2) AdminActions[indexPath.Row].Run();
}
private void Reload() { Title = User?.Nickname ?? $"User {userId}"; if (IsViewLoaded) TableView.ReloadData(); }
private void PromptReason(User user, bool ban) { UIAlertController prompt = UIAlertController.Create(ban ? "Ban user" : "Kick user", "Reason (optional)", UIAlertControllerStyle.Alert); prompt.AddTextField(field => field.AccessibilityLabel = "Reason"); prompt.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); prompt.AddAction(UIAlertAction.Create(ban ? "Ban" : "Kick", UIAlertActionStyle.Destructive, _ => Run(() => model.Client!.KickUserAsync(user.Id, prompt.TextFields?[0].Text ?? "")))); PresentViewController(prompt, true, null); }
private async void Run(Func<Task<GenericResult>> command) { try { GenericResult result = await model.RunAdminAsync(_ => command()); if (!result.Ok) throw new InvalidOperationException(result.Message); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
@@ -124,7 +144,16 @@ internal sealed class MoveUserController : FormController
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel"); NavigationItem.RightBarButtonItem = new("Move", UIBarButtonItemStyle.Done, async (_, _) => await Move()); }
public override nint RowsInSection(UITableView tableView, nint section) => model.Channels.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath) { Channel channel = model.Channels[indexPath.Row]; UITableViewCell cell = TextCell(tableView, indexPath, "channel", channel.Name); cell.Accessory = channel.Id == selected ? UITableViewCellAccessory.Checkmark : UITableViewCellAccessory.None; cell.AccessibilityLabel = channel.Name + (channel.Id == selected ? ", selected" : ""); return cell; }
public override void RowSelected(UITableView tableView, NSIndexPath indexPath) { selected = model.Channels[indexPath.Row].Id; tableView.ReloadData(); }
// Only the two checkmarks move. Reloading the whole table would rebuild every accessibility
// element and drop VoiceOver's focus on the row that was just chosen.
public override void RowSelected(UITableView tableView, NSIndexPath indexPath)
{
int previous = model.Channels.ToList().FindIndex(channel => channel.Id == selected);
selected = model.Channels[indexPath.Row].Id;
NSIndexPath[] rows = previous >= 0 && previous != indexPath.Row
? [indexPath, NSIndexPath.FromRowSection(previous, 0)] : [indexPath];
tableView.ReloadRows(rows, UITableViewRowAnimation.None);
}
private async Task Move() { try { GenericResult result = await model.RunAdminAsync(client => client.MoveUserAsync(user.Id, selected)); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
}
@@ -9,6 +9,9 @@ internal sealed class AppDelegate : UIApplicationDelegate
public override bool FinishedLaunching(UIApplication application, NSDictionary? launchOptions)
{
AppModel.Shared.Load();
#if DEBUG
SimulatorSmoke.RunIfRequested();
#endif
return true;
}
+97 -22
View File
@@ -34,8 +34,12 @@ internal sealed class AppModel
private bool restoreDeafened;
private bool backgrounded;
private int diagnosticPolls;
private bool announcedConnectionLoss;
private bool handingOver;
internal event Action? Changed;
// Raised by the 20 Hz level timer only. Subscribers must be cheap and must not reload a list.
internal event Action? LevelChanged;
internal event Action<ServerIdentityChallenge>? IdentityRequested;
internal IReadOnlyList<ServerProfile> Profiles => profiles;
internal IReadOnlyList<ChatEntry> Messages => messages;
@@ -47,6 +51,10 @@ internal sealed class AppModel
internal bool IsConnecting { get; private set; }
internal bool VoiceJoined => microphoneStream != 0;
internal bool ScreenSharing => broadcast?.IsActive == true;
#if DEBUG
internal bool ScreenRingMapped => broadcast?.IsMappingOpen == true;
internal long ScreenEncodedPackets => broadcast?.EncodedPackets ?? 0;
#endif
internal bool IsBackgrounded => backgrounded;
internal float MicrophoneLevel { get; private set; }
internal string Status { get; private set; } = "Not connected";
@@ -55,8 +63,24 @@ internal sealed class AppModel
internal IReadOnlyList<Channel> Channels => client?.Channels ?? [];
internal IReadOnlyList<User> Users => client?.Users ?? [];
private AppModel() { feedback = new(settings); }
internal void Load() { profiles.Clear(); profiles.AddRange(storage.LoadProfiles()); settings.Load(); IosAudioRouter.Shared.Load(); Notify(); }
private AppModel() { feedback = new(settings); IosNetworkPathMonitor.Shared.InterfaceChanged += OnInterfaceChanged; }
internal void Load() { profiles.Clear(); profiles.AddRange(storage.LoadProfiles()); settings.Load(); IosAudioRouter.Shared.Load(); IosNetworkPathMonitor.Shared.Start(); Notify(); }
// A changed interface has already stranded the control socket on a source address that is
// gone; the media transport rebinds itself, but TCP cannot, and waiting for the keepalive to
// notice costs the user ten seconds of a session that is already dead. Fail it now so the
// reconnect runs while the audio session is still up. Raised on the monitor queue.
private void OnInterfaceChanged(string path)
{
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
VoiceCatClient? owner = client;
if (owner is null || explicitDisconnect || IsConnecting || owner.State != ClientConnectionState.Connected) return;
AddActivity("Network changed; reconnecting.");
handingOver = true; reconnectAttempt = 0;
owner.DropForReconnect($"The network interface changed to {path}.");
});
}
internal void Save() { storage.SaveProfiles(profiles); settings.Save(); }
internal void DidEnterBackground()
@@ -70,7 +94,7 @@ internal sealed class AppModel
internal void WillEnterForeground()
{
backgrounded = false;
IosAudioRouter.Shared.Recover("foreground");
IosAudioRouter.Shared.ResumeForeground();
}
internal void DidBecomeActive()
@@ -103,6 +127,10 @@ internal sealed class AppModel
// handoff; a sleep-paced audio worker stalls when iOS coalesces backgrounded wakeups and
// the call glitches after several minutes in the background.
VoiceCatClient next = new("VoiceCat-iOS", "0.0.1", storage.TofuPath, deviceClockedAudio: true);
// A phone changes path often and the OS reports a blackholed TCP connection late or never.
// The path monitor catches a real handover immediately; this is the backstop for the cases
// it cannot see, such as a NAT rebinding or an upstream route that quietly stops carrying.
next.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(12));
next.ConnectionStateChanged += state =>
{
if (state == ClientConnectionState.Disconnected && next.ConnectionFailure is { } failure)
@@ -129,18 +157,29 @@ internal sealed class AppModel
_ = PumpEventsAsync(next, lifetime.Token);
levelTimer?.Dispose(); levelTimer = new(_ =>
UIApplication.SharedApplication.BeginInvokeOnMainThread(PollAudio), null, 50, 50);
feedback.Play(SoundEvent.Login); feedback.Speak(restoring ? "Reconnected" : "Connected");
// A handover that recovers in under a second should read as a hiccup, not a dropped
// call: announce the return only when the loss itself was announced.
if (!restoring || announcedConnectionLoss) { feedback.Play(SoundEvent.Login); feedback.Speak(restoring ? "Reconnected" : "Connected"); }
announcedConnectionLoss = false; handingOver = false;
if (restoring && restoreChannel != 0) await RestoreSessionAsync(next);
Notify();
}
catch (Exception exception)
{
System.Diagnostics.Debug.WriteLine($"Connection failed: {exception}");
Console.Error.WriteLine($"VoiceCat connection setup failed: {exception}");
IsConnecting = false; Status = exception.Message;
if (ReferenceEquals(client, next)) { client = null; await StopSessionResourcesAsync(); }
if (ReferenceEquals(client, next)) { client = null; await StopSessionResourcesAsync(releaseAudio: !restoring); }
await next.DisposeAsync();
Notify();
if (restoring && !explicitDisconnect) ScheduleReconnect();
if (restoring && !explicitDisconnect)
{
if (!announcedConnectionLoss)
{
announcedConnectionLoss = true; handingOver = false;
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting");
}
ScheduleReconnect();
}
else throw;
}
}
@@ -199,11 +238,15 @@ internal sealed class AppModel
// Handle that state change directly so a Wi-Fi transition cannot strand this session.
CaptureRestoreState(owner);
client = null; Status = "Connection lost";
try { await StopSessionResourcesAsync(); }
try { await StopSessionResourcesAsync(releaseAudio: false); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio cleanup failed: {exception}"); }
try { await owner.DisposeAsync(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Connection cleanup failed: {exception}"); }
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting"); Notify();
// On a handover the reconnect below usually lands within a second, so stay quiet and let
// the first failed attempt be what tells the user. Any other loss is announced at once.
if (handingOver) Status = "Reconnecting…";
else { announcedConnectionLoss = true; feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting"); }
Notify();
ScheduleReconnect();
}
@@ -231,15 +274,35 @@ internal sealed class AppModel
if (microphoneStream != 0)
{
IosAudioEngine.Shared.StopMicrophone(); active.StopStream(microphoneStream); microphoneStream = 0;
await active.SubscribeVoiceAsync(false); MicrophoneLevel = 0; feedback.Play(SoundEvent.VoiceOff); Notify(); return;
if (!ScreenSharing) await active.SubscribeVoiceAsync(false);
MicrophoneLevel = 0; feedback.Play(SoundEvent.VoiceOff); Notify(); return;
}
if (AVFoundation.AVCaptureDevice.GetAuthorizationStatus(AVFoundation.AVAuthorizationMediaType.Audio) == AVFoundation.AVAuthorizationStatus.NotDetermined)
await AVFoundation.AVCaptureDevice.RequestAccessForMediaTypeAsync(AVFoundation.AVAuthorizationMediaType.Audio);
AVFoundation.AVAuthorizationStatus authorization = AVFoundation.AVCaptureDevice.GetAuthorizationStatus(AVFoundation.AVAuthorizationMediaType.Audio);
if (authorization == AVFoundation.AVAuthorizationStatus.NotDetermined)
authorization = await AVFoundation.AVCaptureDevice.RequestAccessForMediaTypeAsync(AVFoundation.AVAuthorizationMediaType.Audio)
? AVFoundation.AVAuthorizationStatus.Authorized : AVFoundation.AVAuthorizationStatus.Denied;
if (authorization != AVFoundation.AVAuthorizationStatus.Authorized)
throw new InvalidOperationException("Microphone access is required to join voice. Allow it in iOS Settings.");
VoiceSubscriptionResult subscribed = await active.SubscribeVoiceAsync();
if (!subscribed.Ok) throw new InvalidOperationException(subscribed.Error);
uint startedStream = 0;
try
{
int channels = IosAudioRouter.Shared.CaptureChannels;
StreamInfo stream = await active.StartStreamAsync(StreamKind.StreamMic, "Microphone", channels);
microphoneStream = stream.StreamId; IosAudioEngine.Shared.StartMicrophone(stream.StreamId, channels); feedback.Play(SoundEvent.VoiceOn); Notify();
startedStream = stream.StreamId;
IosAudioEngine.Shared.StartMicrophone(stream.StreamId, channels);
microphoneStream = stream.StreamId; feedback.Play(SoundEvent.VoiceOn); Notify();
}
catch
{
IosAudioEngine.Shared.StopMicrophone();
if (startedStream != 0)
try { active.StopStream(startedStream); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
if (!ScreenSharing)
try { await active.SubscribeVoiceAsync(false); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
throw;
}
}
internal void ToggleScreenAudio()
@@ -273,7 +336,8 @@ internal sealed class AppModel
internal async Task DisconnectAsync()
{
explicitDisconnect = true; lifetime?.Cancel(); await StopSessionResourcesAsync();
explicitDisconnect = true; handingOver = false; announcedConnectionLoss = false;
lifetime?.Cancel(); await StopSessionResourcesAsync(releaseAudio: true);
VoiceCatClient? old = client; client = null; microphoneStream = 0; IsConnecting = false; Status = "Not connected"; Notify();
if (old is not null) await old.DisposeAsync(); feedback.Play(SoundEvent.Logout); feedback.Speak("Disconnected");
}
@@ -281,7 +345,10 @@ internal sealed class AppModel
private void ScheduleReconnect()
{
ServerProfile? profile = connectedProfile; if (profile is null || explicitDisconnect) return;
int delay = Math.Min(1 << Math.Min(reconnectAttempt++, 5), 30);
// The first attempt runs immediately: after a handover the new path is already up, and a
// second of deliberate silence is the difference between a hiccup and a dropped call.
int delay = reconnectAttempt == 0 ? 0 : Math.Min(1 << Math.Min(reconnectAttempt, 5), 30);
reconnectAttempt++;
CancellationToken token = lifetime?.Token ?? default;
_ = Task.Run(async () =>
{
@@ -325,7 +392,10 @@ internal sealed class AppModel
catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { return; }
lastTalking = talking; feedback.Play(talking ? SoundEvent.VoiceStart : SoundEvent.VoiceStop);
}
Notify();
// Only the voice bar renders the level. Raising the general Changed event at 20 Hz made
// every list reload itself that often, which tears down VoiceOver's element tree under an
// exploring finger; keep the fast signal on its own event.
LevelChanged?.Invoke();
}
private void HandleUserEvent(VoiceCatClient owner, UserEvent value)
@@ -344,10 +414,9 @@ internal sealed class AppModel
{
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
// Presenting either system picker can replace or interrupt the app's audio session.
// Rebuild after the producer becomes active so playback and an existing mic tap are
// attached to the session that will remain in use for the broadcast.
if (ScreenSharing) IosAudioRouter.Shared.Recover("screen sharing started");
// Presenting either picker can change the audio route. Keep a healthy graph; if the
// picker stopped it, Recover resumes the same graph or rebuilds for changed hardware.
IosAudioRouter.Shared.Recover(ScreenSharing ? "screen sharing started" : "screen sharing stopped", force: false);
Notify();
});
}
@@ -367,9 +436,15 @@ internal sealed class AppModel
owner.SetSelfAudioState(restoreMuted, restoreDeafened); AddActivity($"Restored to channel {restoreChannel}{(restoringVoice ? " with voice" : "")}");
}
private async Task StopSessionResourcesAsync()
// `releaseAudio` distinguishes an ended session from an interrupted one. Ending releases the
// AVAudioSession, which on Bluetooth drops the HFP link and costs seconds of renegotiation on
// the way back; a lost connection is a transport event that changed nothing about the audio
// configuration, so it only unbinds the client and leaves the live route in place.
private async Task StopSessionResourcesAsync(bool releaseAudio)
{
levelTimer?.Dispose(); levelTimer = null; IosAudioEngine.Shared.Stop(); microphoneStream = 0; MicrophoneLevel = 0; lastTalking = false;
levelTimer?.Dispose(); levelTimer = null;
if (releaseAudio) IosAudioEngine.Shared.Stop(); else IosAudioEngine.Shared.Detach();
microphoneStream = 0; MicrophoneLevel = 0; lastTalking = false;
if (broadcast is { } pump) { broadcast = null; pump.Changed -= BroadcastChanged; await pump.DisposeAsync(); }
}
}
+112 -17
View File
@@ -9,15 +9,25 @@ internal sealed class BroadcastAudioPump : IAsyncDisposable
private const uint Magic = 0x56434252, Version = 1;
private const int Header = 64, Capacity = 96_000, Frame = 960;
private readonly CancellationTokenSource stop = new();
private readonly short[] scratch = new short[Frame * 2];
private Thread? worker;
private VoiceCatClient? client;
private MemoryMappedFile? map;
private MemoryMappedViewAccessor? view;
private string? path;
private uint streamId;
private bool active;
private int generation;
private readonly short[] scratch = new short[Frame * 2];
private ulong observedWrite;
private ulong idleWrite;
private bool idleWriteSeen;
internal event Action? Changed;
internal bool IsActive => active;
#if DEBUG
internal bool IsMappingOpen => view is not null;
internal long EncodedPackets => streamId == 0 ? 0 : client?.Audio.GetLocalDiagnostics(streamId).EncodedPackets ?? 0;
#endif
internal void Start(VoiceCatClient owner)
{
@@ -29,55 +39,140 @@ internal sealed class BroadcastAudioPump : IAsyncDisposable
private void Run()
{
CancellationToken token = stop.Token;
try
{
while (!token.IsCancellationRequested)
{
try { DrainAsync(token).GetAwaiter().GetResult(); }
catch (Exception exception) when (exception is IOException or UnauthorizedAccessException or InvalidDataException or OperationCanceledException) { }
if (!token.IsCancellationRequested) Thread.Sleep(5);
catch (OperationCanceledException) when (token.IsCancellationRequested) { break; }
catch (Exception exception)
{
// A rejected screen stream is a session error, not an unhandled exception on
// this background Thread. Keep the app and microphone call alive.
Console.Error.WriteLine($"VoiceCat screen audio pump failed: {exception}");
CloseMapping(); StopStream(); SetActive(false);
if (!token.IsCancellationRequested) Thread.Sleep(1000);
}
// The shared file must not remain mapped while the app is idle or suspending.
if (!token.IsCancellationRequested) Thread.Sleep(view is null ? 100 : 5);
}
}
finally { CloseMapping(); }
}
private async Task DrainAsync(CancellationToken token)
{
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
if (root?.Path is null) return;
string path = Path.Combine(root.Path, "voicecat", "broadcast_audio.ring"); if (!File.Exists(path)) return;
using MemoryMappedFile map = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
using MemoryMappedViewAccessor view = map.CreateViewAccessor(0, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
if (view.ReadUInt32(0) != Magic || view.ReadUInt32(4) != Version) throw new InvalidDataException("Unsupported broadcast ring.");
bool active = view.ReadUInt32(16) != 0;
if (!active) { StopStream(); SetActive(false); return; }
if (!OpenMapping()) return;
MemoryMappedViewAccessor pages = view!;
bool active = pages.ReadUInt32(16) != 0;
if (!active) { StopStream(); SetActive(false); CloseMapping(); return; }
VoiceCatClient owner = client ?? throw new IOException("Client disconnected.");
if (streamId == 0)
{
// A ring left marked active by a killed extension must not start a phantom stream.
// Wait for a fresh producer write after this connection opened the mapping.
ulong currentWrite = pages.ReadUInt64(24);
if (currentWrite == observedWrite) return;
observedWrite = currentWrite;
int startGeneration = Volatile.Read(ref generation);
VoiceSubscriptionResult subscription = await owner.SubscribeVoiceAsync(cancellationToken: token).ConfigureAwait(false);
if (!subscription.Ok) throw new InvalidOperationException(subscription.Error);
StreamInfo stream = await owner.StartStreamAsync(StreamKind.StreamScreenAudio, "Screen audio", 2, token).ConfigureAwait(false);
if (startGeneration != Volatile.Read(ref generation) || view.ReadUInt32(16) == 0 || !ReferenceEquals(client, owner))
if (startGeneration != Volatile.Read(ref generation) || pages.ReadUInt32(16) == 0 || !ReferenceEquals(client, owner))
{
try { owner.StopStream(stream.StreamId); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
return;
}
streamId = stream.StreamId; view.Write(32, view.ReadUInt64(24)); SetActive(true);
// Keep the newest 120 ms captured during stream negotiation. This covers the
// largest codec frame plus the input buffer target; dropping through the latest
// write loses short sounds before the first drain can feed them.
ulong latestWrite = pages.ReadUInt64(24);
ulong earliest = latestWrite > (ulong)(Frame * 12) ? latestWrite - (ulong)(Frame * 12) : 0;
pages.Write(32, Math.Max(pages.ReadUInt64(32), earliest));
streamId = stream.StreamId; SetActive(true);
}
ulong write = view.ReadUInt64(24), read = view.ReadUInt64(32);
ulong write = pages.ReadUInt64(24), read = pages.ReadUInt64(32);
if (write - read > Capacity) read = write - Capacity;
while (write - read >= Frame * 2)
{
for (int sample = 0; sample < scratch.Length; sample++)
{
ulong index = (read + (ulong)sample) % Capacity;
scratch[sample] = view.ReadInt16(Header + checked((long)index * sizeof(short)));
scratch[sample] = pages.ReadInt16(Header + checked((long)index * sizeof(short)));
}
if (!owner.Audio.FeedPcm(streamId, scratch, 2)) break;
read += (ulong)scratch.Length;
view.Write(32, read);
pages.Write(32, read);
}
}
// Keep a mapping only while a broadcast is active. An inactive mapping holds an open handle
// into the shared App Group container through suspension, which iOS may terminate as a shared
// file lock (0xdead10cc). Poll the small header with a short-lived read when idle, then map
// once for the active broadcast's high-rate drain.
private bool OpenMapping()
{
path ??= ResolvePath();
if (path is null) return false;
bool present = File.Exists(path);
if (view is not null)
{
if (present) return true;
CloseMapping(); return false;
}
if (!present) { idleWriteSeen = false; return false; }
using (var file = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete))
{
if (file.Length < Header) { idleWriteSeen = false; return false; }
Span<byte> header = stackalloc byte[32];
file.ReadExactly(header);
if (System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header) != Magic ||
System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header[4..]) != Version ||
System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header[16..]) == 0)
{
idleWriteSeen = false;
return false;
}
ulong currentWrite = System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(header[24..]);
if (!idleWriteSeen)
{
idleWrite = currentWrite;
idleWriteSeen = true;
return false;
}
if (currentWrite == idleWrite) return false;
}
MemoryMappedFile candidate = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
try
{
MemoryMappedViewAccessor pages = candidate.CreateViewAccessor(0, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
if (pages.ReadUInt32(0) != Magic || pages.ReadUInt32(4) != Version)
{
pages.Dispose(); throw new InvalidDataException("Unsupported broadcast ring.");
}
map = candidate; view = pages; observedWrite = idleWrite; return true;
}
catch { candidate.Dispose(); throw; }
}
private static string? ResolvePath()
{
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
return root?.Path is null ? null : Path.Combine(root.Path, "voicecat", "broadcast_audio.ring");
}
private void CloseMapping()
{
view?.Dispose(); view = null;
map?.Dispose(); map = null;
observedWrite = 0;
idleWriteSeen = false;
}
private void StopStream()
{
uint id = streamId; streamId = 0; if (id == 0 || client?.State != ClientConnectionState.Connected) return;
try { client.StopStream(id); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
try { client.StopStream(id); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
}
internal void RequestStop() { Interlocked.Increment(ref generation); SetActive(false); }
+198 -21
View File
@@ -39,35 +39,112 @@ internal sealed class IosAudioEngine
private AVAudioConverterInputHandler? inputProvider;
private bool inputProvided;
private bool tapInstalled;
private AVAudioMixerNode? captureSink;
// The voice-processing state the live graph was actually built with, so Reconfigure can tell
// a settings change apart from a re-check of an unchanged graph.
private bool voiceProcessing;
// The input format the tap, the converter and the ring capacity were all built for. Asked of
// the input node rather than of AVAudioSession: the session's reported rate and channel count
// do not settle until after the graph has started, so comparing against them reports a change
// that has not happened and every rebuild reports it again.
private long lastRebuildAt;
// How long a freshly built graph is given to produce its first render callback. Enabling voice
// processing rebuilds both halves of the IO, which takes several hundred milliseconds, posts a
// configuration change and reports the engine as not running while it happens. Everything that
// judges a graph dead has to wait this out, or it tears down the graph it just built and the
// replacement reports exactly the same thing.
private const int SettlingMilliseconds = 2_000;
private double builtInputRate;
private uint builtInputChannels;
// The capture width the tap and converter were built for. The input side of the graph exists
// for the whole session, so this is what decides whether joining voice needs a rebuild at all.
private int builtCaptureChannels;
private long captureCallbacks, capturedFrames, convertedFrames, rejectedFeeds, converterFailures, stereoFrames, stereoDifferentFrames;
private long renderCallbacks, lastRenderTimestamp;
internal bool IsConnected { get; private set; }
internal bool IsRunning => engine?.Running == true;
// True while the last rebuild is still starting up, and so cannot be judged.
internal bool Settling => Environment.TickCount64 - Volatile.Read(ref lastRebuildAt) < SettlingMilliseconds;
// True when the graph's own input no longer has the format its tap and converter were built
// for, which is the only thing a route change can do that AVAudioEngine cannot absorb on its
// own. The engine's view is the one that is stable: it changes when the hardware actually
// changes, which is precisely what a configuration change reports.
internal bool HardwareChanged()
{
if (engine is not { } live || !tapInstalled) return false;
AVAudioFormat format = live.InputNode.GetBusOutputFormat(0);
return format.SampleRate != builtInputRate || format.ChannelCount != builtInputChannels;
}
// The watchdog compares this across ticks: a graph that stops calling back while it still
// reports Running leaves the whole device-clocked pipeline frozen until it is rebuilt.
internal long RenderCallbacks => Interlocked.Read(ref renderCallbacks);
internal int BufferMilliseconds { get => playbackRing.BufferMilliseconds; set => playbackRing.BufferMilliseconds = value; }
internal void StartListening(VoiceCatClient owner)
{
Stop(); client = owner; IsConnected = true; owner.Audio.MixedPcm += ReceiveMixedPcm; Rebuild();
if (client is { } previous) previous.Audio.MixedPcm -= ReceiveMixedPcm;
client = owner; IsConnected = true; owner.Audio.MixedPcm += ReceiveMixedPcm;
// A reconnect after Detach finds the session active and the graph already running on the
// right hardware. Rebuilding it there would release an HFP headset and pay a Bluetooth
// profile renegotiation for a transport blip that changed no audio configuration.
if (engine?.Running == true) { playbackRing.Resynchronize(); return; }
if (ResumeStoppedGraph()) return;
Volatile.Write(ref microphone, null); Rebuild();
}
// Unbinds the client without touching the session or the graph, for a connection that was
// lost rather than ended. Capture keeps feeding a ring nobody drains and Render emits silence
// until StartListening rebinds, which keeps the route and its Bluetooth profile alive.
internal void Detach()
{
if (client is { } owner) owner.Audio.MixedPcm -= ReceiveMixedPcm;
client = null; playbackRing.Resynchronize();
// The route's stream id belongs to the connection that just died. Keep the tap and its
// hardware, but park the route on the unbound id so a rebind cannot feed the next
// connection a stream it never announced.
if (Volatile.Read(ref microphone) is { } stale && stale.StreamId != 0)
Volatile.Write(ref microphone, CreateMicrophoneRoute(0, stale.Channels));
}
internal void StartMicrophone(uint streamId, int channels)
{
Volatile.Write(ref microphone, CreateMicrophoneRoute(streamId, channels)); Rebuild();
// Capture is already running: the graph carries the tap for the whole session. Joining voice
// only names the stream the frames belong to, so it is a state change and not a rebuild. The
// width is the one exception, because the tap format and converter are built around it.
MicrophoneRoute next = CreateMicrophoneRoute(streamId, channels);
bool reusable = tapInstalled && engine?.Running == true && builtCaptureChannels == next.Channels;
Volatile.Write(ref microphone, next);
if (!reusable) Rebuild();
}
internal void StopMicrophone() { Volatile.Write(ref microphone, null); Rebuild(); }
internal void Reconfigure()
// Leaving voice never rebuilds. Capture keeps running into a route nobody reads, exactly as it
// does between connecting and joining, which costs one discarded conversion per callback and
// saves tearing down the voice-processing IO only to build it again on the next join.
internal void StopMicrophone() { Volatile.Write(ref microphone, null); }
// `force` rebuilds unconditionally, which is what a route change, a media-services reset and
// the stall watchdog all need. Callers that are only re-checking a graph they expect to be
// healthy — foregrounding, above all — pass false and get a no-op when nothing has changed.
internal void Reconfigure(bool force = true, string cause = "reconfigure")
{
if (!IsConnected) return;
// A graph that has not finished starting is not a graph to replace. Only an explicit
// configuration change forces its way past this.
if (!force && Settling) return;
MicrophoneRoute? current = Volatile.Read(ref microphone);
int channels = IosAudioRouter.Shared.CaptureChannels;
if (!force && engine?.Running == true && tapInstalled && builtCaptureChannels == channels
&& voiceProcessing == IosAudioRouter.Shared.UsesVoiceProcessing && !HardwareChanged()) return;
if (!force && ResumeStoppedGraph()) return;
if (current is not null)
{
// Stop the old tap before publishing a route with a different sample width. Otherwise
// an in-flight callback could interpret its old converter buffer using the new width.
DestroyGraph();
if (current.Channels != channels) client!.Audio.SetCaptureChannels(current.StreamId, channels);
if (current.Channels != channels && current.StreamId != 0) client?.Audio.SetCaptureChannels(current.StreamId, channels);
Volatile.Write(ref microphone, CreateMicrophoneRoute(current.StreamId, channels));
}
Rebuild();
Rebuild(cause);
}
private static MicrophoneRoute CreateMicrophoneRoute(uint streamId, int channels)
@@ -77,21 +154,58 @@ internal sealed class IosAudioEngine
internal bool EnsureRunning()
{
if (!IsConnected || engine?.Running == true) return true;
// Still starting: report it as running rather than replacing it, since that is what it is
// about to be, and a rebuild here would start the cycle over.
if (Settling) return true;
if (ResumeStoppedGraph()) return true;
Rebuild(); return engine?.Running == true;
}
private void Rebuild()
private bool ResumeStoppedGraph()
{
DestroyGraph(); MicrophoneRoute? route = Volatile.Read(ref microphone); IosAudioRouter.Shared.Apply(route is not null);
if (engine is not { } paused || !tapInstalled ||
builtCaptureChannels != IosAudioRouter.Shared.CaptureChannels ||
voiceProcessing != IosAudioRouter.Shared.UsesVoiceProcessing || HardwareChanged()) return false;
IosAudioRouter.Shared.Apply(true);
bool started = paused.StartAndReturnError(out NSError? error);
if (started)
{
playbackRing.Resynchronize();
// Voice-processing IO may still be starting when Start returns. Give the existing
// graph its settling window before a watchdog judges it, just as a fresh graph gets.
Volatile.Write(ref lastRebuildAt, Environment.TickCount64);
Console.Error.WriteLine($"VC_RESUME running={paused.Running}");
return true;
}
Console.Error.WriteLine($"VC_RESUME running={paused.Running} error={error?.LocalizedDescription ?? "none"}");
return false;
}
private void Rebuild([System.Runtime.CompilerServices.CallerMemberName] string cause = "")
{
Console.Error.WriteLine($"VC_REBUILD cause={cause} running={engine?.Running == true} callbacks={RenderCallbacks}");
DestroyGraph(); MicrophoneRoute? route = Volatile.Read(ref microphone);
// The input node, voice processing and the tap are built once and kept for the session, not
// added when voice is joined. Adding them later means replacing a running graph that has no
// voice processing with one that has it, and that transition is what fails: the new IO unit
// starts and is torn down again within a second, non-deterministically, which is the
// rebuild storm the watchdog then chases. Steady-state voice processing is reliable; only
// the change into it is not. So capture always runs, and joining voice only decides which
// stream its frames belong to — Capture and PumpMicrophoneChunk both discard without one.
bool captures = IsConnected;
int captureChannels = route?.Channels ?? Math.Clamp(IosAudioRouter.Shared.CaptureChannels, 1, 2);
IosAudioRouter.Shared.Apply(captures);
voiceProcessing = IosAudioRouter.Shared.UsesVoiceProcessing;
builtInputRate = 0; builtInputChannels = 0;
var next = new AVAudioEngine();
AVAudioInputNode? input = null;
if (route is not null)
if (captures)
{
// Enabling voice processing rebuilds both sides of AVAudioEngine. Do it before any
// formats are queried or nodes are connected so the graph is built from the final IO.
input = next.InputNode;
if (!input.SetVoiceProcessingEnabled(IosAudioRouter.Shared.UsesVoiceProcessing, out NSError? processingError))
throw new InvalidOperationException(processingError.LocalizedDescription);
throw new InvalidOperationException(processingError?.LocalizedDescription ?? "Could not configure voice processing.");
if (IosAudioRouter.Shared.UsesVoiceProcessing) input.VoiceProcessingAgcEnabled = IosAudioRouter.Shared.AutomaticGainControl;
}
outputFormat = new(AVAudioCommonFormat.PCMFloat32, 48_000, 2, false);
@@ -101,12 +215,16 @@ internal sealed class IosAudioEngine
if (OperatingSystem.IsIOSVersionAtLeast(27)) next.Connect(source, next.MainMixerNode, outputFormat, out connectionError);
else next.Connect(source, next.MainMixerNode, outputFormat);
if (connectionError is not null) throw new InvalidOperationException(connectionError.LocalizedDescription);
if (route is not null)
if (captures)
{
AVAudioFormat inputFormat = input!.GetBusOutputFormat(0);
Console.Error.WriteLine($"VC_GRAPH vpio={IosAudioRouter.Shared.UsesVoiceProcessing} requestedCh={route.Channels} " +
if (inputFormat.SampleRate <= 0 || inputFormat.ChannelCount == 0)
throw new InvalidOperationException("No usable microphone input is available on the current audio route.");
Console.Error.WriteLine($"VC_GRAPH vpio={IosAudioRouter.Shared.UsesVoiceProcessing} requestedCh={captureChannels} " +
$"inputCh={inputFormat.ChannelCount} inputRate={inputFormat.SampleRate}");
microphoneFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, (uint)route.Channels, true);
builtInputRate = inputFormat.SampleRate; builtInputChannels = inputFormat.ChannelCount;
microphoneFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, (uint)captureChannels, true);
builtCaptureChannels = captureChannels;
microphoneConverter = new(inputFormat, microphoneFormat);
uint capacity = checked((uint)Math.Ceiling(MaximumCaptureCallbackFrames * 48_000 / inputFormat.SampleRate) + 64);
convertedMicrophone = new(microphoneFormat, capacity);
@@ -116,17 +234,64 @@ internal sealed class IosAudioEngine
else input.InstallTapOnBus(0, 960, inputFormat, Capture);
if (tapError is not null) throw new InvalidOperationException(tapError.LocalizedDescription);
tapInstalled = true;
// With voice processing the input and output run as one IO unit, and the unit only stays
// up while the input is part of the render chain. A tap alone does not put it there: the
// engine starts and the IO is torn down again within a second, which is why a graph with
// voice processing came up perhaps one time in four. Route the input through a silent
// mixer so it is genuinely rendered, contributing nothing audible.
captureSink = new AVAudioMixerNode();
next.AttachNode(captureSink);
NSError? sinkError = null;
if (OperatingSystem.IsIOSVersionAtLeast(27))
{
next.Connect(input, captureSink, inputFormat, out sinkError);
if (sinkError is null) next.Connect(captureSink, next.MainMixerNode, outputFormat, out sinkError);
}
else
{
next.Connect(input, captureSink, inputFormat);
next.Connect(captureSink, next.MainMixerNode, outputFormat);
}
if (sinkError is not null) throw new InvalidOperationException(sinkError.LocalizedDescription);
captureSink.OutputVolume = 0f;
}
next.Prepare();
if (!next.StartAndReturnError(out NSError? error)) { next.Dispose(); throw new InvalidOperationException(error.LocalizedDescription); }
if (!next.StartAndReturnError(out NSError? error)) { next.Dispose(); throw new InvalidOperationException(error?.LocalizedDescription ?? "The iOS audio graph could not start."); }
engine = next;
{
AVAudioSession live = AVAudioSession.SharedInstance();
Console.Error.WriteLine($"VC_START running={next.Running} builtCh={builtInputChannels} sessionRate={live.SampleRate} " +
$"sessionInCh={live.InputNumberOfChannels} sessionOutCh={live.OutputNumberOfChannels} inputAvailable={live.InputAvailable} " +
$"other={live.OtherAudioPlaying} mode={live.Mode} options={live.CategoryOptions} io={live.IOBufferDuration:F4} " +
$"out={string.Join(',', live.CurrentRoute?.Outputs?.Select(value => value.PortType.ToString()) ?? [])} " +
$"in={string.Join(',', live.CurrentRoute?.Inputs?.Select(value => value.PortType.ToString()) ?? [])}");
// A graph that starts and then stops on its own is the failure that matters, and it is
// invisible at start: check again once the IO has had time to come up.
AVAudioEngine started = next;
System.Threading.Tasks.Task.Delay(750).ContinueWith(_ =>
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
if (!ReferenceEquals(engine, started)) return;
Console.Error.WriteLine($"VC_START_CHECK running={started.Running} render={RenderCallbacks} capture={Interlocked.Read(ref captureCallbacks)}");
}));
}
Volatile.Write(ref lastRebuildAt, Environment.TickCount64);
engineConfigurationObserver = Foundation.NSNotificationCenter.DefaultCenter.AddObserver(
AVAudioEngine.ConfigurationChangeNotification, notification =>
{
if (!ReferenceEquals(notification.Object, next)) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
if (IsConnected && ReferenceEquals(engine, next) && !next.Running) Rebuild();
if (!IsConnected || !ReferenceEquals(engine, next)) return;
// Building this graph is itself what posted most of these: enabling voice
// processing rebuilds the IO, and the engine reads as stopped until that
// finishes. Rebuilding then replaces a graph that was about to run with one
// that reports the same thing, without end.
if (Settling) return;
// A configuration change with the graph still running is usually one
// AVAudioEngine has already absorbed; it matters here only when the hardware
// the tap and converter were built around moved.
if (!next.Running || HardwareChanged()) Rebuild("configuration change");
});
}, next);
}
@@ -184,7 +349,8 @@ internal sealed class IosAudioEngine
private void PumpMicrophoneChunk(VoiceCatClient owner)
{
MicrophoneRoute? route = Volatile.Read(ref microphone);
if (route is null) return;
// Stream id 0 is a route parked by Detach: still capturing, not yet bound to a connection.
if (route is null || route.StreamId == 0) return;
int required = 960 * route.Channels;
if (route.Ring.Read(microphoneFrame.AsSpan(0, required)) == required &&
ReferenceEquals(route, Volatile.Read(ref microphone)) &&
@@ -201,15 +367,24 @@ internal sealed class IosAudioEngine
// This callback is the cadence iOS keeps exact while the app is backgrounded or the device
// is locked, so it owns both managed 20 ms hands-offs: capture into the sender and one mix
// cycle per 20 ms of render demand. Both run allocation-free and without locks or I/O.
Interlocked.Increment(ref renderCallbacks);
long now = System.Diagnostics.Stopwatch.GetTimestamp(), previous = lastRenderTimestamp;
lastRenderTimestamp = now;
VoiceCatClient? owner = Volatile.Read(ref client);
if (owner is null || !IsConnected) microphoneCredit = 0;
else
{
if (previous != 0 && (now - previous) * 1000.0 / System.Diagnostics.Stopwatch.Frequency > 100)
{
Volatile.Read(ref microphone)?.Ring.Resynchronize();
playbackRing.Resynchronize(); owner.Audio.ResynchronizeInputs(); microphoneCredit = 0;
}
microphoneCredit += frames;
while (microphoneCredit >= 960) { microphoneCredit -= 960; PumpMicrophoneChunk(owner); }
// Top the mix ring up to cover this callback plus its configured target so the read
// below never starves and the buffer keeps its chosen buffering latency.
int deficit = frames + playbackRing.TargetFrames - playbackRing.CountFrames;
// below never starves and the buffer keeps its chosen buffering latency. Catch-up is
// capped at one extra cycle so a refill cannot overrun this callback's deadline.
int deficit = Math.Min(frames + playbackRing.TargetFrames - playbackRing.CountFrames, frames + 960);
for (int produced = 0; produced < deficit; produced += 960) owner.Audio.RunCycle();
}
Span<short> input = renderScratch.AsSpan(0, requested);
@@ -230,7 +405,7 @@ internal sealed class IosAudioEngine
{
IsConnected = false; Volatile.Write(ref microphone, null);
if (client is { } owner) owner.Audio.MixedPcm -= ReceiveMixedPcm;
DestroyGraph(); client = null; IosAudioRouter.Shared.Deactivate();
engine?.Pause(); client = null; IosAudioRouter.Shared.Deactivate();
}
private void DestroyGraph()
@@ -243,9 +418,11 @@ internal sealed class IosAudioEngine
if (engine is { } old)
{
if (tapInstalled) old.InputNode.RemoveTapOnBus(0);
old.Stop(); if (source is not null) old.DetachNode(source); old.Dispose();
old.Stop(); if (source is not null) old.DetachNode(source);
if (captureSink is not null) { old.DetachNode(captureSink); captureSink.Dispose(); captureSink = null; }
old.Dispose();
}
tapInstalled = false; pendingInput = null; inputProvider = null;
tapInstalled = false; pendingInput = null; inputProvider = null; lastRenderTimestamp = 0; microphoneCredit = 0;
convertedMicrophone?.Dispose(); convertedMicrophone = null;
microphoneConverter?.Dispose(); microphoneConverter = null;
microphoneFormat?.Dispose(); microphoneFormat = null;
+157 -24
View File
@@ -18,6 +18,27 @@ internal sealed class IosAudioRouter
internal static IosAudioRouter Shared { get; } = new();
private readonly NSUserDefaults defaults = NSUserDefaults.StandardUserDefaults;
private bool applying;
private bool speakerIsExplicit;
// Whether this session actually has a stereo capsule configuration to undo. Clearing one that
// was never applied is not free: SetPreferredPolarPattern(Unknown) drops the built-in array out
// of the beamformed mono configuration VoiceChat mode selects and exposes its four raw
// channels, which the voice-processing IO cannot start against.
private bool stereoApplied;
// Set by an explicit speaker choice and consumed by the next Apply. The override is a one-shot
// request, never steady-state configuration; see SetForceSpeaker.
private bool overridePending;
private System.Threading.Timer? watchdog;
private long lastRenderCallbacks = -1;
private int watchdogMisses;
private int watchdogAttempts;
private long nextWatchdogAttempt;
// A rebuild that does not restore the callbacks is not worth repeating at the same rate, and
// never worth repeating forever: an unbounded retry turns a graph that cannot start into a
// storm of session reconfiguration, which is both what the user hears and what hides the
// reason from the log. Back off, then stop and say so.
private const int MaximumWatchdogAttempts = 4;
private bool watchdogTicking;
private bool interrupted;
internal event Action? Changed;
internal IosAudioPreset Preset { get; private set; } = IosAudioPreset.VoiceChat;
internal IosBluetoothMode BluetoothMode { get; private set; } = IosBluetoothMode.HfpVoice;
@@ -46,11 +67,16 @@ internal sealed class IosAudioRouter
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.preset"), true, out IosAudioPreset preset)) Preset = preset;
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.bluetoothMode"), true, out IosBluetoothMode bluetooth)) BluetoothMode = bluetooth;
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.micMode"), true, out IosMicMode mic)) MicMode = mic;
ForceSpeaker = defaults.BoolForKey("cat.voice.audio.forceSpeaker");
// Speaker output is an output-routing choice, orthogonal to the capture preset, which is
// why it sits outside Advanced audio. An install that predates the explicit choice carries
// a speaker flag the preset set on its behalf; drop that once so it cannot pin a headset
// user to the speaker, and honour the toggle from then on.
if (defaults.BoolForKey("cat.voice.audio.speakerIsExplicit")) ForceSpeaker = defaults.BoolForKey("cat.voice.audio.forceSpeaker");
VoiceProcessing = defaults.ValueForKey(new NSString("cat.voice.audio.voiceProcessing")) is null || defaults.BoolForKey("cat.voice.audio.voiceProcessing");
AutomaticGainControl = defaults.ValueForKey(new NSString("cat.voice.audio.agc")) is null || defaults.BoolForKey("cat.voice.audio.agc");
CaptureChannels = defaults.IntForKey("cat.voice.audio.captureChannels") == 2 ? 2 : Preset == IosAudioPreset.StereoMicrophone ? 2 : 1;
SelectedInputId = defaults.StringForKey("cat.voice.audio.inputPortId"); SelectedDataSourceId = defaults.StringForKey("cat.voice.audio.dataSourceId");
if (Preset == IosAudioPreset.VoiceChat) { SelectedInputId = null; SelectedDataSourceId = null; }
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.polarPattern"), true, out AVAudioDataSourcePolarPattern pattern)) SelectedPolarPattern = pattern;
RefreshRoutes();
}
@@ -68,12 +94,19 @@ internal sealed class IosAudioRouter
// Named presets never carry an Advanced capsule selection across transitions.
// Stereo derives its data source below; mono/voice chat must clear a stale stereo one.
SelectedDataSourceId = null; SelectedPolarPattern = AVAudioDataSourcePolarPattern.Unknown;
if (preset == IosAudioPreset.VoiceChat) ForceSpeaker = true;
}
SaveAndReconfigure();
}
internal void SetForceSpeaker(bool value) { ForceSpeaker = value; SaveAndReconfigure(); }
// Deliberately does not move the preset to Advanced: the speaker is an output choice every
// named preset supports, so a voice-chat user can take a call on the speaker without losing
// the capture configuration the preset stands for.
// The port override is issued once, on the toggle, and never re-issued by a later rebuild.
// Re-asserting it on every Apply means fighting iOS for the route: where the system wants to
// hand output back to a connected headset, each rebuild forces it back to the speaker, the
// route change that follows drives another rebuild, and the audio flips back and forth. The
// DefaultToSpeaker category option below is the part that does persist across rebuilds.
internal void SetForceSpeaker(bool value) { ForceSpeaker = value; speakerIsExplicit = overridePending = true; SaveAndReconfigure(); }
internal void SetVoiceProcessing(bool value) { VoiceProcessing = value; SaveAndReconfigure(); }
internal void SetAutomaticGainControl(bool value) { AutomaticGainControl = value; SaveAndReconfigure(); }
internal void SetCaptureChannels(int value) { CaptureChannels = value == 2 ? 2 : 1; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
@@ -94,8 +127,11 @@ internal sealed class IosAudioRouter
{
AVAudioSession session = AVAudioSession.SharedInstance();
Inputs = session.AvailableInputs?.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray() ?? [];
Outputs = session.CurrentRoute.Outputs.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray();
SelectedInputId ??= session.PreferredInput?.UID; Changed?.Invoke();
Outputs = session.CurrentRoute?.Outputs?.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray() ?? [];
// A route reported by iOS is not an explicit user input choice. Capturing it here
// can pin the built-in mic after a speaker fallback and displace a Bluetooth HFP route
// on the next graph rebuild.
Changed?.Invoke();
}
internal void Apply(bool configureInput)
@@ -109,14 +145,25 @@ internal sealed class IosAudioRouter
if (BluetoothMode == IosBluetoothMode.BuiltInMicSpeaker || ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp) options |= AVAudioSessionCategoryOptions.DefaultToSpeaker;
AVAudioSessionMode mode = CaptureChannels == 2 ? AVAudioSessionMode.Default : MicMode == IosMicMode.Raw ? AVAudioSessionMode.Measurement
: BluetoothMode == IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionMode.VideoRecording : AVAudioSessionMode.VoiceChat;
if (!session.SetCategory(AVAudioSessionCategory.PlayAndRecord, mode, options, out NSError? categoryError)) throw new InvalidOperationException(categoryError.LocalizedDescription);
if (!session.SetCategory(AVAudioSessionCategory.PlayAndRecord, mode, options, out NSError? categoryError)) throw new InvalidOperationException(categoryError?.LocalizedDescription ?? "Could not configure the iOS audio session.");
session.SetPreferredSampleRate(48_000, out _); session.SetPreferredIOBufferDuration(0.02, out _);
if (!session.SetActive(true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? activeError)) throw new InvalidOperationException(activeError?.LocalizedDescription ?? "Could not activate the iOS audio session.");
// Apple requires an active session before selecting a preferred input or data source.
if (configureInput) ApplyInputSelection(session);
if (!session.SetActive(true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? activeError)) throw new InvalidOperationException(activeError.LocalizedDescription);
Console.Error.WriteLine($"VC_ROUTE preset={Preset} requestedCh={CaptureChannels} sessionCh={session.InputNumberOfChannels} " +
$"preferred={session.PreferredInput?.PortName ?? "default"} dataSource={session.InputDataSource?.DataSourceName ?? "default"} " +
$"pattern={session.InputDataSource?.SelectedPolarPattern.ToString() ?? "default"}");
session.OverrideOutputAudioPort(ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _); RefreshRoutes();
// DefaultToSpeaker only decides where audio goes when nothing else is connected, so a
// user who asks for the speaker with a headset attached needs the port override as
// well. Only the toggle itself issues it, and only once.
if (overridePending)
{
overridePending = false;
session.OverrideOutputAudioPort(ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp
? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _);
}
RefreshRoutes();
ResetWatchdog(); EnsureWatchdog();
}
finally { applying = false; }
}
@@ -125,7 +172,8 @@ internal sealed class IosAudioRouter
{
AVAudioSessionPortDescription? port = session.AvailableInputs?.FirstOrDefault(value => value.UID == SelectedInputId);
if (CaptureChannels == 2) port ??= session.AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic);
if (port is null) { if (CaptureChannels == 1) ClearStereoPolarPattern(session); return; }
if (port is null) { if (CaptureChannels == 1 && stereoApplied) ClearStereo(session); return; }
if (!session.SetPreferredInput(port, out NSError? inputError)) throw new InvalidOperationException(inputError?.LocalizedDescription ?? "Could not select the microphone input.");
if (CaptureChannels == 2)
{
// Polar-pattern discovery alone is insufficient on current iPhones: until a stereo
@@ -135,26 +183,25 @@ internal sealed class IosAudioRouter
// it also gives Core Audio an unambiguous left/right mapping before graph creation.
if (!session.SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait, out NSError? orientationError))
throw new InvalidOperationException(orientationError?.LocalizedDescription ?? "Could not set the stereo microphone orientation.");
stereoApplied = true;
}
AVAudioSessionDataSourceDescription? source = CaptureChannels == 2
? port.DataSources?.FirstOrDefault(SupportsStereoPolarPattern)
: port.DataSources?.FirstOrDefault(value => value.DataSourceID.ToString() == SelectedDataSourceId);
if (CaptureChannels == 1 && source is null) ClearStereoPolarPattern(session);
if (CaptureChannels == 1 && source is null && stereoApplied) ClearStereo(session);
if (source is not null)
{
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError.LocalizedDescription);
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError?.LocalizedDescription ?? "Could not select the microphone data source.");
if (CaptureChannels == 2) SetStereoPolarPattern(source);
else if (SelectedPolarPattern != AVAudioDataSourcePolarPattern.Unknown &&
!source.SetPreferredPolarPattern(SelectedPolarPattern, out NSError? patternError))
throw new InvalidOperationException(patternError.LocalizedDescription);
throw new InvalidOperationException(patternError?.LocalizedDescription ?? "Could not select the microphone polar pattern.");
}
if (!session.SetPreferredInput(port, out NSError? inputError)) throw new InvalidOperationException(inputError.LocalizedDescription);
// The working Swift client deliberately does not call
// SetPreferredInputNumberOfChannels: doing so disrupts stereo + A2DP routing. The stereo
// capsule and polar pattern above cause the input node to expose its two-channel format.
if (CaptureChannels == 2 && source is not null &&
!session.SetInputDataSource(source, out NSError? sourceError))
throw new InvalidOperationException(sourceError.LocalizedDescription);
// A port preference is enough. SetInputDataSource only accepts a member of the *current*
// port's InputDataSources, which can still be a different route after SetPreferredInput.
Console.Error.WriteLine($"VC_ROUTE_SELECT port={port.PortName} source={source?.DataSourceName ?? "none"} " +
$"stereoPattern={source is not null && SupportsStereoPolarPattern(source)} " +
$"patterns={string.Join(',', source?.SupportedPolarPatterns?.Select(value => value.ToString()) ?? [])}");
@@ -190,6 +237,8 @@ internal sealed class IosAudioRouter
internal static extern byte SetObject(NativeHandle receiver, NativeHandle selector, NativeHandle value, ref NativeHandle error);
}
private void ClearStereo(AVAudioSession session) { stereoApplied = false; ClearStereoPolarPattern(session); }
private static void ClearStereoPolarPattern(AVAudioSession session)
{
session.SetPreferredInputOrientation(AVAudioStereoOrientation.None, out _);
@@ -204,30 +253,97 @@ internal sealed class IosAudioRouter
{
defaults.SetString(Preset.ToString(), "cat.voice.audio.preset"); defaults.SetString(BluetoothMode.ToString(), "cat.voice.audio.bluetoothMode");
defaults.SetString(MicMode.ToString(), "cat.voice.audio.micMode"); defaults.SetBool(ForceSpeaker, "cat.voice.audio.forceSpeaker");
defaults.SetBool(speakerIsExplicit, "cat.voice.audio.speakerIsExplicit");
defaults.SetBool(VoiceProcessing, "cat.voice.audio.voiceProcessing"); defaults.SetBool(AutomaticGainControl, "cat.voice.audio.agc"); defaults.SetInt(CaptureChannels, "cat.voice.audio.captureChannels");
Set("cat.voice.audio.inputPortId", SelectedInputId); Set("cat.voice.audio.dataSourceId", SelectedDataSourceId); defaults.SetString(SelectedPolarPattern.ToString(), "cat.voice.audio.polarPattern"); defaults.Synchronize();
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(); Changed?.Invoke();
ResetWatchdogAttempts();
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(true, "settings"); Changed?.Invoke();
}
private void Set(string key, string? value) { if (value is null) defaults.RemoveObject(key); else defaults.SetString(value, key); }
internal void Deactivate() => AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out _);
internal void Deactivate()
{
// A released session has no route the next graph can be compared against.
watchdog?.Dispose(); watchdog = null; ResetWatchdog(); ResetWatchdogAttempts();
if (!AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? error))
Console.Error.WriteLine($"VC_DEACTIVATE failed: {error?.LocalizedDescription ?? "no system error"}");
else Console.Error.WriteLine("VC_DEACTIVATE succeeded");
}
internal void EnsureAudio(string reason)
{
if (!IosAudioEngine.Shared.IsConnected) return;
if (!IosAudioEngine.Shared.IsConnected || interrupted) return;
try { IosAudioEngine.Shared.EnsureRunning(); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
}
internal void Recover(string reason)
internal void Recover(string reason, bool force = true)
{
RefreshRoutes();
if (!IosAudioEngine.Shared.IsConnected) return;
if (!IosAudioEngine.Shared.IsConnected || interrupted) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
try { IosAudioEngine.Shared.Reconfigure(); }
try { IosAudioEngine.Shared.Reconfigure(force, reason); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
});
}
// The render callback is the only clock for the device-clocked pipeline, so a graph that
// stops while the app is backgrounded freezes capture, mix and send with no notification to
// recover from. Poll for that and rebuild; a failed rebuild is retried on the next tick.
private void EnsureWatchdog()
{
watchdog ??= new System.Threading.Timer(_ => UIApplication.SharedApplication.BeginInvokeOnMainThread(TickWatchdog),
null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1));
}
private void ResetWatchdog() { lastRenderCallbacks = -1; watchdogMisses = 0; }
private void ResetWatchdogAttempts() { watchdogAttempts = 0; nextWatchdogAttempt = 0; }
// An interruption that ends while the app is suspended never delivers its Ended notification,
// so foregrounding still has to check. It must not rebuild unconditionally though: the `audio`
// background mode keeps the session and graph live across a backgrounding, so the graph is
// almost always healthy here and a rebuild costs a visible glitch plus, on Bluetooth, an HFP
// renegotiation. Ensure it is running, and only reconfigure when it actually stopped.
internal void ResumeForeground()
{
interrupted = false; ResetWatchdog(); RefreshRoutes();
if (IosAudioEngine.Shared.IsConnected && !IosAudioEngine.Shared.IsRunning) Recover("foreground");
else EnsureAudio("foreground");
}
private void TickWatchdog()
{
if (watchdogTicking) return;
watchdogTicking = true;
try
{
// A graph still starting up reports no callbacks yet and reads as not running. Judging
// it there is how the watchdog ends up rebuilding a healthy graph on every tick.
if (!IosAudioEngine.Shared.IsConnected || interrupted || applying || IosAudioEngine.Shared.Settling) { ResetWatchdog(); return; }
long callbacks = IosAudioEngine.Shared.RenderCallbacks;
bool stalled = !IosAudioEngine.Shared.IsRunning || callbacks == lastRenderCallbacks;
lastRenderCallbacks = callbacks;
if (!stalled) { watchdogMisses = 0; ResetWatchdogAttempts(); return; }
// One missed tick can be a route change already rebuilding the graph.
if (++watchdogMisses < 2) return;
if (Environment.TickCount64 < nextWatchdogAttempt) return;
if (watchdogAttempts >= MaximumWatchdogAttempts)
{
if (watchdogAttempts == MaximumWatchdogAttempts)
{
watchdogAttempts++;
Console.Error.WriteLine("VC_WATCHDOG exhausted; the audio graph will not start and is no longer being rebuilt");
}
return;
}
ResetWatchdog();
watchdogAttempts++;
nextWatchdogAttempt = Environment.TickCount64 + Math.Min(2_000 * (1 << watchdogAttempts), 30_000);
try { IosAudioEngine.Shared.Reconfigure(true, $"stall watchdog {watchdogAttempts}"); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio watchdog rebuild failed: {exception}"); }
}
finally { watchdogTicking = false; }
}
private void HandleRouteChange(NSNotification note)
{
NSNumber? value = note.UserInfo?[new NSString("AVAudioSessionRouteChangeReasonKey")] as NSNumber;
@@ -237,12 +353,29 @@ internal sealed class IosAudioRouter
or AVAudioSessionRouteChangeReason.Override
or AVAudioSessionRouteChangeReason.RouteConfigurationChange)
return;
Recover($"route change ({reason})");
// Apply is mid-flight: this notification describes the change Apply is itself making.
if (applying) return;
// Not a forced rebuild. Every reconfiguration moves the route, and moving the route
// notifies here, so answering a route change with an unconditional rebuild is a loop with
// one iteration per notification: forcing the speaker takes a headset out of the route as
// OldDeviceUnavailable, releasing it brings the headset back as NewDeviceAvailable, and
// even joining voice moves the route through SetPreferredInput. AVAudioEngine follows a
// route change on its own; what it cannot absorb is the hardware under the tap changing,
// and Reconfigure tests for that, rebuilding a stopped graph and leaving a healthy
// unchanged one alone. Comparing routes instead cannot work: CurrentRoute still names the
// previous route for a while after a reconfiguration.
Console.Error.WriteLine($"VC_ROUTE_CHANGE reason={reason} running={IosAudioEngine.Shared.IsRunning} " +
$"hardwareChanged={IosAudioEngine.Shared.HardwareChanged()}");
Recover($"route change ({reason})", force: false);
}
private void HandleInterruption(NSNotification note)
{
NSNumber? type = note.UserInfo?[new NSString("AVAudioSessionInterruptionTypeKey")] as NSNumber;
AVAudioSessionInterruptionType interruption = (AVAudioSessionInterruptionType)(type?.UInt32Value ?? 0);
if (interruption == AVAudioSessionInterruptionType.Ended) Recover("interruption ended");
// The system stops the graph on Began and SetActive fails until the interruption clears,
// so suppress recovery until Ended and let the watchdog retry if that rebuild fails.
if (interruption == AVAudioSessionInterruptionType.Began) { interrupted = true; ResetWatchdog(); return; }
if (interruption != AVAudioSessionInterruptionType.Ended) return;
interrupted = false; ResetWatchdog(); Recover("interruption ended");
}
}
@@ -0,0 +1,56 @@
using CoreFoundation;
using Network;
using VoiceCat.Core;
namespace VoiceCat.iOS;
// Watches which interface actually carries traffic so a handover can be acted on the instant it
// happens instead of after the control keepalive proves the old path dead.
//
// The signal must be narrow. Roaming between access points, a lift ride, a minute of bad
// cellular: those keep the same interface and the same source address, so TCP survives them and
// a reconnect would be pure damage. Only the set of satisfied interfaces changing — Wi-Fi to
// cellular, one physical link to another — strands the existing sockets on a source address that
// no longer exists, and that is the only thing reported here.
internal sealed class IosNetworkPathMonitor : IDisposable
{
internal static IosNetworkPathMonitor Shared { get; } = new();
private readonly DispatchQueue queue = new("voicecat.path");
private readonly object gate = new();
private readonly ControlPathWatcher watcher = new();
private NWPathMonitor? monitor;
private bool started;
// Raised on the monitor queue when the carrying interface changed and the new path is usable.
internal event Action<string>? InterfaceChanged;
internal void Start()
{
lock (gate)
{
if (started) return;
started = true;
monitor = new NWPathMonitor();
monitor.SetQueue(queue);
monitor.SnapshotHandler = OnPath;
monitor.Start();
}
}
private void OnPath(NWPath path)
{
List<string> interfaces = [];
if (path.Status == NWPathStatus.Satisfied)
path.EnumerateInterfaces(item => { interfaces.Add($"{item.InterfaceType}:{item.Name}"); return true; });
if (watcher.Observe(path.Status == NWPathStatus.Satisfied, interfaces))
InterfaceChanged?.Invoke(string.Join(",", interfaces));
}
public void Dispose()
{
lock (gate)
{
monitor?.Cancel(); monitor?.Dispose(); monitor = null; started = false; watcher.Reset();
}
}
}
+74 -14
View File
@@ -34,7 +34,13 @@ internal sealed class VoiceControlsView : UIView
private readonly AppModel model; private readonly UIButton join = UIButton.FromType(UIButtonType.System);
private readonly UIButton ptt = UIButton.FromType(UIButtonType.System); private readonly UIButton mute = UIButton.FromType(UIButtonType.System);
private readonly UIButton deafen = UIButton.FromType(UIButtonType.System); private readonly UIProgressView level = new(UIProgressViewStyle.Default);
internal VoiceControlsView(AppModel model) { this.model = model; model.Changed += Refresh; Build(); }
private int renderedLevel = -1;
internal VoiceControlsView(AppModel model) { this.model = model; model.Changed += Refresh; model.LevelChanged += RefreshLevel; Build(); }
protected override void Dispose(bool disposing)
{
if (disposing) { model.Changed -= Refresh; model.LevelChanged -= RefreshLevel; }
base.Dispose(disposing);
}
private void Build()
{
BackgroundColor = UIColor.SecondarySystemBackground; join.TouchUpInside += async (_, _) => await Run(model.ToggleVoiceAsync);
@@ -54,22 +60,50 @@ internal sealed class VoiceControlsView : UIView
bool muted = model.Client?.Audio.MicMuted == true, deafened = model.Client?.Audio.Deafened == true;
mute.SetImage(UIImage.GetSystemImage(muted ? "mic.slash.fill" : "mic.fill"), UIControlState.Normal); mute.AccessibilityLabel = muted ? "Unmute microphone" : "Mute microphone"; mute.Enabled = model.VoiceJoined;
deafen.SetImage(UIImage.GetSystemImage(deafened ? "headphones.slash" : "headphones"), UIControlState.Normal); deafen.AccessibilityLabel = deafened ? "Undeafen" : "Deafen"; deafen.Enabled = model.VoiceJoined;
level.Progress = Math.Clamp(model.MicrophoneLevel * 10, 0, 1); level.AccessibilityLabel = "Microphone level"; level.AccessibilityValue = $"{level.Progress:P0}";
level.AccessibilityLabel = "Microphone level"; RefreshLevel();
}
// Driven at 20 Hz. Rewriting AccessibilityValue on every tick makes VoiceOver re-announce the
// meter continuously while it is focused, so only publish a value that actually changed.
private void RefreshLevel()
{
float progress = Math.Clamp(model.MicrophoneLevel * 10, 0, 1); level.Progress = progress;
int percent = (int)MathF.Round(progress * 100 / 5) * 5;
if (percent == renderedLevel) return;
renderedLevel = percent; level.AccessibilityValue = $"{percent}%";
}
private async Task Run(Func<Task> operation) { try { await operation(); } catch (Exception exception) { if (Window?.RootViewController is { } owner) UiHelpers.ShowError(owner, exception); } }
}
internal sealed class ChannelsController : UITableViewController
{
private readonly AppModel model; private IReadOnlyList<(Channel Channel, int Depth)> Visible => Flatten();
internal ChannelsController(AppModel model) { this.model = model; Title = "Channels"; model.Changed += Reload; }
private readonly AppModel model; private readonly ListRefresher refresher; private IReadOnlyList<(Channel Channel, int Depth)> Visible => Flatten();
internal ChannelsController(AppModel model)
{
this.model = model; Title = "Channels";
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
}
public override void ViewDidLoad()
{
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel");
NavigationItem.RightBarButtonItems = [new("Users", UIBarButtonItemStyle.Plain, (_, _) => NavigationController?.PushViewController(new UsersController(model), true)),
new(UIBarButtonSystemItem.Add, (_, _) => NavigationController?.PushViewController(new ChannelEditorController(model, null), true))]; Reload();
new(UIBarButtonSystemItem.Add, (_, _) => NavigationController?.PushViewController(new ChannelEditorController(model, null), true))];
}
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); UpdateActions(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature()
{
UpdateActions();
return string.Join('\n', Visible.Select(entry =>
$"{entry.Channel.Id}|{entry.Depth}|{entry.Channel.Name}|{entry.Channel.Topic}|{entry.Channel.PasswordProtected}|" +
$"{entry.Channel.Id == model.CurrentChannelId}|{model.Users.Count(user => user.ChannelId == entry.Channel.Id)}"));
}
// The add button is not part of the table, so it can follow every notification cheaply.
private void UpdateActions()
{
if (NavigationItem.RightBarButtonItems is not { Length: > 1 } items) return;
items[1].Enabled = model.Client?.Permissions is { } p && (p.IsAdmin || p.CanCreateTempChannel);
}
private void Reload() { TableView.ReloadData(); NavigationItem.RightBarButtonItems![1].Enabled = model.Client?.Permissions is { } p && (p.IsAdmin || p.CanCreateTempChannel); }
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
@@ -112,7 +146,9 @@ internal sealed class ChannelsController : UITableViewController
internal sealed class ChatController : UIViewController
{
private readonly AppModel model; private readonly UITextView log = new(); private readonly UITextField compose = UiHelpers.Field("Message");
internal ChatController(AppModel model) { this.model = model; Title = "Chat"; model.Changed += Refresh; }
internal ChatController(AppModel model) { this.model = model; Title = "Chat"; }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); model.Changed += Refresh; Refresh(); }
public override void ViewDidDisappear(bool animated) { model.Changed -= Refresh; base.ViewDidDisappear(animated); }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; log.Editable = false; log.Font = UIFont.PreferredBody; log.AccessibilityLabel = "Chat and activity timeline"; log.TranslatesAutoresizingMaskIntoConstraints = false;
@@ -125,7 +161,11 @@ internal sealed class ChatController : UIViewController
{
IEnumerable<(DateTime Time, string Text)> chat = model.Messages.Where(message => !message.Private).Select(message => (message.Timestamp, $"{message.Sender}: {message.Text}"));
IEnumerable<(DateTime Time, string Text)> activity = model.Activity.Select(value => (value.Timestamp, $"• {value.Text}"));
log.Text = string.Join("\n", chat.Concat(activity).OrderBy(value => value.Time).Select(value => $"[{value.Time:t}] {value.Text}")); if (log.Text.Length > 0) log.ScrollRangeToVisible(new(log.Text.Length - 1, 1));
string text = string.Join("\n", chat.Concat(activity).OrderBy(value => value.Time).Select(value => $"[{value.Time:t}] {value.Text}"));
// Reassigning identical text still resets the text view's accessibility state, which
// interrupts VoiceOver mid-read of the transcript.
if (text == log.Text) return;
log.Text = text; if (text.Length > 0) log.ScrollRangeToVisible(new(text.Length - 1, 1));
}
}
@@ -145,8 +185,24 @@ internal sealed class PrivateChatsController : UITableViewController
}).OrderByDescending(peer => peer.Latest?.Timestamp ?? DateTime.MinValue).ThenBy(peer => peer.Name).ToArray();
}
}
internal PrivateChatsController(AppModel model) { this.model = model; Title = "Private Chats"; model.Changed += Reload; }
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "private-peer"); Reload(); }
private readonly ListRefresher refresher;
internal PrivateChatsController(AppModel model)
{
this.model = model; Title = "Private Chats";
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
}
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "private-peer"); }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature()
{
IReadOnlyList<(uint Id, string Name, ChatEntry? Latest, bool Online)> peers = Peers;
// The empty-state view is not a table row, so it is kept current outside the reload.
bool empty = peers.Count == 0, shown = TableView.BackgroundView is not null;
if (empty != shown)
TableView.BackgroundView = empty ? new UILabel { Text = "No other users or private conversations", TextAlignment = UITextAlignment.Center, AccessibilityLabel = "No other users or private conversations" } : null;
return string.Join('\n', peers.Select(peer => $"{peer.Id}|{peer.Name}|{peer.Online}|{peer.Latest?.Text}"));
}
public override nint RowsInSection(UITableView tableView, nint section) => Peers.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
{
@@ -158,7 +214,6 @@ internal sealed class PrivateChatsController : UITableViewController
{
(uint id, string name, _, _) = Peers[indexPath.Row]; tableView.DeselectRow(indexPath, true); NavigationController?.PushViewController(new PrivateConversationController(model, id, name), true);
}
private void Reload() { if (IsViewLoaded) { TableView.ReloadData(); TableView.BackgroundView = Peers.Count == 0 ? new UILabel { Text = "No other users or private conversations", TextAlignment = UITextAlignment.Center, AccessibilityLabel = "No other users or private conversations" } : null; } }
}
internal sealed class PrivateConversationController : UIViewController
@@ -167,7 +222,9 @@ internal sealed class PrivateConversationController : UIViewController
private readonly UITextView transcript = new(); private readonly UITextField compose = UiHelpers.Field("Private message"); private readonly UIButton send = UIButton.FromType(UIButtonType.System);
private User? Peer => model.Users.FirstOrDefault(user => user.Id == peerUserId);
internal PrivateConversationController(AppModel model, uint peerUserId, string fallbackName)
{ this.model = model; this.peerUserId = peerUserId; this.fallbackName = fallbackName; Title = fallbackName; model.Changed += Refresh; }
{ this.model = model; this.peerUserId = peerUserId; this.fallbackName = fallbackName; Title = fallbackName; }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); model.Changed += Refresh; Refresh(); }
public override void ViewDidDisappear(bool animated) { model.Changed -= Refresh; base.ViewDidDisappear(animated); }
public override void ViewDidLoad()
{
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; transcript.Editable = false; transcript.Font = UIFont.PreferredBody; transcript.AccessibilityLabel = $"Private conversation with {fallbackName}"; transcript.TranslatesAutoresizingMaskIntoConstraints = false;
@@ -182,8 +239,11 @@ internal sealed class PrivateConversationController : UIViewController
{
if (!IsViewLoaded) return; string name = Peer?.Nickname ?? fallbackName; Title = name;
IEnumerable<ChatEntry> messages = model.Messages.Where(message => message.Private && message.PeerUserId == peerUserId).OrderBy(message => message.Timestamp);
transcript.Text = string.Join("\n", messages.Select(message => $"[{message.Timestamp:t}] {(message.SenderId == model.SelfUserId ? "You" : message.Sender)}: {message.Text}"));
if (transcript.Text.Length > 0) transcript.ScrollRangeToVisible(new(transcript.Text.Length - 1, 1));
string text = string.Join("\n", messages.Select(message => $"[{message.Timestamp:t}] {(message.SenderId == model.SelfUserId ? "You" : message.Sender)}: {message.Text}"));
if (text != transcript.Text)
{
transcript.Text = text; if (text.Length > 0) transcript.ScrollRangeToVisible(new(text.Length - 1, 1));
}
bool online = Peer is not null; compose.Enabled = online; send.Enabled = online; NavigationItem.RightBarButtonItem!.Enabled = online;
}
}
@@ -5,12 +5,24 @@ namespace VoiceCat.iOS;
internal sealed class ServerListController : UITableViewController
{
private readonly AppModel model;
internal ServerListController(AppModel model) { this.model = model; Title = "Servers"; TabBarItem = new(UITabBarSystemItem.Favorites, 0); }
private readonly AppModel model; private readonly ListRefresher refresher;
internal ServerListController(AppModel model)
{
this.model = model; Title = "Servers"; TabBarItem = new(UITabBarSystemItem.Favorites, 0);
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
}
public override void ViewDidLoad()
{
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "server");
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PresentEditor(null)); model.Changed += Reload;
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PresentEditor(null));
}
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature()
{
// The connection prompt is navigation-bar chrome, not a row, so it follows every change.
NavigationItem.Prompt = model.IsConnecting ? model.Status : null;
return string.Join('\n', model.Profiles.Select(profile => $"{profile.Id}|{profile.DisplayName}|{profile.Authentication}"));
}
public override nint RowsInSection(UITableView tableView, nint section) => model.Profiles.Count;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
@@ -32,7 +44,6 @@ internal sealed class ServerListController : UITableViewController
return UISwipeActionsConfiguration.FromActions([delete, edit]);
}
private void PresentEditor(ServerProfile? profile) => PresentViewController(new UINavigationController(new ServerEditorController(model, profile)), true, null);
private void Reload() { TableView.ReloadData(); NavigationItem.Prompt = model.IsConnecting ? model.Status : null; }
}
internal sealed class ServerEditorController : UIViewController
@@ -9,10 +9,22 @@ namespace VoiceCat.iOS;
internal sealed class SettingsController : UITableViewController
{
private readonly AppModel model;
internal SettingsController(AppModel model) : base(UITableViewStyle.InsetGrouped) { this.model = model; Title = "Settings"; model.Changed += Reload; IosAudioRouter.Shared.Changed += Reload; }
private readonly AppModel model; private readonly ListRefresher refresher;
internal SettingsController(AppModel model) : base(UITableViewStyle.InsetGrouped)
{
this.model = model; Title = "Settings";
refresher = new(this, Signature,
handler => { model.Changed += handler; IosAudioRouter.Shared.Changed += handler; },
handler => { model.Changed -= handler; IosAudioRouter.Shared.Changed -= handler; });
}
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "setting"); }
private void Reload() => TableView.ReloadData();
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature() =>
$"{IosAudioRouter.Shared.Preset}|{IosAudioRouter.Shared.ForceSpeaker}|{model.Settings.AudioBufferMilliseconds}|{model.ScreenSharing}|" +
$"{model.Settings.InputMode}|{model.Settings.VadThreshold}|{model.Settings.InputGain}|{model.Settings.InputNoiseReduction}|" +
$"{model.Settings.EventSounds}|{model.Settings.EventVolume}|{model.Settings.SpokenEvents}|{model.Settings.SelfTalkSounds}|{model.Settings.PushToTalkSound}|" +
$"{model.Client?.Permissions is { } p && (p.IsAdmin || p.CanAdminAccounts)}";
public override nint NumberOfSections(UITableView tableView) => 5;
public override nint RowsInSection(UITableView tableView, nint section) => section switch { 0 => 5, 1 => 4, 2 => 5, 3 => model.Client?.Permissions is { } p && (p.IsAdmin || p.CanAdminAccounts) ? 1 : 0, _ => 2 };
public override string? TitleForHeader(UITableView tableView, nint section) => section switch { 0 => "Audio", 1 => "Voice", 2 => "Notifications", 3 => "Administration", _ => "Server" };
@@ -69,9 +81,19 @@ internal sealed class SettingsController : UITableViewController
internal sealed class AdvancedAudioController : UITableViewController
{
private readonly IosAudioRouter router = IosAudioRouter.Shared;
internal AdvancedAudioController() : base(UITableViewStyle.InsetGrouped) { Title = "Advanced Audio"; router.Changed += () => TableView.ReloadData(); }
private readonly IosAudioRouter router = IosAudioRouter.Shared; private readonly ListRefresher refresher;
internal AdvancedAudioController() : base(UITableViewStyle.InsetGrouped)
{
Title = "Advanced Audio";
refresher = new(this, Signature, handler => IosAudioRouter.Shared.Changed += handler, handler => IosAudioRouter.Shared.Changed -= handler);
}
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "audio"); router.RefreshRoutes(); }
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
private string Signature() =>
$"{router.SelectedInputId}|{router.SelectedDataSourceId}|{router.SelectedPolarPattern}|{router.MicMode}|{router.CaptureChannels}|" +
$"{router.BluetoothMode}|{router.UsesVoiceProcessing}|{router.AutomaticGainControl}|" +
$"{string.Join(',', router.Inputs.Select(value => value.Name))}|{string.Join(',', router.Outputs.Select(value => value.Name))}";
public override nint RowsInSection(UITableView tableView, nint section) => 8;
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath path)
{
@@ -0,0 +1,181 @@
#if DEBUG
using System.Buffers.Binary;
using System.Runtime.InteropServices;
using Foundation;
using UIKit;
using VoiceCat.Core;
namespace VoiceCat.iOS;
// An opt-in simulator gate for the real UIKit model and audio graph. Local development only.
internal static class SimulatorSmoke
{
internal static void RunIfRequested()
{
string? endpoint = Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE");
if (endpoint is null || !RuntimeInformation.RuntimeIdentifier.StartsWith("iossimulator", StringComparison.Ordinal)) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(async () =>
{
try
{
string[] parts = endpoint.Split(':', 3);
Guid id = Guid.Parse("959da92f-f088-452d-a870-4c15007a6fd3");
ServerProfile guest = ServerProfile.Create(parts[0], ushort.Parse(parts[1]), ServerAuthentication.Guest, nickname: "smoke", id: id);
ServerProfile profile = ServerProfile.Create(parts[0], ushort.Parse(parts[1]), ServerAuthentication.Account, "smoke", id: id);
AppModel model = AppModel.Shared;
bool share = Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") == "share";
string? ring = share ? PrepareRing() : null;
model.IdentityRequested += _ => model.ResolveIdentity(true);
if (Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") != "resume")
{
model.UpsertProfile(guest, null);
model.UpsertProfile(profile, parts[2]);
}
model.Load();
if (Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") == "stereo")
IosAudioRouter.Shared.SelectPreset(IosAudioPreset.StereoMicrophone);
profile = model.Profiles.Single(value => value.Id == id);
Console.Error.WriteLine("VC_SIM phase=connect");
await model.ConnectAsync(profile);
Console.Error.WriteLine($"VC_SIM phase=connected audioRunning={IosAudioEngine.Shared.IsRunning}");
Require(model.IsConnected, "Account login did not connect.");
if (ring is not null)
{
// Start a synthetic producer while the microphone is off. This exercises
// screen-only subscription and the real BroadcastChanged audio recovery.
await Task.Delay(300);
Require(!model.ScreenRingMapped && !model.ScreenSharing,
"An unchanged ring from a previous broadcast remained mapped.");
for (int frame = 0; frame < 6; frame++) WriteScreenPcm(ring);
await WaitUntil(() => model.ScreenSharing, "Screen stream did not start.");
Require(IosAudioEngine.Shared.IsRunning, "Screen sharing stopped the audio graph.");
await WaitUntil(() => model.ScreenEncodedPackets > 0,
"Screen PCM captured before stream negotiation was discarded.");
long encodedBefore = model.ScreenEncodedPackets;
for (int frame = 0; frame < 12; frame++)
{
WriteScreenPcm(ring);
await Task.Delay(20);
}
await WaitUntil(() => model.ScreenEncodedPackets > encodedBefore,
"Screen PCM did not reach the encoder while the microphone was off.");
WriteRing(ring, active: false);
await WaitUntil(() => !model.ScreenSharing, "Screen stream did not stop.");
Require(!model.ScreenRingMapped, "The stopped screen ring remained mapped.");
Console.Error.WriteLine("VC_SIM phase=screenSharePassed");
}
await model.ToggleVoiceAsync();
Console.Error.WriteLine($"VC_SIM phase=joined audioRunning={IosAudioEngine.Shared.IsRunning}");
await Task.Delay(2500);
Require(model.VoiceJoined && IosAudioEngine.Shared.IsRunning, "The first voice graph stopped.");
long beforeShare = IosAudioEngine.Shared.RenderCallbacks;
IosAudioRouter.Shared.Recover("simulated screen sharing start", force: false);
await Task.Delay(1000);
Require(IosAudioEngine.Shared.IsRunning && IosAudioEngine.Shared.RenderCallbacks > beforeShare,
"Screen sharing recovery stopped the live audio graph.");
if (ring is not null)
{
WriteRing(ring, active: true);
await Task.Delay(300);
WriteScreenPcm(ring);
await WaitUntil(() => model.ScreenSharing, "Screen stream did not start during a voice call.");
Require(IosAudioEngine.Shared.IsRunning, "Starting screen sharing stopped the voice graph.");
}
await model.ToggleVoiceAsync();
if (ring is not null)
{
Require(model.ScreenSharing, "Leaving the microphone call unsubscribed active screen audio.");
WriteRing(ring, active: false);
await WaitUntil(() => !model.ScreenSharing, "The second screen stream did not stop.");
}
await model.DisconnectAsync();
await Task.Delay(3000);
model.Load();
profile = model.Profiles.Single(value => value.Id == id);
Console.Error.WriteLine("VC_SIM phase=reconnect");
await model.ConnectAsync(profile);
Console.Error.WriteLine($"VC_SIM phase=reconnected audioRunning={IosAudioEngine.Shared.IsRunning}");
Require(model.IsConnected, "Saved account login did not reconnect.");
await Task.Delay(2500);
Console.Error.WriteLine($"VC_SIM phase=reconnectedSettled audioRunning={IosAudioEngine.Shared.IsRunning}");
Require(IosAudioEngine.Shared.IsRunning, "The reconnect audio graph stopped.");
await model.ToggleVoiceAsync();
Console.Error.WriteLine($"VC_SIM phase=rejoined audioRunning={IosAudioEngine.Shared.IsRunning}");
await Task.Delay(2500);
Console.Error.WriteLine($"VC_SIM phase=rejoinedSettled audioRunning={IosAudioEngine.Shared.IsRunning}");
Require(model.VoiceJoined && IosAudioEngine.Shared.IsRunning, "The rejoined voice graph stopped.");
await model.DisconnectAsync();
Console.Error.WriteLine("VC_SIM phase=passed");
}
catch (Exception exception) { Console.Error.WriteLine($"VC_SIM phase=failed exception={exception}"); }
});
}
private static void Require(bool condition, string message)
{
if (!condition) throw new InvalidOperationException(message);
}
private static string PrepareRing()
{
string root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup)?.Path
?? throw new InvalidOperationException("Simulator App Group is unavailable.");
string path = Path.Combine(root, "voicecat", "broadcast_audio.ring");
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
using var file = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.ReadWrite);
file.SetLength(64 + 96_000 * sizeof(short));
byte[] header = new byte[20];
BinaryPrimitives.WriteUInt32LittleEndian(header, 0x56434252);
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(4), 1);
file.Write(header);
file.Dispose();
WriteRing(path, active: true, write: 1920);
return path;
}
private static void WriteRing(string path, bool active, ulong? write = null)
{
using var file = new FileStream(path, FileMode.Open, FileAccess.Write, FileShare.ReadWrite);
byte[] value = new byte[8];
if (write is { } position)
{
BinaryPrimitives.WriteUInt64LittleEndian(value, position);
file.Position = 24; file.Write(value);
}
BinaryPrimitives.WriteUInt32LittleEndian(value, active ? 1u : 0u);
file.Position = 16; file.Write(value, 0, 4);
}
private static void WriteScreenPcm(string path)
{
using var file = new FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite);
Span<byte> index = stackalloc byte[8];
file.Position = 24; file.ReadExactly(index);
ulong write = BinaryPrimitives.ReadUInt64LittleEndian(index);
byte[] pcm = new byte[960 * 2 * sizeof(short)];
for (int sample = 0; sample < pcm.Length / sizeof(short); sample++)
BinaryPrimitives.WriteInt16LittleEndian(pcm.AsSpan(sample * sizeof(short)),
(short)(sample % 64 < 32 ? 6000 : -6000));
int start = (int)(write % 96_000), first = Math.Min(pcm.Length / sizeof(short), 96_000 - start);
file.Position = 64 + start * sizeof(short);
file.Write(pcm.AsSpan(0, first * sizeof(short)));
if (first < pcm.Length / sizeof(short))
{
file.Position = 64;
file.Write(pcm.AsSpan(first * sizeof(short)));
}
BinaryPrimitives.WriteUInt64LittleEndian(index, write + (ulong)(pcm.Length / sizeof(short)));
file.Position = 24; file.Write(index);
}
private static async Task WaitUntil(Func<bool> condition, string error)
{
for (int attempt = 0; attempt < 30; attempt++)
{
if (condition()) return;
await Task.Delay(100);
}
throw new InvalidOperationException(error);
}
}
#endif
+38
View File
@@ -18,3 +18,41 @@ internal static class UiHelpers
field.AccessibilityLabel = placeholder; return field;
}
}
// VoiceOver rebuilds its element tree from scratch on every UITableView.ReloadData. The model
// notifies far more often than any list's content actually changes - the microphone level timer
// alone ticks at 20 Hz - so reloading unconditionally re-announces the row under an exploring
// finger and destroys the element a double tap was aimed at. Reload only when the rendered text
// actually differs, and only while the view is on screen, so an off-screen or popped controller
// stops reloading instead of holding a subscription for the lifetime of the connection.
internal sealed class ListRefresher(UITableViewController owner, Func<string> signature, Action<Action> subscribe, Action<Action> unsubscribe)
{
// Null means nothing has been rendered yet. A signature is never null, so the first
// Refresh after Start or Invalidate always reloads.
private string? rendered;
private bool observing;
internal void Start()
{
if (observing) return;
observing = true; subscribe(Refresh); Refresh();
}
internal void Stop()
{
if (!observing) return;
observing = false; unsubscribe(Refresh);
}
// Forces the next Refresh to reload even when the signature is unchanged, for state the
// signature cannot see.
internal void Invalidate() => rendered = null;
internal void Refresh()
{
if (!observing || !owner.IsViewLoaded) return;
string next = signature();
if (next == rendered) return;
rendered = next; owner.TableView.ReloadData();
}
}
+6 -5
View File
@@ -164,8 +164,9 @@ Before packaging, confirm:
- the bundle identifiers are the stable identifiers above;
- both version pairs match;
- both executables contain `arm64`; and
- the host `Info.plist` contains `CFBundleIconName` and the bundle contains both
`AppIcon60x60@2x.png` (120x120) and `AppIcon76x76@2x~ipad.png` (152x152).
- the host `Info.plist` contains `CFBundleIconName` nested inside both `CFBundleIcons` and
`CFBundleIcons~ipad`, which is where `actool` writes it and not at the top level, and the
bundle contains both `AppIcon60x60@2x.png` (120x120) and `AppIcon76x76@2x~ipad.png` (152x152).
Also decode each embedded profile and verify its name, UUID, application identifier, and
`get-task-allow` value:
@@ -270,9 +271,9 @@ clean build regenerated the app manifests.
## Last verified release build
On 2026-09-22, version `0.0.1`, build `2026092203` was built with .NET 10.0.401 and Xcode 27.0.
On 2026-09-27, version `0.0.1`, build `2026092701` was built with .NET 10.0.401 and Xcode 27.0.
The host and ReplayKit extension passed strict nested-signature validation with App Store Connect
profiles, matching distribution identities, matching versions, the shared App Group, and
`get-task-allow=false`. The host carries `CFBundleIconName` with the 120x120 and 152x152 icons.
The resulting IPA was packaged locally; Apple server-side upload validation remains a separate
gate.
The IPA at `dist/VoiceCat-0.0.1-2026092701.ipa` passed archive integrity checks; Apple server-side
upload validation remains a separate gate.
+6
View File
@@ -68,6 +68,12 @@ need direct native entry points. These are platform adapters, not a second core.
- `proto/voicecat.proto` is the control-plane schema.
- Media uses the fixed header and AEAD construction described in `protocol.md` and
`security.md`.
- Media frame types are a versioned contract: `Voice`, `Keepalive`, `UdpBinding`, and `Rebind`.
`UdpBinding` establishes a peer's endpoint once, in the clear. `Rebind` moves an established
endpoint after the client's source address changes, as on a Wi-Fi/cellular handover; it
carries the binding token in the clear for peer lookup only, and authorization comes from the
AEAD tag over header and token plus the peer's replay window, so a captured rebind cannot be
replayed to redirect someone else's downlink.
- SQLite is the server's persistent store; schema changes require explicit migrations.
- Client profiles and TOFU pins are local platform data.
- The ReplayKit ring layout is separately versioned and frozen.
+39
View File
@@ -133,6 +133,45 @@ CoreDevice can also install the app successfully and then reject only the launch
deployment; rebuilding is unnecessary. Occasional CoreDeviceService initialization timeouts while
listing devices or processes do not imply that an already-confirmed install or launch failed.
## Simulator connection and audio gate
The Debug app has an opt-in simulator smoke gate. Start a local VoiceCat server and create an
account named `smoke`, then build the `iossimulator-arm64` app. The simulator build needs an
App Group entitlement for the saved-password and screen-ring checks; sign the built app
ad hoc with `clients/apple/VoiceCat.iOS/Entitlements.plist` before installing it. A clean
build avoids stale simulator AOT modules after managed code changes.
```bash
dotnet restore clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj \
-p:RuntimeIdentifier=iossimulator-arm64 -p:VoiceCatIosStatic=true --force-evaluate
dotnet clean clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj -c Debug \
-p:RuntimeIdentifier=iossimulator-arm64
dotnet build clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj -c Debug --no-restore \
-p:RuntimeIdentifier=iossimulator-arm64
codesign --force --sign - --entitlements clients/apple/VoiceCat.iOS/Entitlements.plist \
clients/apple/VoiceCat.iOS/bin/Debug/net10.0-ios27.0/iossimulator-arm64/VoiceCat.iOS.app
xcrun simctl install booted \
clients/apple/VoiceCat.iOS/bin/Debug/net10.0-ios27.0/iossimulator-arm64/VoiceCat.iOS.app
```
Grant microphone access with `xcrun simctl privacy booted grant microphone
me.iamtalon.voicecat`. Launch with `SIMCTL_CHILD_VOICECAT_SIM_SMOKE` set to
`127.0.0.1:<port>:<smoke-account-password>` and
`SIMCTL_CHILD_VOICECAT_SIM_SMOKE_MODE=share` using `xcrun simctl launch --console-pty booted
me.iamtalon.voicecat`. The `share` mode checks that a stale ring stays unmapped, then verifies
that screen PCM captured before stream negotiation reaches the encoder. It also exercises
continuous synthetic screen audio with the microphone off and during a voice call, voice join,
disconnect, and reconnect. Look for
`VC_SIM phase=screenSharePassed` and `VC_SIM phase=passed`. Terminate the app, then launch
again with mode `resume` to verify the saved account after a process restart. The simulator
gate also accepts mode `stereo` to select the StereoMicrophone preset before login and voice join;
the simulator exposes only a mono microphone, so check stereo capture on an iPhone. It also
does not validate Bluetooth hardware, the iOS 27 ScreenCaptureKit picker, or suspension on
a physical device.
The simulator restore can rewrite `packages.ios.lock.json`; retain both checked-in iOS
runtime entries when reviewing the diff.
## Verified hardware result
On 2026-09-22, the Debug build completed with .NET 10.0.401 and Xcode 27.0. The host and
+8 -5
View File
@@ -177,6 +177,13 @@ public sealed class AudioEngine : IDisposable
catch (Exception exception) { Failure = exception; stop.Cancel(); }
}
// Drop each capture backlog to its buffer target. The consumer calls this after a stall it
// cannot catch up in place: one bounded gap instead of a queue that ratchets toward its edge.
public void ResynchronizeInputs()
{
foreach (LocalStream stream in Volatile.Read(ref routes).Local) stream.Input.Resynchronize();
}
private void Work()
{
long deadline = Stopwatch.GetTimestamp();
@@ -189,12 +196,8 @@ public sealed class AudioEngine : IDisposable
WaitUntil(deadline);
if ((deadline - Stopwatch.GetTimestamp()) * 1000.0 / Stopwatch.Frequency < -100)
{
// A stall this long cannot be caught up in place. Discarding only the schedule
// deficit would leave the producer backlog queued at a fixed offset forever and
// eventually overflow its ring, so drop back to the buffer targets instead: one
// bounded gap per stall rather than growing latency and feed drops.
deadline = Stopwatch.GetTimestamp();
foreach (LocalStream stream in Volatile.Read(ref routes).Local) stream.Input.Resynchronize();
ResynchronizeInputs();
}
}
}
+50 -7
View File
@@ -25,7 +25,11 @@ internal sealed class ReceiveStream : IDisposable
private readonly byte[][] jitter;
private readonly uint[] timestamps;
private readonly int[] sizes;
private int count, available, offset, missing, waiting;
private int count, available, offset, missing, waiting, stretchCooldown;
// One held frame per half second. Depth still follows a degrading link within a few seconds,
// but a target pinned at its cap can no longer trade a steady stream of concealment against
// depth it will never reach, nor oscillate against CatchUp's trim.
private const int StretchCooldownFrames = 25;
private uint expected;
private bool started, hasTimestamp;
private bool hasMarker;
@@ -40,6 +44,8 @@ internal sealed class ReceiveStream : IDisposable
internal int ConcealedFrames { get; private set; }
internal int DredFrames { get; private set; }
internal int FecFrames { get; private set; }
internal int Overruns { get; private set; }
internal int Stretches { get; private set; }
internal ReceiveStream(uint userId, StreamInfo info, TimeProvider? clock = null)
{
@@ -60,14 +66,22 @@ internal sealed class ReceiveStream : IDisposable
internal bool Enqueue(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
{
int index = written;
if (payload.Length is < 1 or > 1275 || unchecked(index - Volatile.Read(ref read)) >= 64) return false;
if (payload.Length is < 1 or > 1275) return false;
// A stalled consumer (an interrupted or rebuilding iOS graph) stops draining this
// handoff. Refusing new packets would hold a ring of audio that is already too old to
// play and discard the live talkspurt instead, so the newest always wins and the
// consumer — which alone owns `read` — notices the overrun and skips forward.
int slot = index & 63; payload.CopyTo(packets[slot]); headers[slot] = header; lengths[slot] = payload.Length; arrivals[slot] = clock.GetTimestamp();
Volatile.Write(ref written, unchecked(index + 1)); return true;
}
private void Drain()
{
while (read != Volatile.Read(ref written))
int end = Volatile.Read(ref written);
// Leave a margin below the producer rather than resuming exactly 64 back, so a write
// during this drain cannot lap the slot being copied.
if (unchecked(end - read) > 64) { read = unchecked(end - 32); Overruns++; }
while (read != end)
{
int source = read & 63; uint timestamp = headers[source].Timestamp;
if (!hasTimestamp) { hasTimestamp = true; expected = timestamp; }
@@ -78,8 +92,16 @@ internal sealed class ReceiveStream : IDisposable
DropBefore(timestamp); available = offset = missing = waiting = 0;
expected = timestamp; started = false; delta = 0; lastArrival = 0; jitterSamples = 0;
}
// Observed before the acceptance test below: arrival statistics describe the network,
// not what this buffer could use. Measuring only accepted packets let a buffer that
// was too shallow reject the very late arrivals that should have deepened it.
ObserveArrival(timestamp, arrivals[source]);
bool duplicate = false;
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == timestamp) duplicate = true;
// Frames behind `expected` are unplayable: it is the playout clock and never moves
// backward, so a late arrival would sit here forever and, as the oldest entry, would
// also mask the future packet that DRED and FEC recover from. Depth, not late
// tolerance, is what absorbs reordering here.
if ((!started || delta >= 0) && delta % frameSamples == 0 && !duplicate)
{
if (count >= maximumDepth)
@@ -89,7 +111,6 @@ internal sealed class ReceiveStream : IDisposable
int target = Array.IndexOf(sizes, 0);
timestamps[target] = timestamp; sizes[target] = lengths[source];
packets[source].AsSpan(0, lengths[source]).CopyTo(jitter[target]); count++;
ObserveArrival(timestamp, arrivals[source]);
}
Volatile.Write(ref read, unchecked(read + 1));
}
@@ -100,8 +121,10 @@ internal sealed class ReceiveStream : IDisposable
if (lastArrival != 0)
{
int timestampDelta = unchecked((int)(timestamp - lastArrivalTimestamp));
if (timestampDelta <= 0) return;
if (timestampDelta > 0 && timestampDelta <= frameSamples * 10)
// A reordered packet arrives with a negative timestamp delta, and that is precisely
// the arrival the target depth has to absorb. Ignoring it left reordering invisible
// to the estimator. Gaps far beyond a frame are talkspurt silence, not jitter.
if (Math.Abs(timestampDelta) <= frameSamples * 10)
{
double arrivalDelta = clock.GetElapsedTime(lastArrival, arrival).TotalSeconds * 48_000;
double deviation = Math.Abs(arrivalDelta - timestampDelta);
@@ -111,9 +134,12 @@ internal sealed class ReceiveStream : IDisposable
lastArrival = arrival; lastArrivalTimestamp = timestamp;
}
// The playout target never drops below a single frame. A zero target starts playout on one
// packet with no depth at all, so ordinary reordering becomes concealment even when nothing
// was actually lost; recovery modes need a further frame of lookahead on top of that floor.
private int TargetSamples()
{
int recovery = Info.Audio.Dred || Info.Audio.Fec ? frameSamples : 0;
int recovery = Info.Audio.Dred || Info.Audio.Fec ? frameSamples * 2 : frameSamples;
int variation = checked((int)Math.Ceiling(4 * jitterSamples / frameSamples)) * frameSamples;
return Math.Min(5760, recovery + variation);
}
@@ -155,6 +181,23 @@ internal sealed class ReceiveStream : IDisposable
int found = -1;
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == expected) { found = i; break; }
bool decoded = false;
// The playout clock advances one frame per call, so a target that grows mid-call has no
// way to deepen the buffer again. Hold the clock for one frame — concealing instead of
// consuming — so the standing depth can follow a link that has become jittery. Only when
// the expected frame is actually present, otherwise a loss gap would stall playout, and
// CatchUp's trim threshold sits two frames above this so the two cannot oscillate.
if (stretchCooldown > 0) stretchCooldown--;
if (started && found >= 0 && count > 0 && stretchCooldown == 0)
{
int ahead = Newest();
if (unchecked((int)(timestamps[ahead] - expected)) + frameSamples < TargetSamples())
{
stretchCooldown = StretchCooldownFrames;
if (!decoder.TryDecode([], pcm, frameSamples, out _)) pcm.AsSpan(0, frameSamples * channels).Clear();
ConcealedFrames++; Stretches++;
return;
}
}
if (found >= 0)
{
decoded = decoder.TryDecode(jitter[found].AsSpan(0, sizes[found]), pcm, frameSamples, out int result) && result == frameSamples;
+60 -10
View File
@@ -10,7 +10,9 @@ public delegate void EncodedVoiceHandler(VoiceFrameHeader header, ReadOnlySpan<b
internal sealed class ClientMediaTransport : IAsyncDisposable
{
private readonly Socket socket;
private Socket socket;
private readonly IPEndPoint endpoint;
private readonly byte[] token = new byte[MediaEncryptor.RebindTokenSize];
private readonly MediaSessionCrypto crypto;
private readonly CancellationTokenSource stop;
private readonly byte[] binding = new byte[VoiceFrameHeader.Size + 16];
@@ -21,6 +23,13 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
private readonly TaskCompletionSource bound = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly AutoResetEvent sendReady = new(false);
private int senderNapping;
private long lastInbound;
// A media path that has gone quiet for longer than this is treated as lost: the phone has
// most likely changed interface, which strands a connected UDP socket on a dead source
// address. Two missed keepalive echoes.
private const int RecoveryIdleMilliseconds = 5_000;
private const int KeepaliveMilliseconds = 2_000;
internal int Migrations { get; private set; }
internal event EncodedVoiceHandler? Received;
internal Task Bound => bound.Task;
@@ -28,10 +37,13 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
{
if (token.Length != 16) throw new IOException("Invalid UDP binding token.");
this.crypto = crypto;
this.endpoint = endpoint;
token.CopyTo(this.token.AsSpan());
stop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
socket = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
try { socket.Connect(endpoint); }
catch { socket.Dispose(); stop.Dispose(); throw; }
lastInbound = Environment.TickCount64;
new VoiceFrameHeader(MediaFrameType.UdpBinding, 0, 0, 0, 0, 0).Write(binding);
token.CopyTo(binding.AsSpan(VoiceFrameHeader.Size));
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
@@ -52,6 +64,7 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
private void Send()
{
byte[] plain = new byte[1275], packet = new byte[1275 + VoiceFrameHeader.Size + MediaEncryptor.TagSize];
byte[] rebind = new byte[MediaEncryptor.RebindSize];
long nextKeepalive = 0;
try
{
@@ -60,23 +73,39 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
bool drained = false;
try
{
// A bound path that stops echoing keepalives has been lost under us. The
// socket is connected, so it is still pinned to a source address that may no
// longer exist; rebuild it and prove possession of the media key from the
// new one so the relay moves this peer's endpoint.
if (bound.Task.IsCompleted && Environment.TickCount64 - Volatile.Read(ref lastInbound) > RecoveryIdleMilliseconds)
{
Rebuild();
int size = crypto.Encryptor.EncryptRebind(token, rebind);
Volatile.Read(ref socket).Send(rebind.AsSpan(0, size), SocketFlags.None);
Migrations++;
Volatile.Write(ref lastInbound, Environment.TickCount64);
nextKeepalive = 0;
}
if (Environment.TickCount64 >= nextKeepalive)
{
if (!bound.Task.IsCompleted) socket.Send(binding, SocketFlags.None);
socket.Send(keepalive, SocketFlags.None);
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? 5000 : 250);
Socket current = Volatile.Read(ref socket);
if (!bound.Task.IsCompleted) current.Send(binding, SocketFlags.None);
current.Send(keepalive, SocketFlags.None);
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? KeepaliveMilliseconds : 250);
}
while (packets.TryRead(plain, out VoiceFrameHeader header, out int length))
{
drained = true;
int size = crypto.Encryptor.Encrypt(header, plain.AsSpan(0, length), packet);
socket.Send(packet.AsSpan(0, size), SocketFlags.None);
Volatile.Read(ref socket).Send(packet.AsSpan(0, size), SocketFlags.None);
}
}
catch (SocketException exception) when (IsTransientNetworkError(exception))
{
// iOS can briefly lose its UDP route while Wi-Fi and cellular switch.
// Keep the sender and socket alive so the next route can carry media.
// iOS loses its UDP route while Wi-Fi and cellular switch. A connected socket
// stays bound to the vanished source address, so retrying on it never
// recovers; replace it and let the idle check above re-offer the binding.
Rebuild();
nextKeepalive = 0;
Thread.Sleep(100);
}
@@ -98,6 +127,18 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
finally { stop.Cancel(); }
}
// Replaces the UDP socket so the next send leaves over whichever interface is now current.
// Only the sender thread rebuilds; the receive loop picks the new socket up on its next read.
private void Rebuild()
{
if (stop.IsCancellationRequested) return;
Socket replacement = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
try { replacement.Connect(endpoint); }
catch { replacement.Dispose(); return; }
Socket previous = Interlocked.Exchange(ref socket, replacement);
previous.Dispose();
}
private async Task ReceiveAsync()
{
byte[] packet = new byte[65535], plain = new byte[65535];
@@ -106,10 +147,14 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
while (true)
{
int length;
try { length = await socket.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
Socket current = Volatile.Read(ref socket);
try { length = await current.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
catch (SocketException exception) when (exception.SocketErrorCode is SocketError.ConnectionReset or SocketError.MessageSize) { continue; }
catch (SocketException exception) when (IsTransientNetworkError(exception))
{ await Task.Delay(100, stop.Token).ConfigureAwait(false); continue; }
// The sender replaced the socket under us during a migration; read the new one.
catch (ObjectDisposedException) when (!stop.IsCancellationRequested && !ReferenceEquals(current, Volatile.Read(ref socket))) { continue; }
Volatile.Write(ref lastInbound, Environment.TickCount64);
if (!VoiceFrameHeader.TryRead(packet.AsSpan(0, length), out var candidate)) continue;
if (candidate.Type == MediaFrameType.Keepalive && length == VoiceFrameHeader.Size) { bound.TrySetResult(); continue; }
if (candidate.Type != MediaFrameType.Voice || candidate.Codec != 0 ||
@@ -128,9 +173,14 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
public async ValueTask DisposeAsync()
{
stop.Cancel(); socket.Dispose(); sendReady.Set();
stop.Cancel(); Volatile.Read(ref socket).Dispose(); sendReady.Set();
try { sending.Join(); await receiving.ConfigureAwait(false); }
finally { System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding); stop.Dispose(); sendReady.Dispose(); }
finally
{
System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding);
System.Security.Cryptography.CryptographicOperations.ZeroMemory(token);
stop.Dispose(); sendReady.Dispose();
}
}
// A bounded, allocation-free packet handoff. A contending producer drops instead of
+42
View File
@@ -0,0 +1,42 @@
namespace VoiceCat.Core;
/// Decides whether an observed network path change is a genuine handover — one that strands the
/// existing sockets on a source address that no longer exists — or something the connection
/// should be left alone to ride out.
///
/// The distinction matters because the response is a reconnect. Roaming between access points,
/// a tunnel, a minute of unusable cellular: those keep the same interface, so TCP survives them
/// and a reconnect would turn a recoverable glitch into a visible drop. Only the set of
/// interfaces carrying the path changing is reported as a handover.
///
/// Platform-agnostic on purpose: the caller supplies whatever stable interface identity its OS
/// reports (on iOS, NWPath's interface type and name).
public sealed class ControlPathWatcher
{
private readonly object gate = new();
private string signature = "";
/// Returns true when the path moved to a different set of interfaces and is usable again.
/// The first usable path only establishes a baseline; there is nothing to hand over from.
public bool Observe(bool usable, IReadOnlyList<string> interfaces)
{
// An unusable path is an outage, not a handover. Hold the last signature so a link that
// returns on the same interface stays silent, and so a reconnect is never started into a
// path that cannot carry it.
if (!usable) return false;
string[] sorted = [.. interfaces.Where(item => !string.IsNullOrEmpty(item))];
if (sorted.Length == 0) return false;
Array.Sort(sorted, StringComparer.Ordinal);
string current = string.Join(",", sorted);
lock (gate)
{
string previous = signature;
if (previous == current) return false;
signature = current;
return previous.Length != 0;
}
}
/// Forgets the baseline, so the next usable path establishes a new one without reporting.
public void Reset() { lock (gate) signature = ""; }
}
+50 -3
View File
@@ -42,6 +42,37 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
private uint adaptiveLossChannel;
private int adaptiveLossPercent = -1;
private ClientConnectionState state;
private long lastControlInbound;
// A control connection that stops answering is dead even though the socket still looks open.
// A phone that changes interface leaves TCP blackholed rather than reset, and the OS will not
// report it for minutes, so liveness is judged here instead.
private TimeSpan controlKeepaliveInterval = TimeSpan.FromSeconds(10);
private TimeSpan controlSilenceTimeout = TimeSpan.FromSeconds(30);
// Instance scoped so tests can shorten the window without disturbing parallel tests, and so
// a mobile client can tighten it: on a phone the interval is the delay between a handover and
// the reconnect that follows it, which a desktop on one fixed interface never pays.
// Takes effect on the next ConnectAsync; the running keepalive worker keeps its own values.
public void ConfigureControlLiveness(TimeSpan keepalive, TimeSpan silenceTimeout)
{
if (keepalive <= TimeSpan.Zero) throw new ArgumentOutOfRangeException(nameof(keepalive));
if (silenceTimeout <= keepalive) throw new ArgumentOutOfRangeException(nameof(silenceTimeout), "Silence timeout must exceed the keepalive interval.");
controlKeepaliveInterval = keepalive; controlSilenceTimeout = silenceTimeout;
}
internal void SetControlLiveness(TimeSpan keepalive, TimeSpan silenceTimeout)
{ controlKeepaliveInterval = keepalive; controlSilenceTimeout = silenceTimeout; }
// The platform can know the path is gone long before an unanswered keepalive proves it: iOS
// reports an interface change the instant it happens. Failing the connection here hands the
// existing disconnect path a real cause and starts the reconnect immediately instead of
// waiting out the silence timeout on a route that is already dead.
public void DropForReconnect(string reason)
{
if (State == ClientConnectionState.Disconnected) return;
ConnectionFailure ??= new IOException(reason);
connectionLifetime?.Cancel();
}
public event Action<ClientConnectionState>? ConnectionStateChanged;
public ClientConnectionState State { get { lock (stateGate) return state; } }
@@ -178,8 +209,20 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
{
try
{
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(10));
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false)) Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
Volatile.Write(ref lastControlInbound, Environment.TickCount64);
using var timer = new PeriodicTimer(controlKeepaliveInterval);
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false))
{
// Every server reply counts as liveness, so a busy session never trips this; an
// unanswered ping is what exposes a path that has stopped carrying anything.
if (Environment.TickCount64 - Volatile.Read(ref lastControlInbound) > controlSilenceTimeout.TotalMilliseconds)
{
ConnectionFailure ??= new IOException("Server stopped responding on the control connection.");
connectionLifetime?.Cancel();
break;
}
Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
}
}
catch (Exception exception) when (exception is OperationCanceledException or IOException or InvalidOperationException) { }
}
@@ -212,13 +255,17 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
{
await foreach (Envelope message in connection.ReadAsync(cancellationToken).ConfigureAwait(false))
{
Volatile.Write(ref lastControlInbound, Environment.TickCount64);
Apply(message);
if (message.RequestId != 0 && pending.TryRemove(message.RequestId, out var completion)) completion.TrySetResult(message.Clone());
if (!events.Writer.TryWrite(message.Clone())) throw new IOException("Client event queue exhausted; consume events regularly.");
if (message.Disconnect is not null) { connection.CompleteWrites(); break; }
}
}
catch (Exception exception) { failure = exception; ConnectionFailure = exception; }
// A liveness failure has already recorded the real cause and cancelled this read, so do
// not replace it with the cancellation it produced.
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) { }
catch (Exception exception) { failure = exception; ConnectionFailure ??= exception; }
finally
{
connectionLifetime?.Cancel();
+32 -13
View File
@@ -25,20 +25,42 @@ public sealed class MediaDecryptor : IDisposable
if (packet.Overlaps(plaintext)) throw new ArgumentException("Input and output must not overlap.", nameof(plaintext));
VoiceFrameHeader.TryRead(packet, out var candidate);
ulong sequence = candidate.Sequence;
if (initialized && sequence <= highestSequence)
{
ulong offset = highestSequence - sequence;
if (offset >= 64 || (replayWindow & (1UL << (int)offset)) != 0) return false;
}
if (IsReplay(sequence)) return false;
if (!cipher.TryDecrypt(sequence, packet[VoiceFrameHeader.Size..], packet[..VoiceFrameHeader.Size], plaintext[..length])) return false;
Accept(sequence);
header = candidate;
bytesWritten = length;
return true;
}
// Verifies an endpoint-migration frame: header, plaintext binding token, and tag. The token
// is authenticated as additional data, so only the media key holder can move an endpoint,
// and the shared replay window makes a captured frame useless to an on-path observer.
public bool TryVerifyRebind(ReadOnlySpan<byte> packet, out ReadOnlySpan<byte> token)
{
ObjectDisposedException.ThrowIf(disposed, this);
token = default;
if (packet.Length != MediaEncryptor.RebindSize) return false;
if (!VoiceFrameHeader.TryRead(packet, out var candidate) || candidate.Type != MediaFrameType.Rebind) return false;
if (IsReplay(candidate.Sequence)) return false;
int aad = VoiceFrameHeader.Size + MediaEncryptor.RebindTokenSize;
if (!cipher.TryDecrypt(candidate.Sequence, packet[aad..], packet[..aad], [])) return false;
Accept(candidate.Sequence);
token = packet.Slice(VoiceFrameHeader.Size, MediaEncryptor.RebindTokenSize);
return true;
}
private bool IsReplay(ulong sequence)
{
if (!initialized || sequence > highestSequence) return false;
ulong offset = highestSequence - sequence;
return offset >= 64 || (replayWindow & (1UL << (int)offset)) != 0;
}
// Only authenticated counters may move the replay window.
if (!initialized)
private void Accept(ulong sequence)
{
highestSequence = sequence;
replayWindow = 1;
initialized = true;
}
if (!initialized) { highestSequence = sequence; replayWindow = 1; initialized = true; }
else if (sequence > highestSequence)
{
ulong shift = sequence - highestSequence;
@@ -46,9 +68,6 @@ public sealed class MediaDecryptor : IDisposable
highestSequence = sequence;
}
else replayWindow |= 1UL << (int)(highestSequence - sequence);
header = candidate;
bytesWritten = length;
return true;
}
public void Dispose()
+19
View File
@@ -9,6 +9,8 @@ public sealed class MediaEncryptor : IDisposable
private bool disposed;
public const int TagSize = 16;
public const int RebindTokenSize = 16;
public const int RebindSize = VoiceFrameHeader.Size + RebindTokenSize + TagSize;
public MediaEncryptor(ReadOnlySpan<byte> key) : this(key, false) { }
@@ -31,6 +33,23 @@ public sealed class MediaEncryptor : IDisposable
return size;
}
// A rebind frame proves possession of the media key from a new source address. The token
// is plaintext so the relay can find the peer without trialling every key; it is covered by
// the AEAD as additional data, and the counter makes a captured frame unreplayable.
public int EncryptRebind(ReadOnlySpan<byte> token, Span<byte> packet)
{
ObjectDisposedException.ThrowIf(disposed, this);
if (token.Length != RebindTokenSize) throw new ArgumentException("Binding tokens contain 16 bytes.", nameof(token));
ArgumentOutOfRangeException.ThrowIfLessThan(packet.Length, RebindSize);
if (nextSequence == ulong.MaxValue) throw new InvalidOperationException("Media counter exhausted; establish a new session.");
var header = new VoiceFrameHeader(MediaFrameType.Rebind, VoiceFrameFlags.None, 0, 0, nextSequence++, 0);
header.Write(packet);
token.CopyTo(packet.Slice(VoiceFrameHeader.Size, RebindTokenSize));
int aad = VoiceFrameHeader.Size + RebindTokenSize;
cipher.Encrypt(header.Sequence, [], packet[..aad], packet.Slice(aad, TagSize));
return RebindSize;
}
public void Dispose()
{
if (disposed) return;
+4 -1
View File
@@ -6,7 +6,10 @@ public enum MediaFrameType : byte
{
Voice = 1,
Keepalive = 2,
UdpBinding = 3
UdpBinding = 3,
// Authenticated endpoint migration. Carries the peer's binding token in the clear for
// lookup only; the AEAD tag and replay window are what authorize the move.
Rebind = 4
}
[Flags]
@@ -4,6 +4,7 @@ using System.Net;
using System.Net.Sockets;
using System.Security.Cryptography;
using System.Threading.Channels;
using VoiceCat.Crypto;
using VoiceCat.Protocol;
using PacketLossMode = Voicecat.V1.PacketLossMode;
@@ -31,6 +32,7 @@ internal sealed class MediaRelay : IAsyncDisposable
private readonly Channel<byte> changed = Channel.CreateBounded<byte>(1);
private MediaRoute[] routes = [];
private readonly byte[] input = new byte[65535];
private readonly byte[] rebindAck = new byte[VoiceFrameHeader.Size];
private readonly MediaFanout fanout = new();
private readonly Task receiving;
internal Task Completion => receiving;
@@ -91,6 +93,30 @@ internal sealed class MediaRelay : IAsyncDisposable
foreach (MediaRoute route in current)
if (route.Peer.Endpoint?.Equals(sender) == true) { source = route; break; }
// A client whose media source address changed (a phone moving between Wi-Fi and
// cellular) keeps its TLS session but arrives here from an unknown address. The
// binding token alone travels in the clear, so it may only locate the peer; the
// authenticated tag and the peer's replay window are what authorize the move.
if (header.Type == MediaFrameType.Rebind)
{
if (length != MediaEncryptor.RebindSize) continue;
// Locate the peer by token before verifying, so a flood of forged rebinds
// costs one authentication attempt rather than one per connected peer.
MediaPeer? claimed = null;
foreach (MediaRoute route in current)
if (CryptographicOperations.FixedTimeEquals(route.Peer.Token, input.AsSpan(VoiceFrameHeader.Size, MediaEncryptor.RebindTokenSize)))
{ claimed = route.Peer; break; }
if (claimed is null || !claimed.Crypto.Decryptor.TryVerifyRebind(input.AsSpan(0, length), out _)) continue;
var moved = new SocketAddress(sender.Family, sender.Size);
for (int index = 0; index < sender.Size; index++) moved[index] = sender[index];
claimed.Endpoint = moved;
claimed.Activity.Touch();
// Echo a keepalive so the client learns its new path is carrying media.
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(rebindAck);
await SendAsync(rebindAck, sender).ConfigureAwait(false);
continue;
}
if (header.Type == MediaFrameType.UdpBinding)
{
if (length != VoiceFrameHeader.Size + 16 || source is not null) continue;
+25 -3
View File
@@ -28,7 +28,7 @@ public class AudioEngineTests
string source = File.ReadAllText(Path.Combine(FindRoot(), "src", "VoiceCat.Core", "ClientMediaTransport.cs"));
Assert.Contains("new Thread(Send)", source);
Assert.Contains("socket.Send(packet.AsSpan(0, size)", source);
Assert.Contains("Volatile.Read(ref socket).Send(packet.AsSpan(0, size)", source);
Assert.DoesNotContain("Task.Delay(5", source);
Assert.DoesNotContain("SendAsync(packet.AsMemory", source);
}
@@ -253,8 +253,14 @@ public class AudioEngineTests
[InlineData(5)] [InlineData(10)] [InlineData(20)] [InlineData(40)] [InlineData(60)]
public void RecoveryLookaheadTracksChannelFrameDuration(int frameMilliseconds)
{
// One frame of standing depth plus one of recovery lookahead. The depth floor keeps a
// quiet link from playing out with no buffer at all, where reordering alone conceals.
using var stream = new ReceiveStream(2, Stream(frameMilliseconds));
Assert.Equal(frameMilliseconds * 48, stream.TargetDepthSamples);
Assert.Equal(frameMilliseconds * 96, stream.TargetDepthSamples);
StreamInfo plain = Stream(frameMilliseconds); plain.Audio.Fec = false;
using var bare = new ReceiveStream(2, plain);
Assert.Equal(frameMilliseconds * 48, bare.TargetDepthSamples);
}
[Fact]
@@ -446,6 +452,22 @@ public class AudioEngineTests
Assert.Contains("public void Resynchronize()", buffer);
}
[Fact]
public void ResynchronizeInputsDropsStalledBacklogToTheBufferTarget()
{
using var send = new AudioEngine((_, _, _, _) => true, false);
send.DeviceBufferMilliseconds = 40; send.AddLocalStream(Stream());
short[] mono = Tone();
for (int i = 0; i < 40; i++) send.FeedPcm(1, mono, 1);
Assert.True(send.GetLocalDiagnostics(1).BufferedFrames > 1920 * 2);
send.ResynchronizeInputs();
Assert.Equal(1920, send.GetLocalDiagnostics(1).BufferedFrames);
send.ProcessCycle();
Assert.Equal(0, send.GetLocalDiagnostics(1).StarvedCycles);
}
[Fact]
public void RemotePlaybackSettingsAreIndependentForEachUserStream()
{
@@ -469,7 +491,7 @@ public class AudioEngineTests
for (uint i = 0; i < 64; i++) stream.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, length));
stream.Mix(output, false, null); Assert.InRange(stream.Depth, 0, 6);
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); stream.Mix(output, false, null); }
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 10);
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 16);
}
[Fact]
@@ -39,6 +39,7 @@ public class ManagedClientTests
await Assert.ThrowsAsync<InvalidOperationException>(() => Connect(alice, fixture));
Assert.Equal(ClientConnectionState.Connected, alice.State);
await alice.DisconnectAsync();
Assert.Null(alice.ConnectionFailure);
await Event(bob, e => e.UserEvent?.Kind == UserEvent.Types.Kind.Left);
await Connect(alice, fixture); Assert.True((await alice.AuthenticateGuestAsync("Returned")).Ok);
}
+35 -1
View File
@@ -1,6 +1,7 @@
using VoiceCat.Transport;
using System.Net;
using System.Net.Sockets;
using VoiceCat.Crypto;
using VoiceCat.Protocol;
using VoiceCat.Server.Transport;
using Voicecat.V1;
@@ -177,7 +178,7 @@ public sealed class MediaRelayTests
internal sealed class VoicePeer : IAsyncDisposable
{
public Client Client { get; }
private readonly Socket udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
private Socket udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
private readonly IPEndPoint endpoint;
private readonly MediaSessionCrypto crypto;
public ulong LastSequence { get; private set; }
@@ -228,6 +229,39 @@ public sealed class MediaRelayTests
return packet;
}
public async Task SendAsync(byte[] packet) => await udp.SendToAsync(packet, SocketFlags.None, endpoint, Client.Timeout.Token);
// Models a Wi-Fi/cellular handover: the peer keeps its TLS control session but its media
// source address changes, then it re-offers its UDP binding token from the new address.
internal async Task HandoverAsync()
{
udp.Dispose();
udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
udp.Bind(new IPEndPoint(IPAddress.Loopback, 0));
byte[] rebind = new byte[MediaEncryptor.RebindSize];
int size = crypto.Encryptor.EncryptRebind(Client.Authentication!.UdpToken.Span, rebind);
await SendAsync(rebind[..size]);
}
// A rebind captured off the wire must not let anyone else claim the peer's downlink.
internal async Task<byte[]> CaptureRebindAsync()
{
byte[] rebind = new byte[MediaEncryptor.RebindSize];
int size = crypto.Encryptor.EncryptRebind(Client.Authentication!.UdpToken.Span, rebind);
await SendAsync(rebind[..size]);
return rebind[..size];
}
// Returns true when the relay echoes a keepalive to the peer's current source address,
// which is the only signal that the server will route downlink media back to it.
internal async Task<bool> KeepaliveEchoesAsync(int timeoutMilliseconds = 1000)
{
byte[] keepalive = new byte[VoiceFrameHeader.Size];
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
await SendAsync(keepalive);
using var timeout = new CancellationTokenSource(timeoutMilliseconds);
try { byte[] buffer = new byte[65535]; await udp.ReceiveAsync(buffer, SocketFlags.None, timeout.Token); return true; }
catch (OperationCanceledException) { return false; }
}
public async Task<byte[]> ReceivePacketAsync()
{
byte[] buffer = new byte[65535];
@@ -0,0 +1,401 @@
using System.Net;
using System.Net.Sockets;
using VoiceCat.Audio;
using VoiceCat.Core;
using VoiceCat.Codec;
using VoiceCat.Protocol;
using Voicecat.V1;
using Xunit;
using static VoiceCat.Tests.ServerTests;
using Xunit.Abstractions;
namespace VoiceCat.Tests;
// Deterministic network-impairment simulation for the receive path. A virtual millisecond clock
// drives an encoder, an impairment model, and the mixer so bursty loss, jitter, reordering,
// link outages, and a stalled (backgrounded) consumer are reproducible without hardware.
public class NetworkImpairmentTests(ITestOutputHelper output)
{
// A scheduled packet. Arrival is virtual-clock milliseconds; duplicates share a timestamp.
private readonly record struct Wire(int Arrival, uint Sequence, uint Timestamp, bool Marker, int Length, byte[] Payload);
private sealed class Impairment(int seed)
{
private readonly Random random = new(seed);
private bool bursting;
internal double LossPercent, BurstLossPercent, BurstEntryPercent, BurstExitPercent = 30, JitterMs, ReorderPercent, DuplicatePercent;
internal int BaseDelayMs = 20, OutageStartMs = -1, OutageEndMs = -1;
internal bool Dropped(int sendTime)
{
if (OutageStartMs >= 0 && sendTime >= OutageStartMs && sendTime < OutageEndMs) return true;
if (BurstEntryPercent > 0)
{
bursting = bursting ? random.NextDouble() * 100 >= BurstExitPercent : random.NextDouble() * 100 < BurstEntryPercent;
if (bursting) return random.NextDouble() * 100 < BurstLossPercent;
}
return random.NextDouble() * 100 < LossPercent;
}
internal int Arrival(int sendTime)
{
double delay = BaseDelayMs + (JitterMs > 0 ? random.NextDouble() * JitterMs : 0);
if (ReorderPercent > 0 && random.NextDouble() * 100 < ReorderPercent) delay += 45;
return sendTime + (int)delay;
}
internal bool Duplicated() => DuplicatePercent > 0 && random.NextDouble() * 100 < DuplicatePercent;
}
private sealed record Report(string Name, int Frames, int SilentFrames, int LongestSilentRunMs, int Concealed, int Overruns, int Sent, int TargetDepth)
{
internal double SilentPercent => Frames == 0 ? 0 : SilentFrames * 100.0 / Frames;
public override string ToString() =>
$"{Name,-28} silent={SilentPercent,5:F1}% worstGap={LongestSilentRunMs,5}ms concealed={Concealed,4} overruns={Overruns,3} sent={Sent,4} target={TargetDepth,4}";
}
// Runs `durationMs` of a 20 ms mono talkspurt through the impairment model. The consumer
// pumps the mixer every 20 ms of virtual time except inside a stall window, which models an
// iOS render callback that stops being serviced while backgrounded.
private Report Simulate(string name, Impairment impairment, int durationMs = 20_000, bool dred = false, bool fec = true,
int stallStartMs = -1, int stallEndMs = -1)
{
StreamInfo info = AudioEngineTests.Stream(20, dred: dred);
info.Audio.Fec = fec;
var clock = new VirtualClock();
using var stream = new ReceiveStream(7, info, clock);
using var encoder = new OpusEncoder(new() { Bitrate = 32000, ForwardErrorCorrection = fec, DeepRedundancy = dred, ExpectedPacketLossPercent = 20, Complexity = 5 });
var pending = new List<Wire>();
short[] tone = new short[960];
int frames = 0, silent = 0, run = 0, longest = 0, sent = 0;
int[] mix = new int[1920];
for (int now = 0; now <= durationMs; now += 20)
{
CodecTests.FillTone(tone, 960, 1, 48000, now / 20);
byte[] packet = new byte[1275];
int length = encoder.Encode(tone, packet);
uint sequence = (uint)(now / 20);
if (!impairment.Dropped(now))
{
pending.Add(new(impairment.Arrival(now), sequence, sequence * 960, sequence == 0, length, packet));
if (impairment.Duplicated()) pending.Add(new(impairment.Arrival(now) + 5, sequence, sequence * 960, false, length, packet));
}
clock.Set(now);
foreach (Wire wire in pending.Where(w => w.Arrival <= now).OrderBy(w => w.Arrival).ToArray())
{
var flags = wire.Marker ? VoiceFrameFlags.Marker : VoiceFrameFlags.None;
sent++;
stream.Enqueue(new(MediaFrameType.Voice, flags, 0, 42, wire.Sequence, wire.Timestamp), wire.Payload.AsSpan(0, wire.Length));
pending.Remove(wire);
}
if (stallStartMs >= 0 && now >= stallStartMs && now < stallEndMs) continue;
mix.AsSpan().Clear();
stream.Mix(mix, false, null);
frames++;
bool quiet = true;
foreach (int sample in mix) if (sample != 0) { quiet = false; break; }
if (quiet) { silent++; run += 20; longest = Math.Max(longest, run); }
else run = 0;
}
var report = new Report(name, frames, silent, longest, stream.ConcealedFrames, stream.Overruns, sent, stream.TargetDepthSamples);
output.WriteLine(report.ToString());
return report;
}
[Fact]
public void ImpairmentProfileReport()
{
output.WriteLine("--- FEC on ---");
Simulate("clean", new Impairment(1));
Simulate("random loss 2%", new Impairment(2) { LossPercent = 2 });
Simulate("random loss 10%", new Impairment(3) { LossPercent = 10 });
Simulate("bursty loss", new Impairment(4) { BurstEntryPercent = 4, BurstLossPercent = 80, BurstExitPercent = 25 });
Simulate("jitter 60ms", new Impairment(5) { JitterMs = 60 });
Simulate("jitter 120ms", new Impairment(6) { JitterMs = 120 });
Simulate("reorder 5%", new Impairment(7) { ReorderPercent = 5 });
Simulate("duplicate 5%", new Impairment(8) { DuplicatePercent = 5 });
Simulate("wifi switch 3s outage", new Impairment(9) { OutageStartMs = 6000, OutageEndMs = 9000 });
Simulate("bad wifi (loss+jitter)", new Impairment(10) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 });
Simulate("background stall 2s", new Impairment(11), stallStartMs: 6000, stallEndMs: 8000);
Simulate("background stall 10s", new Impairment(13), stallStartMs: 6000, stallEndMs: 16000);
Simulate("stall + jitter", new Impairment(12) { JitterMs = 60 }, stallStartMs: 6000, stallEndMs: 8000);
output.WriteLine("--- FEC off and DRED off ---");
Simulate("nofec clean", new Impairment(21), fec: false);
Simulate("nofec loss 2%", new Impairment(22) { LossPercent = 2 }, fec: false);
Simulate("nofec loss 10%", new Impairment(23) { LossPercent = 10 }, fec: false);
Simulate("nofec jitter 30ms", new Impairment(24) { JitterMs = 30 }, fec: false);
Simulate("nofec reorder 5%", new Impairment(25) { ReorderPercent = 5 }, fec: false);
Simulate("nofec bad wifi", new Impairment(26) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 }, fec: false);
}
// Bounds are set well above the measured result so ordinary codec variation does not make
// them flaky; they exist to catch a structural regression in the receive path, such as the
// depth floor or the arrival estimator being lost again.
[Theory]
// impairment, maxConcealedPercent, maxGapMs
[InlineData("loss2", 12)]
[InlineData("loss10", 12)]
[InlineData("burst", 20)]
[InlineData("jitter60", 12)]
[InlineData("jitter120", 12)]
[InlineData("reorder", 5)]
[InlineData("badwifi", 12)]
public void ImpairedLinksStayIntelligible(string profile, int maxConcealedPercent)
{
Impairment impairment = profile switch
{
"loss2" => new(2) { LossPercent = 2 },
"loss10" => new(3) { LossPercent = 10 },
"burst" => new(4) { BurstEntryPercent = 4, BurstLossPercent = 80, BurstExitPercent = 25 },
"jitter60" => new(5) { JitterMs = 60 },
"jitter120" => new(6) { JitterMs = 120 },
"reorder" => new(7) { ReorderPercent = 5 },
_ => new(10) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 },
};
Report report = Simulate(profile, impairment);
Assert.True(report.Concealed * 100 / report.Frames <= maxConcealedPercent,
$"{profile} concealed {report.Concealed} of {report.Frames} frames.");
Assert.True(report.LongestSilentRunMs <= 200, $"{profile} went silent for {report.LongestSilentRunMs} ms.");
}
// Pure reordering loses no data at all, so it must be absorbed by depth rather than concealed.
// Without the depth floor and an estimator that observes late arrivals this was 47 frames.
[Fact]
public void ReorderingWithoutLossIsAbsorbedRatherThanConcealed()
{
Report report = Simulate("reorder no fec", new Impairment(25) { ReorderPercent = 5 }, fec: false);
Assert.Equal(1000, report.Sent);
Assert.True(report.Concealed <= 15, $"Concealed {report.Concealed} frames despite losing none.");
}
// A consumer that stops draining (an interrupted or rebuilding iOS graph) must not cost the
// live talkspurt. The handoff previously refused new packets while full, discarding 437 of
// 1000 packets across a ten second stall.
[Fact]
public void AStalledConsumerLosesBoundedAudioRatherThanTheLiveTalkspurt()
{
Report report = Simulate("stall", new Impairment(13), stallStartMs: 6000, stallEndMs: 16000);
Assert.Equal(1000, report.Sent);
Assert.InRange(report.Overruns, 1, 4);
Assert.True(report.LongestSilentRunMs <= 200, $"Silent for {report.LongestSilentRunMs} ms after the stall.");
}
// A Wi-Fi/cellular handover changes the client's media source address while TLS survives.
// The relay binds a peer's endpoint once and refuses to move it, and the client stops
// offering its binding token after the first bind, so media must not silently die here.
[Fact]
public async Task MediaSurvivesAHandoverThatChangesTheClientSourceAddress()
{
await using var fixture = new ServerFixture();
await using var alice = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Alice");
await using var bob = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Bob");
StreamAnnounceResult stream = await alice.AnnounceAsync(StreamKind.StreamMic);
await alice.SendAsync(alice.Seal(stream.Ssrc, [1]));
await bob.ReceiveVoiceAsync();
await alice.HandoverAsync();
Assert.True(await alice.KeepaliveEchoesAsync(), "Relay stopped routing to Alice after her media source address changed.");
await alice.SendAsync(alice.Seal(stream.Ssrc, [2]));
(_, byte[] payload) = await bob.ReceiveVoiceAsync();
Assert.Equal<byte[]>([2], payload);
}
// The rebind token travels in the clear so the relay can locate the peer without trialling
// every key. Authorization comes from the AEAD tag and the replay window, so an on-path
// observer who captures a rebind must not be able to redirect the peer's downlink.
[Fact]
public async Task ReplayedRebindFromAnotherAddressCannotStealTheDownlink()
{
await using var fixture = new ServerFixture();
await using var alice = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Alice");
await using var bob = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Bob");
StreamAnnounceResult stream = await alice.AnnounceAsync(StreamKind.StreamMic);
byte[] captured = await alice.CaptureRebindAsync();
Assert.True(await alice.KeepaliveEchoesAsync());
using var attacker = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
attacker.Bind(new IPEndPoint(IPAddress.Loopback, 0));
await attacker.SendToAsync(captured, SocketFlags.None, fixture.Server.MediaEndPoint);
using (var timeout = new CancellationTokenSource(500))
{
byte[] buffer = new byte[65535];
await Assert.ThrowsAnyAsync<OperationCanceledException>(async () =>
await attacker.ReceiveAsync(buffer, SocketFlags.None, timeout.Token));
}
// Alice still owns the path, so her media keeps flowing to Bob.
await alice.SendAsync(alice.Seal(stream.Ssrc, [9]));
(_, byte[] payload) = await bob.ReceiveVoiceAsync();
Assert.Equal<byte[]>([9], payload);
Assert.True(await alice.KeepaliveEchoesAsync());
}
// A phone that changes interface leaves TCP blackholed rather than reset: the socket stays
// open and the OS reports nothing for minutes. Only an unanswered keepalive exposes it, and
// until it does the app shows a live session over a dead path and never reconnects.
[Fact]
public async Task ABlackholedControlConnectionIsDetectedInsteadOfAppearingConnected()
{
await using var fixture = new ServerFixture();
await using var proxy = new BlackholeProxy(fixture.Server.EndPoint);
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(fixture.Directory, "tofu.txt"));
client.SetControlLiveness(TimeSpan.FromMilliseconds(200), TimeSpan.FromSeconds(2));
var disconnected = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
client.ConnectionStateChanged += state => { if (state == ClientConnectionState.Disconnected) disconnected.TrySetResult(); };
await client.ConnectAsync("127.0.0.1", (ushort)proxy.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
await client.AuthenticateGuestAsync("Alice");
Assert.Equal(ClientConnectionState.Connected, client.State);
proxy.Freeze();
await disconnected.Task.WaitAsync(TimeSpan.FromSeconds(15));
Assert.Equal(ClientConnectionState.Disconnected, client.State);
Assert.IsType<IOException>(client.ConnectionFailure);
}
// The reconnect is only worth paying for a path change that TCP genuinely cannot survive.
// Walking a house between access points, or a link that is merely unusable for a while, keeps
// the same interface and the same source address, and a reconnect there would turn a
// recoverable glitch into the visible drop this whole change exists to remove.
[Fact]
public void OnlyAChangedInterfaceCountsAsAHandover()
{
var watcher = new ControlPathWatcher();
// The first usable path is a baseline, not a handover.
Assert.False(watcher.Observe(true, ["wifi:en0"]));
// Roaming between access points, and the same path reported again.
Assert.False(watcher.Observe(true, ["wifi:en0"]));
// A dead spot: unusable, then back on the interface it left from.
Assert.False(watcher.Observe(false, []));
Assert.False(watcher.Observe(false, []));
Assert.False(watcher.Observe(true, ["wifi:en0"]));
// Wi-Fi gives out and cellular takes over: the source address is gone.
Assert.True(watcher.Observe(true, ["cellular:pdp_ip0"]));
Assert.False(watcher.Observe(true, ["cellular:pdp_ip0"]));
// And back onto Wi-Fi on arriving home.
Assert.True(watcher.Observe(true, ["wifi:en0"]));
// Interface order is a reporting detail, not a change.
Assert.True(watcher.Observe(true, ["wifi:en0", "cellular:pdp_ip0"]));
Assert.False(watcher.Observe(true, ["cellular:pdp_ip0", "wifi:en0"]));
// After a reset the next usable path is a baseline again.
watcher.Reset();
Assert.False(watcher.Observe(true, ["wired:en5"]));
Assert.True(watcher.Observe(true, ["wifi:en0"]));
}
// The platform knows the path changed long before an unanswered keepalive can prove it. A
// client told directly must fail the connection at once, with the real cause, so the reconnect
// runs while the audio session is still up instead of after the silence timeout expires.
[Fact]
public async Task ADroppedControlConnectionReconnectsWithoutWaitingOutTheSilenceTimeout()
{
await using var fixture = new ServerFixture();
await using var proxy = new BlackholeProxy(fixture.Server.EndPoint);
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(fixture.Directory, "tofu.txt"));
client.SetControlLiveness(TimeSpan.FromSeconds(30), TimeSpan.FromMinutes(5));
var disconnected = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
client.ConnectionStateChanged += state => { if (state == ClientConnectionState.Disconnected) disconnected.TrySetResult(); };
await client.ConnectAsync("127.0.0.1", (ushort)proxy.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
await client.AuthenticateGuestAsync("Alice");
Assert.Equal(ClientConnectionState.Connected, client.State);
proxy.Freeze();
client.DropForReconnect("The network interface changed to cellular:pdp_ip0.");
await disconnected.Task.WaitAsync(TimeSpan.FromSeconds(5));
Assert.Equal(ClientConnectionState.Disconnected, client.State);
Assert.Contains("cellular:pdp_ip0", Assert.IsType<IOException>(client.ConnectionFailure).Message);
// Dropping an already dead connection is a no-op, not a second failure.
client.DropForReconnect("Ignored.");
Assert.Contains("cellular:pdp_ip0", client.ConnectionFailure!.Message);
}
// A silence timeout at or below the keepalive interval would fail every healthy connection on
// its first tick, so a mobile client tightening these cannot be allowed to invert them.
[Fact]
public async Task ControlLivenessRejectsAWindowThatCannotBeMet()
{
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(Path.GetTempPath(), Path.GetRandomFileName()));
Assert.Throws<ArgumentOutOfRangeException>(() => client.ConfigureControlLiveness(TimeSpan.Zero, TimeSpan.FromSeconds(12)));
Assert.Throws<ArgumentOutOfRangeException>(() => client.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5)));
client.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(12));
}
// Forwards TCP both ways until frozen, after which bytes are swallowed and the sockets are
// left open — what a vanished route looks like to the client, unlike a close or a reset.
private sealed class BlackholeProxy : IAsyncDisposable
{
private readonly Socket listener = new(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
private readonly CancellationTokenSource stop = new();
private readonly IPEndPoint origin;
private volatile bool frozen;
internal IPEndPoint EndPoint { get; }
internal void Freeze() => frozen = true;
internal BlackholeProxy(IPEndPoint origin)
{
this.origin = origin;
listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
listener.Listen(4);
EndPoint = (IPEndPoint)listener.LocalEndPoint!;
_ = AcceptAsync();
}
private async Task AcceptAsync()
{
try
{
while (!stop.IsCancellationRequested)
{
Socket inbound = await listener.AcceptAsync(stop.Token);
Socket outbound = new(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
await outbound.ConnectAsync(origin, stop.Token);
_ = PumpAsync(inbound, outbound);
_ = PumpAsync(outbound, inbound);
}
}
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException) { }
}
private async Task PumpAsync(Socket from, Socket to)
{
byte[] buffer = new byte[16384];
try
{
while (!stop.IsCancellationRequested)
{
int length = await from.ReceiveAsync(buffer, SocketFlags.None, stop.Token);
if (length == 0) break;
if (frozen) continue;
await to.SendAsync(buffer.AsMemory(0, length), SocketFlags.None, stop.Token);
}
}
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException) { }
}
public ValueTask DisposeAsync()
{
stop.Cancel(); listener.Dispose(); stop.Dispose();
return ValueTask.CompletedTask;
}
}
private sealed class VirtualClock : TimeProvider
{
private long milliseconds;
public override long TimestampFrequency => 1000;
public override long GetTimestamp() => milliseconds;
internal void Set(long value) => milliseconds = value;
}
}
@@ -1,4 +1,5 @@
using System.Diagnostics;
using System.Text.RegularExpressions;
namespace VoiceCat.Tests;
@@ -117,6 +118,25 @@ public class PublishServerScriptTests
Assert.DoesNotContain("if (OperatingSystem.IsIOSVersionAtLeast(27)) { model.ToggleScreenAudio();", settings);
}
[Fact]
public async Task IosBroadcastPumpKeepsOneRingMappingAcrossTicks()
{
string root = FindRoot();
string pump = await File.ReadAllTextAsync(Path.Combine(
root, "clients", "apple", "VoiceCat.iOS", "BroadcastAudioPump.cs"));
// A mapping pair plus its container lookup and path strings per 5 ms tick allocated
// steadily at 200 Hz and churned the GC under long calls. The pump opens one mapping on
// the ring's transitions and reuses it; the producer never replaces the ring file.
Assert.Single(Regex.Matches(pump, "MemoryMappedFile.CreateFromFile"));
Assert.Contains("private bool OpenMapping()", pump);
Assert.Contains("private void CloseMapping()", pump);
Assert.Contains("Thread.Sleep(view is null ? 100 : 5)", pump);
Assert.Contains("File.Exists(path)", pump);
Assert.DoesNotContain("using MemoryMappedFile", pump);
Assert.DoesNotContain("using MemoryMappedViewAccessor", pump);
}
[Fact]
public async Task IosCaptureDoesNotDependOnCoalescedManagedTimers()
{
@@ -144,6 +164,42 @@ public class PublishServerScriptTests
Assert.Contains("deviceClockedAudio: true", model);
}
[Fact]
public async Task IosRecoversBackgroundAudioWithoutForegrounding()
{
string root = FindRoot();
string microphone = await File.ReadAllTextAsync(Path.Combine(
root, "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
string router = await File.ReadAllTextAsync(Path.Combine(
root, "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
// The render callback is the only clock for the device-clocked pipeline, so a graph that
// stops while backgrounded froze capture, mix and send until the app was foregrounded.
Assert.Contains("internal long RenderCallbacks", microphone);
Assert.Contains("internal bool IsRunning", microphone);
Assert.Contains("private void TickWatchdog()", router);
Assert.Contains("IosAudioEngine.Shared.RenderCallbacks", router);
Assert.Contains("IosAudioEngine.Shared.IsRunning", router);
// Bounded: a rebuild that does not restore the callbacks backs off and then stops, so a
// graph that cannot start fails visibly instead of churning the session forever.
Assert.Contains("IosAudioEngine.Shared.Reconfigure(true, $\"stall watchdog {watchdogAttempts}\")", router);
Assert.Contains("watchdogAttempts >= MaximumWatchdogAttempts", router);
Assert.Contains("VC_WATCHDOG exhausted", router);
// SetActive fails for as long as an interruption is in force, so Began suppresses
// recovery and the watchdog retries the rebuild that Ended asks for.
Assert.Contains("AVAudioSessionInterruptionType.Began", router);
Assert.Contains("interrupted = true", router);
Assert.Contains("internal void ResumeForeground()", router);
// A stalled render callback resynchronizes its own rings, and a refill is bounded so it
// cannot overrun the callback deadline that it is recovering from.
Assert.Contains("owner.Audio.ResynchronizeInputs()", microphone);
Assert.Contains("playbackRing.Resynchronize()", microphone);
Assert.Contains("Ring.Resynchronize()", microphone);
Assert.Contains("Math.Min(frames + playbackRing.TargetFrames - playbackRing.CountFrames, frames + 960)", microphone);
}
[Fact]
public async Task IosAudioCommitsInputRouteBeforeChannelsAndVoiceProcessingBeforeConnections()
{
@@ -151,8 +207,11 @@ public class PublishServerScriptTests
string router = await File.ReadAllTextAsync(Path.Combine(root, "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
string engine = await File.ReadAllTextAsync(Path.Combine(root, "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
Assert.True(router.IndexOf("ApplyInputSelection(session)", StringComparison.Ordinal) <
router.IndexOf("SetActive(true", StringComparison.Ordinal));
Assert.True(router.IndexOf("SetActive(true", StringComparison.Ordinal) <
router.IndexOf("ApplyInputSelection(session)", StringComparison.Ordinal));
Assert.True(router.IndexOf("session.SetPreferredInput(port", StringComparison.Ordinal) <
router.IndexOf("port.SetPreferredDataSource(source", StringComparison.Ordinal));
Assert.DoesNotContain("session.SetInputDataSource(source", router);
Assert.DoesNotContain("session.SetPreferredInputNumberOfChannels", router);
Assert.DoesNotContain("MaximumInputNumberOfChannels", router);
Assert.Contains("SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait", router);
@@ -166,7 +225,66 @@ public class PublishServerScriptTests
Assert.True(engine.IndexOf("SetVoiceProcessingEnabled", StringComparison.Ordinal) <
engine.IndexOf("next.Connect(source", StringComparison.Ordinal));
Assert.Contains("AVAudioEngine.ConfigurationChangeNotification", engine);
Assert.Contains("ReferenceEquals(engine, next) && !next.Running", engine);
Assert.Contains("if (!IsConnected || !ReferenceEquals(engine, next)) return;", engine);
Assert.Contains("if (!next.Running || HardwareChanged()) Rebuild(\"configuration change\");", engine);
// With voice processing the input and output are one IO unit, and it only stays up while the
// input is rendered. A tap alone leaves it out of the chain and the unit dies seconds later.
Assert.Contains("next.Connect(input, captureSink, inputFormat", engine);
Assert.Contains("captureSink.OutputVolume = 0f;", engine);
// Capture exists for the session, so joining voice names a stream instead of rebuilding.
Assert.Contains("internal void StopMicrophone() { Volatile.Write(ref microphone, null); }", engine);
Assert.Contains("bool captures = IsConnected;", engine);
}
[Fact]
public async Task IosSpeakerOutputIsAnOrthogonalOneShotChoice()
{
string router = await File.ReadAllTextAsync(Path.Combine(
FindRoot(), "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
// Speaker output decides where audio goes, not how it is captured, so it must not take the
// preset with it, and no preset may clear it back.
Assert.Contains("ForceSpeaker = value; speakerIsExplicit = overridePending = true", router);
Assert.DoesNotContain("ForceSpeaker = value; Preset = IosAudioPreset.Advanced", router);
Assert.DoesNotContain("IosAudioPreset.VoiceChat) ForceSpeaker = false", router);
// The stored flag is honoured only once the user has actually chosen, so the value older
// installs inherited from the voice-chat preset cannot pin a headset user to the speaker.
Assert.Contains("cat.voice.audio.speakerIsExplicit", router);
// The port override is the toggle's one-shot request. Re-asserting it on every rebuild
// means fighting the system for the route, which the user sees as audio flipping.
Assert.Contains("if (overridePending)", router);
Assert.Contains("overridePending = false;", router);
// Every reconfiguration moves the route and moving the route notifies the handler, so a
// route change must never force a rebuild: that is one rebuild per notification, without
// end. The graph is rebuilt only when it stopped or when the hardware it was built around
// moved, which is the one thing AVAudioEngine cannot absorb by itself.
Assert.Contains("if (applying) return;", router);
Assert.Contains("Recover($\"route change ({reason})\", force: false)", router);
Assert.DoesNotContain("Recover($\"route change ({reason})\");", router);
string engine = await File.ReadAllTextAsync(Path.Combine(
FindRoot(), "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
// Asked of the graph's own input, not of AVAudioSession and not remembered from a route:
// the session's reported rate and channel count do not settle until after the graph has
// started, and CurrentRoute still names the previous route for a while, so either one
// reports a change that has not happened and every rebuild reports it again.
Assert.Contains("live.InputNode.GetBusOutputFormat(0)", engine);
Assert.Contains("format.SampleRate != builtInputRate || format.ChannelCount != builtInputChannels", engine);
Assert.DoesNotContain("AVAudioSession.SharedInstance().SampleRate", engine);
// Every rebuild names what asked for it, so a storm can be read from a log.
Assert.Contains("VC_REBUILD cause=", engine);
// Building a graph is itself what posts most configuration changes, and the engine reads as
// stopped until voice processing has rebuilt the IO. Nothing may judge a graph dead inside
// that window, or it replaces the graph it just built with one that reports the same thing.
Assert.Contains("internal bool Settling =>", engine);
Assert.Contains("if (!force && Settling) return;", engine);
Assert.Contains("if (Settling) return;", engine);
Assert.Contains("if (Settling) return true;", engine);
Assert.Contains("|| IosAudioEngine.Shared.Settling) { ResetWatchdog(); return; }", router);
Assert.Contains("&& !HardwareChanged()) return;", engine);
Assert.Contains("(!next.Running || HardwareChanged())", engine);
Assert.Contains("if (Preset == IosAudioPreset.VoiceChat) { SelectedInputId = null; SelectedDataSourceId = null; }", router);
Assert.DoesNotContain("SelectedInputId ??= session.PreferredInput", router);
}
[Fact]