Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a918334e06 | ||
|
|
4bad264605 | ||
|
|
3dbe763598 | ||
|
|
c1763c9a5d | ||
|
|
4cc13a27a0 | ||
|
|
5a2dddba89 | ||
|
|
01bae734b8 | ||
|
|
1a0ff957ae | ||
|
|
938b23d4c0 | ||
|
|
7033053184 | ||
|
|
69a9729c28 | ||
|
|
0dad40c9d7 | ||
|
|
0b81b81c0c | ||
|
|
724f7e912d | ||
|
|
f3ac779bf4 | ||
|
|
fb740bcfb2 | ||
|
|
186fe6dbb6 | ||
|
|
f348574bc1 | ||
|
|
1487f2673b |
+75
-7
@@ -1,6 +1,6 @@
|
||||
# VoiceCat status
|
||||
|
||||
Updated: 2026-09-22
|
||||
Updated: 2026-09-25
|
||||
|
||||
## Current state
|
||||
|
||||
@@ -23,11 +23,67 @@ The physical-device iOS voice path now supports ReplayKit fallback, stable Apple
|
||||
capture using a paced 20 ms handoff, and true built-in stereo microphone capture. Stereo was
|
||||
verified on an iPhone 16 Pro Max with a two-channel AVAudioEngine input and distinct left/right
|
||||
samples; the managed Apple binding requires native use of its otherwise-unmapped stereo polar
|
||||
pattern constant.
|
||||
pattern constant. The voice-chat preset now leaves speaker routing off so a connected Bluetooth
|
||||
headset can supply both input and output; speaker routing remains an explicit Advanced setting.
|
||||
Device-selected inputs are not saved as preferences during route refresh. Bluetooth switching
|
||||
still needs device validation.
|
||||
|
||||
The iOS user list now opens a remote-user detail view with independent tuning for each active
|
||||
audio stream. Private messages are grouped into per-user conversations with direct access to the
|
||||
same user and audio controls.
|
||||
same user and audio controls. Lists reload only when their rendered content actually changed and
|
||||
only while on screen, and the 20 Hz microphone level is a separate signal from the general model
|
||||
change, so VoiceOver explore mode no longer re-announces the row under a dragging finger or loses
|
||||
the element a double tap was aimed at.
|
||||
|
||||
The iOS audio graph is rebuilt only when the audio configuration changed. A lost connection
|
||||
unbinds the client but keeps the session, graph, and route alive, so a reconnect rebinds to a
|
||||
live Bluetooth HFP link instead of renegotiating one, and foregrounding ensures the graph is
|
||||
running rather than rebuilding it. The input side — input node, voice processing, and capture tap
|
||||
— is built once per session rather than added when voice is joined, so joining and leaving voice
|
||||
name a stream instead of replacing the graph. The input is rendered through a silent mixer,
|
||||
because the voice-processing IO unit only stays up while the input is in the render chain. Route
|
||||
changes no longer force a rebuild; media-services resets still do, and the stall watchdog now
|
||||
backs off and stops after four failed attempts instead of rebuilding without end.
|
||||
|
||||
An explicit disconnect now pauses the prepared iOS graph and releases the audio session; the
|
||||
next connection resumes that graph before attempting a rebuild. A signed iOS 26 simulator gate
|
||||
verified guest-to-account editing, saved-password login after a process restart, two voice joins,
|
||||
and a screen-audio stream started before the microphone. Screen sharing no longer forces a graph
|
||||
rebuild, and its pump subscribes before announcing the stream. The pump retains up to 120 ms of
|
||||
screen PCM captured during stream negotiation; the simulator gate now checks that this PCM
|
||||
reaches the encoder and passed on iOS 26. Audible screen sharing was confirmed on an iPhone 16
|
||||
Pro Max after deployment. An idle shared broadcast ring is no longer held mapped through suspension. A
|
||||
TestFlight report from build 2026092504 was an iOS
|
||||
`0xdead10cc` shared-file-lock termination; physical-device validation of that mitigation remains
|
||||
open.
|
||||
|
||||
Stereo capture now selects the preferred input only after activating the audio session and no
|
||||
longer calls `SetInputDataSource` with a source from a port that may not yet be current. A Debug
|
||||
build installed on an iPhone 16 Pro Max connected with the StereoMicrophone preset and recorded
|
||||
a two-channel input graph with distinct left/right PCM and encoded packets. The simulator's mono
|
||||
input also completed stereo-preset login, voice join, disconnect, and reconnect.
|
||||
|
||||
Voice-chat capture is not yet reliable on the first attempt: a graph with voice processing still
|
||||
sometimes fails to start and is only recovered by a watchdog rebuild, which costs seconds before
|
||||
audio appears. Speaker output is an explicit choice that no longer changes the preset. Every
|
||||
rebuild logs `VC_REBUILD cause=`, and `VC_START`/`VC_START_CHECK` record whether the graph
|
||||
survived its start; that instrumentation is what the remaining investigation needs.
|
||||
|
||||
The media path now survives changing networks. A client whose source address changes proves
|
||||
possession of its media key from the new address with an authenticated `Rebind` frame and the
|
||||
relay moves its endpoint, instead of the session dying silently in both directions; the client
|
||||
rebuilds its UDP socket rather than retrying on one pinned to a vanished interface. The control
|
||||
connection is judged live by server traffic rather than assumed live, so a blackholed TCP path
|
||||
is detected in 30 s instead of waiting minutes for the OS, and 12 s on iOS. iOS also watches the
|
||||
system path and fails the control connection the moment the carrying interface changes, so a
|
||||
handover reconnects in about a second instead of waiting out the silence timeout; access-point
|
||||
roaming and an unusable-but-unchanged link are deliberately not handovers and are ridden out.
|
||||
The first reconnect attempt is immediate, and a loss is only announced once an attempt has
|
||||
actually failed, so a handover reads as a hiccup rather than a dropped call. A control reconnect
|
||||
still re-authenticates and rejoins: seamless handover needs control-plane session resumption,
|
||||
because the media keys come from the TLS exporter of the connection that was lost. The receive jitter buffer keeps a
|
||||
one-frame depth floor, measures late and reordered arrivals, and can deepen mid-call, and a
|
||||
stalled consumer now costs bounded audio rather than the live talkspurt.
|
||||
|
||||
SQLite schema v4 persists DRED and the channel packet-loss mode. Manual loss remains the default;
|
||||
automatic Fast/Balanced/Stable modes measure each sender's authenticated UDP uplink at the server,
|
||||
@@ -35,6 +91,12 @@ cap the applied Opus hint at 30%, and feed it back over TLS.
|
||||
|
||||
## Release gates
|
||||
|
||||
- Find why an iOS graph with voice processing intermittently starts and then stops within a
|
||||
second, so voice-chat capture comes up on the first attempt rather than after a watchdog rebuild.
|
||||
- On physical iOS 27 hardware, repeat account login after app restart, explicit disconnect and
|
||||
reconnect, microphone permission denial, ReplayKit and ScreenCaptureKit sharing with and
|
||||
without a joined microphone, and background/foreground transitions. Confirm the graph keeps
|
||||
rendering and that `0xdead10cc` does not recur.
|
||||
- Run real multi-person calls on Windows, macOS, and physical iOS hardware, including adaptive
|
||||
20/40/60 ms buffering, duration-aware DRED/FEC, automatic packet-loss feedback, and mismatched
|
||||
input/output endpoints.
|
||||
@@ -42,11 +104,17 @@ cap the applied Opus hint at 30%, and feed it back over TLS.
|
||||
- Verify iOS remote-user tuning and private-conversation navigation with VoiceOver, including
|
||||
multiple streams, users without active streams, and users who disconnect while a view is open.
|
||||
- Exercise iOS background/lock, interruption, Bluetooth, route-change, ReplayKit, and iOS 27
|
||||
ScreenCaptureKit paths on devices. Complete a 30-minute iOS call and Wi-Fi/cellular switching
|
||||
with voice restoration, plus extended mono/stereo/voice-chat switching while joined. Verify
|
||||
Windows desktop/per-app stereo sharing.
|
||||
ScreenCaptureKit paths on devices. The background/lock gate keeps a call active for 15+ minutes
|
||||
backgrounded and screen-locked with no periodic glitches and flat `VC_AUDIO` `feedDrops`/`starved`
|
||||
counters (the render callback now paces the mix and the 20 ms capture handoff, and a watchdog
|
||||
rebuilds a graph that stops calling back). Take a Siri or phone-call interruption while
|
||||
backgrounded and confirm audio resumes without foregrounding. Complete a
|
||||
30-minute iOS call and Wi-Fi/cellular switching with voice restoration (the switch is covered
|
||||
by simulation in `NetworkImpairmentTests`; hardware confirms the real route change), plus extended
|
||||
mono/stereo/voice-chat switching while joined. Verify Windows desktop/per-app stereo sharing.
|
||||
- Complete Developer ID signing/notarization. The iOS host and ReplayKit extension have been
|
||||
distribution-signed and packaged locally; upload the IPA for Apple's server-side validation.
|
||||
distribution-signed and packaged as build `2026092701`; upload the IPA for Apple's server-side
|
||||
validation.
|
||||
- Run the published Linux container and a 30-minute-or-longer server soak.
|
||||
|
||||
## Working rule
|
||||
|
||||
@@ -24,7 +24,10 @@ require `VOICECAT_CODESIGN_IDENTITY`; optional notarization uses `APPLE_ID`, `AP
|
||||
For a physical iOS device, use `build-ios-device.sh` and `deploy-ios-device.sh`. The host and
|
||||
ReplayKit extension require signing profiles with App Group `group.me.iamtalon.voicecat`.
|
||||
Hardware validation must cover VoiceOver, background and lock behavior, interruptions, route
|
||||
changes, Bluetooth, ReplayKit, and iOS 27 ScreenCaptureKit audio.
|
||||
changes, Bluetooth, ReplayKit, and iOS 27 ScreenCaptureKit audio. The opt-in simulator share
|
||||
smoke gate checks that ring PCM captured during stream startup reaches the encoder; audible
|
||||
screen sharing was confirmed on an iPhone 16 Pro Max. The iOS 27 capture path still needs a
|
||||
device check.
|
||||
For iOS voice stability, leave a call joined with the microphone active for at least 30 minutes
|
||||
and confirm speech stays clear and `VC_AUDIO` reports no growing `feedDrops`. While still joined,
|
||||
toggle Wi-Fi off and on, switch between Wi-Fi and cellular, and confirm the app stays open,
|
||||
|
||||
@@ -6,9 +6,18 @@ namespace VoiceCat.iOS;
|
||||
|
||||
internal sealed class UsersController : UITableViewController
|
||||
{
|
||||
private readonly AppModel model; private IReadOnlyList<User> Visible => model.CurrentChannelId == 0 ? model.Users : model.Users.Where(user => user.ChannelId == model.CurrentChannelId).ToArray();
|
||||
internal UsersController(AppModel model) { this.model = model; Title = "Users"; model.Changed += () => TableView.ReloadData(); }
|
||||
private readonly AppModel model; private readonly ListRefresher refresher;
|
||||
private IReadOnlyList<User> Visible => model.CurrentChannelId == 0 ? model.Users : model.Users.Where(user => user.ChannelId == model.CurrentChannelId).ToArray();
|
||||
internal UsersController(AppModel model)
|
||||
{
|
||||
this.model = model; Title = "Users";
|
||||
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "user"); }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature() => string.Join('\n', Visible.Select(user =>
|
||||
$"{user.Id}|{user.Nickname}|{user.Id == model.SelfUserId}|{user.ServerDeafened}|{user.ServerMuted}|{user.SelfDeafened}|{user.SelfMicMuted}|{user.IsGuest}"));
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
|
||||
{
|
||||
@@ -34,7 +43,7 @@ internal sealed class UsersController : UITableViewController
|
||||
|
||||
internal sealed class UserDetailController : FormController
|
||||
{
|
||||
private readonly AppModel model; private readonly uint userId;
|
||||
private readonly AppModel model; private readonly uint userId; private readonly ListRefresher refresher;
|
||||
private User? User => model.Users.FirstOrDefault(value => value.Id == userId);
|
||||
private IReadOnlyList<(string Title, Action Run, bool Destructive)> AdminActions
|
||||
{
|
||||
@@ -55,8 +64,20 @@ internal sealed class UserDetailController : FormController
|
||||
}
|
||||
}
|
||||
internal UserDetailController(AppModel model, uint userId) : base(model.Users.FirstOrDefault(value => value.Id == userId)?.Nickname ?? $"User {userId}")
|
||||
{ this.model = model; this.userId = userId; model.Changed += Reload; }
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "detail"); Reload(); }
|
||||
{
|
||||
this.model = model; this.userId = userId;
|
||||
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "detail"); }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature()
|
||||
{
|
||||
User? user = User; Title = user?.Nickname ?? $"User {userId}";
|
||||
string streams = user is null ? "offline" : string.Join(',', user.Streams.Select(stream =>
|
||||
$"{stream.StreamId}:{stream.Kind}:{stream.Label}:{model.Client?.Audio.GetRemotePlayback(userId, stream.StreamId)}"));
|
||||
return $"{Title}|{streams}|{string.Join(',', AdminActions.Select(action => action.Title))}";
|
||||
}
|
||||
public override nint NumberOfSections(UITableView tableView) => AdminActions.Count == 0 ? 2 : 3;
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => section switch
|
||||
{ 0 => Math.Max(User?.Streams.Count ?? 0, 1), 1 => 1, _ => AdminActions.Count };
|
||||
@@ -93,7 +114,6 @@ internal sealed class UserDetailController : FormController
|
||||
else if (indexPath.Section == 1) NavigationController?.PushViewController(new PrivateConversationController(model, user.Id, user.Nickname), true);
|
||||
else if (indexPath.Section == 2) AdminActions[indexPath.Row].Run();
|
||||
}
|
||||
private void Reload() { Title = User?.Nickname ?? $"User {userId}"; if (IsViewLoaded) TableView.ReloadData(); }
|
||||
private void PromptReason(User user, bool ban) { UIAlertController prompt = UIAlertController.Create(ban ? "Ban user" : "Kick user", "Reason (optional)", UIAlertControllerStyle.Alert); prompt.AddTextField(field => field.AccessibilityLabel = "Reason"); prompt.AddAction(UIAlertAction.Create("Cancel", UIAlertActionStyle.Cancel, null)); prompt.AddAction(UIAlertAction.Create(ban ? "Ban" : "Kick", UIAlertActionStyle.Destructive, _ => Run(() => model.Client!.KickUserAsync(user.Id, prompt.TextFields?[0].Text ?? "")))); PresentViewController(prompt, true, null); }
|
||||
private async void Run(Func<Task<GenericResult>> command) { try { GenericResult result = await model.RunAdminAsync(_ => command()); if (!result.Ok) throw new InvalidOperationException(result.Message); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
|
||||
}
|
||||
@@ -124,7 +144,16 @@ internal sealed class MoveUserController : FormController
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel"); NavigationItem.RightBarButtonItem = new("Move", UIBarButtonItemStyle.Done, async (_, _) => await Move()); }
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => model.Channels.Count;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath) { Channel channel = model.Channels[indexPath.Row]; UITableViewCell cell = TextCell(tableView, indexPath, "channel", channel.Name); cell.Accessory = channel.Id == selected ? UITableViewCellAccessory.Checkmark : UITableViewCellAccessory.None; cell.AccessibilityLabel = channel.Name + (channel.Id == selected ? ", selected" : ""); return cell; }
|
||||
public override void RowSelected(UITableView tableView, NSIndexPath indexPath) { selected = model.Channels[indexPath.Row].Id; tableView.ReloadData(); }
|
||||
// Only the two checkmarks move. Reloading the whole table would rebuild every accessibility
|
||||
// element and drop VoiceOver's focus on the row that was just chosen.
|
||||
public override void RowSelected(UITableView tableView, NSIndexPath indexPath)
|
||||
{
|
||||
int previous = model.Channels.ToList().FindIndex(channel => channel.Id == selected);
|
||||
selected = model.Channels[indexPath.Row].Id;
|
||||
NSIndexPath[] rows = previous >= 0 && previous != indexPath.Row
|
||||
? [indexPath, NSIndexPath.FromRowSection(previous, 0)] : [indexPath];
|
||||
tableView.ReloadRows(rows, UITableViewRowAnimation.None);
|
||||
}
|
||||
private async Task Move() { try { GenericResult result = await model.RunAdminAsync(client => client.MoveUserAsync(user.Id, selected)); if (!result.Ok) throw new InvalidOperationException(result.Message); NavigationController?.PopViewController(true); } catch (Exception exception) { UiHelpers.ShowError(this, exception); } }
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,9 @@ internal sealed class AppDelegate : UIApplicationDelegate
|
||||
public override bool FinishedLaunching(UIApplication application, NSDictionary? launchOptions)
|
||||
{
|
||||
AppModel.Shared.Load();
|
||||
#if DEBUG
|
||||
SimulatorSmoke.RunIfRequested();
|
||||
#endif
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@@ -34,8 +34,12 @@ internal sealed class AppModel
|
||||
private bool restoreDeafened;
|
||||
private bool backgrounded;
|
||||
private int diagnosticPolls;
|
||||
private bool announcedConnectionLoss;
|
||||
private bool handingOver;
|
||||
|
||||
internal event Action? Changed;
|
||||
// Raised by the 20 Hz level timer only. Subscribers must be cheap and must not reload a list.
|
||||
internal event Action? LevelChanged;
|
||||
internal event Action<ServerIdentityChallenge>? IdentityRequested;
|
||||
internal IReadOnlyList<ServerProfile> Profiles => profiles;
|
||||
internal IReadOnlyList<ChatEntry> Messages => messages;
|
||||
@@ -47,6 +51,10 @@ internal sealed class AppModel
|
||||
internal bool IsConnecting { get; private set; }
|
||||
internal bool VoiceJoined => microphoneStream != 0;
|
||||
internal bool ScreenSharing => broadcast?.IsActive == true;
|
||||
#if DEBUG
|
||||
internal bool ScreenRingMapped => broadcast?.IsMappingOpen == true;
|
||||
internal long ScreenEncodedPackets => broadcast?.EncodedPackets ?? 0;
|
||||
#endif
|
||||
internal bool IsBackgrounded => backgrounded;
|
||||
internal float MicrophoneLevel { get; private set; }
|
||||
internal string Status { get; private set; } = "Not connected";
|
||||
@@ -55,8 +63,24 @@ internal sealed class AppModel
|
||||
internal IReadOnlyList<Channel> Channels => client?.Channels ?? [];
|
||||
internal IReadOnlyList<User> Users => client?.Users ?? [];
|
||||
|
||||
private AppModel() { feedback = new(settings); }
|
||||
internal void Load() { profiles.Clear(); profiles.AddRange(storage.LoadProfiles()); settings.Load(); IosAudioRouter.Shared.Load(); Notify(); }
|
||||
private AppModel() { feedback = new(settings); IosNetworkPathMonitor.Shared.InterfaceChanged += OnInterfaceChanged; }
|
||||
internal void Load() { profiles.Clear(); profiles.AddRange(storage.LoadProfiles()); settings.Load(); IosAudioRouter.Shared.Load(); IosNetworkPathMonitor.Shared.Start(); Notify(); }
|
||||
|
||||
// A changed interface has already stranded the control socket on a source address that is
|
||||
// gone; the media transport rebinds itself, but TCP cannot, and waiting for the keepalive to
|
||||
// notice costs the user ten seconds of a session that is already dead. Fail it now so the
|
||||
// reconnect runs while the audio session is still up. Raised on the monitor queue.
|
||||
private void OnInterfaceChanged(string path)
|
||||
{
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
|
||||
{
|
||||
VoiceCatClient? owner = client;
|
||||
if (owner is null || explicitDisconnect || IsConnecting || owner.State != ClientConnectionState.Connected) return;
|
||||
AddActivity("Network changed; reconnecting.");
|
||||
handingOver = true; reconnectAttempt = 0;
|
||||
owner.DropForReconnect($"The network interface changed to {path}.");
|
||||
});
|
||||
}
|
||||
internal void Save() { storage.SaveProfiles(profiles); settings.Save(); }
|
||||
|
||||
internal void DidEnterBackground()
|
||||
@@ -70,7 +94,7 @@ internal sealed class AppModel
|
||||
internal void WillEnterForeground()
|
||||
{
|
||||
backgrounded = false;
|
||||
IosAudioRouter.Shared.Recover("foreground");
|
||||
IosAudioRouter.Shared.ResumeForeground();
|
||||
}
|
||||
|
||||
internal void DidBecomeActive()
|
||||
@@ -99,7 +123,14 @@ internal sealed class AppModel
|
||||
explicitDisconnect = false; IsConnecting = true; connectedProfile = profile;
|
||||
Status = restoring ? "Reconnecting…" : "Connecting…"; Notify();
|
||||
lifetime?.Cancel(); lifetime?.Dispose(); lifetime = new();
|
||||
VoiceCatClient next = new("VoiceCat-iOS", "0.0.1", storage.TofuPath);
|
||||
// The AVAudioSourceNode render callback drives Audio.RunCycle and the 20 ms capture
|
||||
// handoff; a sleep-paced audio worker stalls when iOS coalesces backgrounded wakeups and
|
||||
// the call glitches after several minutes in the background.
|
||||
VoiceCatClient next = new("VoiceCat-iOS", "0.0.1", storage.TofuPath, deviceClockedAudio: true);
|
||||
// A phone changes path often and the OS reports a blackholed TCP connection late or never.
|
||||
// The path monitor catches a real handover immediately; this is the backstop for the cases
|
||||
// it cannot see, such as a NAT rebinding or an upstream route that quietly stops carrying.
|
||||
next.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(12));
|
||||
next.ConnectionStateChanged += state =>
|
||||
{
|
||||
if (state == ClientConnectionState.Disconnected && next.ConnectionFailure is { } failure)
|
||||
@@ -126,18 +157,29 @@ internal sealed class AppModel
|
||||
_ = PumpEventsAsync(next, lifetime.Token);
|
||||
levelTimer?.Dispose(); levelTimer = new(_ =>
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(PollAudio), null, 50, 50);
|
||||
feedback.Play(SoundEvent.Login); feedback.Speak(restoring ? "Reconnected" : "Connected");
|
||||
// A handover that recovers in under a second should read as a hiccup, not a dropped
|
||||
// call: announce the return only when the loss itself was announced.
|
||||
if (!restoring || announcedConnectionLoss) { feedback.Play(SoundEvent.Login); feedback.Speak(restoring ? "Reconnected" : "Connected"); }
|
||||
announcedConnectionLoss = false; handingOver = false;
|
||||
if (restoring && restoreChannel != 0) await RestoreSessionAsync(next);
|
||||
Notify();
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
System.Diagnostics.Debug.WriteLine($"Connection failed: {exception}");
|
||||
Console.Error.WriteLine($"VoiceCat connection setup failed: {exception}");
|
||||
IsConnecting = false; Status = exception.Message;
|
||||
if (ReferenceEquals(client, next)) { client = null; await StopSessionResourcesAsync(); }
|
||||
if (ReferenceEquals(client, next)) { client = null; await StopSessionResourcesAsync(releaseAudio: !restoring); }
|
||||
await next.DisposeAsync();
|
||||
Notify();
|
||||
if (restoring && !explicitDisconnect) ScheduleReconnect();
|
||||
if (restoring && !explicitDisconnect)
|
||||
{
|
||||
if (!announcedConnectionLoss)
|
||||
{
|
||||
announcedConnectionLoss = true; handingOver = false;
|
||||
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting");
|
||||
}
|
||||
ScheduleReconnect();
|
||||
}
|
||||
else throw;
|
||||
}
|
||||
}
|
||||
@@ -196,11 +238,15 @@ internal sealed class AppModel
|
||||
// Handle that state change directly so a Wi-Fi transition cannot strand this session.
|
||||
CaptureRestoreState(owner);
|
||||
client = null; Status = "Connection lost";
|
||||
try { await StopSessionResourcesAsync(); }
|
||||
try { await StopSessionResourcesAsync(releaseAudio: false); }
|
||||
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio cleanup failed: {exception}"); }
|
||||
try { await owner.DisposeAsync(); }
|
||||
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Connection cleanup failed: {exception}"); }
|
||||
feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting"); Notify();
|
||||
// On a handover the reconnect below usually lands within a second, so stay quiet and let
|
||||
// the first failed attempt be what tells the user. Any other loss is announced at once.
|
||||
if (handingOver) Status = "Reconnecting…";
|
||||
else { announcedConnectionLoss = true; feedback.Play(SoundEvent.ConnectionLost); feedback.Speak("Connection lost, reconnecting"); }
|
||||
Notify();
|
||||
ScheduleReconnect();
|
||||
}
|
||||
|
||||
@@ -228,15 +274,35 @@ internal sealed class AppModel
|
||||
if (microphoneStream != 0)
|
||||
{
|
||||
IosAudioEngine.Shared.StopMicrophone(); active.StopStream(microphoneStream); microphoneStream = 0;
|
||||
await active.SubscribeVoiceAsync(false); MicrophoneLevel = 0; feedback.Play(SoundEvent.VoiceOff); Notify(); return;
|
||||
if (!ScreenSharing) await active.SubscribeVoiceAsync(false);
|
||||
MicrophoneLevel = 0; feedback.Play(SoundEvent.VoiceOff); Notify(); return;
|
||||
}
|
||||
if (AVFoundation.AVCaptureDevice.GetAuthorizationStatus(AVFoundation.AVAuthorizationMediaType.Audio) == AVFoundation.AVAuthorizationStatus.NotDetermined)
|
||||
await AVFoundation.AVCaptureDevice.RequestAccessForMediaTypeAsync(AVFoundation.AVAuthorizationMediaType.Audio);
|
||||
AVFoundation.AVAuthorizationStatus authorization = AVFoundation.AVCaptureDevice.GetAuthorizationStatus(AVFoundation.AVAuthorizationMediaType.Audio);
|
||||
if (authorization == AVFoundation.AVAuthorizationStatus.NotDetermined)
|
||||
authorization = await AVFoundation.AVCaptureDevice.RequestAccessForMediaTypeAsync(AVFoundation.AVAuthorizationMediaType.Audio)
|
||||
? AVFoundation.AVAuthorizationStatus.Authorized : AVFoundation.AVAuthorizationStatus.Denied;
|
||||
if (authorization != AVFoundation.AVAuthorizationStatus.Authorized)
|
||||
throw new InvalidOperationException("Microphone access is required to join voice. Allow it in iOS Settings.");
|
||||
VoiceSubscriptionResult subscribed = await active.SubscribeVoiceAsync();
|
||||
if (!subscribed.Ok) throw new InvalidOperationException(subscribed.Error);
|
||||
uint startedStream = 0;
|
||||
try
|
||||
{
|
||||
int channels = IosAudioRouter.Shared.CaptureChannels;
|
||||
StreamInfo stream = await active.StartStreamAsync(StreamKind.StreamMic, "Microphone", channels);
|
||||
microphoneStream = stream.StreamId; IosAudioEngine.Shared.StartMicrophone(stream.StreamId, channels); feedback.Play(SoundEvent.VoiceOn); Notify();
|
||||
startedStream = stream.StreamId;
|
||||
IosAudioEngine.Shared.StartMicrophone(stream.StreamId, channels);
|
||||
microphoneStream = stream.StreamId; feedback.Play(SoundEvent.VoiceOn); Notify();
|
||||
}
|
||||
catch
|
||||
{
|
||||
IosAudioEngine.Shared.StopMicrophone();
|
||||
if (startedStream != 0)
|
||||
try { active.StopStream(startedStream); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
|
||||
if (!ScreenSharing)
|
||||
try { await active.SubscribeVoiceAsync(false); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
internal void ToggleScreenAudio()
|
||||
@@ -270,7 +336,8 @@ internal sealed class AppModel
|
||||
|
||||
internal async Task DisconnectAsync()
|
||||
{
|
||||
explicitDisconnect = true; lifetime?.Cancel(); await StopSessionResourcesAsync();
|
||||
explicitDisconnect = true; handingOver = false; announcedConnectionLoss = false;
|
||||
lifetime?.Cancel(); await StopSessionResourcesAsync(releaseAudio: true);
|
||||
VoiceCatClient? old = client; client = null; microphoneStream = 0; IsConnecting = false; Status = "Not connected"; Notify();
|
||||
if (old is not null) await old.DisposeAsync(); feedback.Play(SoundEvent.Logout); feedback.Speak("Disconnected");
|
||||
}
|
||||
@@ -278,7 +345,10 @@ internal sealed class AppModel
|
||||
private void ScheduleReconnect()
|
||||
{
|
||||
ServerProfile? profile = connectedProfile; if (profile is null || explicitDisconnect) return;
|
||||
int delay = Math.Min(1 << Math.Min(reconnectAttempt++, 5), 30);
|
||||
// The first attempt runs immediately: after a handover the new path is already up, and a
|
||||
// second of deliberate silence is the difference between a hiccup and a dropped call.
|
||||
int delay = reconnectAttempt == 0 ? 0 : Math.Min(1 << Math.Min(reconnectAttempt, 5), 30);
|
||||
reconnectAttempt++;
|
||||
CancellationToken token = lifetime?.Token ?? default;
|
||||
_ = Task.Run(async () =>
|
||||
{
|
||||
@@ -322,7 +392,10 @@ internal sealed class AppModel
|
||||
catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { return; }
|
||||
lastTalking = talking; feedback.Play(talking ? SoundEvent.VoiceStart : SoundEvent.VoiceStop);
|
||||
}
|
||||
Notify();
|
||||
// Only the voice bar renders the level. Raising the general Changed event at 20 Hz made
|
||||
// every list reload itself that often, which tears down VoiceOver's element tree under an
|
||||
// exploring finger; keep the fast signal on its own event.
|
||||
LevelChanged?.Invoke();
|
||||
}
|
||||
|
||||
private void HandleUserEvent(VoiceCatClient owner, UserEvent value)
|
||||
@@ -341,10 +414,9 @@ internal sealed class AppModel
|
||||
{
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
|
||||
{
|
||||
// Presenting either system picker can replace or interrupt the app's audio session.
|
||||
// Rebuild after the producer becomes active so playback and an existing mic tap are
|
||||
// attached to the session that will remain in use for the broadcast.
|
||||
if (ScreenSharing) IosAudioRouter.Shared.Recover("screen sharing started");
|
||||
// Presenting either picker can change the audio route. Keep a healthy graph; if the
|
||||
// picker stopped it, Recover resumes the same graph or rebuilds for changed hardware.
|
||||
IosAudioRouter.Shared.Recover(ScreenSharing ? "screen sharing started" : "screen sharing stopped", force: false);
|
||||
Notify();
|
||||
});
|
||||
}
|
||||
@@ -364,9 +436,15 @@ internal sealed class AppModel
|
||||
owner.SetSelfAudioState(restoreMuted, restoreDeafened); AddActivity($"Restored to channel {restoreChannel}{(restoringVoice ? " with voice" : "")}");
|
||||
}
|
||||
|
||||
private async Task StopSessionResourcesAsync()
|
||||
// `releaseAudio` distinguishes an ended session from an interrupted one. Ending releases the
|
||||
// AVAudioSession, which on Bluetooth drops the HFP link and costs seconds of renegotiation on
|
||||
// the way back; a lost connection is a transport event that changed nothing about the audio
|
||||
// configuration, so it only unbinds the client and leaves the live route in place.
|
||||
private async Task StopSessionResourcesAsync(bool releaseAudio)
|
||||
{
|
||||
levelTimer?.Dispose(); levelTimer = null; IosAudioEngine.Shared.Stop(); microphoneStream = 0; MicrophoneLevel = 0; lastTalking = false;
|
||||
levelTimer?.Dispose(); levelTimer = null;
|
||||
if (releaseAudio) IosAudioEngine.Shared.Stop(); else IosAudioEngine.Shared.Detach();
|
||||
microphoneStream = 0; MicrophoneLevel = 0; lastTalking = false;
|
||||
if (broadcast is { } pump) { broadcast = null; pump.Changed -= BroadcastChanged; await pump.DisposeAsync(); }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,15 +9,25 @@ internal sealed class BroadcastAudioPump : IAsyncDisposable
|
||||
private const uint Magic = 0x56434252, Version = 1;
|
||||
private const int Header = 64, Capacity = 96_000, Frame = 960;
|
||||
private readonly CancellationTokenSource stop = new();
|
||||
private readonly short[] scratch = new short[Frame * 2];
|
||||
private Thread? worker;
|
||||
private VoiceCatClient? client;
|
||||
private MemoryMappedFile? map;
|
||||
private MemoryMappedViewAccessor? view;
|
||||
private string? path;
|
||||
private uint streamId;
|
||||
private bool active;
|
||||
private int generation;
|
||||
private readonly short[] scratch = new short[Frame * 2];
|
||||
private ulong observedWrite;
|
||||
private ulong idleWrite;
|
||||
private bool idleWriteSeen;
|
||||
|
||||
internal event Action? Changed;
|
||||
internal bool IsActive => active;
|
||||
#if DEBUG
|
||||
internal bool IsMappingOpen => view is not null;
|
||||
internal long EncodedPackets => streamId == 0 ? 0 : client?.Audio.GetLocalDiagnostics(streamId).EncodedPackets ?? 0;
|
||||
#endif
|
||||
|
||||
internal void Start(VoiceCatClient owner)
|
||||
{
|
||||
@@ -29,55 +39,140 @@ internal sealed class BroadcastAudioPump : IAsyncDisposable
|
||||
private void Run()
|
||||
{
|
||||
CancellationToken token = stop.Token;
|
||||
try
|
||||
{
|
||||
while (!token.IsCancellationRequested)
|
||||
{
|
||||
try { DrainAsync(token).GetAwaiter().GetResult(); }
|
||||
catch (Exception exception) when (exception is IOException or UnauthorizedAccessException or InvalidDataException or OperationCanceledException) { }
|
||||
if (!token.IsCancellationRequested) Thread.Sleep(5);
|
||||
catch (OperationCanceledException) when (token.IsCancellationRequested) { break; }
|
||||
catch (Exception exception)
|
||||
{
|
||||
// A rejected screen stream is a session error, not an unhandled exception on
|
||||
// this background Thread. Keep the app and microphone call alive.
|
||||
Console.Error.WriteLine($"VoiceCat screen audio pump failed: {exception}");
|
||||
CloseMapping(); StopStream(); SetActive(false);
|
||||
if (!token.IsCancellationRequested) Thread.Sleep(1000);
|
||||
}
|
||||
// The shared file must not remain mapped while the app is idle or suspending.
|
||||
if (!token.IsCancellationRequested) Thread.Sleep(view is null ? 100 : 5);
|
||||
}
|
||||
}
|
||||
finally { CloseMapping(); }
|
||||
}
|
||||
|
||||
private async Task DrainAsync(CancellationToken token)
|
||||
{
|
||||
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
|
||||
if (root?.Path is null) return;
|
||||
string path = Path.Combine(root.Path, "voicecat", "broadcast_audio.ring"); if (!File.Exists(path)) return;
|
||||
using MemoryMappedFile map = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
|
||||
using MemoryMappedViewAccessor view = map.CreateViewAccessor(0, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
|
||||
if (view.ReadUInt32(0) != Magic || view.ReadUInt32(4) != Version) throw new InvalidDataException("Unsupported broadcast ring.");
|
||||
bool active = view.ReadUInt32(16) != 0;
|
||||
if (!active) { StopStream(); SetActive(false); return; }
|
||||
if (!OpenMapping()) return;
|
||||
MemoryMappedViewAccessor pages = view!;
|
||||
bool active = pages.ReadUInt32(16) != 0;
|
||||
if (!active) { StopStream(); SetActive(false); CloseMapping(); return; }
|
||||
VoiceCatClient owner = client ?? throw new IOException("Client disconnected.");
|
||||
if (streamId == 0)
|
||||
{
|
||||
// A ring left marked active by a killed extension must not start a phantom stream.
|
||||
// Wait for a fresh producer write after this connection opened the mapping.
|
||||
ulong currentWrite = pages.ReadUInt64(24);
|
||||
if (currentWrite == observedWrite) return;
|
||||
observedWrite = currentWrite;
|
||||
int startGeneration = Volatile.Read(ref generation);
|
||||
VoiceSubscriptionResult subscription = await owner.SubscribeVoiceAsync(cancellationToken: token).ConfigureAwait(false);
|
||||
if (!subscription.Ok) throw new InvalidOperationException(subscription.Error);
|
||||
StreamInfo stream = await owner.StartStreamAsync(StreamKind.StreamScreenAudio, "Screen audio", 2, token).ConfigureAwait(false);
|
||||
if (startGeneration != Volatile.Read(ref generation) || view.ReadUInt32(16) == 0 || !ReferenceEquals(client, owner))
|
||||
if (startGeneration != Volatile.Read(ref generation) || pages.ReadUInt32(16) == 0 || !ReferenceEquals(client, owner))
|
||||
{
|
||||
try { owner.StopStream(stream.StreamId); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
|
||||
return;
|
||||
}
|
||||
streamId = stream.StreamId; view.Write(32, view.ReadUInt64(24)); SetActive(true);
|
||||
// Keep the newest 120 ms captured during stream negotiation. This covers the
|
||||
// largest codec frame plus the input buffer target; dropping through the latest
|
||||
// write loses short sounds before the first drain can feed them.
|
||||
ulong latestWrite = pages.ReadUInt64(24);
|
||||
ulong earliest = latestWrite > (ulong)(Frame * 12) ? latestWrite - (ulong)(Frame * 12) : 0;
|
||||
pages.Write(32, Math.Max(pages.ReadUInt64(32), earliest));
|
||||
streamId = stream.StreamId; SetActive(true);
|
||||
}
|
||||
ulong write = view.ReadUInt64(24), read = view.ReadUInt64(32);
|
||||
ulong write = pages.ReadUInt64(24), read = pages.ReadUInt64(32);
|
||||
if (write - read > Capacity) read = write - Capacity;
|
||||
while (write - read >= Frame * 2)
|
||||
{
|
||||
for (int sample = 0; sample < scratch.Length; sample++)
|
||||
{
|
||||
ulong index = (read + (ulong)sample) % Capacity;
|
||||
scratch[sample] = view.ReadInt16(Header + checked((long)index * sizeof(short)));
|
||||
scratch[sample] = pages.ReadInt16(Header + checked((long)index * sizeof(short)));
|
||||
}
|
||||
if (!owner.Audio.FeedPcm(streamId, scratch, 2)) break;
|
||||
read += (ulong)scratch.Length;
|
||||
view.Write(32, read);
|
||||
pages.Write(32, read);
|
||||
}
|
||||
}
|
||||
|
||||
// Keep a mapping only while a broadcast is active. An inactive mapping holds an open handle
|
||||
// into the shared App Group container through suspension, which iOS may terminate as a shared
|
||||
// file lock (0xdead10cc). Poll the small header with a short-lived read when idle, then map
|
||||
// once for the active broadcast's high-rate drain.
|
||||
private bool OpenMapping()
|
||||
{
|
||||
path ??= ResolvePath();
|
||||
if (path is null) return false;
|
||||
bool present = File.Exists(path);
|
||||
if (view is not null)
|
||||
{
|
||||
if (present) return true;
|
||||
CloseMapping(); return false;
|
||||
}
|
||||
if (!present) { idleWriteSeen = false; return false; }
|
||||
using (var file = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete))
|
||||
{
|
||||
if (file.Length < Header) { idleWriteSeen = false; return false; }
|
||||
Span<byte> header = stackalloc byte[32];
|
||||
file.ReadExactly(header);
|
||||
if (System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header) != Magic ||
|
||||
System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header[4..]) != Version ||
|
||||
System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(header[16..]) == 0)
|
||||
{
|
||||
idleWriteSeen = false;
|
||||
return false;
|
||||
}
|
||||
ulong currentWrite = System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(header[24..]);
|
||||
if (!idleWriteSeen)
|
||||
{
|
||||
idleWrite = currentWrite;
|
||||
idleWriteSeen = true;
|
||||
return false;
|
||||
}
|
||||
if (currentWrite == idleWrite) return false;
|
||||
}
|
||||
MemoryMappedFile candidate = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
|
||||
try
|
||||
{
|
||||
MemoryMappedViewAccessor pages = candidate.CreateViewAccessor(0, Header + Capacity * sizeof(short), MemoryMappedFileAccess.ReadWrite);
|
||||
if (pages.ReadUInt32(0) != Magic || pages.ReadUInt32(4) != Version)
|
||||
{
|
||||
pages.Dispose(); throw new InvalidDataException("Unsupported broadcast ring.");
|
||||
}
|
||||
map = candidate; view = pages; observedWrite = idleWrite; return true;
|
||||
}
|
||||
catch { candidate.Dispose(); throw; }
|
||||
}
|
||||
|
||||
private static string? ResolvePath()
|
||||
{
|
||||
NSUrl? root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup);
|
||||
return root?.Path is null ? null : Path.Combine(root.Path, "voicecat", "broadcast_audio.ring");
|
||||
}
|
||||
|
||||
private void CloseMapping()
|
||||
{
|
||||
view?.Dispose(); view = null;
|
||||
map?.Dispose(); map = null;
|
||||
observedWrite = 0;
|
||||
idleWriteSeen = false;
|
||||
}
|
||||
|
||||
private void StopStream()
|
||||
{
|
||||
uint id = streamId; streamId = 0; if (id == 0 || client?.State != ClientConnectionState.Connected) return;
|
||||
try { client.StopStream(id); } catch (Exception exception) when (exception is IOException or InvalidOperationException) { }
|
||||
try { client.StopStream(id); } catch (Exception exception) when (exception is IOException or InvalidOperationException or ObjectDisposedException) { }
|
||||
}
|
||||
|
||||
internal void RequestStop() { Interlocked.Increment(ref generation); SetActive(false); }
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
<plist version="1.0"><dict>
|
||||
<key>CFBundleDisplayName</key><string>VoiceCat</string>
|
||||
<key>CFBundleIdentifier</key><string>me.iamtalon.voicecat</string>
|
||||
<key>XSAppIconAssets</key><string>Assets.xcassets/AppIcon.appiconset</string>
|
||||
<key>LSRequiresIPhoneOS</key><true/>
|
||||
<key>NSMicrophoneUsageDescription</key><string>VoiceCat needs microphone access to transmit your voice in channels.</string>
|
||||
<key>UIBackgroundModes</key><array><string>audio</string><string>screen-capture</string></array>
|
||||
|
||||
@@ -31,7 +31,7 @@ internal sealed class IosAudioEngine
|
||||
}
|
||||
private MicrophoneRoute? microphone;
|
||||
private readonly short[] microphoneFrame = new short[960 * 2];
|
||||
private readonly Thread microphonePump;
|
||||
private int microphoneCredit;
|
||||
private AVAudioFormat? microphoneFormat;
|
||||
private AVAudioConverter? microphoneConverter;
|
||||
private AVAudioPcmBuffer? convertedMicrophone;
|
||||
@@ -39,41 +39,112 @@ internal sealed class IosAudioEngine
|
||||
private AVAudioConverterInputHandler? inputProvider;
|
||||
private bool inputProvided;
|
||||
private bool tapInstalled;
|
||||
private AVAudioMixerNode? captureSink;
|
||||
// The voice-processing state the live graph was actually built with, so Reconfigure can tell
|
||||
// a settings change apart from a re-check of an unchanged graph.
|
||||
private bool voiceProcessing;
|
||||
// The input format the tap, the converter and the ring capacity were all built for. Asked of
|
||||
// the input node rather than of AVAudioSession: the session's reported rate and channel count
|
||||
// do not settle until after the graph has started, so comparing against them reports a change
|
||||
// that has not happened and every rebuild reports it again.
|
||||
private long lastRebuildAt;
|
||||
// How long a freshly built graph is given to produce its first render callback. Enabling voice
|
||||
// processing rebuilds both halves of the IO, which takes several hundred milliseconds, posts a
|
||||
// configuration change and reports the engine as not running while it happens. Everything that
|
||||
// judges a graph dead has to wait this out, or it tears down the graph it just built and the
|
||||
// replacement reports exactly the same thing.
|
||||
private const int SettlingMilliseconds = 2_000;
|
||||
private double builtInputRate;
|
||||
private uint builtInputChannels;
|
||||
// The capture width the tap and converter were built for. The input side of the graph exists
|
||||
// for the whole session, so this is what decides whether joining voice needs a rebuild at all.
|
||||
private int builtCaptureChannels;
|
||||
private long captureCallbacks, capturedFrames, convertedFrames, rejectedFeeds, converterFailures, stereoFrames, stereoDifferentFrames;
|
||||
private long renderCallbacks, lastRenderTimestamp;
|
||||
internal bool IsConnected { get; private set; }
|
||||
internal int BufferMilliseconds { get => playbackRing.BufferMilliseconds; set => playbackRing.BufferMilliseconds = value; }
|
||||
|
||||
private IosAudioEngine()
|
||||
internal bool IsRunning => engine?.Running == true;
|
||||
// True while the last rebuild is still starting up, and so cannot be judged.
|
||||
internal bool Settling => Environment.TickCount64 - Volatile.Read(ref lastRebuildAt) < SettlingMilliseconds;
|
||||
// True when the graph's own input no longer has the format its tap and converter were built
|
||||
// for, which is the only thing a route change can do that AVAudioEngine cannot absorb on its
|
||||
// own. The engine's view is the one that is stable: it changes when the hardware actually
|
||||
// changes, which is precisely what a configuration change reports.
|
||||
internal bool HardwareChanged()
|
||||
{
|
||||
microphonePump = new Thread(PumpMicrophone) { IsBackground = true, Name = "VoiceCat iOS microphone pacer", Priority = ThreadPriority.Highest };
|
||||
microphonePump.Start();
|
||||
if (engine is not { } live || !tapInstalled) return false;
|
||||
AVAudioFormat format = live.InputNode.GetBusOutputFormat(0);
|
||||
return format.SampleRate != builtInputRate || format.ChannelCount != builtInputChannels;
|
||||
}
|
||||
// The watchdog compares this across ticks: a graph that stops calling back while it still
|
||||
// reports Running leaves the whole device-clocked pipeline frozen until it is rebuilt.
|
||||
internal long RenderCallbacks => Interlocked.Read(ref renderCallbacks);
|
||||
internal int BufferMilliseconds { get => playbackRing.BufferMilliseconds; set => playbackRing.BufferMilliseconds = value; }
|
||||
|
||||
internal void StartListening(VoiceCatClient owner)
|
||||
{
|
||||
Stop(); client = owner; IsConnected = true; owner.Audio.MixedPcm += ReceiveMixedPcm; Rebuild();
|
||||
if (client is { } previous) previous.Audio.MixedPcm -= ReceiveMixedPcm;
|
||||
client = owner; IsConnected = true; owner.Audio.MixedPcm += ReceiveMixedPcm;
|
||||
// A reconnect after Detach finds the session active and the graph already running on the
|
||||
// right hardware. Rebuilding it there would release an HFP headset and pay a Bluetooth
|
||||
// profile renegotiation for a transport blip that changed no audio configuration.
|
||||
if (engine?.Running == true) { playbackRing.Resynchronize(); return; }
|
||||
if (ResumeStoppedGraph()) return;
|
||||
Volatile.Write(ref microphone, null); Rebuild();
|
||||
}
|
||||
|
||||
// Unbinds the client without touching the session or the graph, for a connection that was
|
||||
// lost rather than ended. Capture keeps feeding a ring nobody drains and Render emits silence
|
||||
// until StartListening rebinds, which keeps the route and its Bluetooth profile alive.
|
||||
internal void Detach()
|
||||
{
|
||||
if (client is { } owner) owner.Audio.MixedPcm -= ReceiveMixedPcm;
|
||||
client = null; playbackRing.Resynchronize();
|
||||
// The route's stream id belongs to the connection that just died. Keep the tap and its
|
||||
// hardware, but park the route on the unbound id so a rebind cannot feed the next
|
||||
// connection a stream it never announced.
|
||||
if (Volatile.Read(ref microphone) is { } stale && stale.StreamId != 0)
|
||||
Volatile.Write(ref microphone, CreateMicrophoneRoute(0, stale.Channels));
|
||||
}
|
||||
|
||||
internal void StartMicrophone(uint streamId, int channels)
|
||||
{
|
||||
Volatile.Write(ref microphone, CreateMicrophoneRoute(streamId, channels)); Rebuild();
|
||||
// Capture is already running: the graph carries the tap for the whole session. Joining voice
|
||||
// only names the stream the frames belong to, so it is a state change and not a rebuild. The
|
||||
// width is the one exception, because the tap format and converter are built around it.
|
||||
MicrophoneRoute next = CreateMicrophoneRoute(streamId, channels);
|
||||
bool reusable = tapInstalled && engine?.Running == true && builtCaptureChannels == next.Channels;
|
||||
Volatile.Write(ref microphone, next);
|
||||
if (!reusable) Rebuild();
|
||||
}
|
||||
|
||||
internal void StopMicrophone() { Volatile.Write(ref microphone, null); Rebuild(); }
|
||||
internal void Reconfigure()
|
||||
// Leaving voice never rebuilds. Capture keeps running into a route nobody reads, exactly as it
|
||||
// does between connecting and joining, which costs one discarded conversion per callback and
|
||||
// saves tearing down the voice-processing IO only to build it again on the next join.
|
||||
internal void StopMicrophone() { Volatile.Write(ref microphone, null); }
|
||||
|
||||
// `force` rebuilds unconditionally, which is what a route change, a media-services reset and
|
||||
// the stall watchdog all need. Callers that are only re-checking a graph they expect to be
|
||||
// healthy — foregrounding, above all — pass false and get a no-op when nothing has changed.
|
||||
internal void Reconfigure(bool force = true, string cause = "reconfigure")
|
||||
{
|
||||
if (!IsConnected) return;
|
||||
// A graph that has not finished starting is not a graph to replace. Only an explicit
|
||||
// configuration change forces its way past this.
|
||||
if (!force && Settling) return;
|
||||
MicrophoneRoute? current = Volatile.Read(ref microphone);
|
||||
int channels = IosAudioRouter.Shared.CaptureChannels;
|
||||
if (!force && engine?.Running == true && tapInstalled && builtCaptureChannels == channels
|
||||
&& voiceProcessing == IosAudioRouter.Shared.UsesVoiceProcessing && !HardwareChanged()) return;
|
||||
if (!force && ResumeStoppedGraph()) return;
|
||||
if (current is not null)
|
||||
{
|
||||
// Stop the old tap before publishing a route with a different sample width. Otherwise
|
||||
// an in-flight callback could interpret its old converter buffer using the new width.
|
||||
DestroyGraph();
|
||||
if (current.Channels != channels) client!.Audio.SetCaptureChannels(current.StreamId, channels);
|
||||
if (current.Channels != channels && current.StreamId != 0) client?.Audio.SetCaptureChannels(current.StreamId, channels);
|
||||
Volatile.Write(ref microphone, CreateMicrophoneRoute(current.StreamId, channels));
|
||||
}
|
||||
Rebuild();
|
||||
Rebuild(cause);
|
||||
}
|
||||
|
||||
private static MicrophoneRoute CreateMicrophoneRoute(uint streamId, int channels)
|
||||
@@ -83,21 +154,58 @@ internal sealed class IosAudioEngine
|
||||
internal bool EnsureRunning()
|
||||
{
|
||||
if (!IsConnected || engine?.Running == true) return true;
|
||||
// Still starting: report it as running rather than replacing it, since that is what it is
|
||||
// about to be, and a rebuild here would start the cycle over.
|
||||
if (Settling) return true;
|
||||
if (ResumeStoppedGraph()) return true;
|
||||
Rebuild(); return engine?.Running == true;
|
||||
}
|
||||
|
||||
private void Rebuild()
|
||||
private bool ResumeStoppedGraph()
|
||||
{
|
||||
DestroyGraph(); MicrophoneRoute? route = Volatile.Read(ref microphone); IosAudioRouter.Shared.Apply(route is not null);
|
||||
if (engine is not { } paused || !tapInstalled ||
|
||||
builtCaptureChannels != IosAudioRouter.Shared.CaptureChannels ||
|
||||
voiceProcessing != IosAudioRouter.Shared.UsesVoiceProcessing || HardwareChanged()) return false;
|
||||
IosAudioRouter.Shared.Apply(true);
|
||||
bool started = paused.StartAndReturnError(out NSError? error);
|
||||
if (started)
|
||||
{
|
||||
playbackRing.Resynchronize();
|
||||
// Voice-processing IO may still be starting when Start returns. Give the existing
|
||||
// graph its settling window before a watchdog judges it, just as a fresh graph gets.
|
||||
Volatile.Write(ref lastRebuildAt, Environment.TickCount64);
|
||||
Console.Error.WriteLine($"VC_RESUME running={paused.Running}");
|
||||
return true;
|
||||
}
|
||||
Console.Error.WriteLine($"VC_RESUME running={paused.Running} error={error?.LocalizedDescription ?? "none"}");
|
||||
return false;
|
||||
}
|
||||
|
||||
private void Rebuild([System.Runtime.CompilerServices.CallerMemberName] string cause = "")
|
||||
{
|
||||
Console.Error.WriteLine($"VC_REBUILD cause={cause} running={engine?.Running == true} callbacks={RenderCallbacks}");
|
||||
DestroyGraph(); MicrophoneRoute? route = Volatile.Read(ref microphone);
|
||||
// The input node, voice processing and the tap are built once and kept for the session, not
|
||||
// added when voice is joined. Adding them later means replacing a running graph that has no
|
||||
// voice processing with one that has it, and that transition is what fails: the new IO unit
|
||||
// starts and is torn down again within a second, non-deterministically, which is the
|
||||
// rebuild storm the watchdog then chases. Steady-state voice processing is reliable; only
|
||||
// the change into it is not. So capture always runs, and joining voice only decides which
|
||||
// stream its frames belong to — Capture and PumpMicrophoneChunk both discard without one.
|
||||
bool captures = IsConnected;
|
||||
int captureChannels = route?.Channels ?? Math.Clamp(IosAudioRouter.Shared.CaptureChannels, 1, 2);
|
||||
IosAudioRouter.Shared.Apply(captures);
|
||||
voiceProcessing = IosAudioRouter.Shared.UsesVoiceProcessing;
|
||||
builtInputRate = 0; builtInputChannels = 0;
|
||||
var next = new AVAudioEngine();
|
||||
AVAudioInputNode? input = null;
|
||||
if (route is not null)
|
||||
if (captures)
|
||||
{
|
||||
// Enabling voice processing rebuilds both sides of AVAudioEngine. Do it before any
|
||||
// formats are queried or nodes are connected so the graph is built from the final IO.
|
||||
input = next.InputNode;
|
||||
if (!input.SetVoiceProcessingEnabled(IosAudioRouter.Shared.UsesVoiceProcessing, out NSError? processingError))
|
||||
throw new InvalidOperationException(processingError.LocalizedDescription);
|
||||
throw new InvalidOperationException(processingError?.LocalizedDescription ?? "Could not configure voice processing.");
|
||||
if (IosAudioRouter.Shared.UsesVoiceProcessing) input.VoiceProcessingAgcEnabled = IosAudioRouter.Shared.AutomaticGainControl;
|
||||
}
|
||||
outputFormat = new(AVAudioCommonFormat.PCMFloat32, 48_000, 2, false);
|
||||
@@ -107,12 +215,16 @@ internal sealed class IosAudioEngine
|
||||
if (OperatingSystem.IsIOSVersionAtLeast(27)) next.Connect(source, next.MainMixerNode, outputFormat, out connectionError);
|
||||
else next.Connect(source, next.MainMixerNode, outputFormat);
|
||||
if (connectionError is not null) throw new InvalidOperationException(connectionError.LocalizedDescription);
|
||||
if (route is not null)
|
||||
if (captures)
|
||||
{
|
||||
AVAudioFormat inputFormat = input!.GetBusOutputFormat(0);
|
||||
Console.Error.WriteLine($"VC_GRAPH vpio={IosAudioRouter.Shared.UsesVoiceProcessing} requestedCh={route.Channels} " +
|
||||
if (inputFormat.SampleRate <= 0 || inputFormat.ChannelCount == 0)
|
||||
throw new InvalidOperationException("No usable microphone input is available on the current audio route.");
|
||||
Console.Error.WriteLine($"VC_GRAPH vpio={IosAudioRouter.Shared.UsesVoiceProcessing} requestedCh={captureChannels} " +
|
||||
$"inputCh={inputFormat.ChannelCount} inputRate={inputFormat.SampleRate}");
|
||||
microphoneFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, (uint)route.Channels, true);
|
||||
builtInputRate = inputFormat.SampleRate; builtInputChannels = inputFormat.ChannelCount;
|
||||
microphoneFormat = new(AVAudioCommonFormat.PCMInt16, 48_000, (uint)captureChannels, true);
|
||||
builtCaptureChannels = captureChannels;
|
||||
microphoneConverter = new(inputFormat, microphoneFormat);
|
||||
uint capacity = checked((uint)Math.Ceiling(MaximumCaptureCallbackFrames * 48_000 / inputFormat.SampleRate) + 64);
|
||||
convertedMicrophone = new(microphoneFormat, capacity);
|
||||
@@ -122,17 +234,64 @@ internal sealed class IosAudioEngine
|
||||
else input.InstallTapOnBus(0, 960, inputFormat, Capture);
|
||||
if (tapError is not null) throw new InvalidOperationException(tapError.LocalizedDescription);
|
||||
tapInstalled = true;
|
||||
// With voice processing the input and output run as one IO unit, and the unit only stays
|
||||
// up while the input is part of the render chain. A tap alone does not put it there: the
|
||||
// engine starts and the IO is torn down again within a second, which is why a graph with
|
||||
// voice processing came up perhaps one time in four. Route the input through a silent
|
||||
// mixer so it is genuinely rendered, contributing nothing audible.
|
||||
captureSink = new AVAudioMixerNode();
|
||||
next.AttachNode(captureSink);
|
||||
NSError? sinkError = null;
|
||||
if (OperatingSystem.IsIOSVersionAtLeast(27))
|
||||
{
|
||||
next.Connect(input, captureSink, inputFormat, out sinkError);
|
||||
if (sinkError is null) next.Connect(captureSink, next.MainMixerNode, outputFormat, out sinkError);
|
||||
}
|
||||
else
|
||||
{
|
||||
next.Connect(input, captureSink, inputFormat);
|
||||
next.Connect(captureSink, next.MainMixerNode, outputFormat);
|
||||
}
|
||||
if (sinkError is not null) throw new InvalidOperationException(sinkError.LocalizedDescription);
|
||||
captureSink.OutputVolume = 0f;
|
||||
}
|
||||
next.Prepare();
|
||||
if (!next.StartAndReturnError(out NSError? error)) { next.Dispose(); throw new InvalidOperationException(error.LocalizedDescription); }
|
||||
if (!next.StartAndReturnError(out NSError? error)) { next.Dispose(); throw new InvalidOperationException(error?.LocalizedDescription ?? "The iOS audio graph could not start."); }
|
||||
engine = next;
|
||||
{
|
||||
AVAudioSession live = AVAudioSession.SharedInstance();
|
||||
Console.Error.WriteLine($"VC_START running={next.Running} builtCh={builtInputChannels} sessionRate={live.SampleRate} " +
|
||||
$"sessionInCh={live.InputNumberOfChannels} sessionOutCh={live.OutputNumberOfChannels} inputAvailable={live.InputAvailable} " +
|
||||
$"other={live.OtherAudioPlaying} mode={live.Mode} options={live.CategoryOptions} io={live.IOBufferDuration:F4} " +
|
||||
$"out={string.Join(',', live.CurrentRoute?.Outputs?.Select(value => value.PortType.ToString()) ?? [])} " +
|
||||
$"in={string.Join(',', live.CurrentRoute?.Inputs?.Select(value => value.PortType.ToString()) ?? [])}");
|
||||
// A graph that starts and then stops on its own is the failure that matters, and it is
|
||||
// invisible at start: check again once the IO has had time to come up.
|
||||
AVAudioEngine started = next;
|
||||
System.Threading.Tasks.Task.Delay(750).ContinueWith(_ =>
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
|
||||
{
|
||||
if (!ReferenceEquals(engine, started)) return;
|
||||
Console.Error.WriteLine($"VC_START_CHECK running={started.Running} render={RenderCallbacks} capture={Interlocked.Read(ref captureCallbacks)}");
|
||||
}));
|
||||
}
|
||||
Volatile.Write(ref lastRebuildAt, Environment.TickCount64);
|
||||
engineConfigurationObserver = Foundation.NSNotificationCenter.DefaultCenter.AddObserver(
|
||||
AVAudioEngine.ConfigurationChangeNotification, notification =>
|
||||
{
|
||||
if (!ReferenceEquals(notification.Object, next)) return;
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
|
||||
{
|
||||
if (IsConnected && ReferenceEquals(engine, next) && !next.Running) Rebuild();
|
||||
if (!IsConnected || !ReferenceEquals(engine, next)) return;
|
||||
// Building this graph is itself what posted most of these: enabling voice
|
||||
// processing rebuilds the IO, and the engine reads as stopped until that
|
||||
// finishes. Rebuilding then replaces a graph that was about to run with one
|
||||
// that reports the same thing, without end.
|
||||
if (Settling) return;
|
||||
// A configuration change with the graph still running is usually one
|
||||
// AVAudioEngine has already absorbed; it matters here only when the hardware
|
||||
// the tap and converter were built around moved.
|
||||
if (!next.Running || HardwareChanged()) Rebuild("configuration change");
|
||||
});
|
||||
}, next);
|
||||
}
|
||||
@@ -183,33 +342,21 @@ internal sealed class IosAudioEngine
|
||||
status = AVAudioConverterInputStatus.NoDataNow; return null!;
|
||||
}
|
||||
|
||||
private void PumpMicrophone()
|
||||
{
|
||||
long deadline = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
while (true)
|
||||
// One paced 20 ms handoff from the capture ring into the encoder. The render callback calls
|
||||
// this at its demand rate: the ring absorbs RemoteIO's burst pattern and the small capture vs
|
||||
// output clock difference, so the sender sees a steady 20 ms feed without a sleep-paced pacer
|
||||
// thread to stall when iOS coalesces a backgrounded app's wakeups.
|
||||
private void PumpMicrophoneChunk(VoiceCatClient owner)
|
||||
{
|
||||
MicrophoneRoute? route = Volatile.Read(ref microphone);
|
||||
VoiceCatClient? owner = client;
|
||||
if (route is not null && owner is not null)
|
||||
{
|
||||
// Stream id 0 is a route parked by Detach: still capturing, not yet bound to a connection.
|
||||
if (route is null || route.StreamId == 0) return;
|
||||
int required = 960 * route.Channels;
|
||||
if (route.Ring.Read(microphoneFrame.AsSpan(0, required)) == required &&
|
||||
ReferenceEquals(route, Volatile.Read(ref microphone)) &&
|
||||
!owner.Audio.FeedPcm(route.StreamId, microphoneFrame.AsSpan(0, required), route.Channels))
|
||||
Interlocked.Increment(ref rejectedFeeds);
|
||||
}
|
||||
deadline += System.Diagnostics.Stopwatch.Frequency / 50;
|
||||
while (true)
|
||||
{
|
||||
long remaining = deadline - System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
if (remaining <= 0) break;
|
||||
double milliseconds = remaining * 1000.0 / System.Diagnostics.Stopwatch.Frequency;
|
||||
if (milliseconds > 2) Thread.Sleep(Math.Max(1, (int)milliseconds - 1)); else Thread.SpinWait(64);
|
||||
}
|
||||
if ((deadline - System.Diagnostics.Stopwatch.GetTimestamp()) * 1000.0 / System.Diagnostics.Stopwatch.Frequency < -100)
|
||||
deadline = System.Diagnostics.Stopwatch.GetTimestamp();
|
||||
}
|
||||
}
|
||||
|
||||
private void ReceiveMixedPcm(ReadOnlySpan<short> pcm) => playbackRing.TryWrite(pcm);
|
||||
|
||||
@@ -217,6 +364,29 @@ internal sealed class IosAudioEngine
|
||||
{
|
||||
int frames = checked((int)frameCount), requested = checked(frames * 2);
|
||||
if (requested > renderScratch.Length) return -1;
|
||||
// This callback is the cadence iOS keeps exact while the app is backgrounded or the device
|
||||
// is locked, so it owns both managed 20 ms hands-offs: capture into the sender and one mix
|
||||
// cycle per 20 ms of render demand. Both run allocation-free and without locks or I/O.
|
||||
Interlocked.Increment(ref renderCallbacks);
|
||||
long now = System.Diagnostics.Stopwatch.GetTimestamp(), previous = lastRenderTimestamp;
|
||||
lastRenderTimestamp = now;
|
||||
VoiceCatClient? owner = Volatile.Read(ref client);
|
||||
if (owner is null || !IsConnected) microphoneCredit = 0;
|
||||
else
|
||||
{
|
||||
if (previous != 0 && (now - previous) * 1000.0 / System.Diagnostics.Stopwatch.Frequency > 100)
|
||||
{
|
||||
Volatile.Read(ref microphone)?.Ring.Resynchronize();
|
||||
playbackRing.Resynchronize(); owner.Audio.ResynchronizeInputs(); microphoneCredit = 0;
|
||||
}
|
||||
microphoneCredit += frames;
|
||||
while (microphoneCredit >= 960) { microphoneCredit -= 960; PumpMicrophoneChunk(owner); }
|
||||
// Top the mix ring up to cover this callback plus its configured target so the read
|
||||
// below never starves and the buffer keeps its chosen buffering latency. Catch-up is
|
||||
// capped at one extra cycle so a refill cannot overrun this callback's deadline.
|
||||
int deficit = Math.Min(frames + playbackRing.TargetFrames - playbackRing.CountFrames, frames + 960);
|
||||
for (int produced = 0; produced < deficit; produced += 960) owner.Audio.RunCycle();
|
||||
}
|
||||
Span<short> input = renderScratch.AsSpan(0, requested);
|
||||
int read = playbackRing.Read(input); input[read..].Clear();
|
||||
int count = Marshal.ReadInt32(outputData), first = IntPtr.Size == 8 ? 8 : 4, stride = IntPtr.Size == 8 ? 16 : 12;
|
||||
@@ -235,7 +405,7 @@ internal sealed class IosAudioEngine
|
||||
{
|
||||
IsConnected = false; Volatile.Write(ref microphone, null);
|
||||
if (client is { } owner) owner.Audio.MixedPcm -= ReceiveMixedPcm;
|
||||
DestroyGraph(); client = null; IosAudioRouter.Shared.Deactivate();
|
||||
engine?.Pause(); client = null; IosAudioRouter.Shared.Deactivate();
|
||||
}
|
||||
|
||||
private void DestroyGraph()
|
||||
@@ -248,9 +418,11 @@ internal sealed class IosAudioEngine
|
||||
if (engine is { } old)
|
||||
{
|
||||
if (tapInstalled) old.InputNode.RemoveTapOnBus(0);
|
||||
old.Stop(); if (source is not null) old.DetachNode(source); old.Dispose();
|
||||
old.Stop(); if (source is not null) old.DetachNode(source);
|
||||
if (captureSink is not null) { old.DetachNode(captureSink); captureSink.Dispose(); captureSink = null; }
|
||||
old.Dispose();
|
||||
}
|
||||
tapInstalled = false; pendingInput = null; inputProvider = null;
|
||||
tapInstalled = false; pendingInput = null; inputProvider = null; lastRenderTimestamp = 0; microphoneCredit = 0;
|
||||
convertedMicrophone?.Dispose(); convertedMicrophone = null;
|
||||
microphoneConverter?.Dispose(); microphoneConverter = null;
|
||||
microphoneFormat?.Dispose(); microphoneFormat = null;
|
||||
|
||||
@@ -18,6 +18,27 @@ internal sealed class IosAudioRouter
|
||||
internal static IosAudioRouter Shared { get; } = new();
|
||||
private readonly NSUserDefaults defaults = NSUserDefaults.StandardUserDefaults;
|
||||
private bool applying;
|
||||
private bool speakerIsExplicit;
|
||||
// Whether this session actually has a stereo capsule configuration to undo. Clearing one that
|
||||
// was never applied is not free: SetPreferredPolarPattern(Unknown) drops the built-in array out
|
||||
// of the beamformed mono configuration VoiceChat mode selects and exposes its four raw
|
||||
// channels, which the voice-processing IO cannot start against.
|
||||
private bool stereoApplied;
|
||||
// Set by an explicit speaker choice and consumed by the next Apply. The override is a one-shot
|
||||
// request, never steady-state configuration; see SetForceSpeaker.
|
||||
private bool overridePending;
|
||||
private System.Threading.Timer? watchdog;
|
||||
private long lastRenderCallbacks = -1;
|
||||
private int watchdogMisses;
|
||||
private int watchdogAttempts;
|
||||
private long nextWatchdogAttempt;
|
||||
// A rebuild that does not restore the callbacks is not worth repeating at the same rate, and
|
||||
// never worth repeating forever: an unbounded retry turns a graph that cannot start into a
|
||||
// storm of session reconfiguration, which is both what the user hears and what hides the
|
||||
// reason from the log. Back off, then stop and say so.
|
||||
private const int MaximumWatchdogAttempts = 4;
|
||||
private bool watchdogTicking;
|
||||
private bool interrupted;
|
||||
internal event Action? Changed;
|
||||
internal IosAudioPreset Preset { get; private set; } = IosAudioPreset.VoiceChat;
|
||||
internal IosBluetoothMode BluetoothMode { get; private set; } = IosBluetoothMode.HfpVoice;
|
||||
@@ -46,11 +67,16 @@ internal sealed class IosAudioRouter
|
||||
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.preset"), true, out IosAudioPreset preset)) Preset = preset;
|
||||
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.bluetoothMode"), true, out IosBluetoothMode bluetooth)) BluetoothMode = bluetooth;
|
||||
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.micMode"), true, out IosMicMode mic)) MicMode = mic;
|
||||
ForceSpeaker = defaults.BoolForKey("cat.voice.audio.forceSpeaker");
|
||||
// Speaker output is an output-routing choice, orthogonal to the capture preset, which is
|
||||
// why it sits outside Advanced audio. An install that predates the explicit choice carries
|
||||
// a speaker flag the preset set on its behalf; drop that once so it cannot pin a headset
|
||||
// user to the speaker, and honour the toggle from then on.
|
||||
if (defaults.BoolForKey("cat.voice.audio.speakerIsExplicit")) ForceSpeaker = defaults.BoolForKey("cat.voice.audio.forceSpeaker");
|
||||
VoiceProcessing = defaults.ValueForKey(new NSString("cat.voice.audio.voiceProcessing")) is null || defaults.BoolForKey("cat.voice.audio.voiceProcessing");
|
||||
AutomaticGainControl = defaults.ValueForKey(new NSString("cat.voice.audio.agc")) is null || defaults.BoolForKey("cat.voice.audio.agc");
|
||||
CaptureChannels = defaults.IntForKey("cat.voice.audio.captureChannels") == 2 ? 2 : Preset == IosAudioPreset.StereoMicrophone ? 2 : 1;
|
||||
SelectedInputId = defaults.StringForKey("cat.voice.audio.inputPortId"); SelectedDataSourceId = defaults.StringForKey("cat.voice.audio.dataSourceId");
|
||||
if (Preset == IosAudioPreset.VoiceChat) { SelectedInputId = null; SelectedDataSourceId = null; }
|
||||
if (Enum.TryParse(defaults.StringForKey("cat.voice.audio.polarPattern"), true, out AVAudioDataSourcePolarPattern pattern)) SelectedPolarPattern = pattern;
|
||||
RefreshRoutes();
|
||||
}
|
||||
@@ -68,12 +94,19 @@ internal sealed class IosAudioRouter
|
||||
// Named presets never carry an Advanced capsule selection across transitions.
|
||||
// Stereo derives its data source below; mono/voice chat must clear a stale stereo one.
|
||||
SelectedDataSourceId = null; SelectedPolarPattern = AVAudioDataSourcePolarPattern.Unknown;
|
||||
if (preset == IosAudioPreset.VoiceChat) ForceSpeaker = true;
|
||||
}
|
||||
SaveAndReconfigure();
|
||||
}
|
||||
|
||||
internal void SetForceSpeaker(bool value) { ForceSpeaker = value; SaveAndReconfigure(); }
|
||||
// Deliberately does not move the preset to Advanced: the speaker is an output choice every
|
||||
// named preset supports, so a voice-chat user can take a call on the speaker without losing
|
||||
// the capture configuration the preset stands for.
|
||||
// The port override is issued once, on the toggle, and never re-issued by a later rebuild.
|
||||
// Re-asserting it on every Apply means fighting iOS for the route: where the system wants to
|
||||
// hand output back to a connected headset, each rebuild forces it back to the speaker, the
|
||||
// route change that follows drives another rebuild, and the audio flips back and forth. The
|
||||
// DefaultToSpeaker category option below is the part that does persist across rebuilds.
|
||||
internal void SetForceSpeaker(bool value) { ForceSpeaker = value; speakerIsExplicit = overridePending = true; SaveAndReconfigure(); }
|
||||
internal void SetVoiceProcessing(bool value) { VoiceProcessing = value; SaveAndReconfigure(); }
|
||||
internal void SetAutomaticGainControl(bool value) { AutomaticGainControl = value; SaveAndReconfigure(); }
|
||||
internal void SetCaptureChannels(int value) { CaptureChannels = value == 2 ? 2 : 1; Preset = IosAudioPreset.Advanced; SaveAndReconfigure(); }
|
||||
@@ -94,8 +127,11 @@ internal sealed class IosAudioRouter
|
||||
{
|
||||
AVAudioSession session = AVAudioSession.SharedInstance();
|
||||
Inputs = session.AvailableInputs?.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray() ?? [];
|
||||
Outputs = session.CurrentRoute.Outputs.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray();
|
||||
SelectedInputId ??= session.PreferredInput?.UID; Changed?.Invoke();
|
||||
Outputs = session.CurrentRoute?.Outputs?.Select(value => new IosAudioPort(value.UID, value.PortName, value.PortType.ToString())).ToArray() ?? [];
|
||||
// A route reported by iOS is not an explicit user input choice. Capturing it here
|
||||
// can pin the built-in mic after a speaker fallback and displace a Bluetooth HFP route
|
||||
// on the next graph rebuild.
|
||||
Changed?.Invoke();
|
||||
}
|
||||
|
||||
internal void Apply(bool configureInput)
|
||||
@@ -109,14 +145,25 @@ internal sealed class IosAudioRouter
|
||||
if (BluetoothMode == IosBluetoothMode.BuiltInMicSpeaker || ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp) options |= AVAudioSessionCategoryOptions.DefaultToSpeaker;
|
||||
AVAudioSessionMode mode = CaptureChannels == 2 ? AVAudioSessionMode.Default : MicMode == IosMicMode.Raw ? AVAudioSessionMode.Measurement
|
||||
: BluetoothMode == IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionMode.VideoRecording : AVAudioSessionMode.VoiceChat;
|
||||
if (!session.SetCategory(AVAudioSessionCategory.PlayAndRecord, mode, options, out NSError? categoryError)) throw new InvalidOperationException(categoryError.LocalizedDescription);
|
||||
if (!session.SetCategory(AVAudioSessionCategory.PlayAndRecord, mode, options, out NSError? categoryError)) throw new InvalidOperationException(categoryError?.LocalizedDescription ?? "Could not configure the iOS audio session.");
|
||||
session.SetPreferredSampleRate(48_000, out _); session.SetPreferredIOBufferDuration(0.02, out _);
|
||||
if (!session.SetActive(true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? activeError)) throw new InvalidOperationException(activeError?.LocalizedDescription ?? "Could not activate the iOS audio session.");
|
||||
// Apple requires an active session before selecting a preferred input or data source.
|
||||
if (configureInput) ApplyInputSelection(session);
|
||||
if (!session.SetActive(true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? activeError)) throw new InvalidOperationException(activeError.LocalizedDescription);
|
||||
Console.Error.WriteLine($"VC_ROUTE preset={Preset} requestedCh={CaptureChannels} sessionCh={session.InputNumberOfChannels} " +
|
||||
$"preferred={session.PreferredInput?.PortName ?? "default"} dataSource={session.InputDataSource?.DataSourceName ?? "default"} " +
|
||||
$"pattern={session.InputDataSource?.SelectedPolarPattern.ToString() ?? "default"}");
|
||||
session.OverrideOutputAudioPort(ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp ? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _); RefreshRoutes();
|
||||
// DefaultToSpeaker only decides where audio goes when nothing else is connected, so a
|
||||
// user who asks for the speaker with a headset attached needs the port override as
|
||||
// well. Only the toggle itself issues it, and only once.
|
||||
if (overridePending)
|
||||
{
|
||||
overridePending = false;
|
||||
session.OverrideOutputAudioPort(ForceSpeaker && BluetoothMode != IosBluetoothMode.BuiltInMicA2dp
|
||||
? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _);
|
||||
}
|
||||
RefreshRoutes();
|
||||
ResetWatchdog(); EnsureWatchdog();
|
||||
}
|
||||
finally { applying = false; }
|
||||
}
|
||||
@@ -125,7 +172,8 @@ internal sealed class IosAudioRouter
|
||||
{
|
||||
AVAudioSessionPortDescription? port = session.AvailableInputs?.FirstOrDefault(value => value.UID == SelectedInputId);
|
||||
if (CaptureChannels == 2) port ??= session.AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic);
|
||||
if (port is null) { if (CaptureChannels == 1) ClearStereoPolarPattern(session); return; }
|
||||
if (port is null) { if (CaptureChannels == 1 && stereoApplied) ClearStereo(session); return; }
|
||||
if (!session.SetPreferredInput(port, out NSError? inputError)) throw new InvalidOperationException(inputError?.LocalizedDescription ?? "Could not select the microphone input.");
|
||||
if (CaptureChannels == 2)
|
||||
{
|
||||
// Polar-pattern discovery alone is insufficient on current iPhones: until a stereo
|
||||
@@ -135,26 +183,25 @@ internal sealed class IosAudioRouter
|
||||
// it also gives Core Audio an unambiguous left/right mapping before graph creation.
|
||||
if (!session.SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait, out NSError? orientationError))
|
||||
throw new InvalidOperationException(orientationError?.LocalizedDescription ?? "Could not set the stereo microphone orientation.");
|
||||
stereoApplied = true;
|
||||
}
|
||||
AVAudioSessionDataSourceDescription? source = CaptureChannels == 2
|
||||
? port.DataSources?.FirstOrDefault(SupportsStereoPolarPattern)
|
||||
: port.DataSources?.FirstOrDefault(value => value.DataSourceID.ToString() == SelectedDataSourceId);
|
||||
if (CaptureChannels == 1 && source is null) ClearStereoPolarPattern(session);
|
||||
if (CaptureChannels == 1 && source is null && stereoApplied) ClearStereo(session);
|
||||
if (source is not null)
|
||||
{
|
||||
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError.LocalizedDescription);
|
||||
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError?.LocalizedDescription ?? "Could not select the microphone data source.");
|
||||
if (CaptureChannels == 2) SetStereoPolarPattern(source);
|
||||
else if (SelectedPolarPattern != AVAudioDataSourcePolarPattern.Unknown &&
|
||||
!source.SetPreferredPolarPattern(SelectedPolarPattern, out NSError? patternError))
|
||||
throw new InvalidOperationException(patternError.LocalizedDescription);
|
||||
throw new InvalidOperationException(patternError?.LocalizedDescription ?? "Could not select the microphone polar pattern.");
|
||||
}
|
||||
if (!session.SetPreferredInput(port, out NSError? inputError)) throw new InvalidOperationException(inputError.LocalizedDescription);
|
||||
// The working Swift client deliberately does not call
|
||||
// SetPreferredInputNumberOfChannels: doing so disrupts stereo + A2DP routing. The stereo
|
||||
// capsule and polar pattern above cause the input node to expose its two-channel format.
|
||||
if (CaptureChannels == 2 && source is not null &&
|
||||
!session.SetInputDataSource(source, out NSError? sourceError))
|
||||
throw new InvalidOperationException(sourceError.LocalizedDescription);
|
||||
// A port preference is enough. SetInputDataSource only accepts a member of the *current*
|
||||
// port's InputDataSources, which can still be a different route after SetPreferredInput.
|
||||
Console.Error.WriteLine($"VC_ROUTE_SELECT port={port.PortName} source={source?.DataSourceName ?? "none"} " +
|
||||
$"stereoPattern={source is not null && SupportsStereoPolarPattern(source)} " +
|
||||
$"patterns={string.Join(',', source?.SupportedPolarPatterns?.Select(value => value.ToString()) ?? [])}");
|
||||
@@ -190,6 +237,8 @@ internal sealed class IosAudioRouter
|
||||
internal static extern byte SetObject(NativeHandle receiver, NativeHandle selector, NativeHandle value, ref NativeHandle error);
|
||||
}
|
||||
|
||||
private void ClearStereo(AVAudioSession session) { stereoApplied = false; ClearStereoPolarPattern(session); }
|
||||
|
||||
private static void ClearStereoPolarPattern(AVAudioSession session)
|
||||
{
|
||||
session.SetPreferredInputOrientation(AVAudioStereoOrientation.None, out _);
|
||||
@@ -204,30 +253,97 @@ internal sealed class IosAudioRouter
|
||||
{
|
||||
defaults.SetString(Preset.ToString(), "cat.voice.audio.preset"); defaults.SetString(BluetoothMode.ToString(), "cat.voice.audio.bluetoothMode");
|
||||
defaults.SetString(MicMode.ToString(), "cat.voice.audio.micMode"); defaults.SetBool(ForceSpeaker, "cat.voice.audio.forceSpeaker");
|
||||
defaults.SetBool(speakerIsExplicit, "cat.voice.audio.speakerIsExplicit");
|
||||
defaults.SetBool(VoiceProcessing, "cat.voice.audio.voiceProcessing"); defaults.SetBool(AutomaticGainControl, "cat.voice.audio.agc"); defaults.SetInt(CaptureChannels, "cat.voice.audio.captureChannels");
|
||||
Set("cat.voice.audio.inputPortId", SelectedInputId); Set("cat.voice.audio.dataSourceId", SelectedDataSourceId); defaults.SetString(SelectedPolarPattern.ToString(), "cat.voice.audio.polarPattern"); defaults.Synchronize();
|
||||
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(); Changed?.Invoke();
|
||||
ResetWatchdogAttempts();
|
||||
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(true, "settings"); Changed?.Invoke();
|
||||
}
|
||||
|
||||
private void Set(string key, string? value) { if (value is null) defaults.RemoveObject(key); else defaults.SetString(value, key); }
|
||||
internal void Deactivate() => AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out _);
|
||||
internal void Deactivate()
|
||||
{
|
||||
// A released session has no route the next graph can be compared against.
|
||||
watchdog?.Dispose(); watchdog = null; ResetWatchdog(); ResetWatchdogAttempts();
|
||||
if (!AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out NSError? error))
|
||||
Console.Error.WriteLine($"VC_DEACTIVATE failed: {error?.LocalizedDescription ?? "no system error"}");
|
||||
else Console.Error.WriteLine("VC_DEACTIVATE succeeded");
|
||||
}
|
||||
internal void EnsureAudio(string reason)
|
||||
{
|
||||
if (!IosAudioEngine.Shared.IsConnected) return;
|
||||
if (!IosAudioEngine.Shared.IsConnected || interrupted) return;
|
||||
try { IosAudioEngine.Shared.EnsureRunning(); }
|
||||
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
|
||||
}
|
||||
|
||||
internal void Recover(string reason)
|
||||
internal void Recover(string reason, bool force = true)
|
||||
{
|
||||
RefreshRoutes();
|
||||
if (!IosAudioEngine.Shared.IsConnected) return;
|
||||
if (!IosAudioEngine.Shared.IsConnected || interrupted) return;
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
|
||||
{
|
||||
try { IosAudioEngine.Shared.Reconfigure(); }
|
||||
try { IosAudioEngine.Shared.Reconfigure(force, reason); }
|
||||
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
|
||||
});
|
||||
}
|
||||
|
||||
// The render callback is the only clock for the device-clocked pipeline, so a graph that
|
||||
// stops while the app is backgrounded freezes capture, mix and send with no notification to
|
||||
// recover from. Poll for that and rebuild; a failed rebuild is retried on the next tick.
|
||||
private void EnsureWatchdog()
|
||||
{
|
||||
watchdog ??= new System.Threading.Timer(_ => UIApplication.SharedApplication.BeginInvokeOnMainThread(TickWatchdog),
|
||||
null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1));
|
||||
}
|
||||
|
||||
private void ResetWatchdog() { lastRenderCallbacks = -1; watchdogMisses = 0; }
|
||||
private void ResetWatchdogAttempts() { watchdogAttempts = 0; nextWatchdogAttempt = 0; }
|
||||
|
||||
// An interruption that ends while the app is suspended never delivers its Ended notification,
|
||||
// so foregrounding still has to check. It must not rebuild unconditionally though: the `audio`
|
||||
// background mode keeps the session and graph live across a backgrounding, so the graph is
|
||||
// almost always healthy here and a rebuild costs a visible glitch plus, on Bluetooth, an HFP
|
||||
// renegotiation. Ensure it is running, and only reconfigure when it actually stopped.
|
||||
internal void ResumeForeground()
|
||||
{
|
||||
interrupted = false; ResetWatchdog(); RefreshRoutes();
|
||||
if (IosAudioEngine.Shared.IsConnected && !IosAudioEngine.Shared.IsRunning) Recover("foreground");
|
||||
else EnsureAudio("foreground");
|
||||
}
|
||||
|
||||
private void TickWatchdog()
|
||||
{
|
||||
if (watchdogTicking) return;
|
||||
watchdogTicking = true;
|
||||
try
|
||||
{
|
||||
// A graph still starting up reports no callbacks yet and reads as not running. Judging
|
||||
// it there is how the watchdog ends up rebuilding a healthy graph on every tick.
|
||||
if (!IosAudioEngine.Shared.IsConnected || interrupted || applying || IosAudioEngine.Shared.Settling) { ResetWatchdog(); return; }
|
||||
long callbacks = IosAudioEngine.Shared.RenderCallbacks;
|
||||
bool stalled = !IosAudioEngine.Shared.IsRunning || callbacks == lastRenderCallbacks;
|
||||
lastRenderCallbacks = callbacks;
|
||||
if (!stalled) { watchdogMisses = 0; ResetWatchdogAttempts(); return; }
|
||||
// One missed tick can be a route change already rebuilding the graph.
|
||||
if (++watchdogMisses < 2) return;
|
||||
if (Environment.TickCount64 < nextWatchdogAttempt) return;
|
||||
if (watchdogAttempts >= MaximumWatchdogAttempts)
|
||||
{
|
||||
if (watchdogAttempts == MaximumWatchdogAttempts)
|
||||
{
|
||||
watchdogAttempts++;
|
||||
Console.Error.WriteLine("VC_WATCHDOG exhausted; the audio graph will not start and is no longer being rebuilt");
|
||||
}
|
||||
return;
|
||||
}
|
||||
ResetWatchdog();
|
||||
watchdogAttempts++;
|
||||
nextWatchdogAttempt = Environment.TickCount64 + Math.Min(2_000 * (1 << watchdogAttempts), 30_000);
|
||||
try { IosAudioEngine.Shared.Reconfigure(true, $"stall watchdog {watchdogAttempts}"); }
|
||||
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio watchdog rebuild failed: {exception}"); }
|
||||
}
|
||||
finally { watchdogTicking = false; }
|
||||
}
|
||||
private void HandleRouteChange(NSNotification note)
|
||||
{
|
||||
NSNumber? value = note.UserInfo?[new NSString("AVAudioSessionRouteChangeReasonKey")] as NSNumber;
|
||||
@@ -237,12 +353,29 @@ internal sealed class IosAudioRouter
|
||||
or AVAudioSessionRouteChangeReason.Override
|
||||
or AVAudioSessionRouteChangeReason.RouteConfigurationChange)
|
||||
return;
|
||||
Recover($"route change ({reason})");
|
||||
// Apply is mid-flight: this notification describes the change Apply is itself making.
|
||||
if (applying) return;
|
||||
// Not a forced rebuild. Every reconfiguration moves the route, and moving the route
|
||||
// notifies here, so answering a route change with an unconditional rebuild is a loop with
|
||||
// one iteration per notification: forcing the speaker takes a headset out of the route as
|
||||
// OldDeviceUnavailable, releasing it brings the headset back as NewDeviceAvailable, and
|
||||
// even joining voice moves the route through SetPreferredInput. AVAudioEngine follows a
|
||||
// route change on its own; what it cannot absorb is the hardware under the tap changing,
|
||||
// and Reconfigure tests for that, rebuilding a stopped graph and leaving a healthy
|
||||
// unchanged one alone. Comparing routes instead cannot work: CurrentRoute still names the
|
||||
// previous route for a while after a reconfiguration.
|
||||
Console.Error.WriteLine($"VC_ROUTE_CHANGE reason={reason} running={IosAudioEngine.Shared.IsRunning} " +
|
||||
$"hardwareChanged={IosAudioEngine.Shared.HardwareChanged()}");
|
||||
Recover($"route change ({reason})", force: false);
|
||||
}
|
||||
private void HandleInterruption(NSNotification note)
|
||||
{
|
||||
NSNumber? type = note.UserInfo?[new NSString("AVAudioSessionInterruptionTypeKey")] as NSNumber;
|
||||
AVAudioSessionInterruptionType interruption = (AVAudioSessionInterruptionType)(type?.UInt32Value ?? 0);
|
||||
if (interruption == AVAudioSessionInterruptionType.Ended) Recover("interruption ended");
|
||||
// The system stops the graph on Began and SetActive fails until the interruption clears,
|
||||
// so suppress recovery until Ended and let the watchdog retry if that rebuild fails.
|
||||
if (interruption == AVAudioSessionInterruptionType.Began) { interrupted = true; ResetWatchdog(); return; }
|
||||
if (interruption != AVAudioSessionInterruptionType.Ended) return;
|
||||
interrupted = false; ResetWatchdog(); Recover("interruption ended");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
using CoreFoundation;
|
||||
using Network;
|
||||
using VoiceCat.Core;
|
||||
|
||||
namespace VoiceCat.iOS;
|
||||
|
||||
// Watches which interface actually carries traffic so a handover can be acted on the instant it
|
||||
// happens instead of after the control keepalive proves the old path dead.
|
||||
//
|
||||
// The signal must be narrow. Roaming between access points, a lift ride, a minute of bad
|
||||
// cellular: those keep the same interface and the same source address, so TCP survives them and
|
||||
// a reconnect would be pure damage. Only the set of satisfied interfaces changing — Wi-Fi to
|
||||
// cellular, one physical link to another — strands the existing sockets on a source address that
|
||||
// no longer exists, and that is the only thing reported here.
|
||||
internal sealed class IosNetworkPathMonitor : IDisposable
|
||||
{
|
||||
internal static IosNetworkPathMonitor Shared { get; } = new();
|
||||
private readonly DispatchQueue queue = new("voicecat.path");
|
||||
private readonly object gate = new();
|
||||
private readonly ControlPathWatcher watcher = new();
|
||||
private NWPathMonitor? monitor;
|
||||
private bool started;
|
||||
|
||||
// Raised on the monitor queue when the carrying interface changed and the new path is usable.
|
||||
internal event Action<string>? InterfaceChanged;
|
||||
|
||||
internal void Start()
|
||||
{
|
||||
lock (gate)
|
||||
{
|
||||
if (started) return;
|
||||
started = true;
|
||||
monitor = new NWPathMonitor();
|
||||
monitor.SetQueue(queue);
|
||||
monitor.SnapshotHandler = OnPath;
|
||||
monitor.Start();
|
||||
}
|
||||
}
|
||||
|
||||
private void OnPath(NWPath path)
|
||||
{
|
||||
List<string> interfaces = [];
|
||||
if (path.Status == NWPathStatus.Satisfied)
|
||||
path.EnumerateInterfaces(item => { interfaces.Add($"{item.InterfaceType}:{item.Name}"); return true; });
|
||||
if (watcher.Observe(path.Status == NWPathStatus.Satisfied, interfaces))
|
||||
InterfaceChanged?.Invoke(string.Join(",", interfaces));
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
lock (gate)
|
||||
{
|
||||
monitor?.Cancel(); monitor?.Dispose(); monitor = null; started = false; watcher.Reset();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -34,7 +34,13 @@ internal sealed class VoiceControlsView : UIView
|
||||
private readonly AppModel model; private readonly UIButton join = UIButton.FromType(UIButtonType.System);
|
||||
private readonly UIButton ptt = UIButton.FromType(UIButtonType.System); private readonly UIButton mute = UIButton.FromType(UIButtonType.System);
|
||||
private readonly UIButton deafen = UIButton.FromType(UIButtonType.System); private readonly UIProgressView level = new(UIProgressViewStyle.Default);
|
||||
internal VoiceControlsView(AppModel model) { this.model = model; model.Changed += Refresh; Build(); }
|
||||
private int renderedLevel = -1;
|
||||
internal VoiceControlsView(AppModel model) { this.model = model; model.Changed += Refresh; model.LevelChanged += RefreshLevel; Build(); }
|
||||
protected override void Dispose(bool disposing)
|
||||
{
|
||||
if (disposing) { model.Changed -= Refresh; model.LevelChanged -= RefreshLevel; }
|
||||
base.Dispose(disposing);
|
||||
}
|
||||
private void Build()
|
||||
{
|
||||
BackgroundColor = UIColor.SecondarySystemBackground; join.TouchUpInside += async (_, _) => await Run(model.ToggleVoiceAsync);
|
||||
@@ -54,22 +60,50 @@ internal sealed class VoiceControlsView : UIView
|
||||
bool muted = model.Client?.Audio.MicMuted == true, deafened = model.Client?.Audio.Deafened == true;
|
||||
mute.SetImage(UIImage.GetSystemImage(muted ? "mic.slash.fill" : "mic.fill"), UIControlState.Normal); mute.AccessibilityLabel = muted ? "Unmute microphone" : "Mute microphone"; mute.Enabled = model.VoiceJoined;
|
||||
deafen.SetImage(UIImage.GetSystemImage(deafened ? "headphones.slash" : "headphones"), UIControlState.Normal); deafen.AccessibilityLabel = deafened ? "Undeafen" : "Deafen"; deafen.Enabled = model.VoiceJoined;
|
||||
level.Progress = Math.Clamp(model.MicrophoneLevel * 10, 0, 1); level.AccessibilityLabel = "Microphone level"; level.AccessibilityValue = $"{level.Progress:P0}";
|
||||
level.AccessibilityLabel = "Microphone level"; RefreshLevel();
|
||||
}
|
||||
|
||||
// Driven at 20 Hz. Rewriting AccessibilityValue on every tick makes VoiceOver re-announce the
|
||||
// meter continuously while it is focused, so only publish a value that actually changed.
|
||||
private void RefreshLevel()
|
||||
{
|
||||
float progress = Math.Clamp(model.MicrophoneLevel * 10, 0, 1); level.Progress = progress;
|
||||
int percent = (int)MathF.Round(progress * 100 / 5) * 5;
|
||||
if (percent == renderedLevel) return;
|
||||
renderedLevel = percent; level.AccessibilityValue = $"{percent}%";
|
||||
}
|
||||
private async Task Run(Func<Task> operation) { try { await operation(); } catch (Exception exception) { if (Window?.RootViewController is { } owner) UiHelpers.ShowError(owner, exception); } }
|
||||
}
|
||||
|
||||
internal sealed class ChannelsController : UITableViewController
|
||||
{
|
||||
private readonly AppModel model; private IReadOnlyList<(Channel Channel, int Depth)> Visible => Flatten();
|
||||
internal ChannelsController(AppModel model) { this.model = model; Title = "Channels"; model.Changed += Reload; }
|
||||
private readonly AppModel model; private readonly ListRefresher refresher; private IReadOnlyList<(Channel Channel, int Depth)> Visible => Flatten();
|
||||
internal ChannelsController(AppModel model)
|
||||
{
|
||||
this.model = model; Title = "Channels";
|
||||
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad()
|
||||
{
|
||||
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "channel");
|
||||
NavigationItem.RightBarButtonItems = [new("Users", UIBarButtonItemStyle.Plain, (_, _) => NavigationController?.PushViewController(new UsersController(model), true)),
|
||||
new(UIBarButtonSystemItem.Add, (_, _) => NavigationController?.PushViewController(new ChannelEditorController(model, null), true))]; Reload();
|
||||
new(UIBarButtonSystemItem.Add, (_, _) => NavigationController?.PushViewController(new ChannelEditorController(model, null), true))];
|
||||
}
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); UpdateActions(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature()
|
||||
{
|
||||
UpdateActions();
|
||||
return string.Join('\n', Visible.Select(entry =>
|
||||
$"{entry.Channel.Id}|{entry.Depth}|{entry.Channel.Name}|{entry.Channel.Topic}|{entry.Channel.PasswordProtected}|" +
|
||||
$"{entry.Channel.Id == model.CurrentChannelId}|{model.Users.Count(user => user.ChannelId == entry.Channel.Id)}"));
|
||||
}
|
||||
// The add button is not part of the table, so it can follow every notification cheaply.
|
||||
private void UpdateActions()
|
||||
{
|
||||
if (NavigationItem.RightBarButtonItems is not { Length: > 1 } items) return;
|
||||
items[1].Enabled = model.Client?.Permissions is { } p && (p.IsAdmin || p.CanCreateTempChannel);
|
||||
}
|
||||
private void Reload() { TableView.ReloadData(); NavigationItem.RightBarButtonItems![1].Enabled = model.Client?.Permissions is { } p && (p.IsAdmin || p.CanCreateTempChannel); }
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => Visible.Count;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
|
||||
{
|
||||
@@ -112,7 +146,9 @@ internal sealed class ChannelsController : UITableViewController
|
||||
internal sealed class ChatController : UIViewController
|
||||
{
|
||||
private readonly AppModel model; private readonly UITextView log = new(); private readonly UITextField compose = UiHelpers.Field("Message");
|
||||
internal ChatController(AppModel model) { this.model = model; Title = "Chat"; model.Changed += Refresh; }
|
||||
internal ChatController(AppModel model) { this.model = model; Title = "Chat"; }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); model.Changed += Refresh; Refresh(); }
|
||||
public override void ViewDidDisappear(bool animated) { model.Changed -= Refresh; base.ViewDidDisappear(animated); }
|
||||
public override void ViewDidLoad()
|
||||
{
|
||||
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; log.Editable = false; log.Font = UIFont.PreferredBody; log.AccessibilityLabel = "Chat and activity timeline"; log.TranslatesAutoresizingMaskIntoConstraints = false;
|
||||
@@ -125,7 +161,11 @@ internal sealed class ChatController : UIViewController
|
||||
{
|
||||
IEnumerable<(DateTime Time, string Text)> chat = model.Messages.Where(message => !message.Private).Select(message => (message.Timestamp, $"{message.Sender}: {message.Text}"));
|
||||
IEnumerable<(DateTime Time, string Text)> activity = model.Activity.Select(value => (value.Timestamp, $"• {value.Text}"));
|
||||
log.Text = string.Join("\n", chat.Concat(activity).OrderBy(value => value.Time).Select(value => $"[{value.Time:t}] {value.Text}")); if (log.Text.Length > 0) log.ScrollRangeToVisible(new(log.Text.Length - 1, 1));
|
||||
string text = string.Join("\n", chat.Concat(activity).OrderBy(value => value.Time).Select(value => $"[{value.Time:t}] {value.Text}"));
|
||||
// Reassigning identical text still resets the text view's accessibility state, which
|
||||
// interrupts VoiceOver mid-read of the transcript.
|
||||
if (text == log.Text) return;
|
||||
log.Text = text; if (text.Length > 0) log.ScrollRangeToVisible(new(text.Length - 1, 1));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -145,8 +185,24 @@ internal sealed class PrivateChatsController : UITableViewController
|
||||
}).OrderByDescending(peer => peer.Latest?.Timestamp ?? DateTime.MinValue).ThenBy(peer => peer.Name).ToArray();
|
||||
}
|
||||
}
|
||||
internal PrivateChatsController(AppModel model) { this.model = model; Title = "Private Chats"; model.Changed += Reload; }
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "private-peer"); Reload(); }
|
||||
private readonly ListRefresher refresher;
|
||||
internal PrivateChatsController(AppModel model)
|
||||
{
|
||||
this.model = model; Title = "Private Chats";
|
||||
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "private-peer"); }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature()
|
||||
{
|
||||
IReadOnlyList<(uint Id, string Name, ChatEntry? Latest, bool Online)> peers = Peers;
|
||||
// The empty-state view is not a table row, so it is kept current outside the reload.
|
||||
bool empty = peers.Count == 0, shown = TableView.BackgroundView is not null;
|
||||
if (empty != shown)
|
||||
TableView.BackgroundView = empty ? new UILabel { Text = "No other users or private conversations", TextAlignment = UITextAlignment.Center, AccessibilityLabel = "No other users or private conversations" } : null;
|
||||
return string.Join('\n', peers.Select(peer => $"{peer.Id}|{peer.Name}|{peer.Online}|{peer.Latest?.Text}"));
|
||||
}
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => Peers.Count;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
|
||||
{
|
||||
@@ -158,7 +214,6 @@ internal sealed class PrivateChatsController : UITableViewController
|
||||
{
|
||||
(uint id, string name, _, _) = Peers[indexPath.Row]; tableView.DeselectRow(indexPath, true); NavigationController?.PushViewController(new PrivateConversationController(model, id, name), true);
|
||||
}
|
||||
private void Reload() { if (IsViewLoaded) { TableView.ReloadData(); TableView.BackgroundView = Peers.Count == 0 ? new UILabel { Text = "No other users or private conversations", TextAlignment = UITextAlignment.Center, AccessibilityLabel = "No other users or private conversations" } : null; } }
|
||||
}
|
||||
|
||||
internal sealed class PrivateConversationController : UIViewController
|
||||
@@ -167,7 +222,9 @@ internal sealed class PrivateConversationController : UIViewController
|
||||
private readonly UITextView transcript = new(); private readonly UITextField compose = UiHelpers.Field("Private message"); private readonly UIButton send = UIButton.FromType(UIButtonType.System);
|
||||
private User? Peer => model.Users.FirstOrDefault(user => user.Id == peerUserId);
|
||||
internal PrivateConversationController(AppModel model, uint peerUserId, string fallbackName)
|
||||
{ this.model = model; this.peerUserId = peerUserId; this.fallbackName = fallbackName; Title = fallbackName; model.Changed += Refresh; }
|
||||
{ this.model = model; this.peerUserId = peerUserId; this.fallbackName = fallbackName; Title = fallbackName; }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); model.Changed += Refresh; Refresh(); }
|
||||
public override void ViewDidDisappear(bool animated) { model.Changed -= Refresh; base.ViewDidDisappear(animated); }
|
||||
public override void ViewDidLoad()
|
||||
{
|
||||
base.ViewDidLoad(); View!.BackgroundColor = UIColor.SystemBackground; transcript.Editable = false; transcript.Font = UIFont.PreferredBody; transcript.AccessibilityLabel = $"Private conversation with {fallbackName}"; transcript.TranslatesAutoresizingMaskIntoConstraints = false;
|
||||
@@ -182,8 +239,11 @@ internal sealed class PrivateConversationController : UIViewController
|
||||
{
|
||||
if (!IsViewLoaded) return; string name = Peer?.Nickname ?? fallbackName; Title = name;
|
||||
IEnumerable<ChatEntry> messages = model.Messages.Where(message => message.Private && message.PeerUserId == peerUserId).OrderBy(message => message.Timestamp);
|
||||
transcript.Text = string.Join("\n", messages.Select(message => $"[{message.Timestamp:t}] {(message.SenderId == model.SelfUserId ? "You" : message.Sender)}: {message.Text}"));
|
||||
if (transcript.Text.Length > 0) transcript.ScrollRangeToVisible(new(transcript.Text.Length - 1, 1));
|
||||
string text = string.Join("\n", messages.Select(message => $"[{message.Timestamp:t}] {(message.SenderId == model.SelfUserId ? "You" : message.Sender)}: {message.Text}"));
|
||||
if (text != transcript.Text)
|
||||
{
|
||||
transcript.Text = text; if (text.Length > 0) transcript.ScrollRangeToVisible(new(text.Length - 1, 1));
|
||||
}
|
||||
bool online = Peer is not null; compose.Enabled = online; send.Enabled = online; NavigationItem.RightBarButtonItem!.Enabled = online;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,12 +5,24 @@ namespace VoiceCat.iOS;
|
||||
|
||||
internal sealed class ServerListController : UITableViewController
|
||||
{
|
||||
private readonly AppModel model;
|
||||
internal ServerListController(AppModel model) { this.model = model; Title = "Servers"; TabBarItem = new(UITabBarSystemItem.Favorites, 0); }
|
||||
private readonly AppModel model; private readonly ListRefresher refresher;
|
||||
internal ServerListController(AppModel model)
|
||||
{
|
||||
this.model = model; Title = "Servers"; TabBarItem = new(UITabBarSystemItem.Favorites, 0);
|
||||
refresher = new(this, Signature, handler => model.Changed += handler, handler => model.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad()
|
||||
{
|
||||
base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "server");
|
||||
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PresentEditor(null)); model.Changed += Reload;
|
||||
NavigationItem.RightBarButtonItem = new(UIBarButtonSystemItem.Add, (_, _) => PresentEditor(null));
|
||||
}
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature()
|
||||
{
|
||||
// The connection prompt is navigation-bar chrome, not a row, so it follows every change.
|
||||
NavigationItem.Prompt = model.IsConnecting ? model.Status : null;
|
||||
return string.Join('\n', model.Profiles.Select(profile => $"{profile.Id}|{profile.DisplayName}|{profile.Authentication}"));
|
||||
}
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => model.Profiles.Count;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath indexPath)
|
||||
@@ -32,7 +44,6 @@ internal sealed class ServerListController : UITableViewController
|
||||
return UISwipeActionsConfiguration.FromActions([delete, edit]);
|
||||
}
|
||||
private void PresentEditor(ServerProfile? profile) => PresentViewController(new UINavigationController(new ServerEditorController(model, profile)), true, null);
|
||||
private void Reload() { TableView.ReloadData(); NavigationItem.Prompt = model.IsConnecting ? model.Status : null; }
|
||||
}
|
||||
|
||||
internal sealed class ServerEditorController : UIViewController
|
||||
|
||||
@@ -9,10 +9,22 @@ namespace VoiceCat.iOS;
|
||||
|
||||
internal sealed class SettingsController : UITableViewController
|
||||
{
|
||||
private readonly AppModel model;
|
||||
internal SettingsController(AppModel model) : base(UITableViewStyle.InsetGrouped) { this.model = model; Title = "Settings"; model.Changed += Reload; IosAudioRouter.Shared.Changed += Reload; }
|
||||
private readonly AppModel model; private readonly ListRefresher refresher;
|
||||
internal SettingsController(AppModel model) : base(UITableViewStyle.InsetGrouped)
|
||||
{
|
||||
this.model = model; Title = "Settings";
|
||||
refresher = new(this, Signature,
|
||||
handler => { model.Changed += handler; IosAudioRouter.Shared.Changed += handler; },
|
||||
handler => { model.Changed -= handler; IosAudioRouter.Shared.Changed -= handler; });
|
||||
}
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "setting"); }
|
||||
private void Reload() => TableView.ReloadData();
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature() =>
|
||||
$"{IosAudioRouter.Shared.Preset}|{IosAudioRouter.Shared.ForceSpeaker}|{model.Settings.AudioBufferMilliseconds}|{model.ScreenSharing}|" +
|
||||
$"{model.Settings.InputMode}|{model.Settings.VadThreshold}|{model.Settings.InputGain}|{model.Settings.InputNoiseReduction}|" +
|
||||
$"{model.Settings.EventSounds}|{model.Settings.EventVolume}|{model.Settings.SpokenEvents}|{model.Settings.SelfTalkSounds}|{model.Settings.PushToTalkSound}|" +
|
||||
$"{model.Client?.Permissions is { } p && (p.IsAdmin || p.CanAdminAccounts)}";
|
||||
public override nint NumberOfSections(UITableView tableView) => 5;
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => section switch { 0 => 5, 1 => 4, 2 => 5, 3 => model.Client?.Permissions is { } p && (p.IsAdmin || p.CanAdminAccounts) ? 1 : 0, _ => 2 };
|
||||
public override string? TitleForHeader(UITableView tableView, nint section) => section switch { 0 => "Audio", 1 => "Voice", 2 => "Notifications", 3 => "Administration", _ => "Server" };
|
||||
@@ -69,9 +81,19 @@ internal sealed class SettingsController : UITableViewController
|
||||
|
||||
internal sealed class AdvancedAudioController : UITableViewController
|
||||
{
|
||||
private readonly IosAudioRouter router = IosAudioRouter.Shared;
|
||||
internal AdvancedAudioController() : base(UITableViewStyle.InsetGrouped) { Title = "Advanced Audio"; router.Changed += () => TableView.ReloadData(); }
|
||||
private readonly IosAudioRouter router = IosAudioRouter.Shared; private readonly ListRefresher refresher;
|
||||
internal AdvancedAudioController() : base(UITableViewStyle.InsetGrouped)
|
||||
{
|
||||
Title = "Advanced Audio";
|
||||
refresher = new(this, Signature, handler => IosAudioRouter.Shared.Changed += handler, handler => IosAudioRouter.Shared.Changed -= handler);
|
||||
}
|
||||
public override void ViewDidLoad() { base.ViewDidLoad(); TableView.RegisterClassForCellReuse(typeof(UITableViewCell), "audio"); router.RefreshRoutes(); }
|
||||
public override void ViewWillAppear(bool animated) { base.ViewWillAppear(animated); refresher.Start(); }
|
||||
public override void ViewDidDisappear(bool animated) { refresher.Stop(); base.ViewDidDisappear(animated); }
|
||||
private string Signature() =>
|
||||
$"{router.SelectedInputId}|{router.SelectedDataSourceId}|{router.SelectedPolarPattern}|{router.MicMode}|{router.CaptureChannels}|" +
|
||||
$"{router.BluetoothMode}|{router.UsesVoiceProcessing}|{router.AutomaticGainControl}|" +
|
||||
$"{string.Join(',', router.Inputs.Select(value => value.Name))}|{string.Join(',', router.Outputs.Select(value => value.Name))}";
|
||||
public override nint RowsInSection(UITableView tableView, nint section) => 8;
|
||||
public override UITableViewCell GetCell(UITableView tableView, NSIndexPath path)
|
||||
{
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
#if DEBUG
|
||||
using System.Buffers.Binary;
|
||||
using System.Runtime.InteropServices;
|
||||
using Foundation;
|
||||
using UIKit;
|
||||
using VoiceCat.Core;
|
||||
|
||||
namespace VoiceCat.iOS;
|
||||
|
||||
// An opt-in simulator gate for the real UIKit model and audio graph. Local development only.
|
||||
internal static class SimulatorSmoke
|
||||
{
|
||||
internal static void RunIfRequested()
|
||||
{
|
||||
string? endpoint = Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE");
|
||||
if (endpoint is null || !RuntimeInformation.RuntimeIdentifier.StartsWith("iossimulator", StringComparison.Ordinal)) return;
|
||||
UIApplication.SharedApplication.BeginInvokeOnMainThread(async () =>
|
||||
{
|
||||
try
|
||||
{
|
||||
string[] parts = endpoint.Split(':', 3);
|
||||
Guid id = Guid.Parse("959da92f-f088-452d-a870-4c15007a6fd3");
|
||||
ServerProfile guest = ServerProfile.Create(parts[0], ushort.Parse(parts[1]), ServerAuthentication.Guest, nickname: "smoke", id: id);
|
||||
ServerProfile profile = ServerProfile.Create(parts[0], ushort.Parse(parts[1]), ServerAuthentication.Account, "smoke", id: id);
|
||||
AppModel model = AppModel.Shared;
|
||||
bool share = Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") == "share";
|
||||
string? ring = share ? PrepareRing() : null;
|
||||
model.IdentityRequested += _ => model.ResolveIdentity(true);
|
||||
if (Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") != "resume")
|
||||
{
|
||||
model.UpsertProfile(guest, null);
|
||||
model.UpsertProfile(profile, parts[2]);
|
||||
}
|
||||
model.Load();
|
||||
if (Environment.GetEnvironmentVariable("VOICECAT_SIM_SMOKE_MODE") == "stereo")
|
||||
IosAudioRouter.Shared.SelectPreset(IosAudioPreset.StereoMicrophone);
|
||||
profile = model.Profiles.Single(value => value.Id == id);
|
||||
Console.Error.WriteLine("VC_SIM phase=connect");
|
||||
await model.ConnectAsync(profile);
|
||||
Console.Error.WriteLine($"VC_SIM phase=connected audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
Require(model.IsConnected, "Account login did not connect.");
|
||||
if (ring is not null)
|
||||
{
|
||||
// Start a synthetic producer while the microphone is off. This exercises
|
||||
// screen-only subscription and the real BroadcastChanged audio recovery.
|
||||
await Task.Delay(300);
|
||||
Require(!model.ScreenRingMapped && !model.ScreenSharing,
|
||||
"An unchanged ring from a previous broadcast remained mapped.");
|
||||
for (int frame = 0; frame < 6; frame++) WriteScreenPcm(ring);
|
||||
await WaitUntil(() => model.ScreenSharing, "Screen stream did not start.");
|
||||
Require(IosAudioEngine.Shared.IsRunning, "Screen sharing stopped the audio graph.");
|
||||
await WaitUntil(() => model.ScreenEncodedPackets > 0,
|
||||
"Screen PCM captured before stream negotiation was discarded.");
|
||||
long encodedBefore = model.ScreenEncodedPackets;
|
||||
for (int frame = 0; frame < 12; frame++)
|
||||
{
|
||||
WriteScreenPcm(ring);
|
||||
await Task.Delay(20);
|
||||
}
|
||||
await WaitUntil(() => model.ScreenEncodedPackets > encodedBefore,
|
||||
"Screen PCM did not reach the encoder while the microphone was off.");
|
||||
WriteRing(ring, active: false);
|
||||
await WaitUntil(() => !model.ScreenSharing, "Screen stream did not stop.");
|
||||
Require(!model.ScreenRingMapped, "The stopped screen ring remained mapped.");
|
||||
Console.Error.WriteLine("VC_SIM phase=screenSharePassed");
|
||||
}
|
||||
await model.ToggleVoiceAsync();
|
||||
Console.Error.WriteLine($"VC_SIM phase=joined audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
await Task.Delay(2500);
|
||||
Require(model.VoiceJoined && IosAudioEngine.Shared.IsRunning, "The first voice graph stopped.");
|
||||
long beforeShare = IosAudioEngine.Shared.RenderCallbacks;
|
||||
IosAudioRouter.Shared.Recover("simulated screen sharing start", force: false);
|
||||
await Task.Delay(1000);
|
||||
Require(IosAudioEngine.Shared.IsRunning && IosAudioEngine.Shared.RenderCallbacks > beforeShare,
|
||||
"Screen sharing recovery stopped the live audio graph.");
|
||||
if (ring is not null)
|
||||
{
|
||||
WriteRing(ring, active: true);
|
||||
await Task.Delay(300);
|
||||
WriteScreenPcm(ring);
|
||||
await WaitUntil(() => model.ScreenSharing, "Screen stream did not start during a voice call.");
|
||||
Require(IosAudioEngine.Shared.IsRunning, "Starting screen sharing stopped the voice graph.");
|
||||
}
|
||||
await model.ToggleVoiceAsync();
|
||||
if (ring is not null)
|
||||
{
|
||||
Require(model.ScreenSharing, "Leaving the microphone call unsubscribed active screen audio.");
|
||||
WriteRing(ring, active: false);
|
||||
await WaitUntil(() => !model.ScreenSharing, "The second screen stream did not stop.");
|
||||
}
|
||||
await model.DisconnectAsync();
|
||||
await Task.Delay(3000);
|
||||
model.Load();
|
||||
profile = model.Profiles.Single(value => value.Id == id);
|
||||
Console.Error.WriteLine("VC_SIM phase=reconnect");
|
||||
await model.ConnectAsync(profile);
|
||||
Console.Error.WriteLine($"VC_SIM phase=reconnected audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
Require(model.IsConnected, "Saved account login did not reconnect.");
|
||||
await Task.Delay(2500);
|
||||
Console.Error.WriteLine($"VC_SIM phase=reconnectedSettled audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
Require(IosAudioEngine.Shared.IsRunning, "The reconnect audio graph stopped.");
|
||||
await model.ToggleVoiceAsync();
|
||||
Console.Error.WriteLine($"VC_SIM phase=rejoined audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
await Task.Delay(2500);
|
||||
Console.Error.WriteLine($"VC_SIM phase=rejoinedSettled audioRunning={IosAudioEngine.Shared.IsRunning}");
|
||||
Require(model.VoiceJoined && IosAudioEngine.Shared.IsRunning, "The rejoined voice graph stopped.");
|
||||
await model.DisconnectAsync();
|
||||
Console.Error.WriteLine("VC_SIM phase=passed");
|
||||
}
|
||||
catch (Exception exception) { Console.Error.WriteLine($"VC_SIM phase=failed exception={exception}"); }
|
||||
});
|
||||
}
|
||||
|
||||
private static void Require(bool condition, string message)
|
||||
{
|
||||
if (!condition) throw new InvalidOperationException(message);
|
||||
}
|
||||
|
||||
private static string PrepareRing()
|
||||
{
|
||||
string root = NSFileManager.DefaultManager.GetContainerUrl(IosConstants.AppGroup)?.Path
|
||||
?? throw new InvalidOperationException("Simulator App Group is unavailable.");
|
||||
string path = Path.Combine(root, "voicecat", "broadcast_audio.ring");
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
|
||||
using var file = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.ReadWrite);
|
||||
file.SetLength(64 + 96_000 * sizeof(short));
|
||||
byte[] header = new byte[20];
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(header, 0x56434252);
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(4), 1);
|
||||
file.Write(header);
|
||||
file.Dispose();
|
||||
WriteRing(path, active: true, write: 1920);
|
||||
return path;
|
||||
}
|
||||
|
||||
private static void WriteRing(string path, bool active, ulong? write = null)
|
||||
{
|
||||
using var file = new FileStream(path, FileMode.Open, FileAccess.Write, FileShare.ReadWrite);
|
||||
byte[] value = new byte[8];
|
||||
if (write is { } position)
|
||||
{
|
||||
BinaryPrimitives.WriteUInt64LittleEndian(value, position);
|
||||
file.Position = 24; file.Write(value);
|
||||
}
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(value, active ? 1u : 0u);
|
||||
file.Position = 16; file.Write(value, 0, 4);
|
||||
}
|
||||
|
||||
private static void WriteScreenPcm(string path)
|
||||
{
|
||||
using var file = new FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite);
|
||||
Span<byte> index = stackalloc byte[8];
|
||||
file.Position = 24; file.ReadExactly(index);
|
||||
ulong write = BinaryPrimitives.ReadUInt64LittleEndian(index);
|
||||
byte[] pcm = new byte[960 * 2 * sizeof(short)];
|
||||
for (int sample = 0; sample < pcm.Length / sizeof(short); sample++)
|
||||
BinaryPrimitives.WriteInt16LittleEndian(pcm.AsSpan(sample * sizeof(short)),
|
||||
(short)(sample % 64 < 32 ? 6000 : -6000));
|
||||
int start = (int)(write % 96_000), first = Math.Min(pcm.Length / sizeof(short), 96_000 - start);
|
||||
file.Position = 64 + start * sizeof(short);
|
||||
file.Write(pcm.AsSpan(0, first * sizeof(short)));
|
||||
if (first < pcm.Length / sizeof(short))
|
||||
{
|
||||
file.Position = 64;
|
||||
file.Write(pcm.AsSpan(first * sizeof(short)));
|
||||
}
|
||||
BinaryPrimitives.WriteUInt64LittleEndian(index, write + (ulong)(pcm.Length / sizeof(short)));
|
||||
file.Position = 24; file.Write(index);
|
||||
}
|
||||
|
||||
private static async Task WaitUntil(Func<bool> condition, string error)
|
||||
{
|
||||
for (int attempt = 0; attempt < 30; attempt++)
|
||||
{
|
||||
if (condition()) return;
|
||||
await Task.Delay(100);
|
||||
}
|
||||
throw new InvalidOperationException(error);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
@@ -18,3 +18,41 @@ internal static class UiHelpers
|
||||
field.AccessibilityLabel = placeholder; return field;
|
||||
}
|
||||
}
|
||||
|
||||
// VoiceOver rebuilds its element tree from scratch on every UITableView.ReloadData. The model
|
||||
// notifies far more often than any list's content actually changes - the microphone level timer
|
||||
// alone ticks at 20 Hz - so reloading unconditionally re-announces the row under an exploring
|
||||
// finger and destroys the element a double tap was aimed at. Reload only when the rendered text
|
||||
// actually differs, and only while the view is on screen, so an off-screen or popped controller
|
||||
// stops reloading instead of holding a subscription for the lifetime of the connection.
|
||||
internal sealed class ListRefresher(UITableViewController owner, Func<string> signature, Action<Action> subscribe, Action<Action> unsubscribe)
|
||||
{
|
||||
// Null means nothing has been rendered yet. A signature is never null, so the first
|
||||
// Refresh after Start or Invalidate always reloads.
|
||||
private string? rendered;
|
||||
private bool observing;
|
||||
|
||||
internal void Start()
|
||||
{
|
||||
if (observing) return;
|
||||
observing = true; subscribe(Refresh); Refresh();
|
||||
}
|
||||
|
||||
internal void Stop()
|
||||
{
|
||||
if (!observing) return;
|
||||
observing = false; unsubscribe(Refresh);
|
||||
}
|
||||
|
||||
// Forces the next Refresh to reload even when the signature is unchanged, for state the
|
||||
// signature cannot see.
|
||||
internal void Invalidate() => rendered = null;
|
||||
|
||||
internal void Refresh()
|
||||
{
|
||||
if (!observing || !owner.IsViewLoaded) return;
|
||||
string next = signature();
|
||||
if (next == rendered) return;
|
||||
rendered = next; owner.TableView.ReloadData();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,10 +10,14 @@ project="$script_dir/../../../native/apple/broadcast/VoiceCatBroadcast.xcodeproj
|
||||
|
||||
mkdir -p "$output"
|
||||
signing=()
|
||||
versioning=(MARKETING_VERSION="${VOICECAT_DISPLAY_VERSION:-0.0.1}")
|
||||
if [[ -n "${VOICECAT_BUILD_NUMBER:-}" ]]; then
|
||||
versioning+=(CURRENT_PROJECT_VERSION="$VOICECAT_BUILD_NUMBER")
|
||||
fi
|
||||
# Both keys must always be set. The extension manifest expands these placeholders, and an
|
||||
# empty CURRENT_PROJECT_VERSION drops CFBundleVersion, which installd rejects. The defaults
|
||||
# match ApplicationDisplayVersion/ApplicationVersion in VoiceCat.iOS.csproj so that the host
|
||||
# and extension versions agree when no release build number is supplied.
|
||||
versioning=(
|
||||
MARKETING_VERSION="${VOICECAT_DISPLAY_VERSION:-0.0.1}"
|
||||
CURRENT_PROJECT_VERSION="${VOICECAT_BUILD_NUMBER:-1}"
|
||||
)
|
||||
if [[ "$sdk" == "iphonesimulator" ]]; then
|
||||
signing+=(CODE_SIGNING_ALLOWED=NO)
|
||||
elif [[ -n "${VOICECAT_BROADCAST_CODESIGN_KEY:-}" && -n "${VOICECAT_BROADCAST_CODESIGN_PROVISION:-}" ]]; then
|
||||
|
||||
@@ -162,8 +162,11 @@ Before packaging, confirm:
|
||||
- both entitlements contain the shared App Group;
|
||||
- both have `get-task-allow=false` and `beta-reports-active=true`;
|
||||
- the bundle identifiers are the stable identifiers above;
|
||||
- both version pairs match; and
|
||||
- both executables contain `arm64`.
|
||||
- both version pairs match;
|
||||
- both executables contain `arm64`; and
|
||||
- the host `Info.plist` contains `CFBundleIconName` nested inside both `CFBundleIcons` and
|
||||
`CFBundleIcons~ipad`, which is where `actool` writes it and not at the top level, and the
|
||||
bundle contains both `AppIcon60x60@2x.png` (120x120) and `AppIcon76x76@2x~ipad.png` (152x152).
|
||||
|
||||
Also decode each embedded profile and verify its name, UUID, application identifier, and
|
||||
`get-task-allow` value:
|
||||
@@ -232,6 +235,34 @@ Xcode command-line builds can report a missing `Xcode-Token` even when the GUI d
|
||||
account. Manual App Store Connect profiles avoid that dependency and make the two-target signing
|
||||
inputs explicit.
|
||||
|
||||
### Missing app icon or `CFBundleIconName`
|
||||
|
||||
App Store Connect rejects uploads with errors 90022, 90023, and 90713 when the host bundle has
|
||||
no `CFBundleIconName`. Local `codesign` cannot catch this; only upload validation does.
|
||||
|
||||
The asset catalog alone is not enough. `Info.plist` must name it:
|
||||
|
||||
```xml
|
||||
<key>XSAppIconAssets</key><string>Assets.xcassets/AppIcon.appiconset</string>
|
||||
```
|
||||
|
||||
This is an **app-manifest key**, not an MSBuild property. The `ReadAppManifest` task reads
|
||||
`XSAppIconAssets` out of `Info.plist` and passes `--app-icon` to `actool`; setting a csproj
|
||||
property of the same name has no effect. The build strips the key from the shipped manifest.
|
||||
|
||||
Without it, `actool` still compiles the catalog but emits an empty
|
||||
`obj/<config>/<tfm>/<rid>/actool/partial-info.plist`, so no icon keys reach the app. Verify the
|
||||
build inputs rather than the bundle, since stale intermediates can leave icon PNGs from an
|
||||
earlier build in place while the manifest has no icon keys:
|
||||
|
||||
```bash
|
||||
cat clients/apple/VoiceCat.iOS/obj/Release/net10.0-ios27.0/ios-arm64/actool/partial-info.plist
|
||||
```
|
||||
|
||||
A correct build writes `CFBundleIcons` and `CFBundleIcons~ipad` there. A single 1024x1024
|
||||
universal entry in `AppIcon.appiconset` is sufficient; `actool` derives the 120x120 and 152x152
|
||||
variants.
|
||||
|
||||
### App and extension versions differ
|
||||
|
||||
Apple validates nested bundle metadata. Confirm the host does not hard-code version keys, the
|
||||
@@ -240,8 +271,9 @@ clean build regenerated the app manifests.
|
||||
|
||||
## Last verified release build
|
||||
|
||||
On 2026-09-22, version `0.0.1`, build `2026092202` was built with .NET 10.0.401 and Xcode 27.0.
|
||||
On 2026-09-27, version `0.0.1`, build `2026092701` was built with .NET 10.0.401 and Xcode 27.0.
|
||||
The host and ReplayKit extension passed strict nested-signature validation with App Store Connect
|
||||
profiles, matching distribution identities, matching versions, the shared App Group, and
|
||||
`get-task-allow=false`. The resulting IPA was packaged locally; Apple server-side upload
|
||||
validation remains a separate gate.
|
||||
`get-task-allow=false`. The host carries `CFBundleIconName` with the 120x120 and 152x152 icons.
|
||||
The IPA at `dist/VoiceCat-0.0.1-2026092701.ipa` passed archive integrity checks; Apple server-side
|
||||
upload validation remains a separate gate.
|
||||
|
||||
@@ -68,6 +68,12 @@ need direct native entry points. These are platform adapters, not a second core.
|
||||
- `proto/voicecat.proto` is the control-plane schema.
|
||||
- Media uses the fixed header and AEAD construction described in `protocol.md` and
|
||||
`security.md`.
|
||||
- Media frame types are a versioned contract: `Voice`, `Keepalive`, `UdpBinding`, and `Rebind`.
|
||||
`UdpBinding` establishes a peer's endpoint once, in the clear. `Rebind` moves an established
|
||||
endpoint after the client's source address changes, as on a Wi-Fi/cellular handover; it
|
||||
carries the binding token in the clear for peer lookup only, and authorization comes from the
|
||||
AEAD tag over header and token plus the peer's replay window, so a captured rebind cannot be
|
||||
replayed to redirect someone else's downlink.
|
||||
- SQLite is the server's persistent store; schema changes require explicit migrations.
|
||||
- Client profiles and TOFU pins are local platform data.
|
||||
- The ReplayKit ring layout is separately versioned and frozen.
|
||||
|
||||
+66
-5
@@ -101,6 +101,28 @@ be a shell sandbox or keychain-access artifact. Run the identity check from an o
|
||||
session before changing certificates. A successful build prints the selected Apple Development
|
||||
identity, provisioning profile, bundle ID, and app ID before linking.
|
||||
|
||||
### Install fails with `MissingBundleVersion`
|
||||
|
||||
CoreDevice error 3002 with `does not have a CFBundleVersion key with a non-zero length string
|
||||
value` means the ReplayKit extension was built without `CURRENT_PROJECT_VERSION`. The extension
|
||||
manifest expands that placeholder, and an empty value drops `CFBundleVersion` entirely, which
|
||||
installd rejects.
|
||||
|
||||
`build-broadcast-extension.sh` now always passes both version placeholders, defaulting to `1`
|
||||
and `0.0.1` to match `ApplicationVersion` and `ApplicationDisplayVersion` in the csproj, so a
|
||||
plain Debug deployment needs no release environment variables. If this reappears, confirm the
|
||||
script is not making `CURRENT_PROJECT_VERSION` conditional again, and check the staged bundle:
|
||||
|
||||
```bash
|
||||
/usr/libexec/PlistBuddy -c 'Print :CFBundleVersion' \
|
||||
dist/ios-managed-device/VoiceCat.iOS.app/PlugIns/VoiceCatBroadcast.appex/Info.plist
|
||||
```
|
||||
|
||||
### Install fails on a dropped connection
|
||||
|
||||
CoreDevice error 4000 with `Connection reset by peer` is a transport failure, not a bundle or
|
||||
signing problem. Rerun the deployment with `--no-build`.
|
||||
|
||||
### Developer disk image cannot be mounted
|
||||
|
||||
CoreDevice errors 10003 or 12040 mean the phone locked. Unlock it, keep the display awake, and
|
||||
@@ -111,10 +133,49 @@ CoreDevice can also install the app successfully and then reject only the launch
|
||||
deployment; rebuilding is unnecessary. Occasional CoreDeviceService initialization timeouts while
|
||||
listing devices or processes do not imply that an already-confirmed install or launch failed.
|
||||
|
||||
## Simulator connection and audio gate
|
||||
|
||||
The Debug app has an opt-in simulator smoke gate. Start a local VoiceCat server and create an
|
||||
account named `smoke`, then build the `iossimulator-arm64` app. The simulator build needs an
|
||||
App Group entitlement for the saved-password and screen-ring checks; sign the built app
|
||||
ad hoc with `clients/apple/VoiceCat.iOS/Entitlements.plist` before installing it. A clean
|
||||
build avoids stale simulator AOT modules after managed code changes.
|
||||
|
||||
```bash
|
||||
dotnet restore clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj \
|
||||
-p:RuntimeIdentifier=iossimulator-arm64 -p:VoiceCatIosStatic=true --force-evaluate
|
||||
dotnet clean clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj -c Debug \
|
||||
-p:RuntimeIdentifier=iossimulator-arm64
|
||||
dotnet build clients/apple/VoiceCat.iOS/VoiceCat.iOS.csproj -c Debug --no-restore \
|
||||
-p:RuntimeIdentifier=iossimulator-arm64
|
||||
codesign --force --sign - --entitlements clients/apple/VoiceCat.iOS/Entitlements.plist \
|
||||
clients/apple/VoiceCat.iOS/bin/Debug/net10.0-ios27.0/iossimulator-arm64/VoiceCat.iOS.app
|
||||
xcrun simctl install booted \
|
||||
clients/apple/VoiceCat.iOS/bin/Debug/net10.0-ios27.0/iossimulator-arm64/VoiceCat.iOS.app
|
||||
```
|
||||
|
||||
Grant microphone access with `xcrun simctl privacy booted grant microphone
|
||||
me.iamtalon.voicecat`. Launch with `SIMCTL_CHILD_VOICECAT_SIM_SMOKE` set to
|
||||
`127.0.0.1:<port>:<smoke-account-password>` and
|
||||
`SIMCTL_CHILD_VOICECAT_SIM_SMOKE_MODE=share` using `xcrun simctl launch --console-pty booted
|
||||
me.iamtalon.voicecat`. The `share` mode checks that a stale ring stays unmapped, then verifies
|
||||
that screen PCM captured before stream negotiation reaches the encoder. It also exercises
|
||||
continuous synthetic screen audio with the microphone off and during a voice call, voice join,
|
||||
disconnect, and reconnect. Look for
|
||||
`VC_SIM phase=screenSharePassed` and `VC_SIM phase=passed`. Terminate the app, then launch
|
||||
again with mode `resume` to verify the saved account after a process restart. The simulator
|
||||
gate also accepts mode `stereo` to select the StereoMicrophone preset before login and voice join;
|
||||
the simulator exposes only a mono microphone, so check stereo capture on an iPhone. It also
|
||||
does not validate Bluetooth hardware, the iOS 27 ScreenCaptureKit picker, or suspension on
|
||||
a physical device.
|
||||
|
||||
The simulator restore can rewrite `packages.ios.lock.json`; retain both checked-in iOS
|
||||
runtime entries when reviewing the diff.
|
||||
|
||||
## Verified hardware result
|
||||
|
||||
On 2026-09-21, the Debug build completed with .NET 10.0.401 and Xcode 27.0. The host and
|
||||
ReplayKit extension were signed under team `FJV8L966W4`, installed wirelessly on Talon's iPhone,
|
||||
and launched as `me.iamtalon.voicecat`; the running UI was confirmed on the device. Audio,
|
||||
background/lock behavior, Bluetooth, ReplayKit, ScreenCaptureKit, and VoiceOver remain separate
|
||||
manual hardware gates.
|
||||
On 2026-09-22, the Debug build completed with .NET 10.0.401 and Xcode 27.0. The host and
|
||||
ReplayKit extension were signed under team `FJV8L966W4` with matching versions, installed on
|
||||
Talon's iPhone (iPhone 16 Pro Max), and launched as `me.iamtalon.voicecat`; the process was
|
||||
confirmed running on the device. Audio, background/lock behavior, Bluetooth, ReplayKit,
|
||||
ScreenCaptureKit, and VoiceOver remain separate manual hardware gates.
|
||||
|
||||
@@ -23,6 +23,7 @@ public sealed class AdaptivePcmBuffer
|
||||
|
||||
public int Channels => channels;
|
||||
public int CountFrames => unchecked(Volatile.Read(ref writtenFrame) - Volatile.Read(ref readFrame));
|
||||
public int TargetFrames => Volatile.Read(ref targetFrames);
|
||||
public int BufferMilliseconds
|
||||
{
|
||||
get => Volatile.Read(ref targetFrames) * 1000 / SampleRate;
|
||||
@@ -50,6 +51,19 @@ public sealed class AdaptivePcmBuffer
|
||||
finally { Volatile.Write(ref producer, 0); }
|
||||
}
|
||||
|
||||
// A consumer that stalls leaves its producer backlog queued at a fixed offset forever: feed
|
||||
// and drain rates are equal in steady state, so the correction above only sheds roughly half
|
||||
// a percent at a time and the queue ratchets toward its hard edge one stall at a time. The
|
||||
// consumer calls this after a detected stall so the backlog becomes one bounded gap instead
|
||||
// of accumulating; queued audio beyond the target is dropped and the next read re-primes.
|
||||
public void Resynchronize()
|
||||
{
|
||||
int written = Volatile.Read(ref writtenFrame), available = unchecked(written - Volatile.Read(ref readFrame));
|
||||
int target = Volatile.Read(ref targetFrames);
|
||||
if (available > target) Volatile.Write(ref readFrame, unchecked(written - target));
|
||||
primed = false; phase = 0;
|
||||
}
|
||||
|
||||
// Returns interleaved samples written. A zero return means the caller should treat the
|
||||
// already-cleared destination as silence. Once primed, short scheduling stalls re-prime
|
||||
// instead of repeatedly clicking at the ring edge.
|
||||
|
||||
@@ -166,6 +166,24 @@ public sealed class AudioEngine : IDisposable
|
||||
Volatile.Write(ref completedEpoch, current.Epoch);
|
||||
}
|
||||
|
||||
// Device-clocked pull entry for platforms whose audio callback is the only reliable cadence:
|
||||
// iOS keeps render callbacks running in the background while sleep-paced threads coalesce
|
||||
// there. The device callback owns this call, so failures are contained instead of thrown and
|
||||
// later cycles stay inert. Constructed with startWorker: false, the caller alone paces it.
|
||||
public void RunCycle()
|
||||
{
|
||||
if (Failure is not null || Volatile.Read(ref disposed) != 0) return;
|
||||
try { ProcessCycle(); }
|
||||
catch (Exception exception) { Failure = exception; stop.Cancel(); }
|
||||
}
|
||||
|
||||
// Drop each capture backlog to its buffer target. The consumer calls this after a stall it
|
||||
// cannot catch up in place: one bounded gap instead of a queue that ratchets toward its edge.
|
||||
public void ResynchronizeInputs()
|
||||
{
|
||||
foreach (LocalStream stream in Volatile.Read(ref routes).Local) stream.Input.Resynchronize();
|
||||
}
|
||||
|
||||
private void Work()
|
||||
{
|
||||
long deadline = Stopwatch.GetTimestamp();
|
||||
@@ -177,7 +195,10 @@ public sealed class AudioEngine : IDisposable
|
||||
deadline += Stopwatch.Frequency / 50;
|
||||
WaitUntil(deadline);
|
||||
if ((deadline - Stopwatch.GetTimestamp()) * 1000.0 / Stopwatch.Frequency < -100)
|
||||
{
|
||||
deadline = Stopwatch.GetTimestamp();
|
||||
ResynchronizeInputs();
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception exception) { Failure = exception; stop.Cancel(); }
|
||||
|
||||
@@ -25,7 +25,11 @@ internal sealed class ReceiveStream : IDisposable
|
||||
private readonly byte[][] jitter;
|
||||
private readonly uint[] timestamps;
|
||||
private readonly int[] sizes;
|
||||
private int count, available, offset, missing, waiting;
|
||||
private int count, available, offset, missing, waiting, stretchCooldown;
|
||||
// One held frame per half second. Depth still follows a degrading link within a few seconds,
|
||||
// but a target pinned at its cap can no longer trade a steady stream of concealment against
|
||||
// depth it will never reach, nor oscillate against CatchUp's trim.
|
||||
private const int StretchCooldownFrames = 25;
|
||||
private uint expected;
|
||||
private bool started, hasTimestamp;
|
||||
private bool hasMarker;
|
||||
@@ -40,6 +44,8 @@ internal sealed class ReceiveStream : IDisposable
|
||||
internal int ConcealedFrames { get; private set; }
|
||||
internal int DredFrames { get; private set; }
|
||||
internal int FecFrames { get; private set; }
|
||||
internal int Overruns { get; private set; }
|
||||
internal int Stretches { get; private set; }
|
||||
|
||||
internal ReceiveStream(uint userId, StreamInfo info, TimeProvider? clock = null)
|
||||
{
|
||||
@@ -60,14 +66,22 @@ internal sealed class ReceiveStream : IDisposable
|
||||
internal bool Enqueue(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
|
||||
{
|
||||
int index = written;
|
||||
if (payload.Length is < 1 or > 1275 || unchecked(index - Volatile.Read(ref read)) >= 64) return false;
|
||||
if (payload.Length is < 1 or > 1275) return false;
|
||||
// A stalled consumer (an interrupted or rebuilding iOS graph) stops draining this
|
||||
// handoff. Refusing new packets would hold a ring of audio that is already too old to
|
||||
// play and discard the live talkspurt instead, so the newest always wins and the
|
||||
// consumer — which alone owns `read` — notices the overrun and skips forward.
|
||||
int slot = index & 63; payload.CopyTo(packets[slot]); headers[slot] = header; lengths[slot] = payload.Length; arrivals[slot] = clock.GetTimestamp();
|
||||
Volatile.Write(ref written, unchecked(index + 1)); return true;
|
||||
}
|
||||
|
||||
private void Drain()
|
||||
{
|
||||
while (read != Volatile.Read(ref written))
|
||||
int end = Volatile.Read(ref written);
|
||||
// Leave a margin below the producer rather than resuming exactly 64 back, so a write
|
||||
// during this drain cannot lap the slot being copied.
|
||||
if (unchecked(end - read) > 64) { read = unchecked(end - 32); Overruns++; }
|
||||
while (read != end)
|
||||
{
|
||||
int source = read & 63; uint timestamp = headers[source].Timestamp;
|
||||
if (!hasTimestamp) { hasTimestamp = true; expected = timestamp; }
|
||||
@@ -78,8 +92,16 @@ internal sealed class ReceiveStream : IDisposable
|
||||
DropBefore(timestamp); available = offset = missing = waiting = 0;
|
||||
expected = timestamp; started = false; delta = 0; lastArrival = 0; jitterSamples = 0;
|
||||
}
|
||||
// Observed before the acceptance test below: arrival statistics describe the network,
|
||||
// not what this buffer could use. Measuring only accepted packets let a buffer that
|
||||
// was too shallow reject the very late arrivals that should have deepened it.
|
||||
ObserveArrival(timestamp, arrivals[source]);
|
||||
bool duplicate = false;
|
||||
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == timestamp) duplicate = true;
|
||||
// Frames behind `expected` are unplayable: it is the playout clock and never moves
|
||||
// backward, so a late arrival would sit here forever and, as the oldest entry, would
|
||||
// also mask the future packet that DRED and FEC recover from. Depth, not late
|
||||
// tolerance, is what absorbs reordering here.
|
||||
if ((!started || delta >= 0) && delta % frameSamples == 0 && !duplicate)
|
||||
{
|
||||
if (count >= maximumDepth)
|
||||
@@ -89,7 +111,6 @@ internal sealed class ReceiveStream : IDisposable
|
||||
int target = Array.IndexOf(sizes, 0);
|
||||
timestamps[target] = timestamp; sizes[target] = lengths[source];
|
||||
packets[source].AsSpan(0, lengths[source]).CopyTo(jitter[target]); count++;
|
||||
ObserveArrival(timestamp, arrivals[source]);
|
||||
}
|
||||
Volatile.Write(ref read, unchecked(read + 1));
|
||||
}
|
||||
@@ -100,8 +121,10 @@ internal sealed class ReceiveStream : IDisposable
|
||||
if (lastArrival != 0)
|
||||
{
|
||||
int timestampDelta = unchecked((int)(timestamp - lastArrivalTimestamp));
|
||||
if (timestampDelta <= 0) return;
|
||||
if (timestampDelta > 0 && timestampDelta <= frameSamples * 10)
|
||||
// A reordered packet arrives with a negative timestamp delta, and that is precisely
|
||||
// the arrival the target depth has to absorb. Ignoring it left reordering invisible
|
||||
// to the estimator. Gaps far beyond a frame are talkspurt silence, not jitter.
|
||||
if (Math.Abs(timestampDelta) <= frameSamples * 10)
|
||||
{
|
||||
double arrivalDelta = clock.GetElapsedTime(lastArrival, arrival).TotalSeconds * 48_000;
|
||||
double deviation = Math.Abs(arrivalDelta - timestampDelta);
|
||||
@@ -111,9 +134,12 @@ internal sealed class ReceiveStream : IDisposable
|
||||
lastArrival = arrival; lastArrivalTimestamp = timestamp;
|
||||
}
|
||||
|
||||
// The playout target never drops below a single frame. A zero target starts playout on one
|
||||
// packet with no depth at all, so ordinary reordering becomes concealment even when nothing
|
||||
// was actually lost; recovery modes need a further frame of lookahead on top of that floor.
|
||||
private int TargetSamples()
|
||||
{
|
||||
int recovery = Info.Audio.Dred || Info.Audio.Fec ? frameSamples : 0;
|
||||
int recovery = Info.Audio.Dred || Info.Audio.Fec ? frameSamples * 2 : frameSamples;
|
||||
int variation = checked((int)Math.Ceiling(4 * jitterSamples / frameSamples)) * frameSamples;
|
||||
return Math.Min(5760, recovery + variation);
|
||||
}
|
||||
@@ -155,6 +181,23 @@ internal sealed class ReceiveStream : IDisposable
|
||||
int found = -1;
|
||||
for (int i = 0; i < sizes.Length; i++) if (sizes[i] != 0 && timestamps[i] == expected) { found = i; break; }
|
||||
bool decoded = false;
|
||||
// The playout clock advances one frame per call, so a target that grows mid-call has no
|
||||
// way to deepen the buffer again. Hold the clock for one frame — concealing instead of
|
||||
// consuming — so the standing depth can follow a link that has become jittery. Only when
|
||||
// the expected frame is actually present, otherwise a loss gap would stall playout, and
|
||||
// CatchUp's trim threshold sits two frames above this so the two cannot oscillate.
|
||||
if (stretchCooldown > 0) stretchCooldown--;
|
||||
if (started && found >= 0 && count > 0 && stretchCooldown == 0)
|
||||
{
|
||||
int ahead = Newest();
|
||||
if (unchecked((int)(timestamps[ahead] - expected)) + frameSamples < TargetSamples())
|
||||
{
|
||||
stretchCooldown = StretchCooldownFrames;
|
||||
if (!decoder.TryDecode([], pcm, frameSamples, out _)) pcm.AsSpan(0, frameSamples * channels).Clear();
|
||||
ConcealedFrames++; Stretches++;
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (found >= 0)
|
||||
{
|
||||
decoded = decoder.TryDecode(jitter[found].AsSpan(0, sizes[found]), pcm, frameSamples, out int result) && result == frameSamples;
|
||||
|
||||
@@ -10,7 +10,9 @@ public delegate void EncodedVoiceHandler(VoiceFrameHeader header, ReadOnlySpan<b
|
||||
|
||||
internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
{
|
||||
private readonly Socket socket;
|
||||
private Socket socket;
|
||||
private readonly IPEndPoint endpoint;
|
||||
private readonly byte[] token = new byte[MediaEncryptor.RebindTokenSize];
|
||||
private readonly MediaSessionCrypto crypto;
|
||||
private readonly CancellationTokenSource stop;
|
||||
private readonly byte[] binding = new byte[VoiceFrameHeader.Size + 16];
|
||||
@@ -19,6 +21,15 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
private readonly Thread sending;
|
||||
private readonly Task receiving;
|
||||
private readonly TaskCompletionSource bound = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
private readonly AutoResetEvent sendReady = new(false);
|
||||
private int senderNapping;
|
||||
private long lastInbound;
|
||||
// A media path that has gone quiet for longer than this is treated as lost: the phone has
|
||||
// most likely changed interface, which strands a connected UDP socket on a dead source
|
||||
// address. Two missed keepalive echoes.
|
||||
private const int RecoveryIdleMilliseconds = 5_000;
|
||||
private const int KeepaliveMilliseconds = 2_000;
|
||||
internal int Migrations { get; private set; }
|
||||
internal event EncodedVoiceHandler? Received;
|
||||
internal Task Bound => bound.Task;
|
||||
|
||||
@@ -26,10 +37,13 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
{
|
||||
if (token.Length != 16) throw new IOException("Invalid UDP binding token.");
|
||||
this.crypto = crypto;
|
||||
this.endpoint = endpoint;
|
||||
token.CopyTo(this.token.AsSpan());
|
||||
stop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||
socket = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
|
||||
try { socket.Connect(endpoint); }
|
||||
catch { socket.Dispose(); stop.Dispose(); throw; }
|
||||
lastInbound = Environment.TickCount64;
|
||||
new VoiceFrameHeader(MediaFrameType.UdpBinding, 0, 0, 0, 0, 0).Write(binding);
|
||||
token.CopyTo(binding.AsSpan(VoiceFrameHeader.Size));
|
||||
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
|
||||
@@ -38,38 +52,74 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
sending.Start();
|
||||
}
|
||||
|
||||
internal bool TrySend(VoiceFrameHeader header, ReadOnlySpan<byte> payload) => packets.TryWrite(header, payload);
|
||||
internal bool TrySend(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
|
||||
{
|
||||
if (!packets.TryWrite(header, payload)) return false;
|
||||
// Wake the sender only when it is actually waiting. The flag is raised before its final
|
||||
// queue check, so a write can never slip between that check and the wait unnoticed.
|
||||
if (Volatile.Read(ref senderNapping) != 0) sendReady.Set();
|
||||
return true;
|
||||
}
|
||||
|
||||
private void Send()
|
||||
{
|
||||
byte[] plain = new byte[1275], packet = new byte[1275 + VoiceFrameHeader.Size + MediaEncryptor.TagSize];
|
||||
byte[] rebind = new byte[MediaEncryptor.RebindSize];
|
||||
long nextKeepalive = 0;
|
||||
try
|
||||
{
|
||||
while (!stop.IsCancellationRequested)
|
||||
{
|
||||
bool drained = false;
|
||||
try
|
||||
{
|
||||
// A bound path that stops echoing keepalives has been lost under us. The
|
||||
// socket is connected, so it is still pinned to a source address that may no
|
||||
// longer exist; rebuild it and prove possession of the media key from the
|
||||
// new one so the relay moves this peer's endpoint.
|
||||
if (bound.Task.IsCompleted && Environment.TickCount64 - Volatile.Read(ref lastInbound) > RecoveryIdleMilliseconds)
|
||||
{
|
||||
Rebuild();
|
||||
int size = crypto.Encryptor.EncryptRebind(token, rebind);
|
||||
Volatile.Read(ref socket).Send(rebind.AsSpan(0, size), SocketFlags.None);
|
||||
Migrations++;
|
||||
Volatile.Write(ref lastInbound, Environment.TickCount64);
|
||||
nextKeepalive = 0;
|
||||
}
|
||||
if (Environment.TickCount64 >= nextKeepalive)
|
||||
{
|
||||
if (!bound.Task.IsCompleted) socket.Send(binding, SocketFlags.None);
|
||||
socket.Send(keepalive, SocketFlags.None);
|
||||
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? 5000 : 250);
|
||||
Socket current = Volatile.Read(ref socket);
|
||||
if (!bound.Task.IsCompleted) current.Send(binding, SocketFlags.None);
|
||||
current.Send(keepalive, SocketFlags.None);
|
||||
nextKeepalive = Environment.TickCount64 + (bound.Task.IsCompleted ? KeepaliveMilliseconds : 250);
|
||||
}
|
||||
while (packets.TryRead(plain, out VoiceFrameHeader header, out int length))
|
||||
{
|
||||
drained = true;
|
||||
int size = crypto.Encryptor.Encrypt(header, plain.AsSpan(0, length), packet);
|
||||
socket.Send(packet.AsSpan(0, size), SocketFlags.None);
|
||||
Volatile.Read(ref socket).Send(packet.AsSpan(0, size), SocketFlags.None);
|
||||
}
|
||||
}
|
||||
catch (SocketException exception) when (IsTransientNetworkError(exception))
|
||||
{
|
||||
// iOS can briefly lose its UDP route while Wi-Fi and cellular switch.
|
||||
// Keep the sender and socket alive so the next route can carry media.
|
||||
// iOS loses its UDP route while Wi-Fi and cellular switch. A connected socket
|
||||
// stays bound to the vanished source address, so retrying on it never
|
||||
// recovers; replace it and let the idle check above re-offer the binding.
|
||||
Rebuild();
|
||||
nextKeepalive = 0;
|
||||
Thread.Sleep(100);
|
||||
}
|
||||
Thread.Sleep(1);
|
||||
if (drained) continue;
|
||||
// Wait on the queue's signal instead of polling every millisecond. A backgrounded
|
||||
// iOS app coalesces timed sleeps, and a polling sender then flushes queued voice in
|
||||
// bursts; a signalled wake is immediate. The keepalive deadline bounds the wait.
|
||||
Volatile.Write(ref senderNapping, 1);
|
||||
try
|
||||
{
|
||||
if (packets.IsEmpty && Environment.TickCount64 < nextKeepalive)
|
||||
sendReady.WaitOne(checked((int)Math.Clamp(nextKeepalive - Environment.TickCount64, 1, 5000)));
|
||||
}
|
||||
finally { Volatile.Write(ref senderNapping, 0); }
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is SocketException or ObjectDisposedException)
|
||||
@@ -77,6 +127,18 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
finally { stop.Cancel(); }
|
||||
}
|
||||
|
||||
// Replaces the UDP socket so the next send leaves over whichever interface is now current.
|
||||
// Only the sender thread rebuilds; the receive loop picks the new socket up on its next read.
|
||||
private void Rebuild()
|
||||
{
|
||||
if (stop.IsCancellationRequested) return;
|
||||
Socket replacement = new(endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
|
||||
try { replacement.Connect(endpoint); }
|
||||
catch { replacement.Dispose(); return; }
|
||||
Socket previous = Interlocked.Exchange(ref socket, replacement);
|
||||
previous.Dispose();
|
||||
}
|
||||
|
||||
private async Task ReceiveAsync()
|
||||
{
|
||||
byte[] packet = new byte[65535], plain = new byte[65535];
|
||||
@@ -85,10 +147,14 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
while (true)
|
||||
{
|
||||
int length;
|
||||
try { length = await socket.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
|
||||
Socket current = Volatile.Read(ref socket);
|
||||
try { length = await current.ReceiveAsync(packet, SocketFlags.None, stop.Token).ConfigureAwait(false); }
|
||||
catch (SocketException exception) when (exception.SocketErrorCode is SocketError.ConnectionReset or SocketError.MessageSize) { continue; }
|
||||
catch (SocketException exception) when (IsTransientNetworkError(exception))
|
||||
{ await Task.Delay(100, stop.Token).ConfigureAwait(false); continue; }
|
||||
// The sender replaced the socket under us during a migration; read the new one.
|
||||
catch (ObjectDisposedException) when (!stop.IsCancellationRequested && !ReferenceEquals(current, Volatile.Read(ref socket))) { continue; }
|
||||
Volatile.Write(ref lastInbound, Environment.TickCount64);
|
||||
if (!VoiceFrameHeader.TryRead(packet.AsSpan(0, length), out var candidate)) continue;
|
||||
if (candidate.Type == MediaFrameType.Keepalive && length == VoiceFrameHeader.Size) { bound.TrySetResult(); continue; }
|
||||
if (candidate.Type != MediaFrameType.Voice || candidate.Codec != 0 ||
|
||||
@@ -107,9 +173,14 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
stop.Cancel(); socket.Dispose();
|
||||
stop.Cancel(); Volatile.Read(ref socket).Dispose(); sendReady.Set();
|
||||
try { sending.Join(); await receiving.ConfigureAwait(false); }
|
||||
finally { System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding); stop.Dispose(); }
|
||||
finally
|
||||
{
|
||||
System.Security.Cryptography.CryptographicOperations.ZeroMemory(binding);
|
||||
System.Security.Cryptography.CryptographicOperations.ZeroMemory(token);
|
||||
stop.Dispose(); sendReady.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
// A bounded, allocation-free packet handoff. A contending producer drops instead of
|
||||
@@ -120,6 +191,7 @@ internal sealed class ClientMediaTransport : IAsyncDisposable
|
||||
private readonly VoiceFrameHeader[] headers = new VoiceFrameHeader[64];
|
||||
private readonly int[] lengths = new int[64];
|
||||
private int read, written, producer;
|
||||
internal bool IsEmpty => read == Volatile.Read(ref written);
|
||||
internal bool TryWrite(VoiceFrameHeader header, ReadOnlySpan<byte> payload)
|
||||
{
|
||||
if (payload.Length is < 1 or > 1275 || Interlocked.CompareExchange(ref producer, 1, 0) != 0) return false;
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
namespace VoiceCat.Core;
|
||||
|
||||
/// Decides whether an observed network path change is a genuine handover — one that strands the
|
||||
/// existing sockets on a source address that no longer exists — or something the connection
|
||||
/// should be left alone to ride out.
|
||||
///
|
||||
/// The distinction matters because the response is a reconnect. Roaming between access points,
|
||||
/// a tunnel, a minute of unusable cellular: those keep the same interface, so TCP survives them
|
||||
/// and a reconnect would turn a recoverable glitch into a visible drop. Only the set of
|
||||
/// interfaces carrying the path changing is reported as a handover.
|
||||
///
|
||||
/// Platform-agnostic on purpose: the caller supplies whatever stable interface identity its OS
|
||||
/// reports (on iOS, NWPath's interface type and name).
|
||||
public sealed class ControlPathWatcher
|
||||
{
|
||||
private readonly object gate = new();
|
||||
private string signature = "";
|
||||
|
||||
/// Returns true when the path moved to a different set of interfaces and is usable again.
|
||||
/// The first usable path only establishes a baseline; there is nothing to hand over from.
|
||||
public bool Observe(bool usable, IReadOnlyList<string> interfaces)
|
||||
{
|
||||
// An unusable path is an outage, not a handover. Hold the last signature so a link that
|
||||
// returns on the same interface stays silent, and so a reconnect is never started into a
|
||||
// path that cannot carry it.
|
||||
if (!usable) return false;
|
||||
string[] sorted = [.. interfaces.Where(item => !string.IsNullOrEmpty(item))];
|
||||
if (sorted.Length == 0) return false;
|
||||
Array.Sort(sorted, StringComparer.Ordinal);
|
||||
string current = string.Join(",", sorted);
|
||||
lock (gate)
|
||||
{
|
||||
string previous = signature;
|
||||
if (previous == current) return false;
|
||||
signature = current;
|
||||
return previous.Length != 0;
|
||||
}
|
||||
}
|
||||
|
||||
/// Forgets the baseline, so the next usable path establishes a new one without reporting.
|
||||
public void Reset() { lock (gate) signature = ""; }
|
||||
}
|
||||
@@ -42,6 +42,37 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
|
||||
private uint adaptiveLossChannel;
|
||||
private int adaptiveLossPercent = -1;
|
||||
private ClientConnectionState state;
|
||||
private long lastControlInbound;
|
||||
// A control connection that stops answering is dead even though the socket still looks open.
|
||||
// A phone that changes interface leaves TCP blackholed rather than reset, and the OS will not
|
||||
// report it for minutes, so liveness is judged here instead.
|
||||
private TimeSpan controlKeepaliveInterval = TimeSpan.FromSeconds(10);
|
||||
private TimeSpan controlSilenceTimeout = TimeSpan.FromSeconds(30);
|
||||
|
||||
// Instance scoped so tests can shorten the window without disturbing parallel tests, and so
|
||||
// a mobile client can tighten it: on a phone the interval is the delay between a handover and
|
||||
// the reconnect that follows it, which a desktop on one fixed interface never pays.
|
||||
// Takes effect on the next ConnectAsync; the running keepalive worker keeps its own values.
|
||||
public void ConfigureControlLiveness(TimeSpan keepalive, TimeSpan silenceTimeout)
|
||||
{
|
||||
if (keepalive <= TimeSpan.Zero) throw new ArgumentOutOfRangeException(nameof(keepalive));
|
||||
if (silenceTimeout <= keepalive) throw new ArgumentOutOfRangeException(nameof(silenceTimeout), "Silence timeout must exceed the keepalive interval.");
|
||||
controlKeepaliveInterval = keepalive; controlSilenceTimeout = silenceTimeout;
|
||||
}
|
||||
|
||||
internal void SetControlLiveness(TimeSpan keepalive, TimeSpan silenceTimeout)
|
||||
{ controlKeepaliveInterval = keepalive; controlSilenceTimeout = silenceTimeout; }
|
||||
|
||||
// The platform can know the path is gone long before an unanswered keepalive proves it: iOS
|
||||
// reports an interface change the instant it happens. Failing the connection here hands the
|
||||
// existing disconnect path a real cause and starts the reconnect immediately instead of
|
||||
// waiting out the silence timeout on a route that is already dead.
|
||||
public void DropForReconnect(string reason)
|
||||
{
|
||||
if (State == ClientConnectionState.Disconnected) return;
|
||||
ConnectionFailure ??= new IOException(reason);
|
||||
connectionLifetime?.Cancel();
|
||||
}
|
||||
|
||||
public event Action<ClientConnectionState>? ConnectionStateChanged;
|
||||
public ClientConnectionState State { get { lock (stateGate) return state; } }
|
||||
@@ -55,12 +86,15 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
|
||||
public bool TryReadEvent(out Envelope? envelope) => events.Reader.TryRead(out envelope);
|
||||
public IAsyncEnumerable<Envelope> ReadEventsAsync(CancellationToken cancellationToken = default) => events.Reader.ReadAllAsync(cancellationToken);
|
||||
|
||||
public VoiceCatClient(string clientName = "VoiceCat .NET", string clientVersion = "0.1.0", string? tofuStorePath = null)
|
||||
// deviceClockedAudio: the platform's audio callback drives Audio.RunCycle() itself (iOS
|
||||
// keeps render callbacks running in the background while sleep-paced threads coalesce there),
|
||||
// so the audio engine must not start its own pacing worker.
|
||||
public VoiceCatClient(string clientName = "VoiceCat .NET", string clientVersion = "0.1.0", string? tofuStorePath = null, bool deviceClockedAudio = false)
|
||||
{
|
||||
this.clientName = clientName;
|
||||
this.clientVersion = clientVersion;
|
||||
pins = new(tofuStorePath ?? Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "VoiceCat", "tofu.txt"));
|
||||
Audio = new(TrySendEncodedVoice);
|
||||
Audio = new(TrySendEncodedVoice, startWorker: !deviceClockedAudio);
|
||||
VoiceReceived += Audio.Receive;
|
||||
}
|
||||
|
||||
@@ -175,8 +209,20 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
|
||||
{
|
||||
try
|
||||
{
|
||||
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(10));
|
||||
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false)) Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
|
||||
Volatile.Write(ref lastControlInbound, Environment.TickCount64);
|
||||
using var timer = new PeriodicTimer(controlKeepaliveInterval);
|
||||
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
// Every server reply counts as liveness, so a busy session never trips this; an
|
||||
// unanswered ping is what exposes a path that has stopped carrying anything.
|
||||
if (Environment.TickCount64 - Volatile.Read(ref lastControlInbound) > controlSilenceTimeout.TotalMilliseconds)
|
||||
{
|
||||
ConnectionFailure ??= new IOException("Server stopped responding on the control connection.");
|
||||
connectionLifetime?.Cancel();
|
||||
break;
|
||||
}
|
||||
Send(new() { Ping = new() { Nonce = checked((ulong)Environment.TickCount64) } });
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or IOException or InvalidOperationException) { }
|
||||
}
|
||||
@@ -209,13 +255,17 @@ public sealed partial class VoiceCatClient : IAsyncDisposable
|
||||
{
|
||||
await foreach (Envelope message in connection.ReadAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
Volatile.Write(ref lastControlInbound, Environment.TickCount64);
|
||||
Apply(message);
|
||||
if (message.RequestId != 0 && pending.TryRemove(message.RequestId, out var completion)) completion.TrySetResult(message.Clone());
|
||||
if (!events.Writer.TryWrite(message.Clone())) throw new IOException("Client event queue exhausted; consume events regularly.");
|
||||
if (message.Disconnect is not null) { connection.CompleteWrites(); break; }
|
||||
}
|
||||
}
|
||||
catch (Exception exception) { failure = exception; ConnectionFailure = exception; }
|
||||
// A liveness failure has already recorded the real cause and cancelled this read, so do
|
||||
// not replace it with the cancellation it produced.
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) { }
|
||||
catch (Exception exception) { failure = exception; ConnectionFailure ??= exception; }
|
||||
finally
|
||||
{
|
||||
connectionLifetime?.Cancel();
|
||||
|
||||
@@ -25,20 +25,42 @@ public sealed class MediaDecryptor : IDisposable
|
||||
if (packet.Overlaps(plaintext)) throw new ArgumentException("Input and output must not overlap.", nameof(plaintext));
|
||||
VoiceFrameHeader.TryRead(packet, out var candidate);
|
||||
ulong sequence = candidate.Sequence;
|
||||
if (initialized && sequence <= highestSequence)
|
||||
{
|
||||
ulong offset = highestSequence - sequence;
|
||||
if (offset >= 64 || (replayWindow & (1UL << (int)offset)) != 0) return false;
|
||||
}
|
||||
if (IsReplay(sequence)) return false;
|
||||
if (!cipher.TryDecrypt(sequence, packet[VoiceFrameHeader.Size..], packet[..VoiceFrameHeader.Size], plaintext[..length])) return false;
|
||||
Accept(sequence);
|
||||
header = candidate;
|
||||
bytesWritten = length;
|
||||
return true;
|
||||
}
|
||||
|
||||
// Verifies an endpoint-migration frame: header, plaintext binding token, and tag. The token
|
||||
// is authenticated as additional data, so only the media key holder can move an endpoint,
|
||||
// and the shared replay window makes a captured frame useless to an on-path observer.
|
||||
public bool TryVerifyRebind(ReadOnlySpan<byte> packet, out ReadOnlySpan<byte> token)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(disposed, this);
|
||||
token = default;
|
||||
if (packet.Length != MediaEncryptor.RebindSize) return false;
|
||||
if (!VoiceFrameHeader.TryRead(packet, out var candidate) || candidate.Type != MediaFrameType.Rebind) return false;
|
||||
if (IsReplay(candidate.Sequence)) return false;
|
||||
int aad = VoiceFrameHeader.Size + MediaEncryptor.RebindTokenSize;
|
||||
if (!cipher.TryDecrypt(candidate.Sequence, packet[aad..], packet[..aad], [])) return false;
|
||||
Accept(candidate.Sequence);
|
||||
token = packet.Slice(VoiceFrameHeader.Size, MediaEncryptor.RebindTokenSize);
|
||||
return true;
|
||||
}
|
||||
|
||||
private bool IsReplay(ulong sequence)
|
||||
{
|
||||
if (!initialized || sequence > highestSequence) return false;
|
||||
ulong offset = highestSequence - sequence;
|
||||
return offset >= 64 || (replayWindow & (1UL << (int)offset)) != 0;
|
||||
}
|
||||
|
||||
// Only authenticated counters may move the replay window.
|
||||
if (!initialized)
|
||||
private void Accept(ulong sequence)
|
||||
{
|
||||
highestSequence = sequence;
|
||||
replayWindow = 1;
|
||||
initialized = true;
|
||||
}
|
||||
if (!initialized) { highestSequence = sequence; replayWindow = 1; initialized = true; }
|
||||
else if (sequence > highestSequence)
|
||||
{
|
||||
ulong shift = sequence - highestSequence;
|
||||
@@ -46,9 +68,6 @@ public sealed class MediaDecryptor : IDisposable
|
||||
highestSequence = sequence;
|
||||
}
|
||||
else replayWindow |= 1UL << (int)(highestSequence - sequence);
|
||||
header = candidate;
|
||||
bytesWritten = length;
|
||||
return true;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
|
||||
@@ -9,6 +9,8 @@ public sealed class MediaEncryptor : IDisposable
|
||||
private bool disposed;
|
||||
|
||||
public const int TagSize = 16;
|
||||
public const int RebindTokenSize = 16;
|
||||
public const int RebindSize = VoiceFrameHeader.Size + RebindTokenSize + TagSize;
|
||||
|
||||
public MediaEncryptor(ReadOnlySpan<byte> key) : this(key, false) { }
|
||||
|
||||
@@ -31,6 +33,23 @@ public sealed class MediaEncryptor : IDisposable
|
||||
return size;
|
||||
}
|
||||
|
||||
// A rebind frame proves possession of the media key from a new source address. The token
|
||||
// is plaintext so the relay can find the peer without trialling every key; it is covered by
|
||||
// the AEAD as additional data, and the counter makes a captured frame unreplayable.
|
||||
public int EncryptRebind(ReadOnlySpan<byte> token, Span<byte> packet)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(disposed, this);
|
||||
if (token.Length != RebindTokenSize) throw new ArgumentException("Binding tokens contain 16 bytes.", nameof(token));
|
||||
ArgumentOutOfRangeException.ThrowIfLessThan(packet.Length, RebindSize);
|
||||
if (nextSequence == ulong.MaxValue) throw new InvalidOperationException("Media counter exhausted; establish a new session.");
|
||||
var header = new VoiceFrameHeader(MediaFrameType.Rebind, VoiceFrameFlags.None, 0, 0, nextSequence++, 0);
|
||||
header.Write(packet);
|
||||
token.CopyTo(packet.Slice(VoiceFrameHeader.Size, RebindTokenSize));
|
||||
int aad = VoiceFrameHeader.Size + RebindTokenSize;
|
||||
cipher.Encrypt(header.Sequence, [], packet[..aad], packet.Slice(aad, TagSize));
|
||||
return RebindSize;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (disposed) return;
|
||||
|
||||
@@ -6,7 +6,10 @@ public enum MediaFrameType : byte
|
||||
{
|
||||
Voice = 1,
|
||||
Keepalive = 2,
|
||||
UdpBinding = 3
|
||||
UdpBinding = 3,
|
||||
// Authenticated endpoint migration. Carries the peer's binding token in the clear for
|
||||
// lookup only; the AEAD tag and replay window are what authorize the move.
|
||||
Rebind = 4
|
||||
}
|
||||
|
||||
[Flags]
|
||||
|
||||
@@ -4,6 +4,7 @@ using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using System.Security.Cryptography;
|
||||
using System.Threading.Channels;
|
||||
using VoiceCat.Crypto;
|
||||
using VoiceCat.Protocol;
|
||||
using PacketLossMode = Voicecat.V1.PacketLossMode;
|
||||
|
||||
@@ -31,6 +32,7 @@ internal sealed class MediaRelay : IAsyncDisposable
|
||||
private readonly Channel<byte> changed = Channel.CreateBounded<byte>(1);
|
||||
private MediaRoute[] routes = [];
|
||||
private readonly byte[] input = new byte[65535];
|
||||
private readonly byte[] rebindAck = new byte[VoiceFrameHeader.Size];
|
||||
private readonly MediaFanout fanout = new();
|
||||
private readonly Task receiving;
|
||||
internal Task Completion => receiving;
|
||||
@@ -91,6 +93,30 @@ internal sealed class MediaRelay : IAsyncDisposable
|
||||
foreach (MediaRoute route in current)
|
||||
if (route.Peer.Endpoint?.Equals(sender) == true) { source = route; break; }
|
||||
|
||||
// A client whose media source address changed (a phone moving between Wi-Fi and
|
||||
// cellular) keeps its TLS session but arrives here from an unknown address. The
|
||||
// binding token alone travels in the clear, so it may only locate the peer; the
|
||||
// authenticated tag and the peer's replay window are what authorize the move.
|
||||
if (header.Type == MediaFrameType.Rebind)
|
||||
{
|
||||
if (length != MediaEncryptor.RebindSize) continue;
|
||||
// Locate the peer by token before verifying, so a flood of forged rebinds
|
||||
// costs one authentication attempt rather than one per connected peer.
|
||||
MediaPeer? claimed = null;
|
||||
foreach (MediaRoute route in current)
|
||||
if (CryptographicOperations.FixedTimeEquals(route.Peer.Token, input.AsSpan(VoiceFrameHeader.Size, MediaEncryptor.RebindTokenSize)))
|
||||
{ claimed = route.Peer; break; }
|
||||
if (claimed is null || !claimed.Crypto.Decryptor.TryVerifyRebind(input.AsSpan(0, length), out _)) continue;
|
||||
var moved = new SocketAddress(sender.Family, sender.Size);
|
||||
for (int index = 0; index < sender.Size; index++) moved[index] = sender[index];
|
||||
claimed.Endpoint = moved;
|
||||
claimed.Activity.Touch();
|
||||
// Echo a keepalive so the client learns its new path is carrying media.
|
||||
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(rebindAck);
|
||||
await SendAsync(rebindAck, sender).ConfigureAwait(false);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (header.Type == MediaFrameType.UdpBinding)
|
||||
{
|
||||
if (length != VoiceFrameHeader.Size + 16 || source is not null) continue;
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using VoiceCat.Audio;
|
||||
using VoiceCat.Codec;
|
||||
using VoiceCat.Core;
|
||||
using VoiceCat.Crypto;
|
||||
using VoiceCat.Protocol;
|
||||
using VoiceCat.Transport;
|
||||
using Voicecat.V1;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
@@ -23,11 +28,68 @@ public class AudioEngineTests
|
||||
string source = File.ReadAllText(Path.Combine(FindRoot(), "src", "VoiceCat.Core", "ClientMediaTransport.cs"));
|
||||
|
||||
Assert.Contains("new Thread(Send)", source);
|
||||
Assert.Contains("socket.Send(packet.AsSpan(0, size)", source);
|
||||
Assert.Contains("Volatile.Read(ref socket).Send(packet.AsSpan(0, size)", source);
|
||||
Assert.DoesNotContain("Task.Delay(5", source);
|
||||
Assert.DoesNotContain("SendAsync(packet.AsMemory", source);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MediaSenderWakesOnQueueSignalInsteadOfTimedPolling()
|
||||
{
|
||||
string source = File.ReadAllText(Path.Combine(FindRoot(), "src", "VoiceCat.Core", "ClientMediaTransport.cs"));
|
||||
|
||||
Assert.Contains("sendReady.WaitOne(", source);
|
||||
Assert.Contains("senderNapping", source);
|
||||
Assert.DoesNotContain("Thread.Sleep(1);", source);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task MediaSenderDeliversQueuedVoicePromptlyWithoutTimedPolling()
|
||||
{
|
||||
// A loopback "relay" answers the binding so the sender's own keepalive pass falls back to
|
||||
// its five second schedule: only the queue signal can then deliver queued voice promptly,
|
||||
// so a lost wake or a polling loop shows up here as a multi-second delay.
|
||||
using var relay = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
|
||||
relay.Bind(new IPEndPoint(IPAddress.Loopback, 0));
|
||||
byte[] key = new byte[32]; Random.Shared.NextBytes(key);
|
||||
using var crypto = new MediaSessionCrypto(new MediaEncryptor(key), new MediaDecryptor(key));
|
||||
await using var transport = new ClientMediaTransport((IPEndPoint)relay.LocalEndPoint!, new byte[16], crypto, CancellationToken.None);
|
||||
|
||||
byte[] datagram = new byte[512];
|
||||
using (var startup = new CancellationTokenSource(TimeSpan.FromSeconds(5)))
|
||||
{
|
||||
SocketReceiveFromResult probe = await relay.ReceiveFromAsync(datagram, SocketFlags.None, new IPEndPoint(IPAddress.Any, 0), startup.Token);
|
||||
Assert.Equal(VoiceFrameHeader.Size + 16, probe.ReceivedBytes);
|
||||
byte[] keepalive = new byte[VoiceFrameHeader.Size];
|
||||
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
|
||||
await relay.SendToAsync(keepalive, SocketFlags.None, probe.RemoteEndPoint);
|
||||
}
|
||||
await transport.Bound.WaitAsync(TimeSpan.FromSeconds(5));
|
||||
|
||||
var stopwatch = System.Diagnostics.Stopwatch.StartNew();
|
||||
Assert.True(transport.TrySend(new(MediaFrameType.Voice, VoiceFrameFlags.None, 0, 7, 0, 0), new byte[80]));
|
||||
await ReceiveVoiceAsync(relay, datagram, 1, TimeSpan.FromSeconds(2));
|
||||
Assert.True(stopwatch.Elapsed < TimeSpan.FromSeconds(2));
|
||||
|
||||
// A burst queued while the sender naps must drain in one wake, not one keepalive pass.
|
||||
stopwatch.Restart();
|
||||
for (uint i = 1; i <= 30; i++)
|
||||
Assert.True(transport.TrySend(new(MediaFrameType.Voice, VoiceFrameFlags.None, 0, 7, 0, i * 960), new byte[80]));
|
||||
await ReceiveVoiceAsync(relay, datagram, 30, TimeSpan.FromSeconds(2));
|
||||
Assert.True(stopwatch.Elapsed < TimeSpan.FromSeconds(2));
|
||||
}
|
||||
|
||||
private static async Task ReceiveVoiceAsync(Socket relay, byte[] datagram, int count, TimeSpan timeout)
|
||||
{
|
||||
int voice = VoiceFrameHeader.Size + 80 + MediaEncryptor.TagSize;
|
||||
using var deadline = new CancellationTokenSource(timeout);
|
||||
while (count > 0)
|
||||
{
|
||||
SocketReceiveFromResult packet = await relay.ReceiveFromAsync(datagram, SocketFlags.None, new IPEndPoint(IPAddress.Any, 0), deadline.Token);
|
||||
if (packet.ReceivedBytes == voice) count--;
|
||||
}
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(-1000)]
|
||||
[InlineData(1000)]
|
||||
@@ -124,6 +186,31 @@ public class AudioEngineTests
|
||||
Assert.Equal(0, lateStarvation);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(20)]
|
||||
[InlineData(40)]
|
||||
[InlineData(120)]
|
||||
public void AdaptivePcmBufferResynchronizeDropsStallBacklogToTheTarget(int bufferMilliseconds)
|
||||
{
|
||||
var buffer = new AdaptivePcmBuffer(1, bufferMilliseconds, 65_536);
|
||||
short[] chunk = new short[960], output = new short[960];
|
||||
chunk.AsSpan().Fill(1234);
|
||||
|
||||
// A stalled consumer leaves every chunk queued: feed and drain rates are equal in steady
|
||||
// state, so this backlog would otherwise sit at a fixed offset until overflow drops begin.
|
||||
for (int i = 0; i < 50; i++) Assert.True(buffer.TryWrite(chunk));
|
||||
Assert.Equal(50 * 960, buffer.CountFrames);
|
||||
|
||||
buffer.Resynchronize();
|
||||
Assert.Equal(bufferMilliseconds * 48, buffer.CountFrames);
|
||||
Assert.Equal(960, buffer.Read(output));
|
||||
Assert.All(output, value => Assert.Equal(1234, value));
|
||||
|
||||
// At or below the target the resynchronization drops nothing.
|
||||
buffer.Resynchronize();
|
||||
Assert.Equal(bufferMilliseconds * 48 - 960, buffer.CountFrames);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OneCaptureMissDoesNotRestartTalkspurtButSustainedStarvationDoes()
|
||||
{
|
||||
@@ -166,8 +253,14 @@ public class AudioEngineTests
|
||||
[InlineData(5)] [InlineData(10)] [InlineData(20)] [InlineData(40)] [InlineData(60)]
|
||||
public void RecoveryLookaheadTracksChannelFrameDuration(int frameMilliseconds)
|
||||
{
|
||||
// One frame of standing depth plus one of recovery lookahead. The depth floor keeps a
|
||||
// quiet link from playing out with no buffer at all, where reordering alone conceals.
|
||||
using var stream = new ReceiveStream(2, Stream(frameMilliseconds));
|
||||
Assert.Equal(frameMilliseconds * 48, stream.TargetDepthSamples);
|
||||
Assert.Equal(frameMilliseconds * 96, stream.TargetDepthSamples);
|
||||
|
||||
StreamInfo plain = Stream(frameMilliseconds); plain.Audio.Fec = false;
|
||||
using var bare = new ReceiveStream(2, plain);
|
||||
Assert.Equal(frameMilliseconds * 48, bare.TargetDepthSamples);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -282,6 +375,99 @@ public class AudioEngineTests
|
||||
Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DeviceClockedRunCyclePacesMixAndCaptureWithNoWorkerThread()
|
||||
{
|
||||
var sent = new List<(uint Timestamp, VoiceFrameFlags Flags)>(512);
|
||||
StreamInfo stream = Stream();
|
||||
using var receive = new AudioEngine((_, _, _, _) => true, false);
|
||||
receive.SetRemoteStreams([new() { Id = 2, ChannelId = 1, Streams = { stream } }], 1, 1);
|
||||
using var send = new AudioEngine((ssrc, timestamp, payload, flags) =>
|
||||
{
|
||||
sent.Add((timestamp, flags));
|
||||
receive.Receive(new(MediaFrameType.Voice, flags, 0, ssrc, 0, timestamp), payload);
|
||||
return true;
|
||||
}, false) { InputMode = AudioInputMode.AlwaysOn, DeviceBufferMilliseconds = 20 };
|
||||
send.AddLocalStream(stream);
|
||||
short[] tone = Tone(); long energy = 0;
|
||||
receive.MixedPcm += pcm => { foreach (short sample in pcm) energy += Math.Abs((int)sample); };
|
||||
|
||||
// Each iteration is one 20 ms render demand quantum: feed the capture handoff, then pull
|
||||
// both mixes, exactly as the iOS render callback drives its device-clocked engine.
|
||||
for (int i = 0; i < 30; i++) { Assert.True(send.FeedPcm(1, tone, 1)); send.RunCycle(); receive.RunCycle(); }
|
||||
Assert.Equal(30, send.GetLocalDiagnostics(1).Cycles);
|
||||
Assert.Equal(0, send.GetLocalDiagnostics(1).StarvedCycles);
|
||||
Assert.Equal(30, sent.Count);
|
||||
Assert.True(energy > 100000);
|
||||
|
||||
long before = GC.GetAllocatedBytesForCurrentThread();
|
||||
for (int i = 0; i < 100; i++) { send.FeedPcm(1, tone, 1); send.RunCycle(); receive.RunCycle(); }
|
||||
Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RunCycleContainsFailuresInsteadOfThrowingIntoRealtimeCallbacks()
|
||||
{
|
||||
using var engine = new AudioEngine((_, _, _, _) => throw new InvalidOperationException("boom"), false)
|
||||
{ InputMode = AudioInputMode.AlwaysOn, DeviceBufferMilliseconds = 20 };
|
||||
engine.AddLocalStream(Stream());
|
||||
Assert.True(engine.FeedPcm(1, Tone(), 1));
|
||||
engine.RunCycle();
|
||||
Assert.IsType<InvalidOperationException>(engine.Failure);
|
||||
engine.RunCycle();
|
||||
Assert.Equal(1, engine.GetLocalDiagnostics(1).Cycles);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DeviceClockedClientLeavesMixCadenceToTheAudioCallback()
|
||||
{
|
||||
string directory = Path.Combine(Path.GetTempPath(), $"voicecat-{Guid.NewGuid():N}");
|
||||
Directory.CreateDirectory(directory);
|
||||
try
|
||||
{
|
||||
await using var paced = new VoiceCatClient(tofuStorePath: Path.Combine(directory, "paced.txt"));
|
||||
await using var pulled = new VoiceCatClient(tofuStorePath: Path.Combine(directory, "pulled.txt"), deviceClockedAudio: true);
|
||||
paced.Audio.AddLocalStream(Stream());
|
||||
pulled.Audio.AddLocalStream(Stream());
|
||||
await Task.Delay(250);
|
||||
Assert.True(paced.Audio.GetLocalDiagnostics(1).Cycles > 0);
|
||||
Assert.Equal(0, pulled.Audio.GetLocalDiagnostics(1).Cycles);
|
||||
pulled.Audio.RunCycle();
|
||||
Assert.Equal(1, pulled.Audio.GetLocalDiagnostics(1).Cycles);
|
||||
}
|
||||
finally
|
||||
{
|
||||
try { Directory.Delete(directory, true); }
|
||||
catch (IOException) { } catch (UnauthorizedAccessException) { }
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ManagedAudioWorkerResynchronizesProducerBacklogAfterStalls()
|
||||
{
|
||||
string source = File.ReadAllText(Path.Combine(FindRoot(), "src", "VoiceCat.Audio", "AudioEngine.cs"));
|
||||
string buffer = File.ReadAllText(Path.Combine(FindRoot(), "src", "VoiceCat.Audio", "AdaptivePcmBuffer.cs"));
|
||||
|
||||
Assert.Contains("stream.Input.Resynchronize()", source);
|
||||
Assert.Contains("public void Resynchronize()", buffer);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ResynchronizeInputsDropsStalledBacklogToTheBufferTarget()
|
||||
{
|
||||
using var send = new AudioEngine((_, _, _, _) => true, false);
|
||||
send.DeviceBufferMilliseconds = 40; send.AddLocalStream(Stream());
|
||||
short[] mono = Tone();
|
||||
for (int i = 0; i < 40; i++) send.FeedPcm(1, mono, 1);
|
||||
Assert.True(send.GetLocalDiagnostics(1).BufferedFrames > 1920 * 2);
|
||||
|
||||
send.ResynchronizeInputs();
|
||||
|
||||
Assert.Equal(1920, send.GetLocalDiagnostics(1).BufferedFrames);
|
||||
send.ProcessCycle();
|
||||
Assert.Equal(0, send.GetLocalDiagnostics(1).StarvedCycles);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RemotePlaybackSettingsAreIndependentForEachUserStream()
|
||||
{
|
||||
@@ -305,7 +491,7 @@ public class AudioEngineTests
|
||||
for (uint i = 0; i < 64; i++) stream.Enqueue(new(MediaFrameType.Voice, 0, 0, 42, i, i * 960), packet.AsSpan(0, length));
|
||||
stream.Mix(output, false, null); Assert.InRange(stream.Depth, 0, 6);
|
||||
for (int i = 0; i < 20; i++) { output.AsSpan().Clear(); stream.Mix(output, false, null); }
|
||||
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 10);
|
||||
Assert.All(output, value => Assert.Equal(0, value)); Assert.InRange(stream.ConcealedFrames, 1, 16);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -39,6 +39,7 @@ public class ManagedClientTests
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(() => Connect(alice, fixture));
|
||||
Assert.Equal(ClientConnectionState.Connected, alice.State);
|
||||
await alice.DisconnectAsync();
|
||||
Assert.Null(alice.ConnectionFailure);
|
||||
await Event(bob, e => e.UserEvent?.Kind == UserEvent.Types.Kind.Left);
|
||||
await Connect(alice, fixture); Assert.True((await alice.AuthenticateGuestAsync("Returned")).Ok);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using VoiceCat.Transport;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using VoiceCat.Crypto;
|
||||
using VoiceCat.Protocol;
|
||||
using VoiceCat.Server.Transport;
|
||||
using Voicecat.V1;
|
||||
@@ -177,7 +178,7 @@ public sealed class MediaRelayTests
|
||||
internal sealed class VoicePeer : IAsyncDisposable
|
||||
{
|
||||
public Client Client { get; }
|
||||
private readonly Socket udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
|
||||
private Socket udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
|
||||
private readonly IPEndPoint endpoint;
|
||||
private readonly MediaSessionCrypto crypto;
|
||||
public ulong LastSequence { get; private set; }
|
||||
@@ -228,6 +229,39 @@ public sealed class MediaRelayTests
|
||||
return packet;
|
||||
}
|
||||
public async Task SendAsync(byte[] packet) => await udp.SendToAsync(packet, SocketFlags.None, endpoint, Client.Timeout.Token);
|
||||
|
||||
// Models a Wi-Fi/cellular handover: the peer keeps its TLS control session but its media
|
||||
// source address changes, then it re-offers its UDP binding token from the new address.
|
||||
internal async Task HandoverAsync()
|
||||
{
|
||||
udp.Dispose();
|
||||
udp = new(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
|
||||
udp.Bind(new IPEndPoint(IPAddress.Loopback, 0));
|
||||
byte[] rebind = new byte[MediaEncryptor.RebindSize];
|
||||
int size = crypto.Encryptor.EncryptRebind(Client.Authentication!.UdpToken.Span, rebind);
|
||||
await SendAsync(rebind[..size]);
|
||||
}
|
||||
|
||||
// A rebind captured off the wire must not let anyone else claim the peer's downlink.
|
||||
internal async Task<byte[]> CaptureRebindAsync()
|
||||
{
|
||||
byte[] rebind = new byte[MediaEncryptor.RebindSize];
|
||||
int size = crypto.Encryptor.EncryptRebind(Client.Authentication!.UdpToken.Span, rebind);
|
||||
await SendAsync(rebind[..size]);
|
||||
return rebind[..size];
|
||||
}
|
||||
|
||||
// Returns true when the relay echoes a keepalive to the peer's current source address,
|
||||
// which is the only signal that the server will route downlink media back to it.
|
||||
internal async Task<bool> KeepaliveEchoesAsync(int timeoutMilliseconds = 1000)
|
||||
{
|
||||
byte[] keepalive = new byte[VoiceFrameHeader.Size];
|
||||
new VoiceFrameHeader(MediaFrameType.Keepalive, 0, 0, 0, 0, 0).Write(keepalive);
|
||||
await SendAsync(keepalive);
|
||||
using var timeout = new CancellationTokenSource(timeoutMilliseconds);
|
||||
try { byte[] buffer = new byte[65535]; await udp.ReceiveAsync(buffer, SocketFlags.None, timeout.Token); return true; }
|
||||
catch (OperationCanceledException) { return false; }
|
||||
}
|
||||
public async Task<byte[]> ReceivePacketAsync()
|
||||
{
|
||||
byte[] buffer = new byte[65535];
|
||||
|
||||
@@ -0,0 +1,401 @@
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using VoiceCat.Audio;
|
||||
using VoiceCat.Core;
|
||||
using VoiceCat.Codec;
|
||||
using VoiceCat.Protocol;
|
||||
using Voicecat.V1;
|
||||
using Xunit;
|
||||
using static VoiceCat.Tests.ServerTests;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
|
||||
// Deterministic network-impairment simulation for the receive path. A virtual millisecond clock
|
||||
// drives an encoder, an impairment model, and the mixer so bursty loss, jitter, reordering,
|
||||
// link outages, and a stalled (backgrounded) consumer are reproducible without hardware.
|
||||
public class NetworkImpairmentTests(ITestOutputHelper output)
|
||||
{
|
||||
// A scheduled packet. Arrival is virtual-clock milliseconds; duplicates share a timestamp.
|
||||
private readonly record struct Wire(int Arrival, uint Sequence, uint Timestamp, bool Marker, int Length, byte[] Payload);
|
||||
|
||||
private sealed class Impairment(int seed)
|
||||
{
|
||||
private readonly Random random = new(seed);
|
||||
private bool bursting;
|
||||
internal double LossPercent, BurstLossPercent, BurstEntryPercent, BurstExitPercent = 30, JitterMs, ReorderPercent, DuplicatePercent;
|
||||
internal int BaseDelayMs = 20, OutageStartMs = -1, OutageEndMs = -1;
|
||||
|
||||
internal bool Dropped(int sendTime)
|
||||
{
|
||||
if (OutageStartMs >= 0 && sendTime >= OutageStartMs && sendTime < OutageEndMs) return true;
|
||||
if (BurstEntryPercent > 0)
|
||||
{
|
||||
bursting = bursting ? random.NextDouble() * 100 >= BurstExitPercent : random.NextDouble() * 100 < BurstEntryPercent;
|
||||
if (bursting) return random.NextDouble() * 100 < BurstLossPercent;
|
||||
}
|
||||
return random.NextDouble() * 100 < LossPercent;
|
||||
}
|
||||
|
||||
internal int Arrival(int sendTime)
|
||||
{
|
||||
double delay = BaseDelayMs + (JitterMs > 0 ? random.NextDouble() * JitterMs : 0);
|
||||
if (ReorderPercent > 0 && random.NextDouble() * 100 < ReorderPercent) delay += 45;
|
||||
return sendTime + (int)delay;
|
||||
}
|
||||
|
||||
internal bool Duplicated() => DuplicatePercent > 0 && random.NextDouble() * 100 < DuplicatePercent;
|
||||
}
|
||||
|
||||
private sealed record Report(string Name, int Frames, int SilentFrames, int LongestSilentRunMs, int Concealed, int Overruns, int Sent, int TargetDepth)
|
||||
{
|
||||
internal double SilentPercent => Frames == 0 ? 0 : SilentFrames * 100.0 / Frames;
|
||||
public override string ToString() =>
|
||||
$"{Name,-28} silent={SilentPercent,5:F1}% worstGap={LongestSilentRunMs,5}ms concealed={Concealed,4} overruns={Overruns,3} sent={Sent,4} target={TargetDepth,4}";
|
||||
}
|
||||
|
||||
// Runs `durationMs` of a 20 ms mono talkspurt through the impairment model. The consumer
|
||||
// pumps the mixer every 20 ms of virtual time except inside a stall window, which models an
|
||||
// iOS render callback that stops being serviced while backgrounded.
|
||||
private Report Simulate(string name, Impairment impairment, int durationMs = 20_000, bool dred = false, bool fec = true,
|
||||
int stallStartMs = -1, int stallEndMs = -1)
|
||||
{
|
||||
StreamInfo info = AudioEngineTests.Stream(20, dred: dred);
|
||||
info.Audio.Fec = fec;
|
||||
var clock = new VirtualClock();
|
||||
using var stream = new ReceiveStream(7, info, clock);
|
||||
using var encoder = new OpusEncoder(new() { Bitrate = 32000, ForwardErrorCorrection = fec, DeepRedundancy = dred, ExpectedPacketLossPercent = 20, Complexity = 5 });
|
||||
var pending = new List<Wire>();
|
||||
short[] tone = new short[960];
|
||||
int frames = 0, silent = 0, run = 0, longest = 0, sent = 0;
|
||||
int[] mix = new int[1920];
|
||||
|
||||
for (int now = 0; now <= durationMs; now += 20)
|
||||
{
|
||||
CodecTests.FillTone(tone, 960, 1, 48000, now / 20);
|
||||
byte[] packet = new byte[1275];
|
||||
int length = encoder.Encode(tone, packet);
|
||||
uint sequence = (uint)(now / 20);
|
||||
if (!impairment.Dropped(now))
|
||||
{
|
||||
pending.Add(new(impairment.Arrival(now), sequence, sequence * 960, sequence == 0, length, packet));
|
||||
if (impairment.Duplicated()) pending.Add(new(impairment.Arrival(now) + 5, sequence, sequence * 960, false, length, packet));
|
||||
}
|
||||
|
||||
clock.Set(now);
|
||||
foreach (Wire wire in pending.Where(w => w.Arrival <= now).OrderBy(w => w.Arrival).ToArray())
|
||||
{
|
||||
var flags = wire.Marker ? VoiceFrameFlags.Marker : VoiceFrameFlags.None;
|
||||
sent++;
|
||||
stream.Enqueue(new(MediaFrameType.Voice, flags, 0, 42, wire.Sequence, wire.Timestamp), wire.Payload.AsSpan(0, wire.Length));
|
||||
pending.Remove(wire);
|
||||
}
|
||||
|
||||
if (stallStartMs >= 0 && now >= stallStartMs && now < stallEndMs) continue;
|
||||
|
||||
mix.AsSpan().Clear();
|
||||
stream.Mix(mix, false, null);
|
||||
frames++;
|
||||
bool quiet = true;
|
||||
foreach (int sample in mix) if (sample != 0) { quiet = false; break; }
|
||||
if (quiet) { silent++; run += 20; longest = Math.Max(longest, run); }
|
||||
else run = 0;
|
||||
}
|
||||
var report = new Report(name, frames, silent, longest, stream.ConcealedFrames, stream.Overruns, sent, stream.TargetDepthSamples);
|
||||
output.WriteLine(report.ToString());
|
||||
return report;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ImpairmentProfileReport()
|
||||
{
|
||||
output.WriteLine("--- FEC on ---");
|
||||
Simulate("clean", new Impairment(1));
|
||||
Simulate("random loss 2%", new Impairment(2) { LossPercent = 2 });
|
||||
Simulate("random loss 10%", new Impairment(3) { LossPercent = 10 });
|
||||
Simulate("bursty loss", new Impairment(4) { BurstEntryPercent = 4, BurstLossPercent = 80, BurstExitPercent = 25 });
|
||||
Simulate("jitter 60ms", new Impairment(5) { JitterMs = 60 });
|
||||
Simulate("jitter 120ms", new Impairment(6) { JitterMs = 120 });
|
||||
Simulate("reorder 5%", new Impairment(7) { ReorderPercent = 5 });
|
||||
Simulate("duplicate 5%", new Impairment(8) { DuplicatePercent = 5 });
|
||||
Simulate("wifi switch 3s outage", new Impairment(9) { OutageStartMs = 6000, OutageEndMs = 9000 });
|
||||
Simulate("bad wifi (loss+jitter)", new Impairment(10) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 });
|
||||
Simulate("background stall 2s", new Impairment(11), stallStartMs: 6000, stallEndMs: 8000);
|
||||
Simulate("background stall 10s", new Impairment(13), stallStartMs: 6000, stallEndMs: 16000);
|
||||
Simulate("stall + jitter", new Impairment(12) { JitterMs = 60 }, stallStartMs: 6000, stallEndMs: 8000);
|
||||
|
||||
output.WriteLine("--- FEC off and DRED off ---");
|
||||
Simulate("nofec clean", new Impairment(21), fec: false);
|
||||
Simulate("nofec loss 2%", new Impairment(22) { LossPercent = 2 }, fec: false);
|
||||
Simulate("nofec loss 10%", new Impairment(23) { LossPercent = 10 }, fec: false);
|
||||
Simulate("nofec jitter 30ms", new Impairment(24) { JitterMs = 30 }, fec: false);
|
||||
Simulate("nofec reorder 5%", new Impairment(25) { ReorderPercent = 5 }, fec: false);
|
||||
Simulate("nofec bad wifi", new Impairment(26) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 }, fec: false);
|
||||
}
|
||||
|
||||
// Bounds are set well above the measured result so ordinary codec variation does not make
|
||||
// them flaky; they exist to catch a structural regression in the receive path, such as the
|
||||
// depth floor or the arrival estimator being lost again.
|
||||
[Theory]
|
||||
// impairment, maxConcealedPercent, maxGapMs
|
||||
[InlineData("loss2", 12)]
|
||||
[InlineData("loss10", 12)]
|
||||
[InlineData("burst", 20)]
|
||||
[InlineData("jitter60", 12)]
|
||||
[InlineData("jitter120", 12)]
|
||||
[InlineData("reorder", 5)]
|
||||
[InlineData("badwifi", 12)]
|
||||
public void ImpairedLinksStayIntelligible(string profile, int maxConcealedPercent)
|
||||
{
|
||||
Impairment impairment = profile switch
|
||||
{
|
||||
"loss2" => new(2) { LossPercent = 2 },
|
||||
"loss10" => new(3) { LossPercent = 10 },
|
||||
"burst" => new(4) { BurstEntryPercent = 4, BurstLossPercent = 80, BurstExitPercent = 25 },
|
||||
"jitter60" => new(5) { JitterMs = 60 },
|
||||
"jitter120" => new(6) { JitterMs = 120 },
|
||||
"reorder" => new(7) { ReorderPercent = 5 },
|
||||
_ => new(10) { LossPercent = 8, JitterMs = 80, ReorderPercent = 3 },
|
||||
};
|
||||
Report report = Simulate(profile, impairment);
|
||||
Assert.True(report.Concealed * 100 / report.Frames <= maxConcealedPercent,
|
||||
$"{profile} concealed {report.Concealed} of {report.Frames} frames.");
|
||||
Assert.True(report.LongestSilentRunMs <= 200, $"{profile} went silent for {report.LongestSilentRunMs} ms.");
|
||||
}
|
||||
|
||||
// Pure reordering loses no data at all, so it must be absorbed by depth rather than concealed.
|
||||
// Without the depth floor and an estimator that observes late arrivals this was 47 frames.
|
||||
[Fact]
|
||||
public void ReorderingWithoutLossIsAbsorbedRatherThanConcealed()
|
||||
{
|
||||
Report report = Simulate("reorder no fec", new Impairment(25) { ReorderPercent = 5 }, fec: false);
|
||||
Assert.Equal(1000, report.Sent);
|
||||
Assert.True(report.Concealed <= 15, $"Concealed {report.Concealed} frames despite losing none.");
|
||||
}
|
||||
|
||||
// A consumer that stops draining (an interrupted or rebuilding iOS graph) must not cost the
|
||||
// live talkspurt. The handoff previously refused new packets while full, discarding 437 of
|
||||
// 1000 packets across a ten second stall.
|
||||
[Fact]
|
||||
public void AStalledConsumerLosesBoundedAudioRatherThanTheLiveTalkspurt()
|
||||
{
|
||||
Report report = Simulate("stall", new Impairment(13), stallStartMs: 6000, stallEndMs: 16000);
|
||||
Assert.Equal(1000, report.Sent);
|
||||
Assert.InRange(report.Overruns, 1, 4);
|
||||
Assert.True(report.LongestSilentRunMs <= 200, $"Silent for {report.LongestSilentRunMs} ms after the stall.");
|
||||
}
|
||||
|
||||
// A Wi-Fi/cellular handover changes the client's media source address while TLS survives.
|
||||
// The relay binds a peer's endpoint once and refuses to move it, and the client stops
|
||||
// offering its binding token after the first bind, so media must not silently die here.
|
||||
[Fact]
|
||||
public async Task MediaSurvivesAHandoverThatChangesTheClientSourceAddress()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var alice = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Alice");
|
||||
await using var bob = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Bob");
|
||||
StreamAnnounceResult stream = await alice.AnnounceAsync(StreamKind.StreamMic);
|
||||
|
||||
await alice.SendAsync(alice.Seal(stream.Ssrc, [1]));
|
||||
await bob.ReceiveVoiceAsync();
|
||||
|
||||
await alice.HandoverAsync();
|
||||
Assert.True(await alice.KeepaliveEchoesAsync(), "Relay stopped routing to Alice after her media source address changed.");
|
||||
|
||||
await alice.SendAsync(alice.Seal(stream.Ssrc, [2]));
|
||||
(_, byte[] payload) = await bob.ReceiveVoiceAsync();
|
||||
Assert.Equal<byte[]>([2], payload);
|
||||
}
|
||||
|
||||
// The rebind token travels in the clear so the relay can locate the peer without trialling
|
||||
// every key. Authorization comes from the AEAD tag and the replay window, so an on-path
|
||||
// observer who captures a rebind must not be able to redirect the peer's downlink.
|
||||
[Fact]
|
||||
public async Task ReplayedRebindFromAnotherAddressCannotStealTheDownlink()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var alice = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Alice");
|
||||
await using var bob = await MediaRelayTests.VoicePeer.ConnectAsync(fixture, "Bob");
|
||||
StreamAnnounceResult stream = await alice.AnnounceAsync(StreamKind.StreamMic);
|
||||
|
||||
byte[] captured = await alice.CaptureRebindAsync();
|
||||
Assert.True(await alice.KeepaliveEchoesAsync());
|
||||
|
||||
using var attacker = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
|
||||
attacker.Bind(new IPEndPoint(IPAddress.Loopback, 0));
|
||||
await attacker.SendToAsync(captured, SocketFlags.None, fixture.Server.MediaEndPoint);
|
||||
using (var timeout = new CancellationTokenSource(500))
|
||||
{
|
||||
byte[] buffer = new byte[65535];
|
||||
await Assert.ThrowsAnyAsync<OperationCanceledException>(async () =>
|
||||
await attacker.ReceiveAsync(buffer, SocketFlags.None, timeout.Token));
|
||||
}
|
||||
|
||||
// Alice still owns the path, so her media keeps flowing to Bob.
|
||||
await alice.SendAsync(alice.Seal(stream.Ssrc, [9]));
|
||||
(_, byte[] payload) = await bob.ReceiveVoiceAsync();
|
||||
Assert.Equal<byte[]>([9], payload);
|
||||
Assert.True(await alice.KeepaliveEchoesAsync());
|
||||
}
|
||||
|
||||
// A phone that changes interface leaves TCP blackholed rather than reset: the socket stays
|
||||
// open and the OS reports nothing for minutes. Only an unanswered keepalive exposes it, and
|
||||
// until it does the app shows a live session over a dead path and never reconnects.
|
||||
[Fact]
|
||||
public async Task ABlackholedControlConnectionIsDetectedInsteadOfAppearingConnected()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var proxy = new BlackholeProxy(fixture.Server.EndPoint);
|
||||
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(fixture.Directory, "tofu.txt"));
|
||||
client.SetControlLiveness(TimeSpan.FromMilliseconds(200), TimeSpan.FromSeconds(2));
|
||||
|
||||
var disconnected = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
client.ConnectionStateChanged += state => { if (state == ClientConnectionState.Disconnected) disconnected.TrySetResult(); };
|
||||
await client.ConnectAsync("127.0.0.1", (ushort)proxy.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
|
||||
await client.AuthenticateGuestAsync("Alice");
|
||||
Assert.Equal(ClientConnectionState.Connected, client.State);
|
||||
|
||||
proxy.Freeze();
|
||||
await disconnected.Task.WaitAsync(TimeSpan.FromSeconds(15));
|
||||
Assert.Equal(ClientConnectionState.Disconnected, client.State);
|
||||
Assert.IsType<IOException>(client.ConnectionFailure);
|
||||
}
|
||||
|
||||
// The reconnect is only worth paying for a path change that TCP genuinely cannot survive.
|
||||
// Walking a house between access points, or a link that is merely unusable for a while, keeps
|
||||
// the same interface and the same source address, and a reconnect there would turn a
|
||||
// recoverable glitch into the visible drop this whole change exists to remove.
|
||||
[Fact]
|
||||
public void OnlyAChangedInterfaceCountsAsAHandover()
|
||||
{
|
||||
var watcher = new ControlPathWatcher();
|
||||
|
||||
// The first usable path is a baseline, not a handover.
|
||||
Assert.False(watcher.Observe(true, ["wifi:en0"]));
|
||||
// Roaming between access points, and the same path reported again.
|
||||
Assert.False(watcher.Observe(true, ["wifi:en0"]));
|
||||
// A dead spot: unusable, then back on the interface it left from.
|
||||
Assert.False(watcher.Observe(false, []));
|
||||
Assert.False(watcher.Observe(false, []));
|
||||
Assert.False(watcher.Observe(true, ["wifi:en0"]));
|
||||
// Wi-Fi gives out and cellular takes over: the source address is gone.
|
||||
Assert.True(watcher.Observe(true, ["cellular:pdp_ip0"]));
|
||||
Assert.False(watcher.Observe(true, ["cellular:pdp_ip0"]));
|
||||
// And back onto Wi-Fi on arriving home.
|
||||
Assert.True(watcher.Observe(true, ["wifi:en0"]));
|
||||
// Interface order is a reporting detail, not a change.
|
||||
Assert.True(watcher.Observe(true, ["wifi:en0", "cellular:pdp_ip0"]));
|
||||
Assert.False(watcher.Observe(true, ["cellular:pdp_ip0", "wifi:en0"]));
|
||||
|
||||
// After a reset the next usable path is a baseline again.
|
||||
watcher.Reset();
|
||||
Assert.False(watcher.Observe(true, ["wired:en5"]));
|
||||
Assert.True(watcher.Observe(true, ["wifi:en0"]));
|
||||
}
|
||||
|
||||
// The platform knows the path changed long before an unanswered keepalive can prove it. A
|
||||
// client told directly must fail the connection at once, with the real cause, so the reconnect
|
||||
// runs while the audio session is still up instead of after the silence timeout expires.
|
||||
[Fact]
|
||||
public async Task ADroppedControlConnectionReconnectsWithoutWaitingOutTheSilenceTimeout()
|
||||
{
|
||||
await using var fixture = new ServerFixture();
|
||||
await using var proxy = new BlackholeProxy(fixture.Server.EndPoint);
|
||||
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(fixture.Directory, "tofu.txt"));
|
||||
client.SetControlLiveness(TimeSpan.FromSeconds(30), TimeSpan.FromMinutes(5));
|
||||
|
||||
var disconnected = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
client.ConnectionStateChanged += state => { if (state == ClientConnectionState.Disconnected) disconnected.TrySetResult(); };
|
||||
await client.ConnectAsync("127.0.0.1", (ushort)proxy.EndPoint.Port, (_, _) => ValueTask.FromResult(true));
|
||||
await client.AuthenticateGuestAsync("Alice");
|
||||
Assert.Equal(ClientConnectionState.Connected, client.State);
|
||||
|
||||
proxy.Freeze();
|
||||
client.DropForReconnect("The network interface changed to cellular:pdp_ip0.");
|
||||
await disconnected.Task.WaitAsync(TimeSpan.FromSeconds(5));
|
||||
Assert.Equal(ClientConnectionState.Disconnected, client.State);
|
||||
Assert.Contains("cellular:pdp_ip0", Assert.IsType<IOException>(client.ConnectionFailure).Message);
|
||||
|
||||
// Dropping an already dead connection is a no-op, not a second failure.
|
||||
client.DropForReconnect("Ignored.");
|
||||
Assert.Contains("cellular:pdp_ip0", client.ConnectionFailure!.Message);
|
||||
}
|
||||
|
||||
// A silence timeout at or below the keepalive interval would fail every healthy connection on
|
||||
// its first tick, so a mobile client tightening these cannot be allowed to invert them.
|
||||
[Fact]
|
||||
public async Task ControlLivenessRejectsAWindowThatCannotBeMet()
|
||||
{
|
||||
await using var client = new VoiceCatClient("Test", "0.0.1", Path.Combine(Path.GetTempPath(), Path.GetRandomFileName()));
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() => client.ConfigureControlLiveness(TimeSpan.Zero, TimeSpan.FromSeconds(12)));
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() => client.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5)));
|
||||
client.ConfigureControlLiveness(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(12));
|
||||
}
|
||||
|
||||
// Forwards TCP both ways until frozen, after which bytes are swallowed and the sockets are
|
||||
// left open — what a vanished route looks like to the client, unlike a close or a reset.
|
||||
private sealed class BlackholeProxy : IAsyncDisposable
|
||||
{
|
||||
private readonly Socket listener = new(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
|
||||
private readonly CancellationTokenSource stop = new();
|
||||
private readonly IPEndPoint origin;
|
||||
private volatile bool frozen;
|
||||
internal IPEndPoint EndPoint { get; }
|
||||
internal void Freeze() => frozen = true;
|
||||
|
||||
internal BlackholeProxy(IPEndPoint origin)
|
||||
{
|
||||
this.origin = origin;
|
||||
listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
|
||||
listener.Listen(4);
|
||||
EndPoint = (IPEndPoint)listener.LocalEndPoint!;
|
||||
_ = AcceptAsync();
|
||||
}
|
||||
|
||||
private async Task AcceptAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
while (!stop.IsCancellationRequested)
|
||||
{
|
||||
Socket inbound = await listener.AcceptAsync(stop.Token);
|
||||
Socket outbound = new(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
|
||||
await outbound.ConnectAsync(origin, stop.Token);
|
||||
_ = PumpAsync(inbound, outbound);
|
||||
_ = PumpAsync(outbound, inbound);
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException) { }
|
||||
}
|
||||
|
||||
private async Task PumpAsync(Socket from, Socket to)
|
||||
{
|
||||
byte[] buffer = new byte[16384];
|
||||
try
|
||||
{
|
||||
while (!stop.IsCancellationRequested)
|
||||
{
|
||||
int length = await from.ReceiveAsync(buffer, SocketFlags.None, stop.Token);
|
||||
if (length == 0) break;
|
||||
if (frozen) continue;
|
||||
await to.SendAsync(buffer.AsMemory(0, length), SocketFlags.None, stop.Token);
|
||||
}
|
||||
}
|
||||
catch (Exception exception) when (exception is OperationCanceledException or SocketException or ObjectDisposedException) { }
|
||||
}
|
||||
|
||||
public ValueTask DisposeAsync()
|
||||
{
|
||||
stop.Cancel(); listener.Dispose(); stop.Dispose();
|
||||
return ValueTask.CompletedTask;
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class VirtualClock : TimeProvider
|
||||
{
|
||||
private long milliseconds;
|
||||
public override long TimestampFrequency => 1000;
|
||||
public override long GetTimestamp() => milliseconds;
|
||||
internal void Set(long value) => milliseconds = value;
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
using System.Diagnostics;
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
namespace VoiceCat.Tests;
|
||||
|
||||
@@ -66,8 +67,11 @@ public class PublishServerScriptTests
|
||||
Assert.Contains("group.me.iamtalon.voicecat", entitlements);
|
||||
Assert.Contains("CODE_SIGN_STYLE=Manual", script);
|
||||
Assert.Contains("PROVISIONING_PROFILE_SPECIFIER=\"$VOICECAT_BROADCAST_CODESIGN_PROVISION\"", script);
|
||||
Assert.Contains("CURRENT_PROJECT_VERSION=\"$VOICECAT_BUILD_NUMBER\"", script);
|
||||
// Both version placeholders must always be passed to xcodebuild. A conditional
|
||||
// CURRENT_PROJECT_VERSION expands to an empty CFBundleVersion, which installd rejects.
|
||||
Assert.Contains("CURRENT_PROJECT_VERSION=\"${VOICECAT_BUILD_NUMBER:-1}\"", script);
|
||||
Assert.Contains("MARKETING_VERSION=\"${VOICECAT_DISPLAY_VERSION:-0.0.1}\"", script);
|
||||
Assert.DoesNotContain("if [[ -n \"${VOICECAT_BUILD_NUMBER:-}\" ]]; then", script);
|
||||
|
||||
string deviceScript = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "build-ios-device.sh"));
|
||||
@@ -81,6 +85,24 @@ public class PublishServerScriptTests
|
||||
Assert.Contains("<string>$(CURRENT_PROJECT_VERSION)</string>", extensionManifest);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosHostDeclaresItsAppIconAssetForAppStoreValidation()
|
||||
{
|
||||
string root = FindRoot();
|
||||
string manifest = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS", "Info.plist"));
|
||||
string iconSet = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS",
|
||||
"Assets.xcassets", "AppIcon.appiconset", "Contents.json"));
|
||||
|
||||
// XSAppIconAssets is an app-manifest key, not an MSBuild property. Without it actool
|
||||
// never receives --app-icon, so the bundle ships without CFBundleIconName and App Store
|
||||
// Connect rejects the upload with errors 90022, 90023, and 90713.
|
||||
Assert.Contains("<key>XSAppIconAssets</key>", manifest);
|
||||
Assert.Contains("<string>Assets.xcassets/AppIcon.appiconset</string>", manifest);
|
||||
Assert.Contains("AppIcon-1024.png", iconSet);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosScreenSharingDeclaresBackgroundCaptureAndKeepsReplayKitFallback()
|
||||
{
|
||||
@@ -96,6 +118,25 @@ public class PublishServerScriptTests
|
||||
Assert.DoesNotContain("if (OperatingSystem.IsIOSVersionAtLeast(27)) { model.ToggleScreenAudio();", settings);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosBroadcastPumpKeepsOneRingMappingAcrossTicks()
|
||||
{
|
||||
string root = FindRoot();
|
||||
string pump = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS", "BroadcastAudioPump.cs"));
|
||||
|
||||
// A mapping pair plus its container lookup and path strings per 5 ms tick allocated
|
||||
// steadily at 200 Hz and churned the GC under long calls. The pump opens one mapping on
|
||||
// the ring's transitions and reuses it; the producer never replaces the ring file.
|
||||
Assert.Single(Regex.Matches(pump, "MemoryMappedFile.CreateFromFile"));
|
||||
Assert.Contains("private bool OpenMapping()", pump);
|
||||
Assert.Contains("private void CloseMapping()", pump);
|
||||
Assert.Contains("Thread.Sleep(view is null ? 100 : 5)", pump);
|
||||
Assert.Contains("File.Exists(path)", pump);
|
||||
Assert.DoesNotContain("using MemoryMappedFile", pump);
|
||||
Assert.DoesNotContain("using MemoryMappedViewAccessor", pump);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosCaptureDoesNotDependOnCoalescedManagedTimers()
|
||||
{
|
||||
@@ -111,6 +152,52 @@ public class PublishServerScriptTests
|
||||
Assert.DoesNotContain("Task.Delay(10", screen);
|
||||
Assert.Contains("MaximumCaptureCallbackFrames = 16_384", microphone);
|
||||
Assert.DoesNotContain("Math.Ceiling(4_096 * 48_000", microphone);
|
||||
|
||||
// Both 20 ms hands-offs run from the AVAudioSourceNode render callback: sleep-paced
|
||||
// threads coalesce when the app is backgrounded and glitch the call after several minutes.
|
||||
string model = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS", "AppModel.cs"));
|
||||
Assert.DoesNotContain("new Thread(", microphone);
|
||||
Assert.DoesNotContain("Thread.Sleep", microphone);
|
||||
Assert.Contains("owner.Audio.RunCycle()", microphone);
|
||||
Assert.Contains("while (microphoneCredit >= 960)", microphone);
|
||||
Assert.Contains("deviceClockedAudio: true", model);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosRecoversBackgroundAudioWithoutForegrounding()
|
||||
{
|
||||
string root = FindRoot();
|
||||
string microphone = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
|
||||
string router = await File.ReadAllTextAsync(Path.Combine(
|
||||
root, "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
|
||||
|
||||
// The render callback is the only clock for the device-clocked pipeline, so a graph that
|
||||
// stops while backgrounded froze capture, mix and send until the app was foregrounded.
|
||||
Assert.Contains("internal long RenderCallbacks", microphone);
|
||||
Assert.Contains("internal bool IsRunning", microphone);
|
||||
Assert.Contains("private void TickWatchdog()", router);
|
||||
Assert.Contains("IosAudioEngine.Shared.RenderCallbacks", router);
|
||||
Assert.Contains("IosAudioEngine.Shared.IsRunning", router);
|
||||
// Bounded: a rebuild that does not restore the callbacks backs off and then stops, so a
|
||||
// graph that cannot start fails visibly instead of churning the session forever.
|
||||
Assert.Contains("IosAudioEngine.Shared.Reconfigure(true, $\"stall watchdog {watchdogAttempts}\")", router);
|
||||
Assert.Contains("watchdogAttempts >= MaximumWatchdogAttempts", router);
|
||||
Assert.Contains("VC_WATCHDOG exhausted", router);
|
||||
|
||||
// SetActive fails for as long as an interruption is in force, so Began suppresses
|
||||
// recovery and the watchdog retries the rebuild that Ended asks for.
|
||||
Assert.Contains("AVAudioSessionInterruptionType.Began", router);
|
||||
Assert.Contains("interrupted = true", router);
|
||||
Assert.Contains("internal void ResumeForeground()", router);
|
||||
|
||||
// A stalled render callback resynchronizes its own rings, and a refill is bounded so it
|
||||
// cannot overrun the callback deadline that it is recovering from.
|
||||
Assert.Contains("owner.Audio.ResynchronizeInputs()", microphone);
|
||||
Assert.Contains("playbackRing.Resynchronize()", microphone);
|
||||
Assert.Contains("Ring.Resynchronize()", microphone);
|
||||
Assert.Contains("Math.Min(frames + playbackRing.TargetFrames - playbackRing.CountFrames, frames + 960)", microphone);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -120,8 +207,11 @@ public class PublishServerScriptTests
|
||||
string router = await File.ReadAllTextAsync(Path.Combine(root, "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
|
||||
string engine = await File.ReadAllTextAsync(Path.Combine(root, "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
|
||||
|
||||
Assert.True(router.IndexOf("ApplyInputSelection(session)", StringComparison.Ordinal) <
|
||||
router.IndexOf("SetActive(true", StringComparison.Ordinal));
|
||||
Assert.True(router.IndexOf("SetActive(true", StringComparison.Ordinal) <
|
||||
router.IndexOf("ApplyInputSelection(session)", StringComparison.Ordinal));
|
||||
Assert.True(router.IndexOf("session.SetPreferredInput(port", StringComparison.Ordinal) <
|
||||
router.IndexOf("port.SetPreferredDataSource(source", StringComparison.Ordinal));
|
||||
Assert.DoesNotContain("session.SetInputDataSource(source", router);
|
||||
Assert.DoesNotContain("session.SetPreferredInputNumberOfChannels", router);
|
||||
Assert.DoesNotContain("MaximumInputNumberOfChannels", router);
|
||||
Assert.Contains("SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait", router);
|
||||
@@ -135,7 +225,66 @@ public class PublishServerScriptTests
|
||||
Assert.True(engine.IndexOf("SetVoiceProcessingEnabled", StringComparison.Ordinal) <
|
||||
engine.IndexOf("next.Connect(source", StringComparison.Ordinal));
|
||||
Assert.Contains("AVAudioEngine.ConfigurationChangeNotification", engine);
|
||||
Assert.Contains("ReferenceEquals(engine, next) && !next.Running", engine);
|
||||
Assert.Contains("if (!IsConnected || !ReferenceEquals(engine, next)) return;", engine);
|
||||
Assert.Contains("if (!next.Running || HardwareChanged()) Rebuild(\"configuration change\");", engine);
|
||||
// With voice processing the input and output are one IO unit, and it only stays up while the
|
||||
// input is rendered. A tap alone leaves it out of the chain and the unit dies seconds later.
|
||||
Assert.Contains("next.Connect(input, captureSink, inputFormat", engine);
|
||||
Assert.Contains("captureSink.OutputVolume = 0f;", engine);
|
||||
// Capture exists for the session, so joining voice names a stream instead of rebuilding.
|
||||
Assert.Contains("internal void StopMicrophone() { Volatile.Write(ref microphone, null); }", engine);
|
||||
Assert.Contains("bool captures = IsConnected;", engine);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task IosSpeakerOutputIsAnOrthogonalOneShotChoice()
|
||||
{
|
||||
string router = await File.ReadAllTextAsync(Path.Combine(
|
||||
FindRoot(), "clients", "apple", "VoiceCat.iOS", "IosAudioRouter.cs"));
|
||||
|
||||
// Speaker output decides where audio goes, not how it is captured, so it must not take the
|
||||
// preset with it, and no preset may clear it back.
|
||||
Assert.Contains("ForceSpeaker = value; speakerIsExplicit = overridePending = true", router);
|
||||
Assert.DoesNotContain("ForceSpeaker = value; Preset = IosAudioPreset.Advanced", router);
|
||||
Assert.DoesNotContain("IosAudioPreset.VoiceChat) ForceSpeaker = false", router);
|
||||
// The stored flag is honoured only once the user has actually chosen, so the value older
|
||||
// installs inherited from the voice-chat preset cannot pin a headset user to the speaker.
|
||||
Assert.Contains("cat.voice.audio.speakerIsExplicit", router);
|
||||
// The port override is the toggle's one-shot request. Re-asserting it on every rebuild
|
||||
// means fighting the system for the route, which the user sees as audio flipping.
|
||||
Assert.Contains("if (overridePending)", router);
|
||||
Assert.Contains("overridePending = false;", router);
|
||||
// Every reconfiguration moves the route and moving the route notifies the handler, so a
|
||||
// route change must never force a rebuild: that is one rebuild per notification, without
|
||||
// end. The graph is rebuilt only when it stopped or when the hardware it was built around
|
||||
// moved, which is the one thing AVAudioEngine cannot absorb by itself.
|
||||
Assert.Contains("if (applying) return;", router);
|
||||
Assert.Contains("Recover($\"route change ({reason})\", force: false)", router);
|
||||
Assert.DoesNotContain("Recover($\"route change ({reason})\");", router);
|
||||
|
||||
string engine = await File.ReadAllTextAsync(Path.Combine(
|
||||
FindRoot(), "clients", "apple", "VoiceCat.iOS", "IosAudioEngine.cs"));
|
||||
// Asked of the graph's own input, not of AVAudioSession and not remembered from a route:
|
||||
// the session's reported rate and channel count do not settle until after the graph has
|
||||
// started, and CurrentRoute still names the previous route for a while, so either one
|
||||
// reports a change that has not happened and every rebuild reports it again.
|
||||
Assert.Contains("live.InputNode.GetBusOutputFormat(0)", engine);
|
||||
Assert.Contains("format.SampleRate != builtInputRate || format.ChannelCount != builtInputChannels", engine);
|
||||
Assert.DoesNotContain("AVAudioSession.SharedInstance().SampleRate", engine);
|
||||
// Every rebuild names what asked for it, so a storm can be read from a log.
|
||||
Assert.Contains("VC_REBUILD cause=", engine);
|
||||
// Building a graph is itself what posts most configuration changes, and the engine reads as
|
||||
// stopped until voice processing has rebuilt the IO. Nothing may judge a graph dead inside
|
||||
// that window, or it replaces the graph it just built with one that reports the same thing.
|
||||
Assert.Contains("internal bool Settling =>", engine);
|
||||
Assert.Contains("if (!force && Settling) return;", engine);
|
||||
Assert.Contains("if (Settling) return;", engine);
|
||||
Assert.Contains("if (Settling) return true;", engine);
|
||||
Assert.Contains("|| IosAudioEngine.Shared.Settling) { ResetWatchdog(); return; }", router);
|
||||
Assert.Contains("&& !HardwareChanged()) return;", engine);
|
||||
Assert.Contains("(!next.Running || HardwareChanged())", engine);
|
||||
Assert.Contains("if (Preset == IosAudioPreset.VoiceChat) { SelectedInputId = null; SelectedDataSourceId = null; }", router);
|
||||
Assert.DoesNotContain("SelectedInputId ??= session.PreferredInput", router);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
Reference in New Issue
Block a user