fix(ios): keep the voice-processing graph up instead of rebuilding it
Build and test / test (macos-latest) (push) Canceled after 0s
Build and test / test (ubuntu-24.04) (push) Canceled after 0s
Build and test / test (windows-latest) (push) Canceled after 0s
Build and test / apple-client (push) Canceled after 0s

Joining voice on the voice-chat preset was unreliable: audio arrived after
several seconds of the route flipping back and forth, sometimes not at all, and
VoiceOver went quiet while it happened. Device logs show why. A graph with
voice processing enabled reports a successful start and is then torn down
within a second, roughly three times in four; every configuration without voice
processing — both microphone presets, and voice chat with processing off — comes
up first time and runs indefinitely.

With voice processing the input and output are one IO unit, and it only stays up
while the input is part of the render chain. The input node carried a tap and no
connection, which leaves it out of that chain. Route the input through a silent
mixer so it is genuinely rendered.

The rest of this is the amplifier rather than the cause, and each part of it
turned one failed start into a storm:

The stall watchdog rebuilt on every missed tick, without bound. That converted a
graph that could not start into endless session reconfiguration, which is what
the user heard and what hid the reason from the log. It now backs off after each
failed attempt and stops after four, logging VC_WATCHDOG exhausted, so a
transient freeze still recovers and a graph that will not start fails visibly.

Nothing waited for a graph to start before judging it dead. Enabling voice
processing rebuilds both halves of the IO, which posts a configuration change
and reads as not running for several hundred milliseconds, so the
configuration-change handler and the watchdog both tore down graphs that were
about to run. A settling window holds them off for two seconds.

A route change forced a full rebuild, and every rebuild moves the route, so one
notification produced the next. Route changes now take the non-forcing path,
which rebuilds a stopped graph and leaves a healthy one alone; the hardware test
it uses reads the input node's format, not AVAudioSession, whose reported rate
and channel count do not settle until after the graph has started.

The input side is built once per session instead of being added when voice is
joined, so joining and leaving voice set a stream id rather than replacing the
graph, and a mono voice-chat apply no longer clears a stereo capsule
configuration it never applied.

Every rebuild now logs its cause, and VC_START/VC_START_CHECK record whether the
graph survived its start. The first-attempt failure is not fixed and is recorded
in PROGRESS.md as a release gate: capture still comes up on a watchdog rebuild
rather than immediately.

The changed logic sits on AVAudioSession and AVAudioEngine, which the net10.0
test project cannot reference, so the behaviour is covered by the existing
source assertions; verification is on device.
This commit is contained in:
2026-09-25 20:56:31 +02:00
parent 1a0ff957ae
commit 01bae734b8
6 changed files with 229 additions and 135 deletions
+51 -27
View File
@@ -19,13 +19,24 @@ internal sealed class IosAudioRouter
private readonly NSUserDefaults defaults = NSUserDefaults.StandardUserDefaults;
private bool applying;
private bool speakerIsExplicit;
private readonly VoiceCat.Core.AudioRouteWatcher route = new();
// Whether this session actually has a stereo capsule configuration to undo. Clearing one that
// was never applied is not free: SetPreferredPolarPattern(Unknown) drops the built-in array out
// of the beamformed mono configuration VoiceChat mode selects and exposes its four raw
// channels, which the voice-processing IO cannot start against.
private bool stereoApplied;
// Set by an explicit speaker choice and consumed by the next Apply. The override is a one-shot
// request, never steady-state configuration; see SetForceSpeaker.
private bool overridePending;
private System.Threading.Timer? watchdog;
private long lastRenderCallbacks = -1;
private int watchdogMisses;
private int watchdogAttempts;
private long nextWatchdogAttempt;
// A rebuild that does not restore the callbacks is not worth repeating at the same rate, and
// never worth repeating forever: an unbounded retry turns a graph that cannot start into a
// storm of session reconfiguration, which is both what the user hears and what hides the
// reason from the log. Back off, then stop and say so.
private const int MaximumWatchdogAttempts = 4;
private bool watchdogTicking;
private bool interrupted;
internal event Action? Changed;
@@ -151,27 +162,16 @@ internal sealed class IosAudioRouter
? AVAudioSessionPortOverride.Speaker : AVAudioSessionPortOverride.None, out _);
}
RefreshRoutes();
// Last, so it describes the route this graph is being built on rather than the one it
// replaced. HandleRouteChange compares against it to recognise its own echo.
route.Built(RouteSignature(session));
ResetWatchdog(); EnsureWatchdog();
}
finally { applying = false; }
}
// Identifies the hardware carrying audio. A graph has to be rebuilt when this changes, because
// the input format changes with it; when it has not changed, there is nothing to rebuild for.
private static string RouteSignature(AVAudioSession session)
{
AVAudioSessionRouteDescription current = session.CurrentRoute;
return string.Join('|', current.Inputs.Select(value => value.UID).Concat(current.Outputs.Select(value => value.UID)));
}
private void ApplyInputSelection(AVAudioSession session)
{
AVAudioSessionPortDescription? port = session.AvailableInputs?.FirstOrDefault(value => value.UID == SelectedInputId);
if (CaptureChannels == 2) port ??= session.AvailableInputs?.FirstOrDefault(value => value.PortType == AVAudioSession.PortBuiltInMic);
if (port is null) { if (CaptureChannels == 1) ClearStereoPolarPattern(session); return; }
if (port is null) { if (CaptureChannels == 1 && stereoApplied) ClearStereo(session); return; }
if (CaptureChannels == 2)
{
// Polar-pattern discovery alone is insufficient on current iPhones: until a stereo
@@ -181,11 +181,12 @@ internal sealed class IosAudioRouter
// it also gives Core Audio an unambiguous left/right mapping before graph creation.
if (!session.SetPreferredInputOrientation(AVAudioStereoOrientation.Portrait, out NSError? orientationError))
throw new InvalidOperationException(orientationError?.LocalizedDescription ?? "Could not set the stereo microphone orientation.");
stereoApplied = true;
}
AVAudioSessionDataSourceDescription? source = CaptureChannels == 2
? port.DataSources?.FirstOrDefault(SupportsStereoPolarPattern)
: port.DataSources?.FirstOrDefault(value => value.DataSourceID.ToString() == SelectedDataSourceId);
if (CaptureChannels == 1 && source is null) ClearStereoPolarPattern(session);
if (CaptureChannels == 1 && source is null && stereoApplied) ClearStereo(session);
if (source is not null)
{
if (!port.SetPreferredDataSource(source, out NSError? portError)) throw new InvalidOperationException(portError.LocalizedDescription);
@@ -236,6 +237,8 @@ internal sealed class IosAudioRouter
internal static extern byte SetObject(NativeHandle receiver, NativeHandle selector, NativeHandle value, ref NativeHandle error);
}
private void ClearStereo(AVAudioSession session) { stereoApplied = false; ClearStereoPolarPattern(session); }
private static void ClearStereoPolarPattern(AVAudioSession session)
{
session.SetPreferredInputOrientation(AVAudioStereoOrientation.None, out _);
@@ -253,14 +256,15 @@ internal sealed class IosAudioRouter
defaults.SetBool(speakerIsExplicit, "cat.voice.audio.speakerIsExplicit");
defaults.SetBool(VoiceProcessing, "cat.voice.audio.voiceProcessing"); defaults.SetBool(AutomaticGainControl, "cat.voice.audio.agc"); defaults.SetInt(CaptureChannels, "cat.voice.audio.captureChannels");
Set("cat.voice.audio.inputPortId", SelectedInputId); Set("cat.voice.audio.dataSourceId", SelectedDataSourceId); defaults.SetString(SelectedPolarPattern.ToString(), "cat.voice.audio.polarPattern"); defaults.Synchronize();
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(); Changed?.Invoke();
ResetWatchdogAttempts();
if (IosAudioEngine.Shared.IsConnected) IosAudioEngine.Shared.Reconfigure(true, "settings"); Changed?.Invoke();
}
private void Set(string key, string? value) { if (value is null) defaults.RemoveObject(key); else defaults.SetString(value, key); }
internal void Deactivate()
{
// A released session has no route the next graph can be compared against.
watchdog?.Dispose(); watchdog = null; ResetWatchdog(); route.Reset();
watchdog?.Dispose(); watchdog = null; ResetWatchdog(); ResetWatchdogAttempts();
AVAudioSession.SharedInstance().SetActive(false, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out _);
}
internal void EnsureAudio(string reason)
@@ -276,7 +280,7 @@ internal sealed class IosAudioRouter
if (!IosAudioEngine.Shared.IsConnected || interrupted) return;
UIApplication.SharedApplication.BeginInvokeOnMainThread(() =>
{
try { IosAudioEngine.Shared.Reconfigure(force); }
try { IosAudioEngine.Shared.Reconfigure(force, reason); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio recovery ({reason}) failed: {exception}"); }
});
}
@@ -291,6 +295,7 @@ internal sealed class IosAudioRouter
}
private void ResetWatchdog() { lastRenderCallbacks = -1; watchdogMisses = 0; }
private void ResetWatchdogAttempts() { watchdogAttempts = 0; nextWatchdogAttempt = 0; }
// An interruption that ends while the app is suspended never delivers its Ended notification,
// so foregrounding still has to check. It must not rebuild unconditionally though: the `audio`
@@ -310,15 +315,29 @@ internal sealed class IosAudioRouter
watchdogTicking = true;
try
{
if (!IosAudioEngine.Shared.IsConnected || interrupted || applying) { ResetWatchdog(); return; }
// A graph still starting up reports no callbacks yet and reads as not running. Judging
// it there is how the watchdog ends up rebuilding a healthy graph on every tick.
if (!IosAudioEngine.Shared.IsConnected || interrupted || applying || IosAudioEngine.Shared.Settling) { ResetWatchdog(); return; }
long callbacks = IosAudioEngine.Shared.RenderCallbacks;
bool stalled = !IosAudioEngine.Shared.IsRunning || callbacks == lastRenderCallbacks;
lastRenderCallbacks = callbacks;
if (!stalled) { watchdogMisses = 0; return; }
if (!stalled) { watchdogMisses = 0; ResetWatchdogAttempts(); return; }
// One missed tick can be a route change already rebuilding the graph.
if (++watchdogMisses < 2) return;
if (Environment.TickCount64 < nextWatchdogAttempt) return;
if (watchdogAttempts >= MaximumWatchdogAttempts)
{
if (watchdogAttempts == MaximumWatchdogAttempts)
{
watchdogAttempts++;
Console.Error.WriteLine("VC_WATCHDOG exhausted; the audio graph will not start and is no longer being rebuilt");
}
return;
}
ResetWatchdog();
try { IosAudioEngine.Shared.Reconfigure(); }
watchdogAttempts++;
nextWatchdogAttempt = Environment.TickCount64 + Math.Min(2_000 * (1 << watchdogAttempts), 30_000);
try { IosAudioEngine.Shared.Reconfigure(true, $"stall watchdog {watchdogAttempts}"); }
catch (Exception exception) { System.Diagnostics.Debug.WriteLine($"Audio watchdog rebuild failed: {exception}"); }
}
finally { watchdogTicking = false; }
@@ -334,13 +353,18 @@ internal sealed class IosAudioRouter
return;
// Apply is mid-flight: this notification describes the change Apply is itself making.
if (applying) return;
// Every rebuild moves the route, and moving the route notifies here. Forcing the speaker
// takes a headset out of the route as OldDeviceUnavailable and releasing it brings the
// headset back as NewDeviceAvailable, neither of which is filtered above, so a rebuild
// that answered its own echo would rebuild again without end. The route the graph was
// built on is what decides: if it still carries audio, there is nothing to recover from.
if (!route.ShouldRebuild(RouteSignature(AVAudioSession.SharedInstance()))) return;
Recover($"route change ({reason})");
// Not a forced rebuild. Every reconfiguration moves the route, and moving the route
// notifies here, so answering a route change with an unconditional rebuild is a loop with
// one iteration per notification: forcing the speaker takes a headset out of the route as
// OldDeviceUnavailable, releasing it brings the headset back as NewDeviceAvailable, and
// even joining voice moves the route through SetPreferredInput. AVAudioEngine follows a
// route change on its own; what it cannot absorb is the hardware under the tap changing,
// and Reconfigure tests for that, rebuilding a stopped graph and leaving a healthy
// unchanged one alone. Comparing routes instead cannot work: CurrentRoute still names the
// previous route for a while after a reconfiguration.
Console.Error.WriteLine($"VC_ROUTE_CHANGE reason={reason} running={IosAudioEngine.Shared.IsRunning} " +
$"hardwareChanged={IosAudioEngine.Shared.HardwareChanged()}");
Recover($"route change ({reason})", force: false);
}
private void HandleInterruption(NSNotification note)
{