Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ios): auto-reconnect + audio-device-change recovery

Browse Source 
Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired
headphones, AirPods) used to leave the iOS client in a dead/zombie state:
the engine went silent, no reconnect was attempted, and a live-session
disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout.

Reconnect (AppState.swift, SessionState.swift):
- Two-layer reconcile. Once SessionState overwrites client.onEvent at auth
  success, AppState.handleConnectEvent no longer sees live-session events.
  Added a weak SessionState.appState; SessionState.handleEvent .disconnected
  calls appState.onLiveSessionDisconnected after the cue -- the single path
  AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect
  snapshots lastSession, stops audio, releases session/VoiceCatClient (io-
  thread join via vc_client_destroy), resets the backoff, and arms
  scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth
  success via existing TOFU_MATCHED auto-confirm + idempotent join_channel).
- NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi
  <-> cellular interface change or path .unsatisfied it calls
  proactiveReconnect: tearing the session down BEFORE the C core notices the
  dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff
  tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature.
  While mid-reconnect a .satisfied path resets the backoff for a fast retry.
- User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect
  and cancel all reconnect state (task + monitor + lastSession + connectedServer).

Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift):
- Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/
  .newDeviceAvailable route-change guard; fires on every externally-initiated
  route change reason except the ones we cause ourselves (.categoryChange/
  .routeConfigurationChange) to avoid a notification loop. Interruption-end
  now always recovers instead of only when .shouldResume is set.
- Added AVAudioEngineConfigurationChange observer on the engine so a system
  self-stop after our route-change handler wins the race is caught.
- IOSAudioEngine.rebuild() does a one-shot reactivation-retry on
  engine.start() failure (iOS sometimes refuses until the session is
  re-reactivated -- the silent-death case).

No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green
via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim).
This commit is contained in:
Talon
2026-06-25 14:57:13 +02:00
parent 44a336cc89
commit 99937c9446
5 changed files with 600 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
PROGRESS.md
View File

@@ -10,6 +10,78 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action ## ▶ Where we left off / next action
- **Done (2026-06-25):** **iOS robustness — auto-reconnect after a network change + audio
recovery when audio devices plug/unplug.** Two layers of bugs the iOS client had:
(a) a `VC_EVENT_DISCONNECTED` from the C core on a Wi-Fi→cellular flip / DNS outage /
server restart used to leave the session dead with no retry; (b) unplugging wired
headphones or AirPods left the engine stopped forever — mic stopped transmitting and
remote audio stayed silent (the server connection itself survived, but the audio graph
did not recover).
The first attempt wired reconnect into `AppState.handleConnectEvent`, but that handler
never runs for a live-session disconnect: once `SessionState.init` overwrites
`client.onEvent` (`SessionState.swift:87`), the `.disconnected` event is delivered to
`SessionState.handleEvent`, which used to play a cue and do nothing else. So the live
session would sit as a zombie for ~30-60 s (the C core's TCP keepalive/reaper timeout)
and then play the "connection lost" sound with no reconnect armed — exactly what the
user saw. The fix below has two parts addressing both the missing reconnect AND the
long wait.
1. **Event-driven reconnect** (`AppState.swift`, `SessionState.swift`): added a
`weak var appState: AppState?` to `SessionState`, set by AppState on auth success.
`SessionState.handleEvent` `.disconnected` now plays the cue and calls
`appState?.onLiveSessionDisconnected()` — the SINGLE path by which AppState learns a
live session dropped (since its own `handleConnectEvent` is bypassed for live-session
events). `onLiveSessionDisconnected` calls a shared `teardownLiveSessionAndReconnect`
that snapshots the live session into `LastSession`, stops the audio engine,
deactivates the AVAudioSession, nil's `session` (which releases `VoiceCatClient` →
`vc_client_destroy` joins the io thread), resets the backoff counter, and arms
`scheduleReconnect`.
2. **Path-driven proactive reconnect** (`AppState.swift`): an `NWPathMonitor`
(`Network.framework`) now runs the whole time we're CONNECTED (started on auth
success, not only when armed for reconnect) and stays armed across reconnects. Its
`pathUpdateHandler` (dispatched to @MainActor) does two things:
- While connected: a primary-interface change (Wi-Fi↔cellular) OR the path becoming
`.unsatisfied` triggers `proactiveReconnect()` — tearing the live session down
BEFORE the C core notices the dead TCP read. This is what collapses the 30-60 s
reaper wait into ~1 s + the first backoff tick. Same-interface refreshes (Wi-Fi
BSSID roams, signal-strength changes) are intentionally ignored (signature
comparison via `pathSignature`); those usually don't break the TCP connection.
- While mid-reconnect (no session): a path becoming `.satisfied` resets the backoff
counter and arms `scheduleReconnect` for a fast-fresh retry.
`userInitiatedDisconnect` distinguishes manual `disconnect()`/`cancelConnect()` (which
set it true → cancel all reconnect state) from a network drop (which leaves it false).
On a successful reconnect, `reconnectAttempt` resets and `lastSession` clears; the
path monitor keeps watching for the next change. On user-initiated disconnect, all
reconnect state (task + path monitor + `lastSession` + `connectedServer`) is cancelled.
3. **Backoff + restore**: exponential backoff 1s → 2s → 4s → 8s → 16s → 30s cap,
indefinite. TOFU pins match on the second connect (`VC_TOFU_MATCHED`) so the identity
gate auto-confirms; on auth success `SessionState.requestRestore` issues a
`joinChannel` and re-arms voice + restores the local mute/deafen state on the
resulting `.joinResult`.
4. **Audio recovery** (`AudioSessionManager.swift`, `IOSVoiceProcessingEngine.swift`):
replaced the route-change handler's narrow `.oldDeviceUnavailable`/
`.newDeviceAvailable` guard with a single intent-gated `recoverAudio()` path that
re-activates the AVAudioSession, re-applies the route config, and rebuilds the
engine; it runs on every externally-initiated route change reason except
`.categoryChange`/`.routeConfigurationChange` (those we cause ourselves and would
loop). Interruption-end now always calls `recoverAudio()` instead of only when
`.shouldResume` is set (which left the session permanently dead after Siri). Added
an `AVAudioEngineConfigurationChange` observer on the engine in `IOSAudioEngine`
that catches the case where iOS stops the engine itself AFTER our route-change
handler already rebuilt it (the previous rebuilds raced the engine's own self-stop
and lost). And `IOSAudioEngine.rebuild()` now does a one-shot reactivation-retry on
`engine.start()` failure — iOS sometimes refuses to start until the AVAudioSession is
re-activated, which is the silent-death case.
**Build:** `scripts/build-ios-client.sh --no-configure` green (Xcode 26.5 / iOS 18.0 sim
SDK, Swift 5 mode). No C ABI / `voicecat.h` / `voicecat.proto` / C core changes; the
existing TOFU auto-confirm (`VC_TOFU_MATCHED`) and idempotent `vc_join_channel` make
reconnect+restore possible without new C ABI. macOS and Windows clients unchanged.
**Next (manual, on-device):** verify unplugging AirPods/wired headphones mid-call keeps
audio going through the loudspeaker; verify Wi-Fi→cellular flip mid-call now triggers a
FAST reconnect (within a couple seconds, not 30-60 s) and lands in the same channel with
voice re-armed; verify tapping Disconnect mid-reconnect-abort cancels cleanly.
- **Done (2026-06-24):** **Three bug fixes — voice join/leave, channel edit defaults, channel-update stream restart.** - **Done (2026-06-24):** **Three bug fixes — voice join/leave, channel edit defaults, channel-update stream restart.**
1. **Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.** Previously 1. **Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.** Previously
"Join Voice" only started the local mic — receiving was always on (gated by channel "Join Voice" only started the local mic — receiving was always on (gated by channel

359
clients/apple/iOS/VoiceCatiOS/AppState.swift
View File

@@ -1,4 +1,5 @@
import Foundation import Foundation
import Network
import VoiceCatCore import VoiceCatCore
struct PendingIdentity: Identifiable { struct PendingIdentity: Identifiable {
@@ -25,6 +26,70 @@ final class AppState {
private(set) var connectingServer: SavedServer? private(set) var connectingServer: SavedServer?
private var identityHandled = false private var identityHandled = false
/// The server we are currently fully connected to. Set on auth success (when `session` is
/// created) and cleared on teardown. Used to build the `lastSession` restore snapshot when a
/// live-session disconnect fires through `SessionState.handleEvent` — `connectingServer` is
/// already nil by then, and `handleConnectEvent`'s `server` parameter is out of scope because
/// `SessionState` owns `client.onEvent` after auth success (see `SessionState.init`).
private var connectedServer: SavedServer?
// MARK: - Reconnect state
//
// The C core surfaces every unexpected connection drop as a `.disconnected` event; nothing
// in the C core auto-reconnects (intentional — reconnect UX is the client's job). Two layers
// drive iOS reconnect:
//
// 1. **Event-driven** (the C core's TCP read eventually fails after the keepalive/reaper
// timeout, ~30-60 s on a hard Wi-Fi drop): `SessionState.handleEvent` `.disconnected`
// plays the audible cue, then calls back into AppState via `onLiveSessionDisconnected`,
// which snapshots the live session, tears it down, and arms `scheduleReconnect`.
// (Live-session events never reach `AppState.handleConnectEvent` — `SessionState.init`
// overwrites `client.onEvent`, so `AppState` cannot see them without the callback.)
//
// 2. **Path-driven** (proactive, much faster): `NWPathMonitor` runs the whole time we are
// connected (started on auth success) and reacts to network changes — a Wi-Fi↔cellular
// flip or the path becoming `.unsatisfied` calls `proactiveReconnect`, which tears the
// live session down BEFORE the C core notices the dead TCP path. This is what makes the
// 30-60 s wait collapse into ~1 s + the backoff tick. While mid-reconnect (no session)
// the same monitor arms a fast-fresh retry whenever a path becomes `.satisfied`.
//
// Manual Disconnect cancels everything (task + path monitor) and clears `lastSession`.
/// True at the top of `disconnect()`/`cancelConnect()` — suppresses auto-reconnect for the
/// `.disconnected` event the core then emits in response to our `vc_disconnect()` call.
private var userInitiatedDisconnect = false
/// Snapshot of the live session state needed to restore after a reconnect. Cleared on
/// successful restore and on user-initiated disconnect.
private struct LastSession {
let server: SavedServer
let channelId: UInt32
let voiceSubscribed: Bool
let micMuted: Bool
let deafened: Bool
}
private var lastSession: LastSession?
/// Reconnect attempt counter — drives exponential backoff. Reset to 0 on successful auth and
/// on a path-driven fast-fresh retry.
private var reconnectAttempt = 0
/// The in-flight reconnect `Task` (sleeps for the backoff, then calls `connectTo`). One
/// at a time; cancelled on user disconnect / successful restore.
private var reconnectTask: Task<Void, Never>?
/// Started on auth success and kept running while connected / mid-reconnect; stopped only on
/// user-initiated disconnect. Its `pathUpdateHandler` (dispatched to @MainActor) handles two
/// cases: a path change while connected → proactive reconnect; a satisfied path while
/// mid-reconnect → fast-fresh retry. See the reconnect-state header comment.
private var pathMonitor: NWPathMonitor?
private let pathQueue = DispatchQueue(label: "cat.voice.network.path")
/// Signature of the last path seen by the monitor (a stable string encoding status + active
/// interface types). The very first path callback (when the monitor starts) sets this and is
/// otherwise ignored — it's the baseline; only subsequent CHANGES are reconnect triggers.
private var lastPathSignature: String?
// MARK: - Server list management // MARK: - Server list management
func addServer(_ server: SavedServer, password: String?) { func addServer(_ server: SavedServer, password: String?) {
@@ -51,14 +116,39 @@ final class AppState {
ServerListStore.shared.save(servers) ServerListStore.shared.save(servers)
} }
// MARK: - Teardown
/// `AppState` is the @Observable app root owned by the SwiftUI `App`; it lives for the
/// whole app process and is torn down only on process exit, at which point OS cleanup
/// suffices. Reconnect state (the in-flight `Task` and the `NWPathMonitor`) is cancelled
/// via `cancelReconnect()` driven by `disconnect()`/`cancelConnect()` and on successful
/// restore — those run on user-initiated teardown, which is the only path that matters.
/// (The `Task` captures `[weak self]` and guards on `nil`/`userInitiatedDisconnect`, so a
/// stray task left running when AppState is gone is a no-op; the monitor similarly guards.)
// MARK: - Connect flow // MARK: - Connect flow
/// Public connect entry. Always starts a fresh session (no restore).
func connectTo(_ server: SavedServer) { func connectTo(_ server: SavedServer) {
connectTo(server, restoring: nil)
}
/// Internal connect that drives the full TLS/auth saga. `restoring` is non-nil for a
/// reconnect attempt following an unexpected disconnect; the captured channel + voice/mic
/// state is handed to the new `SessionState` after auth succeeds.
private func connectTo(_ server: SavedServer, restoring: LastSession?) {
guard !isConnecting else { return } guard !isConnecting else { return }
isConnecting = true isConnecting = true
connectStatus = "Connecting…" connectStatus = (restoring != nil) ? "Reconnecting…" : "Connecting…"
connectingServer = server connectingServer = server
identityHandled = false identityHandled = false
userInitiatedDisconnect = false
// Discard any leftover connecting client. nil'ing the strong ref calls VoiceCatClient's
// deinit, which synchronously joins the C core's io thread (vc_client_destroy) before
// freeing the config-string storage — safe from @MainActor because the io thread never
// blocks on main (it enqueues events via DispatchQueue.main.async and returns).
connectingClient = nil
let config = VoiceCatConfig( let config = VoiceCatConfig(
clientName: "VoiceCat-iOS", clientName: "VoiceCat-iOS",
@@ -69,7 +159,9 @@ final class AppState {
connectingClient = client connectingClient = client
client.onEvent = { [weak self] ev in client.onEvent = { [weak self] ev in
Task { @MainActor [weak self] in self?.handleConnectEvent(ev, server: server) } Task { @MainActor [weak self] in
self?.handleConnectEvent(ev, server: server, restoring: restoring)
}
} }
// Put the core into external-playback mode BEFORE connect, so the flag is set on the // Put the core into external-playback mode BEFORE connect, so the flag is set on the
// io thread before any message is processed. The server sends AuthResult immediately // io thread before any message is processed. The server sends AuthResult immediately
@@ -82,7 +174,9 @@ final class AppState {
client.setExternalPlayback(true) client.setExternalPlayback(true)
client.connect(host: server.host, port: server.port) client.connect(host: server.host, port: server.port)
// Auth is queued immediately — the core serialises it behind TLS + TOFU. // Auth is queued immediately — the core serialises it behind TLS + TOFU. On a reconnect
// the TOFU pin already matches (VC_TOFU_MATCHED), so the identity gate auto-confirms
// inside the .serverIdentity case below and auth proceeds unattended.
switch server.authMode { switch server.authMode {
case .guest: case .guest:
let nick = (server.nickname?.isEmpty == false) ? server.nickname! : "iOS User" let nick = (server.nickname?.isEmpty == false) ? server.nickname! : "iOS User"
@@ -98,6 +192,11 @@ final class AppState {
} }
func disconnect() { func disconnect() {
// Mark BEFORE we ask the core to disconnect, so the .disconnected event the core emits
// in response is treated as user-initiated (no reconnect) rather than an unexpected drop.
userInitiatedDisconnect = true
cancelReconnect()
lastSession = nil
session?.leaveVoice() session?.leaveVoice()
session?.client.disconnect() session?.client.disconnect()
IOSAudioEngine.shared.stop() IOSAudioEngine.shared.stop()
@@ -106,6 +205,7 @@ final class AppState {
connectingClient?.disconnect() connectingClient?.disconnect()
connectingClient = nil connectingClient = nil
connectingServer = nil connectingServer = nil
connectedServer = nil
isConnecting = false isConnecting = false
connectStatus = "" connectStatus = ""
showPasswordPrompt = false showPasswordPrompt = false
@@ -126,26 +226,210 @@ final class AppState {
} }
func cancelConnect() { func cancelConnect() {
// User explicitly cancelled — no reconnect for the resulting .disconnected event.
userInitiatedDisconnect = true
cancelReconnect()
lastSession = nil
connectingClient?.disconnect() connectingClient?.disconnect()
connectingClient = nil connectingClient = nil
connectingServer = nil connectingServer = nil
connectedServer = nil
isConnecting = false isConnecting = false
connectStatus = "" connectStatus = ""
showPasswordPrompt = false showPasswordPrompt = false
pendingIdentity = nil pendingIdentity = nil
} }
// MARK: - Reconnect orchestration
/// Cancel any in-flight reconnect task and stop the path monitor. Safe to call when nothing
/// is armed (no-op). Does NOT touch `userInitiatedDisconnect` or `lastSession` — callers set
/// those as needed (disconnect/cancelConnect clear them; scheduleReconnect keeps them).
private func cancelReconnect() {
reconnectTask?.cancel()
reconnectTask = nil
stopPathMonitor()
// Don't reset `reconnectAttempt` here: scheduleReconnect resets it on successful auth,
// and the path monitor resets it to 0 for a fast fresh attempt on a path-satisfied event.
// If a fresh user connect follows, connectTo() doesn't reset it either, but it doesn't
// need to — `reconnectAttempt` only matters while we're mid-reconnect.
}
/// Arm the next reconnect attempt with exponential backoff (1s → 2s → 4s → 8s → 16s → 30s
/// cap). Cancelled cleanly by `cancelReconnect()` on user disconnect or successful auth.
/// Idempotent: a new call supersedes any in-flight one. The path monitor is armed here and
/// disarmed on cancel; on a path-satisfied event it resets the attempt counter to 0 and
/// re-arms via this same method, yielding a fast refresh after Wi-Fi ↔ cellular transitions.
private func scheduleReconnect() {
guard !userInitiatedDisconnect, let last = lastSession else { return }
reconnectTask?.cancel()
reconnectAttempt = max(1, reconnectAttempt + 1)
let delaySec = min(pow(2.0, Double(reconnectAttempt - 1)), 30.0)
connectStatus = "Reconnecting (attempt \(reconnectAttempt))…"
startPathMonitor()
let task = Task { [weak self, last] in
guard let self else { return }
try? await Task.sleep(nanoseconds: UInt64(delaySec * 1_000_000_000))
if Task.isCancelled { return }
// Re-check under Task: a user disconnect between the sleep and this line must abort.
guard !self.userInitiatedDisconnect else { return }
guard self.lastSession != nil else { return }
guard self.session == nil else { return }
self.connectTo(last.server, restoring: last)
}
reconnectTask = task
}
/// Start (if not already running) the network path monitor. Runs the whole time we are
/// connected (started on auth success) and stays armed across reconnects; stopped only on
/// user-initiated disconnect. The handler dispatches to @MainActor before touching state and
/// does two distinct things:
/// - **While connected** (`session != nil`): a Wi-Fi↔cellular interface change OR the path
/// becoming `.unsatisfied` triggers `proactiveReconnect()` — tearing the live session
/// down before the C core notices the dead TCP read. Without this the disconnect would
/// take 30-60 s (the TCP keepalive/reaper timeout); proactive teardown collapses that to
/// ~1 s + the first backoff tick. Same-interface path refreshes (e.g. a Wi-Fi roam
/// without an IP change) are ignored — likely the connection is still good.
/// - **While mid-reconnect** (`session == nil`, `lastSession != nil`): a path becoming
/// `.satisfied` arms a fast-fresh retry (backoff counter reset, `scheduleReconnect`).
/// This is what makes a Wi-Fi→cellular flip reconnect on roughly the next tick instead
/// of waiting out a long backoff.
private func startPathMonitor() {
guard pathMonitor == nil else { return }
let monitor = NWPathMonitor()
monitor.pathUpdateHandler = { [weak self] path in
Task { @MainActor [weak self] in
guard let self else { return }
guard !self.userInitiatedDisconnect else { return }
let sig = Self.pathSignature(path)
let prevSig = self.lastPathSignature
self.lastPathSignature = sig
// The first callback (when the monitor starts) is the baseline, not a change.
if prevSig == nil { return }
if self.session != nil {
// Connected — tear down + reconnect on a meaningful path change.
// `.unsatisfied` (all radios off) OR a primary-interface change (Wi-Fi↔cellular)
// almost always breaks the live TCP connection; reconnecting proactively beats
// waiting for the C core's keepalive/reaper timeout.
if path.status != .satisfied || sig != prevSig {
self.proactiveReconnect()
}
} else if self.lastSession != nil {
// Mid-reconnect — a path is available again; fast-fresh the next attempt.
if path.status == .satisfied {
self.reconnectAttempt = 0
self.scheduleReconnect()
}
}
}
}
// Listen on a dedicated queue — the path monitor can't share the main queue (it would
// re-enter main if any handler dispatched to main synchronously).
monitor.start(queue: pathQueue)
pathMonitor = monitor
}
private func stopPathMonitor() {
pathMonitor?.cancel()
pathMonitor = nil
lastPathSignature = nil
}
/// A stable string signature of a network path: the path status plus the set of interface
/// types it uses. Two paths with the same signature are treated as equivalent — no
/// reconnect. A signature change is the trigger for `proactiveReconnect`. Used to ignore
/// same-interface refreshes (signal-strength changes, BSSID roams) which usually don't break
/// the TCP connection.
private static func pathSignature(_ path: NWPath) -> String {
guard path.status == .satisfied else { return "unsatisfied" }
var parts: [String] = []
if path.usesInterfaceType(.wifi) { parts.append("wifi") }
if path.usesInterfaceType(.cellular) { parts.append("cellular") }
if path.usesInterfaceType(.wiredEthernet) { parts.append("wired") }
if path.usesInterfaceType(.other) { parts.append("other") }
return parts.isEmpty ? "none" : parts.sorted().joined(separator: "+")
}
// MARK: - Live-session disconnect (called by SessionState)
/// Called by `SessionState.handleEvent` `.disconnected` after the audible cue has already
/// played. Once `SessionState` is created (auth success) it owns `client.onEvent`, so
/// `AppState.handleConnectEvent` never sees live-session events — this callback is the only
/// way AppState learns that a live session dropped. Snapshots the live session state,
/// tears the session down, and arms `scheduleReconnect` so the backoff loop drives a fresh
/// TLS/auth/restoration. Guarded against user-initiated disconnect (which nil's `session`
/// synchronously, so SessionState is gone before the event could fire this callback) — but
/// the guard is cheap insurance.
func onLiveSessionDisconnected() {
guard !userInitiatedDisconnect else { return }
teardownLiveSessionAndReconnect(sound: false)
}
/// Called by `NWPathMonitor` when a path change is detected while a live session exists.
/// Tears the session down immediately — nil'ing `session` releases `VoiceCatClient`, whose
/// `deinit` calls `vc_client_destroy`; that closes the socket and joins the C core's io
/// thread, so the io thread exits in milliseconds rather than blocking on a dead read for
/// ~30-60 s. The proactive tear-down is what collapses the long TCP-reaper wait into a
/// ~1 s reconnect. Plays the audible cue (no `.disconnected` event fires through to
/// `SessionState` for this path, since `SessionState` is being torn down here — so the cue
/// would otherwise be missing).
private func proactiveReconnect() {
guard !userInitiatedDisconnect else { return }
guard session != nil else { return }
teardownLiveSessionAndReconnect(sound: true)
}
/// Shared teardown for a live-session disconnect (event- or path-driven). Snapshots the live
/// session into `lastSession`, stops audio, deactivates the AVAudioSession, releases the
/// session (which releases `VoiceCatClient` → io-thread join), resets the backoff counter,
/// and arms `scheduleReconnect`. `sound` is true for the proactive (path-driven) case — the
/// `.disconnected` event that would have played it never fires because we're tearing down
/// ahead of the C core noticing. The event-driven caller (`onLiveSessionDisconnected`) has
/// ALREADY played the cue via `SessionState.handleEvent`, so it passes `sound: false`.
private func teardownLiveSessionAndReconnect(sound: Bool) {
// Snapshot BEFORE nil'ing `session` — we need the channel + voice/mic state to restore.
if let s = session, let srv = connectedServer {
lastSession = LastSession(
server: srv,
channelId: s.currentChannelId,
voiceSubscribed: s.voiceState.voiceSubscribed,
micMuted: s.voiceState.selfMuted,
deafened: s.voiceState.selfDeafened)
}
IOSAudioEngine.shared.stop()
AudioSessionManager.shared.deactivateSession()
// Releasing `session` releases `VoiceCatClient`; its deinit joins the C core's io thread.
// For a path-driven proactive teardown this is what avoids the 30-60 s reaper timeout.
session = nil
isConnecting = false
connectingClient = nil
connectedServer = nil
if sound {
EventFeedback.shared.play(.connectionLost)
EventFeedback.shared.speak("Network changed — reconnecting")
}
// Reset the backoff counter so the first reconnect attempt after a drop uses the short
// 1 s delay (the immediate path-driven attempt matters most; sustained-outage backoff is
// driven by `scheduleReconnect`'s increment).
reconnectAttempt = 0
scheduleReconnect()
}
// MARK: - Connect event handler // MARK: - Connect event handler
private func handleConnectEvent(_ ev: VoiceCatEvent, server: SavedServer) { private func handleConnectEvent(_ ev: VoiceCatEvent, server: SavedServer,
restoring: LastSession?) {
switch ev.type { switch ev.type {
case .connectionState: case .connectionState:
switch ev.connectionState { switch ev.connectionState {
case .connecting: connectStatus = "Connecting…" case .connecting: connectStatus = (restoring != nil) ? "Reconnecting…" : "Connecting…"
case .tlsHandshake: connectStatus = "TLS handshake…" case .tlsHandshake: connectStatus = "TLS handshake…"
case .authenticating: connectStatus = "Authenticating…" case .authenticating: connectStatus = "Authenticating…"
case .verifyingIdentity: connectStatus = "Verifying server identity…" case .verifyingIdentity: connectStatus = "Verifying server identity…"
case .connected: connectStatus = "Connected" case .connected: connectStatus = "Connected"
default: break default: break
} }
case .serverIdentity: case .serverIdentity:
@@ -163,13 +447,15 @@ final class AppState {
guard let client = connectingClient else { break } guard let client = connectingClient else { break }
let perms = client.getPermissions() let perms = client.getPermissions()
let newSession = SessionState(client: client, selfUserId: ev.userId, permissions: perms) let newSession = SessionState(client: client, selfUserId: ev.userId, permissions: perms)
newSession.appState = self
connectingClient = nil connectingClient = nil
isConnecting = false isConnecting = false
connectStatus = "" connectStatus = ""
showPasswordPrompt = false showPasswordPrompt = false
connectedServer = server
self.session = newSession self.session = newSession
EventFeedback.shared.play(.login) EventFeedback.shared.play(.login)
EventFeedback.shared.speak("Connected") EventFeedback.shared.speak(restoring != nil ? "Reconnected" : "Connected")
// External-playback mode was enabled before connect() so the core never opens a // External-playback mode was enabled before connect() so the core never opens a
// miniaudio device on iOS (the single ordering rule of the unified audio path). // miniaudio device on iOS (the single ordering rule of the unified audio path).
// Now activate the session and start the engine in listening mode so remote audio // Now activate the session and start the engine in listening mode so remote audio
@@ -180,20 +466,67 @@ final class AppState {
print("Audio session activate on connect failed: \(error)") print("Audio session activate on connect failed: \(error)")
} }
IOSAudioEngine.shared.startListening(client: client) IOSAudioEngine.shared.startListening(client: client)
// The path monitor runs the whole time we're connected so a network change fires
// proactiveReconnect immediately instead of waiting for the C core's TCP keepalive
// timeout (~30-60 s on a hard Wi-Fi drop). It stays armed across reconnects and is
// stopped only on user-initiated disconnect.
startPathMonitor()
// Reconnect restore: rejoin the prior channel and re-enable voice/mic if they
// were on. The session is fresh (server auto-places us in Lobby), so the restore
// is driven through SessionState.requestRestore, which issues a JoinChannel then
// (on the resulting .joinResult) re-arms voice + mute/deafen. A successful auth
// means the server is reachable, so the backoff counter resets and `lastSession`
// clears; the path monitor keeps watching for the next change.
if let restoring {
newSession.requestRestore(channelId: restoring.channelId,
voiceSubscribed: restoring.voiceSubscribed,
micMuted: restoring.micMuted,
deafened: restoring.deafened)
reconnectAttempt = 0
lastSession = nil
}
} else { } else {
connectStatus = "Auth failed: \(ev.result.description)" connectStatus = "Auth failed: \(ev.result.description)"
showPasswordPrompt = true showPasswordPrompt = true
} }
case .disconnected: case .disconnected:
if session == nil { cancelConnect() } // This handler runs ONLY during the connecting phase — after auth success
else { // `SessionState.init` overwrites `client.onEvent`, so a live-session disconnect
IOSAudioEngine.shared.stop() // reaches `SessionState.handleEvent` and comes back via
AudioSessionManager.shared.deactivateSession() // `onLiveSessionDisconnected`, not here. Two outcomes for this branch:
session = nil; isConnecting = false // - A reconnect's connecting phase failed (`lastSession != nil`, set by a prior
// teardown) → re-arm `scheduleReconnect` so the backoff loop continues.
// - A fresh connect failed before auth (`lastSession == nil`) → show the error, do
// not auto-reconnect (the user should retry manually once the server is reachable).
connectingClient = nil
isConnecting = false
IOSAudioEngine.shared.stop()
AudioSessionManager.shared.deactivateSession()
if userInitiatedDisconnect {
connectStatus = ""
showPasswordPrompt = false
pendingIdentity = nil
lastSession = nil
connectedServer = nil
cancelReconnect()
} else if lastSession != nil {
// Mid-reconnect drop — keep the backoff loop going.
EventFeedback.shared.play(.connectionLost)
EventFeedback.shared.speak("Connection lost — reconnecting")
scheduleReconnect()
} else {
// Fresh connect failed before auth. Surface the reason; no auto-reconnect.
connectStatus = ev.text ?? "Disconnected"
showPasswordPrompt = false
pendingIdentity = nil
connectedServer = nil
cancelReconnect()
} }
case .error: case .error:
connectStatus = ev.text ?? "Unknown error" connectStatus = ev.text ?? "Unknown error"
if session == nil { isConnecting = false } // Errors don't disconnect us; the .disconnected event handles teardown/reconnect.
default: default:
break break
} }

84
clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift
View File

@@ -35,6 +35,27 @@ final class AudioSessionManager {
name: AVAudioSession.routeChangeNotification, object: nil) name: AVAudioSession.routeChangeNotification, object: nil)
} }
/// The single end-to-end audio recovery path, driven by *intent* (`IOSAudioEngine.isConnected`)
/// — not by session bookkeeping flags that can drift out of sync (e.g. an interruption ended
/// without `.shouldResume`, which used to leave `isSessionActive` false forever). Safe to call
/// speculatively: the underlying calls are idempotent (AVAudioSession.setActive(true),
/// `IOSAudioRouter.applyConfiguration` has a re-entrancy guard, `IOSAudioEngine.reconfigure`
/// no-ops when not connected). Call this whenever the audio environment changes in a way that
/// could have stopped the engine — interruption end, route change, AVAudioEngine
/// configuration-change — and we still want audio back.
func recoverAudio() {
guard IOSAudioEngine.shared.isConnected else { return }
do {
try ensureSessionActive()
} catch {
logger.error("recoverAudio — session activate failed: \(error.localizedDescription)")
}
IOSAudioRouter.shared.applyConfiguration()
if isSessionActive { IOSAudioRouter.shared.applyA2dpSpeakerFallback() }
IOSAudioEngine.shared.reconfigure()
logSessionState("after recoverAudio")
}
/// Activate the AVAudioSession if not already active. Call before any audio I/O: /// Activate the AVAudioSession if not already active. Call before any audio I/O:
/// when the user joins voice, or when a remote stream starts (so playback works even /// when the user joins voice, or when a remote stream starts (so playback works even
/// before the user has joined voice). Idempotent — safe to call multiple times. /// before the user has joined voice). Idempotent — safe to call multiple times.
@@ -110,21 +131,13 @@ final class AudioSessionManager {
logger.info("interruption began — session suspended by system") logger.info("interruption began — session suspended by system")
isSessionActive = false isSessionActive = false
case .ended: case .ended:
let optionsValue = info[AVAudioSessionInterruptionOptionKey] as? UInt ?? 0 // Always attempt recovery when we have a live session. iOS sometimes ends an
let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue) // interruption without the `.shouldResume` hint (e.g. Siri), and the previous
if options.contains(.shouldResume) { // behavior of only reactivating when `.shouldResume` was set left the session
do { // permanently dead — audio never came back. `recoverAudio()` is intent-gated on
IOSAudioRouter.shared.applyConfiguration() // `IOSAudioEngine.isConnected` and idempotent, so speculatively calling it is safe.
try AVAudioSession.sharedInstance().setActive(true) logger.info("interruption ended — recovery requested")
isSessionActive = true recoverAudio()
IOSAudioRouter.shared.applyA2dpSpeakerFallback()
// Rebuild the engine graph against the restored route (both directions).
IOSAudioEngine.shared.reconfigure()
logger.info("interruption ended — session reactivated, engine rebuilt")
} catch {
logger.error("interruption ended — reactivation failed: \(error.localizedDescription)")
}
}
@unknown default: break @unknown default: break
} }
} }
@@ -134,37 +147,31 @@ final class AudioSessionManager {
let reasonValue = info[AVAudioSessionRouteChangeReasonKey] as? UInt, let reasonValue = info[AVAudioSessionRouteChangeReasonKey] as? UInt,
let reason = AVAudioSession.RouteChangeReason(rawValue: reasonValue) let reason = AVAudioSession.RouteChangeReason(rawValue: reasonValue)
else { else {
logger.warning("routeChange — unknown reason, refreshing only") logger.warning("routeChange — unknown reason, refreshing + recovery")
IOSAudioRouter.shared.refreshRoutes() IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil) NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
recoverAudio()
return return
} }
logger.info("routeChange reason=\(self.reasonLabel(reason))") logger.info("routeChange reason=\(self.reasonLabel(reason))")
// Re-apply preferences ONLY on external device plug/unplug. Do NOT re-apply on
// .categoryChange / .routeConfigurationChange — those are triggered by our own
// applyConfiguration() calls (setCategory, setPreferredInput, etc.), and re-applying
// would create an infinite notification loop:
// handleRouteChange → applyConfiguration → setCategory → routeChange → ...
// That loop burns CPU and cycles the audio session on/off — the "glitching" bug.
// IOSAudioRouter.applyConfiguration() also has a re-entrancy guard for synchronous
// notifications, but the reason check here is the primary defense.
if reason == .oldDeviceUnavailable || reason == .newDeviceAvailable {
logger.info("routeChange — external device change, re-applying config")
IOSAudioRouter.shared.applyConfiguration()
// Re-evaluate the A2DP-mode speaker fallback: a Bluetooth unplug should drop us onto
// the loud speaker (not the earpiece), and a replug should hand output back to A2DP.
if isSessionActive {
IOSAudioRouter.shared.applyA2dpSpeakerFallback()
// Rebind the engine (both directions) to the new route. The engine owns the route
// now, so this is the single thing that re-establishes audio after a device change.
IOSAudioEngine.shared.reconfigure()
}
}
IOSAudioRouter.shared.refreshRoutes() IOSAudioRouter.shared.refreshRoutes()
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil) NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
// Recover audio on every externally-initiated route change. `.categoryChange` and
// `.routeConfigurationChange` are fired by our OWN applyConfiguration() calls
// (setCategory, setPreferredInput, ...). Acting on them would create a tight ping-pong
// loop with the re-entrancy guard (handleRouteChange → recoverAudio →
// applyConfiguration → setCategory → routeChange → ...). IOSAudioRouter's guard is the
// backstop that bounds it to ONE extra iteration, but skipping these two reasons avoids
// even that, so we reconfigure only in response to genuine environmental changes.
//
// The recovery set below (oldDeviceUnavailable, newDeviceAvailable, override,
// wakeFromSleep, noSuitableRouteForCategory, unknown) covers headphone/AirPods/wired
// unplug-replug — the previously-reported "audio dies when headphones disconnect" bug.
if reason != .categoryChange && reason != .routeConfigurationChange {
recoverAudio()
}
logSessionState("route change (\(reasonLabel(reason)))") logSessionState("route change (\(reasonLabel(reason)))")
} }
@@ -177,6 +184,7 @@ final class AudioSessionManager {
case .wakeFromSleep: return "wakeFromSleep" case .wakeFromSleep: return "wakeFromSleep"
case .noSuitableRouteForCategory: return "noSuitableRouteForCategory" case .noSuitableRouteForCategory: return "noSuitableRouteForCategory"
case .routeConfigurationChange: return "routeConfigurationChange" case .routeConfigurationChange: return "routeConfigurationChange"
case .unknown: return "unknown"
@unknown default: return "unknown" @unknown default: return "unknown"
} }
} }

47
clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift
View File

@@ -187,6 +187,29 @@ final class IOSAudioEngine {
renderScratch.initialize(repeating: 0, count: renderScratchFrames * 2) renderScratch.initialize(repeating: 0, count: renderScratchFrames * 2)
micDrainScratch = UnsafeMutablePointer<Int16>.allocate(capacity: 960 * 2) micDrainScratch = UnsafeMutablePointer<Int16>.allocate(capacity: 960 * 2)
micDrainScratch.initialize(repeating: 0, count: 960 * 2) micDrainScratch.initialize(repeating: 0, count: 960 * 2)
// AVAudioEngine stops itself on a mid-session route/configuration change (it stops
// if its I/O graph no longer matches the active route). Our route-change handler in
// AudioSessionManager normally rebuilds us before the user notices, but if the engine
// stops itself AFTER our recovery (because the route-change notification raced ahead
// of the engine's own self-stop), nothing restarts it. Catch that case here.
NotificationCenter.default.addObserver(
self, selector: #selector(handleEngineConfigurationChange),
name: .AVAudioEngineConfigurationChange, object: engine)
}
/// The engine stopped itself because its configuration no longer matches the active AVAudio
/// route (this fires after a route change that the route-change handler can't always outrun).
/// Dispatch to main and call the unified `recoverAudio()` — it's intent-gated on
/// `isConnected`, idempotent, and no-ops if the engine is already running (the common case
/// where our route-change handler got there first).
@objc private func handleEngineConfigurationChange(_ notification: Notification) {
Task { @MainActor [weak self] in
guard let self else { return }
guard self.isConnected, !self.engine.isRunning else { return }
logger.info("engine configuration-change — engine stopped itself, recovering")
AudioSessionManager.shared.recoverAudio()
}
} }
// MARK: - Lifecycle // MARK: - Lifecycle
@@ -308,7 +331,29 @@ final class IOSAudioEngine {
inFormat=\(inFmt) outputNode=\(outFmt) outputRoute=[\(route)] inFormat=\(inFmt) outputNode=\(outFmt) outputRoute=[\(route)]
""") """)
} catch { } catch {
logger.error("engine start failed: \(error.localizedDescription)") // iOS occasionally refuses to start the engine immediately after a route change —
// the AVAudioSession needs a re-activation nudge before the engine will start. Do
// ONE recovery attempt: re-activate the session, re-apply the route config, then
// try `engine.start()` again. Recovering here is what fixes the silent-death bug
// where unplugging headphones left the engine stopped forever.
logger.error("engine start failed: \(error.localizedDescription) — attempting one-shot recovery")
do {
try AudioSessionManager.shared.ensureSessionActive()
} catch {
logger.error("recovery — session re-activate failed: \(error.localizedDescription)")
}
IOSAudioRouter.shared.applyConfiguration()
if AudioSessionManager.shared.isActive {
IOSAudioRouter.shared.applyA2dpSpeakerFallback()
}
do {
try engine.start()
logger.info("engine start succeeded after one-shot recovery")
} catch {
logger.error("engine start failed after recovery: \(error.localizedDescription)")
// Not fatal — a subsequent route-change or AVAudioEngine configuration-change
// notification will trigger recoverAudio() and re-attempt the rebuild.
}
} }
// Start the feed pump last, with the current channel count, so it never carries a stale // Start the feed pump last, with the current channel count, so it never carries a stale

91
clients/apple/iOS/VoiceCatiOS/SessionState.swift
View File

@@ -54,6 +54,30 @@ final class SessionState {
var accounts: [Account] = [] var accounts: [Account] = []
var devices: [Device] = [] var devices: [Device] = []
/// Back-reference to the app state. Once `SessionState.init` overwrites `client.onEvent`
/// (`SessionState.swift:87`), `AppState.handleConnectEvent` no longer receives per-session
/// events — so the `.disconnected` event for a LIVE session arrives here in `handleEvent`,
/// not in AppState. This weak ref lets us hand the disconnect back to AppState (which owns
/// the reconnect state machine) so the auto-reconnect fires. Set by AppState on auth success.
weak var appState: AppState?
// MARK: - Reconnect restore state
//
// When the iOS client auto-reconnects after a network drop, AppState captures the prior
// session's channel + voice/mic state and asks the new SessionState (created on auth success)
// to restore it. We rejoin the channel explicitly (the server auto-placed us in Lobby on
// auth), and on the resulting `.joinResult` we re-arm voice subscription + mute/deafen. The
// drive is here, not in AppState, because once SessionState is created it owns
// `client.onEvent` and AppState no longer sees per-session events.
private struct RestoreRequest {
let channelId: UInt32
let voiceSubscribed: Bool
let micMuted: Bool
let deafened: Bool
}
private var pendingRestore: RestoreRequest?
private var didIssueRestoreJoin = false
/// Host side of iOS screen-audio sharing — drains the broadcast extension's App Group ring /// Host side of iOS screen-audio sharing — drains the broadcast extension's App Group ring
/// and feeds the SCREEN_AUDIO stream this session owns. See BroadcastAudioPump. /// and feeds the SCREEN_AUDIO stream this session owns. See BroadcastAudioPump.
private let broadcastPump = BroadcastAudioPump() private let broadcastPump = BroadcastAudioPump()
@@ -186,8 +210,24 @@ final class SessionState {
currentChannelId = ev.channelId currentChannelId = ev.channelId
addActivity("Joined channel") addActivity("Joined channel")
refreshUsers() refreshUsers()
// Reconnect restore: this was our restore-join. Now that the server has
// processed the channel move, re-arm voice subscription (if the user was
// transmitting before the drop) and re-apply the local mute/deafen state.
// The server returns ok even when joining the channel we're already in, so
// this fires reliably for the Lobby-too case.
if didIssueRestoreJoin, let r = pendingRestore, r.channelId == ev.channelId {
didIssueRestoreJoin = false
completeRestore()
}
} else { } else {
addActivity("Join failed: \(ev.result.description)") addActivity("Join failed: \(ev.result.description)")
// Restore-join failed (channel was deleted, became password-protected or
// full while we were away). Give up on the voice/mute restore cleanly so we
// don't leave dangling state or attempt voice without being in a channel.
if didIssueRestoreJoin {
didIssueRestoreJoin = false
pendingRestore = nil
}
} }
case .error: case .error:
addActivity("Error: \(ev.text ?? ev.result.description)") addActivity("Error: \(ev.text ?? ev.result.description)")
@@ -198,9 +238,15 @@ final class SessionState {
case .accountList: case .accountList:
accounts = client.listAccounts() accounts = client.listAccounts()
case .disconnected: case .disconnected:
// Audible cue only — session teardown is driven elsewhere (AppState / UI). // Audible cue, then hand the disconnect back to AppState so its reconnect state
// machine fires. This is the ONLY way AppState learns a live session dropped —
// after auth success, `SessionState.init` overwrites `client.onEvent`, so
// `AppState.handleConnectEvent` never sees this event. (Without this callback, a
// network drop on a live session would just play the cue and leave the session as a
// zombie — the user would have to tap Disconnect manually.)
EventFeedback.shared.play(ev.result == .ok ? .logout : .connectionLost) EventFeedback.shared.play(ev.result == .ok ? .logout : .connectionLost)
EventFeedback.shared.speak(ev.result == .ok ? "Disconnected" : "Connection lost") EventFeedback.shared.speak(ev.result == .ok ? "Disconnected" : "Connection lost")
appState?.onLiveSessionDisconnected()
default: default:
break break
} }
@@ -300,6 +346,49 @@ final class SessionState {
client.leaveVoice() client.leaveVoice()
} }
// MARK: - Reconnect restore
/// Called by `AppState` after a reconnect's auth success to rejoin the prior channel and
/// re-enable the prior voice/mic state. Drives the restore through the `.joinResult` event
/// so we re-arm voice only AFTER the server processed the join — joining voice before the
/// channel move would be rejected server-side. `micMuted`/`deafened` are the user's LOCAL
/// mute/deafen state at the moment of the drop; the server resets those on a fresh auth, so
/// we re-push them via `setMute` after the channel is restored.
func requestRestore(channelId: UInt32, voiceSubscribed: Bool,
micMuted: Bool, deafened: Bool) {
pendingRestore = RestoreRequest(channelId: channelId,
voiceSubscribed: voiceSubscribed,
micMuted: micMuted,
deafened: deafened)
didIssueRestoreJoin = false
if channelId != 0 {
// The server auto-placed us in the Lobby on auth; join our prior channel explicitly.
// `vc_join_channel` is idempotent server-side (joining the channel you're already in
// returns ok), so this is safe even if the prior channel was the Lobby.
client.joinChannel(channelId)
didIssueRestoreJoin = true
} else {
// No prior channel — go straight to the voice/mute restore. (voiceSubscribed with
// channelId == 0 is contradictory; `completeRestore` further guards on
// currentChannelId != 0 before subscribing to voice.)
completeRestore()
}
}
/// Finish the restore after the channel is in place (or there was no channel to restore):
/// re-subscribe to voice if the user was transmitting, and re-apply the local mute/deafen
/// state. Safe to call once per `pendingRestore`; clears it.
private func completeRestore() {
guard let r = pendingRestore else { return }
if r.voiceSubscribed && currentChannelId != 0 {
joinVoice()
}
setMute(r.micMuted, deafened: r.deafened)
addActivity("Restored to channel \(currentChannelId)"
+ (r.voiceSubscribed ? " with voice" : ""))
pendingRestore = nil
}
// MARK: - Screen audio share // MARK: - Screen audio share
/// Called when the broadcast extension becomes active. Announces the SCREEN_AUDIO stream; /// Called when the broadcast extension becomes active. Announces the SCREEN_AUDIO stream;