From 99937c9446d9b6d9e25064d6952e0d719a22307b Mon Sep 17 00:00:00 2001 From: Talon Date: Thu, 25 Jun 2026 14:57:13 +0200 Subject: [PATCH] feat(ios): auto-reconnect + audio-device-change recovery Network drops (e.g. Wi-Fi -> cellular) and audio-device plug/unplug (wired headphones, AirPods) used to leave the iOS client in a dead/zombie state: the engine went silent, no reconnect was attempted, and a live-session disconnect waited 30-60 s for the C core's TCP keepalive/reaper timeout. Reconnect (AppState.swift, SessionState.swift): - Two-layer reconcile. Once SessionState overwrites client.onEvent at auth success, AppState.handleConnectEvent no longer sees live-session events. Added a weak SessionState.appState; SessionState.handleEvent .disconnected calls appState.onLiveSessionDisconnected after the cue -- the single path AppState learns a live session dropped. Shared teardownLiveSessionAndReconnect snapshots lastSession, stops audio, releases session/VoiceCatClient (io- thread join via vc_client_destroy), resets the backoff, and arms scheduleReconnect (exponential 1s -> 30s cap, indefinite, restored on auth success via existing TOFU_MATCHED auto-confirm + idempotent join_channel). - NWPathMonitor now runs WHILE CONNECTED (not only mid-reconnect). On a Wi-Fi <-> cellular interface change or path .unsatisfied it calls proactiveReconnect: tearing the session down BEFORE the C core notices the dead socket collapses the 30-60 s reaper wait into ~1 s + first backoff tick. Same-interface refreshes (BSSID roams) are ignored via pathSignature. While mid-reconnect a .satisfied path resets the backoff for a fast retry. - User-initiated disconnect()/cancelConnect() set userInitiatedDisconnect and cancel all reconnect state (task + monitor + lastSession + connectedServer). Audio recovery (AudioSessionManager.swift, IOSVoiceProcessingEngine.swift): - Intent-gated recoverAudio() replaces the narrow .oldDeviceUnavailable/ .newDeviceAvailable route-change guard; fires on every externally-initiated route change reason except the ones we cause ourselves (.categoryChange/ .routeConfigurationChange) to avoid a notification loop. Interruption-end now always recovers instead of only when .shouldResume is set. - Added AVAudioEngineConfigurationChange observer on the engine so a system self-stop after our route-change handler wins the race is caught. - IOSAudioEngine.rebuild() does a one-shot reactivation-retry on engine.start() failure (iOS sometimes refuses until the session is re-reactivated -- the silent-death case). No C ABI / voicecat.h / proto / core changes. Swift-only. iOS sim build green via scripts/build-ios-client.sh --no-configure (Xcode 26.5 / iOS 18.0 sim). --- PROGRESS.md | 72 ++++ clients/apple/iOS/VoiceCatiOS/AppState.swift | 359 +++++++++++++++++- .../iOS/VoiceCatiOS/AudioSessionManager.swift | 84 ++-- .../IOSVoiceProcessingEngine.swift | 47 ++- .../apple/iOS/VoiceCatiOS/SessionState.swift | 91 ++++- 5 files changed, 600 insertions(+), 53 deletions(-) diff --git a/PROGRESS.md b/PROGRESS.md index 35b9f60..f0a1685 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -10,6 +10,78 @@ up instantly. Newest status at the top. ## ▶ Where we left off / next action +- **Done (2026-06-25):** **iOS robustness — auto-reconnect after a network change + audio + recovery when audio devices plug/unplug.** Two layers of bugs the iOS client had: + (a) a `VC_EVENT_DISCONNECTED` from the C core on a Wi-Fi→cellular flip / DNS outage / + server restart used to leave the session dead with no retry; (b) unplugging wired + headphones or AirPods left the engine stopped forever — mic stopped transmitting and + remote audio stayed silent (the server connection itself survived, but the audio graph + did not recover). + + The first attempt wired reconnect into `AppState.handleConnectEvent`, but that handler + never runs for a live-session disconnect: once `SessionState.init` overwrites + `client.onEvent` (`SessionState.swift:87`), the `.disconnected` event is delivered to + `SessionState.handleEvent`, which used to play a cue and do nothing else. So the live + session would sit as a zombie for ~30-60 s (the C core's TCP keepalive/reaper timeout) + and then play the "connection lost" sound with no reconnect armed — exactly what the + user saw. The fix below has two parts addressing both the missing reconnect AND the + long wait. + + 1. **Event-driven reconnect** (`AppState.swift`, `SessionState.swift`): added a + `weak var appState: AppState?` to `SessionState`, set by AppState on auth success. + `SessionState.handleEvent` `.disconnected` now plays the cue and calls + `appState?.onLiveSessionDisconnected()` — the SINGLE path by which AppState learns a + live session dropped (since its own `handleConnectEvent` is bypassed for live-session + events). `onLiveSessionDisconnected` calls a shared `teardownLiveSessionAndReconnect` + that snapshots the live session into `LastSession`, stops the audio engine, + deactivates the AVAudioSession, nil's `session` (which releases `VoiceCatClient` → + `vc_client_destroy` joins the io thread), resets the backoff counter, and arms + `scheduleReconnect`. + 2. **Path-driven proactive reconnect** (`AppState.swift`): an `NWPathMonitor` + (`Network.framework`) now runs the whole time we're CONNECTED (started on auth + success, not only when armed for reconnect) and stays armed across reconnects. Its + `pathUpdateHandler` (dispatched to @MainActor) does two things: + - While connected: a primary-interface change (Wi-Fi↔cellular) OR the path becoming + `.unsatisfied` triggers `proactiveReconnect()` — tearing the live session down + BEFORE the C core notices the dead TCP read. This is what collapses the 30-60 s + reaper wait into ~1 s + the first backoff tick. Same-interface refreshes (Wi-Fi + BSSID roams, signal-strength changes) are intentionally ignored (signature + comparison via `pathSignature`); those usually don't break the TCP connection. + - While mid-reconnect (no session): a path becoming `.satisfied` resets the backoff + counter and arms `scheduleReconnect` for a fast-fresh retry. + `userInitiatedDisconnect` distinguishes manual `disconnect()`/`cancelConnect()` (which + set it true → cancel all reconnect state) from a network drop (which leaves it false). + On a successful reconnect, `reconnectAttempt` resets and `lastSession` clears; the + path monitor keeps watching for the next change. On user-initiated disconnect, all + reconnect state (task + path monitor + `lastSession` + `connectedServer`) is cancelled. + 3. **Backoff + restore**: exponential backoff 1s → 2s → 4s → 8s → 16s → 30s cap, + indefinite. TOFU pins match on the second connect (`VC_TOFU_MATCHED`) so the identity + gate auto-confirms; on auth success `SessionState.requestRestore` issues a + `joinChannel` and re-arms voice + restores the local mute/deafen state on the + resulting `.joinResult`. + 4. **Audio recovery** (`AudioSessionManager.swift`, `IOSVoiceProcessingEngine.swift`): + replaced the route-change handler's narrow `.oldDeviceUnavailable`/ + `.newDeviceAvailable` guard with a single intent-gated `recoverAudio()` path that + re-activates the AVAudioSession, re-applies the route config, and rebuilds the + engine; it runs on every externally-initiated route change reason except + `.categoryChange`/`.routeConfigurationChange` (those we cause ourselves and would + loop). Interruption-end now always calls `recoverAudio()` instead of only when + `.shouldResume` is set (which left the session permanently dead after Siri). Added + an `AVAudioEngineConfigurationChange` observer on the engine in `IOSAudioEngine` + that catches the case where iOS stops the engine itself AFTER our route-change + handler already rebuilt it (the previous rebuilds raced the engine's own self-stop + and lost). And `IOSAudioEngine.rebuild()` now does a one-shot reactivation-retry on + `engine.start()` failure — iOS sometimes refuses to start until the AVAudioSession is + re-activated, which is the silent-death case. + **Build:** `scripts/build-ios-client.sh --no-configure` green (Xcode 26.5 / iOS 18.0 sim + SDK, Swift 5 mode). No C ABI / `voicecat.h` / `voicecat.proto` / C core changes; the + existing TOFU auto-confirm (`VC_TOFU_MATCHED`) and idempotent `vc_join_channel` make + reconnect+restore possible without new C ABI. macOS and Windows clients unchanged. + **Next (manual, on-device):** verify unplugging AirPods/wired headphones mid-call keeps + audio going through the loudspeaker; verify Wi-Fi→cellular flip mid-call now triggers a + FAST reconnect (within a couple seconds, not 30-60 s) and lands in the same channel with + voice re-armed; verify tapping Disconnect mid-reconnect-abort cancels cleanly. + - **Done (2026-06-24):** **Three bug fixes — voice join/leave, channel edit defaults, channel-update stream restart.** 1. **Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.** Previously "Join Voice" only started the local mic — receiving was always on (gated by channel diff --git a/clients/apple/iOS/VoiceCatiOS/AppState.swift b/clients/apple/iOS/VoiceCatiOS/AppState.swift index d26cb35..3801f20 100644 --- a/clients/apple/iOS/VoiceCatiOS/AppState.swift +++ b/clients/apple/iOS/VoiceCatiOS/AppState.swift @@ -1,4 +1,5 @@ import Foundation +import Network import VoiceCatCore struct PendingIdentity: Identifiable { @@ -25,6 +26,70 @@ final class AppState { private(set) var connectingServer: SavedServer? private var identityHandled = false + /// The server we are currently fully connected to. Set on auth success (when `session` is + /// created) and cleared on teardown. Used to build the `lastSession` restore snapshot when a + /// live-session disconnect fires through `SessionState.handleEvent` — `connectingServer` is + /// already nil by then, and `handleConnectEvent`'s `server` parameter is out of scope because + /// `SessionState` owns `client.onEvent` after auth success (see `SessionState.init`). + private var connectedServer: SavedServer? + + // MARK: - Reconnect state + // + // The C core surfaces every unexpected connection drop as a `.disconnected` event; nothing + // in the C core auto-reconnects (intentional — reconnect UX is the client's job). Two layers + // drive iOS reconnect: + // + // 1. **Event-driven** (the C core's TCP read eventually fails after the keepalive/reaper + // timeout, ~30-60 s on a hard Wi-Fi drop): `SessionState.handleEvent` `.disconnected` + // plays the audible cue, then calls back into AppState via `onLiveSessionDisconnected`, + // which snapshots the live session, tears it down, and arms `scheduleReconnect`. + // (Live-session events never reach `AppState.handleConnectEvent` — `SessionState.init` + // overwrites `client.onEvent`, so `AppState` cannot see them without the callback.) + // + // 2. **Path-driven** (proactive, much faster): `NWPathMonitor` runs the whole time we are + // connected (started on auth success) and reacts to network changes — a Wi-Fi↔cellular + // flip or the path becoming `.unsatisfied` calls `proactiveReconnect`, which tears the + // live session down BEFORE the C core notices the dead TCP path. This is what makes the + // 30-60 s wait collapse into ~1 s + the backoff tick. While mid-reconnect (no session) + // the same monitor arms a fast-fresh retry whenever a path becomes `.satisfied`. + // + // Manual Disconnect cancels everything (task + path monitor) and clears `lastSession`. + + /// True at the top of `disconnect()`/`cancelConnect()` — suppresses auto-reconnect for the + /// `.disconnected` event the core then emits in response to our `vc_disconnect()` call. + private var userInitiatedDisconnect = false + + /// Snapshot of the live session state needed to restore after a reconnect. Cleared on + /// successful restore and on user-initiated disconnect. + private struct LastSession { + let server: SavedServer + let channelId: UInt32 + let voiceSubscribed: Bool + let micMuted: Bool + let deafened: Bool + } + private var lastSession: LastSession? + + /// Reconnect attempt counter — drives exponential backoff. Reset to 0 on successful auth and + /// on a path-driven fast-fresh retry. + private var reconnectAttempt = 0 + + /// The in-flight reconnect `Task` (sleeps for the backoff, then calls `connectTo`). One + /// at a time; cancelled on user disconnect / successful restore. + private var reconnectTask: Task? + + /// Started on auth success and kept running while connected / mid-reconnect; stopped only on + /// user-initiated disconnect. Its `pathUpdateHandler` (dispatched to @MainActor) handles two + /// cases: a path change while connected → proactive reconnect; a satisfied path while + /// mid-reconnect → fast-fresh retry. See the reconnect-state header comment. + private var pathMonitor: NWPathMonitor? + private let pathQueue = DispatchQueue(label: "cat.voice.network.path") + + /// Signature of the last path seen by the monitor (a stable string encoding status + active + /// interface types). The very first path callback (when the monitor starts) sets this and is + /// otherwise ignored — it's the baseline; only subsequent CHANGES are reconnect triggers. + private var lastPathSignature: String? + // MARK: - Server list management func addServer(_ server: SavedServer, password: String?) { @@ -51,14 +116,39 @@ final class AppState { ServerListStore.shared.save(servers) } + // MARK: - Teardown + + /// `AppState` is the @Observable app root owned by the SwiftUI `App`; it lives for the + /// whole app process and is torn down only on process exit, at which point OS cleanup + /// suffices. Reconnect state (the in-flight `Task` and the `NWPathMonitor`) is cancelled + /// via `cancelReconnect()` driven by `disconnect()`/`cancelConnect()` and on successful + /// restore — those run on user-initiated teardown, which is the only path that matters. + /// (The `Task` captures `[weak self]` and guards on `nil`/`userInitiatedDisconnect`, so a + /// stray task left running when AppState is gone is a no-op; the monitor similarly guards.) + // MARK: - Connect flow + /// Public connect entry. Always starts a fresh session (no restore). func connectTo(_ server: SavedServer) { + connectTo(server, restoring: nil) + } + + /// Internal connect that drives the full TLS/auth saga. `restoring` is non-nil for a + /// reconnect attempt following an unexpected disconnect; the captured channel + voice/mic + /// state is handed to the new `SessionState` after auth succeeds. + private func connectTo(_ server: SavedServer, restoring: LastSession?) { guard !isConnecting else { return } isConnecting = true - connectStatus = "Connecting…" + connectStatus = (restoring != nil) ? "Reconnecting…" : "Connecting…" connectingServer = server identityHandled = false + userInitiatedDisconnect = false + + // Discard any leftover connecting client. nil'ing the strong ref calls VoiceCatClient's + // deinit, which synchronously joins the C core's io thread (vc_client_destroy) before + // freeing the config-string storage — safe from @MainActor because the io thread never + // blocks on main (it enqueues events via DispatchQueue.main.async and returns). + connectingClient = nil let config = VoiceCatConfig( clientName: "VoiceCat-iOS", @@ -69,7 +159,9 @@ final class AppState { connectingClient = client client.onEvent = { [weak self] ev in - Task { @MainActor [weak self] in self?.handleConnectEvent(ev, server: server) } + Task { @MainActor [weak self] in + self?.handleConnectEvent(ev, server: server, restoring: restoring) + } } // Put the core into external-playback mode BEFORE connect, so the flag is set on the // io thread before any message is processed. The server sends AuthResult immediately @@ -82,7 +174,9 @@ final class AppState { client.setExternalPlayback(true) client.connect(host: server.host, port: server.port) - // Auth is queued immediately — the core serialises it behind TLS + TOFU. + // Auth is queued immediately — the core serialises it behind TLS + TOFU. On a reconnect + // the TOFU pin already matches (VC_TOFU_MATCHED), so the identity gate auto-confirms + // inside the .serverIdentity case below and auth proceeds unattended. switch server.authMode { case .guest: let nick = (server.nickname?.isEmpty == false) ? server.nickname! : "iOS User" @@ -98,6 +192,11 @@ final class AppState { } func disconnect() { + // Mark BEFORE we ask the core to disconnect, so the .disconnected event the core emits + // in response is treated as user-initiated (no reconnect) rather than an unexpected drop. + userInitiatedDisconnect = true + cancelReconnect() + lastSession = nil session?.leaveVoice() session?.client.disconnect() IOSAudioEngine.shared.stop() @@ -106,6 +205,7 @@ final class AppState { connectingClient?.disconnect() connectingClient = nil connectingServer = nil + connectedServer = nil isConnecting = false connectStatus = "" showPasswordPrompt = false @@ -126,26 +226,210 @@ final class AppState { } func cancelConnect() { + // User explicitly cancelled — no reconnect for the resulting .disconnected event. + userInitiatedDisconnect = true + cancelReconnect() + lastSession = nil connectingClient?.disconnect() connectingClient = nil connectingServer = nil + connectedServer = nil isConnecting = false connectStatus = "" showPasswordPrompt = false pendingIdentity = nil } + // MARK: - Reconnect orchestration + + /// Cancel any in-flight reconnect task and stop the path monitor. Safe to call when nothing + /// is armed (no-op). Does NOT touch `userInitiatedDisconnect` or `lastSession` — callers set + /// those as needed (disconnect/cancelConnect clear them; scheduleReconnect keeps them). + private func cancelReconnect() { + reconnectTask?.cancel() + reconnectTask = nil + stopPathMonitor() + // Don't reset `reconnectAttempt` here: scheduleReconnect resets it on successful auth, + // and the path monitor resets it to 0 for a fast fresh attempt on a path-satisfied event. + // If a fresh user connect follows, connectTo() doesn't reset it either, but it doesn't + // need to — `reconnectAttempt` only matters while we're mid-reconnect. + } + + /// Arm the next reconnect attempt with exponential backoff (1s → 2s → 4s → 8s → 16s → 30s + /// cap). Cancelled cleanly by `cancelReconnect()` on user disconnect or successful auth. + /// Idempotent: a new call supersedes any in-flight one. The path monitor is armed here and + /// disarmed on cancel; on a path-satisfied event it resets the attempt counter to 0 and + /// re-arms via this same method, yielding a fast refresh after Wi-Fi ↔ cellular transitions. + private func scheduleReconnect() { + guard !userInitiatedDisconnect, let last = lastSession else { return } + reconnectTask?.cancel() + reconnectAttempt = max(1, reconnectAttempt + 1) + let delaySec = min(pow(2.0, Double(reconnectAttempt - 1)), 30.0) + connectStatus = "Reconnecting (attempt \(reconnectAttempt))…" + + startPathMonitor() + + let task = Task { [weak self, last] in + guard let self else { return } + try? await Task.sleep(nanoseconds: UInt64(delaySec * 1_000_000_000)) + if Task.isCancelled { return } + // Re-check under Task: a user disconnect between the sleep and this line must abort. + guard !self.userInitiatedDisconnect else { return } + guard self.lastSession != nil else { return } + guard self.session == nil else { return } + self.connectTo(last.server, restoring: last) + } + reconnectTask = task + } + + /// Start (if not already running) the network path monitor. Runs the whole time we are + /// connected (started on auth success) and stays armed across reconnects; stopped only on + /// user-initiated disconnect. The handler dispatches to @MainActor before touching state and + /// does two distinct things: + /// - **While connected** (`session != nil`): a Wi-Fi↔cellular interface change OR the path + /// becoming `.unsatisfied` triggers `proactiveReconnect()` — tearing the live session + /// down before the C core notices the dead TCP read. Without this the disconnect would + /// take 30-60 s (the TCP keepalive/reaper timeout); proactive teardown collapses that to + /// ~1 s + the first backoff tick. Same-interface path refreshes (e.g. a Wi-Fi roam + /// without an IP change) are ignored — likely the connection is still good. + /// - **While mid-reconnect** (`session == nil`, `lastSession != nil`): a path becoming + /// `.satisfied` arms a fast-fresh retry (backoff counter reset, `scheduleReconnect`). + /// This is what makes a Wi-Fi→cellular flip reconnect on roughly the next tick instead + /// of waiting out a long backoff. + private func startPathMonitor() { + guard pathMonitor == nil else { return } + let monitor = NWPathMonitor() + monitor.pathUpdateHandler = { [weak self] path in + Task { @MainActor [weak self] in + guard let self else { return } + guard !self.userInitiatedDisconnect else { return } + let sig = Self.pathSignature(path) + let prevSig = self.lastPathSignature + self.lastPathSignature = sig + // The first callback (when the monitor starts) is the baseline, not a change. + if prevSig == nil { return } + + if self.session != nil { + // Connected — tear down + reconnect on a meaningful path change. + // `.unsatisfied` (all radios off) OR a primary-interface change (Wi-Fi↔cellular) + // almost always breaks the live TCP connection; reconnecting proactively beats + // waiting for the C core's keepalive/reaper timeout. + if path.status != .satisfied || sig != prevSig { + self.proactiveReconnect() + } + } else if self.lastSession != nil { + // Mid-reconnect — a path is available again; fast-fresh the next attempt. + if path.status == .satisfied { + self.reconnectAttempt = 0 + self.scheduleReconnect() + } + } + } + } + // Listen on a dedicated queue — the path monitor can't share the main queue (it would + // re-enter main if any handler dispatched to main synchronously). + monitor.start(queue: pathQueue) + pathMonitor = monitor + } + + private func stopPathMonitor() { + pathMonitor?.cancel() + pathMonitor = nil + lastPathSignature = nil + } + + /// A stable string signature of a network path: the path status plus the set of interface + /// types it uses. Two paths with the same signature are treated as equivalent — no + /// reconnect. A signature change is the trigger for `proactiveReconnect`. Used to ignore + /// same-interface refreshes (signal-strength changes, BSSID roams) which usually don't break + /// the TCP connection. + private static func pathSignature(_ path: NWPath) -> String { + guard path.status == .satisfied else { return "unsatisfied" } + var parts: [String] = [] + if path.usesInterfaceType(.wifi) { parts.append("wifi") } + if path.usesInterfaceType(.cellular) { parts.append("cellular") } + if path.usesInterfaceType(.wiredEthernet) { parts.append("wired") } + if path.usesInterfaceType(.other) { parts.append("other") } + return parts.isEmpty ? "none" : parts.sorted().joined(separator: "+") + } + + // MARK: - Live-session disconnect (called by SessionState) + + /// Called by `SessionState.handleEvent` `.disconnected` after the audible cue has already + /// played. Once `SessionState` is created (auth success) it owns `client.onEvent`, so + /// `AppState.handleConnectEvent` never sees live-session events — this callback is the only + /// way AppState learns that a live session dropped. Snapshots the live session state, + /// tears the session down, and arms `scheduleReconnect` so the backoff loop drives a fresh + /// TLS/auth/restoration. Guarded against user-initiated disconnect (which nil's `session` + /// synchronously, so SessionState is gone before the event could fire this callback) — but + /// the guard is cheap insurance. + func onLiveSessionDisconnected() { + guard !userInitiatedDisconnect else { return } + teardownLiveSessionAndReconnect(sound: false) + } + + /// Called by `NWPathMonitor` when a path change is detected while a live session exists. + /// Tears the session down immediately — nil'ing `session` releases `VoiceCatClient`, whose + /// `deinit` calls `vc_client_destroy`; that closes the socket and joins the C core's io + /// thread, so the io thread exits in milliseconds rather than blocking on a dead read for + /// ~30-60 s. The proactive tear-down is what collapses the long TCP-reaper wait into a + /// ~1 s reconnect. Plays the audible cue (no `.disconnected` event fires through to + /// `SessionState` for this path, since `SessionState` is being torn down here — so the cue + /// would otherwise be missing). + private func proactiveReconnect() { + guard !userInitiatedDisconnect else { return } + guard session != nil else { return } + teardownLiveSessionAndReconnect(sound: true) + } + + /// Shared teardown for a live-session disconnect (event- or path-driven). Snapshots the live + /// session into `lastSession`, stops audio, deactivates the AVAudioSession, releases the + /// session (which releases `VoiceCatClient` → io-thread join), resets the backoff counter, + /// and arms `scheduleReconnect`. `sound` is true for the proactive (path-driven) case — the + /// `.disconnected` event that would have played it never fires because we're tearing down + /// ahead of the C core noticing. The event-driven caller (`onLiveSessionDisconnected`) has + /// ALREADY played the cue via `SessionState.handleEvent`, so it passes `sound: false`. + private func teardownLiveSessionAndReconnect(sound: Bool) { + // Snapshot BEFORE nil'ing `session` — we need the channel + voice/mic state to restore. + if let s = session, let srv = connectedServer { + lastSession = LastSession( + server: srv, + channelId: s.currentChannelId, + voiceSubscribed: s.voiceState.voiceSubscribed, + micMuted: s.voiceState.selfMuted, + deafened: s.voiceState.selfDeafened) + } + IOSAudioEngine.shared.stop() + AudioSessionManager.shared.deactivateSession() + // Releasing `session` releases `VoiceCatClient`; its deinit joins the C core's io thread. + // For a path-driven proactive teardown this is what avoids the 30-60 s reaper timeout. + session = nil + isConnecting = false + connectingClient = nil + connectedServer = nil + if sound { + EventFeedback.shared.play(.connectionLost) + EventFeedback.shared.speak("Network changed — reconnecting") + } + // Reset the backoff counter so the first reconnect attempt after a drop uses the short + // 1 s delay (the immediate path-driven attempt matters most; sustained-outage backoff is + // driven by `scheduleReconnect`'s increment). + reconnectAttempt = 0 + scheduleReconnect() + } + // MARK: - Connect event handler - private func handleConnectEvent(_ ev: VoiceCatEvent, server: SavedServer) { + private func handleConnectEvent(_ ev: VoiceCatEvent, server: SavedServer, + restoring: LastSession?) { switch ev.type { case .connectionState: switch ev.connectionState { - case .connecting: connectStatus = "Connecting…" + case .connecting: connectStatus = (restoring != nil) ? "Reconnecting…" : "Connecting…" case .tlsHandshake: connectStatus = "TLS handshake…" case .authenticating: connectStatus = "Authenticating…" case .verifyingIdentity: connectStatus = "Verifying server identity…" - case .connected: connectStatus = "Connected" + case .connected: connectStatus = "Connected" default: break } case .serverIdentity: @@ -163,13 +447,15 @@ final class AppState { guard let client = connectingClient else { break } let perms = client.getPermissions() let newSession = SessionState(client: client, selfUserId: ev.userId, permissions: perms) + newSession.appState = self connectingClient = nil isConnecting = false connectStatus = "" showPasswordPrompt = false + connectedServer = server self.session = newSession EventFeedback.shared.play(.login) - EventFeedback.shared.speak("Connected") + EventFeedback.shared.speak(restoring != nil ? "Reconnected" : "Connected") // External-playback mode was enabled before connect() so the core never opens a // miniaudio device on iOS (the single ordering rule of the unified audio path). // Now activate the session and start the engine in listening mode so remote audio @@ -180,20 +466,67 @@ final class AppState { print("Audio session activate on connect failed: \(error)") } IOSAudioEngine.shared.startListening(client: client) + // The path monitor runs the whole time we're connected so a network change fires + // proactiveReconnect immediately instead of waiting for the C core's TCP keepalive + // timeout (~30-60 s on a hard Wi-Fi drop). It stays armed across reconnects and is + // stopped only on user-initiated disconnect. + startPathMonitor() + + // Reconnect restore: rejoin the prior channel and re-enable voice/mic if they + // were on. The session is fresh (server auto-places us in Lobby), so the restore + // is driven through SessionState.requestRestore, which issues a JoinChannel then + // (on the resulting .joinResult) re-arms voice + mute/deafen. A successful auth + // means the server is reachable, so the backoff counter resets and `lastSession` + // clears; the path monitor keeps watching for the next change. + if let restoring { + newSession.requestRestore(channelId: restoring.channelId, + voiceSubscribed: restoring.voiceSubscribed, + micMuted: restoring.micMuted, + deafened: restoring.deafened) + reconnectAttempt = 0 + lastSession = nil + } } else { connectStatus = "Auth failed: \(ev.result.description)" showPasswordPrompt = true } case .disconnected: - if session == nil { cancelConnect() } - else { - IOSAudioEngine.shared.stop() - AudioSessionManager.shared.deactivateSession() - session = nil; isConnecting = false + // This handler runs ONLY during the connecting phase — after auth success + // `SessionState.init` overwrites `client.onEvent`, so a live-session disconnect + // reaches `SessionState.handleEvent` and comes back via + // `onLiveSessionDisconnected`, not here. Two outcomes for this branch: + // - A reconnect's connecting phase failed (`lastSession != nil`, set by a prior + // teardown) → re-arm `scheduleReconnect` so the backoff loop continues. + // - A fresh connect failed before auth (`lastSession == nil`) → show the error, do + // not auto-reconnect (the user should retry manually once the server is reachable). + connectingClient = nil + isConnecting = false + IOSAudioEngine.shared.stop() + AudioSessionManager.shared.deactivateSession() + + if userInitiatedDisconnect { + connectStatus = "" + showPasswordPrompt = false + pendingIdentity = nil + lastSession = nil + connectedServer = nil + cancelReconnect() + } else if lastSession != nil { + // Mid-reconnect drop — keep the backoff loop going. + EventFeedback.shared.play(.connectionLost) + EventFeedback.shared.speak("Connection lost — reconnecting") + scheduleReconnect() + } else { + // Fresh connect failed before auth. Surface the reason; no auto-reconnect. + connectStatus = ev.text ?? "Disconnected" + showPasswordPrompt = false + pendingIdentity = nil + connectedServer = nil + cancelReconnect() } case .error: connectStatus = ev.text ?? "Unknown error" - if session == nil { isConnecting = false } + // Errors don't disconnect us; the .disconnected event handles teardown/reconnect. default: break } diff --git a/clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift b/clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift index a1c4cbf..f2a4d36 100644 --- a/clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift +++ b/clients/apple/iOS/VoiceCatiOS/AudioSessionManager.swift @@ -35,6 +35,27 @@ final class AudioSessionManager { name: AVAudioSession.routeChangeNotification, object: nil) } + /// The single end-to-end audio recovery path, driven by *intent* (`IOSAudioEngine.isConnected`) + /// — not by session bookkeeping flags that can drift out of sync (e.g. an interruption ended + /// without `.shouldResume`, which used to leave `isSessionActive` false forever). Safe to call + /// speculatively: the underlying calls are idempotent (AVAudioSession.setActive(true), + /// `IOSAudioRouter.applyConfiguration` has a re-entrancy guard, `IOSAudioEngine.reconfigure` + /// no-ops when not connected). Call this whenever the audio environment changes in a way that + /// could have stopped the engine — interruption end, route change, AVAudioEngine + /// configuration-change — and we still want audio back. + func recoverAudio() { + guard IOSAudioEngine.shared.isConnected else { return } + do { + try ensureSessionActive() + } catch { + logger.error("recoverAudio — session activate failed: \(error.localizedDescription)") + } + IOSAudioRouter.shared.applyConfiguration() + if isSessionActive { IOSAudioRouter.shared.applyA2dpSpeakerFallback() } + IOSAudioEngine.shared.reconfigure() + logSessionState("after recoverAudio") + } + /// Activate the AVAudioSession if not already active. Call before any audio I/O: /// when the user joins voice, or when a remote stream starts (so playback works even /// before the user has joined voice). Idempotent — safe to call multiple times. @@ -110,21 +131,13 @@ final class AudioSessionManager { logger.info("interruption began — session suspended by system") isSessionActive = false case .ended: - let optionsValue = info[AVAudioSessionInterruptionOptionKey] as? UInt ?? 0 - let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue) - if options.contains(.shouldResume) { - do { - IOSAudioRouter.shared.applyConfiguration() - try AVAudioSession.sharedInstance().setActive(true) - isSessionActive = true - IOSAudioRouter.shared.applyA2dpSpeakerFallback() - // Rebuild the engine graph against the restored route (both directions). - IOSAudioEngine.shared.reconfigure() - logger.info("interruption ended — session reactivated, engine rebuilt") - } catch { - logger.error("interruption ended — reactivation failed: \(error.localizedDescription)") - } - } + // Always attempt recovery when we have a live session. iOS sometimes ends an + // interruption without the `.shouldResume` hint (e.g. Siri), and the previous + // behavior of only reactivating when `.shouldResume` was set left the session + // permanently dead — audio never came back. `recoverAudio()` is intent-gated on + // `IOSAudioEngine.isConnected` and idempotent, so speculatively calling it is safe. + logger.info("interruption ended — recovery requested") + recoverAudio() @unknown default: break } } @@ -134,37 +147,31 @@ final class AudioSessionManager { let reasonValue = info[AVAudioSessionRouteChangeReasonKey] as? UInt, let reason = AVAudioSession.RouteChangeReason(rawValue: reasonValue) else { - logger.warning("routeChange — unknown reason, refreshing only") + logger.warning("routeChange — unknown reason, refreshing + recovery") IOSAudioRouter.shared.refreshRoutes() NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil) + recoverAudio() return } logger.info("routeChange reason=\(self.reasonLabel(reason))") - - // Re-apply preferences ONLY on external device plug/unplug. Do NOT re-apply on - // .categoryChange / .routeConfigurationChange — those are triggered by our own - // applyConfiguration() calls (setCategory, setPreferredInput, etc.), and re-applying - // would create an infinite notification loop: - // handleRouteChange → applyConfiguration → setCategory → routeChange → ... - // That loop burns CPU and cycles the audio session on/off — the "glitching" bug. - // IOSAudioRouter.applyConfiguration() also has a re-entrancy guard for synchronous - // notifications, but the reason check here is the primary defense. - if reason == .oldDeviceUnavailable || reason == .newDeviceAvailable { - logger.info("routeChange — external device change, re-applying config") - IOSAudioRouter.shared.applyConfiguration() - // Re-evaluate the A2DP-mode speaker fallback: a Bluetooth unplug should drop us onto - // the loud speaker (not the earpiece), and a replug should hand output back to A2DP. - if isSessionActive { - IOSAudioRouter.shared.applyA2dpSpeakerFallback() - // Rebind the engine (both directions) to the new route. The engine owns the route - // now, so this is the single thing that re-establishes audio after a device change. - IOSAudioEngine.shared.reconfigure() - } - } - IOSAudioRouter.shared.refreshRoutes() NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil) + + // Recover audio on every externally-initiated route change. `.categoryChange` and + // `.routeConfigurationChange` are fired by our OWN applyConfiguration() calls + // (setCategory, setPreferredInput, ...). Acting on them would create a tight ping-pong + // loop with the re-entrancy guard (handleRouteChange → recoverAudio → + // applyConfiguration → setCategory → routeChange → ...). IOSAudioRouter's guard is the + // backstop that bounds it to ONE extra iteration, but skipping these two reasons avoids + // even that, so we reconfigure only in response to genuine environmental changes. + // + // The recovery set below (oldDeviceUnavailable, newDeviceAvailable, override, + // wakeFromSleep, noSuitableRouteForCategory, unknown) covers headphone/AirPods/wired + // unplug-replug — the previously-reported "audio dies when headphones disconnect" bug. + if reason != .categoryChange && reason != .routeConfigurationChange { + recoverAudio() + } logSessionState("route change (\(reasonLabel(reason)))") } @@ -177,6 +184,7 @@ final class AudioSessionManager { case .wakeFromSleep: return "wakeFromSleep" case .noSuitableRouteForCategory: return "noSuitableRouteForCategory" case .routeConfigurationChange: return "routeConfigurationChange" + case .unknown: return "unknown" @unknown default: return "unknown" } } diff --git a/clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift b/clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift index 335974d..51856a8 100644 --- a/clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift +++ b/clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift @@ -187,6 +187,29 @@ final class IOSAudioEngine { renderScratch.initialize(repeating: 0, count: renderScratchFrames * 2) micDrainScratch = UnsafeMutablePointer.allocate(capacity: 960 * 2) micDrainScratch.initialize(repeating: 0, count: 960 * 2) + + // AVAudioEngine stops itself on a mid-session route/configuration change (it stops + // if its I/O graph no longer matches the active route). Our route-change handler in + // AudioSessionManager normally rebuilds us before the user notices, but if the engine + // stops itself AFTER our recovery (because the route-change notification raced ahead + // of the engine's own self-stop), nothing restarts it. Catch that case here. + NotificationCenter.default.addObserver( + self, selector: #selector(handleEngineConfigurationChange), + name: .AVAudioEngineConfigurationChange, object: engine) + } + + /// The engine stopped itself because its configuration no longer matches the active AVAudio + /// route (this fires after a route change that the route-change handler can't always outrun). + /// Dispatch to main and call the unified `recoverAudio()` — it's intent-gated on + /// `isConnected`, idempotent, and no-ops if the engine is already running (the common case + /// where our route-change handler got there first). + @objc private func handleEngineConfigurationChange(_ notification: Notification) { + Task { @MainActor [weak self] in + guard let self else { return } + guard self.isConnected, !self.engine.isRunning else { return } + logger.info("engine configuration-change — engine stopped itself, recovering") + AudioSessionManager.shared.recoverAudio() + } } // MARK: - Lifecycle @@ -308,7 +331,29 @@ final class IOSAudioEngine { inFormat=\(inFmt) outputNode=\(outFmt) outputRoute=[\(route)] """) } catch { - logger.error("engine start failed: \(error.localizedDescription)") + // iOS occasionally refuses to start the engine immediately after a route change — + // the AVAudioSession needs a re-activation nudge before the engine will start. Do + // ONE recovery attempt: re-activate the session, re-apply the route config, then + // try `engine.start()` again. Recovering here is what fixes the silent-death bug + // where unplugging headphones left the engine stopped forever. + logger.error("engine start failed: \(error.localizedDescription) — attempting one-shot recovery") + do { + try AudioSessionManager.shared.ensureSessionActive() + } catch { + logger.error("recovery — session re-activate failed: \(error.localizedDescription)") + } + IOSAudioRouter.shared.applyConfiguration() + if AudioSessionManager.shared.isActive { + IOSAudioRouter.shared.applyA2dpSpeakerFallback() + } + do { + try engine.start() + logger.info("engine start succeeded after one-shot recovery") + } catch { + logger.error("engine start failed after recovery: \(error.localizedDescription)") + // Not fatal — a subsequent route-change or AVAudioEngine configuration-change + // notification will trigger recoverAudio() and re-attempt the rebuild. + } } // Start the feed pump last, with the current channel count, so it never carries a stale diff --git a/clients/apple/iOS/VoiceCatiOS/SessionState.swift b/clients/apple/iOS/VoiceCatiOS/SessionState.swift index 4d0acb0..3690f9b 100644 --- a/clients/apple/iOS/VoiceCatiOS/SessionState.swift +++ b/clients/apple/iOS/VoiceCatiOS/SessionState.swift @@ -54,6 +54,30 @@ final class SessionState { var accounts: [Account] = [] var devices: [Device] = [] + /// Back-reference to the app state. Once `SessionState.init` overwrites `client.onEvent` + /// (`SessionState.swift:87`), `AppState.handleConnectEvent` no longer receives per-session + /// events — so the `.disconnected` event for a LIVE session arrives here in `handleEvent`, + /// not in AppState. This weak ref lets us hand the disconnect back to AppState (which owns + /// the reconnect state machine) so the auto-reconnect fires. Set by AppState on auth success. + weak var appState: AppState? + + // MARK: - Reconnect restore state + // + // When the iOS client auto-reconnects after a network drop, AppState captures the prior + // session's channel + voice/mic state and asks the new SessionState (created on auth success) + // to restore it. We rejoin the channel explicitly (the server auto-placed us in Lobby on + // auth), and on the resulting `.joinResult` we re-arm voice subscription + mute/deafen. The + // drive is here, not in AppState, because once SessionState is created it owns + // `client.onEvent` and AppState no longer sees per-session events. + private struct RestoreRequest { + let channelId: UInt32 + let voiceSubscribed: Bool + let micMuted: Bool + let deafened: Bool + } + private var pendingRestore: RestoreRequest? + private var didIssueRestoreJoin = false + /// Host side of iOS screen-audio sharing — drains the broadcast extension's App Group ring /// and feeds the SCREEN_AUDIO stream this session owns. See BroadcastAudioPump. private let broadcastPump = BroadcastAudioPump() @@ -186,8 +210,24 @@ final class SessionState { currentChannelId = ev.channelId addActivity("Joined channel") refreshUsers() + // Reconnect restore: this was our restore-join. Now that the server has + // processed the channel move, re-arm voice subscription (if the user was + // transmitting before the drop) and re-apply the local mute/deafen state. + // The server returns ok even when joining the channel we're already in, so + // this fires reliably for the Lobby-too case. + if didIssueRestoreJoin, let r = pendingRestore, r.channelId == ev.channelId { + didIssueRestoreJoin = false + completeRestore() + } } else { addActivity("Join failed: \(ev.result.description)") + // Restore-join failed (channel was deleted, became password-protected or + // full while we were away). Give up on the voice/mute restore cleanly so we + // don't leave dangling state or attempt voice without being in a channel. + if didIssueRestoreJoin { + didIssueRestoreJoin = false + pendingRestore = nil + } } case .error: addActivity("Error: \(ev.text ?? ev.result.description)") @@ -198,9 +238,15 @@ final class SessionState { case .accountList: accounts = client.listAccounts() case .disconnected: - // Audible cue only — session teardown is driven elsewhere (AppState / UI). + // Audible cue, then hand the disconnect back to AppState so its reconnect state + // machine fires. This is the ONLY way AppState learns a live session dropped — + // after auth success, `SessionState.init` overwrites `client.onEvent`, so + // `AppState.handleConnectEvent` never sees this event. (Without this callback, a + // network drop on a live session would just play the cue and leave the session as a + // zombie — the user would have to tap Disconnect manually.) EventFeedback.shared.play(ev.result == .ok ? .logout : .connectionLost) EventFeedback.shared.speak(ev.result == .ok ? "Disconnected" : "Connection lost") + appState?.onLiveSessionDisconnected() default: break } @@ -300,6 +346,49 @@ final class SessionState { client.leaveVoice() } + // MARK: - Reconnect restore + + /// Called by `AppState` after a reconnect's auth success to rejoin the prior channel and + /// re-enable the prior voice/mic state. Drives the restore through the `.joinResult` event + /// so we re-arm voice only AFTER the server processed the join — joining voice before the + /// channel move would be rejected server-side. `micMuted`/`deafened` are the user's LOCAL + /// mute/deafen state at the moment of the drop; the server resets those on a fresh auth, so + /// we re-push them via `setMute` after the channel is restored. + func requestRestore(channelId: UInt32, voiceSubscribed: Bool, + micMuted: Bool, deafened: Bool) { + pendingRestore = RestoreRequest(channelId: channelId, + voiceSubscribed: voiceSubscribed, + micMuted: micMuted, + deafened: deafened) + didIssueRestoreJoin = false + if channelId != 0 { + // The server auto-placed us in the Lobby on auth; join our prior channel explicitly. + // `vc_join_channel` is idempotent server-side (joining the channel you're already in + // returns ok), so this is safe even if the prior channel was the Lobby. + client.joinChannel(channelId) + didIssueRestoreJoin = true + } else { + // No prior channel — go straight to the voice/mute restore. (voiceSubscribed with + // channelId == 0 is contradictory; `completeRestore` further guards on + // currentChannelId != 0 before subscribing to voice.) + completeRestore() + } + } + + /// Finish the restore after the channel is in place (or there was no channel to restore): + /// re-subscribe to voice if the user was transmitting, and re-apply the local mute/deafen + /// state. Safe to call once per `pendingRestore`; clears it. + private func completeRestore() { + guard let r = pendingRestore else { return } + if r.voiceSubscribed && currentChannelId != 0 { + joinVoice() + } + setMute(r.micMuted, deafened: r.deafened) + addActivity("Restored to channel \(currentChannelId)" + + (r.voiceSubscribed ? " with voice" : "")) + pendingRestore = nil + } + // MARK: - Screen audio share /// Called when the broadcast extension becomes active. Announces the SCREEN_AUDIO stream;