docs: condense implementation comments
This commit is contained in:
10
PROGRESS.md
10
PROGRESS.md
@@ -10,6 +10,16 @@ up instantly. Newest status at the top.
|
|||||||
|
|
||||||
## ▶ Where we left off / next action
|
## ▶ Where we left off / next action
|
||||||
|
|
||||||
|
- **Done (2026-07-23):** **First comment-density cleanup across core, server, and native
|
||||||
|
clients.** Condensed comments in the highest-noise audio, reconnect, registry, and binding
|
||||||
|
files; removed implementation history and narration; retained ABI ownership, threading,
|
||||||
|
real-time, ordering, and OS-API invariants. Moved the durable iOS audio-routing/pacing rules
|
||||||
|
to `docs/voice.md` and client reconnection policy to `docs/protocol.md`. No behavior, wire
|
||||||
|
format, or C ABI changes. **Verification:** `cmake --build --preset dev` green;
|
||||||
|
`ctest --preset dev` 29/29 green. `dotnet build VoiceCat.slnx` restores dependencies and
|
||||||
|
builds `VoiceCat.Interop` + `VoiceCat.App`, then fails in the unchanged test project because
|
||||||
|
`ExternalPcmTests.cs:49` references internal `NativeMethods` (`CS0122`).
|
||||||
|
|
||||||
- **Done (2026-06-25):** **Fixed iOS AirPods-disconnect reinitialize loop on A2DP presets
|
- **Done (2026-06-25):** **Fixed iOS AirPods-disconnect reinitialize loop on A2DP presets
|
||||||
(Stereo Mic / Mono Mic).** Regression from the 2026-06-25 audio-device-change recovery
|
(Stereo Mic / Mono Mic).** Regression from the 2026-06-25 audio-device-change recovery
|
||||||
commit below, which broadened the route-change recovery set from
|
commit below, which broadened the route-change recovery set from
|
||||||
|
|||||||
@@ -1,19 +1,5 @@
|
|||||||
// Callbacks — the C function pointers passed to `vc_callbacks`. These are the Swift
|
// C callbacks use an unretained `user` context. Client destruction joins callback threads,
|
||||||
// equivalent of the C# client's `[UnmanagedCallersOnly]` static methods (NativeCallbacks.cs).
|
// and transient event pointers are copied before the callback returns.
|
||||||
//
|
|
||||||
// The critical patterns (carried over from the proven C# implementation):
|
|
||||||
// 1. `@convention(c)` closures — plain C function pointers, NOT GC/ARC-managed closures.
|
|
||||||
// A @convention(c) closure cannot capture context, which is why the `user` pointer is
|
|
||||||
// used to resolve back to the VoiceCatClient instance (the C# version uses GCHandle for
|
|
||||||
// the same thing; Swift uses Unmanaged).
|
|
||||||
// 2. `Unmanaged.passUnretained(self).toOpaque()` as the `user` context — a stable raw
|
|
||||||
// pointer to the Swift object WITHOUT incrementing the retain count. This is safe
|
|
||||||
// because `deinit` calls `vc_client_destroy` (which synchronously joins every internal
|
|
||||||
// thread) BEFORE the object's memory is freed — so no callback can fire after the object
|
|
||||||
// is gone. (The C# equivalent: GCHandle.Alloc + GCHandle.Free in Dispose.)
|
|
||||||
// 3. Copy `ev.text` to a Swift `String` INSIDE `onEvent` (via `VoiceCatEvent.from(_:)`)
|
|
||||||
// before returning — the raw pointer is dangling after the callback returns. This is
|
|
||||||
// the #1 lifetime rule from voicecat.h's vc_event doc comment.
|
|
||||||
|
|
||||||
import VoiceCatC
|
import VoiceCatC
|
||||||
import Foundation
|
import Foundation
|
||||||
|
|||||||
@@ -1,36 +1,6 @@
|
|||||||
// VoiceCatClient — the public, Swift-idiomatic surface over libvoicecat. This is the Swift
|
// Swift binding invariants: native config strings outlive the handle, destroy joins callback
|
||||||
// analog of the C# client's `VoiceCatClient.cs` (clients/windows/VoiceCat.Interop).
|
// threads before deallocation, and callback payloads are copied before main-queue delivery.
|
||||||
//
|
// See docs/architecture.md §4 for the complete binding contract.
|
||||||
// Key patterns carried over from the proven C# implementation (see docs/architecture.md §4
|
|
||||||
// per-platform binding notes):
|
|
||||||
//
|
|
||||||
// 1. HANDLE OWNERSHIP: the class owns `vc_client*`; `deinit` calls `vc_client_destroy`
|
|
||||||
// (which synchronously joins every internal thread, so nothing can still be reading the
|
|
||||||
// config-string pointers or firing callbacks by the time it returns).
|
|
||||||
//
|
|
||||||
// 2. CONFIG STRING LIFETIMES: the core stores raw pointers from `vc_config` by value — it
|
|
||||||
// does NOT copy the string data. `client_name`/`client_version`/`tofu_store_path` are
|
|
||||||
// read later, whenever `connect()` actually runs on the io_thread_. So the native CString
|
|
||||||
// storage (`_clientNamePtr` etc.) must outlive the WHOLE client, not just `init`. It's
|
|
||||||
// freed in `deinit`, AFTER `vc_client_destroy` has returned. (C#: Marshal.StringToCoTask
|
|
||||||
// MemUTF8 in ctor, FreeCoTaskMem in Dispose after destroy.)
|
|
||||||
//
|
|
||||||
// 3. EVENT DELIVERY THREAD HANDOFF: `on_event` fires on the core's event thread. Events are
|
|
||||||
// buffered in a lock-protected array and drained on `DispatchQueue.main` — this is the
|
|
||||||
// boundary where the core's thread hands off to the UI thread. The C# analog is
|
|
||||||
// `Channel<VoiceCatEvent>` drained by a 30ms WinForms Timer; the Swift analog is a
|
|
||||||
// coalesced main-queue drain (only one async block scheduled at a time). `on_event`'s
|
|
||||||
// `text` is copied to a Swift `String` inside the callback (Callbacks.swift) before
|
|
||||||
// enqueueing — the raw pointer is dangling by the time the main thread drains.
|
|
||||||
//
|
|
||||||
// 4. LEVEL METER COALESCING: `on_level` fires far more often than `on_event` and
|
|
||||||
// intermediate values are visually irrelevant — coalesced to "latest sample per
|
|
||||||
// stream_id" in a lock-protected dictionary, drained on main alongside events.
|
|
||||||
// (C#: ConcurrentDictionary<uint,float> cleared in PumpEvents.)
|
|
||||||
//
|
|
||||||
// 5. IMMEDIATE vc_free_* ON LIST READS: `listChannels()`/`listUsers()`/etc. walk the native
|
|
||||||
// array, convert to Swift value types, and call `vc_free_*_list` INSIDE the function —
|
|
||||||
// callers never manage native list lifetime. (C#: Marshaling.ToManaged does the same.)
|
|
||||||
|
|
||||||
import VoiceCatC
|
import VoiceCatC
|
||||||
import Foundation
|
import Foundation
|
||||||
@@ -56,26 +26,14 @@ public final class VoiceCatClient {
|
|||||||
|
|
||||||
// MARK: - Stored properties
|
// MARK: - Stored properties
|
||||||
|
|
||||||
/// The opaque C handle (`vc_client*` — Swift imports the incomplete C struct as
|
|
||||||
/// `OpaquePointer`). Set in `init`, passed to every C function, destroyed in `deinit`.
|
|
||||||
private var handle: OpaquePointer?
|
private var handle: OpaquePointer?
|
||||||
|
|
||||||
/// Unmanaged pointer to `self` — passed as `vc_callbacks.user` so the C function-pointer
|
/// Unretained callback context; destroying the handle joins callback threads first.
|
||||||
/// callbacks can resolve back to this instance. `passUnretained` (not `passRetained`)
|
|
||||||
/// because we want normal ARC to control the object's lifetime — `deinit` calls
|
|
||||||
/// `vc_client_destroy` (joins all threads) before the object's memory is freed, so no
|
|
||||||
/// callback can fire with a dangling `user` pointer. See Callbacks.swift.
|
|
||||||
///
|
|
||||||
/// Computed (not stored) to break a circular init dependency: it needs `self`, but
|
|
||||||
/// stored properties must be initialized before `self` is available. `Unmanaged.passUn
|
|
||||||
/// retained(self).toOpaque()` always returns the same address for a given instance, so
|
|
||||||
/// computing it on demand is safe and consistent.
|
|
||||||
private var selfPointer: UnsafeMutableRawPointer {
|
private var selfPointer: UnsafeMutableRawPointer {
|
||||||
Unmanaged.passUnretained(self).toOpaque()
|
Unmanaged.passUnretained(self).toOpaque()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Native CString storage backing `vc_config` — must outlive the whole client (the core
|
/// The core retains these pointers for the handle's lifetime.
|
||||||
/// stores raw pointers, doesn't copy). Freed in `deinit` after `vc_client_destroy`.
|
|
||||||
private var clientNamePtr: UnsafeMutablePointer<CChar>?
|
private var clientNamePtr: UnsafeMutablePointer<CChar>?
|
||||||
private var clientVersionPtr: UnsafeMutablePointer<CChar>?
|
private var clientVersionPtr: UnsafeMutablePointer<CChar>?
|
||||||
private var tofuStorePathPtr: UnsafeMutablePointer<CChar>?
|
private var tofuStorePathPtr: UnsafeMutablePointer<CChar>?
|
||||||
@@ -90,7 +48,6 @@ public final class VoiceCatClient {
|
|||||||
/// Intermediate values are coalesced (only the latest per stream_id is delivered).
|
/// Intermediate values are coalesced (only the latest per stream_id is delivered).
|
||||||
public var onLevel: ((UInt32, Float) -> Void)?
|
public var onLevel: ((UInt32, Float) -> Void)?
|
||||||
|
|
||||||
/// Lock-protected buffers, written from the core's event thread, drained on main.
|
|
||||||
private let bufferLock = NSLock()
|
private let bufferLock = NSLock()
|
||||||
private var eventBuffer: [VoiceCatEvent] = []
|
private var eventBuffer: [VoiceCatEvent] = []
|
||||||
private var levelSamples: [UInt32: Float] = [:]
|
private var levelSamples: [UInt32: Float] = [:]
|
||||||
@@ -98,19 +55,12 @@ public final class VoiceCatClient {
|
|||||||
|
|
||||||
// MARK: - Init / deinit
|
// MARK: - Init / deinit
|
||||||
|
|
||||||
/// Create a client. `config.clientName`/`clientVersion`/`tofuStorePath` are copied to
|
|
||||||
/// native CString storage held for the client's entire lifetime (the core reads them
|
|
||||||
/// later, e.g. when `connect()` runs on the io thread).
|
|
||||||
public init(config: VoiceCatConfig) {
|
public init(config: VoiceCatConfig) {
|
||||||
// Allocate native C strings — must persist until after vc_client_destroy in deinit.
|
|
||||||
// These don't need `self`, so they're safe to set first.
|
|
||||||
self.clientNamePtr = strdup(config.clientName)
|
self.clientNamePtr = strdup(config.clientName)
|
||||||
self.clientVersionPtr = strdup(config.clientVersion)
|
self.clientVersionPtr = strdup(config.clientVersion)
|
||||||
self.tofuStorePathPtr = config.tofuStorePath.flatMap { strdup($0) }
|
self.tofuStorePathPtr = config.tofuStorePath.flatMap { strdup($0) }
|
||||||
self.handle = nil // placeholder — set below after callbacks are wired
|
self.handle = nil // placeholder — set below after callbacks are wired
|
||||||
|
|
||||||
// All stored properties are now initialized → `self` is fully available, so we can
|
|
||||||
// call `selfPointer` (the computed property) to build the callbacks struct.
|
|
||||||
var nativeConfig = vc_config()
|
var nativeConfig = vc_config()
|
||||||
nativeConfig.client_name = UnsafePointer(clientNamePtr)
|
nativeConfig.client_name = UnsafePointer(clientNamePtr)
|
||||||
nativeConfig.client_version = UnsafePointer(clientVersionPtr)
|
nativeConfig.client_version = UnsafePointer(clientVersionPtr)
|
||||||
|
|||||||
@@ -26,41 +26,14 @@ final class AppState {
|
|||||||
private(set) var connectingServer: SavedServer?
|
private(set) var connectingServer: SavedServer?
|
||||||
private var identityHandled = false
|
private var identityHandled = false
|
||||||
|
|
||||||
/// The server we are currently fully connected to. Set on auth success (when `session` is
|
/// Retained after authentication so an interrupted session can be restored.
|
||||||
/// created) and cleared on teardown. Used to build the `lastSession` restore snapshot when a
|
|
||||||
/// live-session disconnect fires through `SessionState.handleEvent` — `connectingServer` is
|
|
||||||
/// already nil by then, and `handleConnectEvent`'s `server` parameter is out of scope because
|
|
||||||
/// `SessionState` owns `client.onEvent` after auth success (see `SessionState.init`).
|
|
||||||
private var connectedServer: SavedServer?
|
private var connectedServer: SavedServer?
|
||||||
|
|
||||||
// MARK: - Reconnect state
|
// MARK: - Reconnect state
|
||||||
//
|
|
||||||
// The C core surfaces every unexpected connection drop as a `.disconnected` event; nothing
|
|
||||||
// in the C core auto-reconnects (intentional — reconnect UX is the client's job). Two layers
|
|
||||||
// drive iOS reconnect:
|
|
||||||
//
|
|
||||||
// 1. **Event-driven** (the C core's TCP read eventually fails after the keepalive/reaper
|
|
||||||
// timeout, ~30-60 s on a hard Wi-Fi drop): `SessionState.handleEvent` `.disconnected`
|
|
||||||
// plays the audible cue, then calls back into AppState via `onLiveSessionDisconnected`,
|
|
||||||
// which snapshots the live session, tears it down, and arms `scheduleReconnect`.
|
|
||||||
// (Live-session events never reach `AppState.handleConnectEvent` — `SessionState.init`
|
|
||||||
// overwrites `client.onEvent`, so `AppState` cannot see them without the callback.)
|
|
||||||
//
|
|
||||||
// 2. **Path-driven** (proactive, much faster): `NWPathMonitor` runs the whole time we are
|
|
||||||
// connected (started on auth success) and reacts to network changes — a Wi-Fi↔cellular
|
|
||||||
// flip or the path becoming `.unsatisfied` calls `proactiveReconnect`, which tears the
|
|
||||||
// live session down BEFORE the C core notices the dead TCP path. This is what makes the
|
|
||||||
// 30-60 s wait collapse into ~1 s + the backoff tick. While mid-reconnect (no session)
|
|
||||||
// the same monitor arms a fast-fresh retry whenever a path becomes `.satisfied`.
|
|
||||||
//
|
|
||||||
// Manual Disconnect cancels everything (task + path monitor) and clears `lastSession`.
|
|
||||||
|
|
||||||
/// True at the top of `disconnect()`/`cancelConnect()` — suppresses auto-reconnect for the
|
/// Distinguishes an explicit disconnect from a transport failure.
|
||||||
/// `.disconnected` event the core then emits in response to our `vc_disconnect()` call.
|
|
||||||
private var userInitiatedDisconnect = false
|
private var userInitiatedDisconnect = false
|
||||||
|
|
||||||
/// Snapshot of the live session state needed to restore after a reconnect. Cleared on
|
|
||||||
/// successful restore and on user-initiated disconnect.
|
|
||||||
private struct LastSession {
|
private struct LastSession {
|
||||||
let server: SavedServer
|
let server: SavedServer
|
||||||
let channelId: UInt32
|
let channelId: UInt32
|
||||||
@@ -70,24 +43,14 @@ final class AppState {
|
|||||||
}
|
}
|
||||||
private var lastSession: LastSession?
|
private var lastSession: LastSession?
|
||||||
|
|
||||||
/// Reconnect attempt counter — drives exponential backoff. Reset to 0 on successful auth and
|
|
||||||
/// on a path-driven fast-fresh retry.
|
|
||||||
private var reconnectAttempt = 0
|
private var reconnectAttempt = 0
|
||||||
|
|
||||||
/// The in-flight reconnect `Task` (sleeps for the backoff, then calls `connectTo`). One
|
|
||||||
/// at a time; cancelled on user disconnect / successful restore.
|
|
||||||
private var reconnectTask: Task<Void, Never>?
|
private var reconnectTask: Task<Void, Never>?
|
||||||
|
|
||||||
/// Started on auth success and kept running while connected / mid-reconnect; stopped only on
|
/// Detects interface changes before TCP keepalive notices a dead path.
|
||||||
/// user-initiated disconnect. Its `pathUpdateHandler` (dispatched to @MainActor) handles two
|
|
||||||
/// cases: a path change while connected → proactive reconnect; a satisfied path while
|
|
||||||
/// mid-reconnect → fast-fresh retry. See the reconnect-state header comment.
|
|
||||||
private var pathMonitor: NWPathMonitor?
|
private var pathMonitor: NWPathMonitor?
|
||||||
private let pathQueue = DispatchQueue(label: "cat.voice.network.path")
|
private let pathQueue = DispatchQueue(label: "cat.voice.network.path")
|
||||||
|
|
||||||
/// Signature of the last path seen by the monitor (a stable string encoding status + active
|
|
||||||
/// interface types). The very first path callback (when the monitor starts) sets this and is
|
|
||||||
/// otherwise ignored — it's the baseline; only subsequent CHANGES are reconnect triggers.
|
|
||||||
private var lastPathSignature: String?
|
private var lastPathSignature: String?
|
||||||
|
|
||||||
// MARK: - Server list management
|
// MARK: - Server list management
|
||||||
@@ -118,14 +81,10 @@ final class AppState {
|
|||||||
|
|
||||||
// MARK: - Connect flow
|
// MARK: - Connect flow
|
||||||
|
|
||||||
/// Public connect entry. Always starts a fresh session (no restore).
|
|
||||||
func connectTo(_ server: SavedServer) {
|
func connectTo(_ server: SavedServer) {
|
||||||
connectTo(server, restoring: nil)
|
connectTo(server, restoring: nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Internal connect that drives the full TLS/auth saga. `restoring` is non-nil for a
|
|
||||||
/// reconnect attempt following an unexpected disconnect; the captured channel + voice/mic
|
|
||||||
/// state is handed to the new `SessionState` after auth succeeds.
|
|
||||||
private func connectTo(_ server: SavedServer, restoring: LastSession?) {
|
private func connectTo(_ server: SavedServer, restoring: LastSession?) {
|
||||||
guard !isConnecting else { return }
|
guard !isConnecting else { return }
|
||||||
isConnecting = true
|
isConnecting = true
|
||||||
@@ -134,10 +93,7 @@ final class AppState {
|
|||||||
identityHandled = false
|
identityHandled = false
|
||||||
userInitiatedDisconnect = false
|
userInitiatedDisconnect = false
|
||||||
|
|
||||||
// Discard any leftover connecting client. nil'ing the strong ref calls VoiceCatClient's
|
// Releasing the wrapper joins the core's I/O thread before freeing native strings.
|
||||||
// deinit, which synchronously joins the C core's io thread (vc_client_destroy) before
|
|
||||||
// freeing the config-string storage — safe from @MainActor because the io thread never
|
|
||||||
// blocks on main (it enqueues events via DispatchQueue.main.async and returns).
|
|
||||||
connectingClient = nil
|
connectingClient = nil
|
||||||
|
|
||||||
let config = VoiceCatConfig(
|
let config = VoiceCatConfig(
|
||||||
@@ -153,20 +109,10 @@ final class AppState {
|
|||||||
self?.handleConnectEvent(ev, server: server, restoring: restoring)
|
self?.handleConnectEvent(ev, server: server, restoring: restoring)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// Put the core into external-playback mode BEFORE connect, so the flag is set on the
|
// Authentication can start audio, so select the external path before connecting.
|
||||||
// io thread before any message is processed. The server sends AuthResult immediately
|
|
||||||
// followed by ServerStateSnapshot; handle_server_state runs ensure_audio_running() on
|
|
||||||
// the io thread, and if external_playback_ were still false at that point the core
|
|
||||||
// would open a hardware miniaudio playback+capture device (see the matching fix in
|
|
||||||
// vc_client::ensure_audio_running). Setting it here — before connect — guarantees the
|
|
||||||
// unified external path is in effect from the first frame. setExternalPlayback only
|
|
||||||
// flips an atomic + forwards to the engine's setter; both are safe pre-connect.
|
|
||||||
client.setExternalPlayback(true)
|
client.setExternalPlayback(true)
|
||||||
client.connect(host: server.host, port: server.port)
|
client.connect(host: server.host, port: server.port)
|
||||||
|
|
||||||
// Auth is queued immediately — the core serialises it behind TLS + TOFU. On a reconnect
|
|
||||||
// the TOFU pin already matches (VC_TOFU_MATCHED), so the identity gate auto-confirms
|
|
||||||
// inside the .serverIdentity case below and auth proceeds unattended.
|
|
||||||
switch server.authMode {
|
switch server.authMode {
|
||||||
case .guest:
|
case .guest:
|
||||||
let nick = (server.nickname?.isEmpty == false) ? server.nickname! : "iOS User"
|
let nick = (server.nickname?.isEmpty == false) ? server.nickname! : "iOS User"
|
||||||
@@ -182,8 +128,7 @@ final class AppState {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func disconnect() {
|
func disconnect() {
|
||||||
// Mark BEFORE we ask the core to disconnect, so the .disconnected event the core emits
|
// Set before disconnect so its event cannot arm reconnect.
|
||||||
// in response is treated as user-initiated (no reconnect) rather than an unexpected drop.
|
|
||||||
userInitiatedDisconnect = true
|
userInitiatedDisconnect = true
|
||||||
cancelReconnect()
|
cancelReconnect()
|
||||||
lastSession = nil
|
lastSession = nil
|
||||||
@@ -232,31 +177,13 @@ final class AppState {
|
|||||||
|
|
||||||
// MARK: - Reconnect orchestration
|
// MARK: - Reconnect orchestration
|
||||||
|
|
||||||
/// Cancel any in-flight reconnect task and stop the path monitor. Safe to call when nothing
|
|
||||||
/// is armed (no-op). Does NOT touch `userInitiatedDisconnect` or `lastSession` — callers set
|
|
||||||
/// those as needed (disconnect/cancelConnect clear them; scheduleReconnect keeps them).
|
|
||||||
///
|
|
||||||
/// `AppState` is the @Observable app root owned by the SwiftUI `App`; it lives for the
|
|
||||||
/// whole app process and is torn down only on process exit, at which point OS cleanup
|
|
||||||
/// suffices. This method is driven by `disconnect()`/`cancelConnect()` and on successful
|
|
||||||
/// restore — those run on user-initiated teardown, which is the only path that matters.
|
|
||||||
/// (The reconnect `Task` captures `[weak self]` and guards on `nil`/`userInitiatedDisconnect`,
|
|
||||||
/// so a stray task left running when AppState is gone is a no-op; the monitor similarly guards.)
|
|
||||||
private func cancelReconnect() {
|
private func cancelReconnect() {
|
||||||
reconnectTask?.cancel()
|
reconnectTask?.cancel()
|
||||||
reconnectTask = nil
|
reconnectTask = nil
|
||||||
stopPathMonitor()
|
stopPathMonitor()
|
||||||
// Don't reset `reconnectAttempt` here: scheduleReconnect resets it on successful auth,
|
|
||||||
// and the path monitor resets it to 0 for a fast fresh attempt on a path-satisfied event.
|
|
||||||
// If a fresh user connect follows, connectTo() doesn't reset it either, but it doesn't
|
|
||||||
// need to — `reconnectAttempt` only matters while we're mid-reconnect.
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Arm the next reconnect attempt with exponential backoff (1s → 2s → 4s → 8s → 16s → 30s
|
/// Schedules the next reconnect with exponential backoff capped at 30 seconds.
|
||||||
/// cap). Cancelled cleanly by `cancelReconnect()` on user disconnect or successful auth.
|
|
||||||
/// Idempotent: a new call supersedes any in-flight one. The path monitor is armed here and
|
|
||||||
/// disarmed on cancel; on a path-satisfied event it resets the attempt counter to 0 and
|
|
||||||
/// re-arms via this same method, yielding a fast refresh after Wi-Fi ↔ cellular transitions.
|
|
||||||
private func scheduleReconnect() {
|
private func scheduleReconnect() {
|
||||||
guard !userInitiatedDisconnect, let last = lastSession else { return }
|
guard !userInitiatedDisconnect, let last = lastSession else { return }
|
||||||
reconnectTask?.cancel()
|
reconnectTask?.cancel()
|
||||||
@@ -270,7 +197,6 @@ final class AppState {
|
|||||||
guard let self else { return }
|
guard let self else { return }
|
||||||
try? await Task.sleep(nanoseconds: UInt64(delaySec * 1_000_000_000))
|
try? await Task.sleep(nanoseconds: UInt64(delaySec * 1_000_000_000))
|
||||||
if Task.isCancelled { return }
|
if Task.isCancelled { return }
|
||||||
// Re-check under Task: a user disconnect between the sleep and this line must abort.
|
|
||||||
guard !self.userInitiatedDisconnect else { return }
|
guard !self.userInitiatedDisconnect else { return }
|
||||||
guard self.lastSession != nil else { return }
|
guard self.lastSession != nil else { return }
|
||||||
guard self.session == nil else { return }
|
guard self.session == nil else { return }
|
||||||
@@ -279,20 +205,6 @@ final class AppState {
|
|||||||
reconnectTask = task
|
reconnectTask = task
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Start (if not already running) the network path monitor. Runs the whole time we are
|
|
||||||
/// connected (started on auth success) and stays armed across reconnects; stopped only on
|
|
||||||
/// user-initiated disconnect. The handler dispatches to @MainActor before touching state and
|
|
||||||
/// does two distinct things:
|
|
||||||
/// - **While connected** (`session != nil`): a Wi-Fi↔cellular interface change OR the path
|
|
||||||
/// becoming `.unsatisfied` triggers `proactiveReconnect()` — tearing the live session
|
|
||||||
/// down before the C core notices the dead TCP read. Without this the disconnect would
|
|
||||||
/// take 30-60 s (the TCP keepalive/reaper timeout); proactive teardown collapses that to
|
|
||||||
/// ~1 s + the first backoff tick. Same-interface path refreshes (e.g. a Wi-Fi roam
|
|
||||||
/// without an IP change) are ignored — likely the connection is still good.
|
|
||||||
/// - **While mid-reconnect** (`session == nil`, `lastSession != nil`): a path becoming
|
|
||||||
/// `.satisfied` arms a fast-fresh retry (backoff counter reset, `scheduleReconnect`).
|
|
||||||
/// This is what makes a Wi-Fi→cellular flip reconnect on roughly the next tick instead
|
|
||||||
/// of waiting out a long backoff.
|
|
||||||
private func startPathMonitor() {
|
private func startPathMonitor() {
|
||||||
guard pathMonitor == nil else { return }
|
guard pathMonitor == nil else { return }
|
||||||
let monitor = NWPathMonitor()
|
let monitor = NWPathMonitor()
|
||||||
@@ -303,12 +215,9 @@ final class AppState {
|
|||||||
let sig = Self.pathSignature(path)
|
let sig = Self.pathSignature(path)
|
||||||
let prevSig = self.lastPathSignature
|
let prevSig = self.lastPathSignature
|
||||||
self.lastPathSignature = sig
|
self.lastPathSignature = sig
|
||||||
// The first callback (when the monitor starts) is the baseline, not a change.
|
|
||||||
if prevSig == nil { return }
|
if prevSig == nil { return }
|
||||||
|
|
||||||
if self.session != nil {
|
if self.session != nil {
|
||||||
// See this method's doc comment for why these conditions trigger a
|
|
||||||
// proactive reconnect.
|
|
||||||
if path.status != .satisfied || sig != prevSig {
|
if path.status != .satisfied || sig != prevSig {
|
||||||
self.proactiveReconnect()
|
self.proactiveReconnect()
|
||||||
}
|
}
|
||||||
@@ -320,8 +229,6 @@ final class AppState {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// Listen on a dedicated queue — the path monitor can't share the main queue (it would
|
|
||||||
// re-enter main if any handler dispatched to main synchronously).
|
|
||||||
monitor.start(queue: pathQueue)
|
monitor.start(queue: pathQueue)
|
||||||
pathMonitor = monitor
|
pathMonitor = monitor
|
||||||
}
|
}
|
||||||
@@ -332,11 +239,6 @@ final class AppState {
|
|||||||
lastPathSignature = nil
|
lastPathSignature = nil
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A stable string signature of a network path: the path status plus the set of interface
|
|
||||||
/// types it uses. Two paths with the same signature are treated as equivalent — no
|
|
||||||
/// reconnect. A signature change is the trigger for `proactiveReconnect`. Used to ignore
|
|
||||||
/// same-interface refreshes (signal-strength changes, BSSID roams) which usually don't break
|
|
||||||
/// the TCP connection.
|
|
||||||
private static func pathSignature(_ path: NWPath) -> String {
|
private static func pathSignature(_ path: NWPath) -> String {
|
||||||
guard path.status == .satisfied else { return "unsatisfied" }
|
guard path.status == .satisfied else { return "unsatisfied" }
|
||||||
var parts: [String] = []
|
var parts: [String] = []
|
||||||
@@ -349,42 +251,19 @@ final class AppState {
|
|||||||
|
|
||||||
// MARK: - Live-session disconnect (called by SessionState)
|
// MARK: - Live-session disconnect (called by SessionState)
|
||||||
|
|
||||||
/// Called by `SessionState.handleEvent` `.disconnected` after the audible cue has already
|
/// Receives disconnects after `SessionState` takes ownership of authenticated events.
|
||||||
/// played. Once `SessionState` is created (auth success) it owns `client.onEvent`, so
|
|
||||||
/// `AppState.handleConnectEvent` never sees live-session events — this callback is the only
|
|
||||||
/// way AppState learns that a live session dropped. Snapshots the live session state,
|
|
||||||
/// tears the session down, and arms `scheduleReconnect` so the backoff loop drives a fresh
|
|
||||||
/// TLS/auth/restoration. Guarded against user-initiated disconnect (which nil's `session`
|
|
||||||
/// synchronously, so SessionState is gone before the event could fire this callback) — but
|
|
||||||
/// the guard is cheap insurance.
|
|
||||||
func onLiveSessionDisconnected() {
|
func onLiveSessionDisconnected() {
|
||||||
guard !userInitiatedDisconnect else { return }
|
guard !userInitiatedDisconnect else { return }
|
||||||
teardownLiveSessionAndReconnect(sound: false)
|
teardownLiveSessionAndReconnect(sound: false)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Called by `NWPathMonitor` when a path change is detected while a live session exists.
|
|
||||||
/// Tears the session down immediately — nil'ing `session` releases `VoiceCatClient`, whose
|
|
||||||
/// `deinit` calls `vc_client_destroy`; that closes the socket and joins the C core's io
|
|
||||||
/// thread, so the io thread exits in milliseconds rather than blocking on a dead read for
|
|
||||||
/// ~30-60 s. The proactive tear-down is what collapses the long TCP-reaper wait into a
|
|
||||||
/// ~1 s reconnect. Plays the audible cue (no `.disconnected` event fires through to
|
|
||||||
/// `SessionState` for this path, since `SessionState` is being torn down here — so the cue
|
|
||||||
/// would otherwise be missing).
|
|
||||||
private func proactiveReconnect() {
|
private func proactiveReconnect() {
|
||||||
guard !userInitiatedDisconnect else { return }
|
guard !userInitiatedDisconnect else { return }
|
||||||
guard session != nil else { return }
|
guard session != nil else { return }
|
||||||
teardownLiveSessionAndReconnect(sound: true)
|
teardownLiveSessionAndReconnect(sound: true)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Shared teardown for a live-session disconnect (event- or path-driven). Snapshots the live
|
|
||||||
/// session into `lastSession`, stops audio, deactivates the AVAudioSession, releases the
|
|
||||||
/// session (which releases `VoiceCatClient` → io-thread join), resets the backoff counter,
|
|
||||||
/// and arms `scheduleReconnect`. `sound` is true for the proactive (path-driven) case — the
|
|
||||||
/// `.disconnected` event that would have played it never fires because we're tearing down
|
|
||||||
/// ahead of the C core noticing. The event-driven caller (`onLiveSessionDisconnected`) has
|
|
||||||
/// ALREADY played the cue via `SessionState.handleEvent`, so it passes `sound: false`.
|
|
||||||
private func teardownLiveSessionAndReconnect(sound: Bool) {
|
private func teardownLiveSessionAndReconnect(sound: Bool) {
|
||||||
// Snapshot BEFORE nil'ing `session` — we need the channel + voice/mic state to restore.
|
|
||||||
if let s = session, let srv = connectedServer {
|
if let s = session, let srv = connectedServer {
|
||||||
lastSession = LastSession(
|
lastSession = LastSession(
|
||||||
server: srv,
|
server: srv,
|
||||||
@@ -395,8 +274,6 @@ final class AppState {
|
|||||||
}
|
}
|
||||||
IOSAudioEngine.shared.stop()
|
IOSAudioEngine.shared.stop()
|
||||||
AudioSessionManager.shared.deactivateSession()
|
AudioSessionManager.shared.deactivateSession()
|
||||||
// Releasing `session` releases `VoiceCatClient`; its deinit joins the C core's io thread.
|
|
||||||
// For a path-driven proactive teardown this is what avoids the 30-60 s reaper timeout.
|
|
||||||
session = nil
|
session = nil
|
||||||
isConnecting = false
|
isConnecting = false
|
||||||
connectingClient = nil
|
connectingClient = nil
|
||||||
@@ -405,9 +282,6 @@ final class AppState {
|
|||||||
EventFeedback.shared.play(.connectionLost)
|
EventFeedback.shared.play(.connectionLost)
|
||||||
EventFeedback.shared.speak("Network changed — reconnecting")
|
EventFeedback.shared.speak("Network changed — reconnecting")
|
||||||
}
|
}
|
||||||
// Reset the backoff counter so the first reconnect attempt after a drop uses the short
|
|
||||||
// 1 s delay (the immediate path-driven attempt matters most; sustained-outage backoff is
|
|
||||||
// driven by `scheduleReconnect`'s increment).
|
|
||||||
reconnectAttempt = 0
|
reconnectAttempt = 0
|
||||||
scheduleReconnect()
|
scheduleReconnect()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,14 +35,7 @@ final class AudioSessionManager {
|
|||||||
name: AVAudioSession.routeChangeNotification, object: nil)
|
name: AVAudioSession.routeChangeNotification, object: nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The single end-to-end audio recovery path, driven by *intent* (`IOSAudioEngine.isConnected`)
|
/// Idempotently restores audio after an interruption or external route change.
|
||||||
/// — not by session bookkeeping flags that can drift out of sync (e.g. an interruption ended
|
|
||||||
/// without `.shouldResume`, which used to leave `isSessionActive` false forever). Safe to call
|
|
||||||
/// speculatively: the underlying calls are idempotent (AVAudioSession.setActive(true),
|
|
||||||
/// `IOSAudioRouter.applyConfiguration` has a re-entrancy guard, `IOSAudioEngine.reconfigure`
|
|
||||||
/// no-ops when not connected). Call this whenever the audio environment changes in a way that
|
|
||||||
/// could have stopped the engine — interruption end, route change, AVAudioEngine
|
|
||||||
/// configuration-change — and we still want audio back.
|
|
||||||
func recoverAudio() {
|
func recoverAudio() {
|
||||||
guard IOSAudioEngine.shared.isConnected else { return }
|
guard IOSAudioEngine.shared.isConnected else { return }
|
||||||
do {
|
do {
|
||||||
@@ -158,27 +151,9 @@ final class AudioSessionManager {
|
|||||||
IOSAudioRouter.shared.refreshRoutes()
|
IOSAudioRouter.shared.refreshRoutes()
|
||||||
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
|
NotificationCenter.default.post(name: .voiceCatDeviceListChanged, object: nil)
|
||||||
|
|
||||||
// Recover audio on every externally-initiated route change. `.categoryChange`,
|
// Ignore notifications caused by our own configuration calls; rebuilding for them
|
||||||
// `.routeConfigurationChange`, and `.override` are fired by our OWN calls:
|
// recursively emits more route changes. Engine-configuration notifications remain
|
||||||
// - `.categoryChange` / `.routeConfigurationChange` ← applyConfiguration()'s
|
// the recovery path if a self-initiated change actually stops AVAudioEngine.
|
||||||
// setCategory / setPreferredInput / ...
|
|
||||||
// - `.override` ← applyA2dpSpeakerFallback()'s overrideOutputAudioPort(.speaker),
|
|
||||||
// which fires on every AirPods disconnect (and reconnect) on an A2DP preset.
|
|
||||||
// Acting on any of these would create a tight ping-pong loop with the re-entrancy
|
|
||||||
// guard (handleRouteChange → recoverAudio → applyA2dpSpeakerFallback →
|
|
||||||
// overrideOutputAudioPort → .override routeChange → recoverAudio → ...). The
|
|
||||||
// `.override` skip is what fixes the AirPods-disconnect reinitialize loop: each
|
|
||||||
// iteration also calls IOSAudioEngine.reconfigure() → rebuild() (a full
|
|
||||||
// stop/restart of AVAudioEngine), which is the audible cycling. IOSAudioRouter's
|
|
||||||
// guard is the backstop that bounds it to ONE extra iteration, but skipping these
|
|
||||||
// three reasons avoids even that, so we reconfigure only in response to genuine
|
|
||||||
// environmental changes.
|
|
||||||
//
|
|
||||||
// The recovery set below (oldDeviceUnavailable, newDeviceAvailable, wakeFromSleep,
|
|
||||||
// noSuitableRouteForCategory, unknown) covers headphone/AirPods/wired unplug-replug
|
|
||||||
// — the previously-reported "audio dies when headphones disconnect" bug. If an
|
|
||||||
// override ever actually stops the AVAudioEngine, the
|
|
||||||
// AVAudioEngineConfigurationChange handler in IOSVoiceProcessingEngine catches it.
|
|
||||||
if reason != .categoryChange && reason != .routeConfigurationChange && reason != .override {
|
if reason != .categoryChange && reason != .routeConfigurationChange && reason != .override {
|
||||||
recoverAudio()
|
recoverAudio()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,48 +4,8 @@ import VoiceCatCore
|
|||||||
|
|
||||||
private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAudioRouter")
|
private let logger = Logger(subsystem: "cat.voice.VoiceCatiOS", category: "IOSAudioRouter")
|
||||||
|
|
||||||
/// iOS audio routing layer — the sole owner of `AVAudioSession` on iOS. On iOS the core never
|
/// Owns `AVAudioSession` routing for the iOS external-audio path.
|
||||||
/// opens a hardware (miniaudio) device: a single `AVAudioEngine` (`IOSAudioEngine`) drives both
|
/// Route configuration and ordering constraints are documented in `docs/voice.md`.
|
||||||
/// capture and playback and the core runs fully external (see docs/voice.md §8). This class just
|
|
||||||
/// configures the *route* — category / mode / options, preferred input, data source, polar
|
|
||||||
/// pattern, stereo capsule — and `IOSAudioEngine` binds to whatever route is established. After
|
|
||||||
/// any change here the engine is rebuilt via `IOSAudioEngine.reconfigure()` (a deterministic
|
|
||||||
/// Swift-only stop → reconfigure → start); there is no second (miniaudio) audio path to hand off
|
|
||||||
/// to, so a change cannot leave one direction dropped.
|
|
||||||
///
|
|
||||||
/// (The core's iOS `ma_context` is still configured with `sessionCategory = none` +
|
|
||||||
/// `noAudioSessionActivate/Deactivate` in `AudioEngine::make_context_config` so that, should the
|
|
||||||
/// core ever open a device, miniaudio would not reset the category — but on iOS it does not.)
|
|
||||||
///
|
|
||||||
/// The three user-facing choices:
|
|
||||||
/// 1. **Input port** — which physical input (built-in mic, Bluetooth HFP, headset,
|
|
||||||
/// USB, AirPlay). For the built-in mic, a sub-selection of **data source**
|
|
||||||
/// (orientation: front/back/top/bottom) and **polar pattern**
|
|
||||||
/// (omni/cardioid/subcardioid/bidirectional).
|
|
||||||
/// 2. **Bluetooth mode** — how Bluetooth headsets are handled:
|
|
||||||
/// - "BT HFP voice" (`.allowBluetoothHFP` + `.allowBluetoothA2DP`): both profiles
|
|
||||||
/// allowed, iOS picks HFP for two-way mic or A2DP for output-only. Mono, AEC on.
|
|
||||||
/// - "Built-in Mic + BT A2DP stereo" (`.allowBluetoothA2DP` only): stereo output,
|
|
||||||
/// built-in mic, no HFP processing.
|
|
||||||
/// - "Built-in Mic + Speaker" (neither): no Bluetooth at all.
|
|
||||||
/// 3. **Mic processing mode** — Standard (`.voiceChat`: AEC/AGC/HPF on) or
|
|
||||||
/// Raw/Studio (`.measurement`: all processing off). Raw mode is allowed always
|
|
||||||
/// but shows a warning when the output route is the speaker (echo risk, no AEC).
|
|
||||||
///
|
|
||||||
/// Additionally, **stereo capture** (2-channel built-in mic) is enabled by switching the
|
|
||||||
/// built-in mic's data source to the `.stereo` polar pattern. The recipe is:
|
|
||||||
/// `setPreferredDataSource(.stereo source)` + `setPreferredPolarPattern(.stereo)` +
|
|
||||||
/// `setPreferredInput(built-in mic)` + `setInputDataSource(stereo source)`. The channel
|
|
||||||
/// count itself must NOT be requested via `setPreferredInputNumberOfChannels(2)` — that
|
|
||||||
/// session-level call collapses the A2DP output route. Instead the core is told to open the
|
|
||||||
/// device with 2 channels via `vc_set_capture_channels(streamId, 2)`, and the AVAudioSession
|
|
||||||
/// input anchor (`setPreferredInput` + `setInputDataSource`) keeps the route stable during
|
|
||||||
/// the HFP→A2DP and mono→stereo reconfigurations.
|
|
||||||
///
|
|
||||||
/// Voice Isolation / Wide Spectrum (iOS 17+/18+) are user-toggleable in Control Center
|
|
||||||
/// for `.voiceChat` apps — surfaced as a hint, not a programmatic toggle.
|
|
||||||
///
|
|
||||||
/// All choices are persisted in `UserDefaults` and re-applied on route changes.
|
|
||||||
@MainActor
|
@MainActor
|
||||||
final class IOSAudioRouter: ObservableObject {
|
final class IOSAudioRouter: ObservableObject {
|
||||||
|
|
||||||
@@ -156,17 +116,10 @@ final class IOSAudioRouter: ObservableObject {
|
|||||||
private let kVoiceProcessing = "cat.voice.audio.voiceProcessing"
|
private let kVoiceProcessing = "cat.voice.audio.voiceProcessing"
|
||||||
private let kAgc = "cat.voice.audio.agc"
|
private let kAgc = "cat.voice.audio.agc"
|
||||||
|
|
||||||
/// Re-entrancy guard: setCategory/setPreferredInput/etc. trigger route-change
|
/// AVAudioSession setters can synchronously emit route-change notifications.
|
||||||
/// notifications synchronously on the same thread. Without this guard,
|
|
||||||
/// handleRouteChange → applyConfiguration → setCategory → route-change notification
|
|
||||||
/// → handleRouteChange → applyConfiguration → ... creates an infinite loop that
|
|
||||||
/// burns CPU and cycles the audio session on/off (the "glitching" bug).
|
|
||||||
private var isApplyingConfiguration = false
|
private var isApplyingConfiguration = false
|
||||||
|
|
||||||
/// Last `overrideOutputAudioPort` value we successfully applied (`.none` or `.speaker`).
|
/// Prevents redundant overrides; `setCategory` invalidates the cached value.
|
||||||
/// See `applyA2dpSpeakerFallback`'s doc comment for why this cache exists.
|
|
||||||
/// `nil` = "unknown / assume not applied" — reset at the top of `applyConfiguration()`
|
|
||||||
/// because `setCategory` can reset the override out from under us, and on first run.
|
|
||||||
private var lastAppliedOutputOverride: AVAudioSession.PortOverride?
|
private var lastAppliedOutputOverride: AVAudioSession.PortOverride?
|
||||||
|
|
||||||
private init() {}
|
private init() {}
|
||||||
@@ -399,16 +352,8 @@ final class IOSAudioRouter: ObservableObject {
|
|||||||
updateWarnings()
|
updateWarnings()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Enable 2-channel capture on the built-in mic. The recipe that achieves stereo mic +
|
/// Anchors the built-in stereo data source without using
|
||||||
/// A2DP Bluetooth output simultaneously:
|
/// `setPreferredInputNumberOfChannels`, which disrupts A2DP routing.
|
||||||
/// 1. `setPreferredDataSource(stereoSource)` on the built-in mic port
|
|
||||||
/// 2. `setPreferredPolarPattern(.stereo)` on that data source
|
|
||||||
/// 3. `setPreferredInput(builtIn)` — anchor the input route explicitly. Without this
|
|
||||||
/// anchor the route can collapse during the mode switch (.voiceChat → .default).
|
|
||||||
/// 4. `setInputDataSource(stereoSource)` — commit the data source at the session level
|
|
||||||
/// The channel count itself is carried by the engine's mic tap (which captures 2 channels)
|
|
||||||
/// plus `vc_set_capture_channels(2)` so the core encodes stereo. We must NOT call
|
|
||||||
/// `setPreferredInputNumberOfChannels(2)` — that session-level call collapses the A2DP route.
|
|
||||||
private func configureStereoCapture(session: AVAudioSession) {
|
private func configureStereoCapture(session: AVAudioSession) {
|
||||||
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
|
guard let builtIn = session.availableInputs?.first(where: { $0.portType == .builtInMic })
|
||||||
else {
|
else {
|
||||||
@@ -530,11 +475,7 @@ final class IOSAudioRouter: ObservableObject {
|
|||||||
|
|
||||||
// MARK: - Selection setters (called from SettingsView pickers)
|
// MARK: - Selection setters (called from SettingsView pickers)
|
||||||
|
|
||||||
/// Shared tail for every setting change: persist, re-apply the AVAudioSession config, refresh
|
/// Persists the selection and rebuilds the engine against the resulting route.
|
||||||
/// the route lists, re-evaluate the A2DP speaker fallback, and rebind the live engine to the
|
|
||||||
/// new route. `IOSAudioEngine.reconfigure()` is a no-op when not connected, so this is safe to
|
|
||||||
/// call from Settings whether or not a session is in progress. There is no longer a second
|
|
||||||
/// (miniaudio) audio path to hand off to, so one engine rebuild is the whole story.
|
|
||||||
private func applyAndReconfigure() {
|
private func applyAndReconfigure() {
|
||||||
savePreferences()
|
savePreferences()
|
||||||
applyConfiguration()
|
applyConfiguration()
|
||||||
@@ -658,23 +599,8 @@ final class IOSAudioRouter: ObservableObject {
|
|||||||
showsA2dpNoAecWarning = (bluetoothMode == .builtInMicBtA2dp)
|
showsA2dpNoAecWarning = (bluetoothMode == .builtInMicBtA2dp)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Route fallback for the A2DP-output presets (Stereo Mic / Studio / BT Headphones + Mono
|
/// Uses the speaker only when an A2DP-capable preset has no external output.
|
||||||
/// Mic, all `.builtInMicBtA2dp`). These presets deliberately omit `.defaultToSpeaker` (it
|
/// The cached override avoids recursively generated route-change notifications.
|
||||||
/// breaks A2DP routing) and skip the `forceSpeaker` override, so when NO external output
|
|
||||||
/// (Bluetooth A2DP / wired / AirPlay) is connected `.playAndRecord` pins output to the quiet
|
|
||||||
/// built-in receiver (earpiece). This routes to the loud built-in speaker instead via a
|
|
||||||
/// post-activation `overrideOutputAudioPort(.speaker)` — the documented "A2DP if connected,
|
|
||||||
/// else speaker" behavior. When an external output IS present we clear the override so A2DP /
|
|
||||||
/// headphones / AirPlay are honored. No-op outside `.builtInMicBtA2dp` mode (other modes pick
|
|
||||||
/// their route via category options). Must be called AFTER the session is active.
|
|
||||||
///
|
|
||||||
/// Idempotent: skips the `overrideOutputAudioPort` call when the desired override already
|
|
||||||
/// matches the last one we successfully applied. Each call fires a `.override` route-change
|
|
||||||
/// notification, and `AudioSessionManager.recoverAudio()` invokes this on every recovery —
|
|
||||||
/// so on an AirPods disconnect, without this guard, override + recoverAudio ping-pong and
|
|
||||||
/// each iteration also rebuilds the AVAudioEngine (the audible reinitialize loop). The cache
|
|
||||||
/// is reset to `nil` at the top of `applyConfiguration()` (setCategory can reset the
|
|
||||||
/// override) and on a failed call (so the next attempt re-derives from the live session).
|
|
||||||
func applyA2dpSpeakerFallback() {
|
func applyA2dpSpeakerFallback() {
|
||||||
guard bluetoothMode == .builtInMicBtA2dp else { return }
|
guard bluetoothMode == .builtInMicBtA2dp else { return }
|
||||||
let session = AVAudioSession.sharedInstance()
|
let session = AVAudioSession.sharedInstance()
|
||||||
|
|||||||
@@ -131,26 +131,9 @@ final class IOSAudioEngine {
|
|||||||
private var micStreamId: UInt32 = 0
|
private var micStreamId: UInt32 = 0
|
||||||
private var captureChannels: UInt32 = 1
|
private var captureChannels: UInt32 = 1
|
||||||
|
|
||||||
// Mic feed pacing. The core sends each captured frame SYNCHRONOUSLY as it arrives
|
// AVAudioEngine may deliver several codec frames per callback. Pace complete 20 ms frames
|
||||||
// (on_capture_frame → encode → sendto, client.cpp) — there is no send pacer in the core. On
|
// through an SPSC ring; never consume partial frames, and recreate the timer when the channel
|
||||||
// desktop miniaudio capture fires one 960-sample frame every 20 ms, so packets leave at a
|
// count changes.
|
||||||
// steady 20 ms. On iOS the AVAudioEngine input tap fires at the hardware IO-buffer period
|
|
||||||
// (often ~40 ms under VPIO), delivering ~2 frames at once: feeding those straight to the core
|
|
||||||
// bursts 2 packets out then goes quiet for ~40 ms, and the receiver's ~40 ms jitter buffer
|
|
||||||
// underruns on every gap → PLC fade ("talking through a slow fan" + ~40–60 ms flutter).
|
|
||||||
//
|
|
||||||
// Fix: pace the feed to a steady 20 ms. The tap converts to int16 and writes to a lock-free
|
|
||||||
// SPSC ring (producer, audio clock); a 20 ms timer releases ONE 960-sample frame per tick to
|
|
||||||
// feedPcm (consumer). The producer's average rate is locked to 48 kHz = exactly one frame per
|
|
||||||
// 20 ms, so it matches the consumer; the ring just absorbs the tap's 2-at-a-time bursts.
|
|
||||||
//
|
|
||||||
// Two correctness rules learned the hard way (these caused the earlier crackle + octave):
|
|
||||||
// 1. NEVER read a partial frame — `read` consumes whatever it returns, so reading <960 would
|
|
||||||
// silently discard those samples (crackle). The timer checks `availableSamples` first and
|
|
||||||
// only reads when a full frame is present; an underrun just skips the tick (nothing lost).
|
|
||||||
// 2. NEVER freeze the channel count in the timer — mono↔stereo preset switches change it. The
|
|
||||||
// timer is torn down and recreated inside `rebuild()`, so it always captures the current
|
|
||||||
// `captureChannels`; the ring is reset while the timer is stopped (no cross-thread race).
|
|
||||||
private let micRing = PCMRing(capacitySamples: 48000 * 2) // ~1 s stereo — ample elastic slack
|
private let micRing = PCMRing(capacitySamples: 48000 * 2) // ~1 s stereo — ample elastic slack
|
||||||
private var micTimer: DispatchSourceTimer?
|
private var micTimer: DispatchSourceTimer?
|
||||||
private let micQueue = DispatchQueue(label: "cat.voice.mic.feedPump")
|
private let micQueue = DispatchQueue(label: "cat.voice.mic.feedPump")
|
||||||
@@ -314,8 +297,7 @@ final class IOSAudioEngine {
|
|||||||
engine.inputNode.isVoiceProcessingAGCEnabled = IOSAudioRouter.shared.agcEnabled
|
engine.inputNode.isVoiceProcessingAGCEnabled = IOSAudioRouter.shared.agcEnabled
|
||||||
}
|
}
|
||||||
|
|
||||||
// (Re)build the playback source node AFTER the VPIO state is set, so it connects against the
|
// The source node must bind to the selected voice-processing output unit.
|
||||||
// correct (voice-processed or plain) output unit — mirrors the proven original ordering.
|
|
||||||
rebuildSourceNode()
|
rebuildSourceNode()
|
||||||
if micActive { installMicTap() }
|
if micActive { installMicTap() }
|
||||||
|
|
||||||
@@ -331,11 +313,7 @@ final class IOSAudioEngine {
|
|||||||
inFormat=\(inFmt) outputNode=\(outFmt) outputRoute=[\(route)]
|
inFormat=\(inFmt) outputNode=\(outFmt) outputRoute=[\(route)]
|
||||||
""")
|
""")
|
||||||
} catch {
|
} catch {
|
||||||
// iOS occasionally refuses to start the engine immediately after a route change —
|
// Route changes can leave AVAudioSession inactive; retry once after reactivation.
|
||||||
// the AVAudioSession needs a re-activation nudge before the engine will start. Do
|
|
||||||
// ONE recovery attempt: re-activate the session, re-apply the route config, then
|
|
||||||
// try `engine.start()` again. Recovering here is what fixes the silent-death bug
|
|
||||||
// where unplugging headphones left the engine stopped forever.
|
|
||||||
logger.error("engine start failed: \(error.localizedDescription) — attempting one-shot recovery")
|
logger.error("engine start failed: \(error.localizedDescription) — attempting one-shot recovery")
|
||||||
do {
|
do {
|
||||||
try AudioSessionManager.shared.ensureSessionActive()
|
try AudioSessionManager.shared.ensureSessionActive()
|
||||||
@@ -476,9 +454,7 @@ final class IOSAudioEngine {
|
|||||||
if frames < state.targetFrames { return } // still filling the cushion (into silence)
|
if frames < state.targetFrames { return } // still filling the cushion (into silence)
|
||||||
state.primed = true
|
state.primed = true
|
||||||
} else if frames == 0 {
|
} else if frames == 0 {
|
||||||
// Underrun: the cushion drained. Grow it (capped) so it won't recur, then re-prime.
|
// Re-prime with a larger cushion; consuming a partial frame would lose samples.
|
||||||
// Never read a partial frame — `read` consumes what it returns, so that would
|
|
||||||
// discard samples (the old crackle bug); skipping loses nothing, the samples wait.
|
|
||||||
if state.targetFrames < PumpState.maxTargetFrames { state.targetFrames += 1 }
|
if state.targetFrames < PumpState.maxTargetFrames { state.targetFrames += 1 }
|
||||||
state.primed = false
|
state.primed = false
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -1,14 +1,11 @@
|
|||||||
import AVFoundation
|
import AVFoundation
|
||||||
import ScreenCaptureKit
|
import ScreenCaptureKit
|
||||||
|
|
||||||
// Which apps' audio the SCREEN_AUDIO stream captures. ScreenCaptureKit filters audio at the
|
// ScreenCaptureKit filters audio by application bundle identifier.
|
||||||
// *application* level (not per-window), so the selection is expressed as bundle IDs. The
|
|
||||||
// picker UI (ScreenSharePickerSheet) produces a `ScreenAudioSelection`; `start()` turns it
|
|
||||||
// into the matching `SCContentFilter`.
|
|
||||||
enum ScreenAudioScope: Equatable {
|
enum ScreenAudioScope: Equatable {
|
||||||
case entireDesktop // whole display — the original behaviour
|
case entireDesktop
|
||||||
case onlyApps([String]) // capture only these bundle IDs
|
case onlyApps([String])
|
||||||
case allExcept([String]) // capture everything except these bundle IDs
|
case allExcept([String])
|
||||||
}
|
}
|
||||||
|
|
||||||
struct ScreenAudioSelection: Equatable {
|
struct ScreenAudioSelection: Equatable {
|
||||||
@@ -20,20 +17,8 @@ struct ScreenAudioSelection: Equatable {
|
|||||||
static let `default` = ScreenAudioSelection()
|
static let `default` = ScreenAudioSelection()
|
||||||
}
|
}
|
||||||
|
|
||||||
// ScreenAudioCapture — macOS system/desktop audio capture for the SCREEN_AUDIO stream.
|
// SCStream requires a minimal video configuration even for audio-only capture. Only its audio
|
||||||
//
|
// output is registered, and current-process audio is excluded to prevent feedback.
|
||||||
// The macOS analog of the Windows WASAPI loopback path (docs/voice.md §9). ScreenCaptureKit
|
|
||||||
// (macOS 13+) captures whatever the system is playing; we convert each audio CMSampleBuffer
|
|
||||||
// (Float32) → int16 interleaved and push 20 ms frames (960 samples/channel @ 48 kHz) into the
|
|
||||||
// core via `vc_stream_feed_pcm` (exposed as `VoiceCatClient.feedPcm`). The core then runs the
|
|
||||||
// same Opus-encode → media-AEAD → UDP path as any other stream — only the *source* is
|
|
||||||
// platform-specific (architecture.md §4).
|
|
||||||
//
|
|
||||||
// Audio-only: we request a 2×2 video plane at 1 fps purely because SCStream needs a video
|
|
||||||
// configuration, and we never add a `.screen` output — only `.audio`. `excludesCurrentProcess
|
|
||||||
// Audio` prevents the self-echo loop of re-capturing our own incoming voice mix.
|
|
||||||
//
|
|
||||||
// `feedPcm` is thread-safe (any thread), so we forward straight from the sample-handler queue.
|
|
||||||
final class ScreenAudioCapture: NSObject, SCStreamOutput, SCStreamDelegate {
|
final class ScreenAudioCapture: NSObject, SCStreamOutput, SCStreamDelegate {
|
||||||
|
|
||||||
/// Receives a full 20 ms frame: (interleaved int16 PCM, samplesPerChannel = 960, channels).
|
/// Receives a full 20 ms frame: (interleaved int16 PCM, samplesPerChannel = 960, channels).
|
||||||
|
|||||||
@@ -1,18 +1,7 @@
|
|||||||
using System.Runtime.InteropServices;
|
using System.Runtime.InteropServices;
|
||||||
|
|
||||||
// WASAPI shared-mode capture from a real hardware INPUT device (a microphone / line-in / aux
|
// Captures a second hardware input for AUX_DEVICE and feeds it through vc_stream_feed_pcm.
|
||||||
// device), plus enumeration of capture endpoints for the aux-stream picker.
|
// Endpoint identifiers are WASAPI-specific and cannot be exchanged with the core's miniaudio ids.
|
||||||
//
|
|
||||||
// This is the input-device analogue of ProcessLoopbackCapture (which captures *render* loopback
|
|
||||||
// via the process-loopback activation hack). Here the source is an ordinary capture endpoint, so
|
|
||||||
// we use the standard IMMDevice.Activate(IAudioClient) path with RCW interfaces — no vtable
|
|
||||||
// gymnastics needed (a normal device's COM objects honour QueryInterface).
|
|
||||||
//
|
|
||||||
// Why client-side capture at all? The core already owns ONE capture device (the mic). It can't
|
|
||||||
// open a second arbitrary input device, so for the aux stream the client captures the device and
|
|
||||||
// feeds 48 kHz / 20 ms int16 frames into the core via vc_stream_feed_pcm — the same external-feed
|
|
||||||
// pipeline screen-audio sharing uses. The device ids here are WASAPI endpoint ids and are NOT the
|
|
||||||
// core's miniaudio ids, so the aux picker is populated independently of vc_list_devices.
|
|
||||||
namespace VoiceCat.App.Audio;
|
namespace VoiceCat.App.Audio;
|
||||||
|
|
||||||
/// <summary>An audio input (capture) endpoint for the aux-stream device picker. <see cref="Id"/>
|
/// <summary>An audio input (capture) endpoint for the aux-stream device picker. <see cref="Id"/>
|
||||||
|
|||||||
@@ -1,17 +1,8 @@
|
|||||||
using System.Runtime.InteropServices;
|
using System.Runtime.InteropServices;
|
||||||
|
|
||||||
// Single-process WASAPI loopback capture via AUDIOCLIENT_ACTIVATION_PARAMS
|
// Process loopback requires MTA activation; blocking activation from the WinForms STA
|
||||||
// (Windows 10 2004+ / Build 19041+).
|
// deadlocks COM completion. The returned interfaces also reject RCW QueryInterface, so audio
|
||||||
//
|
// calls use explicitly owned raw pointers and vtable dispatch.
|
||||||
// Threading: ALL WASAPI init runs on the capture thread (MTA). If called from the
|
|
||||||
// WinForms UI thread (STA), ActivateAudioInterfaceAsync fires ActivateCompleted on
|
|
||||||
// an MTA pool thread; COM marshals that back to the STA pump — but the STA thread is
|
|
||||||
// blocked on CompletionEvent.Wait → deadlock. MTA capture thread avoids this.
|
|
||||||
//
|
|
||||||
// COM QI policy: the COM objects returned by the process-loopback activation path
|
|
||||||
// reject QueryInterface for their own IIDs under .NET's RCW mechanism. Every call
|
|
||||||
// to IAudioClient and IAudioCaptureClient is therefore dispatched via raw vtable
|
|
||||||
// pointer arithmetic, bypassing .NET COM interop entirely.
|
|
||||||
namespace VoiceCat.App.Audio;
|
namespace VoiceCat.App.Audio;
|
||||||
|
|
||||||
public sealed class ProcessLoopbackCapture : IDisposable
|
public sealed class ProcessLoopbackCapture : IDisposable
|
||||||
|
|||||||
@@ -43,8 +43,8 @@ internal struct VcCallbacksNative
|
|||||||
internal struct VcStreamDescNative
|
internal struct VcStreamDescNative
|
||||||
{
|
{
|
||||||
public VcStreamKind Kind;
|
public VcStreamKind Kind;
|
||||||
public IntPtr DeviceId; // unused by vc_stream_start today — device selection is a
|
// Device selection uses vc_set_input_device; stream start passes null.
|
||||||
// separate vc_set_input_device call; always IntPtr.Zero here.
|
public IntPtr DeviceId;
|
||||||
public IntPtr Label;
|
public IntPtr Label;
|
||||||
// Mirrors vc_stream_desc::external_feed. When 1, the core skips its own WASAPI loopback
|
// Mirrors vc_stream_desc::external_feed. When 1, the core skips its own WASAPI loopback
|
||||||
// and the caller feeds PCM via StreamFeedPcm (per-app capture path on Windows).
|
// and the caller feeds PCM via StreamFeedPcm (per-app capture path on Windows).
|
||||||
|
|||||||
@@ -132,7 +132,7 @@ typedef enum vc_event_type {
|
|||||||
* vc_confirm_server_identity. Pins the TLS leaf certificate's own SHA-256 fingerprint
|
* vc_confirm_server_identity. Pins the TLS leaf certificate's own SHA-256 fingerprint
|
||||||
* (verifiable directly from the handshake), NOT the declared Ed25519
|
* (verifiable directly from the handshake), NOT the declared Ed25519
|
||||||
* server_identity_fingerprint from ServerHello — the TLS cert and the server's Ed25519
|
* server_identity_fingerprint from ServerHello — the TLS cert and the server's Ed25519
|
||||||
* identity key are generated independently with no cryptographic binding between them today
|
* identity key are generated independently with no cryptographic binding between them
|
||||||
* (docs/security.md §1.1), so pinning the self-declared value would be circular. The Ed25519
|
* (docs/security.md §1.1), so pinning the self-declared value would be circular. The Ed25519
|
||||||
* fingerprint is still available for human-readable display via
|
* fingerprint is still available for human-readable display via
|
||||||
* vc_get_server_identity_display(), it just isn't the value this gate accepts/rejects on. */
|
* vc_get_server_identity_display(), it just isn't the value this gate accepts/rejects on. */
|
||||||
|
|||||||
@@ -632,9 +632,8 @@ void AudioEngine::on_playback(int16_t* out, ma_uint32 frames) {
|
|||||||
// the engine-wide playback channel count
|
// the engine-wide playback channel count
|
||||||
// dec_channels/frame_samples are bitstream properties (fixed at decoder init); `frames`
|
// dec_channels/frame_samples are bitstream properties (fixed at decoder init); `frames`
|
||||||
// below is the *hardware* playback callback's period, an independent value miniaudio
|
// below is the *hardware* playback callback's period, an independent value miniaudio
|
||||||
// picks on its own — opus_decode's max_samples must be frame_samples, never `frames`
|
// picks on its own — opus_decode's max_samples must be frame_samples, never `frames`.
|
||||||
// (see RemoteStream::ring in audio_engine.h for what went wrong when it was). The ring
|
// The ring decouples the two: top it up by decoding whole Opus frames, then drain exactly
|
||||||
// decouples the two: top it up by decoding whole Opus frames, then drain exactly
|
|
||||||
// `frames` samples-per-channel from it below (silence-padding on underrun = PLC).
|
// `frames` samples-per-channel from it below (silence-padding on underrun = PLC).
|
||||||
const int dec_channels = std::max(1, stream.decoder.channels());
|
const int dec_channels = std::max(1, stream.decoder.channels());
|
||||||
const int frame_samples = stream.decoder.frame_samples();
|
const int frame_samples = stream.decoder.frame_samples();
|
||||||
|
|||||||
@@ -1,7 +1,4 @@
|
|||||||
/*
|
/* Capture, playback, jitter buffering, and mixing. */
|
||||||
* audio/audio_engine.h: capture/playback + DSP + jitter buffer + mixer.
|
|
||||||
*
|
|
||||||
*/
|
|
||||||
#ifndef VOICECAT_AUDIO_AUDIO_ENGINE_H
|
#ifndef VOICECAT_AUDIO_AUDIO_ENGINE_H
|
||||||
#define VOICECAT_AUDIO_AUDIO_ENGINE_H
|
#define VOICECAT_AUDIO_AUDIO_ENGINE_H
|
||||||
|
|
||||||
@@ -394,13 +391,8 @@ class AudioEngine {
|
|||||||
std::atomic<bool> running_{false};
|
std::atomic<bool> running_{false};
|
||||||
std::atomic<float> output_volume_{1.0f};
|
std::atomic<float> output_volume_{1.0f};
|
||||||
|
|
||||||
// Capture-side frame accumulators: miniaudio fires the capture (and loopback) callback at
|
// Device callback periods are independent of codec frame size. These preallocated
|
||||||
// whatever period the hardware/driver chooses — commonly 480 samples (10 ms) on WASAPI
|
// accumulators emit complete frames without allocating on an RT thread.
|
||||||
// shared mode, while the Opus encoder requires exactly frame_samples_ per call (960 for
|
|
||||||
// 20 ms @ 48 kHz). Accumulate incoming PCM until a full frame is ready, then call
|
|
||||||
// capture_cb_. This mirrors the RemoteStream::ring fix on the playback side. Both
|
|
||||||
// accumulators are pre-allocated once in start(); never resized from the RT callback
|
|
||||||
// thread (satisfies architecture.md §3 — no allocation on RT threads).
|
|
||||||
struct CaptureAccum {
|
struct CaptureAccum {
|
||||||
std::vector<int16_t> buf; // pre-sized to frame_samples_ in start()
|
std::vector<int16_t> buf; // pre-sized to frame_samples_ in start()
|
||||||
int count = 0;
|
int count = 0;
|
||||||
@@ -428,15 +420,10 @@ class AudioEngine {
|
|||||||
float gain = 1.0f;
|
float gain = 1.0f;
|
||||||
bool mute = false;
|
bool mute = false;
|
||||||
uint32_t playout_ts = 0;
|
uint32_t playout_ts = 0;
|
||||||
// playout_ts free-runs (advances every callback via PLC), so it must be seeded from, and
|
// Re-seeded from the stream timeline after late joins and transmission gaps.
|
||||||
// periodically re-synced to, the actual stream timeline — otherwise it drifts past the
|
|
||||||
// jitter buffer's drop window across VAD/PTT gaps and late joins and every frame is
|
|
||||||
// dropped/never-due (silent playback). false until the first frame seeds it (on_playback).
|
|
||||||
bool playout_started = false;
|
bool playout_started = false;
|
||||||
|
|
||||||
// Set by push_recv_frame when a kFlagMarker (talkspurt-start) frame arrives; consumed by
|
// A talkspurt marker forces playout-clock reseeding.
|
||||||
// on_playback to force an immediate playout-clock reseed at the new talkspurt, so the
|
|
||||||
// bounded-depth target is re-established cleanly across silence gaps. See on_playback.
|
|
||||||
bool pending_marker = false;
|
bool pending_marker = false;
|
||||||
|
|
||||||
// Diagnostic: times the decode/playback ring underran (produced silence because the
|
// Diagnostic: times the decode/playback ring underran (produced silence because the
|
||||||
@@ -444,12 +431,7 @@ class AudioEngine {
|
|||||||
// "frames arriving but silent / latency starved" signal. Polled via stream_underruns().
|
// "frames arriving but silent / latency starved" signal. Polled via stream_underruns().
|
||||||
std::atomic<uint64_t> underruns{0};
|
std::atomic<uint64_t> underruns{0};
|
||||||
|
|
||||||
// PLC cap (defense-in-depth): consecutive samples produced by packet-loss
|
// Bounds consecutive PLC output so a stale stream eventually becomes silent.
|
||||||
// concealment since the last real decoded frame. Reset to 0 on every real frame.
|
|
||||||
// When it exceeds kPlcCapSamples (audio_engine.cpp), on_playback stops calling
|
|
||||||
// opus_decode(nullptr,0,...) and emits silence instead — bounding the comfort-noise
|
|
||||||
// hiss to ~2 s so a stale stream can never hiss forever even if remove_stream is
|
|
||||||
// never called. See on_playback's decode loop.
|
|
||||||
int64_t plc_samples_since_real = 0;
|
int64_t plc_samples_since_real = 0;
|
||||||
|
|
||||||
// Listener-chosen, local-only noise reduction (docs/voice.md §10). Lazily
|
// Listener-chosen, local-only noise reduction (docs/voice.md §10). Lazily
|
||||||
@@ -483,17 +465,8 @@ class AudioEngine {
|
|||||||
std::atomic<int64_t> last_voice_ms{0};
|
std::atomic<int64_t> last_voice_ms{0};
|
||||||
bool talking = false;
|
bool talking = false;
|
||||||
|
|
||||||
// Decode/playback decoupling ring
|
// Decoding uses the bitstream frame size, while playback drains the device callback
|
||||||
// opus_decode() must be called with max_samples == the encoder's fixed frame size
|
// size. This preallocated ring decouples those clocks and is never resized on the RT path.
|
||||||
// (decoder.frame_samples(), e.g. 960 @ 20ms/48kHz) — that's a property of the bitstream,
|
|
||||||
// not a choice. miniaudio's playback callback period is a *separate*, independently
|
|
||||||
// chosen value (often smaller, e.g. ~480 @ low-latency WASAPI defaults) and must never
|
|
||||||
// be passed to opus_decode as max_samples (doing so made decode fail basically every
|
|
||||||
// callback — silent playback bug, fixed by this ring). on_playback() tops this ring up
|
|
||||||
// by decoding whole Opus frames (decoder's channel count) and drains exactly the
|
|
||||||
// hardware-requested sample count from it each callback, padding with silence (PLC) on
|
|
||||||
// underrun. Sized once in init_ring() (called off the audio thread); never resized from
|
|
||||||
// on_playback (real-time rule).
|
|
||||||
std::vector<int16_t> ring; // capacity = (frame_samples * 8) frames * ring_channels
|
std::vector<int16_t> ring; // capacity = (frame_samples * 8) frames * ring_channels
|
||||||
size_t ring_channels = 1;
|
size_t ring_channels = 1;
|
||||||
size_t ring_head = 0; // next frame (sample-per-channel) to read
|
size_t ring_head = 0; // next frame (sample-per-channel) to read
|
||||||
|
|||||||
@@ -950,9 +950,7 @@ int64_t client_now_ms() {
|
|||||||
.count();
|
.count();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Builds an OpusParams from a wire AudioConfig, applying the same field-by-field mapping on
|
// Maps the complete wire AudioConfig for both local encoders and remote decoders.
|
||||||
// both the send (local-stream encoder) and receive (remote-stream decoder) paths — fixes a
|
|
||||||
// gap where mode/dtx/complexity/application were silently dropped.
|
|
||||||
voicecat::codec::OpusParams opus_params_from_audio_config(const voicecat::v1::AudioConfig& a) {
|
voicecat::codec::OpusParams opus_params_from_audio_config(const voicecat::v1::AudioConfig& a) {
|
||||||
voicecat::codec::OpusParams p;
|
voicecat::codec::OpusParams p;
|
||||||
// Opus always runs at 48 kHz internally: the whole AudioEngine clock is
|
// Opus always runs at 48 kHz internally: the whole AudioEngine clock is
|
||||||
|
|||||||
@@ -333,9 +333,8 @@ void TcpServerConn::wait_closed() {
|
|||||||
|
|
||||||
|
|
||||||
namespace {
|
namespace {
|
||||||
// Try IPv6 dual-stack first (one socket handles both ::1 and 127.0.0.1 — fixes the common
|
// Prefer IPv6 dual-stack so one listener accepts both IPv6 and IPv4 localhost addresses.
|
||||||
// Windows case where `localhost` resolves to ::1 before 127.0.0.1). Falls back to IPv4-only
|
// Fall back to IPv4 when dual-stack binding is unavailable.
|
||||||
// if the OS has IPv6 disabled or the dual-stack bind fails for any reason.
|
|
||||||
asio::ip::tcp::acceptor make_acceptor(asio::io_context& io, uint16_t port) {
|
asio::ip::tcp::acceptor make_acceptor(asio::io_context& io, uint16_t port) {
|
||||||
asio::ip::tcp::acceptor acc(io);
|
asio::ip::tcp::acceptor acc(io);
|
||||||
std::error_code ec;
|
std::error_code ec;
|
||||||
|
|||||||
@@ -314,6 +314,13 @@ message TextMessage {
|
|||||||
it before closing the socket. `code = 0` is reserved for client-initiated graceful
|
it before closing the socket. `code = 0` is reserved for client-initiated graceful
|
||||||
disconnect; server-sent fatal `Disconnect` uses `code ≥ 1` (1 = protocol error,
|
disconnect; server-sent fatal `Disconnect` uses `code ≥ 1` (1 = protocol error,
|
||||||
2 = kicked).
|
2 = kicked).
|
||||||
|
- **Client reconnection is local policy.** The core reports transport loss but does not reconnect.
|
||||||
|
The iOS client snapshots the server, channel, voice subscription, mute, and deafen state, then
|
||||||
|
reconnects with exponential backoff capped at 30 seconds. `NWPathMonitor` proactively replaces
|
||||||
|
a live session when the active interface changes or becomes unavailable, avoiding the TCP
|
||||||
|
keepalive delay; same-interface refreshes are ignored. A user-initiated disconnect cancels the
|
||||||
|
retry task and path monitor. After authentication, the client rejoins the prior channel before
|
||||||
|
restoring voice and local mute/deafen state.
|
||||||
|
|
||||||
## 8. Client-local features (no protocol changes)
|
## 8. Client-local features (no protocol changes)
|
||||||
|
|
||||||
|
|||||||
@@ -281,6 +281,21 @@ Each receiver keeps an **adaptive jitter buffer per ssrc** with **bounded-depth
|
|||||||
and **Advanced** (every knob manual). A2DP output requires an internal-mic preset (the Bluetooth
|
and **Advanced** (every knob manual). A2DP output requires an internal-mic preset (the Bluetooth
|
||||||
device is output-only); the Stereo/Mono Mic presets fall back to the built-in speaker when no
|
device is output-only); the Stereo/Mono Mic presets fall back to the built-in speaker when no
|
||||||
external output is connected (`applyA2dpSpeakerFallback`).
|
external output is connected (`applyA2dpSpeakerFallback`).
|
||||||
|
- **iOS implementation invariants:**
|
||||||
|
- External playback is enabled before connecting because authentication can start the audio
|
||||||
|
engine before the UI receives another turn.
|
||||||
|
- AVAudioEngine callbacks may contain multiple codec frames. The mic path writes them to an
|
||||||
|
SPSC ring and releases complete 20 ms frames at a steady cadence; it never consumes a partial
|
||||||
|
frame, and the pacer is recreated when mono/stereo capture changes.
|
||||||
|
- `AVAudioSession` setters can synchronously emit route-change notifications. Configuration is
|
||||||
|
re-entrancy guarded, and recovery ignores `.categoryChange`, `.routeConfigurationChange`, and
|
||||||
|
`.override` because those reasons are generated by the app's own routing calls. External route
|
||||||
|
changes and engine-configuration notifications still rebuild the graph.
|
||||||
|
- Stereo capture anchors the built-in mic's stereo data source. It does not call
|
||||||
|
`setPreferredInputNumberOfChannels(2)`, which can disrupt A2DP output; the core receives the
|
||||||
|
channel count through `vc_set_capture_channels`.
|
||||||
|
- The A2DP speaker fallback caches its last output override. Reapplying the same override would
|
||||||
|
emit another `.override` notification and recursively trigger recovery.
|
||||||
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
|
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
|
||||||
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
|
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
|
||||||
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard
|
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard
|
||||||
|
|||||||
@@ -186,15 +186,8 @@ void ConnSession::close() {
|
|||||||
state_.store(State::Disconnecting, std::memory_order_release);
|
state_.store(State::Disconnecting, std::memory_order_release);
|
||||||
uint32_t uid = user_id_.load();
|
uint32_t uid = user_id_.load();
|
||||||
if (uid) {
|
if (uid) {
|
||||||
// Broadcast LEFT BEFORE erasing the user, so remaining clients (and the audio
|
// Broadcast before erasing so peers can remove the user's streams. broadcast_left
|
||||||
// engine's remove_stream path on each peer) learn about the departure. This
|
// releases its shared lock before remove_user acquires the unique lock.
|
||||||
// covers ungraceful disconnects (TCP drop, crash, network loss) that previously
|
|
||||||
// silently erased the user from the registry without notifying anyone — which
|
|
||||||
// left stale users in peer client lists and kept Opus PLC hissing forever on
|
|
||||||
// peers whose remove_stream was never triggered. Mirrors kick_user's first half
|
|
||||||
// (session_registry.cpp kick_user). broadcast_left releases its shared_lock
|
|
||||||
// before remove_user acquires the unique_lock, so no deadlock; and send_envelope
|
|
||||||
// on this session is a no-op now that closed_ is true.
|
|
||||||
registry_->broadcast_left(uid, "");
|
registry_->broadcast_left(uid, "");
|
||||||
registry_->remove_user(uid);
|
registry_->remove_user(uid);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -36,7 +36,7 @@ int main(int argc, char** argv) {
|
|||||||
// launcher redirects stdout to a pipe, as the C# interop smoke test's Process does to
|
// launcher redirects stdout to a pipe, as the C# interop smoke test's Process does to
|
||||||
// read the bound port) — go unbuffered so the startup banner (incl. "TCP :<port>") is
|
// read the bound port) — go unbuffered so the startup banner (incl. "TCP :<port>") is
|
||||||
// visible immediately instead of sitting in the CRT's buffer until it fills or the
|
// visible immediately instead of sitting in the CRT's buffer until it fills or the
|
||||||
// process exits. Same fix as tools/vccli/src/main.cpp.
|
// process exits.
|
||||||
std::setvbuf(stdout, nullptr, _IONBF, 0);
|
std::setvbuf(stdout, nullptr, _IONBF, 0);
|
||||||
|
|
||||||
voicecat::server::Config cfg;
|
voicecat::server::Config cfg;
|
||||||
|
|||||||
@@ -169,12 +169,8 @@ int Server::run() {
|
|||||||
});
|
});
|
||||||
|
|
||||||
// ── Keepalive reaper (docs/protocol.md §7) ─────────────────────────────────
|
// ── Keepalive reaper (docs/protocol.md §7) ─────────────────────────────────
|
||||||
// Sweeps every reaper_sweep_ms and drops any session whose last_seen is older than
|
// Drops half-open sessions and broadcasts LEFT so peers remove their streams.
|
||||||
// reaper_timeout_ms. Each close() broadcasts UserEvent::LEFT via the Tier 1 fix, so
|
// Disabled when reaper_timeout_ms <= 0.
|
||||||
// peers learn about the timeout exactly like a normal disconnect — their audio engines
|
|
||||||
// call remove_stream and stop PLC. This catches half-open connections (NAT timeout,
|
|
||||||
// wifi loss without RST, laptop sleep) that never produce a TCP EOF and would otherwise
|
|
||||||
// leave ghost users forever. Disabled when reaper_timeout_ms <= 0.
|
|
||||||
asio::steady_timer reaper_timer(io);
|
asio::steady_timer reaper_timer(io);
|
||||||
std::function<void()> arm_reaper;
|
std::function<void()> arm_reaper;
|
||||||
if (cfg_.reaper_timeout_ms > 0) {
|
if (cfg_.reaper_timeout_ms > 0) {
|
||||||
|
|||||||
@@ -1,10 +1,4 @@
|
|||||||
/*
|
/* Thread-safe in-memory session, channel, user, and media registry. */
|
||||||
* server/session_registry.h — In-memory session, channel, and user registry.
|
|
||||||
*
|
|
||||||
* Tracks all authenticated sessions, the channel tree, user<→>channel assignments,
|
|
||||||
* UDP endpoint bindings, and SSRC<→>session mappings.
|
|
||||||
* Protected by a shared_mutex (many readers, few writers). All methods are thread-safe.
|
|
||||||
*/
|
|
||||||
#ifndef VOICECAT_SERVER_SESSION_REGISTRY_H
|
#ifndef VOICECAT_SERVER_SESSION_REGISTRY_H
|
||||||
#define VOICECAT_SERVER_SESSION_REGISTRY_H
|
#define VOICECAT_SERVER_SESSION_REGISTRY_H
|
||||||
|
|
||||||
@@ -49,19 +43,14 @@ class SessionRegistry {
|
|||||||
public:
|
public:
|
||||||
explicit SessionRegistry(std::shared_ptr<Database> db);
|
explicit SessionRegistry(std::shared_ptr<Database> db);
|
||||||
|
|
||||||
// Load channels from the database, seeding defaults on first run.
|
|
||||||
void load_channels();
|
void load_channels();
|
||||||
|
|
||||||
// Register a session (before auth). Returns the assigned session_id.
|
|
||||||
uint64_t register_session(std::weak_ptr<ConnSession> session);
|
uint64_t register_session(std::weak_ptr<ConnSession> session);
|
||||||
|
|
||||||
// Remove a session (called on disconnect).
|
|
||||||
void unregister_session(uint64_t session_id);
|
void unregister_session(uint64_t session_id);
|
||||||
|
|
||||||
// Add a user once authenticated. Returns the assigned user_id.
|
|
||||||
uint32_t add_user(uint64_t session_id, const voicecat::v1::User& user);
|
uint32_t add_user(uint64_t session_id, const voicecat::v1::User& user);
|
||||||
|
|
||||||
// Remove a user (called on disconnect after auth).
|
|
||||||
void remove_user(uint32_t user_id);
|
void remove_user(uint32_t user_id);
|
||||||
|
|
||||||
// Broadcast a UserEvent::LEFT for a user to all other sessions. Called by
|
// Broadcast a UserEvent::LEFT for a user to all other sessions. Called by
|
||||||
@@ -70,29 +59,21 @@ class SessionRegistry {
|
|||||||
// kick_user(). Takes the shared lock internally; safe to call from ConnSession::close.
|
// kick_user(). Takes the shared lock internally; safe to call from ConnSession::close.
|
||||||
void broadcast_left(uint32_t user_id, const std::string& reason);
|
void broadcast_left(uint32_t user_id, const std::string& reason);
|
||||||
|
|
||||||
// Move a user to a channel. Returns false if channel doesn't exist.
|
|
||||||
bool set_user_channel(uint32_t user_id, uint32_t channel_id);
|
bool set_user_channel(uint32_t user_id, uint32_t channel_id);
|
||||||
|
|
||||||
// Set the user's voice-plane subscription flag on their proto (broadcast-ready).
|
|
||||||
void set_user_voice_subscribed(uint32_t user_id, bool subscribed);
|
void set_user_voice_subscribed(uint32_t user_id, bool subscribed);
|
||||||
|
|
||||||
// Snapshot for ServerStateSnapshot message.
|
|
||||||
std::vector<voicecat::v1::Channel> channel_snapshot() const;
|
std::vector<voicecat::v1::Channel> channel_snapshot() const;
|
||||||
std::vector<voicecat::v1::User> user_snapshot() const;
|
std::vector<voicecat::v1::User> user_snapshot() const;
|
||||||
std::optional<voicecat::v1::User> user_snapshot_user(uint32_t user_id) const;
|
std::optional<voicecat::v1::User> user_snapshot_user(uint32_t user_id) const;
|
||||||
std::optional<std::string> user_nickname(uint32_t user_id) const;
|
std::optional<std::string> user_nickname(uint32_t user_id) const;
|
||||||
|
|
||||||
// Resolve target sessions for a text message relay.
|
|
||||||
std::vector<std::shared_ptr<ConnSession>> resolve_text_targets(
|
std::vector<std::shared_ptr<ConnSession>> resolve_text_targets(
|
||||||
uint64_t sender_session_id, voicecat::v1::TextScope scope, uint32_t target_id) const;
|
uint64_t sender_session_id, voicecat::v1::TextScope scope, uint32_t target_id) const;
|
||||||
|
|
||||||
// Broadcast an envelope to all sessions except the excluded one.
|
|
||||||
void broadcast(const voicecat::v1::Envelope& env, uint64_t exclude_session_id = 0) const;
|
void broadcast(const voicecat::v1::Envelope& env, uint64_t exclude_session_id = 0) const;
|
||||||
|
|
||||||
// Return all sessions whose last_seen is older than max_age_ms (steady_clock ms), i.e.
|
// The caller closes returned sessions outside the registry lock because close re-enters it.
|
||||||
// have not had any inbound TCP or UDP activity in that span. The reaper (server.cpp)
|
|
||||||
// calls close() on each — which broadcasts UserEvent::LEFT via the Tier 1 fix. Locks
|
|
||||||
// only to collect the list; close() runs outside the lock (mirrors kick_user's pattern).
|
|
||||||
std::vector<std::shared_ptr<ConnSession>> find_stale_sessions(int64_t max_age_ms) const;
|
std::vector<std::shared_ptr<ConnSession>> find_stale_sessions(int64_t max_age_ms) const;
|
||||||
|
|
||||||
private:
|
private:
|
||||||
@@ -100,65 +81,42 @@ class SessionRegistry {
|
|||||||
uint64_t exclude_session_id = 0) const;
|
uint64_t exclude_session_id = 0) const;
|
||||||
|
|
||||||
public:
|
public:
|
||||||
// ── Permissions ────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
void set_session_permissions(uint64_t session_id,
|
void set_session_permissions(uint64_t session_id,
|
||||||
const voicecat::v1::Permissions& perms);
|
const voicecat::v1::Permissions& perms);
|
||||||
std::optional<voicecat::v1::Permissions> get_session_permissions(
|
std::optional<voicecat::v1::Permissions> get_session_permissions(
|
||||||
uint64_t session_id) const;
|
uint64_t session_id) const;
|
||||||
|
|
||||||
// ── Moderation ─────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
// Find a live session by its user_id. Returns nullptr if offline.
|
|
||||||
std::shared_ptr<ConnSession> find_session_by_user_id(uint32_t user_id) const;
|
std::shared_ptr<ConnSession> find_session_by_user_id(uint32_t user_id) const;
|
||||||
|
|
||||||
// Forcibly disconnect a user with a reason. Broadcasts UserEvent::LEFT.
|
|
||||||
// Returns true if the user was online.
|
|
||||||
bool kick_user(uint32_t user_id, const std::string& reason);
|
bool kick_user(uint32_t user_id, const std::string& reason);
|
||||||
|
|
||||||
// Kick a user and insert a persistent ban. Returns true if the user was online.
|
|
||||||
bool ban_user(uint32_t user_id, const std::string& reason, int64_t expires_at);
|
bool ban_user(uint32_t user_id, const std::string& reason, int64_t expires_at);
|
||||||
|
|
||||||
// Set server-mute/deafen flags on a user and broadcast the update.
|
|
||||||
bool set_server_mute(uint32_t user_id, bool muted, bool deafened);
|
bool set_server_mute(uint32_t user_id, bool muted, bool deafened);
|
||||||
|
|
||||||
// Move a user to a channel (permission-checked by caller).
|
|
||||||
bool move_user(uint32_t user_id, uint32_t channel_id);
|
bool move_user(uint32_t user_id, uint32_t channel_id);
|
||||||
|
|
||||||
// ── Channel CRUD ───────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
// Create a channel. Returns the new channel id, or 0 on error.
|
|
||||||
uint32_t create_channel(const voicecat::v1::Channel& ch, const std::string& password,
|
uint32_t create_channel(const voicecat::v1::Channel& ch, const std::string& password,
|
||||||
std::string& error);
|
std::string& error);
|
||||||
|
|
||||||
// Update a channel. Returns false on error.
|
|
||||||
bool update_channel(const voicecat::v1::Channel& ch, const std::string& password,
|
bool update_channel(const voicecat::v1::Channel& ch, const std::string& password,
|
||||||
std::string& error);
|
std::string& error);
|
||||||
|
|
||||||
// Delete a channel. Remaining users are moved to Lobby (id=1). Returns false on error.
|
// Deleting a channel moves its users to Lobby.
|
||||||
bool delete_channel(uint32_t channel_id, std::string& error);
|
bool delete_channel(uint32_t channel_id, std::string& error);
|
||||||
|
|
||||||
// Return a channel proto by id, or nullopt.
|
|
||||||
std::optional<voicecat::v1::Channel> get_channel(uint32_t channel_id) const;
|
std::optional<voicecat::v1::Channel> get_channel(uint32_t channel_id) const;
|
||||||
|
|
||||||
// Check a channel password.
|
|
||||||
bool check_channel_password(uint32_t channel_id, const std::string& password) const;
|
bool check_channel_password(uint32_t channel_id, const std::string& password) const;
|
||||||
|
|
||||||
// ── UDP / media ────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
// Register a session's UDP token (called at auth success).
|
|
||||||
void register_udp_token(const std::array<uint8_t, 16>& token, uint64_t session_id);
|
void register_udp_token(const std::array<uint8_t, 16>& token, uint64_t session_id);
|
||||||
|
|
||||||
// Locate a session by its UDP binding token (called by MediaRelay on UDP_BINDING).
|
|
||||||
std::shared_ptr<ConnSession> find_by_udp_token(const std::array<uint8_t, 16>& token) const;
|
std::shared_ptr<ConnSession> find_by_udp_token(const std::array<uint8_t, 16>& token) const;
|
||||||
|
|
||||||
// Associate a UDP endpoint with a session (called by MediaRelay after token verification).
|
|
||||||
void register_udp_endpoint(asio::ip::udp::endpoint ep, uint64_t session_id);
|
void register_udp_endpoint(asio::ip::udp::endpoint ep, uint64_t session_id);
|
||||||
|
|
||||||
// Locate the session that owns a UDP sender endpoint (called per incoming voice packet).
|
|
||||||
std::shared_ptr<ConnSession> find_by_udp_endpoint(const asio::ip::udp::endpoint& ep) const;
|
std::shared_ptr<ConnSession> find_by_udp_endpoint(const asio::ip::udp::endpoint& ep) const;
|
||||||
|
|
||||||
// Assign an SSRC for a new stream. Returns the assigned SSRC.
|
|
||||||
uint32_t assign_ssrc(uint64_t session_id);
|
uint32_t assign_ssrc(uint64_t session_id);
|
||||||
|
|
||||||
// Add/replace a stream entry on a user (called when StreamAnnounce succeeds).
|
// Add/replace a stream entry on a user (called when StreamAnnounce succeeds).
|
||||||
@@ -170,17 +128,12 @@ class SessionRegistry {
|
|||||||
// User proto for broadcasting, or nullopt if user not found.
|
// User proto for broadcasting, or nullopt if user not found.
|
||||||
std::optional<voicecat::v1::User> clear_user_stream(uint32_t user_id, uint32_t stream_id);
|
std::optional<voicecat::v1::User> clear_user_stream(uint32_t user_id, uint32_t stream_id);
|
||||||
|
|
||||||
// Get all sessions in a channel except the one excluded (for SFU relay).
|
|
||||||
std::vector<std::shared_ptr<ConnSession>> find_channel_sessions(
|
std::vector<std::shared_ptr<ConnSession>> find_channel_sessions(
|
||||||
uint32_t channel_id, uint64_t exclude_session_id = 0) const;
|
uint32_t channel_id, uint64_t exclude_session_id = 0) const;
|
||||||
|
|
||||||
// Return the channel_id of a user (0 if not found).
|
|
||||||
uint32_t user_channel(uint32_t user_id) const;
|
uint32_t user_channel(uint32_t user_id) const;
|
||||||
|
|
||||||
// Return a channel's authoritative AudioConfig (per-channel Opus tuning), or nullopt
|
// Returns the authoritative per-channel Opus configuration.
|
||||||
// if the channel doesn't exist. There is no per-id Channel getter today otherwise —
|
|
||||||
// channel_snapshot() copies every channel, which callers needing just one config should
|
|
||||||
// avoid.
|
|
||||||
std::optional<voicecat::v1::AudioConfig> channel_audio_config(uint32_t channel_id) const;
|
std::optional<voicecat::v1::AudioConfig> channel_audio_config(uint32_t channel_id) const;
|
||||||
|
|
||||||
private:
|
private:
|
||||||
|
|||||||
Reference in New Issue
Block a user