Files
voice-cat/clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift
Talon fdcc84fb42 fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI
Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which
achieves stereo mic + A2DP output. Five fixes:

1. configureStereoCapture now calls setPreferredInput +
   setInputDataSource (mirroring TeamTalk5's SoundDevicesModel).
   Previously omitted based on incorrect diagnosis that
   setPreferredInput collapsed A2DP — the real culprit was
   setPreferredInputNumberOfChannels(2), which neither project uses.

2. New C ABI: vc_audio_restart (full stop + re-init, unlike
   suspend/resume which only stop/start). Swift wrapper added.
   The withAudioSuspend wrapper that used it was removed after
   on-device testing showed it killed all audio (including
   VoiceOver) when switching presets — the core's
   set_capture_channels handles engine restart internally.

3. Bluetooth options: Voice Chat preset now includes BOTH
   .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's
   UtilSound.swift:228). Previously HFP-only blocked A2DP headphones.

4. Capture channels now reset when switching stereo→mono via
   selectCaptureChannels/applyPreset. AudioSessionManager tracks
   activeMicStreamId (set by SessionState on join/leave voice).

5. Docs synced: voice.md, tech-stack.md, architecture.md,
   PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2)
   references.

Verified: ctest --preset dev 21/21 green, iOS client builds.
Stereo mic + A2DP output still needs on-device debugging — the
core recipe is correct but iOS 26 route behavior requires
hands-on testing with a debugger.
2026-06-19 16:58:21 +02:00

539 lines
23 KiB
Swift
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

// VoiceCatClient the public, Swift-idiomatic surface over libvoicecat. This is the Swift
// analog of the C# client's `VoiceCatClient.cs` (clients/windows/VoiceCat.Interop).
//
// Key patterns carried over from the proven C# implementation (see docs/architecture.md §4
// per-platform binding notes):
//
// 1. HANDLE OWNERSHIP: the class owns `vc_client*`; `deinit` calls `vc_client_destroy`
// (which synchronously joins every internal thread, so nothing can still be reading the
// config-string pointers or firing callbacks by the time it returns).
//
// 2. CONFIG STRING LIFETIMES: the core stores raw pointers from `vc_config` by value it
// does NOT copy the string data. `client_name`/`client_version`/`tofu_store_path` are
// read later, whenever `connect()` actually runs on the io_thread_. So the native CString
// storage (`_clientNamePtr` etc.) must outlive the WHOLE client, not just `init`. It's
// freed in `deinit`, AFTER `vc_client_destroy` has returned. (C#: Marshal.StringToCoTask
// MemUTF8 in ctor, FreeCoTaskMem in Dispose after destroy.)
//
// 3. EVENT DELIVERY THREAD HANDOFF: `on_event` fires on the core's event thread. Events are
// buffered in a lock-protected array and drained on `DispatchQueue.main` this is the
// boundary where the core's thread hands off to the UI thread. The C# analog is
// `Channel<VoiceCatEvent>` drained by a 30ms WinForms Timer; the Swift analog is a
// coalesced main-queue drain (only one async block scheduled at a time). `on_event`'s
// `text` is copied to a Swift `String` inside the callback (Callbacks.swift) before
// enqueueing the raw pointer is dangling by the time the main thread drains.
//
// 4. LEVEL METER COALESCING: `on_level` fires far more often than `on_event` and
// intermediate values are visually irrelevant coalesced to "latest sample per
// stream_id" in a lock-protected dictionary, drained on main alongside events.
// (C#: ConcurrentDictionary<uint,float> cleared in PumpEvents.)
//
// 5. IMMEDIATE vc_free_* ON LIST READS: `listChannels()`/`listUsers()`/etc. walk the native
// array, convert to Swift value types, and call `vc_free_*_list` INSIDE the function
// callers never manage native list lifetime. (C#: Marshaling.ToManaged does the same.)
import VoiceCatC
import Foundation
/// The Swift wrapper around `vc_client*`. Owns the native handle for its entire lifetime;
/// `deinit` destroys it. Events and level meters are delivered on the main queue via the
/// `onEvent` / `onLevel` closures.
///
/// Thread-safety: the public methods are not thread-safe call them from the main thread
/// (the standard AppKit/SwiftUI pattern). The internal event/level buffers are thread-safe
/// (lock-protected) because they're written from the core's event thread.
public final class VoiceCatClient {
// MARK: - Stored properties
/// The opaque C handle (`vc_client*` Swift imports the incomplete C struct as
/// `OpaquePointer`). Set in `init`, passed to every C function, destroyed in `deinit`.
private var handle: OpaquePointer?
/// Unmanaged pointer to `self` passed as `vc_callbacks.user` so the C function-pointer
/// callbacks can resolve back to this instance. `passUnretained` (not `passRetained`)
/// because we want normal ARC to control the object's lifetime `deinit` calls
/// `vc_client_destroy` (joins all threads) before the object's memory is freed, so no
/// callback can fire with a dangling `user` pointer. See Callbacks.swift.
///
/// Computed (not stored) to break a circular init dependency: it needs `self`, but
/// stored properties must be initialized before `self` is available. `Unmanaged.passUn
/// retained(self).toOpaque()` always returns the same address for a given instance, so
/// computing it on demand is safe and consistent.
private var selfPointer: UnsafeMutableRawPointer {
Unmanaged.passUnretained(self).toOpaque()
}
/// Native CString storage backing `vc_config` must outlive the whole client (the core
/// stores raw pointers, doesn't copy). Freed in `deinit` after `vc_client_destroy`.
private var clientNamePtr: UnsafeMutablePointer<CChar>?
private var clientVersionPtr: UnsafeMutablePointer<CChar>?
private var tofuStorePathPtr: UnsafeMutablePointer<CChar>?
// MARK: - Event / level delivery (main-queue)
/// Called on the main queue for every event, in order, never coalesced. Set this from
/// the main thread (AppKit/SwiftUI) to drive your UI.
public var onEvent: ((VoiceCatEvent) -> Void)?
/// Called on the main queue with the latest RMS level per stream_id since the last drain.
/// Intermediate values are coalesced (only the latest per stream_id is delivered).
public var onLevel: ((UInt32, Float) -> Void)?
/// Lock-protected buffers, written from the core's event thread, drained on main.
private let bufferLock = NSLock()
private var eventBuffer: [VoiceCatEvent] = []
private var levelSamples: [UInt32: Float] = [:]
private var drainScheduled = false
// MARK: - Init / deinit
/// Create a client. `config.clientName`/`clientVersion`/`tofuStorePath` are copied to
/// native CString storage held for the client's entire lifetime (the core reads them
/// later, e.g. when `connect()` runs on the io thread).
public init(config: VoiceCatConfig) {
// Allocate native C strings must persist until after vc_client_destroy in deinit.
// These don't need `self`, so they're safe to set first.
self.clientNamePtr = strdup(config.clientName)
self.clientVersionPtr = strdup(config.clientVersion)
self.tofuStorePathPtr = config.tofuStorePath.flatMap { strdup($0) }
self.handle = nil // placeholder set below after callbacks are wired
// All stored properties are now initialized `self` is fully available, so we can
// call `selfPointer` (the computed property) to build the callbacks struct.
var nativeConfig = vc_config()
nativeConfig.client_name = UnsafePointer(clientNamePtr)
nativeConfig.client_version = UnsafePointer(clientVersionPtr)
nativeConfig.log_level = config.logLevel.cValue
nativeConfig.tofu_store_path = UnsafePointer(tofuStorePathPtr)
let callbacks = Callbacks.make(user: selfPointer)
self.handle = vc_client_create(&nativeConfig, callbacks)
if handle == nil {
free(clientNamePtr); clientNamePtr = nil
free(clientVersionPtr); clientVersionPtr = nil
if let tofuStorePathPtr { free(tofuStorePathPtr); self.tofuStorePathPtr = nil }
fatalError("vc_client_create returned nil")
}
}
deinit {
if let handle {
// Joins every internal thread synchronously no callbacks can fire after this
// returns, so the selfPointer and config-string pointers are safe to free.
vc_client_destroy(handle)
self.handle = nil
}
// Free config strings AFTER destroy (the core may have been reading them up until
// destroy joined the io thread).
free(clientNamePtr); clientNamePtr = nil
free(clientVersionPtr); clientVersionPtr = nil
if let tofuStorePathPtr { free(tofuStorePathPtr); self.tofuStorePathPtr = nil }
}
// MARK: - Internal: event/level enqueue (called from the core's event thread)
/// Called by Callbacks.onEvent on the core's event thread. Buffers the event and
/// schedules a coalesced main-queue drain.
internal func enqueueEvent(_ event: VoiceCatEvent) {
bufferLock.lock()
eventBuffer.append(event)
let shouldSchedule = !drainScheduled
drainScheduled = true
bufferLock.unlock()
if shouldSchedule {
DispatchQueue.main.async { [weak self] in self?.drain() }
}
}
/// Called by Callbacks.onLevel on the core's event thread. Coalesces to latest-per-stream
/// and schedules a coalesced main-queue drain.
internal func enqueueLevel(_ streamId: UInt32, _ rms: Float) {
bufferLock.lock()
levelSamples[streamId] = rms
let shouldSchedule = !drainScheduled
drainScheduled = true
bufferLock.unlock()
if shouldSchedule {
DispatchQueue.main.async { [weak self] in self?.drain() }
}
}
/// Drains buffered events + coalesced levels on the main queue. Only one drain is
/// scheduled at a time (debounced via `drainScheduled`).
private func drain() {
bufferLock.lock()
let events = eventBuffer
eventBuffer.removeAll()
let levels = levelSamples
levelSamples.removeAll()
drainScheduled = false
bufferLock.unlock()
for event in events { onEvent?(event) }
for (streamId, rms) in levels { onLevel?(streamId, rms) }
}
// MARK: - Lifecycle (statics)
/// The core's version string (e.g. "VoiceCat 0.0.1 (protocol v1)"). Static literal never freed.
public static var versionString: String {
String(cString: vc_version_string())
}
/// Human-readable description of a result code. Static literal never freed.
public static func resultString(_ code: VoiceCatResult) -> String {
String(cString: vc_result_string(code.cValue))
}
// MARK: - Connection & auth (async; results via onEvent)
@discardableResult
public func connect(host: String, port: UInt16) -> VoiceCatResult {
VoiceCatResult(vc_connect(handle, host, port))
}
@discardableResult
public func disconnect() -> VoiceCatResult {
VoiceCatResult(vc_disconnect(handle))
}
@discardableResult
public func authenticateGuest(_ nickname: String) -> VoiceCatResult {
VoiceCatResult(vc_authenticate_guest(handle, nickname))
}
@discardableResult
public func authenticateUser(_ username: String, password: String) -> VoiceCatResult {
VoiceCatResult(vc_authenticate_user(handle, username, password))
}
// MARK: - TOFU server-identity gate (M4)
/// Accept or reject the pending server-identity check. Call after a `.serverIdentity`
/// event. `accept=true` on firstConnect/mismatch updates the pin file and proceeds;
/// `accept=false` aborts (emits `.disconnected` with `.crypto`). See docs/security.md §1.1.
@discardableResult
public func confirmServerIdentity(accept: Bool) -> VoiceCatResult {
VoiceCatResult(vc_confirm_server_identity(handle, accept ? 1 : 0))
}
/// The Ed25519 identity fingerprint from ServerHello, hex-formatted DISPLAY ONLY, not
/// the value the TOFU gate pins on (see docs/security.md §1.1). Empty if not yet available.
/// Uses the two-call idiom: query size with nil buffer, then allocate + fetch.
public func getServerIdentityDisplay() -> String {
var len: Int = 0
_ = vc_get_server_identity_display(handle, nil, 0, &len)
if len == 0 { return "" }
let buf = UnsafeMutablePointer<CChar>.allocate(capacity: len + 1)
defer { buf.deallocate() }
_ = vc_get_server_identity_display(handle, buf, len + 1, &len)
return String(cString: buf)
}
// MARK: - Channels
/// Join a channel. Result arrives as a `.joinResult` event (not via the return value,
/// which only reflects "request queued"). `password` is for password-protected channels.
@discardableResult
public func joinChannel(_ channelId: UInt32, password: String? = nil) -> VoiceCatResult {
VoiceCatResult(vc_join_channel(handle, channelId, password))
}
@discardableResult
public func leaveChannel() -> VoiceCatResult {
VoiceCatResult(vc_leave_channel(handle))
}
/// Pull the current channel tree. Re-call after `.channelList`/`.userJoined`/`.userLeft`/
/// `.userUpdated` events. The native list is freed inside this call callers never
/// manage native lifetime.
public func listChannels() -> [Channel] {
var native = vc_channel_list()
_ = vc_list_channels(handle, &native)
return Marshaling.channels(&native)
}
public func listUsers() -> [User] {
var native = vc_user_list()
_ = vc_list_users(handle, &native)
return Marshaling.users(&native)
}
public func listUserStreams(_ userId: UInt32) -> [StreamSummary] {
var native = vc_stream_summary_list()
let r = vc_list_user_streams(handle, userId, &native)
guard r == VC_OK else { return [] }
return Marshaling.streamSummaries(&native)
}
// MARK: - Local media streams
/// Start a mic / screen-audio / aux stream. Returns `(result, streamId)` `streamId`
/// is non-zero on success. The `label` and `deviceId` C strings are only needed for the
/// duration of the call (the core copies what it needs), so we use temporary strdup'd
/// buffers freed via `defer`.
@discardableResult
public func startStream(_ descriptor: StreamDescriptor) -> (VoiceCatResult, UInt32) {
var streamId: UInt32 = 0
let labelPtr = strdup(descriptor.label)
defer { free(labelPtr) }
let deviceIdPtr = descriptor.deviceId.flatMap { strdup($0) }
defer { if let deviceIdPtr { free(deviceIdPtr) } }
var desc = vc_stream_desc()
desc.kind = descriptor.kind.cValue
desc.device_id = deviceIdPtr.map { UnsafePointer($0) }
desc.label = UnsafePointer(labelPtr)
let r = vc_stream_start(handle, &desc, &streamId)
return (VoiceCatResult(r), streamId)
}
@discardableResult
public func stopStream(_ streamId: UInt32) -> VoiceCatResult {
VoiceCatResult(vc_stream_stop(handle, streamId))
}
@discardableResult
public func setInputDevice(streamId: UInt32, deviceId: String?) -> VoiceCatResult {
VoiceCatResult(vc_set_input_device(handle, streamId, deviceId))
}
/// Set the capture channel count for a local MIC stream (1 = mono, 2 = stereo interleaved).
/// Takes effect on the next AudioEngine restart (immediately if already running). Used by
/// the iOS `IOSAudioRouter` when the user picks stereo built-in mic capture.
@discardableResult
public func setCaptureChannels(streamId: UInt32, channels: UInt32) -> VoiceCatResult {
VoiceCatResult(vc_set_capture_channels(handle, streamId, channels))
}
@discardableResult
public func setInputMode(_ mode: VoiceCatInputMode) -> VoiceCatResult {
VoiceCatResult(vc_set_input_mode(handle, mode.cValue))
}
/// VAD threshold: normalized RMS 0.01.0 (default ~0.025). Takes effect immediately.
@discardableResult
public func setVadThreshold(_ threshold: Float) -> VoiceCatResult {
VoiceCatResult(vc_set_vad_threshold(handle, threshold))
}
@discardableResult
public func setPushToTalk(_ active: Bool) -> VoiceCatResult {
VoiceCatResult(vc_set_push_to_talk(handle, active ? 1 : 0))
}
@discardableResult
public func setSelfMute(micMuted: Bool, deafened: Bool) -> VoiceCatResult {
VoiceCatResult(vc_set_self_mute(handle, micMuted ? 1 : 0, deafened ? 1 : 0))
}
// MARK: - AVAudioSession interruption hooks (iOS)
/// Pause miniaudio device I/O. Call when AVAudioSession interruption begins.
@discardableResult
public func audioSuspend() -> VoiceCatResult {
VoiceCatResult(vc_audio_suspend(handle))
}
/// Resume miniaudio device I/O. Call after re-activating AVAudioSession.
@discardableResult
public func audioResume() -> VoiceCatResult {
VoiceCatResult(vc_audio_resume(handle))
}
/// Full audio engine restart uninitialize and re-initialize the capture and playback
/// devices so they pick up a new AVAudioSession route. Call this AFTER reconfiguring
/// AVAudioSession (setCategory, setPreferredInput, setPreferredPolarPattern, etc.) so the
/// core's devices reopen against the new route. Unlike `audioSuspend()`/`audioResume()`
/// (which only stop/start the existing devices, leaving them bound to the route that was
/// active when they were opened), this fully re-initializes them. Mirrors TeamTalk5's
/// closeSoundDevices()/initSoundInputDevice()/initSoundOutputDevice() pattern. Safe to
/// call when the engine is not running (it will just start it).
@discardableResult
public func audioRestart() -> VoiceCatResult {
VoiceCatResult(vc_audio_restart(handle))
}
// MARK: - Receive-side, per remote stream (LOCAL no protocol traffic; docs/voice.md §10)
@discardableResult
public func setRemoteStream(userId: UInt32, streamId: UInt32, gain: Float,
muted: Bool, noiseReduction: Bool) -> VoiceCatResult {
VoiceCatResult(vc_set_remote_stream(handle, userId, streamId, gain,
muted ? 1 : 0, noiseReduction ? 1 : 0))
}
public func getRemoteStream(userId: UInt32, streamId: UInt32) -> (VoiceCatResult, RemoteStreamState?) {
var state = vc_remote_stream_state()
let r = vc_get_remote_stream(handle, userId, streamId, &state)
guard r == VC_OK else { return (VoiceCatResult(r), nil) }
return (VoiceCatResult(r), Marshaling.remoteStreamState(state))
}
public func getStreamAudioConfig(userId: UInt32, streamId: UInt32) -> (VoiceCatResult, AudioConfig?) {
var cfg = vc_audio_config()
let r = vc_get_stream_audio_config(handle, userId, streamId, &cfg)
guard r == VC_OK else { return (VoiceCatResult(r), nil) }
return (VoiceCatResult(r), Marshaling.audioConfig(cfg))
}
// MARK: - Text
@discardableResult
public func sendText(scope: VoiceCatTextScope, targetId: UInt32, text: String) -> VoiceCatResult {
VoiceCatResult(vc_send_text(handle, scope.cValue, targetId, text))
}
// MARK: - Device enumeration (works pre-connect)
public func listDevices(_ kind: VoiceCatDeviceKind) -> [Device] {
var native = vc_device_list()
_ = vc_list_devices(handle, kind.cValue, &native)
return Marshaling.devices(&native)
}
// MARK: - M5: Moderation
@discardableResult
public func kickUser(_ userId: UInt32, reason: String? = nil) -> VoiceCatResult {
VoiceCatResult(vc_kick_user(handle, userId, reason))
}
@discardableResult
public func banUser(_ userId: UInt32, reason: String? = nil,
expiresUnixMs: UInt64 = 0) -> VoiceCatResult {
VoiceCatResult(vc_ban_user(handle, userId, reason, expiresUnixMs))
}
@discardableResult
public func setPermission(_ userId: UInt32, perms: Permissions) -> VoiceCatResult {
var native = vc_permissions()
native.can_create_temp_channel = perms.canCreateTempChannel ? 1 : 0
native.can_kick = perms.canKick ? 1 : 0
native.can_ban = perms.canBan ? 1 : 0
native.can_move_users = perms.canMoveUsers ? 1 : 0
native.can_admin_accounts = perms.canAdminAccounts ? 1 : 0
native.is_admin = perms.isAdmin ? 1 : 0
return VoiceCatResult(vc_set_permission(handle, userId, &native))
}
@discardableResult
public func setServerMute(_ userId: UInt32, muted: Bool, deafened: Bool) -> VoiceCatResult {
VoiceCatResult(vc_set_server_mute(handle, userId, muted ? 1 : 0, deafened ? 1 : 0))
}
@discardableResult
public func moveUser(_ userId: UInt32, toChannel channelId: UInt32) -> VoiceCatResult {
VoiceCatResult(vc_move_user(handle, userId, channelId))
}
// MARK: - M5: Channel admin
@discardableResult
public func createChannel(_ info: ChannelEdit) -> VoiceCatResult {
var native = vc_channel_info()
Self.populateChannelInfo(&native, from: info)
defer { Self.freeChannelInfoStrings(&native) }
return VoiceCatResult(vc_create_channel(handle, &native))
}
@discardableResult
public func editChannel(_ info: ChannelEdit) -> VoiceCatResult {
var native = vc_channel_info()
Self.populateChannelInfo(&native, from: info)
defer { Self.freeChannelInfoStrings(&native) }
return VoiceCatResult(vc_edit_channel(handle, &native))
}
@discardableResult
public func deleteChannel(_ channelId: UInt32) -> VoiceCatResult {
VoiceCatResult(vc_delete_channel(handle, channelId))
}
// MARK: - M5: Account admin
@discardableResult
public func createAccount(_ username: String, password: String) -> VoiceCatResult {
VoiceCatResult(vc_create_account(handle, username, password))
}
@discardableResult
public func resetPassword(_ username: String, newPassword: String) -> VoiceCatResult {
VoiceCatResult(vc_reset_password(handle, username, newPassword))
}
@discardableResult
public func deleteAccount(_ username: String) -> VoiceCatResult {
VoiceCatResult(vc_delete_account(handle, username))
}
/// Request the account list result arrives as a `.accountList` event, then call
/// `listAccounts()` to pull the cached list.
@discardableResult
public func requestAccountList() -> VoiceCatResult {
VoiceCatResult(vc_list_accounts(handle))
}
public func listAccounts() -> [Account] {
var native = vc_account_list()
_ = vc_get_account_list(handle, &native)
return Marshaling.accounts(&native)
}
public func getPermissions() -> Permissions {
var native = vc_permissions()
_ = vc_get_permissions(handle, &native)
return Marshaling.permissions(native)
}
}
// MARK: - Helpers for vc_channel_info / vc_audio_config construction
extension VoiceCatClient {
/// Populate a `vc_channel_info` from a Swift `ChannelEdit`. The string fields
/// (`name`/`topic`/`password`) are strdup'd the caller MUST call
/// `freeChannelInfoStrings(_:)` after the C call returns (the core copies what it needs
/// during the call, so the temporary buffers can be freed via `defer`).
internal static func populateChannelInfo(_ native: inout vc_channel_info, from info: ChannelEdit) {
native.id = info.id
native.parent_id = info.parentId
native.name = UnsafePointer(strdup(info.name))
native.topic = UnsafePointer(strdup(info.topic))
native.password_protected = info.passwordProtected ? 1 : 0
native.password = (info.passwordProtected && !(info.password?.isEmpty ?? true))
? UnsafePointer(strdup(info.password!)) : nil
native.max_users = info.maxUsers
native.sort_order = info.sortOrder
native.audio = info.audio.toNative()
}
/// Free the strdup'd string fields of a `vc_channel_info` populated by
/// `populateChannelInfo`. Call this in a `defer` after the C call.
internal static func freeChannelInfoStrings(_ native: inout vc_channel_info) {
if let p = native.name { free(UnsafeMutablePointer(mutating: p)); native.name = nil }
if let p = native.topic { free(UnsafeMutablePointer(mutating: p)); native.topic = nil }
if let p = native.password { free(UnsafeMutablePointer(mutating: p)); native.password = nil }
}
}
extension AudioConfig {
/// Convert to a native `vc_audio_config`.
internal func toNative() -> vc_audio_config {
var n = vc_audio_config()
n.codec = codec
n.mode = stereo ? 1 : 0
n.sample_rate = sampleRate
n.bitrate_bps = bitrateBps
n.frame_ms = frameMs
n.application = application
n.fec = fec ? 1 : 0
n.expected_packet_loss = expectedPacketLoss
n.dtx = dtx ? 1 : 0
n.complexity = complexity
return n
}
}