2026-06-19 02:10:25 +02:00
|
|
|
import Foundation
|
|
|
|
|
import AVFoundation
|
|
|
|
|
import VoiceCatCore
|
|
|
|
|
|
|
|
|
|
// MARK: - Helper types
|
|
|
|
|
|
|
|
|
|
struct ChatMessage: Identifiable {
|
|
|
|
|
let id = UUID()
|
|
|
|
|
let timestamp: Date
|
|
|
|
|
let senderName: String
|
|
|
|
|
let text: String
|
|
|
|
|
let scope: VoiceCatTextScope
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
struct ActivityEntry: Identifiable {
|
|
|
|
|
let id = UUID()
|
|
|
|
|
let timestamp: Date
|
|
|
|
|
let text: String
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
struct VoiceState {
|
|
|
|
|
var micActive = false
|
|
|
|
|
var selfMuted = false
|
|
|
|
|
var selfDeafened = false
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
var serverMuted = false
|
|
|
|
|
var serverDeafened = false
|
2026-06-19 02:10:25 +02:00
|
|
|
var inputMode: VoiceCatInputMode = .voiceActivation
|
|
|
|
|
var vadThreshold: Float = 0.025
|
feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
|
|
|
var inputGain: Float = 1.0
|
2026-06-19 02:10:25 +02:00
|
|
|
var level: Float = 0.0
|
|
|
|
|
var currentDeviceId: String?
|
|
|
|
|
var localStreamId: UInt32 = 0
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
var screenSharing = false
|
|
|
|
|
var screenStreamId: UInt32 = 0
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - SessionState
|
|
|
|
|
|
|
|
|
|
@Observable
|
|
|
|
|
@MainActor
|
|
|
|
|
final class SessionState {
|
|
|
|
|
let client: VoiceCatClient
|
|
|
|
|
let selfUserId: UInt32
|
|
|
|
|
|
|
|
|
|
var channels: [Channel] = []
|
|
|
|
|
var users: [User] = []
|
|
|
|
|
var currentChannelId: UInt32 = 0
|
|
|
|
|
var messages: [ChatMessage] = []
|
|
|
|
|
var activityLog: [ActivityEntry] = []
|
|
|
|
|
var voiceState = VoiceState()
|
|
|
|
|
var permissions: Permissions
|
|
|
|
|
var accounts: [Account] = []
|
|
|
|
|
var devices: [Device] = []
|
|
|
|
|
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
/// Host side of iOS screen-audio sharing — drains the broadcast extension's App Group ring
|
|
|
|
|
/// and feeds the SCREEN_AUDIO stream this session owns. See BroadcastAudioPump.
|
|
|
|
|
private let broadcastPump = BroadcastAudioPump()
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
init(client: VoiceCatClient, selfUserId: UInt32, permissions: Permissions) {
|
|
|
|
|
self.client = client
|
|
|
|
|
self.selfUserId = selfUserId
|
|
|
|
|
self.permissions = permissions
|
feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
|
|
|
loadAndApplyVoiceSettings()
|
2026-06-19 02:10:25 +02:00
|
|
|
refreshChannels()
|
|
|
|
|
refreshUsers()
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
syncSelfChannel()
|
2026-06-19 02:10:25 +02:00
|
|
|
refreshDevices()
|
|
|
|
|
client.onEvent = { [weak self] ev in
|
|
|
|
|
Task { @MainActor [weak self] in self?.handleEvent(ev) }
|
|
|
|
|
}
|
|
|
|
|
client.onLevel = { [weak self] _, rms in
|
|
|
|
|
Task { @MainActor [weak self] in self?.voiceState.level = rms }
|
|
|
|
|
}
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
broadcastPump.onBroadcastStarted = { [weak self] in self?.startScreenShare() }
|
|
|
|
|
broadcastPump.onBroadcastFinished = { [weak self] in self?.stopScreenShare() }
|
|
|
|
|
broadcastPump.start()
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
deinit {
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
broadcastPump.stop()
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - Event dispatch
|
|
|
|
|
|
|
|
|
|
func handleEvent(_ ev: VoiceCatEvent) {
|
|
|
|
|
switch ev.type {
|
|
|
|
|
case .channelList:
|
|
|
|
|
refreshChannels()
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
syncSelfChannel()
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
case .userJoined:
|
|
|
|
|
// ev.text = nickname, ev.channelId = the channel they joined (per voicecat.h).
|
|
|
|
|
if ev.userId != selfUserId && ev.channelId == currentChannelId {
|
|
|
|
|
EventFeedback.shared.play(.channelJoin)
|
|
|
|
|
EventFeedback.shared.speak("\(ev.text ?? "Someone") joined")
|
|
|
|
|
}
|
|
|
|
|
refreshUsers()
|
|
|
|
|
syncSelfChannel()
|
|
|
|
|
case .userLeft:
|
|
|
|
|
// Capture the leaving user's prior nickname/channel before refreshUsers() drops them.
|
|
|
|
|
if ev.userId != selfUserId,
|
|
|
|
|
let gone = users.first(where: { $0.id == ev.userId }),
|
|
|
|
|
gone.channelId == currentChannelId {
|
|
|
|
|
EventFeedback.shared.play(.channelLeave)
|
|
|
|
|
EventFeedback.shared.speak("\(gone.nickname) left")
|
|
|
|
|
}
|
2026-06-19 02:10:25 +02:00
|
|
|
refreshUsers()
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
syncSelfChannel()
|
|
|
|
|
case .userUpdated:
|
|
|
|
|
refreshUsers()
|
|
|
|
|
syncSelfChannel()
|
|
|
|
|
if let me = users.first(where: { $0.id == selfUserId }) {
|
|
|
|
|
applyServerMuteState(muted: me.serverMuted, deafened: me.serverDeafened)
|
|
|
|
|
}
|
2026-06-19 02:10:25 +02:00
|
|
|
case .textMessage:
|
|
|
|
|
let sender = users.first(where: { $0.id == ev.userId })?.nickname ?? "Unknown"
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
let body = ev.text ?? ""
|
|
|
|
|
let isSelf = ev.userId == selfUserId
|
|
|
|
|
let isPrivate = ev.textScope == .private
|
2026-06-19 02:10:25 +02:00
|
|
|
messages.append(ChatMessage(
|
|
|
|
|
timestamp: Date(timeIntervalSince1970: Double(ev.timestampUnixMs) / 1000),
|
|
|
|
|
senderName: sender,
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
text: body,
|
2026-06-19 02:10:25 +02:00
|
|
|
scope: ev.textScope))
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
EventFeedback.shared.play(isPrivate
|
|
|
|
|
? (isSelf ? .pmSent : .pmRecv)
|
|
|
|
|
: (isSelf ? .channelSent : .channelRecv))
|
|
|
|
|
if !isSelf {
|
|
|
|
|
EventFeedback.shared.speak(isPrivate
|
|
|
|
|
? "Private message from \(sender): \(body)"
|
|
|
|
|
: "\(sender): \(body)")
|
|
|
|
|
}
|
2026-06-19 02:10:25 +02:00
|
|
|
case .talkState:
|
|
|
|
|
let talking = ev.u32a != 0
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
if ev.userId == selfUserId {
|
|
|
|
|
EventFeedback.shared.play(talking ? .vaStart : .vaStop)
|
|
|
|
|
}
|
2026-06-19 02:10:25 +02:00
|
|
|
let who = users.first(where: { $0.id == ev.userId })?.nickname ?? "user \(ev.userId)"
|
|
|
|
|
addActivity(talking ? "\(who) started talking" : "\(who) stopped talking")
|
|
|
|
|
case .streamStarted:
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
// Our own SCREEN_AUDIO stream is live — begin draining the broadcast ring into it,
|
|
|
|
|
// in the stream's effective channel mode (downmix to mono if the channel is mono).
|
|
|
|
|
if ev.userId == selfUserId && ev.streamId == voiceState.screenStreamId {
|
|
|
|
|
let sid = voiceState.screenStreamId
|
|
|
|
|
let (r, cfg) = client.getStreamAudioConfig(userId: selfUserId, streamId: sid)
|
|
|
|
|
let channels: UInt32 = (r == .ok && cfg?.stereo == true) ? 2 : 1
|
|
|
|
|
let c = client
|
|
|
|
|
broadcastPump.beginFeeding(streamChannels: channels) { pcm, samples, ch in
|
|
|
|
|
c.feedPcm(streamId: sid, pcm: pcm, samplesPerChannel: samples, channels: ch)
|
|
|
|
|
}
|
|
|
|
|
addActivity("Sharing screen audio (\(channels == 2 ? "stereo" : "mono"))")
|
|
|
|
|
break
|
|
|
|
|
}
|
2026-06-19 13:46:20 +02:00
|
|
|
// A remote user started a stream — ensure the audio session is active so we can
|
2026-06-20 03:03:34 +02:00
|
|
|
// hear them even if we haven't joined voice ourselves.
|
2026-06-19 13:46:20 +02:00
|
|
|
if ev.userId != selfUserId {
|
|
|
|
|
do {
|
|
|
|
|
try AudioSessionManager.shared.ensureSessionActive()
|
|
|
|
|
} catch {
|
|
|
|
|
addActivity("Audio session activate failed: \(error)")
|
|
|
|
|
}
|
|
|
|
|
}
|
fix(ios): stop miniaudio from clobbering AVAudioSession (stereo->A2DP output death)
The real root cause of "selecting Stereo Mic kills headphone/A2DP output on Join
Voice." Every prior fix worked on the Swift IOSAudioRouter under the false premise
that "miniaudio does NOT touch AVAudioSession on iOS." It does: the core opened
devices via ma_device_init(nullptr, ...), and with a NULL context miniaudio 0.11.25
runs an iOS "hack" that sets the session category by device type, then
ma_context_init__coreaudio calls setCategory()+setActive() on every device open --
capture -> AVAudioSessionCategoryRecord with zero options. That wipes the
.playAndRecord category, the mode, and .allowBluetoothA2DP / .mixWithOthers /
.allowAirPlay that IOSAudioRouter had just configured, killing headphone/A2DP (and
even wired) output. Stereo presets break worst because they rely on the A2DP output
route the wipe removes. TeamTalk avoids this by opening RemoteIO/VPIO AudioUnits
directly and leaving the session entirely to the app.
Fix (core, cross-platform safe): AudioEngine now owns a ma_context built by
make_context_config() with coreaudio.sessionCategory = ma_ios_session_category_none
and noAudioSessionActivate/Deactivate = MA_TRUE, and routes all ma_device_init calls
(playback, capture, loopback) plus enumerate_devices through it. miniaudio no longer
touches AVAudioSession; IOSAudioRouter is the sole owner (the session is already
activated on connect in AppState before any device opens). Context is lazily inited
in start(), reused across restarts, uninited in ~AudioEngine.
Adds TEMP AudioSessionManager.logSessionState() diagnostics (after activate, on route
change, on .streamStarted) to verify on-device that the category stays
PlayAndRecord+allowBluetoothA2DP instead of flipping to Record. Remove once confirmed.
Windows: cmake --build --preset dev clean; ctest --preset dev 21/21.
iOS build + on-device verification pending on Mac.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:34:07 +02:00
|
|
|
AudioSessionManager.shared.logSessionState("stream started (user \(ev.userId))")
|
2026-06-19 02:10:25 +02:00
|
|
|
addActivity("Stream started (user \(ev.userId))")
|
|
|
|
|
case .streamStopped:
|
|
|
|
|
addActivity("Stream stopped (user \(ev.userId))")
|
|
|
|
|
case .joinResult:
|
|
|
|
|
if ev.result == .ok {
|
|
|
|
|
currentChannelId = ev.channelId
|
|
|
|
|
addActivity("Joined channel")
|
|
|
|
|
refreshUsers()
|
|
|
|
|
} else {
|
|
|
|
|
addActivity("Join failed: \(ev.result.description)")
|
|
|
|
|
}
|
|
|
|
|
case .error:
|
|
|
|
|
addActivity("Error: \(ev.text ?? ev.result.description)")
|
|
|
|
|
case .genericResult:
|
|
|
|
|
if ev.result != .ok {
|
|
|
|
|
addActivity("Operation failed: \(ev.result.description)")
|
|
|
|
|
}
|
|
|
|
|
case .accountList:
|
|
|
|
|
accounts = client.listAccounts()
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
case .disconnected:
|
|
|
|
|
// Audible cue only — session teardown is driven elsewhere (AppState / UI).
|
|
|
|
|
EventFeedback.shared.play(ev.result == .ok ? .logout : .connectionLost)
|
|
|
|
|
EventFeedback.shared.speak(ev.result == .ok ? "Disconnected" : "Connection lost")
|
2026-06-19 02:10:25 +02:00
|
|
|
default:
|
|
|
|
|
break
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
private func addActivity(_ text: String) {
|
|
|
|
|
activityLog.append(ActivityEntry(timestamp: Date(), text: text))
|
|
|
|
|
if activityLog.count > 500 { activityLog.removeFirst() }
|
|
|
|
|
}
|
|
|
|
|
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
// MARK: - Self-channel / server-mute sync
|
|
|
|
|
|
|
|
|
|
/// Sync currentChannelId from the self user's channelId in the user list. Mirrors macOS
|
|
|
|
|
/// MainWindowController.swift:461,491,522. The server auto-places every authed user into
|
|
|
|
|
/// the Lobby (channel 1) on connect, but without this sync currentChannelId stays 0 and
|
|
|
|
|
/// the mic button (gated on currentChannelId == 0) stays permanently dimmed.
|
|
|
|
|
private func syncSelfChannel() {
|
|
|
|
|
if let me = users.first(where: { $0.id == selfUserId }) {
|
|
|
|
|
currentChannelId = me.channelId
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Apply server-side mute/deafen state — mirrors macOS MainWindowController.swift:693-700.
|
|
|
|
|
/// iOS was previously ignoring server mute/deafen entirely.
|
|
|
|
|
private func applyServerMuteState(muted: Bool, deafened: Bool) {
|
|
|
|
|
if muted && !voiceState.serverMuted { addActivity("You have been server-muted") }
|
|
|
|
|
if deafened && !voiceState.serverDeafened { addActivity("You have been server-deafened") }
|
|
|
|
|
if !muted && voiceState.serverMuted { addActivity("Server mute cleared") }
|
|
|
|
|
if !deafened && voiceState.serverDeafened { addActivity("Server deafen cleared") }
|
|
|
|
|
voiceState.serverMuted = muted
|
|
|
|
|
voiceState.serverDeafened = deafened
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
// MARK: - Data refresh
|
|
|
|
|
|
|
|
|
|
func refreshChannels() { channels = client.listChannels() }
|
|
|
|
|
func refreshUsers() { users = client.listUsers() }
|
|
|
|
|
func refreshDevices() { devices = client.listDevices(.input) }
|
|
|
|
|
|
|
|
|
|
// MARK: - Voice controls
|
|
|
|
|
|
|
|
|
|
func joinChannel(_ channelId: UInt32, password: String = "") {
|
|
|
|
|
client.joinChannel(channelId, password: password.isEmpty ? nil : password)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func leaveChannel() {
|
|
|
|
|
client.leaveChannel()
|
|
|
|
|
currentChannelId = 0
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func startMicStream() {
|
|
|
|
|
AVAudioApplication.requestRecordPermission { [weak self] granted in
|
|
|
|
|
DispatchQueue.main.async {
|
|
|
|
|
guard let self else { return }
|
|
|
|
|
if granted {
|
|
|
|
|
self.doStartMicStream()
|
|
|
|
|
} else {
|
|
|
|
|
self.addActivity("Microphone permission denied — grant in Settings > Privacy > Microphone")
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
private func doStartMicStream() {
|
|
|
|
|
do {
|
2026-06-19 13:46:20 +02:00
|
|
|
try AudioSessionManager.shared.ensureSessionActive()
|
2026-06-19 02:10:25 +02:00
|
|
|
} catch {
|
|
|
|
|
addActivity("AVAudioSession activate failed: \(error)")
|
|
|
|
|
return
|
|
|
|
|
}
|
2026-06-22 02:38:01 +02:00
|
|
|
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
// Unified iOS path: the core is always external (set at connect via setExternalPlayback +
|
|
|
|
|
// every MIC stream external_feed), and `IOSAudioEngine` drives capture + playback. So the
|
|
|
|
|
// mic stream is just started with external_feed=true and the engine is told the mic is now
|
|
|
|
|
// active — no setExternalPlayback toggle, no audioRestart ordering, no VPIO/miniaudio fork.
|
2026-06-22 02:38:01 +02:00
|
|
|
let desc = StreamDescriptor(kind: .mic, deviceId: voiceState.currentDeviceId, label: "Mic",
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
externalFeed: true)
|
2026-06-19 02:10:25 +02:00
|
|
|
let (result, streamId) = client.startStream(desc)
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
guard result == .ok else {
|
2026-06-19 02:10:25 +02:00
|
|
|
addActivity("Failed to start mic: \(result.description)")
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
return
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
voiceState.micActive = true
|
|
|
|
|
voiceState.localStreamId = streamId
|
|
|
|
|
EventFeedback.shared.play(.voiceOn)
|
|
|
|
|
|
|
|
|
|
let channels = IOSAudioRouter.shared.captureChannels.channelCount
|
|
|
|
|
if channels != 1 {
|
|
|
|
|
client.setCaptureChannels(streamId: streamId, channels: channels)
|
|
|
|
|
}
|
|
|
|
|
// Engage the mic: installs the input tap and (per preset) VPIO, in one engine rebuild.
|
|
|
|
|
IOSAudioEngine.shared.startMic(streamId: streamId, channels: channels)
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func stopMicStream() {
|
fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.
Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.
- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
playback, conditional mic tap, VPIO/AGC; one rebuild() backing
startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
AppState wires external playback + listening at connect, stop at
disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.
No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
2026-06-23 02:45:53 +02:00
|
|
|
// Disengage the mic (removes the tap + VPIO) but keep the engine running for any remaining
|
|
|
|
|
// remote audio. Then stop the core's MIC stream. The core stays external throughout — no
|
|
|
|
|
// setExternalPlayback toggle, no audioRestart.
|
|
|
|
|
IOSAudioEngine.shared.stopMic()
|
2026-06-19 02:10:25 +02:00
|
|
|
if voiceState.localStreamId != 0 {
|
|
|
|
|
client.stopStream(voiceState.localStreamId)
|
|
|
|
|
voiceState.localStreamId = 0
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
EventFeedback.shared.play(.voiceOff)
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
voiceState.micActive = false
|
|
|
|
|
voiceState.level = 0
|
2026-06-19 13:46:20 +02:00
|
|
|
// Do NOT deactivate the AVAudioSession here — the user may still want to hear
|
|
|
|
|
// remote audio (other people talking). The session is deactivated only when
|
|
|
|
|
// disconnecting from the server (see AppState.disconnect / .disconnected event).
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
// MARK: - Screen audio share
|
|
|
|
|
|
|
|
|
|
/// Called when the broadcast extension becomes active. Announces the SCREEN_AUDIO stream;
|
|
|
|
|
/// feeding begins on the resulting `.streamStarted` event (see handleEvent). The actual
|
|
|
|
|
/// system-audio capture happens in the ReplayKit upload extension (a separate process).
|
|
|
|
|
private func startScreenShare() {
|
|
|
|
|
guard voiceState.screenStreamId == 0 else { return }
|
|
|
|
|
guard currentChannelId != 0 else {
|
|
|
|
|
addActivity("Screen audio ignored — join a channel first")
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
let (result, streamId) = client.startStream(
|
|
|
|
|
StreamDescriptor(kind: .screenAudio, deviceId: nil, label: "Screen audio"))
|
|
|
|
|
if result == .ok {
|
|
|
|
|
voiceState.screenStreamId = streamId
|
|
|
|
|
voiceState.screenSharing = true
|
|
|
|
|
addActivity("Screen audio share starting…")
|
|
|
|
|
} else {
|
|
|
|
|
addActivity("Failed to start screen audio: \(result.description)")
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Called when the broadcast ends (or on disconnect). Stops feeding and the stream.
|
|
|
|
|
private func stopScreenShare() {
|
|
|
|
|
broadcastPump.endFeeding()
|
|
|
|
|
if voiceState.screenStreamId != 0 {
|
|
|
|
|
client.stopStream(voiceState.screenStreamId)
|
|
|
|
|
voiceState.screenStreamId = 0
|
|
|
|
|
}
|
|
|
|
|
if voiceState.screenSharing {
|
|
|
|
|
voiceState.screenSharing = false
|
|
|
|
|
addActivity("Stopped sharing screen audio")
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
func setMute(_ muted: Bool, deafened: Bool) {
|
|
|
|
|
client.setSelfMute(micMuted: muted, deafened: deafened)
|
|
|
|
|
voiceState.selfMuted = muted
|
|
|
|
|
voiceState.selfDeafened = deafened
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func setInputMode(_ mode: VoiceCatInputMode) {
|
|
|
|
|
client.setInputMode(mode)
|
|
|
|
|
voiceState.inputMode = mode
|
feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
|
|
|
UserDefaults.standard.set(Int(mode.rawValue), forKey: DefaultsKey.inputMode)
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func setVadThreshold(_ threshold: Float) {
|
|
|
|
|
client.setVadThreshold(threshold)
|
|
|
|
|
voiceState.vadThreshold = threshold
|
feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver
Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were
applied to the core + UI but never saved, so every relaunch reset to VAD
defaults. Each client now persists them and re-applies on connect:
- iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes)
- macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings
(settings window also restores the VAD slider from the stored threshold)
- Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json,
mirrors FeedbackSettings) loaded/applied in MainForm
Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in
on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings,
and a 0-300% (default 100%) mic-volume slider on all three clients.
Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0),
so channel messages went nowhere; now passes session.currentChannelId.
Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press
.contextMenu only (invisible to VoiceOver); UserRow now also exposes the same
buttons via .accessibilityActions (no visual change).
Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes,
reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build
SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built
(WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
|
|
|
UserDefaults.standard.set(threshold, forKey: DefaultsKey.vadThreshold)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func setInputGain(_ gain: Float) {
|
|
|
|
|
client.setInputGain(gain)
|
|
|
|
|
voiceState.inputGain = gain
|
|
|
|
|
UserDefaults.standard.set(gain, forKey: DefaultsKey.inputGain)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - Persisted input settings
|
|
|
|
|
|
|
|
|
|
private enum DefaultsKey {
|
|
|
|
|
static let inputMode = "voice.inputMode"
|
|
|
|
|
static let vadThreshold = "voice.vadThreshold"
|
|
|
|
|
static let inputGain = "voice.inputGain"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Restore the saved input mode / VAD threshold / mic gain and push them into the core so a
|
|
|
|
|
/// relaunch keeps the user's transmission settings instead of resetting to VAD defaults.
|
|
|
|
|
private func loadAndApplyVoiceSettings() {
|
|
|
|
|
let d = UserDefaults.standard
|
|
|
|
|
if d.object(forKey: DefaultsKey.inputMode) != nil {
|
|
|
|
|
let raw = UInt32(d.integer(forKey: DefaultsKey.inputMode))
|
|
|
|
|
voiceState.inputMode = VoiceCatInputMode(rawValue: raw) ?? .voiceActivation
|
|
|
|
|
}
|
|
|
|
|
if d.object(forKey: DefaultsKey.vadThreshold) != nil {
|
|
|
|
|
voiceState.vadThreshold = d.float(forKey: DefaultsKey.vadThreshold)
|
|
|
|
|
}
|
|
|
|
|
if d.object(forKey: DefaultsKey.inputGain) != nil {
|
|
|
|
|
voiceState.inputGain = d.float(forKey: DefaultsKey.inputGain)
|
|
|
|
|
}
|
|
|
|
|
client.setInputMode(voiceState.inputMode)
|
|
|
|
|
client.setVadThreshold(voiceState.vadThreshold)
|
|
|
|
|
client.setInputGain(voiceState.inputGain)
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
private var pttEngaged = false
|
2026-06-19 02:10:25 +02:00
|
|
|
func setPushToTalk(_ active: Bool) {
|
|
|
|
|
client.setPushToTalk(active)
|
feat(clients): event sound effects + optional text-to-speech
Add audible cues and optional spoken announcements for session events
(join/leave, channel + PM sent/recv, login, logout/connection-lost,
mic on/off, voice-activity, PTT) across all three clients, driven off
the shared C ABI vc_event stream so the mapping stays consistent.
TTS is off by default; when enabled it announces events and reads
message/PM bodies aloud. Master toggles + a sound-volume slider; the
per-utterance voice-activity and PTT cues default off. WAVs ship from
assets/sounds/.
Windows (built + verified): new VoiceCat.App/Notifications/ layer
(FeedbackSettings -> %AppData%\VoiceCat\feedback.json, SoundPlayerPool
via System.Media.SoundPlayer, SpeechAnnouncer via Prismatoid 0.3.0,
EventFeedback dispatcher); MainForm hooks; NotificationSettingsForm
under Settings > Notifications; csproj adds the Prismatoid PackageRef
and copies the WAVs into sounds\.
macOS + iOS (written, not yet built -- needs a Mac): shared
VoiceCatCore/Feedback/ (SoundEvent, EventFeedback = AVAudioPlayer pool
+ native AVSpeechSynthesizer, FeedbackSettings over UserDefaults); WAVs
bundled via Package.swift resources (.process). Hooks in SessionState/
AppState (iOS) and MainWindowController (macOS); settings UI in
SettingsView (iOS) and SettingsWindowController (macOS).
No core/server code touched; ctest --preset dev unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:20:11 +02:00
|
|
|
// Play the PTT cue only on the press transition (the gesture fires repeatedly while held).
|
|
|
|
|
if active && !pttEngaged { EventFeedback.shared.play(.ptt) }
|
|
|
|
|
pttEngaged = active
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - Text
|
|
|
|
|
|
|
|
|
|
func sendText(_ text: String, scope: VoiceCatTextScope, targetId: UInt32 = 0) {
|
|
|
|
|
client.sendText(scope: scope, targetId: targetId, text: text)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - Admin
|
|
|
|
|
|
|
|
|
|
func kickUser(_ userId: UInt32, reason: String) {
|
|
|
|
|
client.kickUser(userId, reason: reason.isEmpty ? nil : reason)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func banUser(_ userId: UInt32, reason: String, expiresUnixMs: UInt64) {
|
|
|
|
|
client.banUser(userId, reason: reason.isEmpty ? nil : reason, expiresUnixMs: expiresUnixMs)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func moveUser(_ userId: UInt32, toChannel channelId: UInt32) {
|
|
|
|
|
client.moveUser(userId, toChannel: channelId)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func setPermissions(_ userId: UInt32, perms: Permissions) {
|
|
|
|
|
client.setPermission(userId, perms: perms)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func setServerMute(_ userId: UInt32, muted: Bool, deafened: Bool) {
|
|
|
|
|
client.setServerMute(userId, muted: muted, deafened: deafened)
|
|
|
|
|
}
|
|
|
|
|
|
feat(clients): expose all channel codec params + guest nickname everywhere
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.
- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
(backward-compatible Codable), shown in Guest mode, wired into the guest auth
path -- guests could not set a display name on either before (only Windows)
Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
2026-06-21 04:03:50 +02:00
|
|
|
func createChannel(_ info: ChannelEdit) {
|
2026-06-19 02:10:25 +02:00
|
|
|
client.createChannel(info)
|
|
|
|
|
}
|
|
|
|
|
|
feat(clients): expose all channel codec params + guest nickname everywhere
Channel create/edit UIs only surfaced a subset of the core's vc_audio_config,
and DRED was exposed nowhere. While adding it, found a latent ABI mismatch:
both Swift AudioConfig and the C# VcAudioConfigNative blittable struct were one
int short of the native vc_audio_config (missing the trailing `dred`), so native
read past the managed struct in vc_create_channel/vc_edit_channel.
- core marshaling: thread `dred` through Swift (Models/Marshaling/toNative) and
C# (Structs/Models/Marshaling/VoiceCatClient) -- fixes the ABI gap + enables it
- windows: add the one missing DRED checkbox to ChannelEditDialog
- macos: ChannelEditSheet now exposes application, sample rate, packet loss,
complexity, and DRED (was stereo/bitrate/frame/FEC/DTX only)
- ios: rebuild ChannelEditView into a full create+edit form (all params); add
SessionState.editChannel + an admin Edit swipe action (iOS had no edit UI)
- guest nickname: add a dedicated `nickname` to SavedServer on macOS+iOS
(backward-compatible Codable), shown in Guest mode, wired into the guest auth
path -- guests could not set a display name on either before (only Windows)
Verified: macOS + iOS (sim, arm64) xcodebuild BUILD SUCCEEDED; core ctest 22/23
(only external_pcm aborts on a pre-existing shutdown mutex race; no C++ changed).
2026-06-21 04:03:50 +02:00
|
|
|
func editChannel(_ info: ChannelEdit) {
|
|
|
|
|
client.editChannel(info)
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
func deleteChannel(_ channelId: UInt32) {
|
|
|
|
|
client.deleteChannel(channelId)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func fetchAccountList() {
|
|
|
|
|
client.requestAccountList()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func createAccount(username: String, password: String) {
|
|
|
|
|
client.createAccount(username, password: password)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func deleteAccount(username: String) {
|
|
|
|
|
client.deleteAccount(username)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func resetPassword(username: String, newPassword: String) {
|
|
|
|
|
client.resetPassword(username, newPassword: newPassword)
|
|
|
|
|
}
|
|
|
|
|
}
|