2026-06-19 02:10:25 +02:00
|
|
|
import SwiftUI
|
|
|
|
|
import VoiceCatCore
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
import ReplayKit
|
2026-06-19 02:10:25 +02:00
|
|
|
|
|
|
|
|
struct VoiceControlsView: View {
|
|
|
|
|
@Bindable var session: SessionState
|
2026-06-21 15:47:20 +02:00
|
|
|
@StateObject private var router = IOSAudioRouter.shared
|
2026-06-19 02:10:25 +02:00
|
|
|
|
|
|
|
|
var body: some View {
|
|
|
|
|
HStack(spacing: 20) {
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
// Join/Leave Voice button (mirrors macOS micToggleButton)
|
2026-06-19 02:10:25 +02:00
|
|
|
if session.voiceState.inputMode == .pushToTalk {
|
|
|
|
|
PTTButton(session: session)
|
|
|
|
|
} else {
|
|
|
|
|
Button {
|
|
|
|
|
if session.voiceState.micActive {
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
session.leaveVoice()
|
2026-06-19 02:10:25 +02:00
|
|
|
} else {
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
session.joinVoice()
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
} label: {
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
Text(session.voiceState.micActive ? "Leave Voice" : "Join Voice")
|
|
|
|
|
.font(.body.weight(.semibold))
|
|
|
|
|
.frame(minWidth: 110)
|
|
|
|
|
.padding(.vertical, 8)
|
|
|
|
|
.padding(.horizontal, 12)
|
|
|
|
|
.background(session.voiceState.micActive ? Color.green.opacity(0.2) : Color.accentColor.opacity(0.15), in: RoundedRectangle(cornerRadius: 8))
|
|
|
|
|
.foregroundStyle(session.voiceState.micActive ? .green : .accentColor)
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
.disabled(session.currentChannelId == 0)
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
.accessibilityLabel(session.voiceState.micActive ? "Leave Voice" : "Join Voice")
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Level meter
|
|
|
|
|
LevelMeterView(level: session.voiceState.level)
|
|
|
|
|
.frame(width: 80, height: 8)
|
|
|
|
|
.accessibilityHidden(true)
|
|
|
|
|
|
|
|
|
|
Spacer()
|
|
|
|
|
|
2026-06-21 15:47:20 +02:00
|
|
|
// Speaker output toggle — force the built-in speaker instead of the earpiece when
|
|
|
|
|
// no headphones/BT are connected. Mirrors the persisted Settings ▸ Audio toggle.
|
|
|
|
|
Button {
|
|
|
|
|
router.setForceSpeaker(!router.forceSpeaker)
|
|
|
|
|
} label: {
|
|
|
|
|
Image(systemName: router.forceSpeaker ? "speaker.wave.2.fill" : "speaker.fill")
|
|
|
|
|
.font(.title3)
|
|
|
|
|
.foregroundStyle(router.forceSpeaker ? Color.accentColor : .primary)
|
|
|
|
|
}
|
|
|
|
|
.accessibilityLabel(router.forceSpeaker
|
|
|
|
|
? "Speaker on — turn off to use the earpiece"
|
|
|
|
|
: "Speaker off — turn on for speakerphone")
|
|
|
|
|
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
// Self mute (disabled when not in voice)
|
2026-06-19 02:10:25 +02:00
|
|
|
Button {
|
|
|
|
|
session.setMute(!session.voiceState.selfMuted, deafened: session.voiceState.selfDeafened)
|
|
|
|
|
} label: {
|
|
|
|
|
Image(systemName: session.voiceState.selfMuted ? "mic.slash" : "mic")
|
|
|
|
|
.font(.title3)
|
|
|
|
|
.foregroundStyle(session.voiceState.selfMuted ? .red : .primary)
|
|
|
|
|
}
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
.disabled(!session.voiceState.micActive)
|
2026-06-19 02:10:25 +02:00
|
|
|
.accessibilityLabel(session.voiceState.selfMuted ? "Unmute microphone" : "Mute microphone")
|
|
|
|
|
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
// Self deafen (disabled when not in voice)
|
2026-06-19 02:10:25 +02:00
|
|
|
Button {
|
|
|
|
|
session.setMute(session.voiceState.selfMuted, deafened: !session.voiceState.selfDeafened)
|
|
|
|
|
} label: {
|
|
|
|
|
Image(systemName: session.voiceState.selfDeafened ? "headphones.slash" : "headphones")
|
|
|
|
|
.font(.title3)
|
|
|
|
|
.foregroundStyle(session.voiceState.selfDeafened ? .red : .primary)
|
|
|
|
|
}
|
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture
Three iOS client problems fixed plus a new core stereo-mic capture ABI:
1. Channel-id sync bug (mic button permanently dimmed): SessionState never
synced currentChannelId from the self user's channelId on connect, so the
mic button (gated on currentChannelId == 0) stayed dimmed. Added
syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522);
called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult.
Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState.
2. Join/Leave Voice button: replaced icon-only mic toggle with explicit
text button (parity with macOS). Mute/deafen disable when not in voice.
3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input
port selection, built-in mic orientation/polar patterns, Bluetooth
HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay,
UserDefaults persistence. AudioSessionManager delegates to it.
4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels()
lets the core open the mic device in stereo (2-ch interleaved). LocalStream
gains capture_channels; ensure_audio_running reads it; audio_engine.cpp
capture_accum_ + on_capture updated to channel-aware accumulation. Test
test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper
VoiceCatClient.setCaptureChannels.
5. Settings UI rework: AVAudioSession-derived input/output tree replaces
miniaudio device picker.
6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj).
swift-tools-version 6.0 with swiftLanguageModes .v5.
Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md
updated; stale 'vc_audio_suspend/resume deferred' claims corrected.
Verified: ctest --preset dev 21/21 green; swift test 6/6 green;
xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
|
|
|
.disabled(!session.voiceState.micActive)
|
2026-06-19 02:10:25 +02:00
|
|
|
.accessibilityLabel(session.voiceState.selfDeafened ? "Undeafen" : "Deafen")
|
|
|
|
|
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
// Share screen audio (ReplayKit system broadcast picker). The picker launches the
|
|
|
|
|
// broadcast upload extension; the host's BroadcastAudioPump then owns + feeds the
|
|
|
|
|
// SCREEN_AUDIO stream. Tinted while sharing.
|
|
|
|
|
BroadcastPickerButton(isSharing: session.voiceState.screenSharing)
|
|
|
|
|
.frame(width: 32, height: 32)
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
// Disconnect
|
|
|
|
|
Button(role: .destructive) {
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
session.leaveVoice()
|
2026-06-19 02:10:25 +02:00
|
|
|
session.client.disconnect()
|
|
|
|
|
} label: {
|
|
|
|
|
Image(systemName: "phone.down.fill")
|
|
|
|
|
.font(.title3)
|
|
|
|
|
.foregroundStyle(.red)
|
|
|
|
|
}
|
|
|
|
|
.accessibilityLabel("Disconnect from server")
|
|
|
|
|
}
|
|
|
|
|
.padding(.horizontal, 16)
|
|
|
|
|
.padding(.vertical, 10)
|
|
|
|
|
.background(.bar)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// MARK: - PTT Button (DragGesture instead of NSEvent on iOS)
|
|
|
|
|
|
|
|
|
|
private struct PTTButton: View {
|
|
|
|
|
@Bindable var session: SessionState
|
|
|
|
|
@GestureState private var isPressing = false
|
|
|
|
|
|
|
|
|
|
var body: some View {
|
|
|
|
|
Circle()
|
|
|
|
|
.fill(isPressing ? Color.blue : Color(.systemGray4))
|
|
|
|
|
.frame(width: 44, height: 44)
|
|
|
|
|
.overlay {
|
|
|
|
|
Image(systemName: "mic.fill")
|
|
|
|
|
.foregroundStyle(isPressing ? .white : .primary)
|
|
|
|
|
}
|
|
|
|
|
.gesture(
|
|
|
|
|
DragGesture(minimumDistance: 0)
|
|
|
|
|
.updating($isPressing) { _, state, _ in state = true }
|
|
|
|
|
.onChanged { _ in
|
|
|
|
|
if !isPressing { return }
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
if !session.voiceState.micActive { session.joinVoice() }
|
2026-06-19 02:10:25 +02:00
|
|
|
session.setPushToTalk(true)
|
|
|
|
|
UIImpactFeedbackGenerator(style: .medium).impactOccurred()
|
|
|
|
|
}
|
|
|
|
|
.onEnded { _ in
|
|
|
|
|
session.setPushToTalk(false)
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
session.leaveVoice()
|
2026-06-19 02:10:25 +02:00
|
|
|
}
|
|
|
|
|
)
|
|
|
|
|
.accessibilityLabel("Push to talk, hold to transmit")
|
|
|
|
|
.accessibilityAddTraits(.isButton)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
// MARK: - Broadcast picker
|
|
|
|
|
|
|
|
|
|
/// Wraps `RPSystemBroadcastPickerView` (which contains its own button) and points it at our
|
|
|
|
|
/// broadcast upload extension. Tapping it shows the system broadcast picker; the user starts the
|
|
|
|
|
/// broadcast and our extension launches.
|
|
|
|
|
private struct BroadcastPickerButton: UIViewRepresentable {
|
|
|
|
|
let isSharing: Bool
|
|
|
|
|
|
|
|
|
|
func makeUIView(context: Context) -> RPSystemBroadcastPickerView {
|
|
|
|
|
let picker = RPSystemBroadcastPickerView(frame: CGRect(x: 0, y: 0, width: 32, height: 32))
|
|
|
|
|
picker.preferredExtension = "cat.voice.VoiceCatiOS.broadcast"
|
|
|
|
|
picker.showsMicrophoneButton = false
|
|
|
|
|
return picker
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func updateUIView(_ uiView: RPSystemBroadcastPickerView, context: Context) {
|
|
|
|
|
uiView.tintColor = isSharing ? .systemGreen : .label
|
2026-06-21 03:02:58 +02:00
|
|
|
// RPSystemBroadcastPickerView owns an inner UIButton that VoiceOver focuses; its default
|
|
|
|
|
// label is the picker's image name ("module icon"). Label the inner button directly —
|
|
|
|
|
// a SwiftUI .accessibilityLabel on the representable doesn't reach it.
|
|
|
|
|
let label = isSharing ? "Stop sharing screen audio" : "Share screen audio"
|
|
|
|
|
for case let button as UIButton in uiView.subviews {
|
|
|
|
|
button.accessibilityLabel = label
|
|
|
|
|
}
|
feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the
existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No
C++/protocol/codec changes -- the core was already ready (the Windows-only
loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits
for fed PCM). Audio only; video is dropped.
macOS (in-process):
- ScreenAudioCapture.swift drives an audio-only SCStream
(excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's
mono/stereo mode, and calls feedPcm. Capture starts on the self
.streamStarted event (effective config known then). Wired into
MainWindowController.screenAudioClicked().
iOS (forward-to-host, single session):
- VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes
.audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and
writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does
not link libvoicecat.
- Host BroadcastAudioPump drains the ring (reacting to the extension's
Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing
to mono when the channel is mono. Screen audio appears as a second stream
of the same user; no credentials persisted. UI is RPSystemBroadcastPicker
View in VoiceControlsView. Removes the speculative BroadcastCredentials.
Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
2026-06-21 00:14:31 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-19 02:10:25 +02:00
|
|
|
// MARK: - Level Meter
|
|
|
|
|
|
|
|
|
|
private struct LevelMeterView: View {
|
|
|
|
|
let level: Float
|
|
|
|
|
|
|
|
|
|
var body: some View {
|
|
|
|
|
GeometryReader { geo in
|
|
|
|
|
ZStack(alignment: .leading) {
|
|
|
|
|
RoundedRectangle(cornerRadius: 4)
|
|
|
|
|
.fill(Color(.systemGray5))
|
|
|
|
|
RoundedRectangle(cornerRadius: 4)
|
|
|
|
|
.fill(levelColor)
|
|
|
|
|
.frame(width: geo.size.width * CGFloat(min(level * 10, 1.0)))
|
|
|
|
|
.animation(.linear(duration: 0.05), value: level)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
private var levelColor: Color {
|
|
|
|
|
if level > 0.15 { return .orange }
|
|
|
|
|
if level > 0.05 { return .green }
|
|
|
|
|
return .green.opacity(0.5)
|
|
|
|
|
}
|
|
|
|
|
}
|