feat(apple): screen-audio sharing -- macOS ScreenCaptureKit, iOS ReplayKit
Implement system/desktop audio sharing on the Apple clients, feeding the existing SCREEN_AUDIO Opus -> AEAD -> UDP path via vc_stream_feed_pcm. No C++/protocol/codec changes -- the core was already ready (the Windows-only loopback is #ifdef VOICECAT_HAS_LOOPBACK; off Windows the stream just waits for fed PCM). Audio only; video is dropped. macOS (in-process): - ScreenAudioCapture.swift drives an audio-only SCStream (excludesCurrentProcessAudio), converts Float32 -> int16 in the channel's mono/stereo mode, and calls feedPcm. Capture starts on the self .streamStarted event (effective config known then). Wired into MainWindowController.screenAudioClicked(). iOS (forward-to-host, single session): - VoiceCatBroadcast: a ReplayKit Broadcast Upload Extension consumes .audioApp only, resamples to 48kHz int16 stereo (AVAudioConverter), and writes a shared App Group SPSC ring (BroadcastAudioRing.swift). It does not link libvoicecat. - Host BroadcastAudioPump drains the ring (reacting to the extension's Darwin notifications) and feeds the SCREEN_AUDIO stream it owns, downmixing to mono when the channel is mono. Screen audio appears as a second stream of the same user; no credentials persisted. UI is RPSystemBroadcastPicker View in VoiceControlsView. Removes the speculative BroadcastCredentials. Docs: voice.md s9, CLAUDE.md status, PROGRESS.md.
This commit is contained in:
@@ -1,5 +1,6 @@
|
||||
import SwiftUI
|
||||
import VoiceCatCore
|
||||
import ReplayKit
|
||||
|
||||
struct VoiceControlsView: View {
|
||||
@Bindable var session: SessionState
|
||||
@@ -58,6 +59,14 @@ struct VoiceControlsView: View {
|
||||
.disabled(!session.voiceState.micActive)
|
||||
.accessibilityLabel(session.voiceState.selfDeafened ? "Undeafen" : "Deafen")
|
||||
|
||||
// Share screen audio (ReplayKit system broadcast picker). The picker launches the
|
||||
// broadcast upload extension; the host's BroadcastAudioPump then owns + feeds the
|
||||
// SCREEN_AUDIO stream. Tinted while sharing.
|
||||
BroadcastPickerButton(isSharing: session.voiceState.screenSharing)
|
||||
.frame(width: 32, height: 32)
|
||||
.accessibilityLabel(session.voiceState.screenSharing
|
||||
? "Stop sharing screen audio" : "Share screen audio")
|
||||
|
||||
// Disconnect
|
||||
Button(role: .destructive) {
|
||||
session.stopMicStream()
|
||||
@@ -108,6 +117,26 @@ private struct PTTButton: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Broadcast picker
|
||||
|
||||
/// Wraps `RPSystemBroadcastPickerView` (which contains its own button) and points it at our
|
||||
/// broadcast upload extension. Tapping it shows the system broadcast picker; the user starts the
|
||||
/// broadcast and our extension launches.
|
||||
private struct BroadcastPickerButton: UIViewRepresentable {
|
||||
let isSharing: Bool
|
||||
|
||||
func makeUIView(context: Context) -> RPSystemBroadcastPickerView {
|
||||
let picker = RPSystemBroadcastPickerView(frame: CGRect(x: 0, y: 0, width: 32, height: 32))
|
||||
picker.preferredExtension = "cat.voice.VoiceCatiOS.broadcast"
|
||||
picker.showsMicrophoneButton = false
|
||||
return picker
|
||||
}
|
||||
|
||||
func updateUIView(_ uiView: RPSystemBroadcastPickerView, context: Context) {
|
||||
uiView.tintColor = isSharing ? .systemGreen : .label
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Level Meter
|
||||
|
||||
private struct LevelMeterView: View {
|
||||
|
||||
Reference in New Issue
Block a user