Add managed UIKit iOS client
.NET port / test (macos-latest) (push) Canceled after 0s
.NET port / test (ubuntu-24.04) (push) Canceled after 0s
.NET port / test (windows-latest) (push) Canceled after 0s
.NET port / apple-client (push) Canceled after 0s
.NET port / cpp-conformance (push) Canceled after 0s

This commit is contained in:
2026-09-19 15:43:37 +02:00
parent d0a72176ba
commit c6715028c1
41 changed files with 1110 additions and 40 deletions
+23
View File
@@ -0,0 +1,23 @@
# iOS broadcast-audio ring format
The ReplayKit extension and host app exchange audio through `broadcast_audio.ring` in the
`group.me.iamtalon.voicecat` App Group. Version 1 is a 64-byte little-endian header followed by
96,000 signed 16-bit PCM samples (one second of stereo at 48 kHz).
| Offset | Type | Meaning |
|---:|---|---|
| 0 | `uint32` | Magic `0x56434252` (`VCBR`) |
| 4 | `uint32` | Version, currently `1` |
| 8 | `uint32` | Channel count; active writers use `2` |
| 12 | `uint32` | Sample rate; active writers use `48000` |
| 16 | `uint32` | Active flag (`0` or `1`) |
| 20 | 4 bytes | Reserved, zero |
| 24 | `uint64` | Monotonic producer/write sample index |
| 32 | `uint64` | Monotonic consumer/read sample index |
| 40 | 24 bytes | Reserved, zero |
The payload starts at byte 64 and contains interleaved little-endian `int16` PCM. The aligned
indices are single-producer/single-consumer counters; producer and consumer publish their owned
index with a release barrier and read the other index with an acquire barrier. A writer drops a
whole input chunk when it cannot fit. Changing this layout requires a new version and compatible
readers; offsets in version 1 must never be repurposed.
+16 -2
View File
@@ -31,7 +31,7 @@ and *how to drive the binaries by hand*.
| Windows client (C# / WinForms) | [§7](#7-windows-client-c--winforms) | `dotnet build clients/windows/VoiceCat.slnx` |
| Managed macOS client (C# AppKit) | [§8](#8-macos-client-appkit) | `dotnet build clients/apple/dotnet/VoiceCat.Apple.slnx` |
| Swift macOS migration oracle | [§8](#8-macos-client-appkit) | `scripts/build-macos-client.sh` |
| iOS client (SwiftUI / simulator) | [§9](#9-ios-client-swiftui) | `scripts/build-ios-client.sh` |
| Managed iOS client (C# UIKit / simulator) | [§9](#9-ios-client-uikit) | `./dotnet/build-native-ios.sh && dotnet build clients/apple/dotnet/VoiceCat.iOS/VoiceCat.iOS.csproj` |
| Launch iOS app on simulator | [§9](#9-ios-client-swiftui) | `scripts/run-ios-simulator.sh` |
| Swift core + tests | [§8](#8-macos-client-appkit) | `cd clients/apple && swift test` |
@@ -419,7 +419,21 @@ Or use the per-artifact script which builds and stages to `dist/macos-client/`:
scripts/build-macos-client.sh
```
## 9. iOS client (SwiftUI)
## 9. iOS client (UIKit)
The replacement iOS client is a native C# UIKit app at
`clients/apple/dotnet/VoiceCat.iOS`. Build its static media shim and simulator bundle with:
```bash
./dotnet/build-native-ios.sh
dotnet restore clients/apple/dotnet/VoiceCat.Apple.slnx
dotnet build clients/apple/dotnet/VoiceCat.iOS/VoiceCat.iOS.csproj \
-c Debug -r iossimulator-arm64 --no-restore
```
The build invokes Xcode for the retained Swift ReplayKit extension and embeds the resulting
appex. The older SwiftUI application below remains the migration oracle during parity and
physical-device accessibility testing.
The iOS client is an Xcode project (`clients/apple/iOS/VoiceCatiOS.xcodeproj`) that
links `libvoicecat` via the same `VoiceCatCore` Swift Package as the macOS client.
+6 -1
View File
@@ -497,7 +497,7 @@ The Swift AppKit code maps almost line-for-line:
NativeAOT for macOS app bundles is supported but adds a step. Nothing blocking; just not
free.
### 8.3 iOS — the SwiftUI gap
### 8.3 iOS — the SwiftUI gap (decision implemented 2026-09-19)
This is the only client with no mechanical path, because **SwiftUI has no C# equivalent.**
Three options:
@@ -512,6 +512,11 @@ Three options:
with the accessibility commitments already made twice in the docs. Budget it as the largest
single client task.
The decision is now implemented in `clients/apple/dotnet/VoiceCat.iOS`: a `net10.0-ios`
UIKit host uses the managed core, statically links the Opus/RNNoise shim through
`__Internal`, and embeds the retained Swift ReplayKit extension. The Swift app remains the
migration oracle until the UIKit client completes its device, live-call and VoiceOver gates.
What ports cleanly regardless:
- `IOSAudioRouter.swift` (31 KB) — `AVAudioSession` is fully bound. `SetPreferredDataSource`,
`SetPreferredPolarPattern`, `AllowBluetoothA2DP`, `MeasurementMode` all exist in C#. The
+5 -4
View File
@@ -85,13 +85,14 @@ exists from M1 so the protocol can be exercised long before any GUI.
deployment target.
- AVAudioSession not needed on macOS (CoreAudio via the core directly).
**iOS (Swift/SwiftUI) — pending:**
- SwiftUI app consuming the same `VoiceCatCore` package.
**iOS (.NET/UIKit) — replacement in progress 2026-09-19:**
- Native C# UIKit app consuming `VoiceCat.Core`; UIKit was chosen over MAUI for direct audio
lifecycle control and the strongest VoiceOver surface.
- ~~AVAudioSession, mic permission, foreground voice.~~ ✓ Done — `IOSAudioRouter` drives
all iOS audio routing (input ports, orientation/polar patterns, HFP/A2DP, Standard/Raw
mic mode, stereo capture), `vc_audio_suspend`/`vc_audio_resume` for interruptions.
- ReplayKit broadcast extension for `SCREEN_AUDIO` — feeds `CMSampleBuffer` audio via
`vc_stream_feed_pcm` (see architecture.md §4).
- The small Swift ReplayKit broadcast extension remains and writes its versioned App Group
PCM ring; the C# host drains it into `SCREEN_AUDIO`. See `broadcast-ring-format.md`.
**Exit:** non-technical user installs a client, saves a server, and joins.
+4 -4
View File
@@ -37,16 +37,16 @@ explicit resampling (speexdsp/libsamplerate) is only needed when a device can't
## 2. Clients
### macOS / iOS — Swift
### Apple clients — .NET native UI with a Swift ReplayKit exception
| Concern | Choice | Notes |
|---------|--------|-------|
| Language | **Swift 5.9+** | Direct **Swift↔C interop** — the C ABI (`voicecat.h`) is imported as a Clang module (`import VoiceCatC`) via a module map in the XCFramework headers; no manual struct/function redeclaration (unlike the C# P/Invoke layer). A Swift wrapper (`VoiceCatCore` package) provides Swift-idiomatic types on top. |
| UI — macOS | **AppKit** | Chosen over SwiftUI for the most mature, granular **VoiceOver** accessibility story (per-control `accessibilityLabel`/`accessibilityHelp`/`accessibilityRole`, `NSAccessibility.post(.announcement)` for live announcements) — the same rationale that drove the Windows client to WinForms over WinUI 3 for screen-reader (NVDA/JAWS/Narrator) UIA support (resolved decision in `docs/roadmap.md`). macOS 14 (Sonoma) deployment target. |
| UI — iOS | **SwiftUI** | iOS has a narrower control surface (no channel-tree moderation, etc.) and SwiftUI's VoiceOver support is sufficient; revisit if gaps emerge. iOS 18.0 deployment target (unlocks newest AVAudioSession APIs: stereo capture, polar patterns, data sources). |
| UI — iOS | **C# / UIKit (`net10.0-ios`)** | Native UIKit keeps direct lifecycle/audio control and predictable VoiceOver semantics. MAUI was rejected because a cross-platform abstraction provides no benefit for this platform-specific client. iOS 18.0 deployment target. |
| Shared core | **VoiceCatCore** Swift Package | One Swift library wrapping the C ABI, consumed by both the macOS AppKit app and the iOS SwiftUI app. Mirrors the C# `VoiceCat.Interop` layer. Events delivered on `@MainActor` via a coalesced `DispatchQueue.main` drain (the Swift analog of C#'s `Channel<VoiceCatEvent>` + 30ms WinForms Timer pump). |
| Audio session (iOS) | **AVAudioSession** + **IOSAudioRouter** | App owns category `.playAndRecord`, mic permission, interruption/route-change handling; calls `vc_audio_suspend`/`vc_audio_resume`/`vc_audio_restart` (implemented) on the core. All iOS audio routing (input port selection, mic orientation/polar patterns, HFP vs A2DP, measurement/raw mode, stereo capture via `.stereo` polar pattern + `setPreferredInput` + `setInputDataSource`) is driven from Swift via `AVAudioSession` *before* the core (miniaudio) opens its device — miniaudio does NOT touch `AVAudioSession` on iOS. The `IOSAudioRouter` singleton owns this; the core is told the channel count via `vc_set_capture_channels`. When settings change mid-session, devices are suspended (`vc_audio_suspend`), the session is reconfigured, and devices are restarted (`vc_audio_restart`) to pick up the new route. macOS uses CoreAudio via the core directly. |
| Packaging | Swift Package + Xcode project | Core shipped as an **XCFramework** binary target — a fat static library (`libvoicecat-fat.a`) bundling `libvoicecat.a` + all vcpkg static deps (protobuf/mbedtls/sodium/opus/sqlite3/spdlog/asio), so the Swift Package links a single self-contained `.a` per slice. macOS slice validated; iOS device + sim slices are scaffolding. |
| Audio session (iOS) | **AVAudioSession + AVAudioEngine from C#** | The UIKit host owns category, permission, interruptions, routes, VPIO capture and planar playback. Converted PCM crosses bounded managed rings; callbacks allocate no managed objects, lock, or block. |
| Packaging | .NET Apple workloads + Xcode appex | The managed iOS host statically links merged Opus/RNNoise archives. MSBuild invokes Xcode to build/embed the small Swift ReplayKit extension, which communicates through the versioned App Group PCM ring. |
| Future | CallKit / PushKit | For background VoIP + incoming-call UX on iOS. Post-v1. |
### Windows — C# (shipped in M4, 2026-06-17)