2026-06-15 21:09:09 +02:00
|
|
|
|
# PROGRESS — VoiceCat
|
|
|
|
|
|
|
|
|
|
|
|
Living status. **Update this file in the same commit as your work** so the next agent picks
|
|
|
|
|
|
up instantly. Newest status at the top.
|
|
|
|
|
|
|
|
|
|
|
|
- **Date convention:** ISO (YYYY-MM-DD).
|
|
|
|
|
|
- Statuses: `[ ]` not started · `[~]` in progress · `[x]` done.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## ▶ Where we left off / next action
|
|
|
|
|
|
|
2026-09-19 19:33:10 +02:00
|
|
|
|
- **In progress (2026-09-19): managed iOS/UIKit replacement at feature parity.** Chose native UIKit over MAUI
|
2026-09-19 15:43:37 +02:00
|
|
|
|
to preserve direct AVAudioSession/AVAudioEngine control and native VoiceOver semantics.
|
|
|
|
|
|
Added the .NET 10 iOS application, App Group profile/TOFU/Keychain migration, saved-server
|
|
|
|
|
|
and connected channel/chat/settings flows, managed client event/reconnect handling, and
|
|
|
|
|
|
foreground microphone/playback through an allocation-free callback and bounded PCM rings.
|
|
|
|
|
|
The Opus/RNNoise shim now cross-compiles as merged static device and simulator archives and
|
|
|
|
|
|
binds through `__Internal`. The Swift ReplayKit upload extension is retained, its ring ABI
|
|
|
|
|
|
is documented/versioned, and the managed pump drains it into a screen-audio stream; MSBuild
|
2026-09-19 19:33:10 +02:00
|
|
|
|
builds and embeds the appex. Added hierarchical protected-channel management, full channel
|
|
|
|
|
|
codec editing, roster moderation and per-user receive tuning, permissions, account
|
|
|
|
|
|
administration, persistent VAD/PTT/always-on voice controls, event feedback, advanced iOS
|
|
|
|
|
|
audio routing, reconnect restoration, and an always-visible voice bar. iOS 27 uses a small
|
|
|
|
|
|
dynamically loaded ScreenCaptureKit audio bridge; iOS 18–26 retain ReplayKit, with both producers
|
|
|
|
|
|
feeding the frozen ring ABI. Added physical-device build/deploy wrappers and expanded the
|
|
|
|
|
|
managed administration round-trip test. Corrected the historical App Group and extension
|
|
|
|
|
|
bundle IDs. CI installs the iOS workload and builds both managed Apple clients. **Next:** run
|
|
|
|
|
|
the physical-device VoiceOver, route-change, background/lock and real multi-human call matrix;
|
|
|
|
|
|
retain the Swift app as release oracle until those observable gates pass.
|
2026-09-19 15:43:37 +02:00
|
|
|
|
|
2026-09-19 00:39:04 +02:00
|
|
|
|
- **In progress (2026-09-19): managed macOS functional-parity checkpoint.** Extended the
|
|
|
|
|
|
.NET AppKit client across the remaining Swift desktop surface: persistent audio and
|
|
|
|
|
|
notification settings; VAD, focus-scoped configurable PTT and always-on modes; input,
|
|
|
|
|
|
output and auxiliary-device selection; input/output/auxiliary gain; stereo microphone;
|
|
|
|
|
|
RNNoise; self mute/deafen; event sounds and speech; modeless private conversations;
|
|
|
|
|
|
speaking and mute state; moderation, permissions, channel configuration and account
|
|
|
|
|
|
administration. Added typed administration helpers to `VoiceCat.Core` and per-local-stream
|
|
|
|
|
|
gain to `VoiceCat.Audio`. ScreenCaptureKit now publishes desktop audio with entire-desktop,
|
|
|
|
|
|
application-only and application-exclusion scopes, including optional VoiceOver/speech
|
|
|
|
|
|
exclusion, without introducing a native protocol dependency. The managed client imports
|
|
|
|
|
|
Swift profiles, TOFU pins and Keychain passwords, preserving a one-time JSON backup when it
|
|
|
|
|
|
converts profile storage. Added a Release publishing script with an ad-hoc validation path
|
|
|
|
|
|
and Developer ID/notarization path; consistently re-signing the nested .NET runtime fixed a
|
|
|
|
|
|
hardened-runtime Team-ID launch failure. Signing now operates on a separate distribution
|
|
|
|
|
|
copy so it cannot corrupt MSBuild's incremental app bundle or signing cache. **Verified:**
|
|
|
|
|
|
the ad-hoc Release app passes strict deep signature verification and remains running after
|
|
|
|
|
|
launch; Debug and Release Apple builds
|
|
|
|
|
|
succeed; 190/198 managed tests pass with eight optional native/published-server oracles
|
|
|
|
|
|
skipped; native `dev` CTest passes 29/29. The administration protocol test exercises typed
|
|
|
|
|
|
result round trips and the profile test proves Swift import plus backup. **Next:** manually
|
|
|
|
|
|
exercise ScreenCaptureKit selection and a real multi-human call, complete the VoiceOver
|
|
|
|
|
|
navigation/announcement matrix, and run the credentialed Developer ID notarization path.
|
|
|
|
|
|
Keep the Swift macOS app as the release oracle until all three gates pass; only then remove
|
|
|
|
|
|
its replaceable app/UI code.
|
|
|
|
|
|
|
2026-09-18 18:51:53 +02:00
|
|
|
|
- **In progress (2026-09-18): managed macOS client and selectable audio path.** Added a
|
2026-09-16 22:13:34 +02:00
|
|
|
|
separate .NET 10 AppKit solution with application/menu lifecycle, guest/account connection,
|
2026-09-16 22:05:40 +02:00
|
|
|
|
explicit TOFU approval, channel selection, roster/chat, disconnect state, accessibility
|
|
|
|
|
|
labels, sandbox entitlements and deterministic disposal. Voice join now opens bounded
|
2026-09-18 18:51:53 +02:00
|
|
|
|
stereo playback, subscribes, announces a managed microphone stream, converts the selected
|
2026-09-16 22:05:40 +02:00
|
|
|
|
Core Audio input to interleaved 48 kHz int16 and feeds `VoiceCat.Audio`; leave/disconnect
|
|
|
|
|
|
tears the path down. Channel changes suspend and resume voice so streams renegotiate.
|
2026-09-18 18:51:53 +02:00
|
|
|
|
Playback uses `AVAudioSourceNode` plus the shared tested `PcmRing`, converting to Core Audio's
|
|
|
|
|
|
native planar Float32 layout; the render callback allocates no managed memory and never locks
|
|
|
|
|
|
or blocks. Added a
|
2026-09-16 22:05:40 +02:00
|
|
|
|
macOS CI gate for the real workload, native Opus/RNNoise shim and Apple solution. Local
|
2026-09-16 22:13:34 +02:00
|
|
|
|
server profiles now persist validated host, port, authentication mode and identity through
|
|
|
|
|
|
`VoiceCat.Core`; malformed profile files do not prevent startup, writes replace atomically,
|
|
|
|
|
|
and passwords are deliberately excluded from the model and JSON. Three behavior tests cover
|
2026-09-16 22:16:18 +02:00
|
|
|
|
profile round trips, corrupt input and account validation. Protected channels are labeled
|
|
|
|
|
|
and prompt for a session-only password before interrupting active voice; cancel and rejected
|
2026-09-16 22:22:03 +02:00
|
|
|
|
joins restore the selected channel and voice session. Saved account profiles can opt into a
|
|
|
|
|
|
macOS Keychain generic-password item keyed by profile ID; secrets are written only after
|
|
|
|
|
|
successful authentication and removed when remembering is disabled or the profile is deleted.
|
2026-09-16 22:24:52 +02:00
|
|
|
|
The composer can now target the current channel or any connected user, and labels private
|
|
|
|
|
|
messages with the other participant.
|
2026-09-18 18:51:53 +02:00
|
|
|
|
On Apple Silicon macOS 27, SDK/workload set 10.0.401 now builds against the explicit
|
|
|
|
|
|
`net10.0-macos27.0` preview target. The build selects native arm64 Homebrew `protoc`,
|
|
|
|
|
|
deduplicates the media dylib during app bundling and uses macOS 27's error-returning
|
|
|
|
|
|
AVFoundation overloads while preserving macOS 14–26 calls. Debug disables hardened runtime
|
|
|
|
|
|
so its ad-hoc-signed .NET dylibs load without a developer identity; Release remains hardened.
|
|
|
|
|
|
The arm64 bundle is valid, launches successfully and displays the connection window on a
|
|
|
|
|
|
physical macOS 27 host. The app now creates missing private-file parent directories on first
|
|
|
|
|
|
launch, requests microphone authorization explicitly, and enumerates/selects actual Core Audio
|
|
|
|
|
|
input and output devices; changing either device while joined restarts voice cleanly. Native
|
|
|
|
|
|
validation on macOS also exposed a PCM-tap teardown race:
|
|
|
|
|
|
disabling either external sink is now a quiescence barrier, so callback state can be released
|
|
|
|
|
|
safely without adding locks or waits to the real-time path. **Verified:** managed locked
|
|
|
|
|
|
restore and warning-free Release build; 188/196 managed tests passed with eight optional
|
|
|
|
|
|
native/published-server oracle tests skipped; native `dev` build and 29/29 CTest tests passed;
|
|
|
|
|
|
both PCM teardown regressions plus external playback passed five consecutive runs. Live testing
|
|
|
|
|
|
selected the internal MacBook microphone, proved nonzero mic energy at a peer, and played a
|
|
|
|
|
|
clean sustained peer tone in the DTX-disabled Music Room. Lobby's later hiss was confirmed as
|
|
|
|
|
|
expected Opus DTX comfort noise for a synthetic sine, not playback corruption. Per user
|
|
|
|
|
|
direction, final listen quality will be exercised in real multi-human calls after feature
|
2026-09-19 00:39:04 +02:00
|
|
|
|
completion rather than an artificial ten-minute tone. This checkpoint is superseded by the
|
|
|
|
|
|
2026-09-19 functional-parity entry above.
|
2026-09-16 17:34:16 +02:00
|
|
|
|
|
2026-09-16 17:08:17 +02:00
|
|
|
|
- **Done (2026-09-16): Linux production packaging checkpoint.** Added a real TLS 1.3
|
|
|
|
|
|
`--health-check` with optional certificate pin verification. Linux x64 now has separate
|
|
|
|
|
|
locked self-contained publish graphs and invariant-globalization startup without a system
|
|
|
|
|
|
ICU dependency. Replaced the native Docker build with a non-root managed multi-stage build
|
|
|
|
|
|
on Microsoft's chiseled runtime-dependencies image; Compose uses a read-only root,
|
|
|
|
|
|
capability drop and no-new-privileges. Added a hardened DynamicUser systemd unit, install
|
|
|
|
|
|
helper, Linux smoke and reusable concurrent text/voice soak driver. **Verified:** the
|
|
|
|
|
|
published ELF starts in WSL, completes TLS health with its emitted pin, creates mode-0700
|
|
|
|
|
|
state, and exits cleanly on SIGTERM; `systemd-analyze verify` accepts the unit. The
|
|
|
|
|
|
published Windows server completed four soak cycles / 16 independent client sessions.
|
|
|
|
|
|
Dockerfile execution could not be checked because the local Docker daemon is unavailable.
|
|
|
|
|
|
**Next:** run the 30-minute+ soak and container build in release infrastructure, publish/
|
|
|
|
|
|
sign the image, and perform Windows NVDA/listen gates. Then begin the C# AppKit client port.
|
|
|
|
|
|
|
2026-09-16 16:55:17 +02:00
|
|
|
|
- **Done (2026-09-16): Managed CLI and cross-generation client exit.** Added `VoiceCat.Cli`
|
|
|
|
|
|
with interactive channel text, TOFU, channel selection and deterministic headless
|
|
|
|
|
|
text/tone verification. Independent process tests run two managed CLIs in channels 1 and 2
|
|
|
|
|
|
and prove text plus decoded bidirectional voice. A separate process test pairs it with the
|
|
|
|
|
|
existing C++ `vccli` in channel 2 and verifies text and voice in both directions. This
|
|
|
|
|
|
completes the behavioral Phase 6 criterion. **Verified:** 193 managed tests with all
|
|
|
|
|
|
conformance variables enabled, locked restore, package audit and 29/29 CTest tests.
|
|
|
|
|
|
**Next:** resume broad server production
|
|
|
|
|
|
deployment (Linux publish/container/service and soak), then the remaining manual Windows
|
|
|
|
|
|
listen/NVDA release checks before beginning the C# AppKit port.
|
|
|
|
|
|
|
2026-09-16 16:48:06 +02:00
|
|
|
|
- **Done (2026-09-16): Managed client/audio and Windows cutover checkpoint.** Added
|
|
|
|
|
|
`VoiceCat.Core` with TOFU-gated TLS, correlated concurrent requests, immutable snapshots,
|
|
|
|
|
|
reconnects, bounded client events, UDP binding and authenticated encrypted media. Added
|
|
|
|
|
|
`VoiceCat.Audio` with allocation-free real-time cycles, bounded PCM/jitter queues,
|
|
|
|
|
|
5/10/20/40/60 ms Opus reframing, VAD/PTT, DTX, DRED → FEC → PLC, RNNoise, per-stream
|
|
|
|
|
|
controls and stereo mixing. Shared TLS/media primitives moved to `VoiceCat.Crypto`.
|
|
|
|
|
|
The WinForms app now references a managed compatibility facade over these libraries and
|
|
|
|
|
|
uses C# WASAPI capture, loopback and playback. Active streams renegotiate after channel
|
|
|
|
|
|
moves while capture-facing IDs stay stable. Per-app mixing no longer allocates or locks in
|
|
|
|
|
|
its real-time loop. The self-contained Windows publish includes only the permitted
|
|
|
|
|
|
`voicecat_media.dll`; a real default-device capture/playback plus main-form smoke passed.
|
|
|
|
|
|
Managed tests cover two-way decoded PCM voice, text, TOFU, concurrent requests, reconnects,
|
|
|
|
|
|
bounded jitter, zero-allocation cycles, recovery order and stream continuity across a
|
|
|
|
|
|
mono→stereo channel move. The old Interop project remains only as a migration oracle and
|
|
|
|
|
|
its nine native ABI tests remain green. **Verified:** 190 managed tests with every native
|
|
|
|
|
|
conformance/published-server variable enabled, nine Windows native-oracle tests, 29 CTest
|
|
|
|
|
|
tests, locked restores, both self-contained publishes and the permissive license audit.
|
|
|
|
|
|
**Next:** add the managed CLI and explicit managed
|
|
|
|
|
|
client ↔ C++ CLI conversation, then finish production deployment work. The Phase 5/7
|
|
|
|
|
|
manual ten-minute listen and NVDA gates are not yet signed off; Apple rewrites follow.
|
|
|
|
|
|
|
2026-09-15 23:24:11 +02:00
|
|
|
|
- **Done (2026-09-15): Server deployment hardening checkpoint.** Pushed existing work
|
|
|
|
|
|
through `653131b` to `origin/dotnet/foundations`. Added CLI/environment configuration,
|
|
|
|
|
|
all-interface port 8384 defaults, config/fingerprint commands, local administrator
|
|
|
|
|
|
provisioning with hidden/stdin/environment password input, JSON readiness, an exclusive
|
|
|
|
|
|
instance lock, transport-failure observation and ten-second graceful shutdown.
|
|
|
|
|
|
Password-attempt buckets are shared by IP/account across reconnects, bounded to 4096
|
|
|
|
|
|
keys and enforce configurable burst/refill plus escalating backoff before Argon2.
|
|
|
|
|
|
Added self-contained win-x64 publishing with separate checked runtime lock files;
|
|
|
|
|
|
server publishes without the audio/codec shim. Behavior tests verify precedence,
|
|
|
|
|
|
invalid input, cross-connection/account throttling, readiness, duplicate-instance refusal,
|
|
|
|
|
|
active TLS shutdown and real published-executable provisioning/fingerprint persistence.
|
|
|
|
|
|
**Verified:** 174 managed tests with all conformance/published-server variables enabled;
|
|
|
|
|
|
warning-free Release build, native build/29 CTest tests and permissive package audit.
|
|
|
|
|
|
**Next (user priority):** managed client-core and audio prerequisites, then Windows
|
|
|
|
|
|
WinForms cutover before either Apple GUI. Windows is already C# at the UI layer but
|
|
|
|
|
|
still depends on native protocol/audio. Linux publishing/container/service packaging
|
|
|
|
|
|
and operational soak remain before broad production rollout; do not claim Phase 4
|
|
|
|
|
|
production rollout or GUI parity complete without those platform/behavior checks.
|
|
|
|
|
|
|
2026-09-15 23:11:09 +02:00
|
|
|
|
- **Done (2026-09-15): Managed channel and administration checkpoint.** Reaper committed
|
|
|
|
|
|
as `274b850`. Added protected joins and capacity checks, leave-to-Lobby, persisted channel
|
|
|
|
|
|
create/edit/delete with events, parent/cycle validation and Lobby protection. Native
|
|
|
|
|
|
salted BLAKE2b channel hashes work in both directions; empty edit passwords preserve
|
|
|
|
|
|
protection. Channel edits, moves and deletion clear streams before further media routing.
|
|
|
|
|
|
Session permissions gate kick/ban/move/server-mute/deafen and account create/reset/delete/
|
|
|
|
|
|
list. Only administrators grant permissions; temporary-channel permission cannot create
|
|
|
|
|
|
permanent channels. Kick/ban retire media and send one reason-bearing LEFT. Account bans
|
|
|
|
|
|
persist by username, guest bans by address; Unix-millisecond expiry converts to database
|
|
|
|
|
|
seconds, fixing the native handler's unit mismatch. Bounded Argon2 work remains outside
|
|
|
|
|
|
the session lock; account lists exclude hashes and respect the frame limit. The existing
|
|
|
|
|
|
C++ CLI successfully creates protected channels and creates/lists accounts against the
|
|
|
|
|
|
managed server. Fixed its temporary channel-string pointers and zero audio defaults.
|
|
|
|
|
|
A native sample-rate regression intermittently measured host microphone audio alongside
|
|
|
|
|
|
its injected tone; changed that test to external capture/playback and kept callback state
|
|
|
|
|
|
alive through client shutdown, with synchronized energy reads.
|
|
|
|
|
|
**Verified:** 169/169 managed tests with all native conformance enabled, zero skips;
|
|
|
|
|
|
warning-free managed Release build, native dev build and 29/29 CTest tests; diff check.
|
|
|
|
|
|
**Next:** production configuration, administrator provisioning/publishing and remaining
|
|
|
|
|
|
server readiness checks (including auth rate limiting). Phase 4 is still in progress.
|
|
|
|
|
|
Then managed audio/core/CLI, Windows cutover, C# AppKit and UIKit clients; preserve Swift
|
|
|
|
|
|
ReplayKit extension and freeze the shared-ring contract before the iOS cutover.
|
|
|
|
|
|
|
2026-09-15 22:58:16 +02:00
|
|
|
|
- **Done (2026-09-15): Managed media-aware reaper.** Voice checkpoint committed as
|
|
|
|
|
|
`05eacb3`. Added `VoiceServerOptions` (name/guests/capacity, handshake deadline,
|
|
|
|
|
|
idle timeout and sweep interval), preserving the previous constructor overload.
|
|
|
|
|
|
Default expiry/sweep are 45 s / 15 s; zero idle timeout disables reaping. Parsed TCP
|
|
|
|
|
|
envelopes, authenticated owned-stream voice and exact bound-endpoint UDP keepalives
|
|
|
|
|
|
refresh one monotonic session timestamp. Invalid media does not refresh it. Removed
|
|
|
|
|
|
the independent 60-second TCP-only timeout so media-active sessions remain connected.
|
|
|
|
|
|
Reaping sends a fatal disconnect and retires presence/media routing with one LEFT
|
|
|
|
|
|
event. Shutdown awaits active control/media/reaper loops and unfinished handshakes.
|
|
|
|
|
|
Tests inject `TimeProvider` timestamps to verify TCP/UDP activity, rejected media,
|
|
|
|
|
|
single departure events, disabled reaping and shutdown. **Verified:** 160/160 managed
|
|
|
|
|
|
tests with all native conformance enabled; managed Release build has zero warnings;
|
|
|
|
|
|
native dev build and 29/29 CTest tests green; `git diff --check` passes.
|
|
|
|
|
|
**Next:** protected channel joins and channel CRUD, permissions/moderation/account
|
|
|
|
|
|
administration, then production configuration/publishing. Phase 4 remains in progress.
|
|
|
|
|
|
Follow with audio/core/managed CLI, switch Windows to the managed library, then C#
|
|
|
|
|
|
AppKit/UIKit clients. Keep the Swift broadcast extension and frozen shared-ring boundary.
|
|
|
|
|
|
|
2026-09-15 22:53:54 +02:00
|
|
|
|
- **Done (2026-09-15): Phase 4 encrypted voice checkpoint.** Existing pending codec/DSP
|
|
|
|
|
|
and initial server work committed as `4067bab`. Managed server now binds UDP on the
|
|
|
|
|
|
TCP port number, issues 16-byte session tokens, supports subscription and multi-stream
|
|
|
|
|
|
signaling, and authenticates/reseals encoded Opus to eligible channel subscribers.
|
|
|
|
|
|
Crypto and endpoints have one UDP-loop owner; control handlers publish immutable
|
|
|
|
|
|
routing snapshots. Rejects invalid tokens, malformed/forged/replayed media and SSRCs
|
|
|
|
|
|
not owned by the sender. Stop, unsubscribe, channel movement and disconnect update
|
|
|
|
|
|
routing; retired keys are cleared without requiring subsequent UDP traffic.
|
|
|
|
|
|
First endpoint binding is fixed for the session (reconnect to change it), unlike
|
|
|
|
|
|
the C++ oracle's permissive rebinding policy. Packet formats/protocol v2 are unchanged.
|
|
|
|
|
|
Two actual C++ `vccli` processes authenticate, join, chat and exchange mono/stereo
|
|
|
|
|
|
voice through the managed server. Native voice oracle additionally verifies three
|
|
|
|
|
|
concurrent streams with bidirectional decoded PCM energy/metadata, without hardware.
|
|
|
|
|
|
Added finite `vccli --test-tone-ms` and fixed normal `--voice` to subscribe first.
|
|
|
|
|
|
**Verified:** 154/154 managed tests, no skips with TLS/database/voice/CLI variables;
|
|
|
|
|
|
native dev build and 29/29 CTest tests; independent native media staging; warning-free
|
|
|
|
|
|
managed Release build; identical regenerated wire/password fixtures; C++ DSP within
|
|
|
|
|
|
one PCM unit; 22 permissive package licenses; `git diff --check` passes. Fan-out core
|
|
|
|
|
|
allocates zero managed bytes for 50 subscribers; transport scheduling and crypto
|
|
|
|
|
|
fallback are excluded. Transport test delivers all 2,500 packets at a paced 50 pps.
|
|
|
|
|
|
**Next:** media-aware keepalive/reaper, then protected channel joins, channel CRUD,
|
|
|
|
|
|
permissions/moderation/account administration and production configuration. Phase 4
|
|
|
|
|
|
remains in progress. Audio, managed client/CLI, Windows cutover and C# AppKit/UIKit
|
|
|
|
|
|
follow; keep Swift ReplayKit extension and freeze its ring contract before iOS.
|
|
|
|
|
|
|
2026-09-15 22:51:33 +02:00
|
|
|
|
- **In progress (2026-09-15): Phase 4 managed server control plane.** Added TLS socket
|
|
|
|
|
|
orchestration, bounded framing/queues, guest and password authentication, persisted
|
|
|
|
|
|
channels, state snapshots, channel joins, text routing, ping and disconnect events.
|
|
|
|
|
|
The existing C++ CLI authenticates and sends text through the managed server.
|
|
|
|
|
|
Managed Argon2id verification passes libsodium fixtures, including UTF-8 and embedded
|
|
|
|
|
|
NUL passwords. The C++ database oracle proves existing account/channel import and
|
|
|
|
|
|
C++ verification of managed-created accounts without password resets. **Verified:**
|
|
|
|
|
|
142/142 managed tests with all native interoperability checks enabled, warning-free
|
|
|
|
|
|
Release build, regenerated password fixtures identical, and 22 permissive package
|
|
|
|
|
|
licenses; native dev build and 29/29 CTest tests green. Locked restore passes.
|
|
|
|
|
|
CI requires CLI/database checks in its C++ conformance job. Codec/DSP and the first
|
|
|
|
|
|
server slice are committed together on `dotnet/foundations` as a validated checkpoint.
|
|
|
|
|
|
**Next:** encrypted UDP binding/SFU relay and stream signaling.
|
|
|
|
|
|
UDP voice, streams, administration, protected channel joins and production configuration
|
|
|
|
|
|
remain pending; this is the first control-plane checkpoint, not Phase 4 completion.
|
|
|
|
|
|
|
2026-09-15 22:53:54 +02:00
|
|
|
|
### .NET control-plane checkpoint handoff / discoveries (2026-09-15)
|
2026-09-15 22:51:33 +02:00
|
|
|
|
|
|
|
|
|
|
- **Working tree:** stay on `dotnet/foundations`, tracking `origin/dotnet/foundations`.
|
|
|
|
|
|
Foundation `b76181d` and TLS checkpoint `2df79cd` were committed and pushed.
|
|
|
|
|
|
The codec/DSP port and first managed server checkpoint were subsequently committed
|
|
|
|
|
|
together, including new projects, native bindings/oracles, tests and docs.
|
|
|
|
|
|
See the latest checkpoint commit; no push is requested for this session.
|
|
|
|
|
|
- **Style/scope:** write idiomatic .NET in `dotnet/`; do not copy C++ code or comment
|
|
|
|
|
|
style. The existing implementation is the behavior/wire oracle. No wire changes
|
|
|
|
|
|
were made. Read `docs/porting-to-dotnet.md`, `docs/api-dotnet.md`, `dotnet/README.md`
|
|
|
|
|
|
and the relevant protocol/security/voice sections before the next subsystem.
|
|
|
|
|
|
- **Implemented projects:** `VoiceCat.Protocol` (existing protobuf + framing),
|
|
|
|
|
|
`VoiceCat.Crypto` (media crypto/replay, TLS/exporters, identity/TOFU, password hashing),
|
|
|
|
|
|
`VoiceCat.Codec` (Opus/PLC/DRED), `VoiceCat.Dsp` (RNNoise/energy VAD), and
|
|
|
|
|
|
`VoiceCat.Server` (real TLS control server + compatible SQLite account/channel store).
|
|
|
|
|
|
`dotnet/oracle/` contains optional native wire, TLS, DSP, password and database
|
|
|
|
|
|
conformance executables. `ServerTests` exercises real sockets and the existing CLI;
|
|
|
|
|
|
`AccountStoreTests` proves native database import and password verification both ways.
|
|
|
|
|
|
- **TLS discovery:** BouncyCastle destroys exporter secrets after its handshake
|
|
|
|
|
|
callback. Export keys inside `NotifyHandshakeComplete`, not after the socket loop
|
|
|
|
|
|
notices readiness. Preserve label `voicecat media v1` and contexts `[0]` / `[1]`.
|
|
|
|
|
|
A `TlsSession` has one owner; the control connection loop owns all TLS calls.
|
|
|
|
|
|
Certificate acceptance is a synchronous leaf-SHA256 pin gate, not normal PKI.
|
|
|
|
|
|
New certificates carry the Ed25519 public key in their SAN, but verification of
|
|
|
|
|
|
the ServerHello identity against that SAN remains pending. Partial credential
|
|
|
|
|
|
sets must fail rather than silently generate a new server identity.
|
|
|
|
|
|
- **Native codec discoveries:** the actual pinned Opus is **1.5.2**, despite older
|
|
|
|
|
|
design comments referring to 1.6. Standalone builds use checksum-pinned upstream
|
|
|
|
|
|
sources with DRED/Deep PLC enabled. DRED needs a **30 ms minimum** in this release;
|
|
|
|
|
|
20 ms produces no redundancy. DRED encoding at 8/12 kHz is explicitly unsupported;
|
|
|
|
|
|
decoding works at all five rates. Recovery offset defaults to one missing frame's
|
|
|
|
|
|
samples before the next packet's start (the older C++ zero offset is not a guide).
|
|
|
|
|
|
Fixed-signature C wrappers avoid the Apple ARM64 varargs ABI issue with Opus CTLs.
|
|
|
|
|
|
Windows DLL staging must omit the MinGW `lib` prefix. MinGW and MSVC builds pass;
|
|
|
|
|
|
iOS needs later static packaging. Device audio callbacks/rings are not implemented.
|
|
|
|
|
|
- **DSP behavior:** RNNoise processes complete 480-sample mono chunks at 48 kHz;
|
|
|
|
|
|
other rates pass through, and partial chunks at 48 kHz are rejected. Native C++
|
|
|
|
|
|
conformance allows one PCM unit for rounding. VAD hang time uses monotonic
|
|
|
|
|
|
`TimeProvider` timestamps, starts closed and does not replace noise suppression.
|
|
|
|
|
|
The combined allocation test proves zero managed allocations across 1,000 cycles.
|
|
|
|
|
|
- **Password/database discoveries:** use the existing BouncyCastle Argon2 engine
|
|
|
|
|
|
with strict libsodium PHC parsing; no additional Konscious dependency or password
|
|
|
|
|
|
reset is needed. Keep UTF-8 bytes unchanged, including embedded NUL. New hashes use
|
|
|
|
|
|
Argon2id v19, 64 MiB, two iterations, parallelism one, salt 16/output 32 bytes.
|
|
|
|
|
|
Verification is bounded to 128 MiB, ten iterations, parallelism four and 1024 UTF-8
|
|
|
|
|
|
password bytes; excessive imported costs fail closed. Two per-store password
|
|
|
|
|
|
workers bound CPU/memory use. Failed login does not update `last_login`.
|
|
|
|
|
|
Keep SQLite schema v2; accept v1 migration and reject unknown versions.
|
|
|
|
|
|
**Seed both default channels only when the entire channel table is empty**;
|
|
|
|
|
|
an existing single Lobby is an intentional configuration and must be preserved.
|
|
|
|
|
|
- **SQLite dependency discovery:** the initial `Microsoft.Data.Sqlite` 10.0.5 bundle
|
|
|
|
|
|
pulled an older vulnerable SQLite native dependency, rejected by warnings-as-errors
|
|
|
|
|
|
restore. The implementation uses `Microsoft.Data.Sqlite.Core` 10.0.5,
|
|
|
|
|
|
SQLitePCLRaw bundle 3.0.2 and explicitly pinned SourceGear SQLite 3.50.4.2 instead.
|
|
|
|
|
|
SourceGear's native package lacks a NuGet license expression; the audit has an
|
|
|
|
|
|
exact-version/repository-identity exception for its public-domain SQLite build.
|
|
|
|
|
|
NativeAOT publishing/trimming has not been verified for this solution.
|
2026-09-15 22:53:54 +02:00
|
|
|
|
- **Previous control-plane checkpoint limits:** CLI binds loopback; positional arguments are data
|
2026-09-15 22:51:33 +02:00
|
|
|
|
directory and TCP port. Guests are enabled there, and the hosting API can disable
|
|
|
|
|
|
them. Accounts can be provisioned through `AccountStore` or native administration;
|
|
|
|
|
|
automatic bootstrap/admin CLI is pending. Authentication enters unprotected Lobby
|
|
|
|
|
|
id 1 subject to capacity. Server owns text sender ids/timestamps. Connections cap
|
|
|
|
|
|
at 64; queues cap at 32 incoming/64 outgoing envelopes, payloads at 64 KiB (shared
|
|
|
|
|
|
framer allows 16 MiB). Slow consumers disconnect. TLS handshake timeout is 15 s;
|
|
|
|
|
|
receive-idle timeout after handshake is 60 s. No UDP port/media features are
|
|
|
|
|
|
advertised, and voice subscription fails explicitly. Protected joins, channel CRUD,
|
|
|
|
|
|
streams, SFU, moderation/admin handlers, configuration compatibility and full reaper
|
|
|
|
|
|
behavior remain pending. **Do not mark Phase 4 or voice interoperability complete.**
|
|
|
|
|
|
|
|
|
|
|
|
To reproduce the last successful validation on Windows, run in **PowerShell**:
|
|
|
|
|
|
|
|
|
|
|
|
```powershell
|
|
|
|
|
|
./dotnet/build-native.ps1 -Generator Ninja -CCompiler C:/tools/msys64/ucrt64/bin/cc.exe
|
|
|
|
|
|
cmake --preset dev -DVOICECAT_BUILD_DOTNET_ORACLE=ON
|
|
|
|
|
|
cmake --build --preset dev
|
|
|
|
|
|
ctest --preset dev
|
|
|
|
|
|
dotnet restore dotnet/VoiceCat.slnx --locked-mode
|
|
|
|
|
|
$env:VOICECAT_TLS_ORACLE = (Resolve-Path build/dev/bin/voicecat-dotnet-tls-oracle.exe).Path
|
|
|
|
|
|
$env:VOICECAT_DATABASE_ORACLE = (Resolve-Path build/dev/bin/voicecat-dotnet-database-oracle.exe).Path
|
|
|
|
|
|
$env:VOICECAT_VCCLI = (Resolve-Path build/dev/bin/vccli.exe).Path
|
2026-09-15 22:53:54 +02:00
|
|
|
|
$env:VOICECAT_VOICE_ORACLE = (Resolve-Path build/dev/bin/voicecat-dotnet-voice-oracle.exe).Path
|
2026-09-15 22:51:33 +02:00
|
|
|
|
dotnet test dotnet/VoiceCat.slnx -c Release --no-restore
|
|
|
|
|
|
./dotnet/check-licenses.ps1
|
|
|
|
|
|
```
|
|
|
|
|
|
|
2026-09-15 22:58:16 +02:00
|
|
|
|
Last results: **160/160 managed tests, no skips with those variables set; 29/29 native
|
2026-09-15 22:51:33 +02:00
|
|
|
|
CTest tests; warning-free Release build; 22 package licenses approved; locked restore
|
|
|
|
|
|
and `git diff --check` passed.** Without the variables, native interoperability tests
|
|
|
|
|
|
skip; that is not equivalent verification. Desktop CI stages codec/DSP bindings on
|
|
|
|
|
|
Windows/Linux/macOS. Its Linux C++ job requires TLS, CLI and database interoperability
|
|
|
|
|
|
and regenerates wire/password/DSP fixtures. Only Windows was run locally this session.
|
|
|
|
|
|
|
2026-09-15 22:53:54 +02:00
|
|
|
|
**Voice behavior now verified:** real clients bind UDP and exchange encrypted Opus,
|
|
|
|
|
|
preserving SSRC/timestamp/flags while resealing with recipient-specific counters.
|
|
|
|
|
|
Never decode audio on the SFU. The two-C++-client voice/text criterion passes;
|
|
|
|
|
|
the remaining Phase 4 server behaviors still need implementation and conformance tests.
|
2026-09-15 22:51:33 +02:00
|
|
|
|
|
|
|
|
|
|
- **Done (2026-09-15): Codec/DSP desktop port.** TLS checkpoint `2df79cd` committed
|
|
|
|
|
|
and pushed to `origin/dotnet/foundations`. Added span-based Opus wrappers, safe native
|
|
|
|
|
|
handle ownership, RNNoise processing, monotonic energy VAD, and fixed-signature
|
|
|
|
|
|
native bindings. Round-trip/PLC behavior passes across 40 supported formats with
|
|
|
|
|
|
the existing Opus build. Independent native staging builds checksum-pinned upstream
|
|
|
|
|
|
Opus 1.5.2 with DRED enabled and the existing RNNoise model. Actual dropped-frame
|
|
|
|
|
|
DRED recovery passes across 40 decoder formats; DRED encoding at 8/12 kHz is
|
|
|
|
|
|
explicitly rejected (tests encode those packets at 16 kHz). This release requires
|
|
|
|
|
|
a 30 ms redundancy floor; the older 20 ms setting emits no DRED. C++ denoising
|
|
|
|
|
|
conformance is within one PCM unit, and 1,000 codec/DSP cycles allocate zero managed
|
|
|
|
|
|
bytes. **Verified:** Release build with no warnings; 127/127 managed tests with
|
|
|
|
|
|
native TLS interoperability enabled; native dev build and 29/29 CTest tests green;
|
|
|
|
|
|
16 permissive NuGet licenses. MinGW and Visual Studio native builds pass. Desktop
|
|
|
|
|
|
CI now builds/stages the bindings before testing. iOS static packaging and device
|
|
|
|
|
|
audio remain later phases. **Next:** Phase 4 managed server; prove persisted
|
|
|
|
|
|
libsodium Argon2id hash compatibility before account/database implementation.
|
|
|
|
|
|
|
2026-09-15 18:04:20 +02:00
|
|
|
|
- **Done (2026-09-15): TLS exporter interoperability and persisted credentials.** Foundation commit
|
|
|
|
|
|
`b76181d` pushed to `origin/dotnet/foundations`. Added a nonblocking managed TLS 1.3
|
|
|
|
|
|
session with certificate acceptance gate and directional media factories. Managed
|
|
|
|
|
|
loopback and C++ interoperability pass; exporter keys are captured inside BouncyCastle's
|
|
|
|
|
|
handshake callback before its exporter secrets are destroyed. The C++ TLS oracle
|
|
|
|
|
|
authenticates encrypted challenges in both directions against the existing mbedTLS
|
|
|
|
|
|
context. Added explicit persisted TOFU pins, compatible Ed25519 identity/PEM import,
|
|
|
|
|
|
new certificate identity SAN, and rejection of incomplete credential sets.
|
|
|
|
|
|
**Verified:** 42/42 managed tests with the native TLS oracle enabled; native dev build
|
|
|
|
|
|
and 29/29 CTest tests green; 16 permissive package licenses verified. Complete socket
|
|
|
|
|
|
orchestration and managed server/client state remain pending.
|
|
|
|
|
|
**Next:** Phase 3 codec/DSP wrappers and native packaging.
|
|
|
|
|
|
|
2026-09-15 17:54:16 +02:00
|
|
|
|
- **Done (2026-09-15): Initial .NET wire/crypto port** on `dotnet/foundations`, from `cs-port`.
|
|
|
|
|
|
Added `dotnet/` solution, schema code generation, pipe framing, immutable voice headers,
|
|
|
|
|
|
directional media encryption/decryption, and xUnit conformance tests. Both platform
|
|
|
|
|
|
and managed crypto paths are tested. Added optional C++ fixture oracle, managed CI,
|
|
|
|
|
|
dependency lock files, license audit, and `docs/api-dotnet.md`. **Verified:** managed
|
|
|
|
|
|
Release build, 34/34 tests including C++ golden bytes, and 16 permissive package
|
|
|
|
|
|
licenses. Fresh `cmake --build --preset dev` and `ctest --preset dev` green (29/29);
|
|
|
|
|
|
regenerating the C++ fixtures produces identical bytes. Native codec/audio packaging
|
|
|
|
|
|
is deferred to its implementation phase. Next checkpoint: BouncyCastle TLS 1.3
|
|
|
|
|
|
exporter interoperability with the existing server.
|
|
|
|
|
|
|
2026-07-23 13:37:05 +02:00
|
|
|
|
- **Done (2026-07-23):** **First comment-density cleanup across core, server, and native
|
|
|
|
|
|
clients.** Condensed comments in the highest-noise audio, reconnect, registry, and binding
|
|
|
|
|
|
files; removed implementation history and narration; retained ABI ownership, threading,
|
|
|
|
|
|
real-time, ordering, and OS-API invariants. Moved the durable iOS audio-routing/pacing rules
|
|
|
|
|
|
to `docs/voice.md` and client reconnection policy to `docs/protocol.md`. No behavior, wire
|
|
|
|
|
|
format, or C ABI changes. **Verification:** `cmake --build --preset dev` green;
|
|
|
|
|
|
`ctest --preset dev` 29/29 green. `dotnet build VoiceCat.slnx` restores dependencies and
|
|
|
|
|
|
builds `VoiceCat.Interop` + `VoiceCat.App`, then fails in the unchanged test project because
|
|
|
|
|
|
`ExternalPcmTests.cs:49` references internal `NativeMethods` (`CS0122`).
|
|
|
|
|
|
|
2026-06-25 16:18:21 +02:00
|
|
|
|
- **Done (2026-06-25):** **Fixed iOS AirPods-disconnect reinitialize loop on A2DP presets
|
|
|
|
|
|
(Stereo Mic / Mono Mic).** Regression from the 2026-06-25 audio-device-change recovery
|
|
|
|
|
|
commit below, which broadened the route-change recovery set from
|
|
|
|
|
|
`{oldDeviceUnavailable, newDeviceAvailable}` to "everything except
|
|
|
|
|
|
categoryChange/routeConfigurationChange". That added `.override` to the recovery set, and
|
|
|
|
|
|
`.override` is fired by our own `applyA2dpSpeakerFallback()` →
|
|
|
|
|
|
`overrideOutputAudioPort(.speaker)` — which `recoverAudio()` calls on every recovery. On an
|
|
|
|
|
|
A2DP preset with AirPods connected, disconnecting them ran:
|
|
|
|
|
|
`oldDeviceUnavailable` → `recoverAudio()` → `applyA2dpSpeakerFallback()` (no external
|
|
|
|
|
|
output now) → `overrideOutputAudioPort(.speaker)` → `.override` routeChange →
|
|
|
|
|
|
`recoverAudio()` → `applyConfiguration()` (setCategory resets the override) →
|
|
|
|
|
|
`applyA2dpSpeakerFallback()` → `overrideOutputAudioPort(.speaker)` → `.override` → …
|
|
|
|
|
|
Each iteration also called `IOSAudioEngine.reconfigure()` → `rebuild()` (a full
|
|
|
|
|
|
stop/restart of `AVAudioEngine`), which is the audible reinitialize loop + CPU spin the
|
|
|
|
|
|
user reported. Voice Chat (`.btHfpVoice`) and Built-in Mic + Speaker were unaffected
|
|
|
|
|
|
because `applyA2dpSpeakerFallback` early-returns for non-A2DP modes (no
|
|
|
|
|
|
`overrideOutputAudioPort` call, no `.override` notification).
|
|
|
|
|
|
|
|
|
|
|
|
Two-part fix (no C ABI / proto / docs changes — pure Swift iOS-app target):
|
|
|
|
|
|
1. **`AudioSessionManager.handleRouteChange`** (`AudioSessionManager.swift:182`): added
|
|
|
|
|
|
`.override` to the skip list alongside `.categoryChange`/`.routeConfigurationChange`.
|
|
|
|
|
|
`.override` is only ever fired by our own `overrideOutputAudioPort` call, so treating
|
|
|
|
|
|
it as a recovery reason is the loop by definition. The
|
|
|
|
|
|
`AVAudioEngineConfigurationChange` observer in `IOSVoiceProcessingEngine` remains as
|
|
|
|
|
|
the backstop for the case where an override actually stops the engine.
|
|
|
|
|
|
2. **`IOSAudioRouter.applyA2dpSpeakerFallback`** (`IOSAudioRouter.swift`): made idempotent
|
|
|
|
|
|
via a `lastAppliedOutputOverride` tracker. Skips the `overrideOutputAudioPort` call
|
|
|
|
|
|
when the desired override (`.none` for external output present, `.speaker` otherwise)
|
|
|
|
|
|
already matches the last successfully applied value — so even if some other path
|
|
|
|
|
|
re-enters, the redundant override (and its `.override` notification) isn't fired. The
|
|
|
|
|
|
tracker is reset to `nil` at the top of `applyConfiguration()` (setCategory can reset
|
|
|
|
|
|
the override) and on a failed call. Defense-in-depth on top of fix 1.
|
|
|
|
|
|
|
|
|
|
|
|
**Build:** `xcodebuild -project clients/apple/iOS/VoiceCatiOS.xcodeproj -scheme VoiceCatiOS
|
|
|
|
|
|
-destination 'generic/platform=iOS' build` green (Xcode 26.5 / iOS 18.0). The standalone
|
|
|
|
|
|
`swift test` in `clients/apple/` fails with `no such module 'VoiceCatC'` — pre-existing
|
|
|
|
|
|
(confirmed by stashing the changes: fails identically without them); the `VoiceCatC` C ABI
|
|
|
|
|
|
XCFramework isn't on SwiftPM's resolver path in this workspace. Not caused by this change
|
|
|
|
|
|
(the edit is in the iOS app target, not the `VoiceCatCore` SwiftPM package).
|
|
|
|
|
|
**Next (manual, on-device):** connect on the Stereo Mic preset, join voice, disconnect
|
|
|
|
|
|
AirPods — expect ONE `oldDeviceUnavailable` → one `recoverAudio` → one `engine started` →
|
|
|
|
|
|
one `override` routeChange (skipped, no further `recoverAudio`) and steady audio through
|
|
|
|
|
|
the loudspeaker. Also sanity-check AirPods reconnect and wired headphone plug/unplug
|
|
|
|
|
|
recover exactly once.
|
|
|
|
|
|
|
2026-06-25 14:57:13 +02:00
|
|
|
|
- **Done (2026-06-25):** **iOS robustness — auto-reconnect after a network change + audio
|
|
|
|
|
|
recovery when audio devices plug/unplug.** Two layers of bugs the iOS client had:
|
|
|
|
|
|
(a) a `VC_EVENT_DISCONNECTED` from the C core on a Wi-Fi→cellular flip / DNS outage /
|
|
|
|
|
|
server restart used to leave the session dead with no retry; (b) unplugging wired
|
|
|
|
|
|
headphones or AirPods left the engine stopped forever — mic stopped transmitting and
|
|
|
|
|
|
remote audio stayed silent (the server connection itself survived, but the audio graph
|
|
|
|
|
|
did not recover).
|
|
|
|
|
|
|
|
|
|
|
|
The first attempt wired reconnect into `AppState.handleConnectEvent`, but that handler
|
|
|
|
|
|
never runs for a live-session disconnect: once `SessionState.init` overwrites
|
|
|
|
|
|
`client.onEvent` (`SessionState.swift:87`), the `.disconnected` event is delivered to
|
|
|
|
|
|
`SessionState.handleEvent`, which used to play a cue and do nothing else. So the live
|
|
|
|
|
|
session would sit as a zombie for ~30-60 s (the C core's TCP keepalive/reaper timeout)
|
|
|
|
|
|
and then play the "connection lost" sound with no reconnect armed — exactly what the
|
|
|
|
|
|
user saw. The fix below has two parts addressing both the missing reconnect AND the
|
|
|
|
|
|
long wait.
|
|
|
|
|
|
|
|
|
|
|
|
1. **Event-driven reconnect** (`AppState.swift`, `SessionState.swift`): added a
|
|
|
|
|
|
`weak var appState: AppState?` to `SessionState`, set by AppState on auth success.
|
|
|
|
|
|
`SessionState.handleEvent` `.disconnected` now plays the cue and calls
|
|
|
|
|
|
`appState?.onLiveSessionDisconnected()` — the SINGLE path by which AppState learns a
|
|
|
|
|
|
live session dropped (since its own `handleConnectEvent` is bypassed for live-session
|
|
|
|
|
|
events). `onLiveSessionDisconnected` calls a shared `teardownLiveSessionAndReconnect`
|
|
|
|
|
|
that snapshots the live session into `LastSession`, stops the audio engine,
|
|
|
|
|
|
deactivates the AVAudioSession, nil's `session` (which releases `VoiceCatClient` →
|
|
|
|
|
|
`vc_client_destroy` joins the io thread), resets the backoff counter, and arms
|
|
|
|
|
|
`scheduleReconnect`.
|
|
|
|
|
|
2. **Path-driven proactive reconnect** (`AppState.swift`): an `NWPathMonitor`
|
|
|
|
|
|
(`Network.framework`) now runs the whole time we're CONNECTED (started on auth
|
|
|
|
|
|
success, not only when armed for reconnect) and stays armed across reconnects. Its
|
|
|
|
|
|
`pathUpdateHandler` (dispatched to @MainActor) does two things:
|
|
|
|
|
|
- While connected: a primary-interface change (Wi-Fi↔cellular) OR the path becoming
|
|
|
|
|
|
`.unsatisfied` triggers `proactiveReconnect()` — tearing the live session down
|
|
|
|
|
|
BEFORE the C core notices the dead TCP read. This is what collapses the 30-60 s
|
|
|
|
|
|
reaper wait into ~1 s + the first backoff tick. Same-interface refreshes (Wi-Fi
|
|
|
|
|
|
BSSID roams, signal-strength changes) are intentionally ignored (signature
|
|
|
|
|
|
comparison via `pathSignature`); those usually don't break the TCP connection.
|
|
|
|
|
|
- While mid-reconnect (no session): a path becoming `.satisfied` resets the backoff
|
|
|
|
|
|
counter and arms `scheduleReconnect` for a fast-fresh retry.
|
|
|
|
|
|
`userInitiatedDisconnect` distinguishes manual `disconnect()`/`cancelConnect()` (which
|
|
|
|
|
|
set it true → cancel all reconnect state) from a network drop (which leaves it false).
|
|
|
|
|
|
On a successful reconnect, `reconnectAttempt` resets and `lastSession` clears; the
|
|
|
|
|
|
path monitor keeps watching for the next change. On user-initiated disconnect, all
|
|
|
|
|
|
reconnect state (task + path monitor + `lastSession` + `connectedServer`) is cancelled.
|
|
|
|
|
|
3. **Backoff + restore**: exponential backoff 1s → 2s → 4s → 8s → 16s → 30s cap,
|
|
|
|
|
|
indefinite. TOFU pins match on the second connect (`VC_TOFU_MATCHED`) so the identity
|
|
|
|
|
|
gate auto-confirms; on auth success `SessionState.requestRestore` issues a
|
|
|
|
|
|
`joinChannel` and re-arms voice + restores the local mute/deafen state on the
|
|
|
|
|
|
resulting `.joinResult`.
|
|
|
|
|
|
4. **Audio recovery** (`AudioSessionManager.swift`, `IOSVoiceProcessingEngine.swift`):
|
|
|
|
|
|
replaced the route-change handler's narrow `.oldDeviceUnavailable`/
|
|
|
|
|
|
`.newDeviceAvailable` guard with a single intent-gated `recoverAudio()` path that
|
|
|
|
|
|
re-activates the AVAudioSession, re-applies the route config, and rebuilds the
|
|
|
|
|
|
engine; it runs on every externally-initiated route change reason except
|
|
|
|
|
|
`.categoryChange`/`.routeConfigurationChange` (those we cause ourselves and would
|
|
|
|
|
|
loop). Interruption-end now always calls `recoverAudio()` instead of only when
|
|
|
|
|
|
`.shouldResume` is set (which left the session permanently dead after Siri). Added
|
|
|
|
|
|
an `AVAudioEngineConfigurationChange` observer on the engine in `IOSAudioEngine`
|
|
|
|
|
|
that catches the case where iOS stops the engine itself AFTER our route-change
|
|
|
|
|
|
handler already rebuilt it (the previous rebuilds raced the engine's own self-stop
|
|
|
|
|
|
and lost). And `IOSAudioEngine.rebuild()` now does a one-shot reactivation-retry on
|
|
|
|
|
|
`engine.start()` failure — iOS sometimes refuses to start until the AVAudioSession is
|
|
|
|
|
|
re-activated, which is the silent-death case.
|
|
|
|
|
|
**Build:** `scripts/build-ios-client.sh --no-configure` green (Xcode 26.5 / iOS 18.0 sim
|
|
|
|
|
|
SDK, Swift 5 mode). No C ABI / `voicecat.h` / `voicecat.proto` / C core changes; the
|
|
|
|
|
|
existing TOFU auto-confirm (`VC_TOFU_MATCHED`) and idempotent `vc_join_channel` make
|
|
|
|
|
|
reconnect+restore possible without new C ABI. macOS and Windows clients unchanged.
|
|
|
|
|
|
**Next (manual, on-device):** verify unplugging AirPods/wired headphones mid-call keeps
|
|
|
|
|
|
audio going through the loudspeaker; verify Wi-Fi→cellular flip mid-call now triggers a
|
|
|
|
|
|
FAST reconnect (within a couple seconds, not 30-60 s) and lands in the same channel with
|
|
|
|
|
|
voice re-armed; verify tapping Disconnect mid-reconnect-abort cancels cleanly.
|
|
|
|
|
|
|
2026-06-24 14:29:39 +02:00
|
|
|
|
- **Done (2026-06-24):** **Three bug fixes — voice join/leave, channel edit defaults, channel-update stream restart.**
|
|
|
|
|
|
1. **Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.** Previously
|
|
|
|
|
|
"Join Voice" only started the local mic — receiving was always on (gated by channel
|
|
|
|
|
|
membership alone). Added a protocol-level voice subscription concept: new
|
|
|
|
|
|
`SubscribeVoiceRequest`/`UnsubscribeVoiceRequest`/`VoiceSubscriptionResult` proto messages
|
|
|
|
|
|
(`core/proto/voicecat.proto`), `User.voice_subscribed` field, `vc_join_voice`/`vc_leave_voice`
|
|
|
|
|
|
C ABI functions (`core/include/voicecat.h`), `VC_EVENT_VOICE_STATE` event, server-side
|
|
|
|
|
|
`voice_subscribed_` flag on `ConnSession` checked by the SFU relay's recipient filter
|
|
|
|
|
|
(`SessionRegistry::find_channel_sessions` excludes non-subscribers; `MediaRelay::on_udp_frame`
|
|
|
|
|
|
also skips non-subscribed senders). The core client gates `sync_remote_streams` on
|
|
|
|
|
|
`voice_subscribed_`, tears down all remote decoders + stops local streams on leave, and
|
|
|
|
|
|
re-syncs from the session model on join. All three clients (Windows/macOS/iOS) rewired
|
|
|
|
|
|
their Join/Leave Voice button to call `joinVoice`+start mic / `leaveVoice`+core stops mic.
|
|
|
|
|
|
The configured input mode (PTT/VAD/AlwaysOn) takes effect on join — no extra mic button.
|
|
|
|
|
|
Text chat works regardless of voice subscription. **Apple clients not yet compile-verified
|
|
|
|
|
|
(Windows environment).**
|
|
|
|
|
|
2. **Channel edit dialog now shows the channel's actual current settings.** The read struct
|
|
|
|
|
|
`vc_channel` (`voicecat.h`) was missing `sort_order` and `audio` fields — only the write
|
|
|
|
|
|
struct `vc_channel_info` had them. Extended `vc_channel` with both (additive, no ABI break),
|
|
|
|
|
|
updated the session model (`session::Channel`) and `apply_snapshot`/`apply_channel_event`
|
|
|
|
|
|
to populate them, and updated `vc_list_channels` marshaling. All three clients now build
|
|
|
|
|
|
the edit descriptor from the actual channel info instead of hardcoded defaults.
|
|
|
|
|
|
3. **Channel parameter updates now automatically restart everyone's streams.** Previously
|
|
|
|
|
|
editing a channel's audio config (codec/bitrate/sample-rate/FEC/DTX/etc.) persisted and
|
|
|
|
|
|
broadcast a `ChannelEvent::UPDATED`, but no layer restarted streams — encoders/decoders
|
|
|
|
|
|
are frozen at announce time. `handle_channel_event` (`core/src/core/client.cpp`) now
|
|
|
|
|
|
detects audio-config changes on the user's current channel and calls
|
|
|
|
|
|
`restart_active_streams_for_channel`, which stop→starts each active local stream. The
|
|
|
|
|
|
server reads the updated channel config on re-announce, and peers' `sync_remote_streams`
|
|
|
|
|
|
wire up fresh decoders at the new ssrc. The `LocalStream` struct now retains the stream
|
|
|
|
|
|
label across restarts. No server or protocol change needed.
|
|
|
|
|
|
|
2026-06-23 15:40:54 +02:00
|
|
|
|
- **[ ] Soon — jitter buffer should measure REAL arrival jitter (RFC 3550), not sender
|
|
|
|
|
|
timestamps.** `JitterBuffer::push` (`core/src/audio/audio_engine.cpp:84-108`) estimates
|
|
|
|
|
|
jitter from `gap = ts - last_push_ts_`, where `ts` is the **sender's timestamp** — which is
|
|
|
|
|
|
perfectly regular (`ls.timestamp += samples` every frame, independent of when the packet is
|
|
|
|
|
|
actually sent). So `diff` is always ~0, `jitter_est_` stays 0, and `target_depth_ms_` is
|
|
|
|
|
|
pinned at its ~20 ms floor. The buffer is therefore **blind to real network/arrival jitter
|
|
|
|
|
|
and to bursty senders** — it never deepens. Combined with the playout deliberately seeding
|
|
|
|
|
|
to near-zero depth (`on_playback`, ~line 715), the receiver tolerates only a *steady*
|
|
|
|
|
|
sender. This is exactly why the iOS mic needed a send-side pacing cushion (below) and why
|
|
|
|
|
|
genuine network jitter would also cause underruns. **Fix:** measure inter-arrival jitter
|
|
|
|
|
|
the RFC 3550 way — `D = (arrival_j - arrival_i) - (ts_j - ts_i)` using a wall-clock arrival
|
|
|
|
|
|
stamp captured in `push()` — and drive `target_depth_ms_` off that EWMA (keep the existing
|
|
|
|
|
|
marker/silence-gap outlier rejection). Then the receiver absorbs bursts itself and the iOS
|
|
|
|
|
|
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
|
|
|
|
|
|
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
|
|
|
|
|
|
|
2026-06-24 13:14:01 +02:00
|
|
|
|
- **Done (2026-06-24):** **Windows PTT can now work system-wide (in the background).** Previously
|
|
|
|
|
|
the PTT key was focus-scoped (WinForms `KeyDown`/`KeyUp`, dead the moment the window lost
|
|
|
|
|
|
focus). Added an AV-safe global path using the **Raw Input API** (`RegisterRawInputDevices` +
|
|
|
|
|
|
`WM_INPUT` with `RIDEV_INPUTSINK`) — *not* a `WH_KEYBOARD_LL` low-level hook, which is the
|
|
|
|
|
|
keylogger pattern AV heuristics flag (worse for our unsigned MinGW binary). New
|
|
|
|
|
|
`clients/windows/VoiceCat.App/Native/RawInput.cs` (P/Invoke + structs); `MainForm` overrides
|
|
|
|
|
|
`OnHandleCreated`/`OnHandleDestroyed`/`WndProc` to register the keyboard sink and handle
|
|
|
|
|
|
`WM_INPUT`, gates the focus-scoped `KeyDown`/`KeyUp` handlers off when system-wide is on, makes
|
|
|
|
|
|
the `Deactivate` force-release conditional, and adds a `GetAsyncKeyState` watchdog on the pump
|
|
|
|
|
|
timer so a missed key-up (RDP/lock-screen focus switch) can't leave PTT stuck. New
|
|
|
|
|
|
`VoiceSettings.SystemWidePtt` (default ON) with a "Works in the background (system-wide)"
|
|
|
|
|
|
checkbox in the Audio settings PTT section. Build green (`dotnet build`, 0 warnings). **Next
|
|
|
|
|
|
(manual):** verify background PTT against a live server, and confirm the binary trips no AV
|
|
|
|
|
|
keyboard-hook detection.
|
|
|
|
|
|
|
2026-06-23 21:11:03 +02:00
|
|
|
|
- **Done (2026-06-23):** **Fixed: receive-side noise reduction silently skipped on stereo mic
|
|
|
|
|
|
streams (regression from stereo-mic capture below).** The per-listener NR toggle
|
|
|
|
|
|
(`vc_set_remote_stream(... noise_reduction)`) did nothing on Windows/macOS/iOS — the UI and
|
|
|
|
|
|
the whole C-ABI→core path were correctly wired, but the decode loop gated the RNNoise pass on
|
|
|
|
|
|
`dec_channels == 1` (`core/src/audio/audio_engine.cpp`), an old proxy for "this stream is
|
|
|
|
|
|
voice" that assumed *stereo ⇒ screen-share*. The stereo-mic commit broke it: a stereo mic with
|
|
|
|
|
|
**send-side NR off** transmits stereo Opus, so the receiver decoded `dec_channels == 2` and
|
|
|
|
|
|
skipped NR entirely (gain/mute have no channel guard, which is why only NR looked broken).
|
|
|
|
|
|
**Fix:** thread the stream *kind* through `init_recv_stream` into `RemoteStream::is_voice`
|
|
|
|
|
|
(set from `si.kind() == STREAM_MIC` in `client.cpp`), gate receive NR on `is_voice` instead of
|
|
|
|
|
|
channel count, and fold a stereo voice frame to mono → denoise → duplicate back across both
|
|
|
|
|
|
channels in place (symmetric with the send-side downmix; RNNoise is mono-only). A stereo voice
|
|
|
|
|
|
stream now plays mono while NR is on; a screen-audio share is never touched. New test
|
|
|
|
|
|
`tests/test_recv_noise_reduction.cpp` drives `AudioEngine` and asserts a stereo voice stream's
|
|
|
|
|
|
noise floor collapses with NR on (RMS 1046 → 0.1) while a screen-audio share stays unchanged
|
|
|
|
|
|
(RMS ≈ 1015). Full `ctest --preset dev` green — **29/29**. Docs: voice.md §10. Clients need no
|
|
|
|
|
|
change (shared-core fix). Not yet re-verified two-client E2E on real hardware.
|
|
|
|
|
|
|
2026-06-23 20:48:26 +02:00
|
|
|
|
- **Done (2026-06-23):** **Stereo mic capture on Windows & macOS desktop clients.** Both
|
|
|
|
|
|
desktop mics were hard-mono: `ensure_audio_running()` defaults `capture_channels = 1` and
|
|
|
|
|
|
neither client ever called `vc_set_capture_channels` (only iOS did). Added a **"Stereo
|
|
|
|
|
|
microphone" toggle** to each client's Audio settings (off by default, persisted —
|
|
|
|
|
|
`VoiceSettings.StereoMic` on Windows, `MainWindowController.stereoMic` /
|
|
|
|
|
|
`voice.stereoMic` UserDefaults on macOS). It's applied to the core when the mic stream
|
|
|
|
|
|
starts (stored on the stream before the announce round-trip, so the first device open picks
|
|
|
|
|
|
it up) and live in settings via `vc_set_capture_channels` + `vc_audio_restart`. Exposed both
|
|
|
|
|
|
ABI calls in the Windows interop (`NativeMethods`/`VoiceCatClient`); the macOS wrapper already
|
|
|
|
|
|
had them. **Core fix:** `encode_and_send_frame` (`core/src/core/client.cpp`) now folds a
|
|
|
|
|
|
stereo mic frame to mono when the channel is mono — previously the `channels == 2` branch
|
|
|
|
|
|
encoded interleaved L/R directly even on a mono channel, feeding a mono `opus_encode` 2× its
|
|
|
|
|
|
samples (wrong pitch / garbage). Real stereo still only reaches the wire on a **stereo
|
|
|
|
|
|
channel** (encoder channel count = channel's Opus mode); on a mono channel the mic is cleanly
|
|
|
|
|
|
downmixed. Test: `test_stereo_mic_mono_channel` in `tests/test_vad_ptt_devices.cpp`. Full
|
|
|
|
|
|
`ctest --preset dev` green — 28/28. macOS Xcode build not compiled here (Windows host); the
|
|
|
|
|
|
Swift changes follow existing `nrChanged`/`setInputDevice` patterns. Docs: voice.md §8.
|
|
|
|
|
|
|
2026-06-23 17:47:49 +02:00
|
|
|
|
- **Done (2026-06-23):** **Fixed iOS dual-stream / crackly mic — core opened a second
|
|
|
|
|
|
(miniaudio) capture device alongside the AVAudioEngine tap.** Symptom: with two clients in
|
|
|
|
|
|
a channel, the remote end heard the iOS mic **twice** and crackly. With Voice Chat + a BT
|
|
|
|
|
|
headset, both the BT mic and the internal mic were captured; with Stereo Mic, both a mono
|
|
|
|
|
|
and a stereo copy of the internal mic were sent simultaneously. Root cause is a timing gap
|
|
|
|
|
|
in `vc_client::ensure_audio_running()` (`core/src/core/client.cpp`): `external_capture` was
|
|
|
|
|
|
only set when a MIC stream already existed, but `ensure_audio_running` is also called from
|
|
|
|
|
|
`sync_remote_streams` (triggered by the post-auth `ServerStateSnapshot`) **before** the user
|
|
|
|
|
|
joins voice — so with no MIC stream, `external_capture` stayed `false` and
|
|
|
|
|
|
`AudioEngine::start()` opened a real miniaudio capture device. Later the user joined voice →
|
|
|
|
|
|
`IOSAudioEngine.startMic` installed the AVAudioEngine input tap → `feedPcm` →
|
|
|
|
|
|
`inject_capture` → `on_capture_frame`. The miniaudio device was still open (the engine was
|
|
|
|
|
|
already `running()`, so the later `ensure_audio_running` early-returned and never applied
|
|
|
|
|
|
`external_feed`), and `on_capture_frame` encodes+sends every frame with **no deduplication**
|
|
|
|
|
|
→ the mic was sent twice. The two unsynchronized capture clocks interleaving in the encoder
|
|
|
|
|
|
is the crackle; the mono miniaudio device + stereo AVAudioEngine tap is the "mono and stereo
|
|
|
|
|
|
at the same time" on Stereo Mic.
|
|
|
|
|
|
- **Fix 1 (core, `core/src/core/client.cpp:ensure_audio_running`):** force
|
|
|
|
|
|
`p.external_capture = true` whenever `external_playback_` is set. In iOS unified mode the
|
|
|
|
|
|
core must never open a hardware capture device — the AVAudioEngine owns the only mic path.
|
|
|
|
|
|
No-op on desktop (`external_playback_` is never set there).
|
|
|
|
|
|
- **Fix 2 (iOS, `clients/apple/iOS/VoiceCatiOS/AppState.swift`):** move
|
|
|
|
|
|
`client.setExternalPlayback(true)` from the `authResult` handler to **before**
|
|
|
|
|
|
`client.connect(...)`. The server sends `AuthResult` immediately followed by
|
|
|
|
|
|
`ServerStateSnapshot`; `handle_server_state` runs `ensure_audio_running` on the io thread
|
|
|
|
|
|
before the main thread drains `authResult`, so setting the flag post-auth raced. Setting it
|
|
|
|
|
|
pre-connect guarantees `external_playback_` is true before any message is processed —
|
|
|
|
|
|
eliminating the playback-device race too (the mixer timer + AVAudioEngine playback path +
|
|
|
|
|
|
VPIO AEC reference are correct from the first frame).
|
|
|
|
|
|
- **Verify:** `cmake --build --preset dev` clean; `ctest --preset dev` = 24/28 — the 4
|
|
|
|
|
|
failures (`vad_ptt_devices`, `external_pcm`, `frame_ms_reframe`, `channel_samplerate`) are
|
|
|
|
|
|
a **pre-existing** teardown `mutex lock failed` race, reproduced identically with the
|
|
|
|
|
|
changes stashed. `external_playback` (the one test exercising this code path) **passes**.
|
|
|
|
|
|
No xcframework rebuild needed (no new symbols). **Next (manual, on device):** two clients
|
|
|
|
|
|
in a channel — Voice Chat + BT, and Stereo Mic — confirm the remote end hears the iOS mic
|
|
|
|
|
|
once, clean (no duplicate, no crackle); confirm the iOS user hears the remote user cleanly
|
|
|
|
|
|
with AEC working in Voice Chat.
|
|
|
|
|
|
|
2026-06-23 15:40:54 +02:00
|
|
|
|
- **Done (2026-06-23):** **Fixed iOS mic flutter / crackle / octave-up.** The iOS mic was
|
|
|
|
|
|
unusable: a consistent ~40–60 ms flutter with volume fade ("talking through a slow fan") on
|
|
|
|
|
|
every preset. Root cause: the core sends each captured frame **synchronously**
|
|
|
|
|
|
(`on_capture_frame` → `encode_and_send_frame`, no send pacer), so packet cadence == capture
|
|
|
|
|
|
cadence; and the receiver's playout keeps **near-zero buffering** by design and its jitter
|
|
|
|
|
|
estimate is blind to arrival timing (see RFC-3550 item above). That's smooth only for a
|
|
|
|
|
|
*steady* sender (desktop miniaudio = steady 20 ms), but the iOS `AVAudioEngine` input tap
|
|
|
|
|
|
delivers ~2 frames per ~40 ms callback (more under VPIO) → bursty → receiver underruns → PLC
|
|
|
|
|
|
fade.
|
|
|
|
|
|
- **Fix (iOS-only, `clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift`):** the
|
|
|
|
|
|
mic tap converts to 48 kHz int16 and writes a lock-free SPSC ring; a 20 ms feed pump
|
|
|
|
|
|
drains it and calls `feedPcm` at a **steady** cadence so packets leave the core every
|
|
|
|
|
|
20 ms (what the receiver expects). The pump **primes a small prebuffer cushion**
|
|
|
|
|
|
(`PumpState.targetFrames`, 3 frames ≈ 60 ms, self-healing up to ~120 ms on underrun)
|
|
|
|
|
|
before releasing, so the tap's bursts can't drain it to empty. Two correctness rules
|
|
|
|
|
|
(each had bit us): never read a partial frame (`read` consumes what it returns →
|
|
|
|
|
|
discarding partials caused crackle), and rebuild the pump with the current channel count
|
|
|
|
|
|
every `rebuild()` (a frozen channel count fed mono-as-stereo = octave-up on a Stereo→Voice
|
|
|
|
|
|
Chat switch). Trade-off: ~60–120 ms added mic-send latency — unavoidable when de-bursting
|
|
|
|
|
|
for a near-zero-buffer receiver; the RFC-3550 fix above would let us shrink it.
|
|
|
|
|
|
- **Verify:** `xcodebuild` Debug **BUILD SUCCEEDED** (iOS Simulator, arm64). Audible test
|
|
|
|
|
|
requires a real device (simulator has no real mic route): mic should be smooth on Voice
|
|
|
|
|
|
Chat / Mono Mic / Stereo Mic, including switching presets while live (no octave).
|
|
|
|
|
|
|
2026-06-23 14:25:30 +02:00
|
|
|
|
- **Done (2026-06-23):** **Fixed Apple client link failure (stale xcframework missing
|
|
|
|
|
|
RNNoise).** Both `VoiceCatMac` and `VoiceCatiOS` failed to link with `Undefined symbols for
|
|
|
|
|
|
architecture arm64: _rnnoise_create / _rnnoise_destroy / _rnnoise_process_frame`. Root
|
|
|
|
|
|
cause: `clients/apple/scripts/build-xcframework.sh` merged vcpkg deps into the fat static
|
|
|
|
|
|
lib but NOT the locally-built vendored `librnnoise.a` (a CMake target from
|
|
|
|
|
|
`third_party/rnnoise/`, linked privately into `voicecat` via `VOICECAT_HAS_NS` — not a
|
|
|
|
|
|
vcpkg dep). The xcframework had been rebuilt at 14:17 after the RNNoise commit but still
|
|
|
|
|
|
omitted the symbols, so every slice's `libvoicecat-fat.a` referenced `_rnnoise_*` with no
|
|
|
|
|
|
defining object. The iOS slices were also stale (pre-rnnoise) and absent from the
|
|
|
|
|
|
xcframework entirely.
|
|
|
|
|
|
- **Fix:** `build-xcframework.sh` now collects `.a` files from `build/<preset>/lib/`
|
|
|
|
|
|
(excluding `libvoicecat*`) in addition to `vcpkg_installed/<triplet>/lib/`, so vendored
|
|
|
|
|
|
CMake-target static libs like `librnnoise.a` are merged into the fat lib. Future-proof:
|
|
|
|
|
|
any new vendored static-lib target landing in `build/<preset>/lib/` is picked up
|
|
|
|
|
|
automatically. README "Fat static library" section updated.
|
|
|
|
|
|
- **Verify:** rebuilt `VoiceCatCore.xcframework --all` → all 3 slices (macos-arm64,
|
|
|
|
|
|
ios-arm64, ios-arm64-simulator) now carry 10 `_rnnoise_*` symbols each; fat lib
|
|
|
|
|
|
~30 MB → ~33 MB. `xcodebuild` Debug **BUILD SUCCEEDED** for `VoiceCatMac`,
|
|
|
|
|
|
`VoiceCatiOS` (iphonesimulator arm64), and `VoiceCatiOS` (iphoneos arm64,
|
|
|
|
|
|
`CODE_SIGNING_ALLOWED=NO`). No core/ABI/proto changes — xcframework artifact only.
|
|
|
|
|
|
|
2026-06-23 13:30:54 +02:00
|
|
|
|
- **Done (2026-06-23):** **Remote-stream noise suppression — real backend (RNNoise).** The
|
|
|
|
|
|
two-sided NR plumbing (`RemoteStream::recv_ns` + `vc_set_remote_stream(... noise_reduction)`)
|
|
|
|
|
|
was wired but **inert** — `ApmProcessor::create()` returned a no-op passthrough, because the
|
|
|
|
|
|
originally-planned `webrtc-audio-processing` won't build on Windows/macOS. Replaced with
|
|
|
|
|
|
**RNNoise** (BSD-3 + CC0), vendored at `third_party/rnnoise/` (the vcpkg port is `!windows
|
|
|
|
|
|
!arm`), built as a standalone C static lib + `VOICECAT_HAS_NS`. One `RnnoiseProcessor`
|
|
|
|
|
|
(`core/src/audio/apm_processor.cpp`) now backs **both** NR paths:
|
|
|
|
|
|
- **Receive-side** (per-listener, per-`ssrc`): lit up automatically via the factory; gated to
|
|
|
|
|
|
mono streams (`audio_engine.cpp` ~L791).
|
|
|
|
|
|
- **Send-side** (mic, new): `vc_set_input_noise_reduction(client, enable)` ABI +
|
|
|
|
|
|
`vc_client::mic_ns_`, run before input gain/VAD in `on_capture_frame`. A stereo mic is
|
|
|
|
|
|
downmixed to mono **only when NR is on**; with NR off a stereo mic keeps full stereo.
|
|
|
|
|
|
- RNNoise is mono/48 kHz/480-sample; our clock is fixed 48 kHz and Opus frame sizes are all
|
|
|
|
|
|
multiples of 480, so no resampling. RT-safe: alloc at construction, lock-free in the callback.
|
|
|
|
|
|
- **Verify status:** `ctest --preset dev` green — **28/28** (new `noise_suppression` test:
|
|
|
|
|
|
feeds white noise through `ApmProcessor::create()`, measures **99.9%** RMS reduction). Build
|
|
|
|
|
|
clean on the `dev` MinGW preset. **Next (manual):** add the on/off toggles to the client UIs
|
|
|
|
|
|
(Windows Audio Settings dialog, macOS/iOS settings) calling the two ABIs; build `windows-client`
|
|
|
|
|
|
+ `apple-dev` presets to confirm RNNoise compiles under MinGW-DLL and arm64; two-client E2E.
|
|
|
|
|
|
|
2026-06-23 12:56:09 +02:00
|
|
|
|
- **Done (2026-06-23):** **Aux outgoing stream (mic + a second input device) — Windows + macOS.**
|
|
|
|
|
|
Users can now transmit a second hardware input device (e.g. line-in / aux) alongside the mic, with
|
|
|
|
|
|
its own device picker and volume, from Audio Settings. **No core/ABI/proto changes** — the aux is
|
|
|
|
|
|
a `VC_STREAM_AUX_DEVICE` stream started with `external_feed=1`, captured client-side, and fed via
|
|
|
|
|
|
`vc_stream_feed_pcm` (the same external-feed pipeline screen-audio uses). Per-kind `local_streams_`
|
|
|
|
|
|
already allows mic + screen + one aux to coexist; volume is a client-side gain multiply (the core's
|
|
|
|
|
|
`vc_set_input_gain` is mic-only/global). The aux is always-on (the core never gates `AUX_DEVICE` on
|
|
|
|
|
|
VAD/PTT) and is tied to the voice session (started on Join Voice when enabled, stopped on Leave).
|
|
|
|
|
|
- **Windows:** new `Audio/InputDeviceCapture.cs` (WASAPI shared-mode capture from a real input
|
|
|
|
|
|
endpoint via `IMMDevice.Activate(IAudioClient)`, 48 kHz/s16, 20 ms frames) + `InputDeviceEnumerator`
|
|
|
|
|
|
(WASAPI capture-endpoint list — separate from the core's miniaudio ids). Aux section in
|
|
|
|
|
|
`AudioSettingsForm.cs` (enable checkbox, device combo, refresh, volume slider, accessible names,
|
|
|
|
|
|
live-apply + Cancel revert via callbacks). Lifecycle in `MainForm.cs` (`_auxStreamId` +
|
|
|
|
|
|
`InputDeviceCapture`). Persisted in `VoiceSettings.cs` (`AuxEnabled/AuxDeviceId/AuxGain`).
|
|
|
|
|
|
- **macOS:** new `Audio/InputDeviceCapture.swift` (AVAudioEngine input-node tap pinned to the chosen
|
|
|
|
|
|
Core Audio device via `kAudioOutputUnitProperty_CurrentDevice`; AVAudioConverter → 48 kHz int16;
|
|
|
|
|
|
20 ms framing modelled on `ScreenAudioCapture`) + `InputDeviceEnumerator` (Core Audio device list
|
|
|
|
|
|
by stable UID). Aux section in `SettingsWindowController.swift`; lifecycle + UserDefaults
|
|
|
|
|
|
persistence (`voice.aux*`) in `MainWindowController.swift`. New file added to `project.pbxproj`.
|
|
|
|
|
|
- **Verify status:** Windows C# solution builds clean (0 warn/0 err); `ctest` core suite unchanged
|
|
|
|
|
|
(no core edits). **Next (manual):** on a Mac, build `VoiceCatMac.xcodeproj`; then two-client E2E —
|
|
|
|
|
|
enable aux on a second input device, confirm two distinct streams for the sender and that the aux
|
|
|
|
|
|
volume slider moves the aux level independently of the mic; confirm persistence across relaunch.
|
|
|
|
|
|
|
2026-06-23 03:35:26 +02:00
|
|
|
|
- **Done (2026-06-23):** **Input-settings persistence, mic input gain, + two iOS bugs (all 3
|
|
|
|
|
|
clients).** Four fixes:
|
|
|
|
|
|
1. **Input settings now persist.** Transmission mode (VAD/PTT/Always-On), VAD threshold, and the
|
|
|
|
|
|
new mic gain were applied to the core + UI but never saved, so every relaunch reset to VAD
|
|
|
|
|
|
defaults. Each client now persists them and re-applies on connect: iOS via `UserDefaults`
|
|
|
|
|
|
(`SessionState.loadAndApplyVoiceSettings` + setter writes, keys `voice.*`); macOS via
|
|
|
|
|
|
`UserDefaults` (`MainWindowController` `didSet` + `loadPersistedAudioSettings`, also restores
|
|
|
|
|
|
the VAD slider from the stored threshold); Windows via new
|
|
|
|
|
|
`VoiceCat.App/Models/VoiceSettings.cs` (JSON at `%AppData%\VoiceCat\voice.json`, mirrors
|
|
|
|
|
|
`FeedbackSettings`) loaded/applied in `MainForm`.
|
|
|
|
|
|
2. **Microphone input gain.** New global send-side API `vc_set_input_gain` (voicecat.h →
|
|
|
|
|
|
`client.cpp::on_capture_frame`, applied to MIC PCM before the VAD gate, clamped to int16) plus
|
|
|
|
|
|
Swift (`setInputGain`) and C# (`SetInputGain`) bindings. Mic-volume slider (0–300 %, default
|
|
|
|
|
|
100 %) added to all three clients' input settings, persisted with the rest.
|
|
|
|
|
|
3. **iOS chat send fixed.** `ChatView` called `sendText(scope:.channel)` with no `targetId` (→ 0),
|
|
|
|
|
|
so channel messages went nowhere; now passes `session.currentChannelId`.
|
|
|
|
|
|
4. **iOS per-user tuning reachable via VoiceOver.** The tuning sheet was long-press
|
|
|
|
|
|
`.contextMenu` only (invisible to VoiceOver); `UserRow` now also exposes the same buttons as
|
|
|
|
|
|
`.accessibilityActions` (no visual change), so the actions rotor reaches tuning + admin actions.
|
|
|
|
|
|
- **Verified:** core `cmake --build --preset dev` clean; `ctest --preset dev` = 24/27 (the 3
|
|
|
|
|
|
failures — `external_pcm`, `frame_ms_reframe`, `channel_samplerate` — are a pre-existing
|
|
|
|
|
|
teardown crash on this machine, reproduced identically with the changes stashed). xcframework
|
|
|
|
|
|
rebuilt (`--all`); **VoiceCatMac** and **VoiceCatiOS** (arm64 sim) → BUILD SUCCEEDED;
|
|
|
|
|
|
`VoiceCat.Interop` (`dotnet build`) succeeded. **Windows App not built** (WinForms
|
|
|
|
|
|
net10.0-windows can't build on macOS) — changes follow existing patterns; needs a Windows
|
|
|
|
|
|
build + manual check.
|
|
|
|
|
|
- **Next (manual):** on each client, set PTT + non-default VAD/mic-gain, relaunch → settings
|
|
|
|
|
|
restored; boost a quiet mic and confirm others hear it louder; iOS send a channel message;
|
|
|
|
|
|
iOS VoiceOver → focus a user → actions rotor opens tuning.
|
|
|
|
|
|
|
2026-06-22 20:02:20 +02:00
|
|
|
|
- **Done (2026-06-22):** **Fixed growing voice latency (jitter-buffer depth ratchet).** Symptom:
|
|
|
|
|
|
end-to-end latency grew to multiple seconds and "drifted backward," reset only by leaving/
|
|
|
|
|
|
rejoining voice (DTX/FEC/DRED on, 10% loss). Root cause was **not** the codec settings (10% loss
|
|
|
|
|
|
is just an `OPUS_SET_PACKET_LOSS_PERC` encoder hint; FEC/DRED add no standing latency) but the
|
|
|
|
|
|
receiver playout logic in `core/src/audio/audio_engine.cpp`: the playout clock free-ran in real
|
|
|
|
|
|
time while the sender omitted silence from its timestamps and set **no header flags at all**, and
|
|
|
|
|
|
the only correction snapped the clock to the *oldest* buffered frame (could only *add* latency) —
|
|
|
|
|
|
with `target_depth_ms_` computed but never enforced, so latency could only grow or be reset.
|
|
|
|
|
|
**Fix:** bounded-depth playout — (re)seed to the *leading edge* (newest frame) on start/marker/
|
|
|
|
|
|
starve, and **frame-skip catch-up** that trims a backlog beyond `target + hysteresis` (the missing
|
|
|
|
|
|
downward force). Plus hardening: adaptive late-drop window, talkspurt `kFlagMarker`/`kFlagDtx`
|
|
|
|
|
|
now actually stamped by the sender (`client.cpp` send path) and consumed on recv, EWMA outlier
|
|
|
|
|
|
rejection (silence gaps/stragglers no longer poison the estimate), duplicate counting, ring-
|
|
|
|
|
|
underrun diagnostics (`stream_underruns`/`stream_duplicates`). New regression test
|
|
|
|
|
|
`tests/test_jitter_depth.cpp` asserts depth stays bounded (<200 ms) while arrivals outrun playout
|
|
|
|
|
|
for ~4 s. `ctest --preset dev` green — **27/27**. Docs: `docs/voice.md` §5 rewritten.
|
|
|
|
|
|
- **Next (manual E2E):** two clients in a channel, DTX/FEC/DRED on — talk in alternating bursts
|
|
|
|
|
|
for several minutes and confirm latency stays low/stable (no backward drift, no rejoin needed).
|
|
|
|
|
|
|
2026-06-22 15:20:11 +02:00
|
|
|
|
- **Windows done / Apple awaiting Mac build (2026-06-22):** **Event sound effects + optional
|
|
|
|
|
|
text-to-speech for all clients.** Clients now play a cue per session event and can optionally
|
|
|
|
|
|
speak it (TTS off by default; when on it announces joins/leaves and reads message/PM bodies).
|
|
|
|
|
|
One canonical event→sound mapping (defined off the shared C ABI `vc_event` stream) is mirrored
|
|
|
|
|
|
across all three clients; `self` vs others is `user_id == self_user_id`, and outgoing messages
|
|
|
|
|
|
echo back as events so sent/recv cues need no separate send-path hook. Conservative defaults
|
|
|
|
|
|
(join/leave, channel/PM sent+recv, login, logout, connection-lost, mic on/off ON; per-utterance
|
|
|
|
|
|
self voice-activity `va_start/va_stop` and the PTT cue OFF). WAVs ship from `assets/sounds/`.
|
|
|
|
|
|
- **Windows (built + verified):** new `VoiceCat.App/Notifications/` (`FeedbackSettings` →
|
|
|
|
|
|
`%AppData%\VoiceCat\feedback.json`, `SoundPlayerPool` via `System.Media.SoundPlayer`,
|
|
|
|
|
|
`SpeechAnnouncer` via the **Prismatoid** NuGet 0.3.0, `EventFeedback` dispatcher); hooks in
|
|
|
|
|
|
`Forms/MainForm.cs`; `Forms/NotificationSettingsForm.cs` under a new **Settings ▸ Notifications**
|
|
|
|
|
|
menu. `.csproj` adds the Prismatoid PackageRef and copies the WAVs into `sounds\`. `dotnet build`
|
|
|
|
|
|
clean; WAVs + `Prismatoid.dll` confirmed in output. Note: `SoundPlayer` has no gain control, so
|
|
|
|
|
|
volume is honoured as a mute gate (0 = silent) — swap to NAudio if finer/overlap control is needed.
|
|
|
|
|
|
- **macOS + iOS (written, NOT yet built — needs a Mac):** shared `Sources/VoiceCatCore/Feedback/`
|
|
|
|
|
|
(`SoundEvent`, `EventFeedback` = `AVAudioPlayer` pool + native `AVSpeechSynthesizer`,
|
|
|
|
|
|
`FeedbackSettings` over `UserDefaults`); WAVs copied into `Sources/VoiceCatCore/Sounds/` and
|
|
|
|
|
|
bundled via `Package.swift` `resources: [.process("Sounds")]` (`Bundle.module`). Hooks: iOS
|
|
|
|
|
|
`SessionState.handleEvent` (+ split `userJoined`/`userLeft`, added a `.disconnected` cue case),
|
|
|
|
|
|
`AppState` auth-success login cue, PTT cue in `setPushToTalk`; macOS `MainWindowController`
|
|
|
|
|
|
handlers + NSEvent PTT monitor. Settings UI: iOS `SettingsView` Notifications section
|
|
|
|
|
|
(`@AppStorage`), macOS `SettingsWindowController` checkboxes + volume slider. No `.pbxproj`
|
|
|
|
|
|
edits needed (shared files are SPM-managed; app files already in the projects).
|
|
|
|
|
|
- **Next:** on a Mac, `clients/apple/scripts/build-xcframework.sh --all` then build
|
|
|
|
|
|
VoiceCatMac/VoiceCatiOS; fix compile fallout. **Watch the iOS audio session:** cues/TTS play over
|
|
|
|
|
|
the live VPIO `playAndRecord` session — verify they mix and don't duck/interrupt the call or get
|
|
|
|
|
|
silenced by the mute switch (most likely bug site). Then run `ctest --preset dev` (unchanged —
|
|
|
|
|
|
no core/server code touched).
|
|
|
|
|
|
|
2026-06-22 12:48:13 +02:00
|
|
|
|
- **Done (2026-06-22):** **UDP media now shares the TCP port (self-host port-forward fix).** Symptom: a
|
|
|
|
|
|
remote self-hosted server (`iamtalon.me:8384`, TCP+UDP 8384 forwarded) accepted TCP connections but
|
|
|
|
|
|
passed no voice. Root cause: `Config::media_port` defaulted to `0` = OS-assigned, and `main.cpp`'s
|
|
|
|
|
|
`--port` only set `bind_port` (TCP) — so the UDP relay bound a *random high port*, advertised it to
|
|
|
|
|
|
clients in HELLO (`udp_port`), and clients sent voice there. With only `8384/udp` forwarded those
|
|
|
|
|
|
packets were dropped → connect OK, no audio. This contradicted `docs/deployment.md` ("Control and media
|
|
|
|
|
|
share one port number on TCP+UDP"). **Fix (`server/src/server.cpp`):** media follows bind_port when
|
|
|
|
|
|
`media_port == 0` — `media_want = cfg_.media_port != 0 ? cfg_.media_port : cfg_.bind_port`. The
|
|
|
|
|
|
`0 = OS-assigned` escape hatch survives when `bind_port` is also 0, so tests that bind ephemeral ports
|
|
|
|
|
|
are unaffected (kept the logic in server.cpp rather than hardcoding 8384 as the default, which would
|
|
|
|
|
|
collide parallel tests on UDP 8384). Banner now reads `TCP :8384 UDP :8384`. Build + `ctest --preset
|
|
|
|
|
|
dev` green (24/24); live-verified banner with `--port 8390` → `UDP :8390`. **Action for self-hosters:**
|
|
|
|
|
|
redeploy and confirm the startup banner shows matching TCP/UDP ports; the existing single forward rule
|
|
|
|
|
|
is now correct. If voice still fails, watch the server's rate-limited `[media] dropped frames —
|
|
|
|
|
|
unmapped-endpoint=…` line (NAT source-port rewrite would be the next suspect).
|
|
|
|
|
|
|
2026-06-23 02:45:53 +02:00
|
|
|
|
- **Done (2026-06-23, Swift-only — no core/ABI change; awaiting on-device verification):** **iOS audio
|
|
|
|
|
|
stack unified — one always-external `AVAudioEngine`, miniaudio dropped on iOS.** The iOS audio path was
|
|
|
|
|
|
a fragile hybrid: Voice-Chat-class presets ran a native VPIO `AVAudioEngine` (core external) while
|
|
|
|
|
|
Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every bug lived in the seam
|
|
|
|
|
|
(lingering miniaudio capture unit fighting VPIO, the `audioRestart` ordering dance, the route-change
|
|
|
|
|
|
"glitching" loop, stereo↔mono stickiness, "can't hear anyone"), and switching presets/routes mid-call
|
|
|
|
|
|
routinely dropped input, output, or both. **Fix: drive *all* iOS audio through one `AVAudioEngine` with
|
|
|
|
|
|
the core fully external at all times** — `vc_set_external_playback(1)` once at connect, every MIC stream
|
|
|
|
|
|
`external_feed=1`, mic via `vc_stream_feed_pcm`, playback via `vc_set_mixed_output_sink`.
|
|
|
|
|
|
- `IOSVoiceProcessingEngine.swift` → **`IOSAudioEngine`** (same file): always-on `AVAudioSourceNode`
|
|
|
|
|
|
playback (runs whenever connected, so remote audio plays before you join voice); conditional mic tap;
|
|
|
|
|
|
VPIO + AGC toggled per config. One private `rebuild()` (stop → set VPIO → install tap → start) backs
|
|
|
|
|
|
`startListening`/`stop`/`startMic`/`stopMic`/`reconfigure`/`setCaptureChannels`. Kept the `PCMRing`
|
|
|
|
|
|
and ring-stats diagnostics.
|
|
|
|
|
|
- `IOSAudioRouter`: presets cut from seven to **four** — Voice Chat (VPIO mono, system output),
|
|
|
|
|
|
Stereo Mic / Mono Mic (internal built-in mic regardless of output, A2DP-capable, no VPIO), Advanced
|
|
|
|
|
|
(manual). New persisted `voiceProcessingEnabled` (master AEC+NS) + `agcEnabled`; setters now call
|
|
|
|
|
|
`IOSAudioEngine.reconfigure()` instead of `client.audioRestart()` + `reconcileVoicePath`. Kept the
|
|
|
|
|
|
proven AVAudioSession recipes (category/mode/options, stereo capsule, `applyA2dpSpeakerFallback`).
|
|
|
|
|
|
- `AudioSessionManager` slimmed (drops `client`/`activeMicStreamId`/`reconcileVoicePath`; adds
|
|
|
|
|
|
`isActive`); interruption-end & device-change now `reconfigure()` the engine. `SessionState`
|
|
|
|
|
|
`doStartMicStream`/`stopMicStream` collapsed to start-stream + `startMic`/`stopMic` (no
|
|
|
|
|
|
`setExternalPlayback`/`audioRestart` toggling); `reconcileVoicePath` deleted. `AppState` sets external
|
|
|
|
|
|
playback + `startListening` at connect, `stop()` at disconnect. `SettingsView` → four presets +
|
|
|
|
|
|
Advanced VPIO/AGC toggles.
|
|
|
|
|
|
- **No core/ABI/test change** — relies on the already-shipped `vc_set_external_playback` /
|
|
|
|
|
|
`external_feed` / `vc_set_mixed_output_sink` / `vc_stream_feed_pcm` path (`test_external_pcm`,
|
|
|
|
|
|
`test_external_playback`). `xcodebuild` iOS device Debug **BUILD SUCCEEDED**. **Rebuild the
|
|
|
|
|
|
xcframework is NOT required** (no new symbols).
|
|
|
|
|
|
- **Next (user, on device):** two iPhones in a channel — verify BOTH directions survive every
|
|
|
|
|
|
transition and are never silent unless intended: Voice Chat (no echo, NR), listen-only before joining,
|
|
|
|
|
|
join↔leave repeatedly, switch Voice Chat↔Stereo↔Mono↔Advanced *while in voice*, A2DP connect/unplug,
|
|
|
|
|
|
wired connect/unplug, phone-call interruption + resume, screen-audio share.
|
|
|
|
|
|
|
|
|
|
|
|
- **Superseded by the 2026-06-23 unification above (2026-06-22):** **iOS real echo cancellation / noise
|
|
|
|
|
|
suppression via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
|
2026-06-22 02:38:01 +02:00
|
|
|
|
AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core uses miniaudio's
|
|
|
|
|
|
plain RemoteIO units — so `.voiceChat` mode alone never engaged AEC. Fix moves both mic capture and
|
|
|
|
|
|
playback to a native Swift `AVAudioEngine` (`setVoiceProcessingEnabled`) on the AEC presets, with the
|
|
|
|
|
|
core in external mode.
|
|
|
|
|
|
- **Core (done, builds + tests green):** new ABI `vc_set_mixed_output_sink` + `vc_set_external_playback`
|
|
|
|
|
|
(voicecat.h PATCH→2). `AudioEngine` gains a mixer-timer thread that drives `on_playback` (decode+mix)
|
|
|
|
|
|
on a ~20 ms cadence with NO hardware playback device and ships the final mix to the mixed-output
|
|
|
|
|
|
sink; `start()` also skips the hardware capture device when the MIC stream is `external_feed`
|
|
|
|
|
|
(`AudioParams.external_capture`). New white-box test `test_external_playback` (23/24;
|
|
|
|
|
|
pre-existing `external_pcm` teardown crash on Darwin 25.5 is UNRELATED — original tree crashes too).
|
|
|
|
|
|
- **Swift (done, builds):** `VoiceCatCore` wrappers (`externalFeed` on `StreamDescriptor`,
|
|
|
|
|
|
`setMixedOutputSink`, `setExternalPlayback`); new `IOSVoiceProcessingEngine.swift` (VPIO
|
|
|
|
|
|
`AVAudioEngine`: mic tap→`feedPcm`, mixed-sink lock-free ring→`AVAudioSourceNode`);
|
|
|
|
|
|
`IOSAudioRouter.currentConfigUsesVoiceProcessing` gates the path per preset; `SessionState`
|
|
|
|
|
|
join/leave + `reconcileVoicePath()` switch between VPIO and the miniaudio path; Voice Chat defaults
|
|
|
|
|
|
to speaker; SettingsView shows AEC/NS state. **Rebuild the xcframework** before building the app:
|
|
|
|
|
|
`clients/apple/scripts/build-xcframework.sh --all` (new ABI symbols). `xcodebuild` iOS sim Debug
|
|
|
|
|
|
BUILD SUCCEEDED.
|
2026-06-22 03:43:00 +02:00
|
|
|
|
- **Post-verification fixes (2026-06-22, Swift-only — no core/ABI change):** two on-device bugs fixed.
|
|
|
|
|
|
- **Voice Chat (VPIO) silent playback:** `SessionState.doStartMicStream()` called `audioRestart()`
|
|
|
|
|
|
BEFORE `startStream`, so when the engine was already running (a remote stream had started it) it
|
|
|
|
|
|
reopened with `external_capture=false` and opened a hardware miniaudio capture device; the
|
|
|
|
|
|
announce-result restart then early-returned (engine already running) so that device was never
|
|
|
|
|
|
dropped and fought the `AVAudioEngine` VPIO unit, silencing playback. Fix: set
|
|
|
|
|
|
`setExternalPlayback` first, then `startStream` (which stores `external_feed` synchronously), THEN
|
|
|
|
|
|
`audioRestart()` — the core reopens in full external mode (no hardware devices). Added VPIO
|
|
|
|
|
|
diagnostics (graph/route formats at start; ring written/read totals at teardown).
|
|
|
|
|
|
- **Stereo Mic / Studio quiet earpiece:** the `.builtInMicBtA2dp` presets omit `.defaultToSpeaker`
|
|
|
|
|
|
(it breaks A2DP) and skip `forceSpeaker`, so with no Bluetooth connected output pinned to the quiet
|
|
|
|
|
|
receiver. New `IOSAudioRouter.applyA2dpSpeakerFallback()` overrides to the built-in speaker when no
|
|
|
|
|
|
external (A2DP/wired/AirPlay) output is present, clears the override when one is — called after
|
|
|
|
|
|
activation and on device-change route changes (`AudioSessionManager`).
|
2026-06-22 02:38:01 +02:00
|
|
|
|
- **Next (user, on device):** two iPhones on speaker, Voice Chat preset → confirm (a) no echo, (b)
|
2026-06-22 03:43:00 +02:00
|
|
|
|
background noise suppressed, (c) speaker output by default AND remote audio is now audible; then
|
|
|
|
|
|
Stereo Mic / Studio with no BT → confirm loud speaker (not earpiece), and A2DP takes over when a BT
|
|
|
|
|
|
headset connects. Tune the mixer-timer/ring sizing if there's under/overrun.
|
2026-06-22 02:38:01 +02:00
|
|
|
|
|
2026-06-21 19:51:43 +02:00
|
|
|
|
- **Done (2026-06-21):** **Docker + Linux deployment + GitHub Actions cross-build.** Added the complete Linux server
|
|
|
|
|
|
deployment story (the only missing platform — Windows and macOS already have native
|
|
|
|
|
|
binaries):
|
|
|
|
|
|
- `Dockerfile` — multi-stage (builder: `ubuntu:24.04` + vcpkg + `cmake --preset
|
|
|
|
|
|
server-release`; runtime: `ubuntu:24.04`, non-root `voicecat` user, `/data` volume,
|
|
|
|
|
|
TCP+UDP 8384). vcpkg is fetched via the GitHub archive tarball at the exact
|
|
|
|
|
|
`builtin-baseline` commit (`d46283cf…`), avoiding a full git-history clone. BuildKit
|
|
|
|
|
|
cache mounts on `/vcpkg/downloads`, `/vcpkg/buildtrees`, `/vcpkg/packages` (scoped by
|
|
|
|
|
|
`TARGETARCH`) keep rebuilds fast. Both `voicecat-server` and `voicecat-admin` are
|
|
|
|
|
|
copied into the runtime image.
|
|
|
|
|
|
- `docker-compose.yml` — single-service compose file with `restart: unless-stopped`,
|
|
|
|
|
|
named volume `voicecat-data`, and port mappings for TCP+UDP 8384. `command:` shows
|
|
|
|
|
|
how to set `--name`.
|
|
|
|
|
|
- `.dockerignore` — excludes `.git/`, `build/`, `clients/` (Swift/C# code), `docs/`,
|
|
|
|
|
|
markdown, editor config; build context is just `core/`, `server/`, `tools/`, `cmake/`,
|
|
|
|
|
|
and the three root CMake/vcpkg files.
|
|
|
|
|
|
- `deploy/linux/voicecat.service` — hardened systemd unit (non-root, `ProtectSystem`,
|
|
|
|
|
|
`NoNewPrivileges`, `AmbientCapabilities=CAP_NET_BIND_SERVICE`) for bare-metal deploys.
|
|
|
|
|
|
- Multi-arch: `docker buildx build --platform linux/amd64,linux/arm64 .` works without
|
|
|
|
|
|
any triplet override — `cmake/voicecat-toolchain.cmake` auto-detects from the host
|
|
|
|
|
|
arch cmake sees inside the buildx container.
|
|
|
|
|
|
- Quick start: `docker compose up -d` (or `docker run -d -p 8384:8384/tcp -p
|
|
|
|
|
|
8384:8384/udp -v voicecat-data:/data voicecat`). First run auto-generates identity
|
|
|
|
|
|
+ cert + DB; check logs for fingerprint + admin password.
|
|
|
|
|
|
- **GitHub Actions** (`.github/workflows/build-linux.yml`): primary cross-platform
|
|
|
|
|
|
binary build path — amd64 uses `ubuntu-24.04`, arm64 uses `ubuntu-24.04-arm`
|
|
|
|
|
|
(native, not QEMU). Triggers on push to main (when C++/cmake files change) and
|
|
|
|
|
|
manually via `workflow_dispatch`. Downloads land as 90-day artifacts.
|
|
|
|
|
|
`scripts/build-linux-binaries.sh` is the local Docker fallback (needs ~10–15 GB
|
|
|
|
|
|
free disk; suits Linux dev machines, not Windows Docker Desktop).
|
|
|
|
|
|
|
2026-06-21 17:45:28 +02:00
|
|
|
|
- **Done (2026-06-21):** **Fix permanent voice-loss bug + harden the UDP media path (protocol v2).**
|
|
|
|
|
|
Field report: two iOS users lost all audio mid-call after a bad-network blip and could not
|
|
|
|
|
|
recover even by restarting the apps. Root causes found in the UDP media path:
|
|
|
|
|
|
1. **Anti-replay window poisoned by unauthenticated packets (the trigger).**
|
|
|
|
|
|
`SodiumMediaCrypto::open()` advanced `recv_highest_` from the plaintext header `seq`
|
|
|
|
|
|
*before* verifying the AEAD tag and never rolled it back on failure. One corrupted/forged
|
|
|
|
|
|
frame (a bit-flip on flaky wifi) shoved the high-water mark far ahead, after which every
|
|
|
|
|
|
legitimate frame was rejected as "too old" — permanently. Fixed by reordering to
|
|
|
|
|
|
replay-check → authenticate → update (RFC 3711 §3.3): the window is now touched only after
|
|
|
|
|
|
a successful tag check. Regression test in `test_media_aead.cpp`
|
|
|
|
|
|
(`test_corrupted_seq_does_not_poison_window`) — fails on the old code, passes now.
|
|
|
|
|
|
2. **16-bit seq wrap with no rollover counter.** The wire header carried only the low 16 bits
|
|
|
|
|
|
of the nonce counter (zero-extended on receive); after 65,536 frames the reconstructed
|
|
|
|
|
|
nonce diverged and all frames failed auth. **Wire format widened to a full u64 seq**
|
|
|
|
|
|
(`voice_frame.h`: header 14 → 20 bytes, `seq` u16 → u64; `crypto.cpp`, `client.cpp`,
|
|
|
|
|
|
`media_relay.cpp` updated; `JitterBuffer::Frame::seq` widened). This is a **versioned wire
|
|
|
|
|
|
change → `VOICECAT_PROTOCOL_VERSION` 1 → 2**; the `Hello` handshake rejects on mismatch
|
|
|
|
|
|
(`conn_session.cpp`). The voice frame is parsed only in `core/`+`server/`+`tests/`, so the
|
|
|
|
|
|
Swift/C# clients need only a rebuild — no parser changes.
|
|
|
|
|
|
3. **Server leaked UDP state on disconnect.** `SessionRegistry::unregister_session()` now also
|
|
|
|
|
|
frees `udp_endpoints_`/`udp_tokens_`/`ssrc_to_session_` (scan-and-erase by session id).
|
|
|
|
|
|
4. **Diagnostics.** `MediaRelay` now emits rate-limited dropped-frame counters
|
|
|
|
|
|
(unmapped-endpoint / no-recv-crypto / open-failed) so a wedged media path is observable.
|
|
|
|
|
|
- **Verified:** `cmake --build --preset dev` clean; `ctest --preset dev -E external_pcm`
|
|
|
|
|
|
**22/22 pass** (incl. `m2_voice` e2e relay + the two new AEAD regressions). `external_pcm`
|
|
|
|
|
|
still aborts on the **pre-existing** CoreAudio shutdown mutex race (confirmed identical on a
|
|
|
|
|
|
clean baseline checkout under the same harness — unrelated to these changes). Docs updated:
|
|
|
|
|
|
`voice.md` §2 (header), `protocol.md` (v2 + negotiation), `security.md` (authenticate-then-advance).
|
|
|
|
|
|
|
2026-06-21 04:03:50 +02:00
|
|
|
|
- **Done (2026-06-21):** **Expose all channel codec params + guest nickname in every client.**
|
|
|
|
|
|
- **DRED everywhere + ABI fix.** `dred` (Opus 1.6 Deep REDundancy) existed in the C ABI
|
|
|
|
|
|
(`vc_audio_config.dred`) and proto but was absent from *both* client marshaling layers — a
|
|
|
|
|
|
latent ABI mismatch: Swift `AudioConfig` and the C# `VcAudioConfigNative` blittable struct
|
|
|
|
|
|
were each one `int` short of the native struct passed to `vc_create_channel`/`vc_edit_channel`.
|
|
|
|
|
|
Added `dred` through Swift (`Models.swift`, `Marshaling.swift`, `VoiceCatClient.toNative`) and
|
|
|
|
|
|
C# (`Structs.cs`, `Models.cs`, `Marshaling.cs`, `VoiceCatClient.cs`).
|
|
|
|
|
|
- **Windows:** added the one missing DRED checkbox to `ChannelEditDialog` (all other params
|
|
|
|
|
|
were already present).
|
|
|
|
|
|
- **macOS:** `ChannelEditSheet` now exposes the previously-hidden params — application profile,
|
|
|
|
|
|
sample rate, expected packet loss, complexity, and DRED (was only stereo/bitrate/frame/FEC/DTX).
|
|
|
|
|
|
- **iOS:** `ChannelEditView` was name+topic only; rebuilt into a full create **and edit** form
|
|
|
|
|
|
(General: name/topic/parent/password/max-users/sort-order; Audio: stereo/bitrate/sample-rate/
|
|
|
|
|
|
frame/application/packet-loss/complexity/FEC/DTX/DRED). Added `SessionState.editChannel` and an
|
|
|
|
|
|
"Edit" swipe action (admins) in `ChannelTreeView` + `ChannelBrowserView` (iOS previously had no
|
|
|
|
|
|
edit-channel UI at all). Note: the channel list doesn't carry the current audio config, so on
|
|
|
|
|
|
edit the audio fields start from codec defaults — same limitation as macOS/Windows.
|
|
|
|
|
|
- **Guest nickname.** Guests could not set a display name on iOS *or* macOS (the field was
|
|
|
|
|
|
absent/disabled; only Windows had it). Added a dedicated `nickname` to `SavedServer` on both
|
|
|
|
|
|
(backward-compatible Codable), a Nickname field shown in Guest mode (`AddServerView` /
|
|
|
|
|
|
`AddServerSheet`), and wired the guest auth path to use it (`AppState`, `ConnectWindowController`).
|
|
|
|
|
|
- **Verified:** `xcodebuild` Debug — macOS BUILD SUCCEEDED; iOS (sim, `ARCHS=arm64`) BUILD
|
|
|
|
|
|
SUCCEEDED. Core `ctest --preset dev` 22/23 (only `external_pcm` aborts on a pre-existing
|
|
|
|
|
|
shutdown mutex race; no C++ was changed). Windows C# not buildable on macOS — changes reviewed.
|
|
|
|
|
|
|
2026-06-21 03:02:58 +02:00
|
|
|
|
- **Done (2026-06-21):** **iOS iPhone-layout UX fixes.** (1) Channels are now a **drill-down**
|
|
|
|
|
|
on iPhone — new `ChannelBrowserView` (root list of top-level channels) → `ChannelDetailView`
|
|
|
|
|
|
(people in the channel + sub-channels + an explicit "Join Channel" button with password
|
|
|
|
|
|
prompt). The iPad 3-column `NavigationSplitView` is unchanged. (2) Extracted a self-contained
|
|
|
|
|
|
`UserRow` (context menu + sheets) from `UserListView` so admin actions are reused in the
|
|
|
|
|
|
drill-down. (3) Fixed the **off-screen chat compose box**: `MainView` now places
|
|
|
|
|
|
`VoiceControlsView` via `.safeAreaInset(edge: .bottom)` instead of a floating `.overlay`, so
|
|
|
|
|
|
it reserves layout space above the tab bar and cooperates with keyboard avoidance. (4)
|
|
|
|
|
|
**Collapsed Activity into Chat** like macOS/Windows: `ChatView` renders a merged, time-sorted
|
|
|
|
|
|
timeline of `messages` + `activityLog` (activity rows in gray); the separate Activity tab and
|
|
|
|
|
|
`ActivityLogView.swift` are removed. `xcodebuild` Debug for `generic/platform=iOS` BUILD
|
|
|
|
|
|
SUCCEEDED (sim slice still arm64-only → simulator run N/A). Next: on-device check of the
|
|
|
|
|
|
drill-down + compose box + unified timeline.
|
|
|
|
|
|
|
2026-06-22 12:28:53 +02:00
|
|
|
|
- **Done (2026-06-22):** **Windows exclude mode is now a real native exclude + self-echo
|
|
|
|
|
|
removal.** The "All apps except selected" mode previously captured the *complement of a frozen
|
|
|
|
|
|
app snapshot* in INCLUDE mode (missed late-launched apps, system sounds; wasted captures on
|
|
|
|
|
|
silent windows). It now opens a **single `ProcessLoopbackCapture` in EXCLUDE mode**
|
|
|
|
|
|
(`AUDIOCLIENT_PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE`) of the one chosen app — true
|
|
|
|
|
|
system-mix-minus-one, dynamic. `AppAudioPickerDialog` enforces single-selection in exclude
|
|
|
|
|
|
mode (the API takes one target PID). Added an **"Exclude VoiceCat's own audio (prevents echo)"**
|
|
|
|
|
|
checkbox (default on, entire-desktop only) that routes the desktop capture through the same
|
|
|
|
|
|
EXCLUDE path targeting `Environment.ProcessId`, killing the whole-device self-echo loop.
|
|
|
|
|
|
Touched `ProcessAudioMixer.cs` (`ResolveCaptures`), `AppAudioPickerDialog.cs`, `MainForm.cs`,
|
|
|
|
|
|
`AudioSessionEnumerator.cs` (`EntireDesktop(bool ExcludeSelf)`); docs in voice.md §9. No C++ /
|
|
|
|
|
|
ABI changes. `dotnet build` clean. **Still to verify on-device:** exclude actually silences
|
|
|
|
|
|
the chosen app while the rest plays, late-launched apps appear without restart, and the
|
|
|
|
|
|
self-exclude checkbox removes the echo.
|
|
|
|
|
|
|
2026-06-21 00:14:31 +02:00
|
|
|
|
- **Done (2026-06-21):** **Screen-audio sharing on macOS + iOS.** macOS uses ScreenCaptureKit
|
|
|
|
|
|
(`ScreenAudioCapture.swift`) → `vc_stream_feed_pcm`; iOS uses a ReplayKit Broadcast Upload
|
|
|
|
|
|
Extension (`VoiceCatBroadcast`) that forwards captured `.audioApp` PCM through a shared App
|
|
|
|
|
|
Group SPSC ring (`BroadcastAudioRing.swift`) to the host's `BroadcastAudioPump`, which owns
|
|
|
|
|
|
the `SCREEN_AUDIO` stream and feeds it — single session, no creds on disk. No C++ changes
|
|
|
|
|
|
(the core was already ready). macOS `xcodebuild` Debug BUILD SUCCEEDED; iOS app + extension
|
|
|
|
|
|
build for device (the xcframework sim slice is arm64-only, so x86_64-simulator link is N/A).
|
|
|
|
|
|
Next: on-device end-to-end verification (two clients hear the shared audio; iOS broadcast
|
|
|
|
|
|
start/stop). NOTE: `ctest --preset dev` is 22/23 — `external_pcm` passes its assertions but
|
|
|
|
|
|
aborts at shutdown (`mutex lock failed`), a **pre-existing** teardown crash unrelated to this
|
|
|
|
|
|
change (no C++ was modified).
|
|
|
|
|
|
|
2026-06-21 13:35:01 +02:00
|
|
|
|
- **Done (2026-06-21):** **macOS per-app screen-audio selection.** Before sharing, a new
|
|
|
|
|
|
`ScreenSharePickerSheet` lets the user choose scope — share Everything / Only selected apps /
|
|
|
|
|
|
All except selected apps — plus a first-class **"Exclude screen reader (VoiceOver) audio"**
|
|
|
|
|
|
toggle. `ScreenAudioCapture` now takes a `ScreenAudioSelection` and builds the matching
|
|
|
|
|
|
`SCContentFilter` (`including:` / `excludingApplications:`); app list comes from
|
|
|
|
|
|
`SCShareableContent`. macOS `xcodebuild` Debug BUILD SUCCEEDED. iOS deliberately untouched —
|
|
|
|
|
|
ReplayKit only delivers the mixed system stream, so per-app/VoiceOver filtering is impossible
|
|
|
|
|
|
there (documented in voice.md §9). **Still to verify on-device:** which process actually
|
|
|
|
|
|
carries VoiceOver speech (VoiceOver app vs. `com.apple.speech.speechsynthesisd`) — the exclude
|
|
|
|
|
|
set covers both candidates in `ScreenAudioCapture.screenReaderBundleIDs`; confirm exclusion
|
|
|
|
|
|
actually silences it in a real share.
|
|
|
|
|
|
|
2026-06-20 23:30:52 +02:00
|
|
|
|
- **Done (2026-06-20):** **macOS client UI overhaul** — mirrors the Windows client's UI
|
|
|
|
|
|
overhaul (commit 97fa659 + 540ec13), adapted to Mac-native conventions. Also fixed and
|
|
|
|
|
|
verified the previously-uncompiled Swift changes from the external PCM feed/tap commit
|
|
|
|
|
|
(615d2a8). The main window is now just toolbar + channels + users + chat; audio device
|
|
|
|
|
|
settings (input mode, VAD, PTT key, device picker, level meter) moved to a modeless
|
|
|
|
|
|
Settings window (⌘,). Details in M5 section below. `swift test` 10/10; `xcodebuild` Debug
|
|
|
|
|
|
+ Release BUILD SUCCEEDED with 0 Swift warnings.
|
|
|
|
|
|
Next: live manual verification (toolbar toggles, unified log colors, PM windows, channel
|
|
|
|
|
|
counts, volume slider, settings window); then iOS ReplayKit and macOS ScreenCaptureKit
|
|
|
|
|
|
consumers of `vc_stream_feed_pcm`.
|
|
|
|
|
|
|
2026-06-20 02:34:07 +02:00
|
|
|
|
- **Awaiting on-device verification:** **iOS stereo mic kills headphone/A2DP output — REAL
|
|
|
|
|
|
root cause found & fixed** (2026-06-20, on Windows; verify on Mac). All prior "fixes" (the
|
|
|
|
|
|
2026-06-19 entries below) targeted the Swift `IOSAudioRouter` on the false premise that
|
|
|
|
|
|
"miniaudio does NOT touch AVAudioSession on iOS." **It does.** The core opened its miniaudio
|
|
|
|
|
|
devices with `ma_device_init(nullptr, ...)`; with a NULL context, miniaudio (0.11.25) runs an
|
|
|
|
|
|
iOS "hack" (`miniaudio.h` ~44057) that picks a session category by device type, then
|
|
|
|
|
|
`ma_context_init__coreaudio` (~36552) calls `setCategory()` + `setActive()` on **every device
|
|
|
|
|
|
open** — capture → `AVAudioSessionCategoryRecord` with **zero options**. That wiped the
|
|
|
|
|
|
`.playAndRecord` category, the mode, and `.allowBluetoothA2DP`/`.mixWithOthers`/`.allowAirPlay`
|
|
|
|
|
|
that `IOSAudioRouter` had just configured → headphone/A2DP (and even wired) output died. The
|
|
|
|
|
|
stereo presets broke worst because they depend on the A2DP output route the wipe removed.
|
|
|
|
|
|
TeamTalk never hits this: its SDK opens RemoteIO/VPIO AudioUnits directly and leaves the
|
|
|
|
|
|
session entirely to the app (`UtilSound.swift`); miniaudio insists on managing it.
|
|
|
|
|
|
- **Fix (core, cross-platform safe):** `AudioEngine` now owns a `ma_context` built by
|
|
|
|
|
|
`make_context_config()` with `coreaudio.sessionCategory = ma_ios_session_category_none` +
|
|
|
|
|
|
`noAudioSessionActivate`/`noAudioSessionDeactivate = MA_TRUE`, and passes it to **all**
|
|
|
|
|
|
`ma_device_init` calls (playback, capture, loopback) and to `enumerate_devices`'s context.
|
|
|
|
|
|
miniaudio now never touches AVAudioSession; the Swift `IOSAudioRouter` is the sole owner
|
|
|
|
|
|
(session is already activated on connect in `AppState.swift:authResult`, before any device
|
|
|
|
|
|
opens, so removing miniaudio's self-activation is safe). Context is lazily inited in
|
|
|
|
|
|
`start()`, reused across restarts, uninited in `~AudioEngine`.
|
|
|
|
|
|
Files: `core/src/audio/audio_engine.{h,cpp}`.
|
|
|
|
|
|
- **TEMP diagnostics (remove after verification):** `AudioSessionManager.logSessionState(_:)`
|
|
|
|
|
|
logs category/mode/options/route; called after `ensureSessionActive`, on every route change,
|
|
|
|
|
|
and on `.streamStarted` (right after the core opens its devices). On Mac, watch the log when
|
|
|
|
|
|
joining voice with the Stereo Mic preset: category must stay `…PlayAndRecord` with
|
|
|
|
|
|
`allowBluetoothA2DP` and the output route must remain the headphones/A2DP device — NOT flip
|
|
|
|
|
|
to `…Record`. If confirmed, delete the `logSessionState` calls + method and the prior
|
|
|
|
|
|
band-aid comments in `IOSAudioRouter`/`audio_engine.cpp` can be trimmed.
|
2026-06-20 17:52:09 +02:00
|
|
|
|
- **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 23/23
|
|
|
|
|
|
(22/22 prior + `test_external_pcm` new binary). iOS build & on-device run still to be done by the user on the Mac.
|
2026-06-20 02:34:07 +02:00
|
|
|
|
|
2026-06-20 17:52:09 +02:00
|
|
|
|
- **Done (2026-06-20):** **External PCM feed/tap API (`vc_stream_feed_pcm` +
|
|
|
|
|
|
`vc_set_pcm_sink`)** — see detail in M5 section below. `ctest --preset dev` 23/23 (was 22/22 + 1 new test binary with 3 sub-tests).
|
|
|
|
|
|
Next: iOS ReplayKit and macOS ScreenCaptureKit consumers of this API. A public,
|
2026-06-19 03:51:14 +02:00
|
|
|
|
documented API for driving audio streams with externally-provided PCM instead of (or in
|
|
|
|
|
|
addition to) miniaudio's hardware device. Motivated by four concrete use cases — all in our
|
|
|
|
|
|
roadmap — that the current "miniaudio owns the device" model can't serve:
|
|
|
|
|
|
1. **ReplayKit Broadcast Upload Extension (iOS `SCREEN_AUDIO`)** — the extension is a
|
|
|
|
|
|
*separate process* with a ~50 MB memory cap and can't link the full `AudioEngine`
|
|
|
|
|
|
(`ma_device`, capture/playback threads). It needs to feed `CMSampleBuffer` audio (system
|
|
|
|
|
|
app audio) into the encode path without any audio hardware. The current plan in
|
|
|
|
|
|
`docs/voice.md §9` says the extension links "a minimal slice of the core (Opus encode +
|
|
|
|
|
|
media send only)" — a public feed-PCM API *is* that minimal slice. The extension links
|
|
|
|
|
|
Opus + the feed entry point, no `ma_device` needed.
|
|
|
|
|
|
2. **ScreenCaptureKit (macOS `SCREEN_AUDIO`)** — `SCStream` delivers `CMSampleBuffer` in a
|
|
|
|
|
|
callback; convert to int16 and feed. No need to route through miniaudio's device layer.
|
|
|
|
|
|
**This is how macOS screen-audio actually gets implemented** — today it does NOT work:
|
|
|
|
|
|
`VOICECAT_HAS_LOOPBACK` is Windows-only (`core/CMakeLists.txt:88-95`), so on macOS
|
|
|
|
|
|
`AudioEngine::start_loopback_capture()` hits the `#else` stub (`audio_engine.cpp:647-649`)
|
|
|
|
|
|
and returns `false`. The macOS client's "Share Screen Audio" button
|
|
|
|
|
|
(`MainWindowController.swift:800-816`) calls `startStream(.screenAudio)` which announces
|
|
|
|
|
|
the stream to peers but captures **zero audio** — peers hear silence. The button is left
|
|
|
|
|
|
in place (not touched per user request); it'll work once this API + a ScreenCaptureKit
|
|
|
|
|
|
tap ship on Mac.
|
|
|
|
|
|
3. **Bots** — music bot, TTS bot, radio relay, transcription bot. They create a
|
|
|
|
|
|
`SCREEN_AUDIO`/`AUX_DEVICE` stream and feed synthesized or decoded PCM via the feed API.
|
|
|
|
|
|
No audio hardware required — runs headless on a server. Today the only way to feed
|
|
|
|
|
|
external PCM is `vc_test_inject_capture` (TEST-ONLY, name signals "don't ship this") or
|
|
|
|
|
|
re-implementing Opus encode + AEAD + UDP framing yourself (~500 lines of duplicated
|
|
|
|
|
|
crypto/codec code per consumer).
|
|
|
|
|
|
4. **Custom clients / accessibility** — soundboard, DAW integration, TTS of incoming chat,
|
|
|
|
|
|
recording/transcription of remote audio. Need either feed (send) or tap (receive) or
|
|
|
|
|
|
both.
|
|
|
|
|
|
|
|
|
|
|
|
**What we already have (input half, gated as test-only):** `vc_test_inject_capture
|
|
|
|
|
|
(stream_id, pcm, samples)` (`voicecat.h`, `client.cpp:1452`) feeds raw int16 PCM into the
|
|
|
|
|
|
encode pipeline via `AudioEngine::inject_capture(kind, pcm, n)`. It works for any stream
|
|
|
|
|
|
kind, supports multiple concurrent injection taps (one ring buffer per local kind), and
|
|
|
|
|
|
goes through the full encode → AEAD → UDP path. The encode path already handles
|
|
|
|
|
|
`channels == 1 || 2` (proven by the WASAPI stereo loopback work, 2026-06-17 entry below).
|
|
|
|
|
|
The only problems: it's marked TEST-ONLY in the header, the name signals "don't use this in
|
|
|
|
|
|
production," and it hardcodes mono (no `channels` parameter).
|
|
|
|
|
|
|
|
|
|
|
|
**What's missing (output half):** today decoded remote audio is mixed and pushed to the
|
|
|
|
|
|
miniaudio playback device (`on_playback`). There's no way for an external consumer to
|
|
|
|
|
|
intercept the decoded PCM of a specific remote stream — it all goes to the hardware device.
|
|
|
|
|
|
A bot that wants to record, transcribe, or re-broadcast remote audio has no hook.
|
|
|
|
|
|
|
|
|
|
|
|
**Plan (API design — clean, append-only, no struct changes, ABI-stable):**
|
|
|
|
|
|
- **`vc_stream_feed_pcm`** — promote `vc_test_inject_capture` to a public, documented API
|
|
|
|
|
|
and add a `channels` parameter:
|
|
|
|
|
|
```c
|
|
|
|
|
|
/* External PCM feed — replaces the hardware capture device for this stream. Caller
|
|
|
|
|
|
provides interleaved int16 PCM at the stream's sample rate. The core frames it,
|
|
|
|
|
|
encodes (Opus), seals (AEAD), and sends (UDP). Works for any stream kind
|
|
|
|
|
|
(MIC/SCREEN_AUDIO/AUX_DEVICE). The stream must be started first (vc_stream_start);
|
|
|
|
|
|
this just replaces the capture source. channels = 1 (mono) or 2 (stereo interleaved).
|
|
|
|
|
|
Thread-safe; may be called from any thread including audio callbacks. */
|
|
|
|
|
|
vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id,
|
|
|
|
|
|
const int16_t* pcm, size_t samples_per_channel,
|
|
|
|
|
|
uint32_t channels);
|
|
|
|
|
|
```
|
|
|
|
|
|
- **`vc_set_pcm_sink`** — symmetric output side: receive decoded remote audio as int16 PCM
|
|
|
|
|
|
instead of (or in addition to) the hardware playback device:
|
|
|
|
|
|
```c
|
|
|
|
|
|
/* External PCM tap — receive decoded, mixed remote audio as int16 PCM. The callback
|
|
|
|
|
|
fires on the audio thread with the mixed output for a specific remote stream. Pass
|
|
|
|
|
|
cb=NULL to disable (default: disabled, hardware playback only). When enabled, PCM is
|
|
|
|
|
|
delivered to the sink AND the hardware device (dual output) so a bot can record
|
|
|
|
|
|
without disabling local monitoring. user_id+stream_id identify the source stream.
|
|
|
|
|
|
The callback MUST NOT block — copy what you need and return (same contract as
|
|
|
|
|
|
vc_callbacks.on_event). */
|
|
|
|
|
|
typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
|
|
|
|
|
|
const int16_t* pcm, size_t samples_per_channel,
|
|
|
|
|
|
uint32_t channels, uint32_t sample_rate);
|
|
|
|
|
|
vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
|
|
|
|
|
|
```
|
|
|
|
|
|
- **Core changes:**
|
|
|
|
|
|
- `core/include/voicecat.h` — add `vc_pcm_sink_cb` typedef + the two function
|
|
|
|
|
|
declarations (append-only, after `vc_test_inject_capture`). Full doc comments on both
|
|
|
|
|
|
(contract, thread-safety, lifetime, use cases).
|
|
|
|
|
|
- `core/src/voicecat.cpp` — thin C trampolines → `vc_client::stream_feed_pcm` /
|
|
|
|
|
|
`set_pcm_sink`.
|
|
|
|
|
|
- `core/src/core/client.{h,cpp}` — `stream_feed_pcm`: validates `stream_id`, looks up
|
|
|
|
|
|
the `LocalStream`'s kind, calls `audio_engine_.inject_capture(kind, pcm, n)` (existing
|
|
|
|
|
|
path) with the channel count forwarded. `set_pcm_sink`: stores the callback + user
|
|
|
|
|
|
pointer; `on_playback` (or a new fan-out in the mixer) invokes it per remote stream
|
|
|
|
|
|
alongside the existing hardware write. Keep `vc_test_inject_capture` as a deprecated
|
|
|
|
|
|
alias calling `stream_feed_pcm(..., channels=1)` for source compatibility.
|
|
|
|
|
|
- `core/src/audio/audio_engine.{h,cpp}` — `inject_capture` already exists per-kind; add
|
|
|
|
|
|
a `channels` parameter to the ring-buffer write path (or a parallel stereo-aware
|
|
|
|
|
|
variant). The encode path in `client.cpp::on_capture_frame` already handles
|
|
|
|
|
|
`channels==2` via the stereo encode branch — just plumb the value through. For the
|
|
|
|
|
|
sink: add a `pcm_sink_` member (callback + user); in `on_playback` after mixing, if the
|
|
|
|
|
|
sink is set, copy the mixed PCM for the current stream and invoke the callback. The
|
|
|
|
|
|
copy must stay off the RT-critical path — document the non-blocking contract.
|
|
|
|
|
|
- **Skeleton stub path:** update `client.cpp`'s `#else` (no-deps) stub section to add
|
|
|
|
|
|
`vc_stream_feed_pcm`/`vc_set_pcm_sink` returning `VC_ERR_NOT_IMPLEMENTED` — keeps the
|
|
|
|
|
|
skeleton preset green.
|
|
|
|
|
|
- **Swift `VoiceCatCore`:** add `feedPcm(streamId:pcm:samplesPerChannel:channels:)` and
|
|
|
|
|
|
`setPcmSink(_:user:)` (the Swift wrapper around `vc_pcm_sink_cb` — a
|
|
|
|
|
|
`@convention(c)` closure + `Unmanaged` context, mirroring `Callbacks.swift`). Wraps both
|
|
|
|
|
|
new ABI functions.
|
|
|
|
|
|
- **C# `VoiceCat.Interop`:** add `StreamFeedPcm(streamId, pcm, samples, channels)` (with
|
|
|
|
|
|
`int16[]` marshaling) and `SetPcmSink` (delegates via `[UnmanagedCallersOnly]` thunk,
|
|
|
|
|
|
mirroring the event-callback pattern). Wraps both new ABI functions.
|
|
|
|
|
|
- **Tests:**
|
|
|
|
|
|
- `tests/test_external_pcm.cpp` (new) — `test_feed_pcm_round_trip`: two clients, A feeds
|
|
|
|
|
|
a known mono sine wave via `vc_stream_feed_pcm` on a MIC stream, B receives via the
|
|
|
|
|
|
normal decode path and asserts energy matches. `test_feed_pcm_stereo`: same with
|
|
|
|
|
|
`channels=2`, assert L≠R end-to-end (mirrors the WASAPI loopback stereo test).
|
|
|
|
|
|
`test_pcm_sink`: B sets a `vc_pcm_sink_cb`, A feeds PCM, assert the sink callback
|
|
|
|
|
|
receives the decoded PCM with matching energy. All headless, no audio hardware.
|
|
|
|
|
|
- `clients/apple/Tests/VoiceCatCoreTests/` — Swift wrapper round-trip for `feedPcm`.
|
|
|
|
|
|
- `clients/windows/VoiceCat.Interop.Tests/` — C# wrapper round-trip.
|
|
|
|
|
|
- **Docs:**
|
|
|
|
|
|
- `docs/architecture.md §4` — new subsection on external PCM feed/tap: the contract
|
|
|
|
|
|
(caller provides interleaved int16 at the stream's sample rate; core frames/encodes/
|
|
|
|
|
|
seals/sends for feed; core decodes/mixes/delivers for sink; sink callback must not
|
|
|
|
|
|
block), the use cases (ReplayKit, ScreenCaptureKit, bots, custom clients), and the
|
|
|
|
|
|
relationship to `vc_test_inject_capture` (deprecated alias).
|
|
|
|
|
|
- `docs/voice.md §9` — update the iOS ReplayKit and macOS ScreenCaptureKit rows: both
|
|
|
|
|
|
now consume `vc_stream_feed_pcm` instead of a "minimal slice of the core." Update the
|
|
|
|
|
|
iOS detail bullets: the extension links Opus + `vc_stream_feed_pcm` (not a parallel
|
|
|
|
|
|
media stack). Add a macOS ScreenCaptureKit note: convert `CMSampleBuffer` → int16,
|
|
|
|
|
|
feed via `vc_stream_feed_pcm` — this is how macOS screen-audio actually ships.
|
|
|
|
|
|
- `docs/protocol.md` — no protocol changes (the feed/sink are client-local; the wire
|
|
|
|
|
|
format is identical whether PCM came from miniaudio or an external source). Note this
|
|
|
|
|
|
explicitly.
|
|
|
|
|
|
- `docs/roadmap.md` — add a milestone entry; update the iOS ReplayKit and macOS
|
|
|
|
|
|
ScreenCaptureKit pending items to reference `vc_stream_feed_pcm`.
|
|
|
|
|
|
- **Implementation order:**
|
|
|
|
|
|
1. C ABI + core (`voicecat.h`, `voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`) +
|
|
|
|
|
|
skeleton stub. Verify `ctest --preset dev` green.
|
|
|
|
|
|
2. `tests/test_external_pcm.cpp` — the three behavior tests. Verify green.
|
|
|
|
|
|
3. Swift `VoiceCatCore` wrapper + `VoiceCatCoreTests` round-trip.
|
|
|
|
|
|
4. C# `VoiceCat.Interop` wrapper + `VoiceCatClientSmokeTests` round-trip.
|
|
|
|
|
|
5. Docs (`architecture.md`, `voice.md`, `protocol.md`, `roadmap.md`, header comments).
|
|
|
|
|
|
6. **Then** ReplayKit (iOS) and ScreenCaptureKit (macOS) become ~100-line consumers of
|
|
|
|
|
|
this API instead of parallel media stacks.
|
|
|
|
|
|
- **Verification:** `ctest --preset dev` green (3 new tests); `swift test` green; `dotnet
|
|
|
|
|
|
test` green; `xcodebuild` (skeleton) green. The feed/sink tests are fully headless — no
|
|
|
|
|
|
audio hardware, no simulator, no device — so they run in CI on every platform.
|
|
|
|
|
|
- **Files to touch:**
|
|
|
|
|
|
- Core C++: `core/include/voicecat.h`, `core/src/voicecat.cpp`,
|
|
|
|
|
|
`core/src/core/client.{h,cpp}`, `core/src/audio/audio_engine.{h,cpp}`.
|
|
|
|
|
|
- Tests: `tests/test_external_pcm.cpp` (new), `tests/CMakeLists.txt`.
|
|
|
|
|
|
- Swift: `clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift`,
|
|
|
|
|
|
`clients/apple/Sources/VoiceCatCore/Callbacks.swift`,
|
|
|
|
|
|
`clients/apple/Tests/VoiceCatCoreTests/ExternalPcmTests.swift` (new).
|
|
|
|
|
|
- C#: `clients/windows/VoiceCat.Interop/VoiceCatClient.cs`,
|
|
|
|
|
|
`clients/windows/VoiceCat.Interop/NativeMethods.cs`,
|
|
|
|
|
|
`clients/windows/VoiceCat.Interop.Tests/ExternalPcmTests.cs` (new).
|
|
|
|
|
|
- Docs: `docs/architecture.md`, `docs/voice.md`, `docs/protocol.md`, `docs/roadmap.md`.
|
|
|
|
|
|
- **ABI stability:** append-only — two new functions + one new typedef, no existing
|
|
|
|
|
|
structs/enums changed. `vc_test_inject_capture` stays as a deprecated alias for source
|
|
|
|
|
|
compatibility. Treat as a deliberate, versioned ABI event per `docs/protocol.md §8`.
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Relationship to the iOS audio routing plan:** orthogonal. The iOS routing layer controls
|
|
|
|
|
|
*which hardware route* miniaudio opens (AVAudioSession config in Swift). This plan is about
|
|
|
|
|
|
*bypassing miniaudio's hardware entirely* (external PCM feed/tap). Both ship; they don't
|
|
|
|
|
|
conflict. ReplayKit/ScreenCaptureKit consume this API; the iOS routing layer controls the
|
|
|
|
|
|
*mic* path which still uses miniaudio's device.
|
2026-06-19 03:51:14 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
---
|
2026-06-19 03:51:14 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
## Recent completed work
|
2026-06-19 16:58:21 +02:00
|
|
|
|
|
2026-06-22 16:56:49 +02:00
|
|
|
|
All items below are `[x]` done; `ctest --preset dev` 26/26 on Windows after all.
|
|
|
|
|
|
|
|
|
|
|
|
- **Per-channel sample_rate as a bandwidth cap** (2026-06-22): the channel `sample_rate` field
|
|
|
|
|
|
was inert (the codec is pinned to 48 kHz). Made it meaningful without changing the 48 kHz
|
|
|
|
|
|
clock: it's carried as `OpusParams::max_bandwidth_hz` and applied via `OPUS_SET_MAX_BANDWIDTH`
|
|
|
|
|
|
in `OpusEncoder::init` (8000→narrowband … 48000→full). Made it **channel-authoritative** on
|
|
|
|
|
|
the server (`conn_session.cpp` no longer overrides effective `sample_rate` with the client's
|
|
|
|
|
|
always-48000 request — like `frame_ms`/`mode`). `vc_get_stream_audio_config` now reports the
|
|
|
|
|
|
channel's configured rate for own streams too. New ctest `channel_samplerate`: a 7 kHz tone is
|
|
|
|
|
|
attenuated ~1000× on an 8 kHz channel vs a 48 kHz channel. Files: `opus_codec.{h,cpp}`,
|
|
|
|
|
|
`client.cpp`, `server/src/conn_session.cpp`, `docs/voice.md`, `tests/test_channel_samplerate.cpp`,
|
|
|
|
|
|
`tests/CMakeLists.txt`. (Future: a true non-48k stack is possible but unnecessary — 48 kHz is
|
|
|
|
|
|
what nearly all hard/software runs at; the bandwidth cap covers the narrowband use case.)
|
2026-06-22 16:45:02 +02:00
|
|
|
|
|
|
|
|
|
|
- **Non-20ms channel frame_ms fix** (2026-06-22): the AudioEngine capture clock is fixed at
|
|
|
|
|
|
48 kHz / 20 ms (960-sample frames), but a channel may set any Opus `frame_ms` (2.5…60 ms,
|
|
|
|
|
|
docs/voice.md §3) and the server enforces it unclamped. The send path handed the engine's
|
|
|
|
|
|
960-sample frame straight to an encoder configured for the channel's window — silently
|
|
|
|
|
|
ignoring `frame_ms > 20` and **breaking `frame_ms < 20` entirely** (receiver sized its decode
|
|
|
|
|
|
buffer too small → `OPUS_BUFFER_TOO_SMALL` → dead audio). Affected the hardware mic AND
|
|
|
|
|
|
`vc_stream_feed_pcm`. Fix: `vc_client::on_capture_frame` now reframes each captured/fed block
|
|
|
|
|
|
to `ls.frame_samples` via a per-`LocalStream` accumulator (pre-sized at announce, no RT-thread
|
|
|
|
|
|
alloc) before `encode_and_send_frame`; the 20 ms case stays a zero-copy fast path. Also pinned
|
|
|
|
|
|
the codec to 48 kHz internally in `opus_params_from_audio_config` (was honoring a non-48k
|
|
|
|
|
|
effective sample_rate against a 48k PCM clock). New ctest `frame_ms_reframe` (40 ms accumulate
|
|
|
|
|
|
+ 10 ms split round trips). Files: `client.{h,cpp}`, `voicecat.h` (feed doc), `docs/voice.md`,
|
|
|
|
|
|
`tests/test_frame_ms_reframe.cpp`, `tests/CMakeLists.txt`.
|
2026-06-20 17:52:09 +02:00
|
|
|
|
|
|
|
|
|
|
- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped.
|
|
|
|
|
|
Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API.
|
|
|
|
|
|
Adds symmetric PCM sink on the playback thread. Swift wrapper (`feedPcm`/`setPcmSink` in
|
|
|
|
|
|
`VoiceCatClient.swift`, 4 XCTest smoke tests). C# wrapper (`StreamFeedPcm`/`SetPcmSink` in
|
|
|
|
|
|
`VoiceCatClient.cs` + `NativeMethods.cs`, 4 xUnit smoke tests in `ExternalPcmTests.cs`).
|
|
|
|
|
|
Three new headless C++ ctests. Docs: architecture.md §4 new subsection, voice.md §9 updated,
|
|
|
|
|
|
protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Files: `voicecat.h`,
|
|
|
|
|
|
`voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`, `tests/test_external_pcm.cpp`,
|
|
|
|
|
|
`tests/CMakeLists.txt`, Swift + C# wrappers.
|
2026-06-19 16:58:21 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **iOS A2DP + stereo root cause fix** (2026-06-20): miniaudio's NULL-context `ma_device_init`
|
|
|
|
|
|
was calling `AVAudioSession setCategory(Record)` on every device open, wiping the session
|
|
|
|
|
|
config `IOSAudioRouter` had set. Fixed by sharing a `ma_context` with
|
|
|
|
|
|
`sessionCategory=none` + `noAudioSessionActivate/Deactivate=MA_TRUE` — miniaudio never
|
|
|
|
|
|
touches `AVAudioSession`; `IOSAudioRouter` is the sole owner. Files: `audio_engine.{h,cpp}`.
|
2026-06-19 16:58:21 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **iOS audio routing overhaul** (2026-06-19): Full `IOSAudioRouter` singleton drives all
|
|
|
|
|
|
`AVAudioSession` config before miniaudio opens devices. Fixed stereo mic polar-pattern setup
|
|
|
|
|
|
(WWDC20 recipe: `setPreferredInput` + `setInputDataSource` + `.stereo` polar pattern + no
|
|
|
|
|
|
`setPreferredInputNumberOfChannels`). Added `vc_audio_restart` ABI (full stop+reinit for
|
|
|
|
|
|
close→reconfigure→reopen ordering). Added `vc_set_capture_channels` ABI (core stereo-mic
|
|
|
|
|
|
support). AVAudioSession activated proactively on `.authResult`, not lazily on
|
|
|
|
|
|
`.streamStarted`. Join/Leave Voice button added (parity with macOS). Channel-id sync fixed
|
|
|
|
|
|
(mic button was permanently dimmed). iOS deployment target raised to 18.0.
|
2026-06-19 03:39:12 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **iOS SwiftUI client** (2026-06-19): `VoiceCatiOS.xcodeproj` at `clients/apple/iOS/`.
|
|
|
|
|
|
Full feature parity with macOS/Windows: saved server list (JSON + Keychain, App Group
|
|
|
|
|
|
`group.cat.voice.VoiceCat`), TOFU, connect flow, channel tree, user list with context menus,
|
|
|
|
|
|
chat, admin sheets, voice controls, settings. `xcodebuild` → BUILD SUCCEEDED.
|
2026-06-19 03:39:12 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **macOS AppKit client** (2026-06-18): `VoiceCatMac.xcodeproj` at `clients/apple/macOS/`.
|
|
|
|
|
|
Fixed compile errors (`NSAccessibility` call-site arg order, `StreamSummary.id` vs
|
|
|
|
|
|
`.streamId`) and linker issues (`OTHER_LDFLAGS = -lc++`, `ONLY_ACTIVE_ARCH = YES` for
|
|
|
|
|
|
Release). Debug + Release both BUILD SUCCEEDED.
|
2026-06-19 02:10:25 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Swift `VoiceCatCore` package + XCFramework** (2026-06-18): Shared Swift wrapper at
|
|
|
|
|
|
`clients/apple/`. `build-xcframework.sh` merges `libvoicecat.a` + 107 vcpkg static deps into
|
|
|
|
|
|
a fat `.a` via `libtool -static`. 6/6 Swift tests green (real server, mirrors C# Interop
|
|
|
|
|
|
tests). Supports macOS-arm64 + iOS-arm64 + iOS-sim slices.
|
2026-06-18 15:26:47 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **macOS port validated** (2026-06-18): 21/21 on macOS. Three cross-platform bugs fixed:
|
|
|
|
|
|
missing `<netdb.h>` in POSIX test branch; SIGPIPE kills (added `SIG_IGN`); use-after-free of
|
|
|
|
|
|
Asio kqueue reactor on server shutdown (fixed `TcpAcceptor` shutdown/connection-drain
|
|
|
|
|
|
sequence).
|
2026-06-18 15:26:47 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **CMake preset cleanup** (2026-06-18): `m1-dev`→`dev`, `dev`→`skeleton`, `m2-dev` dropped.
|
|
|
|
|
|
New `release`, `server-release` (stripped), `apple-dev`/`apple-ios`/`apple-ios-sim`. Cross-
|
|
|
|
|
|
platform triplet auto-resolved by `cmake/voicecat-toolchain.cmake`.
|
2026-06-18 14:20:38 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Disconnect, keepalive & reaper** (2026-06-18): Client sends `Ping` every 15 s; server
|
|
|
|
|
|
reaper drops sessions after 45 s; UDP `KEEPALIVE` every 5 s keeps NAT alive. `vc_disconnect`
|
|
|
|
|
|
sends graceful `Disconnect` proto. Stale-user LEFT broadcast on drop. PLC capped at ~2 s.
|
|
|
|
|
|
Three new tests: `test_disconnect_left`, `test_plc_cap`, `test_reaper_timeout`.
|
2026-06-18 13:24:42 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Stereo screen-audio loopback** (2026-06-17): WASAPI loopback opens in channel's
|
|
|
|
|
|
stereo/mono mode (was hardcoded mono). Real stereo flows end-to-end through loopback → encode
|
|
|
|
|
|
→ decode → mixer. New `test_loopback_stereo_capture`.
|
2026-06-18 03:16:01 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Windows screen-audio UI wired** (2026-06-17): `btnScreenShareToggle` in `MainForm.cs`.
|
|
|
|
|
|
No core/proto/ABI changes — all the plumbing was already there. `dotnet test` 4/4 green.
|
2026-06-18 01:18:33 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Bug fixes** (2026-06-16 – 2026-06-17):
|
|
|
|
|
|
- *AEAD nonce desync in SFU relay* — relay forwarded sender's `seq` verbatim; recipient
|
|
|
|
|
|
nonce reconstruction used the wrong counter. Fixed by rewriting the outgoing `seq` field
|
|
|
|
|
|
to the recipient's `peek_send_counter()`.
|
|
|
|
|
|
- *Playout clock free-ran* — `playout_ts` advanced even during VAD/PTT silence gaps,
|
|
|
|
|
|
eventually dropping all frames as too-late. Fixed with resync in `on_playback` via
|
|
|
|
|
|
`JitterBuffer::peek_front_ts()`.
|
|
|
|
|
|
- *Stale users after disconnect* — `ConnSession::close()` didn't broadcast `UserEvent::LEFT`
|
|
|
|
|
|
before erasing. Fixed; PLC cap added as defense-in-depth.
|
|
|
|
|
|
- *"Randomly bumped to Lobby"* — server excluded the actor from its own state-change
|
|
|
|
|
|
broadcasts. Fixed: `UserEvent::UPDATED` now goes to all clients including the actor.
|
|
|
|
|
|
- *Silent playback after join* — `opus_decode` received hardware callback frame count as
|
|
|
|
|
|
`max_samples` instead of the Opus frame size. Fixed with a decode ring buffer.
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Milestones (see [docs/roadmap.md](docs/roadmap.md) for full detail)
|
|
|
|
|
|
|
|
|
|
|
|
- [x] **M0 — Scaffolding** ✓ complete
|
2026-06-15 23:48:44 +02:00
|
|
|
|
- [x] **M1 — Control plane** ✓ complete (2026-06-15)
|
2026-06-16 01:31:14 +02:00
|
|
|
|
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16)
|
2026-06-16 14:12:37 +02:00
|
|
|
|
- [x] **M3 — Multi-stream & per-channel tuning** ✓ complete (2026-06-16)
|
2026-06-19 02:10:25 +02:00
|
|
|
|
- [x] **M4 — Native clients** — Windows WinForms ✓ (2026-06-17); macOS AppKit ✓ (2026-06-18); iOS SwiftUI ✓ (2026-06-19)
|
2026-06-17 15:08:05 +02:00
|
|
|
|
- [~] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
## M0 — Scaffolding ✓
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
Repo layout (`core/ server/ tools/ clients/ tests/`), CMake + vcpkg manifest, C ABI header
|
|
|
|
|
|
(`voicecat.h`), proto source of truth, core stubs for all six subsystems, `voicecat-server` +
|
|
|
|
|
|
`vccli` skeletons, smoke CTest, `.clang-format`/`.gitattributes`/`.gitignore`.
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-15 23:48:44 +02:00
|
|
|
|
## M1 — Control plane ✓ (completed 2026-06-15)
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Exit criterion:** `test_m1_integration` — two clients authenticate over TLS 1.3 (guest +
|
|
|
|
|
|
Argon2id), exchange channel + private text. ~1 s.
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
`FrameCodec`, `TlsContext` (mbedTLS 1.3, ECDSA-P256 self-signed, TOFU pins TLS leaf-cert
|
|
|
|
|
|
SHA-256), `WorkerPool`, `Database` (SQLite + Argon2id), `ServerIdentityManager`,
|
|
|
|
|
|
`ConnSession` state machine, `SessionRegistry`, `vc_client` full M1 C ABI, `voicecat-admin`
|
|
|
|
|
|
CLI, dual-stack `TcpAcceptor`. **Key bug fixed:** `send_frame` double-framing — `encode_envelope`
|
|
|
|
|
|
was pre-framing the protobuf; fixed by passing raw protobuf bytes.
|
2026-06-15 21:09:09 +02:00
|
|
|
|
|
2026-06-16 01:31:14 +02:00
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## M2 — Voice, single stream ✓ (completed 2026-06-16)
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Exit criterion:** `test_m2_voice` + `test_voice_client_abi` — two headless clients auth, bind
|
|
|
|
|
|
UDP, 50 Opus frames relayed + re-encrypted by SFU, B receives ≥25 and decrypts. ~4 s.
|
2026-06-16 01:31:14 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
14-byte UDP voice header, `SodiumMediaCrypto` (ChaCha20-Poly1305 + 64-bit anti-replay),
|
|
|
|
|
|
`OpusEncoder`/`OpusDecoder` (FEC, PLC), `UdpMediaChannel`, `JitterBuffer`, `AudioEngine`
|
|
|
|
|
|
(miniaudio), `MediaRelay` SFU. **Key bug fixed:** `on_playback` passed hardware callback frame
|
|
|
|
|
|
count as `opus_decode` max_samples; fixed with a per-stream decode ring buffer.
|
2026-06-16 01:31:14 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-16 14:12:37 +02:00
|
|
|
|
## M3 — Multi-stream & per-channel tuning ✓ (completed 2026-06-16)
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Exit criterion:** `test_m3_multistream` — client A runs two concurrent streams (MIC +
|
|
|
|
|
|
SCREEN_AUDIO); B sees both; per-stream gain/mute/NR independent; effective Opus config matches
|
|
|
|
|
|
channel's server-enforced settings. ~2.4 s.
|
2026-06-16 14:12:37 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
Fixed server `stream_id` counter bug (always wrote `1`). Per-channel `AudioConfig` populated
|
|
|
|
|
|
(Lobby: mono/24kbps/VOIP + DTX; Music Room: stereo/128kbps/AUDIO). `LocalStream` map,
|
|
|
|
|
|
`pending_announce_kind_`, `run_talk_timer()`, thread-join race in `teardown_voice()` fixed.
|
|
|
|
|
|
New C ABI: `vc_get_stream_audio_config`, `vc_test_inject_capture`.
|
2026-06-16 14:12:37 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
## Post-M3 follow-up ✓ (completed 2026-06-16)
|
2026-06-16 16:11:52 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- **Device enumeration** — `vc_list_devices`/`vc_set_input_device`; opaque hex device ids;
|
|
|
|
|
|
`vc_free_device_list` now frees. Works pre-connect.
|
|
|
|
|
|
- **VAD/PTT gate** — `EnergyVadProcessor` (RMS threshold ~0.025, 300 ms hang-time);
|
|
|
|
|
|
`vc_set_input_mode`/`vc_set_push_to_talk`; MIC-only (SCREEN_AUDIO/AUX_DEVICE bypass).
|
|
|
|
|
|
- **True stereo playback** — `playback_channels=2`; stereo decoded L→L R→R in mixer; mono
|
|
|
|
|
|
upmixed L=R; hardware fallback to mono on failure.
|
|
|
|
|
|
- **WASAPI loopback** — `loopback_device_` with `ma_device_type_loopback`;
|
|
|
|
|
|
`VOICECAT_HAS_LOOPBACK` macro (Windows-only). `vccli --share-screen-audio`.
|
2026-06-16 16:11:52 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Known deferred (still open):** AEC/NS/AGC (no working Windows/MSVC WebRTC APM build);
|
|
|
|
|
|
process-specific WASAPI loopback; RT-thread rule violation in `on_capture_frame` (mutex lock
|
|
|
|
|
|
on audio callback thread — pre-existing, needs lock-free ring-buffer refactor).
|
2026-06-16 16:11:52 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
## M4 — Native clients ✓ (completed 2026-06-17 – 2026-06-19)
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Exit criterion:** `ctest --preset dev` 21/21 green; `dotnet build` 0 warnings; `xcodebuild`
|
|
|
|
|
|
BUILD SUCCEEDED (macOS + iOS); manually verified: connect, TOFU, channel tree, join, voice,
|
|
|
|
|
|
text, device pickers, level meter on each platform.
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**New C ABI (additive):** `vc_list_channels`/`vc_list_users`/`vc_list_user_streams`,
|
|
|
|
|
|
`vc_join_channel`, `VC_EVENT_SERVER_IDENTITY` + `vc_confirm_server_identity`,
|
|
|
|
|
|
`vc_config::tofu_store_path`, `VC_INPUT_ALWAYS_ON`, `vc_set_vad_threshold`,
|
|
|
|
|
|
`vc_audio_suspend`/`vc_audio_resume`, `vc_audio_restart`, `vc_set_capture_channels`.
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**Windows** (`clients/windows/`): `VoiceCat.Interop` (P/Invoke, `[UnmanagedCallersOnly]`),
|
|
|
|
|
|
`VoiceCat.App` (ConnectDialog, ServerIdentityDialog, MainForm with full M5 moderation UI,
|
2026-06-24 13:14:01 +02:00
|
|
|
|
PerUserTuningDialog, PttKeyCaptureDialog), `VoiceCat.Interop.Tests`. PTT can be system-wide
|
|
|
|
|
|
(Raw Input / WM_INPUT) or focus-scoped, toggled in Audio settings (default system-wide).
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**macOS** (`clients/apple/macOS/VoiceCatMac.xcodeproj`): NSOutlineView channel tree,
|
|
|
|
|
|
NSTableView user list, NSTextView chat, voice controls, full VoiceOver accessibility, admin
|
|
|
|
|
|
menu, 17 Swift source files. `build-xcframework.sh` produces `VoiceCatCore.xcframework`.
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
**iOS** (`clients/apple/iOS/VoiceCatiOS.xcodeproj`): SwiftUI, `NavigationSplitView`/`TabView`,
|
|
|
|
|
|
`OutlineGroup` channel tree, `IOSAudioRouter` AVAudioSession driver, 24 Swift source files,
|
|
|
|
|
|
iOS 18.0 deployment target. App Group `group.cat.voice.VoiceCat` for Keychain sharing.
|
2026-06-17 00:52:02 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-17 15:08:05 +02:00
|
|
|
|
## M5 — Moderation, polish, and beyond [~] (in progress 2026-06-17)
|
|
|
|
|
|
|
|
|
|
|
|
**Exit criterion:** four ABI-level tests green (`test_m5_permissions`,
|
|
|
|
|
|
`test_m5_kick_ban_move_mute`, `test_m5_admin_accounts`, `test_m5_channel_crud`);
|
|
|
|
|
|
`vccli` can drive all moderation/admin/channel operations against a live server.
|
|
|
|
|
|
|
2026-06-20 13:14:39 +02:00
|
|
|
|
- [x] **Server-side moderation & permissions** — per-session `Permissions`, kick/ban/move/
|
|
|
|
|
|
server-mute, channel CRUD, DB schema v2 (`channels`, `bans`), BLAKE2b channel passwords.
|
|
|
|
|
|
- [x] **C ABI** — `vc_kick_user`, `vc_ban_user`, `vc_set_permission`, `vc_set_server_mute`,
|
|
|
|
|
|
`vc_move_user`, `vc_create_channel`, `vc_edit_channel`, `vc_delete_channel`,
|
|
|
|
|
|
`vc_create_account`, `vc_reset_password`, `vc_delete_account`, `vc_list_accounts`,
|
|
|
|
|
|
`vc_get_permissions`; events `VC_EVENT_GENERIC_RESULT`, `VC_EVENT_ACCOUNT_LIST`.
|
|
|
|
|
|
- [x] **Four M5 tests** passing — `ctest --preset dev` 21/21.
|
|
|
|
|
|
- [x] **vccli** M5 flags: `--kick`, `--ban`, `--move`, `--server-mute`/`-unmute`/`-deafen`/
|
|
|
|
|
|
`-undeafen`, `--set-permission`, channel CRUD, account CRUD, `--username`/`--password`.
|
|
|
|
|
|
- [x] **All three client UIs** (Windows WinForms, macOS AppKit, iOS SwiftUI) expose the full
|
|
|
|
|
|
M5 moderation and admin surface.
|
|
|
|
|
|
- [x] **Docs** — `docs/protocol.md`, `docs/security.md` kept in sync.
|
2026-06-20 13:40:47 +02:00
|
|
|
|
- [x] **DRED/audio-quality polish** — done (2026-06-20). `bool dred` added to `AudioConfig`
|
|
|
|
|
|
proto (field 11) and `vc_audio_config` C ABI. Encoder: `OPUS_SET_DRED_DURATION(2)` when
|
|
|
|
|
|
enabled (20 ms of ML redundancy per packet). Decoder: `OpusDREDDecoder` + per-stream
|
|
|
|
|
|
`OpusDRED` scratch pre-allocated; `JitterBuffer::try_copy_front_payload` peeks at the next
|
|
|
|
|
|
buffered packet on every PLC step; if DRED data is present, `opus_decoder_dred_decode`
|
|
|
|
|
|
reconstructs the lost frame — otherwise falls back to standard PLC. New test:
|
|
|
|
|
|
`test_dred_toggle` (ctest 22/22). Files: `voicecat.proto`, `voicecat.h`,
|
|
|
|
|
|
`opus_codec.{h,cpp}`, `audio_engine.{h,cpp}`, `client.cpp`, `session.{h,cpp}`.
|
2026-06-22 20:18:29 +02:00
|
|
|
|
- [x] **In-band FEC decoder wiring** — done (2026-06-22). The encoder set `OPUS_SET_INBAND_FEC`
|
|
|
|
|
|
all along, but the decoder never invoked it — the loss path went DRED → PLC, so FEC redundancy
|
|
|
|
|
|
was emitted (and paid for in bitrate) but never consumed. Wired the FEC recovery into
|
|
|
|
|
|
`AudioEngine::on_playback`'s loss branch between DRED and PLC: copy the next buffered packet
|
|
|
|
|
|
once, try DRED, else (if the stream negotiated FEC) `decode(next_pkt, …, fec=true)`, else PLC.
|
|
|
|
|
|
Added per-stream `RemoteStream::fec_enabled_`, captured from `OpusParams` in
|
|
|
|
|
|
`init_recv_stream`. Recovery priority is now **DRED → FEC → PLC**. ctest 27/27 green. Files:
|
|
|
|
|
|
`audio_engine.{h,cpp}`, `docs/voice.md`.
|
2026-06-20 13:40:47 +02:00
|
|
|
|
- [ ] **DRED toggle in client UIs** — expose the `dred` flag in all three channel-config UIs
|
|
|
|
|
|
so admins can enable it per channel. Windows: `ChannelEditForm` / `vc_channel_info.audio.dred`
|
|
|
|
|
|
checkbox. macOS AppKit: channel-edit sheet. iOS SwiftUI: channel-edit form. All three UIs
|
|
|
|
|
|
already have full channel CRUD wired; this is an additive checkbox on the existing audio-config
|
|
|
|
|
|
section. (Core/protocol/ABI all done — this is UI-only work.)
|
2026-06-21 00:14:31 +02:00
|
|
|
|
- [x] **macOS ScreenCaptureKit screen-audio** — done 2026-06-21. `ScreenAudioCapture.swift`
|
|
|
|
|
|
drives an `SCStream` (audio-only, `excludesCurrentProcessAudio`), converts Float32 →
|
|
|
|
|
|
int16 in the channel's mono/stereo mode, and calls `vc_stream_feed_pcm`. Capture starts on
|
|
|
|
|
|
the self `.streamStarted` event (when the effective config is known); wired into
|
|
|
|
|
|
`MainWindowController.screenAudioClicked()`.
|
|
|
|
|
|
- [x] **iOS ReplayKit Broadcast Extension** (`VoiceCatBroadcast`) — done 2026-06-21.
|
|
|
|
|
|
Forward-to-host design: the extension (`SampleHandler.swift`) captures `.audioApp`,
|
|
|
|
|
|
converts to 48 kHz int16 stereo, and writes a shared App Group SPSC ring
|
|
|
|
|
|
(`BroadcastAudioRing.swift`); the host's `BroadcastAudioPump` owns the `SCREEN_AUDIO`
|
|
|
|
|
|
stream and feeds via `vc_stream_feed_pcm` (single session, no creds on disk). UI is an
|
|
|
|
|
|
`RPSystemBroadcastPickerView` in `VoiceControlsView`. (Replaced the speculative
|
|
|
|
|
|
`BroadcastCredentials.swift` self-connecting design, now removed.)
|
2026-06-20 17:52:09 +02:00
|
|
|
|
- [x] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — done
|
|
|
|
|
|
2026-06-20. Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public API with
|
|
|
|
|
|
stereo support. Adds a symmetric PCM sink fired on the playback thread per decoded remote
|
|
|
|
|
|
stream. Full wrappers for Swift (`feedPcm`/`setPcmSink`) and C# (`StreamFeedPcm`/
|
|
|
|
|
|
`SetPcmSink`). Three new C++ ctests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`,
|
|
|
|
|
|
`test_pcm_sink`), 4 Swift XCTest smoke tests, 4 C# xUnit smoke tests. Docs updated
|
|
|
|
|
|
(architecture.md §4 new subsection, voice.md §9 updated, protocol.md §8 explicit
|
|
|
|
|
|
no-protocol-change note, roadmap.md M5 entry). `ctest --preset dev` 23/23.
|
2026-06-20 23:30:52 +02:00
|
|
|
|
- [x] **macOS client UI overhaul** — done 2026-06-20. Mirrors the Windows client's UI
|
|
|
|
|
|
overhaul (toolbar, unified log, PM windows, channel counts, output volume, keyboard
|
|
|
|
|
|
shortcuts), adapted to Mac-native conventions:
|
|
|
|
|
|
- **NSToolbar**: Join Voice, Share Screen Audio, Mute, Deafen (SF Symbol toggle buttons),
|
|
|
|
|
|
and Output Volume slider (NSSlider 0–100, default 80). Voice actions + mute/deafen +
|
|
|
|
|
|
output volume moved out of the bottom voice panel into the toolbar. Bottom panel keeps
|
|
|
|
|
|
input-mode segmented control / VAD slider / PTT key / device picker / level meter.
|
|
|
|
|
|
- **Unified log**: chat `NSTextView` + activity `NSTableView` collapsed into a single
|
|
|
|
|
|
`NSTextView` — activity events in `secondaryLabelColor` (gray), chat in default color.
|
|
|
|
|
|
Removed `activityTableView` and `activityLog` array.
|
|
|
|
|
|
- **Private messaging**: scope dropdown removed; compose bar always sends to the current
|
|
|
|
|
|
channel. Each PM conversation opens in its own modeless `PrivateMessageWindowController`
|
|
|
|
|
|
(NSWindow). Incoming `.textMessage` with `.private` scope routed to the right window;
|
|
|
|
|
|
outgoing PMs echoed by server arrive through the same path. "Send Private Message…"
|
|
|
|
|
|
added to user context menu. "New Private Message…" (⌘⇧N) opens `UserPickerSheet`
|
|
|
|
|
|
listing all server users.
|
|
|
|
|
|
- **Channel counts**: outline view renders `"Name (n)"` with live user counts;
|
|
|
|
|
|
`refreshChannelTree()` called on `.userJoined`/`.userLeft` (was missing).
|
|
|
|
|
|
- **Voice menu** (⌘⇧V join/leave, ⌘⇧S share screen, ⌘⇧M mute, ⌘⇧D deafen) and **Messages
|
|
|
|
|
|
menu** (⌘⇧N new PM) added to `NSApp.mainMenu` via `NSMenuItem` key equivalents with
|
|
|
|
|
|
`[.command, .shift]` mask. Removed on `windowWillClose`. Mac-native: ⌘ not Ctrl, dispatched
|
|
|
|
|
|
by the responder chain (no custom key monitor needed).
|
|
|
|
|
|
- **Output volume**: `setOutputVolume(_:)` wrapper added to `VoiceCatClient.swift` (was
|
|
|
|
|
|
missing — the C ABI + C# wrapper shipped in commit 97fa659 but the Swift wrapper was
|
|
|
|
|
|
never added). Wired end-to-end: toolbar slider → `client.setOutputVolume(gain)`.
|
|
|
|
|
|
- **Part A (uncompiled Swift fix)**: the external PCM feed/tap Swift wrapper (commit
|
|
|
|
|
|
615d2a8) was never compiled — the local xcframework predating the `voicecat.h` PCM
|
|
|
|
|
|
additions. Fixed: rebuilt xcframework (regenerated module map), fixed `UInt`→`Int` type
|
|
|
|
|
|
mismatch in `feedPcm` (Swift imports `size_t` as `Int` not `UInt`), added
|
|
|
|
|
|
`VoiceCatPcmSinkCallback` typealias (Swift-idiomatic alias for the C `vc_pcm_sink_cb`
|
|
|
|
|
|
so consumers don't need to directly import `VoiceCatC`). `swift test` 10/10 green.
|
|
|
|
|
|
- **Audio settings moved to Settings window**: the bottom voice panel (input mode, VAD
|
|
|
|
|
|
slider, PTT key, device picker, level meter) was removed from the main window and moved
|
|
|
|
|
|
into a new `SettingsWindowController` — a modeless window opened via the app menu's
|
|
|
|
|
|
"Settings…" (⌘,) item. The main window is now just toolbar + channels + users + chat.
|
|
|
|
|
|
Source-of-truth for audio settings (`selectedInputMode`, `vadThresholdValue`,
|
|
|
|
|
|
`selectedInputDeviceId`, `pttKeyCode`) lives in `MainWindowController` so voice start can
|
|
|
|
|
|
apply them even before the settings window has been opened; `SettingsWindowController`
|
|
|
|
|
|
reads from and writes back to those properties and applies changes to the client
|
|
|
|
|
|
immediately when voice is active. The level meter is forwarded from
|
|
|
|
|
|
`MainWindowController.handleLevel` → `settingsWindowController.updateLevel(rms:)`.
|
|
|
|
|
|
`keyCodeName` helper deduplicated (was duplicated in `PttKeyCaptureSheet.swift` +
|
|
|
|
|
|
`MainWindowController.swift` — now shared from `MainWindowController.swift`).
|
|
|
|
|
|
- Files: `MainWindowController.swift` (overhauled), `PrivateMessageWindowController.swift`
|
|
|
|
|
|
(new), `UserPickerSheet.swift` (new), `SettingsWindowController.swift` (new),
|
|
|
|
|
|
`VoiceCatClient.swift` (setOutputVolume + VoiceCatPcmSinkCallback typealias + feedPcm
|
|
|
|
|
|
type fix), `ExternalPcmTests.swift` (use typealias), `PttKeyCaptureSheet.swift` (removed
|
|
|
|
|
|
duplicate `keyCodeName`), `VoiceCatMac.xcodeproj/project.pbxproj` (register 3 new files).
|
|
|
|
|
|
- **Platform-specific adaptations** (vs. Windows): `NSToolbar` instead of `ToolStrip`;
|
|
|
|
|
|
global menu bar + `NSMenuItem` key equivalents (⌘ not Ctrl, responder-chain dispatched);
|
|
|
|
|
|
PM windows as modeless `NSWindow`s; picker as Mac sheet; gray = `secondaryLabelColor`;
|
|
|
|
|
|
SF Symbols for toolbar icons.
|
2026-06-17 15:08:05 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-06-15 21:09:09 +02:00
|
|
|
|
## Decisions log
|
|
|
|
|
|
|
|
|
|
|
|
All architecture/scope decisions are settled and recorded in
|
|
|
|
|
|
[docs/roadmap.md §2 "Resolved decisions"](docs/roadmap.md) and reflected across `docs/`.
|
|
|
|
|
|
If you make a *new* decision, record it there and link it here.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## How to update this file
|
|
|
|
|
|
|
|
|
|
|
|
1. Check off tasks as you complete them; flip a milestone to `[x]` only when its **exit
|
|
|
|
|
|
criterion test** passes.
|
|
|
|
|
|
2. Keep the **"Where we left off / next action"** block at the top accurate — it's the first
|
|
|
|
|
|
thing the next agent reads.
|
|
|
|
|
|
3. When you start a milestone, copy its task list from `docs/roadmap.md` into a section here.
|
