Bump to v3.0.0: Opus live latency, codec rename, profile-resume after update, save-through-lock
* New "Opus, live latency" codec mode: 2.5 ms frames (120 samples/ch at 48 kHz)
via the float-input encode path. End-to-end codec delay drops to ~5 ms (vs
~12.5 ms at standard 10 ms Opus). Test on LAN: 400 pps/lane, zero missed /
reordered / duplicate packets, ~15 ms one-way saved end-to-end.
* Wire-format change: AudioFormatInfo.FrameDurationMilliseconds renamed to
FrameSamplesPerChannel (int sample-count at announced sample rate). Removes
the lossy 48000*ms/1000 conversion that couldn't represent 2.5 ms. v3 <-> v3
exact; v3 <-> v2 still passes audio (Opus decoder is self-describing from
packet TOC) but v2 side over-sizes its buffer wildly. v2.x profiles auto-
migrate via <120 sentinel rule in RemSoundSettingsStore (anything below 120
is treated as legacy ms and multiplied by 48). Profile JSON key kept as
OpusFrameMilliseconds via [JsonPropertyName] so old profile files still load.
* Codec dropdown rebuilt with use-case names: "PCM 48K 24 bit - uncompressed",
"Opus, broadcast quality - loss tolerant", "Opus, live latency - for jamming
and monitoring". Middle 10 ms option retired; saved 480-sample profiles
collapse to broadcast quality (safer-side default).
* Profile auto-resume after self-update: RemSoundUpdater writes a one-shot
_resume-after-update.txt sentinel containing the active profile title before
exit; Program.Main reads + deletes it on next start and silently loads that
profile, skipping the picker. Helper batch's robocopy /XF excludes the
sentinel and the failure-branch cleans it up if the install aborts. Falls
through to normal startup behaviour (StartWithProfileTitle or picker) if the
sentinel is missing, empty, or names a profile that no longer exists.
* Read-only profile saves now go through on explicit Ctrl+S / File -> Save
with a one-time TaskDialog warning ("Save anyway" / "Cancel" + Do-not-show-
again). Lock continues to suppress the automatic unsaved-changes prompt on
close / profile switch (its main job). AppConfig.SaveOnReadOnlyMessageSuppressed
renamed to SaveOnReadOnlyWarningSuppressed; v2.x suppression flag is silently
discarded since the behaviour changed and the user needs to see the warning
once on each machine.
* Manual (readme.html) updated: codec table rewritten with the three new
choices and corrected bandwidth figures, send-rate description updated, new
sections "The same profile picks up automatically after an update" and
"Saving on purpose while a profile is locked".
* Subsumes the never-separately-released v2.2 work: native Opus encoder
(~97% less per-second memory churn on Opus send path via Concentus.Native),
efficiency tidy-ups (item 4 ASIO probe rate, item 6 WaitHandle, item 7
snapshot cache, items 14/16 heartbeat + discovery), legacy cleanup
(items 30/34/35/36: KeepAlive infrastructure, drift drop/repeat/accumulator
fields, fan-out cache stat). New diagnostic columns cpu/memMB/wsMB/
allocKBps/captureMs/sendMs/recvMs/renderMs gated on Enable-logs.
About dialog updated with v3.0 block at top; v2.2 block retained for the
subsumed work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Ednunp
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
6d6d6897e4
commit
af0e7c3fff
@@ -265,3 +265,59 @@ The remaining items in the original list are now categorised honestly:
|
||||
* **Big remaining lever, requires real project**: replacement of NAudio for direct WASAPI/ASIO P/Invoke. Year-scale work. Not on the table now.
|
||||
|
||||
**RemSound's efficiency is in a healthy state.** Steady-state PCM-sending: 1.6-4.7 % CPU, 150 KB/s allocs. Receive side: 10 % CPU (mostly NAudio + ASIO host thread, not ours), 178 KB/s. Opus is no longer special. No further efficiency work is recommended in this round; the next time someone surfaces a real performance complaint, this document should be the starting point for measuring before guessing.
|
||||
|
||||
---
|
||||
|
||||
## Design note (forward-looking): low-latency Opus mode, v3.0 candidate
|
||||
|
||||
Captured 2026-05-23 — Ed asked whether we can get Opus closer to PCM-latency, citing Jamulus as the reference for "very very fast" Opus. Recorded here so a future session has the lay of the land without re-deriving it from scratch.
|
||||
|
||||
### What Jamulus actually does
|
||||
|
||||
Jamulus's low latency comes from a stack of choices, not one trick:
|
||||
|
||||
1. **Opus Custom CELT**, not standard Opus. Opus Custom is a non-standard build of libopus exposing the CELT layer directly (it skips the SILK/CELT decision layer, and accepts non-standard frame sizes / sample rates). Requires libopus to be compiled with `--enable-custom-modes` — that flag is OFF by default in stock builds, including the binaries that ship in the `Concentus.Native` NuGet package we use today.
|
||||
2. **2.67 ms frames** (128 samples at 48 kHz), or 5.33 ms (256 samples). That's the dominant latency win. Algorithmic latency in Opus is roughly `frame size + 2.5 ms look-ahead` — at 2.67 ms frames you're at ~5 ms total instead of the ~12.5 ms a 10 ms standard Opus frame gives.
|
||||
3. **Complexity 1** (we use 10) — cheaper encode, marginally lower quality at the same bitrate.
|
||||
4. **CBR, no FEC, int16 input, joint stereo** — Jamulus prioritises predictable bandwidth and minimum per-packet overhead over robustness. We use VBR + FEC + float input because RemSound runs over WAN via Tailscale and needs to survive packet loss.
|
||||
5. **Tiny jitter buffer** — often 1-2 frames. RemSound's is sized for forgiveness, not LAN snappiness.
|
||||
|
||||
### What RemSound v3.0 adds (shipped 2026-05-23)
|
||||
|
||||
Standard libopus in `OPUS_APPLICATION_RESTRICTED_LOWDELAY` mode goes down to 2.5 ms frames at 48 kHz — same algorithmic latency as Opus Custom at the same frame size. The "Custom" bit gives you sub-frame-size flexibility (any sample count, any sample rate), not lower latency per se. So most of the Jamulus latency advantage is available from stock libopus, and that's what v3.0 pursues.
|
||||
|
||||
**Test result (2026-05-23, desktop↔laptop on LAN, ASIO Audient + Tight rate + auto-tune):** at the new 2.5 ms mode, send-accum dropped from 5 ms to 1.3 ms (3.75 ms shaved), auto-tune pulled the receive buffer to 19 ms (vs 24–29 ms at 10 ms Opus, 5–10 ms shaved). End-to-end one-way ≈ 30 ms vs ≈ 45 ms at standard 10 ms. **~15 ms shaved one-way, ~30 ms round-trip.** Wire integrity perfect over the test session: 400 packets/sec/lane, zero missed, zero reordered, zero duplicates, zero FEC recoveries triggered. CPU on the sender doubled to ~12 % (was ~5 % at PCM), receiver ~9–23 %. Memory rock-solid (gen0/1/2 zero most ticks).
|
||||
|
||||
**Refactor shipped in v3.0:**
|
||||
|
||||
* `AudioFormatInfo.FrameDurationMilliseconds` → `FrameSamplesPerChannel`. The wire field at byte offset 28 of the format payload now carries the exact sample-count per channel at the announced sample rate. v2.x receivers reading a v3.0 format packet misinterpret 480 (= 10 ms in samples) as 480 ms; the actual Opus decode still works because the decoder is self-describing from the packet TOC byte, but their buffer sizing will be wildly off. v2 ↔ v3 audio works but with absurd buffer latency. v3 ↔ v3 is exact.
|
||||
* `OpusEncoderState` constructor now takes `frameSamplesPerChannel` directly — the lossy `48000 × ms / 1000` conversion (which couldn't represent 2.5 ms) is gone.
|
||||
* `AudioSender.OpusFrameMilliseconds` → `OpusFrameSamplesPerChannel`; `ConfigureCodec(codec, samples)` accepts 120 to 2880 (= 2.5 ms to 60 ms).
|
||||
* `SenderLane` constructor + `OnCodecChanged` signatures and `EnsureFormatPacketSent` propagate sample-counts straight through.
|
||||
* `MainFormChoices.CodecChoice` field renamed `OpusFrameMs` → `OpusFrameSamples`.
|
||||
* `MainForm.cs` codec dropdown: PCM, Opus 20 ms (960), Opus 10 ms (480), Opus 2.5 ms experimental (120). `EffectiveOpusFrameSamples` halves for Tight rate with a floor of 120 (so picking 2.5 ms + Tight stays at 2.5 ms; picking 20 ms + Tight gives 10 ms). `FormatCodecLabel` derives ms display from samples / 48 with one decimal so "Opus 2.5ms" displays cleanly. `ResolveCodecIndex` maps 960/480/120 to dropdown indices.
|
||||
* `RemSoundSettingsStore.LoadOpusFrameSamplesPerChannel` / `SaveOpusFrameSamplesPerChannel` (in-memory cache rename). The load path includes a `<120` sentinel migration: any persisted value below 120 is treated as legacy v2.x integer-ms and multiplied by 48. The ranges don't overlap (max legitimate ms = 60, min legitimate samples = 120), so the disambiguation is unambiguous.
|
||||
* `Profile.OpusFrameMilliseconds` → `Profile.OpusFrameSamplesPerChannel`. The JSON key is **kept** as `"OpusFrameMilliseconds"` via `[JsonPropertyName]`, so v2.x profile files still load (the `<120` sentinel migration converts their ms value on read).
|
||||
* `AudioReceiver.ActiveStreamFrameMs` rounds samples-per-channel up to the next integer ms so the auto-tune always overestimates rather than underestimates the codec floor.
|
||||
* `StreamSession.MatchesFormat` + `HandleOpus` use `FrameSamplesPerChannel` directly; the buffer-size calc dropped its `× SampleRate / 1000` conversion.
|
||||
|
||||
Wire-format compatibility: v3.x ↔ v3.x exact; v2.x ↔ v3.x produces garbled buffer sizing but the Opus stream still decodes (TOC-self-describing). Users on the field upgrade both ends together.
|
||||
|
||||
### What 2.5 ms gives you in real-world test
|
||||
|
||||
* **~5 ms codec latency** (vs ~12.5 ms at the standard Opus 10 ms choice) — about 7.5 ms shaved off the end-to-end path. Within ~0.17 ms of Jamulus's Opus Custom 2.67 ms frame.
|
||||
* **4× the packet rate per lane** (200 pps → 800 pps) — measurably more CPU per second, more pressure on the kernel UDP send path, slightly worse single-packet-loss tolerance.
|
||||
* **Same encoder otherwise** — RESTRICTED_LOWDELAY, native libopus, FEC on, bitrate unchanged.
|
||||
|
||||
### What Opus Custom would add on top of 2.5 ms (deferred to v3.1+ if ever)
|
||||
|
||||
Once 2.5 ms standard libopus is in real-world use, the remaining gap to Jamulus is:
|
||||
|
||||
* **Sub-2.5 ms frames** (Jamulus uses 64 samples = 1.33 ms, below stock libopus's floor). Saves ~1.2 ms more codec latency. At that point we're below the CPU-scheduling jitter floor on most consumer Windows hardware — the saving is theoretical more than perceptible.
|
||||
* **Skipping standard Opus packet framing** — saves 1-2 bytes per packet (about 1 % of total bandwidth at 2.5 ms frames). Trivial.
|
||||
|
||||
Cost to add: half a day to a day of native-binary work (build libopus with `--enable-custom-modes`, P/Invoke layer for `opus_custom_mode_create` / `opus_custom_encoder_create` / `opus_custom_encode_float` / matching decode, wire-format extension to mark packets as custom-mode, NuGet/packaging plumbing).
|
||||
|
||||
**Recommendation: do not pursue Opus Custom unless 2.5 ms in real-world testing shows a perceptible gap to Jamulus and the listening tests prove the missing ~1 ms matters.** Most users won't be able to tell.
|
||||
|
||||
Skip the tight-jitter receive-buffer mode (1-2 frames) unless it surfaces as a complaint independently — the current buffer sizing is what makes Wi-Fi tolerable.
|
||||
|
||||
Reference in New Issue
Block a user