From 68bd2cf2ceddba47b5e09dff6181f740f2a1cc6f Mon Sep 17 00:00:00 2001 From: Jage9 Date: Sat, 21 Feb 2026 16:42:42 -0500 Subject: [PATCH] Add core architecture and workflow documentation --- README.md | 10 ++++ docs/audio-architecture.md | 53 +++++++++++++++++++++ docs/controls.md | 63 +++++++++++++++++++++++++ docs/item-schema.md | 1 - docs/item-types.md | 97 ++++++++++++++++++++++++++++++++++++++ docs/protocol-notes.md | 42 +++++++++++++++++ docs/runtime-flow.md | 54 +++++++++++++++++++++ 7 files changed, 319 insertions(+), 1 deletion(-) create mode 100644 docs/audio-architecture.md create mode 100644 docs/controls.md create mode 100644 docs/item-types.md create mode 100644 docs/protocol-notes.md create mode 100644 docs/runtime-flow.md diff --git a/README.md b/README.md index a747230..1cc8418 100644 --- a/README.md +++ b/README.md @@ -38,3 +38,13 @@ Summary: - Client sounds: `client/public/sounds/` - Server config template: `server/config.example.toml` - Server runtime items: `server/runtime/items.json` + +## Documentation + +- Controls/keymap: `docs/controls.md` +- Audio architecture and layers: `docs/audio-architecture.md` +- Item behavior by type: `docs/item-types.md` +- Runtime lifecycle flow: `docs/runtime-flow.md` +- Protocol behavior notes: `docs/protocol-notes.md` +- Item schema reference: `docs/item-schema.md` +- Local dev commands: `docs/local.md` diff --git a/docs/audio-architecture.md b/docs/audio-architecture.md new file mode 100644 index 0000000..96b4c92 --- /dev/null +++ b/docs/audio-architecture.md @@ -0,0 +1,53 @@ +# Audio Architecture + +## Audio Domains + +- Voice: remote WebRTC peer audio. +- Media: radio station streams. +- Item: looping item emit sounds (`emitSound`). +- World: one-shot spatial world events from others (movement/teleport and item-use spatial sounds). +- UI: interface tones and status cues (not layer-controlled). + +## Layer Toggles + +Runtime toggles in normal mode: + +- `1`: voice +- `2`: item +- `3`: media +- `4`: world + +Persisted in local storage key `chatGridAudioLayers`. + +## Layer Off Behavior + +Layer off prefers unsubscribe/cleanup instead of only muting: + +- Voice: remote peer audio graph is detached; resumes by reattaching stored remote streams. +- Media: `RadioStationRuntime.cleanupAll()` and no sync/update processing until re-enabled. +- Item: `ItemEmitRuntime.cleanupAll()` and no sync/update processing until re-enabled. +- World: world one-shots are not played while disabled. + +## Item Sound Model + +- `useSound`: one-shot played on successful `item_use` (`item_use_sound` packet). +- `emitSound`: continuous looping spatial source attached to an item runtime. + +Current defaults: + +- `radio_station`: `useSound=none`, `emitSound=none` +- `dice`: `useSound=sounds/roll.ogg`, `emitSound=none` +- `wheel`: `useSound=sounds/spin.ogg`, `emitSound=none` +- `clock`: `useSound=none`, `emitSound=sounds/clock.ogg` + +`emitSound` uses a base gain multiplier of `0.3` before spatial attenuation. + +## Spatialization + +- Distance attenuation uses hearing radius from game state. +- Stereo panning follows horizontal offset. +- Mono output mode collapses pan to center. + +## Stale Stream Mitigation + +Radio stream startup appends a cache-busting query token on runtime creation to avoid stale buffered playback after reconnect/layer re-enable. diff --git a/docs/controls.md b/docs/controls.md new file mode 100644 index 0000000..707642b --- /dev/null +++ b/docs/controls.md @@ -0,0 +1,63 @@ +# Controls Reference + +This document is the authoritative keymap for the client. + +## Normal Mode + +### Movement +- `Arrow Keys`: Move +- `C`: Speak coordinates +- `Escape`: Press once for disconnect prompt, press again to disconnect + +### Users, Nickname, Chat +- `L`: Locate nearest user +- `Shift+L`: List users and teleport to selected user with `Enter` +- `Shift+U`: Speak connected users +- `N`: Edit nickname +- `/`: Start chat +- `,` / `.`: Previous/next message +- `<` / `>`: First/last message + +### Items +- `I`: Locate nearest item +- `Shift+I`: List items and teleport to selected item with `Enter` +- `A`: Add item +- `O`: Edit item properties +- `Shift+O`: Inspect all item properties +- `D`: Pick up/drop item +- `Shift+D`: Delete item +- `U`: Use item + +### Audio +- `P`: Ping server +- `M`: Mute/unmute local microphone +- `Shift+M`: Toggle stereo/mono output +- `Shift+1` (`!`): Toggle loopback monitor +- `1`: Toggle voice layer +- `2`: Toggle item layer (emit sounds) +- `3`: Toggle media layer (radio) +- `4`: Toggle world layer (other-user world sounds) +- `E`: Effect select menu +- `-` / `=`: Lower/raise active effect value + +## Text Entry Modes (`nickname`, `chat`, `itemPropertyEdit`) + +- `Enter`: Confirm +- `Escape`: Cancel +- `ArrowLeft` / `ArrowRight`: Move cursor by character +- `Ctrl+ArrowLeft` / `Ctrl+ArrowRight`: Move cursor by word (notepad-style) +- `Home` / `End`: Move to start/end +- `Backspace`: Delete previous character +- `Ctrl+A`: Select all (replace-on-next-type) +- `Ctrl+C`: Copy current text +- `Ctrl+X`: Cut current text +- `Ctrl+V`: Paste + +## Menu/List Navigation Modes + +Applies to effect select, user/item list modes, item selection, item property list, and property option select. + +- `ArrowUp` / `ArrowDown`: Move selection +- `Enter`: Confirm selection +- `Escape`: Exit/cancel +- First-letter navigation: jump to next matching entry diff --git a/docs/item-schema.md b/docs/item-schema.md index 968a155..5d0cb73 100644 --- a/docs/item-schema.md +++ b/docs/item-schema.md @@ -105,7 +105,6 @@ } ``` -- `timeZone`: one of the supported IANA zones listed below. - `timeZone`: one representative IANA zone per world UTC offset. Includes: `America/Anchorage`, `America/Argentina/Buenos_Aires`, `America/Chicago`, `America/Detroit`, `America/Halifax`, `America/Indiana/Indianapolis`, `America/Kentucky/Louisville`, diff --git a/docs/item-types.md b/docs/item-types.md new file mode 100644 index 0000000..c478032 --- /dev/null +++ b/docs/item-types.md @@ -0,0 +1,97 @@ +# Item Types + +This is behavior-focused documentation for item types and their defaults. + +## Shared Item Behavior + +- Items are server-authoritative. +- Global per-type fields are injected by the server and are not persisted per-instance: + - `capabilities` + - `useSound` + - `emitSound` + - `useCooldownMs` (from item catalog) +- Instance fields are persisted in `server/runtime/items.json`. + +## `radio_station` + +### Defaults +- Title: `radio` +- Params: + - `streamUrl=""` + - `enabled=true` + - `channel="stereo"` + - `volume=50` + - `effect="off"` + - `effectValue=50` +- Global: + - `useSound=none` + - `emitSound=none` + - `useCooldownMs=1000` + +### Use +- `use` toggles `enabled` on/off and broadcasts chat status. + +### Validation +- `channel`: `stereo | mono | left | right` +- `volume`: integer `0..100` +- `effect`: `reverb | echo | flanger | high_pass | low_pass | off` +- `effectValue`: number `0..100` with `0.1` precision + +## `dice` + +### Defaults +- Title: `Dice` +- Params: + - `sides=6` + - `number=2` +- Global: + - `useSound=sounds/roll.ogg` + - `emitSound=none` + - `useCooldownMs=1000` + +### Use +- Rolls `number` dice with `sides` sides and reports values + total. + +### Validation +- `sides`: integer `1..100` +- `number`: integer `1..100` + +## `wheel` + +### Defaults +- Title: `wheel` +- Params: + - `spaces="yes, no"` +- Global: + - `useSound=sounds/spin.ogg` + - `emitSound=none` + - `useCooldownMs=4000` + +### Use +- Announces spin immediately. +- Result is sent after delay. + +### Validation +- `spaces`: comma-delimited values +- At least 1 entry +- Max 100 entries +- Max 80 chars per entry + +## `clock` + +### Defaults +- Title: `clock` +- Params: + - `timeZone="America/Detroit"` + - `use24Hour=false` +- Global: + - `useSound=none` + - `emitSound=sounds/clock.ogg` + - `useCooldownMs=1000` + +### Use +- Reports current time from item timezone and format. + +### Validation +- `timeZone`: one of `CLOCK_TIME_ZONE_OPTIONS` in `server/app/item_catalog.py` +- `use24Hour`: boolean or on/off style input diff --git a/docs/protocol-notes.md b/docs/protocol-notes.md new file mode 100644 index 0000000..ee52c8d --- /dev/null +++ b/docs/protocol-notes.md @@ -0,0 +1,42 @@ +# Protocol Notes + +This is a behavior guide for packet semantics beyond raw schemas. + +## Direction + +- Client packet schema lives in `server/app/models.py` (`ClientPacket`). +- Browser-side validation/parsing lives in `client/src/network/protocol.ts`. +- Keep these synchronized on every protocol change. + +## Client -> Server + +- `update_position`: authoritative player position update. +- `update_nickname`: nickname change request (server enforces uniqueness). +- `chat_message`: player chat. +- `ping`: latency measurement. +- `item_add`, `item_pickup`, `item_drop`, `item_delete`, `item_use`, `item_update`: item actions. + +## Server -> Client + +- `welcome`: initial snapshot with users/items. +- `signal`: forwarded WebRTC offer/answer/ICE. +- `update_position`, `update_nickname`, `user_left`: presence updates. +- `chat_message`: system and user chat stream. +- `pong`: ping response. +- `nickname_result`: accepted/rejected nickname result. +- `item_upsert`: full item replacement after mutation. +- `item_remove`: item deletion. +- `item_action_result`: action success/failure and user-facing message. +- `item_use_sound`: spatial one-shot sound on successful item use (if `useSound` configured). + +## Item Packet Behavior + +- `item_upsert` is full-state replacement for one item, not partial patch. +- `item_action_result` messages are intended for direct screen-reader/user status feedback. +- `item_use_sound` contains absolute item world coordinates (`x`, `y`) and sound path. + +## Validation Boundaries + +- Server is authoritative for all action validation and normalization. +- Client validates incoming packet shapes and applies runtime behavior. +- Client-side item edit validation is convenience only; server remains source of truth. diff --git a/docs/runtime-flow.md b/docs/runtime-flow.md new file mode 100644 index 0000000..333b008 --- /dev/null +++ b/docs/runtime-flow.md @@ -0,0 +1,54 @@ +# Runtime Flow + +## Connect Flow + +1. User clicks connect. +2. Client validates nickname and sets up local media. +3. Client connects signaling websocket. +4. Server sends `welcome` with users/items snapshot. +5. Client: + - sends initial `update_position` + - sends initial `update_nickname` + - creates peer runtimes for known users + - syncs item runtimes (`radio`, `emit`) + - applies audio layer state + - starts game loop + +## Main Loop + +Each frame: + +- Handle local movement input. +- Update spatial voice audio. +- Update spatial radio audio. +- Update spatial item emit audio. +- Draw canvas scene. + +## Message Handling + +Core incoming message effects: + +- `signal`: WebRTC negotiation and ICE exchange. +- `update_position`: update peer position; may play movement/teleport world sound. +- `update_nickname`: update peer display name. +- `chat_message`: append/readable status; optional system sound class. +- `item_upsert`: replace item snapshot and resync item runtimes. +- `item_remove`: remove item and cleanup runtimes. +- `item_action_result`: success/error status for actions. +- `item_use_sound`: play one-shot spatial sample (world layer gated). + +## Disconnect/Cleanup + +On disconnect: + +- Close signaling. +- Stop local media tracks. +- Cleanup peers and all audio runtimes. +- Reset UI/mode state and lists. + +## Runtime Components + +- `PeerManager`: peer connection lifecycle and remote track attach. +- `RadioStationRuntime`: shared stream sources + per-item output/effects/spatialization. +- `ItemEmitRuntime`: per-item looping emit source + spatialization. +- `AudioEngine`: shared audio context, samples, effects, voice graph.