alchemist-accessibility/accessibility/docs/scratch.md

# SCRATCH — current working state

**Read this first on any handoff.** Living document: update it whenever state changes, before ending a turn.

Last updated: 2026-08-01 · Currently: **Phases 0–3 done + menus/dialogue.
Phase 4 mostly done. Phase 5 (status, popups) next.**

**Game is playable start-to-finish by ear** from main menu through the intro
conversation into gameplay — verified by the testbed, **13/13 checks**.

### Run the testbed (do this after any change)
```
cd "D:/code/Sulphur Memories Alchemist 0.4.2 sanctified"
rm -rf ~/AppData/Roaming/LOVE/Alchemist        # start from a clean save
"C:/Program Files/LOVE/lovec.exe" . --a11y-test
```
It drives the game by code via `love.event.push`, navigating by *reading* focus
rather than counting keypresses. Every bug listed below was caught by it.

---

## What this project is

Blind-accessibility mod for *Sulphur Memories: Alchemist* 0.4.2 (LÖVE 11.5, Lua/LuaJIT), extracted to
`D:\code\Sulphur Memories Alchemist 0.4.2 sanctified`. All mod code lives under `accessibility/`.
Design goal: **change as little game code as possible** so the mod re-applies cleanly to new versions.

Full research is in `accessibility/docs/` — read `README.md` there for the map.

---

## Confirmed decisions (do not relitigate)

| # | Decision |
|---|---|
| D1 | Dev uses system LÖVE at `C:\Program Files\LOVE\` (`lovec.exe` for console output). Distribution deferred. |
| D2 | Tracker hierarchy: category (`Ctrl+PgUp/Dn`) → kind (`PgUp/Dn`) → instance (`Shift+PgUp/Dn`), `Home` = distance+direction. |
| D3 | **Small explicit patches** to game files are allowed where they simplify things — but every state descriptor also carries a `probe` spec so we can flip to zero-edit `debug.getupvalue` mode if the game ships fused on Steam. `render` is shared by both backends. |
| D4 | Six tracker categories confirmed and coverage-verified: Creatures / Items / Exits / Interactables / Terrain features / Alchemy. |
| — | Screen reader: **NVDA**. Running the game via `lovec` during dev is approved. |
| — | User is blind — all verification must be describable in text; don't rely on screenshots. |

## Still-open questions (in `04-open-questions.md`)

**Answered:** Q1 (dev LÖVE), Q2 (tracker hierarchy), Q3 (augment, not relay),
Q4 (speak everything queued, cancelled by explicit action), Q5 (wall tones on
step, stop-and-replay — confirmed working by ear), Q7/Q10 (patches allowed,
probe backend kept open for Steam), Q9 (NVDA, lovec approved).

**Still open:** Q6 (integrate our keys into the game's rebinding UI, or keep a
separate scheme) · Q8 (full keyboard-reachability audit as an explicit
deliverable?). Neither blocks Phase 4.

---

## Key facts worth not rediscovering

- **`console.print(msg, colorTemplate)`** in `modules/console.lua` is the *single* chokepoint for all game log text. Repeated identical messages coalesce into `"msg (x3)"` via a `times` counter. Buffer entries expire after 3s — useless as history, so we keep our own.
- **`level.getCursorMessage()`** returns the description under the examine cursor (actor first, then top visible tile). The game already has a full arrow-key examine mode: `states/ingame_observe.lua`.
- **`inventoryCursorPopup.set(...)`** — one patch narrates focus in `inventory`, `craft`, `refine` and `trade`. ⚠ `levelCursorPopup` is deliberately **not** patched: it fires from an animation-completion callback and lands a keypress late. `ingame_observe` polls `level.cursorX/Y` instead.
- **All 28 states already implement `keypressed`.** The gap is narration, not reachability.
- **`modules/tile.lua:234`**: `tile.tryInteract = (self.loot or self.tryInteractFunction)`. ⚠ This is **not** a "worth tracking" signal — `wall` has a flavour-only `tryInteractFunction`. Structural tiles (`blocksMovement and blocksVision`) must be excluded first. See the D4 correction in `05-decisions.md`.
- **Template counts:** 231 tiles, 40 actors, 344 items. Parent inheritance is *already flattened* by `modules/tile/preprocess.lua` during `love.load`, so read fields directly.
- **6 un-nameable mechanical templates to exclude:** `blank`, `throw_blocker`, `movement_blocker`, `fire_detector`, `wall_features_position_dummy`, `ladder_down_hidden`.
- **`debug.getupvalue` works** on this Lua build — verified reading `selectedOption` / `optionsList` from a state's `keypressed`.
- **Tolk constraint:** `ffi.load("Tolk")` uses the OS loader, so DLLs must sit next to `love.exe` — they can *never* live inside a `.love` archive.
- `tolk.lua` calls `Tolk_Load()` at require time with no error handling, and `Love2talk.lua` leaks `backend` as a global. Always `pcall` the require.

---

## Current status

### Done
- Full architecture research → `docs/01-architecture.md`
- Hooking strategy → `docs/02-hooking-strategy.md`
- Phased plan → `docs/03-implementation-plan.md`
- Open questions → `docs/04-open-questions.md`
- Decisions + category coverage proof → `docs/05-decisions.md`

### Done — Phase 0 (bootstrap) · commit `5c581ed`
- `git init`, pristine tree committed and tagged **`vanilla-0.4.2`** (commit `bbb70d0`,
  1838 files, `accessibility/` deliberately excluded so the tag is genuinely clean).
- love2talk vendored to `accessibility/vendor/love2talk/` — see `VENDORED.md` there
  for the five local modifications.
- **DLL elevation problem solved.** `Tolk.dll` etc. live in `accessibility/lib/` and
  are loaded by *absolute path* via `love.filesystem.getSource()`, with
  `SetDllDirectoryA` pointing at that folder so Tolk can find its own screen-reader
  clients. No admin rights, no copying into `Program Files`. Tracked in git via a
  `!accessibility/lib/*.dll` un-ignore.
- `accessibility/speech.lua` — pcall-guarded, frame-queued, `logToConsole = true`.
- `accessibility/init.lua` — patch registry + `states.switch` / `states.update` hooks
  + startup self-test.
- `accessibility/stateNames.lua` — spoken names for all 28 states.
- **One line added to `main.lua`.** Nothing else in the game touched.

**Verified working** via `lovec.exe .`:
```
[accessibility] TTS initialised
[accessibility] TTS: available
[accessibility] speech channel: yes
[speech]! Main menu
```

### Done — Phase 1: console history + review keys · + wall tones
- `accessibility/history.lua` — unbounded (1000-entry ring) history. Mirrors the
  game's `(xN)` coalescing so speech matches the screen. Review cursor **holds
  still while new lines arrive** and only follows the tail when not being used.
- `accessibility/hotkeys.lua` — global key layer in front of `states.keypressed`.
- `accessibility/patches/console.lua` — wraps `console.print` (calls the real one
  *first*, so a mod bug can't swallow an on-screen message) and `console.clear`.
- `accessibility/walltones.lua` — see below.
- `accessibility/tests/test_history.lua` — 16 assertions, all passing.
  Run with: `lua accessibility/tests/test_history.lua` (pure Lua, no LÖVE needed).

**Auto-speak policy (Q4, answered):** speak everything, queued and joined in order
at end of frame. Any keypress that reaches the game counts as an explicit action
and silences the pending backlog (`hotkeys.onGameKey` → `speech.silence`).

**Verified in-game** by the testbed (console capture + `(x2)` fold + review keys)
and by Talon at the keyboard (wall tones).

### Keymap so far

| Key | Action |
|---|---|
| `[` / `]` | Previous / next console history line |
| `Shift+[` / `Shift+]` | Oldest / newest line |
| `\` | Repeat current line |
| `Shift+\` | Copy current line to clipboard |
| `/` | Scan all 8 neighbours, clockwise from north (terse) |
| `;` | Where am I — level, in-game time, coordinates |
| `Ctrl`+numpad 1–9 | Probe that direction without moving (`Ctrl+kp5` = standing on) |
| `Ctrl+PageUp` / `Ctrl+PageDown` | Tracker: previous / next **category** |
| `PageUp` / `PageDown` | Tracker: previous / next **kind** |
| `Shift+PageUp` / `Shift+PageDown` | Tracker: previous / next **instance** |
| `Home` | Tracker: distance, direction and path length to current instance |

Game keys worth knowing: **`x`** enters examine mode (not `e`), `f` finds exits
while examining, `j` journal, `i` inventory, `c` character, `a` actions menu.

Reserved for the tracker (Phase 3): `PageUp`/`PageDown`, `Home`, with Ctrl/Shift.

**Free scancodes** (game binds none of these): `[ ] \ ; ' , / - =`, `h k l m n o
p u v w`, `F1`–`F12`, `pageup pagedown home end insert`, `kp0 kp. kp+ kp- kp* kp/`.

**Game-claimed scancodes — never bind these unmodified:**
`. 0-9 \` a b backspace c d delete down e escape f g i j kp1-kp9 kpenter lalt
lctrl left lshift mouse3 q r ralt rctrl return right rshift s space t tab up x y z`

⚠ **`keybindings.getCommand()` dispatches on scancode alone and ignores
modifiers.** Ctrl and Shift are *held* modifiers in game (`moveCamera`,
`repeatAction`/`adjustByFive`). So `Ctrl+Up` is seen by the game as plain "up" —
a movement. Never bind a modifier combo over a game-claimed scancode. Combos over
*unclaimed* scancodes (e.g. `Ctrl+PageUp`) are safe. Always consume our keys.

### Wall tones (Q5, answered: on step, 3D-positioned, radius 5)

Key design point, in `walltones.lua`: **direction and distance are decoupled.**
The game's listener sits at the camera, 4 units above the plane, with an *inverse*
distance model. Placing a tone at its wall's real position would (a) make far
walls quieter — the opposite of what was asked — and (b) collapse the pan toward
centre, since the 4-unit vertical offset dominates a 2-unit horizontal one.

So: the source is placed at a **fixed radius** from the listener in the wall's
direction, at the listener's own height (pure horizontal pan, constant
attenuation), and **volume is computed explicitly** from distance. Positioned
relative to the listener via `love.audio.getPosition()`, so panning stays correct
even while the camera is panned away.

Step detection **polls** `playerActor.x/y` each frame instead of hooking movement
— free, and covers walking, pathing, portals and pushes by construction.

⚠ `config.louderWhenFurther = true` as requested, but this inverts the usual
convention (proximity cues normally get louder as you approach, and a near wall is
the more urgent fact). Flip the flag in `walltones.lua` if it reads backwards.

Respects `FoV`/`rememberedFoV` — never leaks unseen map geometry.

### Done — Menu narration (Phase 4, pulled forward) + testbed

Pulled forward because menus were a **blocker**: Talon could hear the screen name
but not the selected option, so the game could not be started at all.

- `accessibility/menus.lua` — descriptor driver. Supports both D3 backends
  (`probe` via `debug.getupvalue`, `patched` via injected announce calls); the
  per-screen `render` is shared. Includes the probe self-test.
- `accessibility/states/{main_menu,settings_menu,load_game,new_game,narration}.lua`
- `accessibility/testbed.lua` — drives the game by code. `lovec.exe . --a11y-test`

**Verified end to end: main menu → new game → name entry → intro narration →
gameplay, 8/8 checks passing.** Console patch smoke-tested including the `(x2)`
fold and review keys.

#### Three real bugs the testbed caught

1. **`love.keypressed` was bound to the pre-wrapper function.** `states.setup()`
   does `love[callback] = love[callback] or states[callback]`, capturing **by
   value**, and it runs before us. `main.lua` defines `love.load/update/
   mousepressed/mousereleased/quit` but NOT `keypressed`/`textinput` — so those
   were frozen to the original. Replacing the `states` table field alone did
   nothing. **Fix: reassign `love.keypressed`/`love.textinput` after wrapping.**
   `states.switch`/`states.update` were fine — looked up on the table at call time.
   ⚠ Any future callback we wrap needs the same treatment.

2. **Focus moves with no keypress.** `main_menu.mousemoved` assigns
   `selectedOption` straight from cursor position, so wherever the mouse rests
   silently overrides selection — including at startup. **Fix: poll focus in the
   update hook** rather than announcing only on keypress. `announceScreen` now
   speaks only the screen name and lets the poll add focus a frame later
   (`justSwitched` makes that first one non-interrupting so it appends).

3. **Probes may need chaining.** `narration.keypressed` doesn't touch
   `displayData`/`currentPage`/`currentLoadedLine` — it only calls
   `continueNarration()`, so they are upvalues of *that* function. The descriptor
   reaches `continueNarration` as an upvalue of `keypressed`, then reads through
   it. Expect this pattern in other states.

#### Also learned
- `states.lua` loads states via `require("states/"..name)` — a **SLASH**.
  `require()` caches on the exact string, so `require("states.main_menu")` would
  load a *second independent copy* with its own upvalues. Always match the slash.
- **Plain movement is silent** — `actions/move.lua` only prints when a move fails
  (bound, immobilized). Walking proves nothing about the console patch; drive
  `console.print` directly to test it.

### Done — dialogue narration + wall tone retrigger fix

**`ingame_dialogue` descriptor.** Speaks speaker + message on a *node* change, but
only the response on a *selection* change (re-reading the whole speech every time
you move between responses would be unusable). Responses can also be picked
directly with number keys 1–0, which the game already supports.

Two more bugs found and fixed:

4. **Nested `states.switch` announced in reverse.** `ingame.open()` sees
   `world.pendingDialogue` and switches straight to `ingame_dialogue`. Since we
   announce *after* `realSwitch` returns, the inner announcement ("Conversation")
   fired first and the outer ("Game") then interrupted and clobbered it.
   **Fix: after switching, if `states.getCurrent() ~= newState`, stay quiet** —
   a nested switch already announced.

5. **Double numbering.** The game already prefixes response text with its number
   (`"1. (Continue)"`, matching the number key that selects it), so adding ours
   gave `"1. 1. (Continue)"`. Now only added when absent.

**Wall tones — retrigger fix (per Talon: "stop previous tones and play new ones").**
Tones are **2.33 s, mono, 44.1 kHz**. Steps land far faster than that and up to
four directions sound at once, so they stacked into a continuous drone. Now each
direction stops its previous source before starting a new one, so it reads as a
per-step cue layered over footsteps, and the volume you hear always reflects
where you're standing now. `playing` is cleared on level change.

*(Mono matters: OpenAL only spatialises mono sources. Stereo would not pan.)*

### Done — Phase 2: tile examination + surroundings

- `accessibility/describe.lua` — shared spatial formatting. 8-point compass,
  **Chebyshev** distance (the game moves 8-directionally, so Euclidean would
  overstate diagonals), and `tileFacts()` / `tile()`. Verbosity setting
  (`terse`/`normal`/`verbose`) since sweeping 8 neighbours and inspecting one
  tile want different amounts of detail.
- `accessibility/states/ingame_observe.lua` — narrates the game's examine cursor.
- `accessibility/surroundings.lua` — probe/scan/where-am-I without moving.
- `accessibility/patches/popups.lua` — wraps `inventoryCursorPopup.set`, covering
  **inventory, craft, refine, trade** in one patch.

**Q3 answered by default: augment, not just relay.** The game's descriptions
("A wooden chair.") carry no position — fine on screen, useless by ear. We append
distance + direction, and blocked/has-items/water at normal verbosity.

#### Design note: why examine mode is polled, not popup-driven
`levelCursorPopup.set` is called from an **animation-completion callback**, so
unless the player enables the game's `quickExaminationPopup` option the text
lands a full keypress late — in testing you heard the *previous* tile while
standing on the next. The `ingame_observe` descriptor polls `level.cursorX/Y`
instead: same-frame, setting-independent, no game edit. `levelCursorPopup` is
therefore deliberately **not** patched; only `inventoryCursorPopup` is.

*(The level model is a closure instance, not a module, so it can't be patched at
require time — reading its public fields is the decoupled route.)*

#### Testability seam
`hotkeys.isCtrlDown` / `isShiftDown` are overridable fields, not direct calls to
`love.keyboard.isScancodeDown`. Real hardware state can't be faked by
`love.event.push`, so without this seam every modifier-gated hotkey would be
untestable. The testbed swaps them per keypress.

### Done — Phase 3: object tracker

- `accessibility/categories.lua` — flag-first classification into the six
  categories, plus the 6-template exclusion list and reuse of the game's `exit`
  tag.
- `accessibility/tracker.lua` — scan, cache, three-level navigation, announcements.

Verified in the starting house:
```
Interactables, 12 kinds
a bed, 1, nearest 1 tile south-west, 2 of 12
an open door 1 of 1, 2 tiles north-west
an open door, 2 tiles north-west, 2 steps by path      <- Home
```

**Caching:** signature is `level name : player x : player y : turn`. Position
matters because everything sorts by distance; turn matters because creatures move
and containers empty. Scans run on keypress only, never per frame.

**FoV respected** — creatures only when currently visible, tiles when visible or
remembered. The tracker never reveals what a sighted player couldn't see.

**`Home` reports path length, not just offset.** Straight-line distance is badly
misleading indoors: a chest 3 tiles north through a wall can be 20 steps away.
Uses the game's own `calculatePathFromPlayer`.

#### ⚠ Correction to D4 — `tryInteract` is NOT a clean "worth tracking" signal
`wall` carries a **flavour-only** `tryInteractFunction` ("You touch the wall.
It's cold and rough."), so flag-first classification promoted every wall segment
into Interactables — 13 in the starting house, burying the chest and cauldron.
**Fix: exclude structural tiles (`blocksMovement and blocksVision`) before the
`tryInteract` test.** Containers and furniture block movement but not sight, so
they survive. 13 kinds/32 instances → 12 kinds/19 instances.
`docs/05-decisions.md` D4 has been amended.

*(Not a bug: Items/Alchemy are empty in the tutorial house because
`chest_tutorial` literally prints "find nothing of use for now" — it has no loot
yet. Verify against a later area before trusting those two categories.)*

### Done — Phase 4 (most of it): menu descriptors

**16 states now have descriptors** — all registered in `describedStates` in
`init.lua`, all resolving in the startup self-test:

`main_menu` · `settings_menu` · `load_game` · `new_game` · `narration` ·
`ingame_dialogue` · `ingame_observe` · `actions_menu` · `worldmap` ·
`alchemy_table` · `journal` · `character` · `inventory` ·
`menu_text` / `ingame_message` / `menu_message` (shared `_textscreen.lua`)

Verified live: `Journal. Quests, A simple commission, 1 of 1` ·
`Character, 1 of 3` · `Actions. Interact, 1 of 1` ·
`Inventory. All items, empty, slot 1`. **21/21 testbed checks.**

#### Two driver improvements (both prompted by real failures)

- **`menus.getAnyUpvalue`** — finds a file-local across *all* of a state's
  functions plus one level into inner closures. Safe because a Lua file-local is
  a single shared upvalue, so any closure referencing it sees the same variable.
  Needed because `actions_menu` and `worldmap` build their lists in `open()` and
  never mention them in `keypressed`. The generic probe path now falls back to it.
- **`menus.hasUpvalue`** — checks a local *exists* rather than *holds a value*.
  `getUpvalue` can't tell "no such local" from "declared but nil", which made the
  self-test cry PROBE BROKEN on `worldmap` (whose `local options` is nil until
  the screen opens). Self-test now reports `ok (unset until opened)`.

#### Ownership rule: descriptors beat the popup patch
`patches/popups.lua` now **stands down for any state that has a descriptor.**
`inventoryCursorPopup.set` only fires when there *is* an item, so on an empty
slot or empty tab it said nothing — you couldn't tell whether the cursor moved.
The `inventory` descriptor reports item-or-empty plus slot and tab; letting both
speak would double up on every item.

### Known gaps (next up)
- **Phase 4 remainder**: `research`, `config_menu`, `keybindings_menu` have no
  descriptors. `craft` / `refine` / `trade` are covered only by the
  `inventoryCursorPopup` patch — so, like inventory was, they will be silent on
  empty slots and say nothing about the quantity **slider** in the confirmation
  window (`modules/slider.lua`, Left/Right adjusts, Shift for ×5).
- **Phase 5**: player status key (HP, statuses, current action), popup
  announcements (`popupStack.add`), damage alerts
  (`ingame.checkInterruptions`), mod settings.
- `ingame` itself has no descriptor.
- **Unverified against real content:** tracker categories Items / Alchemy /
  Exits / Creatures / Terrain (tutorial house has none), and inventory item
  names (inventory is empty in the prologue).
- `worldmap`, `journal`, `character`, `research`, `inventory`, `craft`, `refine`,
  `trade`, `actions_menu`, `alchemy_table`, `keybindings_menu`, `config_menu`,
  `menu_text`, `ingame_message`, `menu_message` still have no descriptors.

### Next — Phase 2: tile examination
Patch `views/levelCursorPopup.set` to narrate the game's existing examine mode,
augmented with distance/direction/passability. Then the surroundings explorer
(`Alt`+numpad to probe a direction without moving). Needs Q3 (verbosity) answered.

---

## Invariants for all mod code

1. Every wrapper calls through to the original and returns its results verbatim (`return realFn(...)` — preserve multiple returns).
2. Speech must never throw. Missing DLL ⇒ silent no-op, game still runs.
3. No behaviour change for sighted players.
4. Never reveal information outside `FoV` / `rememberedFoV` — the tracker must not become a cheat.
5. Grid scans (`xMax × yMax`) happen on keypress or turn change, never per frame.
6. Our keys fall through to the game handler unless we own them.

---

## Modified game files (keep this list exhaustive)

| File | Change | Why |
|---|---|---|
| `main.lua` | +1 line: `require("accessibility").initialize()` after `console.prepare()` | Sole mod entry point. Placed there because templates, keybindings, `states.setup()` and the console canvas are all ready by then, and it's before the first `states.switch()`. |

Verify this list is exhaustive at any time with:
```
git diff --stat vanilla-0.4.2 -- . ':!accessibility' ':!.gitignore'
```
Currently: `main.lua | 1 +`.

### How to run
```
cd "D:/code/Sulphur Memories Alchemist 0.4.2 sanctified"
"C:/Program Files/LOVE/lovec.exe" .
```
`lovec` (not `love`) gives the console, where `[speech]` lines mirror every
utterance — speech is verifiable from the log, not only by ear. `!` after
`[speech]` means the utterance interrupted.

---

## Scratchpad / notes

- `.luacheckrc` uses `std = "min+alch"` with an explicit globals list — add mod globals there to keep lint clean.
- Upstream quirk (not ours to fix): `modules/tile/preprocess.lua` flattens parents in a single `pairs()` pass, so a grandchild may inherit from a not-yet-flattened parent depending on hash order.
- Tolk DLL sources on this machine: `D:\code\cafe`, `D:\code\dragonrpg-mapmaker`, `D:\code\DragonWarriorAccess`.