Audio: host clock on physics steps, review fixes, world / NPC audio hook-in

Review fixes: - The audio host takes each physics step exactly once (one-step pulses latched, nothing while paused or silenced, at most 4 steps per frame), so pushes and other one-frame events are never dropped or doubled. e2e at 60 fps and above is byte-identical; 30 / 45 fps change only by the latch and the step pair. - Removed the emitter PROFILES fallback, the stray trace dump and the audition script; data-gated tests are #[ignore = "needs the private install data"] and fail loudly when run without it; per-post log lines behind SKATE_AUDIO_TRACE; checked bank parsing, finite output, camera-cut Doppler reset, fewer render allocations. - The 15 verdict-settled A/B switches are removed; their "on" behaviour is the only one (byte-identical). World / NPC audio, hookable by engine systems and mods: - Map-change fix: world and NPC hosts reset when the map's banks are unloaded, so surviving objects sound again. - Engine surface (skate-game::world_audio): TrafficAudio / PedAudio / NpcSkaterAudio components, horn / alarm / speech messages, read-back of who is audible; retail limits (4 cars within 40 m, 15 peds within 50 m with footsteps for the nearest 3, one NPC skater within 30 m), measured in the recompilation. - Mod surface (Lua API 2, `sdk.world_audio.*`); test mod `mods/world-audio-test/` (cars, peds, a ghost skater). - Per-skater audio state builder (local player output identical); remote players take the NPC skater slot; an opt-in non-retail "more audible" setting. - Docs: doc 15 "Hooking up world audio", doc 11 sections, PR plan.
2026-10-26 19:47:44 +00:00 · 2026-10-03 16:21:12 -06:00
1 parent df012ec5dd
commit a4ec831a02
@@ -362,9 +362,7 @@ this keeps retail's bed-to-player balance. Before this the beds were 11 dB too l

 ### Dev tools

- `py -3.13 tools/audio_audition.py` → `.local/audio-audition/index.html`: every cue's current samples
-  (parsed from `cues.rs`), each bank in full grouped into runs of similar clips, grains and beds; players at
-  25 % volume; links the installed WAVs, copies nothing.
+- (Removed 2026-10-03: `tools/audio_audition.py`, the cue audition page; see "PR #32 review fixes, step 1".)
 - `cargo run -p skate-data --release --example audio_surfaces -- <map.skate>` lists audio surface IDs per
  map; with `AT=x,y,z[,r]` it lists the collision surfaces (audio ID, physics type, slope) within r m
  (default 3) of a point, e.g. a HUD position the user reports.
@@ -453,7 +451,7 @@ Each step came from the user's report of an in-play session plus the `AUDIO_*` l
 ## Files

 - New: `tools/asset_pipeline/audio_formats.py`, `audio_export.py`, `test_audio_formats.py`,
-  `tools/audio_audition.py`, `crates/skate-game/src/game_audio/{mod,library,voices,ambience,cues,skate_events}.rs`,
+  `tools/audio_audition.py` (removed 2026-10-03), `crates/skate-game/src/game_audio/{mod,library,voices,ambience,cues,skate_events}.rs`,
  `crates/skate-data/examples/audio_surfaces.rs`.
 - 2026-10-02 world audio:
  - `game_audio/emitters.rs` replaces `water.rs`;
@@ -3438,6 +3436,9 @@ Research for the native audio runtime (specs in progress, 2026-10-02) also draws
  [BurnoutDecomp/b5-decomp](https://github.com/BurnoutDecomp/b5-decomp): RenderWare Audio plug-in
  identities, the SndPlayer1 and filter/resampler details, opcode names. b5-decomp has no licence, so it
  is reference only.
+- XNA Math / [DirectXMath](https://github.com/microsoft/DirectXMath) (Microsoft, MIT License): the
+  `XMVectorSin` / `XMVectorCos` polynomials the FrequencyShiftSsb oscillator uses (`skate-audio-fma`); the
+  coefficients are read from the shipped image.
 - [mitsevox/tw2004](https://github.com/mitsevox/tw2004) (CC0): Tiger Woods 07 RenderWare Audio function
  listings.
 No code from these is included.
@@ -4403,3 +4404,202 @@ change surface too (tag 3: surface 0 → 1; tag 5 was metal, now concrete).
 ### Riding body-collision posts settled (2026-10-03, recomp session `audiox_ride_20261003_141046`, 0 malformed)

 User ride (~4.5 min, 2 bails). Outside the bails (+3 s): body-poster posts from NPC skaters 163 (0.70/s, local72 0), from the local rider 49 (0.21/s), and the local ones are only three bursts, no steady riding posts: 45.9 s (4), 172.7 s (19; screenshot: landing a 7.5 ft stair gap) and 202.6 s (26; pushing into a parked car, vehicle material 36). So steady riding gives 0 local body posts in retail, as in ours; the earlier ~1.7/s was NPC skaters plus such events. Follow-up: check that ours posts body hits on a big-drop landing and on bumping a car (a state-log session with both).
+
+### PR #32 review fixes, step 1: the host clock is the physics steps; guards, tests, cleanups (2026-10-03, headless)
+
+**Problem (code review of PR #32).** `skate_events::observe` runs once per physics step (FixedUpdate, Virtual
+time, 16.6666 ms) and overwrote `Cues.riding` each step. `native::mixmap_frame` (Update) ran its own `Time<Real>`
+accumulator (`mix_clock += delta`, ticks = mix_clock / (1/60), cap 4) and ran inputs / process / update on whatever
+sample was in `Cues`. So the two grids drifted:
+- a frame with 2 physics steps and 1 mix tick lost the first step's one-step pulse (`+335`, the push plant's rise);
+- a frame with 1 mix tick and 0 steps re-processed a stale sample;
+- while the menu paused `Time<Virtual>`, `mixmap_frame` and the grain bed kept ticking on the stale sample;
+- below 60 fps the components saw every second step (still true, see below: that is retail's behaviour too).
+
+**Root cause.** Two clocks for one stream of samples. Retail's audio manager (`sub_82485190`) reads the record the
+game steps wrote; it never runs on a sample twice and never on a frame the game didn't step.
+
+**Change.**
+- `skate_events::Cues` counts the steps published since the host last took them (`Cues::publish` /
+  `take_steps`). While steps are pending, the one-step pulses of the new step are OR-ed with the pending one
+  (`latch_pulses`: `push_trigger` = `+335`). Every other field is a level (the latest step's value is what retail's
+  per-frame manager reads too), a latch (`+228`, `+468`, `+192` / `+692`), a 4-frame max (`+668`, `+496..`) or a
+  counter (`Riding::pushes`); the components find their own edges (landing, bail, grind) against the last sample
+  they processed, so those survive a 2-step frame without latching.
+- `native::HostClock::pass` (used by `mixmap_frame` and the e2e harness): takes the frame's steps. No step, or the
+  game silenced (menu, replay, the multiplayer menu that doesn't pause): no pass (no inputs, process, tick or
+  update; a silenced frame drops its steps and pulses, as the stream is paused). Otherwise ticks = steps, at most
+  `MAX_STEPS_PER_FRAME` = 4; the console cadence advances by those ticks.
+- **The cap (4 steps = 2 console evaluations).** Retail runs its audio manager once per rendered frame and never
+  catches up: a long frame is one call with a long dt. Our host counts steps to stay on the console's 30 Hz grid at any
+  frame rate, so it bounds the catch-up instead. 4 keeps frame-rate independence down to 15 fps (the old accumulator's
+  cap) while a hitch (Bevy runs up to 15 physics steps after a 250 ms frame) can't release a burst of evaluations,
+  Jitter steps or poster calls. The pulses of the dropped steps stay latched.
+- `Native::frame_ticks`: the grain bed (`grain_bed::update`) runs only on frames with a pass, dt = ticks × 1/60 (as
+  the e2e harness always did), not every rendered frame on `Time<Real>`. Pause no longer advances its envelopes.
+- Class_Seams' console cadence (`seam_frame`) still runs on every rendered frame (its own 30 Hz grid), now on game
+  time (`Time<Virtual>`, still while paused) and not while silenced. The rendered board's interpolation pair is the
+  last two physics steps (`Riding::wheels_before`, `PlayerAudio::step_wheels`), so a frame that takes two steps
+  interpolates between the right two.
+- Pause.in0 is no longer written as 32767: nothing ticks while silenced, so no evaluation would read it.
+- **Camera cuts.** `Presentation::cuts` counts the engine's snaps (teleport flag, camera discontinuity, cadence
+  change). On a change, or a map change (`CurrentMap::generation`), `mixmap_frame` resets the listener's last camera
+  (`PlayerAudio::reset_listener`: no Doppler velocity from the jump) and the seam pair (no interpolation from the old
+  place), and counts `Native::cuts`; the world / NPC hosts drop their camera velocity on a change of it.
+
+**Evidence / proof.**
+- e2e bench (`optimisation/tools/e2e_bench.sh`, 13 scenarios + 4 whole sessions; baseline built from `git archive
+  HEAD` in `.local/clk-base-src`): `row`, `fps60`, `fps144`, `fps300` byte-identical to the baseline (audio and voice
+  logs). `fps30` / `fps45` differ, as intended. Attribution: with the latch and the step pair both disabled in a
+  throw-away build, fps30 / fps45 are byte-identical to the baseline too, so these two are the only differences.
+  - The latch: in the 4 real sessions, 16 of 34 logged push edges fell on a row without a pass at 30 fps and were
+    dropped before; now each reaches one pass (also `ollies_log`'s one push).
+  - The step pair: at < 60 fps the old change-tracking pair spanned two steps.
+  - The e2e harness already modelled the ideal host at ≥ 60 fps (one pass per row); it now uses `HostClock` and
+    `Cues` itself, so the harness and the game share the scheduling code.
+- Unit test `native::tests::the_host_clock_takes_every_physics_step_once`: a Bevy app with `TimePlugin`, the physics
+  period and `TimeUpdateStrategy::ManualDuration` frame times (alternating 1 / 2 steps, frames without a step, 30 fps,
+  a 300 ms hitch = 15 steps, a paused second, a silenced stretch that still steps). Every step is taken by exactly one
+  frame; a pass sees a pulse iff one of its steps had one (the hitch frame merges three); ticks = steps capped at 4;
+  no pass while paused / silenced or without a step; console evaluations = ticks / 2.
+
+**Other review items in this step.**
+- Removed `trace-downtown-lag.json` (now ignored as `/trace-*.json`, `TRACE_PLAY.bat` writes it) and
+  `tools/audio_audition.py` (the audition pages are gone; the "Dev tools" and "Files" entries above are updated).
+- **No fallback, for real.** `emitters.rs` no longer plays the measured `PROFILES` table through Bevy voices when the
+  native runtime is absent: without it the emitters are silent (the runtime's start logs why). Removed with it: the
+  relay / loop patterns, the shuffle bag, `Library::sample_seconds` and the manifest's `seconds` field read.
+  `native::follow_volume`: Bevy voices always play at 1 / `RETAIL_SCALE` with the native fold and swapped ears (before,
+  an install without the runtime kept ×2 and Bevy's mirrored panning). Identical with the runtime.
+- Data-gated tests: 68 tests in `skate-audio` and `game_audio` (96 skip sites, incl. the world / NPC ones) are
+  `#[ignore = "needs the private install data"]`, and a missing piece now panics ("missing private data: …") instead
+  of passing silently. Run them with `-- --ignored` (or `--include-ignored`). On this machine every ignored test passes
+  (skate-audio 51, game_audio 21, older diagnostics included). The wall-clock assertion in the prefetch test is gone (timings are printed).
+- `AUDIO_NATIVE post` / `release` lines (one per component post, under the runtime lock) only with
+  `SKATE_AUDIO_TRACE=1`. No tool reads them.
+- Guards: `formats::abk` template end is a checked add; `Runtime::fill_stereo` turns non-finite samples into 0 for
+  the device (host-side safety, not a parity change: finite samples pass untouched; e2e reads the bus before it).
+- Allocations: the evaluator reuses unloaded bank slots (map changes no longer grow the list); `Evaluator::walk`
+  reuses its order snapshot; `redeliver` / `release` read their client lists in place; `destroy` reads the module's
+  object lists through the bank's `Arc` (it copied them). `grain_bed::update` no longer clones the seam wobbles and
+  builds a `PlayerTuning` every frame. `tests/render_alloc.rs` gains a hand-built AEMS bank (`eval::synthetic`, the
+  evaluator tests' builder made public and hidden from docs) with a playing program, a redeliver every console frame
+  and a release: 0 allocations in 480 warm blocks; the allocation counter is per thread now. All e2e outputs of
+  these changes are byte-identical (bench `clk_final` vs `clk_new`, every mode).
+- clippy `approx_constant` errors in the `dsp::pan` tests (`FRAC_1_SQRT_2`); 0 clippy errors in `skate-audio`.
+- `.gitignore`: `*.wav` is scoped to the folders retail audio is written to (`/assets/private`, `/data`,
+  `/maps/private`, `/.local`); no untracked WAV became visible. Mods and the SDK may ship WAVs.
+- Credit: the FSS `sin` / `cos` polynomials are XNA Math's `XMVectorSin` / `XMVectorCos` (successor DirectXMath,
+  Microsoft, MIT); see Credits.
+
+**Files.** `game_audio/{skate_events,native,grain_bed,player_audio,e2e,emitters,library,world_sources,npc_skaters}.rs`,
+`game_audio/native/prefetch.rs`, `presentation.rs` (the cut counter), `skate-audio/src/{eval/mod.rs,eval/synthetic.rs,
+eval/tests.rs,formats/abk.rs,runtime.rs,dsp/pan.rs}`, `skate-audio/tests/render_alloc.rs`, every data-gated test file,
+`skate-audio-fma/src/lib.rs`, `.gitignore`.
+
+**Open questions.**
+- Slow motion: the physics timer period changes (`physics/clock.rs`), so the host now ticks slower in real time with
+  it (it followed real time before). What retail's audio manager does in slow motion is not traced (`+220` time
+  scale is written 1.0).
+- The world / NPC hosts keep NodeIds across `unload_map_banks` (not fixed here; the world build-out step).
+- Needs the user's listening check at a low or uneven frame rate (pushes) and after a menu pause.
+
+### World audio hook-in, phases P0–P2: the map-change fix, the engine / mod surface, the dev test mod (2026-10-03, headless)
+
+Full write-up, field tables and examples: [doc 15](15-world-audio.md). Design: `.claude/notes/world-audio-hookin-spec.md`.
+
+- **P0, map change.** `Native::unload_map_banks` destroyed every instance of the 13 world banks, but `WorldHost`
+  kept the dead nodes and "already posted" objects, so an owner that survived the change was silent for the rest of
+  its life. Now `Native::map_epoch` is bumped by the unload. On a change, `WorldHost` / `NpcHost` release every node,
+  clear the pools / records, deactivate the 3DObjPos blocks, stop the ped Splice steps and the NPC bed, and drop
+  their objects; the owners still published are claimed and posted afresh. The evaluation count is a
+  `saturating_sub`. The world hosts' `dt` is now 1/30 per console evaluation and 1/60 per tick with
+  `SKATE_AEMS_MIX_CONSOLE=0` (it was `CONSOLE_DT` either way, double the real time in the non-console mode; that mode
+  went with the A/B switches, below, so it is 1/30 now).
+  Regression test `world_owners_post_again_after_a_map_change`, which fails without the reset.
+- **Retail limits from the gap runs** (spec §7.3): traffic list cut at 40 m horizontal (4 nearest); ped list cut at
+  50 m (15 nearest); ped footsteps for the 3 nearest (`S+68`); `S+148` / `S+156` = the distance / 20 m. Applied by
+  the hosts (radii) and the bridge (footsteps-on, the pair).
+- **P1, the engine surface** `crate::world_audio`: `TrafficAudio`, `PedAudio`, `NpcSkaterAudio`, `AudioVelocity`,
+  the read-back `WorldAudioInstance`, `LivingWorldAudio`, `WorldAudioStats`, the messages `PedSpeechEvent` /
+  `VehicleHorn` / `VehicleAlarm` (8 s). Bridge: `game_audio/world_bridge.rs`.
+  - The per-skater builder `skate_events::skater_audio_state` + `SkaterAudioMemory` is a pure move of `observe`'s
+    code. 1,500 production physics steps (`audio_state_capture`) gave a published state identical line for line
+    to the pre-refactor capture, and the builder equals `observe` on every step.
+  - `AudioState::rolling(&LiteSkater)` is the documented minimal fill for skaters not simulated with the player's
+    physics.
+- **User decisions included** (2026-10-03):
+  - mod cars may opt in to retail traffic engine sounds (`body=`);
+  - remote multiplayer players take the NPC skater instance, first in the list (non-retail; a lite state with the
+    ground's material);
+  - opt-in non-retail "more audible" (`settings/audio.json` `"more_audible_world": true`: 8 / 24 / 3 instances;
+    the MixMap is built with them; instance 0 unchanged);
+  - no traffic-light sounds.
+- **Mods:** `sdk.world_audio.spawn / update / event / remove / read / info` (capability `world_audio` = 1). 16
+  objects per mod, 64 in all; an object is parked after 0.5 s without updates; everything is cleaned up on disable,
+  reload or failure. The existing `sdk.audio.*` is unchanged.
+- **P2, the dev test publisher** `mods/world-audio-test/`:
+  - 16 cars (c00–c08 records, honks, skids, F8 alarm);
+  - 20 peds (walk / jog / run, warn / cheer / slam);
+  - a ghost NPC skater replaying a window of one of the user's state logs (`logs/<name>.tsv` or
+    `SKATE_AUDIO_STATE_LOGS`);
+  - debug boxes by audibility.
+
+  The e2e state-log replay moved unchanged from `e2e.rs` to `game_audio/state_replay.rs` for the ghost.
+- **Proof for the local player:** the e2e bench is byte-identical before / after (13 scenarios + 4 sessions, row and
+  fps300). Tests: the bridge unit tests, the ghost through the real host (claim, release at range, a map change), and
+  the skate-mods validation plus a 400-frame run of the test mod.
+- **Learned for the next pass (P3), from G3:** for an NPC, Wheels runs (layers 0 / 1) and Clothing runs; Tricks and
+  Treatment are local only; OffBoard creates packets but plays no steps. Instance 1 changes hands often (12× in
+  74 s).
+
+### A/B switches removed (2026-10-03, user decision; separate change after the world audio P0–P2)
+
+**Decision (the user):** remove the 15 verdict-settled A/B env switches. Their ON behaviour becomes the only
+behaviour.
+
+**Removed:**
+- `SKATE_AEMS_SEAM_PULSE`, `_MIX_CONSOLE`, `_PUSH_PLANT`, `_PLANT_LIFT`, `_BODY_IMPACTS`, `_GRIND_ONOFF`,
+  `_AIR_TIME_STATE`, `_SUBMIX`, `_FOOTSTEP_SUBMIX`, `_REVERB_INPUTS`, `_REVERB_ZONES`, `_BODY_CONSOLE`,
+  `_DECK_CONSOLE`, `_BODY_CURVE`, `_SLEW_CONSOLE`;
+- their e2e equivalents: `E2E_MIX_CONSOLE`, `_BODY_CONSOLE`, `_DECK_CONSOLE`, `_BODY_CURVE`, `_PLANT_LIFT`,
+  `_BODY_IMPACTS`, `_GRIND_ONOFF`, `_SLEW_CONSOLE`, `_FOOTSTEP_SUBMIX`, `_REVERB_INPUTS`, `_PUSH_PLANT`,
+  `_AIR_TIME_STATE`;
+- the dead comment-only names `SKATE_AEMS_PLAYER` / `SKATE_AEMS_FOOTSTEPS`.
+
+**Kept:** `SKATE_AEMS_WORLD`, `_WORLD_PREFETCH`, `_NPC_SKATERS`, `SKATE_AUDIO_FMA`, the tools (`SKATE_AUDIO_TRACE` /
+`TIMING` / `STATE_LOG` / `SET`, `SKATE_AEMS_BANKS`), and the e2e harness's part switches (`E2E_ROLLING`,
+`E2E_CONTACTS`, `E2E_BUSES`, …).
+
+**Dead code removed with them (game crate):**
+- the old 60 Hz MixMap evaluation (`HostClock::pass` has no `console` parameter; every pass is on the console
+  grid, the eEQChain clear included);
+- the per-tick Class_Seams process (`PlayerAudio::seam_console`; the seams run in `seam_frame`);
+- the fixed Reverb.in5 path;
+- the zone-less reverb;
+- the per-call body / deck posters (`PlayerAudio::body_console` / `deck_console`);
+- the dt-scaled bed slews (`Bed::slew_console`);
+- the air-flag `+236` count and the animation-contact push plant (`air_time_236` / `push_plant` lost their switch
+  parameters);
+- `set_review_ports` / `set_body_curve`;
+- the e2e harness's non-console row mode.
+
+**Kept on purpose:** the `skate-audio` components' own on / off fields (`Contacts::plant_lift_on` / `body_on` /
+`body_speed_on`, `Grind::onoff`, `CollisionManager::submix`, `Option` body / deck / slew calls). The game always
+turns them on. Their unit tests use "off" to isolate one part, so these are component configuration, not switches.
+Their docs no longer name env vars.
+
+**The e2e `row` mode is now the game's host at 60 fps.** Before, it ran the removed old 60 Hz evaluation and the
+per-tick seams.
+
+**Proof (byte-identical):**
+- e2e bench before (`wa_new` row / fps300, `sw_base` fps30 / fps60 / fps144) vs after (`sw_new`, all five
+  modes): fps30, fps60, fps144 and fps300 IDENTICAL for the 13 scenarios and the 4 whole sessions (26 + 8 outputs
+  each);
+- the new `row` renders are identical to the old `fps60` renders, which is what the game ran at 60 fps;
+- `audio_state_capture` (1,500 production physics steps through `observe`) is identical to its capture from before
+  both changes;
+- every unit and data test passes.
+- two data tests (`seams_play_their_retail_bank`, `seam_hits_shuffle_their_samples`) drove the removed per-tick seams.
+  They now call `seam_frame` before each pass, as `mixmap_frame` does, and pass with the same picture (pattern 11
+  at 10 / 20 / 30 km/h: 39 / 67 / 93 voices; brick: all 16 samples).
@@ -0,0 +1,271 @@
+# Hooking up world audio (traffic, pedestrians, NPC skaters)
+
+Part of PR #32 (branch `gameplay/audio`; doc [11](11-audio.md) has the ported audio itself).
+
+## Problem
+
+Skate 3's living world has its own sounds: traffic engines with Doppler, horns, skids and car
+alarms; pedestrians' footsteps and speech; and the board sounds of an AI skater near the camera. The
+native audio port (`crates/skate-audio`, `skate_audio::world`) already plays all of them the way
+retail does. But the engine has no traffic, pedestrian or AI-skater system yet. Nothing upstream or
+in any fork has one, and no issue asks for one (prior-work check, 2026-10-03). The sounds therefore
+had no way in.
+
+The goal (the user, 2026-10-03): "build out everything that can then easily be hooked to by the game
+engine once they are added", in this PR, and moddable like the rest of the engine.
+
+## The change in one paragraph
+
+A future engine system adds **components** to its own entities: `TrafficAudio`, `PedAudio`,
+`NpcSkaterAudio`, plus an optional `AudioVelocity`. It sends a few **messages**: `PedSpeechEvent`,
+`VehicleHorn` and `VehicleAlarm`. A **bridge** turns these into the audio hosts' inputs every frame.
+The hosts apply **retail's limits** to decide who is audible, and the bridge marks the audible
+entities with `WorldAudioInstance`. Lua mods reach the same path through `sdk.world_audio`. A dev
+mod, `mods/world-audio-test`, makes it all audible now.
+
+## Engine surface (`crates/skate-game/src/world_audio.rs`)
+
+Position and heading come from the entity's `GlobalTransform`: its +Z axis is the forward
+direction, the game's convention, and retail's vehicle `+112` is likewise the world matrix's forward
+row (recomp gap run G1). Velocity comes from the transform's change per frame unless the entity has
+an `AudioVelocity`. Give one to anything that teleports or is a kinematic proxy, because Doppler and
+the 3-D rates read it.
+
+**Lifetime = the entity.** To publish, insert the component. To release, despawn the entity or
+remove the component. Owner ids are `Entity::to_bits()`, so a reused index counts as a new owner.
+
+### `TrafficAudio` (retail `SFXObj_TrafficEngine` / `TrafficHorn` / `TrafficSkids`)
+
+| field | retail | meaning | who fills it | default |
+|---|---|---|---|---|
+| `engine` | vehicle record `+168` | the `aud_traffic_engine` record: sedans / hatchback `c01_family01`, sports / muscle `c03_sports01`, taxi / patrol `c04_taxi01`, SUV / pickup / minivan `c05_truck01` (entity → vehicle spec → record, G1). The engine's patch override picks c06 / c07 / c08 itself. | the vehicle system, from the model | — (an unknown name is logged once and stays silent) |
+| `speed` | `+148` | m/s | the vehicle system | the length of the velocity |
+| `load` | `+144` | the driver's signed acceleration in m/s². G1 measured −15.6 in a hard stop, up to +3 pulling away, and 0 while cruising. It goes into the engine and skid words × 3000. | the driving AI | derived from the speed change |
+| `horn` | `+156` | `None`, `Honk(1..=5)` or `Alarm` | the AI (honk decisions) | `None` |
+| `skidding` | `+160` | the tyres skid | the AI (hard braking or a swerve) | false |
+
+### `PedAudio` (retail `SFXObj_PedestrianSFX` / `PedestrianSpeech`, ped state S, G2)
+
+| field | retail | meaning | default |
+|---|---|---|---|
+| `voice` | `S+84` | the model = the speech voice id 41–96 | none (no speech) |
+| `shoe_class` | `S+132` | 1–5 per model (`aud_characteristics`). Most models are 2; no model uses 1, which is silent. | 2 |
+| `weight` | `[obj+28]+144` | 1–5. It changes per ped in retail (1 beyond about 12 m, 2–5 nearer) and its meaning is open. | 1 |
+| `close_range` | `S+96 == 64` | the security guards' close-range footstep levels | false |
+| `feet_down` | `S+74` / `S+73` | the walk animation's foot plants (A, B) | up |
+| `foot_materials` | `S+140` / `S+144` | the audio surface materials under the feet | 0 (retail's pavements read 0 in every line) |
+| `footsteps_on` | `S+68` | footsteps on | retail's rule: the 3 nearest peds in the list |
+| `speech_distance` | `S+148` / `S+156` | distance to the listener / the model's far threshold. Beyond the threshold the far `_f` lines are used. | the 3-D distance / 20 m (retail for regular peds) |
+| `speech_value` | `S+136` | the state graph's speech value | 0. Send `PedSpeechEvent` rather than writing it. |
+
+### `NpcSkaterAudio` (the MixMap Player slot's second instance)
+
+| field | meaning | default |
+|---|---|---|
+| `list_order` | the skater list position (retail walks its list in order) | spawn order |
+| `state` | this frame's `AudioState`. A skater simulated with the player's physics uses `game_audio::skate_events::skater_audio_state(physics, skater, &mut memory, dt)`, the same builder the local player uses (proved identical, below). Anything else uses `AudioState::rolling(&LiteSkater { .. })`: speed, wheels, materials, grind / air flags. With that fill, rolling, surfaces, seams, grinds and landings sound; tricks and foot / body foley stay silent. | none (not published) |
+| `remote` | a remote multiplayer player (see below) | false |
+
+### Messages, resources and the read-back
+
+- `PedSpeechEvent { ped, value: SpeechValue }`: sets the ped's speech value, so PedestrianSpeech sees
+  the change and requests a line. `SpeechValue::from_name` accepts the state-graph names (`DoWarning`,
+  `LongCheer`, …), the short names `warn` / `cheer` / `slam` / `flee` / `knockdown` / `nearby`, and
+  numbers.
+- `VehicleHorn { vehicle, kind, seconds }`: holds horn kind `kind` for the caller's time. The length
+  is the AI's choice, not retail data.
+- `VehicleAlarm { vehicle }`: retail's alarm, horn state 6 for **8 s**.
+- `LivingWorldAudio { expected }`: set it at map load when the system will publish. The 13 world
+  banks then decode on the prefetch worker instead of the game thread.
+- `WorldAudioInstance { slot, instance }` (read-back): the bridge inserts it on the entities that
+  hold a MixMap instance. Only those are audible. An engine system can use it to skip per-frame
+  audio work for the others, such as an NPC's `AudioState`.
+- `WorldAudioStats`: published and audible counts, the instance layout, and whether the
+  "more audible" setting is on.
+
+### Who is audible (retail's limits, applied by the hosts)
+
+| pool | instances | rule | source |
+|---|---|---|---|
+| Traffic | 4 | the 4 nearest within **40 m**, measured horizontally | G1: every holder was among the 4 nearest in 311 of 311 holder-seconds; the list is cut at 39.994 m |
+| Pedestrians | 15 | the 15 nearest within **50 m**; footsteps for the **3** nearest | G2: manager `sub_824F2890`; `+68 == (index < 3)` in 2,803 of 2,803 lines |
+| NPC skater | 1 | the first in list order within **30 m** of the camera, held until it is 30 m away or more | `sub_824F1FB0` / `sub_824F8EF8` |
+
+**Opt-in "more audible" setting (not retail; user decision 2026-10-03):** `settings/audio.json`
+`"more_audible_world": true`, or `SKATE_AUDIO_MORE_AUDIBLE=1` for one run. The MixMap is then built
+with 8 traffic, 24 pedestrian and 4 Player instances (3 NPC / remote skaters). The extra objects
+get retail's lookups, curves and posts; only their number is not retail. Instance 0 of every slot
+is unchanged. The setting is read at start. The runtime has one NPC grain bed, so only NPC instance
+1 has granular rolling. Default: off (retail).
+
+**Remote multiplayer players (not retail; user decision 2026-10-03):** another real player nearby
+takes the NPC skater instance, ahead of the NPCs in the list order. Retail's online behaviour is
+not traced. The remote state carries only the body and pose, so the bridge gives remote players a
+lite state from their root transform. The material under them comes from the same stock line query
+the wheel lines use.
+
+**No traffic-light or crossing sounds:** Skate 3 has none (the user, 2026-10-03).
+
+### Example: an engine system
+
+```rust
+use crate::world_audio::*;
+
+// A traffic system spawning a taxi (heading = the entity's +Z):
+commands.spawn((Transform::from_xyz(10.0, 0.0, 4.0), TrafficAudio::new("c04_taxi01")));
+
+// Its driving AI each frame:
+fn drive(mut cars: Query<(&mut Transform, &mut TrafficAudio, &Car)>) {
+    for (mut t, mut audio, car) in &mut cars {
+        t.translation = car.position;
+        audio.load = Some(car.acceleration); // m/s², or None to derive it
+        audio.skidding = car.acceleration < -8.0;
+    }
+}
+
+// The AI honks at a skater in the lane, a car is hit:
+honks.write(VehicleHorn { vehicle, kind: 2, seconds: 0.8 });
+alarms.write(VehicleAlarm { vehicle });
+
+// A ped system: walk animation foot plants, reactions from its state graph:
+commands.spawn((transform, PedAudio { voice: Some(59), shoe_class: 5, ..Default::default() }));
+speech.write(PedSpeechEvent { ped, value: SpeechValue::WARN });
+
+// An AI skater simulated with the player's physics, near the camera:
+npc.state = Some(skate_events::skater_audio_state(&physics, &skater, &mut memory, dt));
+```
+
+## Mod surface (`sdk.world_audio`, API 2, capability `world_audio` = 1)
+
+Mods publish through the same components: each key becomes an entity. The rules follow the existing
+mod rules. Keys belong to the calling mod. The limits are **16 objects per mod and 64 in all**.
+Options are validated before anything is allocated, and a field of another kind is an error. An
+object that is not updated for **0.5 s is parked** (speed 0, feet up, horn off). Disabling,
+reloading or a failing mod removes everything it published. The existing `sdk.audio.*` (the mod's
+own WAVs) is unchanged.
+
+```lua
+sdk.world_audio.spawn('car1', 'traffic', {engine='c04_taxi01', position=p, heading=h})
+sdk.world_audio.update('car1', {position=p, velocity=v, speed=12, load=-3, skidding=false})
+sdk.world_audio.event('car1', 'horn', {kind=3, seconds=1.2})   -- or 'alarm' (8 s)
+sdk.world_audio.spawn('taxi', 'traffic', {engine='c04_taxi01', body='chassis'})  -- a mod car opts in to a retail engine sound
+sdk.world_audio.spawn('ped1', 'ped', {voice=59, shoe_class=3, position=p})
+sdk.world_audio.update('ped1', {position=p, feet={true,false}})
+sdk.world_audio.event('ped1', 'speech', {value='warn'})        -- name or number
+sdk.world_audio.spawn('npc1', 'skater', {position=p, speed=6})  -- lite: rolling from these fields
+sdk.world_audio.spawn('ghost', 'skater', {source='state_log:state_20261003_143434', from=30, seconds=20, position=p})
+local r = sdk.world_audio.read('car1')   -- {kind='traffic', audible=true, instance=2, parked=false}
+local info = sdk.world_audio.info()      -- {more_audible=false, instances={traffic=4, peds=15, skaters=1}, ...}
+sdk.world_audio.remove('car1')
+```
+
+`body=` makes an object follow one of the mod's physics bodies (position, rotation, velocity). This
+is how a mod car opts in to the retail traffic engine sound; that choice is the user's
+(2026-10-03) and is not retail. A **ghost** replays one of the user's recorded audio state logs
+(`SKATE_AUDIO_STATE_LOG` writes them) at full fidelity. It reads `logs/<name>.tsv` in the mod
+folder, or `<name>.tsv` in the folder named by `SKATE_AUDIO_STATE_LOGS`. A log with any malformed
+line is refused.
+
+## The dev test publisher (`mods/world-audio-test/`)
+
+The mod is dev-only: it should not ship upstream unless the user asks. It drives everything around
+the spot where the mod starts:
+
+- **16 cars** on four rectangular lanes, cycling through the nine engine records `c00`–`c08`. Each
+  car accelerates at +2.5 m/s², cruises at 8–15 m/s for 3–8 s, then brakes: hard (−12 m/s², with
+  skids) or soft (−4). One car honks every 10–15 s with kind 1–5. One taxi is parked, and **F8**
+  fires its alarm.
+- **20 peds** on back-and-forth lines: 16 walk (1.3 m/s), 3 jog (4) and 1 runs (8), so the three
+  `sk8_foley` step ids play. A gait clock alternates the feet; it is dev-only, not retail. Shoe
+  classes are 2–5, and the voices cycle through 41–96. Reactions:
+  - warn (11) when you pass within 1.5 m at more than 3 m/s;
+  - cheer (23) on a landed trick within 10 m;
+  - slam (25) on a bail within 10 m.
+
+  Speech requests are logged (`AUDIO_WORLD speech`). Playback comes in the next pass.
+- **A ghost NPC skater** replays the 20 s window of a recorded state log, 5 m beside the start.
+  Settings: `ghost_log`, `ghost_from`.
+- Debug **boxes**: green = holds an instance (audible), grey = not. An on-screen line shows the
+  audible counts, the limits and the ghost's status. **F9** re-centres everything on the skater.
+
+## Proofs and tests
+
+- **The map-change bug (P0).** `unload_map_banks` (a map change) destroyed every instance of the 13
+  world banks. `WorldHost` kept its dead nodes and "already posted" objects, so an owner that
+  survived the change redelivered to a dead node and stayed silent. The fix:
+  - `Native::map_epoch` is bumped by `unload_map_banks`;
+  - on a change both hosts release every node, clear their pools, deactivate the 3DObjPos blocks,
+    stop the ped Splice steps and the NPC bed, and drop their objects;
+  - the evaluation count is `saturating_sub`;
+  - the evaluation `dt` follows the MixMap cadence: 1/30 per console evaluation. The old 60 Hz
+    mode, where `CONSOLE_DT` doubled it, was removed with the A/B switches the same day (doc 11).
+
+  Test `world_owners_post_again_after_a_map_change` (data-gated): a taxi and a ped publish, the C04
+  engine sounds, the map changes, and the same ids post afresh and sound again. Without the epoch
+  reset the test fails ("the same owner sounds again after the map change"). The ghost test below
+  checks the NPC host's reset the same way.
+- **The per-skater builder** (`skater_audio_state`, `SkaterAudioMemory`) is a pure move of
+  `observe`'s code. Test `audio_state_capture` (data-gated) drives 1,500 production physics steps
+  headless (push, carves, two ollies, a powerslide, a roll-out) and runs `observe` after each. Its
+  published samples are **identical line for line** to a capture taken before the refactor. On
+  every step the builder, run on its own memory, also gives the same `AudioState` as `observe`.
+- **The local player's output:** the e2e bench (13 scenarios and 4 whole sessions, row and fps300)
+  is **byte-identical** before and after this work (`wa_base` vs `wa_new`). The state-log replay
+  that e2e uses was moved, unchanged, into `game_audio/state_replay.rs` so the ghost can use it.
+- Bridge unit tests (a minimal Bevy app):
+  - components become owners;
+  - the ped footsteps-on and far-threshold rules hold, and remote players come first;
+  - the alarm, horn and speech messages work;
+  - the read-back inserts and removes `WorldAudioInstance`;
+  - despawning releases, and nothing happens without components.
+- `a_ghost_claims_instance_1_releases_and_survives_a_map_change` (data-gated): a real user log as a
+  ghost holds Player instance 1 within 30 m and sounds. It is released at 400 m, and after a map
+  change it claims instance 1 and sounds again.
+- skate-mods:
+  - command validation per kind, and the event options;
+  - the Lua wrappers cross the serde boundary;
+  - the bundled test mod runs 400 frames with no Lua error or invalid command, stays within 128
+    commands per callback, and its F9 re-centre spawns everything again.
+
+  `check_mod` validates the mod.
+
+## Files
+
+- New: `crates/skate-game/src/world_audio.rs`, `game_audio/world_bridge.rs`,
+  `game_audio/state_replay.rs`, `crates/skate-game/src/modding/world_audio.rs`,
+  `crates/skate-mods/src/world_audio.rs`, `crates/skate-game/src/tests/audio_state_capture.rs`,
+  `mods/world-audio-test/{mod.json,main.lua}`.
+- Changed:
+  - `game_audio/{world_sources,npc_skaters,native,skate_events,e2e,mod}.rs`;
+  - `skate_audio::{player::state (LiteSkater, AudioState::rolling), world::skaters (Slots::with_records)}`;
+  - `skate-mods` `vm.rs`, `api.lua`, `lib.rs`;
+  - `modding/mod.rs`;
+  - `sdk/skate.lua`;
+  - `.gitignore` (`mods/world-audio-test/logs/`).
+
+## Open questions and the next pass (P3–P5)
+
+- **Speech playback** in the host: the manager, the library and the streams. The level and pan
+  mapping is still open.
+- **The NPC instance's components.** G3 (2026-10-03) settled which ones run for an NPC:
+  - Wheels runs (spin streams on layers 0 / 1; layer 2 and the post-start parameter writes are
+    local only);
+  - Clothing runs (`sk8_foley` 73 / 74; body slide / cloth falls on bails; the start-block float is
+    0 for non-local; the eq-chain pick takes `local72`);
+  - Tricks and Treatment are local only;
+  - OffBoard creates its footstep packets but plays no steps for NPCs;
+  - the bail grunt `sub_824BF5F8` posts speech.
+
+  Instance 1 changes hands often (12 times in 74 s), so claims and releases must stay cheap.
+- **PedBodyFall** (decode the object), **Tazer** (needs AEMS op 38).
+- **Traffic bindings:**
+  - the `TrafficCarPhysics.in0` writer (|v_car − v_listener| clamped to 35, slewed by 100 /s,
+    × 32767 / 35; opens the A11 near boost);
+  - the 3DObjPos 4.1 / 4.2 points (front and rear ±1 m along the heading, R+80 / R+96, likely);
+  - retail's frozen record when a held car leaves the 40 m list (recorded behaviour, not ported).
+- The per-model tables from `aud_characteristics` (shoe class, kind, far threshold 20 / 30 m) as a
+  setup export, so the engine names a model and gets the rest. The horn kinds per model are open.
+- Moving process / update into `mixmap_frame` (retail's split; the inputs are one console frame
+  old today).
+- The wider mod surface (P4): content overrides (banks, samples, programs, speech lines), mod
+  emitters on the native voice graph, tuning read / write, `sdk.audio.post`, and audio event hooks.
@@ -16,9 +16,9 @@ verification, open questions).
 | F | Tooling | Streaming map validator (`--validate-maps`) wired into setup | [06](06-map-validator.md) | `18f41a3` | E (speed only) | Draft upstream [#28](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/28) |
 | H | Gameplay + rendering | Water: detection and bail, shallow water solid / deep water floats, board floats, water camera + vignette, entry splash, water look (family 33 cube blur + tint, anti-tiling, body-size calming, slower waves), animated water (frame-state buffer fix, ocean PCA from default.xex, XEX2 unpacker), ripple scale | [09](09-water.md) | `gameplay/water` (`af1b350`…`55776dc`), upstream branch `water/retail-water` (`8a6e328`, one commit on `60efdef`) | — (carries #28's `install.py` `spawn()` split verbatim) | Draft upstream [#30](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/30) |
 | I | Test fix | `sky_shader_validates`: add the `VERTEX_POSITIONS` shader def Bevy sets for real meshes | [10](10-sky-shader-test.md) | `b7865d0` | — | Not opened; tiny, can go upstream alone |
-| J | Feature (audio) | Game audio: setup group `audio` extracts the disc's audio (vgmstream r2117) and the native data (AEMS banks, MixMap, grains, Splice banks, vault tuning); a native port of the retail audio runtime (`crates/skate-audio`, `crates/skate-audio-fma`) plays all world and player audio; volume settings and `--mute` | [11](11-audio.md) | `gameplay/audio` (latest pushed `870940e`); upstream PR branch `audio/retail-audio` mirrors it exactly (`843cca9`). History: batch commit `f438017` "New audio who dis" (merged as `ca7b4f3`, 2026-10-02), then follow-up commits | H (branched from `gameplay/water`; uses its water detection — `collision_feedback.flags.material_12`, `is_water_tag`). Overlaps upstream #4/#1 (retail-exact player audio via transcribed game code + memory image; not adopted — see doc 11): both edit `tools/asset_pipeline/asset_exports.py` and `install.py`, different asset layouts | Draft upstream [#32](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/32), **Work in Progress**. State 2026-10-03: native runtime is the only path (interim cue tables removed); every retail player component ported; world emitters, location sets and zone ambience from retail data; water splash from retail's mechanism; console-cadence posters, |Δv·n|, the NPC grain bed and the bail speed curve (bridge graph on the body impacts; uncommitted). User: "basically retail, if there is a difference i couldn't tell". Open: the turn / brake slews' console cadence (user's call; the recomp's fast release is its frame rate), world / NPC stages waiting on ped / vehicle systems. Doc 11 has the full history; #32's description is kept current (standing rule). Commits only when the user asks. |
+| J | Feature (audio) | Game audio: setup group `audio` extracts the disc's audio (vgmstream r2117) and the native data (AEMS banks, MixMap, grains, Splice banks, vault tuning); a native port of the retail audio runtime (`crates/skate-audio`, `crates/skate-audio-fma`) plays all world and player audio; volume settings and `--mute` | [11](11-audio.md), [15](15-world-audio.md) | `gameplay/audio` (latest pushed `870940e`); upstream PR branch `audio/retail-audio` mirrors it exactly (`843cca9`). History: batch commit `f438017` "New audio who dis" (merged as `ca7b4f3`, 2026-10-02), then follow-up commits | H (branched from `gameplay/water`, #30, uses its water detection) and **#35** (50 solver iterations; needed for retail-like pops — #32 carries the same commit until #35 merges). Overlaps upstream #4/#1 (not adopted; doc 11) | Draft upstream [#32](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/32), **Work in Progress**. State 2026-10-03: native runtime is the only path; every retail player component ported; user: "basically retail". Road to ready (`.claude/todo/pr32-ready.md`): code-review fixes (host clock = physics steps, stray files, data tests, fallback) → world / NPC audio built out in #32 with an engine- and mod-facing hook-in surface (P0–P2 done 2026-10-03, uncommitted: map-change fix, `world_audio` components + `sdk.world_audio`, dev mod `mods/world-audio-test`; doc 15; P3–P5 next) → publish the specs, repoint `.claude/` references → moddability pass → second optimisation pass → move the non-audio tools to L → full regression check. #32's description is kept current (standing rule). Commits only when the user asks. |
 | G | Performance (setup) | Skip decoding duplicate stream copies; run the customiser beside the maps; parallel clothing library and pro roster, plus a native RefPack DLL in `Build.ps1` | [08](08-setup-performance.md) | `8327e94` (+ overlap hunks of `install.py` in `18f41a3`) | — | Draft upstream [#29](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/29) |
-| K | Gameplay fix (physics) | Board/skater solver: 50 constraint iterations (retail live value) instead of `RWMaxIterations` = 25 | [12](12-solver-iterations.md) | uncommitted (on `gameplay/audio` working tree) | — (one-file change in `physics/settings.rs`) | Ready apart from the asset-gated `customiser_equipment_reaches_ground_force_and_torque` question (doc 12) |
+| K | Gameplay fix (physics) | Board/skater solver: 50 constraint iterations (retail live value) instead of `RWMaxIterations` = 25 | [12](12-solver-iterations.md) | `physics/solver-iterations` (`3f6e989`, cherry-picked from `gameplay/audio`'s `7a9e6dc` onto `main` `7ae67f2`) | — (one-file change in `physics/settings.rs`) | Open upstream: [#35](https://github.com/SK8-ENGINE/skate-3-rust-engine/pull/35) (2026-10-03). #32 depends on it for retail-like pops and carries the same commit until #35 merges. Open question in the PR: the asset-gated `customiser_equipment_reaches_ground_force_and_torque` test (doc 12). Tests: 309 pass; the 2 failures (`pipelines_accept_…`, `sky_shader_validates`) also fail on `main`. |
 | L | Tooling | Development and research tools in `tools/<name>/` (regression checks, setup equivalence, collision / world-stream / audio-file / vault readers, audio e2e and bench; recomp trace and code search, reference only) | [14](14-published-tools.md) | uncommitted (on `gameplay/audio` working tree) | `audio-e2e` / `audio-bench` / `audio-file-inspect` need J's native audio and setup modules; `setup-equivalence/compare_refpack.py` needs G's RefPack DLL; `regression-checks/check_maps.py` uses F's `--validate-maps` (falls back to `--check-assets` without it) | Not opened; the generic folders could go first, the audio ones with or after J |

 Opened 2026-10-01 as drafts from per-PR branches cut from upstream `60efdef`
@@ -20,6 +20,7 @@ Each document stands alone. File paths are relative to the repository root.
 | 12 | [Board solver: 50 constraint iterations](12-solver-iterations.md) | Game physics (Rust) | Done; maps identical, water traces as expected; one asset-gated customiser test fails (open question) |
 | 13 | [Research hooks for the recompilation](13-recomp-research-hooks.md) | Research tooling (external: skate3recomp `research-hooks` branch) | Published; reference and information gathering only (the recompilation is not a perfect copy of the console game) |
 | 14 | [Published development and research tools](14-published-tools.md) | Tooling (`tools/<name>/`, Python and shell) | Done (uncommitted); scripts compile and print their usage; the two `recomp-*` folders are for the recomp's research hooks, reference only |
+| 15 | [Hooking up world audio](15-world-audio.md) | Game audio (Rust), mod API (Lua) | Part of #32 (uncommitted). Engine-facing components (`TrafficAudio`, `PedAudio`, `NpcSkaterAudio`) and `sdk.world_audio` for mods feed the ported retail traffic / ped / NPC-skater audio with retail's limits; the map-change bug fixed; a dev test mod (cars, peds, a ghost skater) makes it audible now |

 ## Environment used for verification