diff --git a/docs/hails-additions/11-audio.md b/docs/hails-additions/11-audio.md index 8a8671b8..4b63f2ce 100644 --- a/docs/hails-additions/11-audio.md +++ b/docs/hails-additions/11-audio.md @@ -4397,3 +4397,9 @@ Our replay of the session's tag-66 grinds (e2e, `E2E_FPS=60`): **For the user in game.** Concrete ledges and curbs (e.g. that University spot) grind with the concrete GRINDS layer and the Skate_Collisions on / off sounds, no metal ring. Metal rails still ring. Some other concrete / stone grinds change surface too (tag 3: surface 0 → 1; tag 5 was metal, now concrete). + +**Listening (user, 2026-10-03, couch launcher, bin\ with the turn / brake console cadence and the grind tag - 1 fix; session `state_20261003_135706`, 0 malformed; asked about the University concrete ledge, a metal rail, carving, braking, enclosed spots (reverb) and fast riding above 46 km/h):** "all of it sounded really good" + +### Riding body-collision posts settled (2026-10-03, recomp session `audiox_ride_20261003_141046`, 0 malformed) + +User ride (~4.5 min, 2 bails). Outside the bails (+3 s): body-poster posts from NPC skaters 163 (0.70/s, local72 0), from the local rider 49 (0.21/s), and the local ones are only three bursts, no steady riding posts: 45.9 s (4), 172.7 s (19; screenshot: landing a 7.5 ft stair gap) and 202.6 s (26; pushing into a parked car, vehicle material 36). So steady riding gives 0 local body posts in retail, as in ours; the earlier ~1.7/s was NPC skaters plus such events. Follow-up: check that ours posts body hits on a big-drop landing and on bumping a car (a state-log session with both). diff --git a/docs/hails-additions/13-recomp-research-hooks.md b/docs/hails-additions/13-recomp-research-hooks.md index 108b6402..70eb0e6f 100644 --- a/docs/hails-additions/13-recomp-research-hooks.md +++ b/docs/hails-additions/13-recomp-research-hooks.md @@ -61,3 +61,15 @@ build the recomp and set up the paths themselves. [skate3recomp](https://github.com/mchughalex/skate3recomp) by @mchughalex, the [rexglue SDK](https://github.com/rexglue/rexglue-sdk), and [Xenia](https://github.com/xenia-project/xenia)'s Xbox 360 research. + +## Playing it from the couch, and closing it + +- Our couch launcher (local, `.local\steam\`) starts the traced recomp from Steam / Steam Link with one shortcut per + trace mode, checks each trace for malformed lines afterwards, and keeps one game running at a time. +- The recomp's settings overlay opens with **Escape** or **F1** and has **"Quit to the desktop"**; it is + controller-navigable once open. On the `research-hooks` branch, holding **LB + RB + Back** for ~1 s opens it from the + controller (cvars `skate3_menu_pad_chord` / `skate3_menu_pad_hold_ms`); a quick tap doesn't. Back alone opens Skate 3's + Instant Replay, which shows underneath while holding. Start isn't used because it opens Skate 3's own pause menu. +- Closing with Steam's "Exit game" is safe for the data (the trace is written continuously). +- A heavy trace (`audio,audiox,dsp` plus the audio capture) can coincide with stalls; the trace writer stops before a + stall, so a stall can be missing from the trace. Drop `dsp` if stalls recur. diff --git a/docs/hails-additions/14-published-tools.md b/docs/hails-additions/14-published-tools.md index 9c5158f9..8ad2d517 100644 --- a/docs/hails-additions/14-published-tools.md +++ b/docs/hails-additions/14-published-tools.md @@ -80,3 +80,9 @@ timing and threading), so its traces are evidence for values and logic, not for [rexglue SDK](https://github.com/rexglue/rexglue-sdk), [Xenia](https://github.com/xenia-project/xenia)'s Xbox 360 research, [vgmstream](https://github.com/vgmstream/vgmstream) for audio decoding, and the vendored map extraction tools in `tools/vendor/`. + +## Added 2026-10-03 + +- `tools/recomp-trace/bail_impacts.py`: per-bail ragdoll impacts and sound posts (BAILSTEP / BAILREG). +- `tools/recomp-trace/collision_posts.py`: collision-sound posts by poster and owner, with the local-rider flag. +- `tools/recomp-trace/trace.py`: knows the newer line kinds (BAIL*, COLLPOST, BANDQ, LOCALTEST). diff --git a/tools/recomp-trace/README.md b/tools/recomp-trace/README.md index fe8af3f8..2163fd7c 100644 --- a/tools/recomp-trace/README.md +++ b/tools/recomp-trace/README.md @@ -28,6 +28,8 @@ optionally a float32 stereo capture of the mixed output, and screenshots. A sess | `grain_trace_stats.py` | Grain voice starts (PLAY lines matching a `.grain` member of your disc): starts per member, per second and inter-start intervals. | | `retail_windows.py` | Straight four-wheel rolling windows of the session's capture per 5 km/h band (level, octave bands), next to the engine's e2e renders folded the same way (`tools/audio-e2e`). | | `veh_trace.py` | Traffic summary from the traffic hook lines: per vehicle speed, target speed, manoeuvre and lane changes, stops, plus the horn / skid / engine lines. | +| `bail_impacts.py` | Per bail (BAILSTEP / BAILREG lines): the ragdoll's update interval, then per body region the impacts the game wrote, the per-update velocity change along the contact normal (percentiles), whether the largest ones are contact stops or drives, and the body / cloth / concrete sound posts in the bail. Flags hitches over 40 ms. | +| `collision_posts.py` | Collision-sound posts (COLLPOST) split into bail windows and the rest, per posting function and owner: rates, material pairs, tiers and the local-rider flag (`local72`), so the player's own sounds can be told apart from NPC skaters' and props'. | ## Inputs @@ -47,8 +49,14 @@ py -3.13 tools/recomp-trace/retail_relay.py sessions/my_session SomeBank --from py -3.13 tools/recomp-trace/grec_level.py sessions/my_session/trace.tsv py -3.13 tools/recomp-trace/retail_windows.py sessions/my_session .local/audio-e2e --contact-callers 82AAAA,82BBBB py -3.13 tools/recomp-trace/first_pass.py sessions/my_session/trace.tsv SOMEKIND +py -3.13 tools/recomp-trace/bail_impacts.py sessions/my_bail_session/trace.tsv +py -3.13 tools/recomp-trace/collision_posts.py sessions/my_ride_session/trace.tsv --top 6 ``` +Both refuse a trace with malformed lines of their kinds (record again rather than parse around bad data). Before +comparing collision-sound rates with another engine, filter to the local rider (`local72` = 1): NPC skaters post +into the same manager. + ## Example output `trace.py` (made-up counts): diff --git a/tools/recomp-trace/bail_impacts.py b/tools/recomp-trace/bail_impacts.py new file mode 100644 index 00000000..4a727bad --- /dev/null +++ b/tools/recomp-trace/bail_impacts.py @@ -0,0 +1,133 @@ +"""Per-bail summary of the recomp's ragdoll body impacts (BAILSTEP / BAILREG, category audiox) and the body posts. + +For each bail (BAILSTEP lines less than 1 s apart): the update interval, then per body region (0..7): contact +steps, the parts seen, the impact the game wrote (max, p50, p90, share over the tier-1 / tier-2 band floors), +|dv.n| percentiles (the part's velocity change along the contact normal per update), and for the largest 10 % +of |dv.n| the median signed normal velocity before / after the update plus the share that stop or bounce +(sign change, or |v_new.n| < 0.25 |v_old.n|): a contact stop shows into-surface v_old.n going to ~0 or reversing; +a drive changes v.n without that pattern. Then the body / cloth / concrete Splice posts (SPLC ids, category +audio) in the bail window -0.5 s .. +0.5 s, by region and tier, and an all-bails |dv.n| distribution. + +Velocities in the skeleton state are finite differences x 60, so dv is per update; compare against +our per-step dv with the update interval in mind (the `step ms` line). + +usage: py -3.13 bail_impacts.py [--gap MS] +Refuses a trace with malformed BAIL* lines (bad data is unusable; record again). +""" +import argparse +import sys +from collections import Counter, defaultdict +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from trace import Trace # noqa: E402 + +# Body poster ids by region and tier (doc 11 "Bail density: ours rendered, the impact scale checked"). +POST_IDS = { + 952: ('head', 0), 953: ('head', 1), 1032: ('head', 2), + 949: ('torso', 0), 950: ('torso', 1), 951: ('torso', 2), + 947: ('legs', 0), 948: ('legs', 1), 1031: ('legs', 2), + 956: ('arms', 0), 957: ('arms', 1), 1030: ('arms', 2), + 954: ('cloth skin', 0), 955: ('cloth denim', 0), + 958: ('concrete', 0), 1047: ('concrete', 1), 991: ('concrete', 2), +} +BANDS = (0.1, 0.25) # the body records' lowest tier-1 / tier-2 floors (torso, head) for a rough share + + +def q(values, p): + if not values: + return float('nan') + s = sorted(values) + return s[min(len(s) - 1, int(p * (len(s) - 1) + 0.5))] + + +def fnum(text): + try: + return float(text) + except ValueError: + return float('nan') + + +def group_bails(steps, gap): + bails = [] + for ms, f in steps: + if not bails or ms - bails[-1][-1][0] > gap: + bails.append([]) + bails[-1].append((ms, f)) + return bails + + +def summarise(path, gap, out=print): + t = Trace(path) + bad = {k: n for k, n in t.malformed().items() if k.startswith('BAIL') and n} + if bad: + out(f'REFUSED: malformed BAIL* lines {bad}; the session is unusable, record it again') + return 2 + steps = t.lines.get('BAILSTEP', []) + regs = t.lines.get('BAILREG', []) + out(f'{path}: BAILLOCAL {len(t.lines.get("BAILLOCAL", []))}, BAILSTEP {len(steps)}, BAILREG {len(regs)}') + for ms, f in t.lines.get('BAILLOCAL', []): + out(f' local {ms:.0f} ms: character {f[0]} object {f[1]} X {f[2]} how {f[3]} entry {f[4]}') + if not steps: + out('no BAILSTEP lines (no bail of the local player, or the local block was not found)') + return 1 + splc = [(ms, int(f[1])) for ms, f in t.lines.get('SPLC', []) if len(f) >= 2 and f[1].lstrip('-').isdigit()] + all_dvn = defaultdict(list) + for n, bail in enumerate(group_bails(steps, gap), 1): + a, b = bail[0][0], bail[-1][0] + step_ms = [fnum(f[1]) for _, f in bail if fnum(f[1]) > 0] + dts = Counter(f[2] for _, f in bail) + hows = Counter(f[9] for _, f in bail) + out(f'\nbail {n}: {a / 1000:.2f}..{b / 1000:.2f} s, {len(bail)} updates; step ms p10/p50/p90 ' + f'{q(step_ms, .1):.2f}/{q(step_ms, .5):.2f}/{q(step_ms, .9):.2f}; dt field {dict(dts)}; how {dict(hows)}; ' + f'bail/end bits {dict(Counter((f[3], f[4]) for _, f in bail))}') + hitches = [s for s in step_ms if s > 40] + if hitches: + out(f' HITCH: {len(hitches)} updates > 40 ms (max {max(hitches):.0f} ms) inside this bail; mark or exclude ' + f'its dv data') + if all((f[3], f[4]) == ('0', '0') for _, f in bail): + out(' NOTE: bail/end bits 0 throughout: opened by another GREC owner\'s bail (an NPC skater), not the local ' + 'rider; exclude') + per = defaultdict(list) + for ms, f in regs: + if a - 1 <= ms <= b + 1: # BAILREG lines precede their BAILSTEP in the same ms + per[int(f[1])].append(f) + out(' reg steps parts imp max p50 p90 >.10 >.25 | |dv.n| p50 p90 max | top10% v_old.n v_new.n stop') + for i in sorted(per): + rows = per[i] + imp = [fnum(f[3]) for f in rows] + dvn = [fnum(f[4]) for f in rows] + all_dvn[i] += dvn + parts = ','.join(p for p, _ in Counter(f[2] for f in rows).most_common(3)) + cut = q(dvn, .9) + top = [f for f in rows if fnum(f[4]) >= cut] + vo = [fnum(f[5]) for f in top] + vn = [fnum(f[6]) for f in top] + stop = sum(1 for o, w in zip(vo, vn) if (o * w < 0) or abs(w) < 0.25 * abs(o)) / max(len(top), 1) + share = [sum(1 for x in imp if x > band) / len(imp) for band in BANDS] + out(f' {i:>3} {len(rows):>5} {parts:<16} {max(imp):7.3f} {q(imp, .5):5.3f} {q(imp, .9):5.3f} ' + f'{share[0]:4.0%} {share[1]:4.0%} | {q(dvn, .5):10.3f} {q(dvn, .9):5.3f} {max(dvn):5.3f} | ' + f'{q(vo, .5):14.3f} {q(vn, .5):7.3f} {stop:4.0%}') + posts = Counter(POST_IDS[i] for ms, i in splc if a - 500 <= ms <= b + 500 and i in POST_IDS) + by_region = defaultdict(lambda: [0, 0, 0]) + for (region, tier), c in posts.items(): + by_region[region][tier] += c + out(' posts (tier 0/1/2): ' + ('; '.join(f'{r} {v[0]}/{v[1]}/{v[2]}' for r, v in sorted(by_region.items())) + or 'none (category audio off, or no body post)')) + out('\nall bails, |dv.n| per region: p50 / p90 / p99 / max (n)') + for i in sorted(all_dvn): + v = all_dvn[i] + out(f' {i}: {q(v, .5):.3f} / {q(v, .9):.3f} / {q(v, .99):.3f} / {max(v):.3f} ({len(v)})') + return 0 + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split('\n')[0]) + ap.add_argument('trace') + ap.add_argument('--gap', type=float, default=1000.0, help='ms between BAILSTEP lines that starts a new bail') + args = ap.parse_args() + return summarise(args.trace, args.gap) + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/recomp-trace/collision_posts.py b/tools/recomp-trace/collision_posts.py new file mode 100644 index 00000000..ff7e1916 --- /dev/null +++ b/tools/recomp-trace/collision_posts.py @@ -0,0 +1,120 @@ +"""Collision sound manager posts (COLLPOST, category audiox of the research hooks) by poster and owner. + +The poster names below map the hook's `caller` field as documented in the hooks' COLLPOST line description. + +Splits the posts into bail windows (a GREC-local state's +676 / +677 set, from FIRSTHIT paired with the GRECX line of +the same call, plus 0.5 s) and the rest ("riding / idle"), and per window kind prints, per posting function and GREC +owner (the board object whose audio state the poster's object uses; 0 = not a GREC-local player, e.g. an NPC skater or a +prop), the post count and rate, the material pairs and sounds, local72 values and the top caller chains. Also counts +the body-material sounds 955 / 954 / 956 / 947 (denim / skin / arms / legs, tier 0) in the SPLC lines of the same +windows, so the poster's sound field can be checked against the ids actually resolved. + +usage: py -3.13 collision_posts.py [--top N] +Refuses a trace with malformed COLLPOST lines (bad data is unusable; record again). +""" +import argparse +import sys +from collections import Counter, defaultdict +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from trace import Trace # noqa: E402 + +CALLERS = { + '82496258': 'poster 82496258', '824BA630': 'poster 824BA630', '824BB0E0': 'grind start', + '824BC188': 'body poster', '824BCEB0': 'first-hit torso', '824BD000': 'deck impact', + '824E3BE0': 'physics prop', '00000000': 'other', +} +BODY_IDS = {955: 'denim', 954: 'skin', 956: 'arms', 947: 'legs'} + + +def bail_windows(t): + """[(start, end)] ms where any GREC-local state had +676 or +677 set, padded by 500 ms.""" + owner = None + events = [] + rows = [] + for kind in ('GRECX', 'FIRSTHIT'): + rows += [(ms, kind, f) for ms, f in t.lines.get(kind, [])] + # Trace keeps per-kind lists; re-merge by time, GRECX before FIRSTHIT at equal ms (same call order). + rows.sort(key=lambda r: (r[0], 0 if r[1] == 'GRECX' else 1)) + active = {} + for ms, kind, f in rows: + if kind == 'GRECX': + owner = f[0] + elif owner is not None: + on = f[3] not in ('0', '-1') or f[4] not in ('0', '-1') + if active.get(owner) != on: + active[owner] = on + events.append((ms, owner, on)) + wins = [] + open_since = {} + for ms, o, on in events: + if on: + open_since[o] = ms + elif o in open_since: + wins.append((open_since.pop(o) - 500, ms + 500)) + end = max((ms for ms, _ in t.lines.get('MARK', [])), default=float('inf')) + wins += [(a - 500, end) for a in open_since.values()] + wins.sort() + merged = [] + for a, b in wins: + if merged and a <= merged[-1][1]: + merged[-1] = (merged[-1][0], max(merged[-1][1], b)) + else: + merged.append((a, b)) + return merged + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument('trace') + ap.add_argument('--top', type=int, default=3) + a = ap.parse_args() + t = Trace(a.trace) + bad = t.malformed().get('COLLPOST', 0) + if bad: + print(f'REFUSED: {bad} malformed COLLPOST lines; the session is unusable, record it again') + return 2 + posts = t.lines.get('COLLPOST', []) + if not posts: + print('no COLLPOST lines (category audiox off, or an old build)') + return 1 + wins = bail_windows(t) + first = min(ms for ms, _ in posts) + marks = [ms for ms, f in t.lines.get('MARK', []) if f and f[0].startswith('script start')] + t0 = marks[0] if marks else first + t1 = max(ms for ms, _ in t.lines.get('MARK', [(posts[-1][0], [])])) + in_bail = lambda ms: any(x <= ms <= y for x, y in wins) # noqa: E731 + bail_ms = sum(min(y, t1) - max(x, t0) for x, y in wins if y > t0 and x < t1) + span = {'bail': max(bail_ms, 1.0), 'other': max(t1 - t0 - bail_ms, 1.0)} + print(f'{a.trace}: {len(posts)} COLLPOST; window {t0 / 1000:.1f}..{t1 / 1000:.1f} s; bail windows ' + f'{len(wins)} ({bail_ms / 1000:.1f} s)') + groups = defaultdict(list) + for ms, f in posts: + if not t0 <= ms <= t1: + continue + groups[('bail' if in_bail(ms) else 'other', f[0], f[3])].append(f) + for key in sorted(groups): + kind, fn, owner = key + g = groups[key] + rate = len(g) / (span[kind] / 1000) + print(f'\n[{kind}] {CALLERS.get(fn, fn)} owner {owner}: {len(g)} posts, {rate:.2f}/s') + pairs = Counter((f[5], f[6], f[7], f[8]) for f in g) + print(' matA matB tierA tierB: ' + ', '.join(f'{p[0]}/{p[1]} t{p[2]}/{p[3]} x{n}' for p, n in pairs.most_common(6))) + sounds = Counter(int(f[9], 16) for f in g) + Counter(int(f[10], 16) for f in g) + print(' sound fields: ' + ', '.join(f'{s:X} x{n}' for s, n in sounds.most_common(8))) + print(' local72: ' + ', '.join(f'{v} x{n}' for v, n in Counter(f[4] for f in g).most_common())) + print(' objects: ' + ', '.join(f'{v} x{n}' for v, n in Counter(f[1] for f in g).most_common(3))) + for c, n in Counter(f[15] for f in g).most_common(a.top): + print(f' chain {c} x{n}') + splc = Counter() + for ms, f in t.lines.get('SPLC', []): + if t0 <= ms <= t1 and len(f) >= 2 and f[1].isdigit() and int(f[1]) in BODY_IDS: + splc[('bail' if in_bail(ms) else 'other', int(f[1]))] += 1 + print('\nSPLC body sounds (all owners): ' + ', '.join( + f'{k} {i} {BODY_IDS[i]} {n} ({n / (span[k] / 1000):.2f}/s)' for (k, i), n in sorted(splc.items()))) + return 0 + + +if __name__ == '__main__': + sys.exit(main())