diff --git a/docs/hails-additions/04-map-spawns.md b/docs/hails-additions/04-map-spawns.md new file mode 100644 index 00000000..442dee8d --- /dev/null +++ b/docs/hails-additions/04-map-spawns.md @@ -0,0 +1,149 @@ +# 4. Authored map spawns + +Each converted `.skate` map stores one spawn (position + heading). The game +uses it as the map's **startup position** (the saved default map) and as the +menu's **default landing zone**; `teleports.json` destinations are only used +when the player picks one. So the baked spawn must be right for every map. + +## Problems observed in play + +- **MegaPark:** the skater fell forever. Log: `PhysicsGround->PhysicsAir` + 9 ticks after spawning, never landing; the respawn (`Teleporting`) returned + to the same point, so it looped. +- **Industrial Skate Park, StartPark:** spawned on the roof; only "a small + part" of the park was visible. +- **Industrial, DownTown:** spawn "not where the original game puts you". + +## Root cause + +`tools/asset_pipeline/map_writer.py` (`SpawnSelector`) chose, for every +district except University, the **upward-wound collision triangle nearest the +world origin** and spawned 1 m above its centre. Two properties of the retail +data defeat that: + +1. Skate-park origins sit at a corner of the park, not its middle. +2. Retail collision winding is inconsistent: some park floors face down + (StartPark's and Industrial Skate Park's ground floors at y = 0), so they + were never candidates. + +Measured with the collision extracted from each park: + +| Map | Old spawn | What was there | +|---|---|---| +| MegaPark | (21.7, 21.6, −19.7) | 6.3 m² double-sided panel at the park edge; **nothing below it** | +| IndustrialSkatePark | (−1.6, 40.3, 2.0) | roof (interior: floor 0 m, level 15 m, ceiling 39.6 m, roof 43 m) | +| StartPark | (1.5, 33.2, −9.8) | roof | + +The retail game does not use geometry at all: it has authored start +locators (below). The converter never read them. + +## Intermediate attempt (kept as a fallback) + +A geometric `ground_spawn()` was added first: ignore winding, search outward +from the area-weighted centre of flat surfaces, and accept a point that is +the lowest surface at its XZ, has ≥ 2.5 m headroom, and has no flat surface +within 25 m (by triangle extent) more than 1 m lower. + +**Regression found in play:** applied to city maps, "lowest surface" is +under the streets. Industrial's spawn became (−864.7, −3.0, 414.1); the player +stood on collision under the map and the world looked invisible (back faces +culled from below). Fix: only districts whose collision spans ≤ 500 m +(`GROUND_MAX_EXTENT`, i.e. the parks) use `ground_spawn`; larger districts keep +the original rule unchanged. A unit test pins this. Lesson recorded: a spawn +change moves every map's startup position, so all maps must be checked. + +`ground_spawn` now only applies if a district has no authored start. + +## Final change: authored starts from retail data + +New `tools/asset_pipeline/map_starts.py` resolves each district's start from +the game's own data, in priority order: + +1. **`world` row start locator** — field `Hash_3735C5C12E8E7AE1` in + `skatercollections` names the start locator of each skate park + (`dist_megapark` → `Z_MegaPark_SkatePark`, etc.). +2. **Chosen `fe_locations` row** for districts with several authored + locations and no single start (`DEFAULT_LOCATIONS`, a project choice — + see below). +3. **The district's only `fe_locations` row** (BlackBoxPark, SkateSchool). + +The locator's matrix comes from the district's own +`data/content/global_locators/**//*.rx2` EB0009 records (parsed with +the existing `teleports.location_records`; row 3 = position, row 2 = forward). +Missing or ambiguous locators fall back to the geometric rule. + +Heading is now written into the `.skate` header (the `f32` after the spawn, +previously always 0): `heading = atan2(forward.x, forward.z)`, matching the +game's `Mat3::from_rotation_y(heading)` (forward = (sin h, 0, cos h)). The +convention was checked against a logged teleport (Aletown: locator forward +(0.1536, 0, 0.9881) = the game's `requested_at`). + +`_install` computes the starts once before map conversion and writes +`/map-starts.json`; each `convert_map` job reads its district's entry. + +### Resulting defaults + +| Map | Locator | Source | Position | Heading | +|---|---|---|---|---| +| BlackBoxPark | `A_BlackBoxPark` | fe_locations `dist_blackboxpark` | (10.55, 0.08, 6.05) | 90° | +| DownTown | `Z_DT_SkatePark` (Rosalita Skate Park) | **choice** → fe `dist_downtown_skatepark` | (−254.05, 40.54, 32.39) | −85.5° | +| DownTownSkatePark | `Z_SP_DT_Start` | world `dist_downtownpark` | (66.05, 0.0, −64.02) | 0° | +| Industrial | `Z_Ind_ReclaimedHastings` (Haystings Park) | **choice** → fe `dist_industrial_reclaimed` | (−89.16, 1.01, −138.35) | −45.4° | +| IndustrialSkatePark | `Z_SP_Ind_SP_Start` | world `dist_industrialpark` | (−64.01, 0.75, 0.0) | 0° | +| MaloofMoneyCup | `A_Maloof_Street` | **choice** → fe `dist_maloof_street` | (0.56, −0.06, −43.42) | 180° | +| MegaPark | `Z_MegaPark_SkatePark` | world `dist_megapark` | (230.63, 0.48, −73.08) | −90° | +| SkateSchool | `tele_world_to_school_dest_locator_01` | fe `dist_skateschool` | (−31.28, 0.11, −31.5) | 0° | +| StartPark | `Z_SP_SkateParkStart2` | world `dist_startpark` | (−64.01, 0.0, 0.0) | 0° | +| University | `ftl_univ_02_challengelocator_01` (career start) | **choice** → fe `gamestart` | (294.64, 74.12, −432.6) | 114.5° | + +### Project choices (maintainers may prefer others) + +Retail has no single start for the cities (you always arrive at a location +picked in the map menu) or for Maloof (two locations). Chosen: + +- **University:** the retail career start (`gamestart`). +- **Industrial:** Haystings Park, the district's skate park. +- **DownTown:** Rosalita Skate Park; retail data also links the DownTown + skate-park world to this spot. +- **MaloofMoneyCup:** the street course. + +Previous University spawn was a fixed point next to Super-Ultra Mega-Park, +(330, 133.005, −710). + +## Other findings + +- Locator files for the parks live under `global_locators/PARK/`, not `BAM/`. +- `teleports.json` marks MegaPark's only FE destination ("Stadium – Monster + Park", locator `vert_skp5_01_challengelocator_01`) unavailable: that locator + does not exist in the shipped content. Unrelated to spawns; left as is. +- The map-conversion work runs in subprocesses (`map_job.py`) capped at 3 + workers; see performance notes elsewhere. + +## Files + +- `tools/asset_pipeline/map_starts.py` (new) +- `tools/asset_pipeline/install.py` — computes starts before conversion; + `convert_map` uses them. +- `tools/asset_pipeline/map_writer.py` — `ground_spawn`, `GROUND_*` + constants, `SpawnSelector` changes, `spawn_point` no longer prunes by + distance for non-University districts, `write(..., prepared_heading=)`. +- `tools/asset_pipeline/test_map_writer.py`, `tools/asset_pipeline/test_map_starts.py`. + +Pipeline fingerprints change for the `maps` and `environment` groups, so an +existing install rebuilds those groups on its next asset refresh. + +## Verification + +- Unit tests (28 across the setup/map suites) pass, including: roof over a + down-facing floor, floating panel over the void, city-sized districts + keeping the original rule, heading convention. +- Collision analysis per park (extract with the collision consumer, then + query all surfaces at an XZ) confirmed the old failures and the + ground_spawn results. +- In play (before authored starts): Industrial with the restored city rule + spawned visible and grounded. +- Authored starts, after an asset refresh (maps + environment groups, 0 + warnings): every map passes `skate3rust --check-assets`, and each baked + spawn and heading read back from the `.skate` header matches the table + above exactly. +- In-game confirmation of the authored starts: pending. diff --git a/tools/asset_pipeline/install.py b/tools/asset_pipeline/install.py index 56a9b0f4..27f41b35 100644 --- a/tools/asset_pipeline/install.py +++ b/tools/asset_pipeline/install.py @@ -174,7 +174,13 @@ def convert_map(archive,work,maps,stage,game_exe,log,report): build_archive(manifest_path,collision) finished('collision_archive') final=maps/(label+'.skate') - write_map(manifest_path,final,collision,report,prepared_spawn=spawn.result(label)) + # Authored retail start (map_starts.py) when _install found one; else geometry. + starts=work/'map-starts.json' + start=json.loads(starts.read_text(encoding='utf-8')).get(district) if starts.is_file() else None + if start:report(f"{label}: authored start {start['locator']} ({start['source']})") + write_map(manifest_path,final,collision,report, + prepared_spawn=tuple(start['position']) if start else spawn.result(label), + prepared_heading=start['heading'] if start else 0.) finished('write_map') from .dynamic_props import export as write_props caches=list((work/'dmo/cache').glob('DMO_*')) @@ -337,6 +343,13 @@ def _install(iso,base,game_exe,report,game_root=None,refresh=False,finalize=None if 'maps' in groups: archives=list((game_root/'data/content').glob('worldDIST_*.big')) archives.sort(key=lambda p:(p.stem!='worldDIST_University',p.name.lower())) + from .map_starts import prepare as map_starts + try: + starts=map_starts(game_root,converted) + except CONTENT_ERRORS as error: + starts={} + report(f'Authored map starts unavailable ({error}); using geometric spawns') + (work/'map-starts.json').write_text(json.dumps(starts),encoding='utf-8') workers=map_workers() report(f'Converting {len(archives)} maps with {workers} workers') def map_job(archive): diff --git a/tools/asset_pipeline/map_starts.py b/tools/asset_pipeline/map_starts.py new file mode 100644 index 00000000..1e92a0e5 --- /dev/null +++ b/tools/asset_pipeline/map_starts.py @@ -0,0 +1,100 @@ +"""Authored default spawn (position + heading) per district from retail data. + +The map spawn is every map's startup position and the menu's "default landing +zone". Retail data names it directly for skate parks; other districts use one +retail FE location. Positions/headings come from the district's own +global_locators EB0009 records (see teleports.py), never from geometry. + +Priority per district: +1. `world` row start locator (field Hash_3735C5C12E8E7AE1; the skate parks), +2. DEFAULT_LOCATIONS: an fe_locations row chosen for districts with several + (cities, Maloof) — project choices, documented in docs/hails-additions, +3. the district's only fe_locations row (BlackBoxPark, SkateSchool). +Districts without a match keep the geometric rule in map_writer. +""" +import json +import math +import tempfile +from pathlib import Path + +from tools.owned_game.big import BigArchive +from .environment import Collections, key_hash +from .teleports import cstring, location_records +from .vlt import vault + +START_LOCATOR = 'Hash_3735C5C12E8E7AE1' +DEFAULT_LOCATIONS = { + 'DIST_University': 'gamestart', # retail career start + 'DIST_Industrial': 'dist_industrial_reclaimed', # Haystings Park + 'DIST_DownTown': 'dist_downtown_skatepark', # Rosalita Skate Park + 'DIST_MaloofMoneyCup': 'dist_maloof_street', # street course +} + + +def heading(forward): + """Game convention: forward = (sin h, 0, cos h) (Mat3::from_rotation_y).""" + return math.atan2(forward[0], forward[2]) + + +def locators(game_root): + found = {} + for path in sorted((Path(game_root) / 'data/content/global_locators').rglob('*.rx2')): + for record in location_records(path.read_bytes()): + found.setdefault((path.parent.name.casefold(), record['locator']), []).append(record['matrix']) + return found + + +def prepare(game_root, converted): + """Return {district: {locator, source, position, heading}}.""" + game_root = Path(game_root) + collections = Collections(converted) + database = BigArchive(game_root / 'data/big/db.big') + with tempfile.TemporaryDirectory(prefix='skate-map-starts-') as tmp: + entries = [e for e in database.entries if e.path in + ('data/db/skatercollections.bin', 'data/db/skatercollections.vlt')] + database.extract_entries(entries, Path(tmp)) + _, binary, _ = vault(Path(tmp) / 'data/db/skatercollections') + + def stream_of(world_key): + world, _ = collections.resolve('world', world_key) + value = world.get(key_hash('WorldStream')) + return value['data'] if value else None + + wanted = {} + fe_rows = {} + for (cls, key), row in collections.rows.items(): + if cls == key_hash('world'): + fields, _ = collections.resolve('world', key) + start, stream = fields.get(key_hash(START_LOCATOR)), stream_of(key) + if start and stream and stream.startswith('DIST_'): + # Parks list several world rows (full/empty/tutorial); the + # stream's own row is the one named after it. + if row['key'].casefold() == ('dist_' + stream[5:]).casefold() or stream not in wanted: + wanted[stream] = (start['data'], 'world ' + row['key']) + elif cls == key_hash('fe_locations'): + fields, _ = collections.resolve(cls, key) + location = cstring(binary, int(fields[key_hash('location')]['data'], 16)).decode('utf-8') + world = fields.get(key_hash('World')) + stream = stream_of(int(world['data'][:16], 16)) if world else None + fe_rows[row['key']] = (location, stream) + for district, key in DEFAULT_LOCATIONS.items(): + if district not in wanted and key in fe_rows: + wanted[district] = (fe_rows[key][0], 'fe_locations ' + key) + by_stream = {} + for key, (location, stream) in fe_rows.items(): + if stream: + by_stream.setdefault(stream, []).append((location, key)) + for stream, rows in by_stream.items(): + if stream not in wanted and len({location for location, _ in rows}) == 1: + wanted[stream] = (rows[0][0], 'fe_locations ' + rows[0][1]) + + records = locators(game_root) + result = {} + for district, (locator, source) in sorted(wanted.items()): + matrices = records.get((district.casefold(), locator), []) + if len({json.dumps(m) for m in matrices}) != 1: + continue # missing or ambiguous: keep the geometric rule + m = matrices[0] + result[district] = dict(locator=locator, source=source, + position=[float(v) for v in m[3][:3]], heading=heading(m[2])) + return result diff --git a/tools/asset_pipeline/map_writer.py b/tools/asset_pipeline/map_writer.py index 6365f5f9..a4447156 100644 --- a/tools/asset_pipeline/map_writer.py +++ b/tools/asset_pipeline/map_writer.py @@ -45,16 +45,67 @@ def write_textures(output,root,textures): if name is not None:pending.append(pool.submit(packed_texture,root,name,textures[name])) def normalise(a):return a/np.maximum(np.linalg.norm(a,axis=1,keepdims=True),1e-20) +GROUND_HEADROOM=2.5 # clear metres above a spawn surface +GROUND_RADIUS=25. # neighbourhood searched for lower ground +GROUND_STEP=1. # a surface this far above nearby ground is a roof/platform +GROUND_CANDIDATES=512 +# Only compact districts (skate parks) use ground_spawn. City districts are +# kilometres wide with collision under the streets, where "lowest surface" +# is underground; they keep the original rule. +GROUND_MAX_EXTENT=500. + +def ground_spawn(triangles): + """Walkable ground nearest the area-weighted centre of flat surfaces. + + Retail collision winding is not consistent (some park floors face down), + so orientation is ignored. A candidate must be the lowest surface at its + XZ, have headroom, and have no ground more than GROUND_STEP lower nearby; + that rejects roofs and floating panels at a park's edge. Returns None if + no candidate qualifies.""" + t=np.asarray(triangles,dtype=np.float64) + a,b,c=t[:,0],t[:,1],t[:,2] + ab=b-a;ac=c-a + cross=np.cross(ab,ac);length=np.linalg.norm(cross,axis=1) + flat=(length>=4)&(np.abs(cross[:,1])>=.9*length) + if not flat.any():return None + points=((a+b+c)/3)[flat];weights=length[flat] + middle=(points[:,[0,2]]*weights[:,None]).sum(0)/weights.sum() + order=np.argsort(((points[:,[0,2]]-middle)**2).sum(1)) + det=ab[:,0]*ac[:,2]-ac[:,0]*ab[:,2] + low=np.minimum(np.minimum(a,b),c);high=np.maximum(np.maximum(a,b),c) + flat_low=low[flat];flat_high=high[flat] + for index in order[:GROUND_CANDIDATES]: + x,y,z=points[index] + # Extents, not centres: one large floor triangle can reach the candidate. + near=((flat_low[:,0]<=x+GROUND_RADIUS)&(flat_high[:,0]>=x-GROUND_RADIUS) + &(flat_low[:,2]<=z+GROUND_RADIUS)&(flat_high[:,2]>=z-GROUND_RADIUS)) + if flat_low[near,1].min()=1e-12)).nonzero()[0] + d=det[under];dx=x-a[under,0];dz=z-a[under,2] + first=(dx*ac[under,2]-ac[under,0]*dz)/d;second=(ab[under,0]*dz-dx*ab[under,2])/d + inside=(first>=-1e-5)&(second>=-1e-5)&(first+second<=1.00001) + heights=(a[under,1]+first*ab[under,1]+second*ac[under,1])[inside] + if heights.size and heights.min()y+.05] + if above.size and above.min()-yselector.best[0]+.01:continue selector.consider(decode_rx2_clustered_meshes((root/entry['rx2']).read_bytes())) return selector.result(manifest['map_name']) -def write(manifest_path,output,collision,report=lambda _:None, *, render_only=False, prepared_spawn=None): +def write(manifest_path,output,collision,report=lambda _:None, *, render_only=False, prepared_spawn=None, prepared_heading=0.): root=manifest_path.parent;m=json.loads(manifest_path.read_text());textures=m['textures'] ids={name:i+1 for i,name in enumerate(sorted(textures))} excluded=set(m['normal_texture_policy']['excluded_texture_ids']) @@ -171,7 +224,8 @@ def write(manifest_path,output,collision,report=lambda _:None, *, render_only=Fa 1.,.96,.86,.42,.56,.92,1.,.18,.34,.10,1.,1.,1.] rails=m['grind_splines'];output.parent.mkdir(parents=True,exist_ok=True) with output.open('wb') as f: - f.write(b'SKATE14\0');u(f,0x12345678);string(f,m['map_name']);floats(f,*spawn,0.,*environment) + # Heading (radians about +Y; forward = (sin h, 0, cos h)) follows the spawn. + f.write(b'SKATE14\0');u(f,0x12345678);string(f,m['map_name']);floats(f,*spawn,0. if render_only else prepared_heading,*environment) u(f,nm,len(ids),nv,ni,0,len(rails),0,0,0);f.write(mats.getvalue()) write_textures(f,root,textures) stored(f,vertices.getvalue());stored(f,indices.getvalue());stored(f,b'') diff --git a/tools/asset_pipeline/test_map_starts.py b/tools/asset_pipeline/test_map_starts.py new file mode 100644 index 00000000..2e4d690c --- /dev/null +++ b/tools/asset_pipeline/test_map_starts.py @@ -0,0 +1,23 @@ +import math +import unittest + +from .map_starts import DEFAULT_LOCATIONS, heading + + +class MapStartTests(unittest.TestCase): + def test_heading_matches_game_rotation_convention(self): + # skate-game: basis = Mat3::from_rotation_y(heading), forward = (sin h, 0, cos h). + for forward, expected in [((0, 0, 1), 0.), ((1, 0, 0), math.pi / 2), + ((-1, 0, 0), -math.pi / 2), ((0.1536, 0, 0.9881), math.atan2(0.1536, 0.9881))]: + h = heading(forward) + self.assertAlmostEqual(h, expected, places=6) + self.assertAlmostEqual(math.sin(h), forward[0] / math.hypot(forward[0], forward[2]), places=6) + self.assertAlmostEqual(math.cos(h), forward[2] / math.hypot(forward[0], forward[2]), places=6) + + def test_chosen_defaults_cover_districts_without_a_retail_start(self): + self.assertEqual(set(DEFAULT_LOCATIONS), + {'DIST_University', 'DIST_Industrial', 'DIST_DownTown', 'DIST_MaloofMoneyCup'}) + + +if __name__ == '__main__': + unittest.main() diff --git a/tools/asset_pipeline/test_map_writer.py b/tools/asset_pipeline/test_map_writer.py index bb2589ce..6786f9fe 100644 --- a/tools/asset_pipeline/test_map_writer.py +++ b/tools/asset_pipeline/test_map_writer.py @@ -59,5 +59,28 @@ class MapWriterTests(unittest.TestCase): with self.assertRaisesRegex(ValueError,'No supported spawn'): SpawnSelector('DIST_Test').result('test') + def test_spawn_prefers_ground_over_roofs_and_floating_panels(self): + def tri(a,b,c):return SimpleNamespace(a=a,b=b,c=c) + def mesh(*triangles):return SimpleNamespace(bounds_min=(-1e3,-1e3,-1e3),bounds_max=(1e3,1e3,1e3),triangles=list(triangles)) + # Retail park: floor wound downward, roof above it wound upward. + floor=mesh(tri((-40,0,-40),(40,0,-40),(-40,0,40)),tri((40,0,-40),(40,0,40),(-40,0,40))) + roof=mesh(tri((-40,30,-40),(-40,30,40),(40,30,-40)),tri((40,30,-40),(-40,30,40),(40,30,40))) + selector=SpawnSelector('DIST_Park');selector.consider([roof,floor]) + self.assertEqual(selector.result('Park')[1],1.) + # A small panel over the void in the middle of two floor slabs. + slabs=mesh(tri((-60,0,-20),(-60,0,20),(-10,0,-20)),tri((10,0,-20),(60,0,20),(60,0,-20))) + panel=mesh(tri((-2,20,-2),(-2,20,2),(2,20,-2))) + selector=SpawnSelector('DIST_Park');selector.consider([panel,slabs]) + x,y,z=selector.result('Park') + self.assertEqual(y,1.) + self.assertGreater(abs(x),10) + # City-sized districts keep the original rule: the same roof/floor + # layout scaled past GROUND_MAX_EXTENT spawns on the upward roof, as + # before, instead of the "lowest surface" (underground in real cities). + big_floor=mesh(tri((-400,0,-400),(400,0,-400),(-400,0,400)),tri((400,0,-400),(400,0,400),(-400,0,400))) + big_roof=mesh(tri((-400,30,-400),(-400,30,400),(400,30,-400)),tri((400,30,-400),(-400,30,400),(400,30,400))) + selector=SpawnSelector('DIST_City');selector.consider([big_roof,big_floor]) + self.assertEqual(selector.result('City')[1],31.) + if __name__=='__main__':unittest.main()