Publish rolling Experimental builds from main and support updater discovery

.github/workflows/release.yml

@@ -1,39 +1,72 @@
 name: Windows release
 on:
+  push:
+    branches: [main]
   release:
     types: [published]
   workflow_dispatch:
 permissions:
-  contents: write
+  contents: read
 jobs:
   windows:
+    if: github.event_name != 'release' || github.event.release.tag_name != 'experimental'
     runs-on: windows-2025
+    timeout-minutes: 90
+    concurrency:
+      group: windows-build-${{ github.event_name == 'release' && github.event.release.tag_name || github.ref }}
+      cancel-in-progress: true
     steps:
       - uses: actions/checkout@v4
       - uses: dtolnay/rust-toolchain@stable
       - uses: actions/setup-python@v5
         with:
           python-version: '3.13'
-      - name: Install setup packager
-        run: python -m pip install pyinstaller==6.19.0 numpy==2.2.6 Pillow==11.3.0
-      - name: Check updater without starting game
+          cache: pip
+          cache-dependency-path: tools/requirements-setup.txt
+      - uses: Swatinem/rust-cache@v2
+        with:
+          shared-key: windows-release
+          cache-on-failure: true
+      - name: Check updater and publisher without starting game
+        run: python -m unittest discover -s tools -p "test_*release*.py" -v
+      - name: Check updater protocol
         run: python -m unittest discover -s tools -p test_updater.py -v
       - name: Build Windows package
         shell: pwsh
         env:
-          RELEASE_TAG: ${{ github.event.release.tag_name }}
+          RELEASE_TAG: ${{ github.event_name == 'release' && github.event.release.tag_name || 'experimental' }}
         run: ./scripts/Build-Release.ps1
       - uses: actions/upload-artifact@v4
         with:
           name: skate3rust-windows-x64
+          retention-days: 7
           path: |
             target/skate3rust-windows-x64.zip
             target/skate3rust-windows-x64.zip.sha256
             target/release.json
-      - name: Attach to release
+  publish:
+    needs: windows
+    if: github.event_name == 'release' || github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    concurrency:
+      group: windows-release-publish
+      cancel-in-progress: false
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/download-artifact@v4
+        with:
+          name: skate3rust-windows-x64
+          path: target
+      - name: Publish rolling experimental
+        if: github.event_name != 'release'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: python tools/publish_experimental.py --directory target
+      - name: Attach numbered release
         if: github.event_name == 'release'
-        shell: pwsh
         env:
           GH_TOKEN: ${{ github.token }}
           RELEASE_TAG: ${{ github.event.release.tag_name }}
-        run: gh release upload "$env:RELEASE_TAG" target/skate3rust-windows-x64.zip target/skate3rust-windows-x64.zip.sha256 target/release.json --clobber
+        run: gh release upload "$RELEASE_TAG" target/skate3rust-windows-x64.zip target/skate3rust-windows-x64.zip.sha256 target/release.json --clobber

README.md

@@ -10,14 +10,17 @@ Includes skating, tricks, grinds, offboard movement, difficulty settings and
 
 ## Mods and agent authoring
 
-Drop mod ZIPs into top-level `mods/` and use `PLAY-MARIO-KART.bat` for the integrated
-mod build in this checkout. **Point your coding agent at [sdk/AGENTS.md](sdk/AGENTS.md)**
+Drop mod ZIPs into top-level `mods/` and enable them in the Mods menu. **Point your coding agent at [sdk/AGENTS.md](sdk/AGENTS.md)**
 to make a mod. See [package structure](docs/mod-packages.md), [Lua API](docs/lua-modding.md),
 [vehicle API](docs/vehicle-sdk.md), [multiplayer mod SDK](docs/multiplayer-mods.md) and [Mixamo workflow](docs/mixamo-vehicle-workflow.md).
 Editable examples live in `sdk/examples/`; `tools/package_mod.py` validates and packages them.
 
 ## Play
 
+[Download Experimental](https://github.com/SK8-ENGINE/skate-3-rust-engine/releases/tag/experimental).
+Successful `main` builds replace this prerelease. Choose **Latest** in Updates
+for experimental updates; **Stable** is the default.
+
 Extract the Windows release ZIP and run `skate3rust.exe`. Select your Skate 3
 Xbox 360 ISO, or select `default.xex` in an extracted game folder. Keep its
 `data` folder alongside it. Setup prepares the skater, animations and all disc maps, then
@@ -40,8 +43,8 @@ Escape opens difficulty and graphics settings.
 
 Development builds use a prepared asset set in `assets/private/` or the
 installed asset directory. `scripts/Build-Release.ps1` builds the portable Windows
-package and requires Python 3.13. Published GitHub releases build and attach
-the package automatically.
+package and requires Python 3.13. GitHub Actions builds `main` automatically;
+numbered releases are published separately.
 
 Custom animations and climbing support remain available, but no custom clips
 are shipped. The included format-demo map is original procedural content.

docs/updates.md

@@ -44,13 +44,40 @@ exceeds its page/time bound fails rather than claiming the player is up to date.
 Older releases without this metadata are skipped. Existing installations predating
 the updater need one manual portable-package upgrade to bootstrap it.
 
-At implementation time the configured repository was private with no releases.
-Normal players need publicly accessible releases. For private testing, launch with
-`SKATE_UPDATE_GITHUB_TOKEN` set to a token with read access to repository contents.
-The updater does not store this credential; it uses authenticated API asset URLs
-and strips authorization on redirects. Never place tokens in launch arguments,
-tracked scripts or release packages. No repository visibility or release was
-changed by this implementation.
+The repository and release downloads are public. For private forks/testing,
+`SKATE_UPDATE_GITHUB_TOKEN` supplies a token with read access to repository contents.
+The updater never stores it and strips authorization on redirects.
+
+## Rolling Experimental
+
+Every push to `main` builds the Windows package. Successful builds update one
+mutable prerelease tagged `experimental`; it is never marked as GitHub's Latest
+stable release. Manual runs on `main` can publish it too. Builds on other branches
+produce only Actions artifacts. Published numbered releases retain the stable
+workflow. Do not mark the rolling release immutable.
+
+Select **Latest** in Updates to receive Experimental; **Stable** stays the default
+and ignores prereleases. Download the first Experimental ZIP manually if your
+installation predates this rolling-release updater.
+
+Builds share this workflow's existing increasing run number. Experimental asset
+names include that number: `skate3rust-windows-x64-build-N.zip`, its `.sha256`, and
+`release-N.json`. The manifest is uploaded last. The updater ignores incomplete
+sets and compares build numbers even when the release tag has not changed. The
+current and previous complete sets are kept; a very old pending download can
+expire after another publication and should be checked again. The internal ZIP
+folder and installed `release.json` names remain unchanged.
+
+Superseded compilation runs are cancelled. Publication is serialized and never
+cancelled by a newer push; stale source revisions and older/equal builds are not
+published. A failed build leaves the previous download available. Failed uploads
+cannot make a partial package eligible. The release body has a current download
+link, build/revision identity and ten recent commit summaries. Actions artifacts
+expire after seven days; dependency caching reduces repeat compilation cost.
+
+The publisher uses GitHub's workflow token with contents-write permission only in
+the publish job. No personal token or game assets are included. Standard hosted
+runners are free for public repositories; private forks use their own allowance.
 
 The ZIP checksum is mandatory; GitHub's asset digest is also checked when present.
 The internal/external manifests and individual executable hashes must agree.

tools/publish_experimental.py

@@ -0,0 +1,115 @@
+"""Publish one rolling prerelease, with complete build-specific asset sets."""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import re
+import shutil
+import subprocess
+import zipfile
+
+from updater import REPO, PACKAGE, PREFIX, identity, program_metadata, package_name
+
+TAG = 'experimental'
+
+
+def gh(*args, payload=None):
+    command = ['gh', *args]
+    if payload is not None:
+        command += ['--input', '-']
+    result = subprocess.run(command, input=json.dumps(payload) if payload is not None else None,
+                            text=True, capture_output=True)
+    if result.returncode:
+        raise RuntimeError(result.stderr.strip() or 'GitHub command failed')
+    return result.stdout
+
+
+def api(path, method='GET', payload=None):
+    return json.loads(gh('api', f'repos/{REPO}/{path}', '--method', method, payload=payload) or 'null')
+
+
+def complete_builds(assets):
+    names = {a['name'] for a in assets if a.get('state') == 'uploaded'}
+    result = []
+    for name in names:
+        match = re.fullmatch(r'release-([1-9][0-9]*)\.json', name)
+        if match:
+            build = int(match[1])
+            package = f'skate3rust-windows-x64-build-{build}.zip'
+            if {package, package + '.sha256'} <= names:
+                result.append(build)
+    return sorted(result, reverse=True)
+
+
+def validate(directory):
+    meta = json.loads((directory/'release.json').read_text(encoding='utf-8-sig'))
+    identity(meta)
+    program_metadata(meta)
+    if meta['tag'] != TAG:
+        raise ValueError('Expected experimental package')
+    archive = directory/PACKAGE
+    with archive.open('rb') as stream:
+        digest = hashlib.file_digest(stream, 'sha256').hexdigest()
+    checksum = (directory/(PACKAGE+'.sha256')).read_text().split()
+    if checksum != [digest, PACKAGE]:
+        raise ValueError('Package checksum mismatch')
+    with zipfile.ZipFile(archive) as z:
+        if json.loads(z.read(PREFIX+'release.json').decode('utf-8-sig')) != meta:
+            raise ValueError('Package identity mismatch')
+        for name, expected in meta['files'].items():
+            if hashlib.sha256(z.read(PREFIX+name)).hexdigest() != expected:
+                raise ValueError('Program checksum mismatch')
+    return meta, digest
+
+
+def publish(directory):
+    meta, digest = validate(directory)
+    if api('git/ref/heads/main')['object']['sha'] != meta['revision']:
+        print('A newer main commit exists; leaving Experimental unchanged.')
+        return
+    # Include drafts so interrupted first publication can resume without another release.
+    releases = json.loads(gh('api', f'repos/{REPO}/releases?per_page=100', '--paginate', '--slurp'))
+    release = next((r for page in releases for r in page if r['tag_name'] == TAG), None)
+    if release and (release.get('immutable') or not release.get('prerelease')):
+        raise ValueError('Experimental must be a mutable prerelease')
+    if release and any(n >= meta['build'] for n in complete_builds(release.get('assets', []))) and not release['draft']:
+        print('This build or a newer build is already published.')
+        return
+    if release is None:
+        release = api('releases', 'POST', dict(tag_name=TAG, target_commitish=meta['revision'],
+                      name='Experimental', draft=True, prerelease=True, make_latest='false'))
+    package = package_name(meta)
+    shutil.copyfile(directory/PACKAGE, directory/package)
+    (directory/(package+'.sha256')).write_text(digest+'  '+package+'\n', encoding='ascii')
+    manifest = f"release-{meta['build']}.json"
+    shutil.copyfile(directory/'release.json', directory/manifest)
+    # Payload/checksum first, manifest last: clients ignore an incomplete upload.
+    for name in (package, package+'.sha256', manifest):
+        gh('release', 'upload', TAG, str(directory/name), '--repo', REPO, '--clobber')
+    current = api(f"releases/{release['id']}")
+    if meta['build'] not in complete_builds(current['assets']):
+        raise ValueError('Uploaded build is incomplete')
+    changes = gh('api', f'repos/{REPO}/commits?sha={meta["revision"]}&per_page=10')
+    commits = json.loads(changes)
+    notes = (f"Rolling build **{meta['build']}**, commit `{meta['revision'][:12]}`.\n\n"
+             f"[Download Windows ZIP](https://github.com/{REPO}/releases/download/{TAG}/{package})\n\n"
+             "Experimental builds may contain regressions. Choose Latest in the in-game Updates window "
+             "to receive these builds. Stable remains the default. Your existing maps and assets are kept.\n\n"
+             "### Recent commits\n\n" + '\n'.join(
+                 f"- {c['sha'][:8]} {c['commit']['message'].splitlines()[0]}" for c in commits))
+    api(f"releases/{release['id']}", 'PATCH', dict(name='Experimental', body=notes,
+        target_commitish=meta['revision'], draft=False, prerelease=True, make_latest='false'))
+    api('git/refs/tags/'+TAG, 'PATCH', dict(sha=meta['revision'], force=True))
+    # Keep the previous complete download for clients already fetching it.
+    keep = set(complete_builds(current['assets'])[:2])
+    for asset in current['assets']:
+        match = re.fullmatch(r'(?:release-([1-9][0-9]*)\.json|skate3rust-windows-x64-build-([1-9][0-9]*)\.zip(?:\.sha256)?)', asset['name'])
+        if match and int(match[1] or match[2]) not in keep:
+            api(f"releases/assets/{asset['id']}", 'DELETE')
+    print(f"Published Experimental build {meta['build']}")
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--directory', type=Path, required=True)
+    publish(parser.parse_args().directory)

tools/test_experimental_release.py

@@ -0,0 +1,41 @@
+"""Publisher checks without network or a game process."""
+import hashlib
+import json
+from pathlib import Path
+import tempfile
+import unittest
+import zipfile
+import publish_experimental as p
+from test_updater import metadata
+
+
+class RollingReleaseTests(unittest.TestCase):
+    def test_only_complete_build_sets_are_retained(self):
+        assets = [dict(name=n, state='uploaded') for n in (
+            'release-3.json', 'skate3rust-windows-x64-build-3.zip',
+            'skate3rust-windows-x64-build-3.zip.sha256', 'release-4.json')]
+        self.assertEqual(p.complete_builds(assets), [3])
+        assets[0]['state'] = 'new'
+        self.assertEqual(p.complete_builds(assets), [])
+
+    def test_package_validation_and_corruption(self):
+        with tempfile.TemporaryDirectory() as temp:
+            root = Path(temp)
+            meta = {**metadata(), 'tag': 'experimental'}
+            meta['files'] = {n: hashlib.sha256(b'program').hexdigest() for n in meta['files']}
+            (root/'release.json').write_text(json.dumps(meta))
+            with zipfile.ZipFile(root/p.PACKAGE, 'w') as z:
+                z.writestr(p.PREFIX+'release.json', json.dumps(meta))
+                for name in meta['files']:
+                    z.writestr(p.PREFIX+name, b'program')
+            digest = hashlib.sha256((root/p.PACKAGE).read_bytes()).hexdigest()
+            checksum = root/(p.PACKAGE+'.sha256')
+            checksum.write_text(digest+'  '+p.PACKAGE)
+            self.assertEqual(p.validate(root), (meta, digest))
+            checksum.write_text('0'*64+'  '+p.PACKAGE)
+            with self.assertRaisesRegex(ValueError, 'checksum'):
+                p.validate(root)
+
+
+if __name__ == '__main__':
+    unittest.main()

tools/test_updater.py

@@ -42,6 +42,26 @@ class UpdaterTests(unittest.TestCase):
             self.assertEqual(u.discover(metadata(1), 'Latest', threading.Event())[0], 4)
             self.assertIsNone(u.discover(metadata(5), 'Latest', threading.Event()))
 
+    def test_rolling_release_same_tag_new_build_and_partial_upload(self):
+        def release():
+            names = []
+            for build in (2, 3):
+                names += [f'release-{build}.json', f'skate3rust-windows-x64-build-{build}.zip',
+                          f'skate3rust-windows-x64-build-{build}.zip.sha256']
+            names += ['release-4.json']  # Incomplete upload must not be offered.
+            return dict(id=1, tag_name='experimental', draft=False, prerelease=True,
+                        published_at='date', assets=[dict(name=n, state='uploaded', browser_download_url=n) for n in names])
+        def fetch(url, *args):
+            if url.startswith(u.API):
+                return json.dumps([release()]).encode()
+            build = int(url.removeprefix('release-').removesuffix('.json'))
+            return json.dumps({**metadata(build), 'tag': 'experimental'}).encode()
+        with patch.object(u, 'fetch', fetch):
+            current = {**metadata(2), 'tag': 'experimental'}
+            self.assertEqual(u.discover(current, 'Latest', threading.Event())[0], 3)
+            self.assertIsNone(u.discover(current, 'Stable', threading.Event()))
+            self.assertIsNone(u.discover({**current, 'build': 3}, 'Latest', threading.Event()))
+
     def package(self, extra=None):
         meta = metadata()
         meta['files'] = {n: hashlib.sha256(b'new').hexdigest() for n in u.FILES[:-1]}

tools/updater.py

@@ -130,6 +130,20 @@ def eligible(release, channel):
             and (channel == 'Latest' or not release.get('prerelease')))
 
 
+def package_name(meta):
+    # Rolling asset names are derived from a validated integer, never a path
+    # supplied by release notes or an arbitrary manifest field.
+    return f'skate3rust-windows-x64-build-{identity(meta)}.zip' if meta['tag'] == 'experimental' else PACKAGE
+
+
+def release_candidates(assets, rolling):
+    if not rolling:
+        return [('release.json', PACKAGE)]
+    builds = sorted((int(m[1]) for name in assets
+                     if (m := re.fullmatch(r'release-([1-9][0-9]*)\.json', name))), reverse=True)
+    return [(f'release-{n}.json', f'skate3rust-windows-x64-build-{n}.zip') for n in builds]
+
+
 def discover(current, channel, cancel):
     current_build = identity(current)
     candidates = []
@@ -145,17 +159,19 @@ def discover(current, channel, cancel):
             if not eligible(release, channel):
                 continue
             assets = {a['name']: a for a in release.get('assets', []) if a.get('state') == 'uploaded'}
-            if not {PACKAGE, PACKAGE + '.sha256', 'release.json'} <= assets.keys():
-                continue
-            try:
-                meta = json.loads(fetch(asset_url(assets['release.json']), cancel, 65536))
-                build = identity(meta)
-                program_metadata(meta)
-                if meta['tag'] != release['tag_name'] or build <= current_build:
+            for manifest_name, package in release_candidates(assets, release.get('tag_name') == 'experimental'):
+                if not {package, package + '.sha256', manifest_name} <= assets.keys():
                     continue
-            except (ValueError, KeyError):
-                continue
-            candidates.append((build, release['id'], release, assets, meta))
+                try:
+                    meta = json.loads(fetch(asset_url(assets[manifest_name]), cancel, 65536))
+                    build = identity(meta)
+                    program_metadata(meta)
+                    if (meta['tag'] != release['tag_name'] or build <= current_build
+                            or package_name(meta) != package):
+                        continue
+                except (ValueError, KeyError):
+                    continue
+                candidates.append((build, release['id'], release, assets, meta))
         if len(releases) < 100:
             break
     else:
@@ -165,14 +181,15 @@ def discover(current, channel, cancel):
 
 def stage(candidate, directory, cancel, progress):
     _, _, release, assets, meta = candidate
-    checksum = fetch(asset_url(assets[PACKAGE + '.sha256']), cancel, 1024).decode('ascii').split()
-    if len(checksum) != 2 or checksum[1] != PACKAGE or not re.fullmatch('[0-9a-fA-F]{64}', checksum[0]):
+    package = package_name(meta)
+    checksum = fetch(asset_url(assets[package + '.sha256']), cancel, 1024).decode('ascii').split()
+    if len(checksum) != 2 or checksum[1] != package or not re.fullmatch('[0-9a-fA-F]{64}', checksum[0]):
         raise ValueError('Invalid release checksum')
-    archive = fetch(asset_url(assets[PACKAGE]), cancel, 1024 * 1024 * 1024, progress)
+    archive = fetch(asset_url(assets[package]), cancel, 1024 * 1024 * 1024, progress)
     digest = hashlib.sha256(archive).hexdigest()
     if digest != checksum[0].lower():
         raise ValueError('Package checksum mismatch')
-    api_digest = assets[PACKAGE].get('digest')
+    api_digest = assets[package].get('digest')
     if api_digest and api_digest != 'sha256:' + digest:
         raise ValueError('GitHub asset digest mismatch')
     import io
@@ -291,7 +308,7 @@ def main(request=None):
     ttk.Label(win, text='Updates — Update downloads, then closes and restarts the game.').pack(pady=8)
     choice = ttk.Combobox(win, textvariable=channel, values=('Stable', 'Latest'), state='readonly')
     choice.pack()
-    ttk.Label(win, text='Stable: published releases. Latest: includes experimental prereleases.').pack()
+    ttk.Label(win, text='Stable: published releases. Latest: rolling Experimental and newer releases.').pack()
     notes = tk.Text(win, wrap='word', height=19)
     notes.pack(fill='both', expand=True, padx=12, pady=8)
     notes.configure(state='disabled')