diff --git a/README.md b/README.md index 67ba67a..5e45002 100644 --- a/README.md +++ b/README.md @@ -12,13 +12,13 @@ Validated: - The registration chain ran live against Apple (2026-10-02→04, spec rev 26): Albert activation, the packed APNs courier, the full GSA sign-in (SRP, trusted-device 2FA, PET delivery, postdata), then IDS authenticate and register — both green, `mailto:` handle active. The earlier 409 refusals turned out to be Apple's validation-stamp replay guard (one mint = one device record), documented in the spec. - Two live unauthenticated probes against Apple succeeded: both bag endpoints return plists this stack parses, and certificate pinning validates Apple's real TLS chains. A drift check ships with the repo (`:engine:runProbe`). - The courier's auth model is settled: the packed path needs no client certificate; the legacy path answers the handshake with a TLS 1.3 post-handshake `certificate_required` alert. -- The ADI anisette crypto is reimplemented natively, no emulation or helper binary: the white-box OTP ciphers, the GSA provisioning exchange, and the spim/cpim/ptm wire framing are transcribed clean-room from analysis of a compiled ADI implementation and verified byte-exact against oracle vectors (192 tests green, spec rev 27). No anisette server, no Unicorn, no Mac relay. +- The ADI anisette crypto is reimplemented natively, no emulation or helper binary: the white-box OTP ciphers, the GSA provisioning exchange, and the spim/cpim/ptm wire framing are transcribed clean-room from analysis of a compiled ADI implementation and verified byte-exact against oracle vectors, then live-confirmed against a genuine client's provisioning run under an instrumented debugger (194 tests green, spec rev 28). No anisette server, no Unicorn, no Mac relay. Not validated yet: - No real message send or receive over the courier; renewal across sessions is untested. - The crypto envelopes have round-trip tests but no known-answer vectors from a live device. -- The ADI provisioning exchange has not yet run against Apple's live servers; two wire details (two cpim signature fields, the provisioned-state byte mapping) await one instrumented observation of a genuine client and are `TODO(capture)` in the code and spec. +- The ADI provisioning wire shapes were confirmed from an instrumented genuine client (spec rev 28): the provisioned-state mapping (metadata/client-secret/mid/flavor), the PTM layout with its embedded decrypt seed, the 276-byte cpim outer message, and the gate echo position are all live-pinned. Three byte groups remain `TODO(capture)`: the 32-byte cpim tail group, the 12-byte plaintext tail, and the spim trailing's first 16 bytes. - A few wire details are capture-bound guesses, tracked in the spec's open-questions checklist. - PQ3 is unsupported. diff --git a/docs/imessage-protocol-spec.md b/docs/imessage-protocol-spec.md index b6ad4ce..d07d3d7 100644 --- a/docs/imessage-protocol-spec.md +++ b/docs/imessage-protocol-spec.md @@ -1,6 +1,6 @@ # iMessage protocol spec (clean-room) -**Status:** draft — revision 26. The full revision history is at the bottom ([Revision history](#revision-history)); the sections below state the facts as of rev 26. +**Status:** draft — revision 28. The full revision history is at the bottom ([Revision history](#revision-history)); the sections below state the facts as of rev 28. **Firewall rule:** this document contains facts only — no code, no function/type names, no copied expression from any reference implementation. Wire-format field names are quoted as on-the-wire protocol data (observable in captures, capture-arbitrated), never as source expression. Facts derived from SSPL/AGPL sources are used facts-only; nothing from them is reproduced. Where the source-derived fact sets agree, the fact is stated once with both citations; where they disagree, the disagreement is stated and tagged `TODO(capture):` — running mitmproxy captures arbitrate. All `TODO(capture)` items are collected in §7. @@ -21,7 +21,7 @@ - Current shipped flow (OpenBubbles, 2026): the Mac-side helper (`Mac-Hardware-Info` 1.2.0, notarized) only *reads* the machine identifiers — serial, board ID, model, etc. — via public IORegistry/sysctl APIs (docs: "High Sierra and above", no upper macOS version bound) and exports them as a shareable code (≤ 20 users per code, optional one-time-use/48-hour mode); it never invokes an Apple private framework [OB-MHI], [OB-FAQ]. The client then performs the three-step exchange above itself, in software, keyed with those identifiers and paired with an Apple-Account login — the [JJ] emulation path with an official identifier-export helper in front of it. `TODO(capture):` **C56** — the client-side generation procedure behind the three-step exchange: what the client computes with the `id-initialize-validation` session info (the key-establishment math and blob assembly), how it interacts with the Apple-Account/GSA login and the anisette header set (C6) at activation time, and which pieces are fixed protocol constants vs per-account/per-session state. — 2026-09-10: steps 1–2 of C56 are recorded (bullet above); step 3 — key establishment + signing — is closed source: the public OpenAbsinthe-Stub repo is an official mock whose methods are stubs (its README states the real repository is closed source), so the spec records only the API contract: context creation consumes the certificate chain + hardware configuration and outputs the session-info-request blob; key establishment consumes the server response; signing produces the final blob with no further inputs (all state retained in the context) [RP:macos.rs], [RP:activation.rs], [RP:auth.rs], [OA]. Remaining: wire-verification of the blob against one live activation; the enc-variant derivation rides C57. — 2026-09-11 (rev 14 extraction): the five obfuscation labels are now MAPPED to their fields (previously only listed together): serial → `Gq3489ugfi`, platform UUID → `Fyp98tpgj`, root-disk UUID → `kbjfrfpoJU`, ROM → `oycqAZloTNDm`, MLB → `abKPld1EcMni` [OA:nac.rs]; the derivation producing each `_enc` byte variant is still closed source. The hardware-configuration field set is recorded in full: product name (Apple product-type form, e.g. `Macmini9,1`), MAC address (exactly 6 raw bytes), platform serial, platform UUID, root-disk UUID, board ID, OS build number, ROM (raw bytes), MLB, plus the five `_enc` raw-byte variants [OA:nac.rs]. Step 3's closure is confirmed at the source level: the crate holding the key-establishment math ships as a closed-source stub whose mock methods panic; only the API contract is observable [OA:README.md], [OA:nac.rs]. - The helper's export-code format (2026-09-11, rev 15 — **C58 closed** by facts-only extraction from the helper's public upstream repo, which is unlicensed — facts only): the payload is a 4-byte ASCII magic `OABS`, a one-byte flag (1 = "prevent sharing" mode, 0 otherwise), then a raw protobuf message (no length prefix; protobuf starts at offset 5). Protobuf field map: outer — 1 nested identifier message, 2 macOS product version (sysctl `kern.osproductversion`), 3 protocol version (hardcoded `1640`), 4 device id (the IOPlatformUUID again), 5 a hardcoded iCloudHelper user-agent string, 6 a hardcoded AOSKit version string; nested identifier message — 1 product name (e.g. `MacBookPro18,3`), 2 MAC address (6 raw bytes), 3 platform serial, 4 platform UUID, 5 root-disk UUID, 6 board ID (Apple Silicon: literal prefix `Mac-` + lowercase hex of the device-tree board-id bytes), 7 OS build number (sysctl `kern.osversion`), 8/9/10/12/14 the five `_enc` obfuscated blobs read from the `IOPower:/` registry under Apple's key names (`Gq3489ugfi` serial, `Fyp98tpgj` platform UUID, `kbjfrfpoJU` root-disk UUID, `oycqAZloTNDm` ROM, `abKPld1EcMni` MLB — matching §1.1's label mapping), 11 ROM (Intel: the device-tree options ROM value; Apple Silicon: last 6 bytes of SHA-256 of `/chosen` `unique-chip-id`), 13 MLB string. Two delivery modes: plain = standard base64 of the payload (clipboard text, or the raw payload bytes as a QR code at medium error correction); restricted = AES-256-CBC/PKCS7 over the payload, ciphertext prefixed with ASCII `Salted__` + 8-byte random salt (legacy OpenSSL `enc` container), key/IV derived by the legacy EVP_BytesToKey KDF (repeated MD5 of previous-hash ‖ password ‖ salt to 48 bytes: first 32 = key, next 16 = IV) keyed by a human activation code (`MB` + 4×4 chars from the alphabet `ABCDEFGHJKLMNPQRSTUVWXYZ123456789`), then base64. The restricted mode additionally uploads the blob (JSON `data` + `id` = lowercase hex SHA-256 of the activation code) to the vendor's server (`hw.openbubbles.app/code`) — the ≤20-user/one-time/48-hour mechanics live SERVER-side; nothing is embedded in the payload (no checksum, signature, compression, or expiry). No public parser exists (the client apps are closed-source); the construction above is the parse specification [MHG:ViewController.swift], [MHG:HwIdentifiers.swift], [MHG:mac_hw_info.pb.swift]. - The validation blob is submitted as the `validation-data` field of the IDS register body and, base64, in the `X-Mme-Nas-Qualify` header of the iCloud account login [RP:ids/user.rs], [RP:auth.rs]. `TODO(capture):` C5 — exact byte shape of the blob a genuine Mac produces, and the current exchange over the two validation bag URLs. — 2026-10-04 (rev 26, live): **REPLAY GUARD — one mint = one device record.** Apple keys a validation blob's consumption server-side: re-submitting an already-consumed blob at the sign-in/register pair answers HTTP 409, which is a REPLAY refusal, NOT a blob rejection — a fresh blob under the same device identity succeeds where the reused one failed, and a captured blob of external provenance succeeds outright (session 17: a captured stamp replaced the local mint and the 409 vanished). Client consequences: mint or fetch a fresh blob per registration attempt (the 15-minute validity above makes stale reuse doubly fatal), and mint the sign-in's blob AT SIGN-IN TIME — never a login-start blob carried across the 2FA wait. A caller-injected blob from any source (local mint, relay Mac, staged capture) is a legitimate stamp; the injection points are the register body's `validation-data` and the sign-in's `X-Mme-Nas-Qualify` header. -- Hardware-related anisette headers seen in activation flows: `X-Apple-I-SRL-NO` (serial), `X-Apple-I-MLB` (logic-board serial), `X-Apple-I-ROM` (hex ROM, lowercase) [RP:relay.rs], [APA:anisette]. `TODO(capture):` C6 — the full anisette header set a current client actually sends, including which optional hardware headers appear. — 2026-09-10: the header-set half of C6 is closed by facts-only extraction: per-request anisette headers `X-Apple-I-Client-Time`, `X-Apple-I-TimeZone`, `X-Apple-Locale`, `X-Apple-I-MD-RINFO`, `X-Mme-Device-Id`, `X-Apple-I-MD` (base64 OTP), `X-Apple-I-MD-M` (base64 machine ID), plus per-platform hardware headers — macOS: `X-Apple-I-MLB`, `X-Apple-I-ROM` (lowercase hex), `X-Apple-I-SRL-NO` [APA:omnisette], [APA:icloud-auth], [RP:relay.rs]. They are produced by an ADI provisioning exchange with gsa.apple.com (start/finish provisioning returning machine tokens), then a local OTP [APA:omnisette]. GSA context (cross-referenced with C6): the Albert activation itself needs NO Apple-Account login — only the FairPlay-signed activation-info plist (§1.2); the GSA login (full Apple-ID password, SRP over gsa.apple.com/grandslam/GsService2, s2k/s2k_fo PBKDF2, 2FA-capable) feeds the delegate logins that yield the IDS `auth-token`/`profile-id` and the PET token [RP:auth.rs], [APA:icloud-auth]. The extraction confirms the validation blob rides base64 in the `X-Mme-Nas-Qualify` header of the iCloud sign-in POST (recorded above) [RP:auth.rs]. — 2026-09-11 (rev 14 extraction round over the anisette crates): the value-format and provisioning halves of C6 are closed. Per-request value forms: `X-Apple-I-Client-Time` = UTC now truncated to whole seconds, RFC 3339/ISO-8601 with the offset suffix replaced by `Z` (provisioning requests send the raw offset form without the Z — a recorded inconsistency between code paths); `X-Apple-I-TimeZone` = literal `UTC` (not the device zone); `X-Apple-Locale` = `en_US` hardcoded (a companion `X-Apple-I-Locale: en_US` variant exists on the browser-style header set); `X-Apple-I-MD-RINFO` = verbatim passthrough of the string returned in the finish-provisioning response; `X-Mme-Device-Id` = lowercase hyphenated UUID string of 16 locally random bytes (`keychain_identifier`); `X-Apple-I-MD-M` = base64 (standard, padded) of the 60-byte `mid`; `X-Apple-I-MD-LU` = lowercase hex (no dashes) SHA-256 over the same 16 bytes (real macOS instead uses AuthKit's local-user UUID) — present in some header sets, omitted in others (disagreement — C57). Header sets are cached 60 s; a fresh OTP is generated per cache miss. The ADI provisioning flow: `GET https://gsa.apple.com/grandslam/GsService2/lookup` → XML plist whose `urls` keys `midStartProvisioning`/`midFinishProvisioning` give the endpoint URLs (never hardcoded); POST to the start URL with an XML plist body of two empty top-level dicts `Header` and `Request` → response `Response` → `spim` (base64); a local session is built from `spim` (+ empty hostuuid, DSID `-2`, flavor `Mac`=1 / `IOS`=3, flavor chosen by whether the client-info string contains `iPhone OS`) yielding `cpim` (base64); POST to the finish URL with `Request` = {`cpim`} → response `Response` → `ptm`, `tk` (both base64) and `X-Apple-I-MD-RINFO`; finishing the session with the decoded `tk`/`ptm` yields the machine state: `client_secret` (32 bytes), `mid` (60 bytes, Apple-issued at provisioning — reused for every OTP until re-provisioning, which happens only on a not-provisioned failure, e.g. ADI error -45061), `metadata` (variable blob). Provisioning HTTP specifics: `Content-Type: application/x-www-form-urlencoded` despite the XML-plist body (intentional), title-cased header names forced, HTTP/1.1, Apple root CA pinned (embedded DER), and a fixed companion header set on all three requests: akd `User-Agent`, `X-Apple-Baa-E: -10000`, `X-Apple-Baa-Avail: 2`, `X-Apple-Baa-UE: AKAuthenticationError:-7066|com.apple.devicecheck.error.baa:-10000`, `X-Apple-Host-Baa-E: -7066`, `X-Apple-Client-App-Name: akd`, `Accept: */*`, `Accept-Language: en-US,en;q=0.9`, plus `X-Apple-I-MD-LU`, `X-Mme-Device-Id`, `X-Mme-Client-Info`, `X-Apple-I-Client-Time` and any caller-supplied hardware headers. State persists as an XML plist with `keychain_identifier` (data) + the provisioned fields (`client_secret`, `mid`, `metadata`, `rinfo`, `flavor`) [APA:omnisette]. CLOSED-SOURCE RESIDUE: the OTP (`X-Apple-I-MD`) and trusted-device 2FA-code derivations live inside a closed-source crate — its public repo is a stub with `todo!()` methods; recorded is only the call contract: inputs (32-byte `client_secret`, 60-byte `mid`, `metadata`, flavor) → raw OTP bytes (base64 into the header) / 32-bit 2FA code [OA-stub], [APA:omnisette]. — rides C57/C56 residue; the seam stays injectable in the code. — 2026-10-07 (rev 27, clean-room RE of the OpenBubbles ADI library — unstripped Linux `.so`, oracle-verified; the derivation is no longer closed residue): the OTP is fully local and pure. `gen = unix_seconds / 30`; dispatch by OTP flavor byte: 1 (IOS) → `gen_otp` (0x17fba60), 0 (Mac) → `gen_otp_ios` (0x1801d90) — the binary's naming is inverted relative to the flavor (a distinct numbering from the §1.1 session flavor `Mac`=1/`IOS`=3). Cipher inputs: `one` = the machine's `client_info` (required ≥ 0x190 B IOS / ≥ 0xd0 B Mac, length-checked) and `two` = the 32-byte `mid`; output 16 bytes. The wire `X-Apple-I-MD` frame is 28 B: `u32be(5)` ‖ `u32be(16)` ‖ sig16 ‖ `u32be(flavor)`. The 2FA code is `SHA-1(sig16 ‖ secret60) mod 1_000_000` over the machine's full 60-byte secret slice (`machine[0x38..0x74]`; two standard SHA-1 blocks — an earlier report's "48-byte" reading is wrong, 0x18013a2/0x18013bb/0x18013ef). Provisioning framing, asm-pinned: `spim` = `field1` u32 BE @0 ‖ payload-length u32 BE @4 ‖ payload @8 (payload must exceed 59 B, 0x17efb33; more than 23 trailing bytes rejected with error 0x41); `cpim` = 9 fields (u32be(32) ‖ `encsec(rng2)` 32 B ‖ u32be(32) ‖ rng1 32 B ‖ 60 B blob = caller-extra ‖ rng3 pad ‖ u64 BE of arg9 = −2 ‖ u32 BE of the byte-swapped LE dword at rng3[8..12] ‖ two 256-bit tail fields fed by `encrypt(session, rng16, data)` + `spim_map_signature` — the tail's exact wire assembly is `TODO(capture)`); `ptm` is lenient (u32 BE then the whole tail — every candidate shape accepted, error 0); finish requires `tk` exactly 16 B (error 0x8, 0x17f1f11), key16 = `encrypt(session, NULL iv, tk)` (0x17f1eac), decrypt block count = payload.size/16 (0x17f2548), yielding the 0x78-byte provisioned struct (+0x00 16 B, +0x10 u64, +0x18 u64, +0x20 16 B, +0x30..0x70 64 B metadata, +0x70 u32, +0x74 u8 flavor — which bytes are `client_secret` vs `mid` vs `metadata` is `TODO(capture)`). Kotlin: `registration` module `ClearAdiOtp`/`ClearAdiProvision`/`ClearAdiFraming`/`ClearAnisetteProvider`, 192 tests green incl. byte-exact oracle vectors. +- Hardware-related anisette headers seen in activation flows: `X-Apple-I-SRL-NO` (serial), `X-Apple-I-MLB` (logic-board serial), `X-Apple-I-ROM` (hex ROM, lowercase) [RP:relay.rs], [APA:anisette]. `TODO(capture):` C6 — the full anisette header set a current client actually sends, including which optional hardware headers appear. — 2026-09-10: the header-set half of C6 is closed by facts-only extraction: per-request anisette headers `X-Apple-I-Client-Time`, `X-Apple-I-TimeZone`, `X-Apple-Locale`, `X-Apple-I-MD-RINFO`, `X-Mme-Device-Id`, `X-Apple-I-MD` (base64 OTP), `X-Apple-I-MD-M` (base64 machine ID), plus per-platform hardware headers — macOS: `X-Apple-I-MLB`, `X-Apple-I-ROM` (lowercase hex), `X-Apple-I-SRL-NO` [APA:omnisette], [APA:icloud-auth], [RP:relay.rs]. They are produced by an ADI provisioning exchange with gsa.apple.com (start/finish provisioning returning machine tokens), then a local OTP [APA:omnisette]. GSA context (cross-referenced with C6): the Albert activation itself needs NO Apple-Account login — only the FairPlay-signed activation-info plist (§1.2); the GSA login (full Apple-ID password, SRP over gsa.apple.com/grandslam/GsService2, s2k/s2k_fo PBKDF2, 2FA-capable) feeds the delegate logins that yield the IDS `auth-token`/`profile-id` and the PET token [RP:auth.rs], [APA:icloud-auth]. The extraction confirms the validation blob rides base64 in the `X-Mme-Nas-Qualify` header of the iCloud sign-in POST (recorded above) [RP:auth.rs]. — 2026-09-11 (rev 14 extraction round over the anisette crates): the value-format and provisioning halves of C6 are closed. Per-request value forms: `X-Apple-I-Client-Time` = UTC now truncated to whole seconds, RFC 3339/ISO-8601 with the offset suffix replaced by `Z` (provisioning requests send the raw offset form without the Z — a recorded inconsistency between code paths); `X-Apple-I-TimeZone` = literal `UTC` (not the device zone); `X-Apple-Locale` = `en_US` hardcoded (a companion `X-Apple-I-Locale: en_US` variant exists on the browser-style header set); `X-Apple-I-MD-RINFO` = verbatim passthrough of the string returned in the finish-provisioning response; `X-Mme-Device-Id` = lowercase hyphenated UUID string of 16 locally random bytes (`keychain_identifier`); `X-Apple-I-MD-M` = base64 (standard, padded) of the 60-byte `mid`; `X-Apple-I-MD-LU` = lowercase hex (no dashes) SHA-256 over the same 16 bytes (real macOS instead uses AuthKit's local-user UUID) — present in some header sets, omitted in others (disagreement — C57). Header sets are cached 60 s; a fresh OTP is generated per cache miss. The ADI provisioning flow: `GET https://gsa.apple.com/grandslam/GsService2/lookup` → XML plist whose `urls` keys `midStartProvisioning`/`midFinishProvisioning` give the endpoint URLs (never hardcoded); POST to the start URL with an XML plist body of two empty top-level dicts `Header` and `Request` → response `Response` → `spim` (base64); a local session is built from `spim` (+ empty hostuuid, DSID `-2`, flavor `Mac`=1 / `IOS`=3, flavor chosen by whether the client-info string contains `iPhone OS`) yielding `cpim` (base64); POST to the finish URL with `Request` = {`cpim`} → response `Response` → `ptm`, `tk` (both base64) and `X-Apple-I-MD-RINFO`; finishing the session with the decoded `tk`/`ptm` yields the machine state: `client_secret` (32 bytes), `mid` (60 bytes, Apple-issued at provisioning — reused for every OTP until re-provisioning, which happens only on a not-provisioned failure, e.g. ADI error -45061), `metadata` (variable blob). Provisioning HTTP specifics: `Content-Type: application/x-www-form-urlencoded` despite the XML-plist body (intentional), title-cased header names forced, HTTP/1.1, Apple root CA pinned (embedded DER), and a fixed companion header set on all three requests: akd `User-Agent`, `X-Apple-Baa-E: -10000`, `X-Apple-Baa-Avail: 2`, `X-Apple-Baa-UE: AKAuthenticationError:-7066|com.apple.devicecheck.error.baa:-10000`, `X-Apple-Host-Baa-E: -7066`, `X-Apple-Client-App-Name: akd`, `Accept: */*`, `Accept-Language: en-US,en;q=0.9`, plus `X-Apple-I-MD-LU`, `X-Mme-Device-Id`, `X-Mme-Client-Info`, `X-Apple-I-Client-Time` and any caller-supplied hardware headers. State persists as an XML plist with `keychain_identifier` (data) + the provisioned fields (`client_secret`, `mid`, `metadata`, `rinfo`, `flavor`) [APA:omnisette]. CLOSED-SOURCE RESIDUE: the OTP (`X-Apple-I-MD`) and trusted-device 2FA-code derivations live inside a closed-source crate — its public repo is a stub with `todo!()` methods; recorded is only the call contract: inputs (32-byte `client_secret`, 60-byte `mid`, `metadata`, flavor) → raw OTP bytes (base64 into the header) / 32-bit 2FA code [OA-stub], [APA:omnisette]. — rides C57/C56 residue; the seam stays injectable in the code. — 2026-10-07 (rev 27, clean-room RE of the OpenBubbles ADI library — unstripped Linux `.so`, oracle-verified; the derivation is no longer closed residue): the OTP is fully local and pure. `gen = unix_seconds / 30`; dispatch by OTP flavor byte: 1 (IOS) → `gen_otp` (0x17fba60), 0 (Mac) → `gen_otp_ios` (0x1801d90) — the binary's naming is inverted relative to the flavor (a distinct numbering from the §1.1 session flavor `Mac`=1/`IOS`=3). Cipher inputs: `one` = the machine's `client_info` (required ≥ 0x190 B IOS / ≥ 0xd0 B Mac, length-checked) and `two` = the 32-byte `mid`; output 16 bytes. The wire `X-Apple-I-MD` frame is 28 B: `u32be(5)` ‖ `u32be(16)` ‖ sig16 ‖ `u32be(flavor)`. The 2FA code is `SHA-1(sig16 ‖ secret60) mod 1_000_000` over the machine's full 60-byte secret slice (`machine[0x38..0x74]`; two standard SHA-1 blocks — an earlier report's "48-byte" reading is wrong, 0x18013a2/0x18013bb/0x18013ef). Provisioning framing, asm-pinned: `spim` = `field1` u32 BE @0 ‖ payload-length u32 BE @4 ‖ payload @8 (payload must exceed 59 B, 0x17efb33; more than 23 trailing bytes rejected with error 0x41); `cpim` = 9 fields (u32be(32) ‖ `encsec(rng2)` 32 B ‖ u32be(32) ‖ rng1 32 B ‖ 60 B blob = caller-extra ‖ rng3 pad ‖ u64 BE of arg9 = −2 ‖ u32 BE of the byte-swapped LE dword at rng3[8..12] ‖ two 256-bit tail fields fed by `encrypt(session, rng16, data)` + `spim_map_signature` — the tail's exact wire assembly is `TODO(capture)`); `ptm` is lenient (u32 BE then the whole tail — every candidate shape accepted, error 0); finish requires `tk` exactly 16 B (error 0x8, 0x17f1f11), key16 = `encrypt(session, NULL iv, tk)` (0x17f1eac), decrypt block count = payload.size/16 (0x17f2548), yielding the 0x78-byte provisioned struct (+0x00 16 B, +0x10 u64, +0x18 u64, +0x20 16 B, +0x30..0x70 64 B metadata, +0x70 u32, +0x74 u8 flavor — which bytes are `client_secret` vs `mid` vs `metadata` is `TODO(capture)`). Kotlin: `registration` module `ClearAdiOtp`/`ClearAdiProvision`/`ClearAdiFraming`/`ClearAnisetteProvider`, 192 tests green incl. byte-exact oracle vectors. — 2026-10-07 (rev 28, live capture: one real OpenBubbles provisioning run under gdb, `registration/dev/clearadi/captures/WIRE-NOTES.md`; the framer is corrected): `spim` trailing is NOT capped at 23 B — the live spim carries 99 trailing bytes and parses fine (the earlier 0x41 pin was a probe-harness artifact); the trailing is server material the client relays. `cpim` = an OUTER 276-B message `[u32be(5)][iv 16][u32be(160)][encout 160][X 32][u32be(52)][u32be(4)][Y 52]` whose 160-B encrypt plaintext = `u32be(32) ‖ encsec(rng2) ‖ u32be(32) ‖ rng1 ‖ 60 B blob (caller-extra ‖ rng3 pad) ‖ u64 BE arg9 = −2 (live-confirmed) ‖ u32 BE unix-seconds ‖ u32be(1) ‖ 12 B (source TODO)`; the iv is 16 plain RNG bytes — the earlier 'IV assembled from the Spim's trailing fields' reading was actually the thread-RNG pool (words at r14+0x10, cursor at r14+0x110). `spim_map_signature` (0x17fe2a0) args are (rdi = in, rsi = out): the live call signs a constant 16-B block `e955a4f0…` → `f877c30e…`, and NEITHER that signature nor the SHA-256(encout) digest appears anywhere in the captured wire — the §6 'two 256-bit tail fields' reading is falsified. The `[u32be(52)][u32be(4)][Y 52]` group is the spim's trailing[16:76] relayed verbatim; X (32 B) = `TODO(capture)`. `ptm` = `[u32 BE 4][seed 16][u32 BE payloadLen][payload][tail]` (live 503 B: seed = ptm[4:20] feeds decryptPTM as its chain seed via rcx; block count = payloadLen/16 = 28); key16 = `encrypt(session, NULL iv, tk)` confirmed byte-exact against the live call. ProvisionedData = the 0x78-B Rust struct `{metadata: Vec (400 B live = decrypt out[4:404], length u32 BE at out[0]), client_secret = rng2 (32 B), mid = rng3 (60 B), flavor u8 (0x01 on a live Mac run — distinct from the OTP flavor numbering)}`; the finish gate is the decrypt output's **out[408:440] == rng1 echo** (NOT out[0:32]); the OTP-path report's ProvisionedMachine note (mid[32]@+0x18, secret[60]@+0x38) had the two swapped. Kotlin `ClearAdiFraming`/`ClearAnisetteProvider` restructured to the live shapes; 194 tests green. Remaining `TODO(capture)`: the X32 source, the 12-B plaintext tail, trailing[0:16]'s role. ### 1.2 Activation endpoints and certificates @@ -331,6 +331,7 @@ Ordered by value. IDs match the inline `TODO(capture):` markers above. ## Revision history +- Rev 28 (2026-10-07, live capture — one real OpenBubbles provisioning run under gdb over the unstripped Linux `.so`; dumps + method in `registration/dev/clearadi/captures/WIRE-NOTES.md`): the §1.1 anisette framer is corrected on every point the previous session left `TODO(capture)`. The cpim is an OUTER 276-B message `[u32be(5)][iv 16][u32be(160)][encout 160][X 32][u32be(52)][u32be(4)][Y 52]` over a 160-B plaintext; the PTM is `[u32 BE 4][seed 16][u32 BE payloadLen][payload]` with the seed riding into decryptPTM as its chain seed; ProvisionedData = `{metadata Vec (400 B), client_secret = rng2 (32 B), mid = rng3 (60 B), flavor u8}` with the finish gate = the decrypt output's out[408:440] == rng1 echo; the spim carries 99 trailing bytes (the ≤23-B cap was a probe artifact) and trailing[16:76] is relayed verbatim as the wire tail's `Y` group. Superseded readings recorded dead: the 'IV from Spim trailing fields' (it is the thread-RNG pool), the 'two 256-bit sig/sha tail fields' (neither the spim_map_signature output nor the SHA-256 digest appears on the wire), and the OTP-path ProvisionedMachine mid/secret ordering. Remaining `TODO(capture)`: the X32 source, the 12-B plaintext tail, trailing[0:16]. Kotlin `ClearAdiFraming`/`ClearAnisetteProvider` restructured to the live shapes; 194 tests green. - Rev 26 (2026-10-02→04, live registration of this protocol family against real Apple endpoints — Albert activation, APNs courier (packed ALPN), the full GSA chain through 2FA, and IDS authenticate + register all green; user-gated with a real Apple ID, `mailto:` handle ACTIVE): six live-verified fact groups beyond the extraction record. (1) **The 409's root cause is the validation-stamp REPLAY guard (§1.1)** — one mint = one device record: a consumed blob re-submitted answers HTTP 409 `ICLOUD_UNSUPPORTED_DEVICE`, a fresh (or captured-elsewhere) blob succeeds; the earlier readings (request shape, blob provenance) are dead. §1.2 authenticate + §1.4 register completed green after the stamp route was fixed, with the `mailto:` handle active (§1.3/§1.4). (2) **The courier's packed format is live (§3.1/§3.2, C19 CLOSED)** — the server selects `apns-pack-v1:4096:4096` and speaks the tag-based packed codec (full wire shape recorded from the wire); its chain anchors at the `Apple Server Authentication CA` intermediate, no TLS client certificate is demanded on that path, and the post-connect set-state + filter sequence is mandatory (unfiltered connections are dropped within seconds, invalidating the connect token). (3) **Apple's live GSA sign-in response is kebab-case** — delegate `service-data`/`status-message` (camelCase recordings become the fallback), and the sign-in body's client id rides `client-id` (§1.5). (4) **Anisette freshness is per-run and per-call** — fresh per SRP run incl. the post-2FA re-run (stale → `ec` −22421), fresh per 2FA call (frozen set → HTTP 434); the trigger/validate GETs carry the plist content type, and failed verifies name the real failure in the body plist's `Status.ec`/`em` (§1.5). (5) **The GSA `postdata` liveness event is recorded in full (§1.5)** — services `icloud`/`imessage`/`facetime`, `X-Apple-I-UrlSwitch-Info`/`X-Apple-HB-Token` binding, `ptkn` (−800012 with a dead push token — the courier connection must stay open across the registration), and the order sign-in FIRST, postdata AFTER (correcting the earlier reversed note). (6) **Albert activation refinements (§1.2)** — `DeviceCertRequest` is ``(base64(PEM)), DER rejected; the accepted `FairPlayCertChain` is leaf + FairPlay CA + Apple Root CA concatenated; the response's `` block may arrive wrapped in a larger `` (parse the span); refusals state their reason in wrapped-UI page text. Code side: the packed courier codec + ALPN negotiation, `GsaPostdata`, the kebab-case parsing, the per-run/per-call anisette providers, the ``-span extraction, and the three-cert FairPlay chain all landed; the validation stamp stays caller-injected with the replay guard documented. - Rev 25 (2026-09-29, first LIVE GSA login of this protocol family — SRP init/complete, both 2FA steps, and the PET-bearing re-login against `gsa.apple.com` with a real Apple ID, user-gated; the delegate sign-in POST was attempted and refused): the §1.5 chain is live-confirmed end-to-end up to the delegate sign-in, with five behavioral facts beyond the extraction record. (1) `au` (`trustedDeviceSecondaryAuth` observed) rides INSIDE the complete response's `Status` dict — reading it at the `Response` level silently turns a 2FA login into a "logged in, no PET" dead end (§1.5 2FA). (2) The PET's second live-confirmed delivery channel: the post-2FA re-login's decrypted `spd` token map `t` (21 service tokens observed incl. `com.apple.gs.idms.pet`), while a no-2FA login's `spd` carries only `com.apple.gs.appleid.auth` and no PET — an earlier implementation inference that a no-2FA login suffices for the PET is dead; 2FA is mandatory for the PET on this account class (§1.5 PET). (3) The trusted-device trigger creates the code on the devices and must precede the prompt; triggering twice before one validate invalidates the outstanding code and re-prompts (§1.5 2FA). (4) Anisette is one-time per request: headers minted at flow start are spent by the GSA hops, and the delegate sign-in with a reused set is refused — mint fresh per POST, matching the reference client's per-request provider (§1.5 delegate logins). Post-login ADI behavior confirmed live: after an account login the machine re-provisions for the personal DSID (GetLoginCode → not-provisioned → full exchange succeeds), which is the personal-`X-Apple-I-MD-M` path, not a defect. (5) The delegate sign-in answered HTTP 409 `ICLOUD_UNSUPPORTED_DEVICE` with the recorded error-plist shape (§1.5 delegate logins) — a device-identity-bound refusal, the current open blocker. Velocity note: a persisted-password-hash re-login was rejected once with -22406 ("enter the correct password") after four full logins in 30 minutes, with the identical hash derivation succeeding twice earlier in the same window — treat as server-side velocity throttling, not a derivation defect. - Rev 24 (2026-09-28, first live probes of this implementation — two unauthenticated bag GETs + a no-client-cert courier TLS handshake, runner `:engine:runProbe`; nothing else touched): (1) C18 SETTLED — the courier demands the Albert client certificate via a fatal TLS 1.3 `certificate_required` alert after a successful handshake, on both ports (§3.1); (2) the bags are TYPED CONFIG DICTIONARIES, not URL maps — the live IDS bag carries ints/bools/floats/string-lists alongside 400+ URLs and a `bag-expiry-timestamp` (~17-day observed TTL; 403 entries live vs 405 on 2026-09-10, two new push-debounce keys in, four A/B-test keys out); the APNs bag is 65 stable entries incl. BAA-authentication flags (`APNSEnableBAAAuthenticationForDevices` etc. — C17/C20-relevant); (3) AppleTrust's embedded-root pinning validates the live chains of `init.ess.apple.com` and `courier.push.apple.com` (no root served — chain anchored at the embedded root); (4) rev-23's container-fact follow-ups: sort-version 0 and the ref-width rule are now capture/probe-consistent. diff --git a/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAdiFraming.kt b/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAdiFraming.kt index 9c9a8b0..df44e79 100644 --- a/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAdiFraming.kt +++ b/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAdiFraming.kt @@ -22,23 +22,23 @@ class SpimFormatException(message: String) : Exception(message) * (17efb82). * * The asm continues with trailing reads ([T;N] 17eee41, `read_bytes_const` 17eeecd — the - * 0x20 size setup is at 17ef00b — and two u32 reads 17ef05f/17ef3e8). Empirically pinned - * against the live library: a Spim whose bytes after the payload are ≤ 23 parses fine and - * ≥ 24 fails with error 0x41 regardless of the payload length (60…256 tested). Their exact - * semantics are TODO(live-capture). + * 0x20 size setup is at 17ef00b — and two u32 reads 17ef05f/17ef3e8). The live capture + * (session 19f) shows a real Spim carrying **99 trailing bytes** that parses fine — the + * earlier "≤ 23 leftover" pin (error 0x41) was an artifact of the probe harness, not the + * reader. The trailing bytes are server material the client relays verbatim into the Cpim + * wire tail (see [Cpim]) and their leading words feed the encrypt IV selection pool. */ data class Spim( val field1: Int, val payloadLength: Int, val payload: ByteArray, + val trailing: ByteArray, ) { companion object { /** Minimum reader requirement (`cmp rdx,0x3; ja` 17eec6c): the first u32 must fit. */ private const val MIN_LEN = 4 /** `cmp QWORD PTR […],0x3b; ja` 17efb33 — the payload Vec must exceed 59 bytes. */ private const val MIN_PAYLOAD = 60 - /** Live-pinned leftover tolerance: > 23 trailing bytes ⇒ error 0x41. TODO(live-capture). */ - private const val MAX_LEFTOVER = 23 fun parse(bytes: ByteArray): Spim { if (bytes.size < MIN_LEN) throw SpimFormatException("spim shorter than the first u32 (${bytes.size} B)") @@ -52,11 +52,8 @@ data class Spim( throw SpimFormatException("spim payload of $payloadLength B rejected (must exceed 59 B, 17efb33)") } val payload = bytes.copyOfRange(8, 8 + payloadLength) - val leftover = bytes.size - 8 - payloadLength - if (leftover > MAX_LEFTOVER) { - throw SpimFormatException("spim has $leftover trailing bytes; the live binary tolerates ≤ 23 (error 0x41)") - } - return Spim(field1, payloadLength, payload) + val trailing = bytes.copyOfRange(8 + payloadLength, bytes.size) + return Spim(field1, payloadLength, payload, trailing) } private fun beU32(b: ByteArray, off: Int): Int = @@ -75,43 +72,54 @@ fun interface ClearAdiRng { } /** - * The two trailing 256-bit Cpim fields (provisioning.md §6 items 8–9, written as deku - * `write_bits(0x100)` calls at 17effc9/17f0338/17f0444). The asm shows the material they are - * derived from — `spim_map_signature` over a zeroed 16-byte block (17f0914–17f0931) and a - * SHA-256 over the `encrypt(session, iv16, payload)` output, where the IV is assembled from - * the parsed Spim's trailing fields (17f0848–17f0888) — but their exact wire assembly is not - * provable from the dump alone: TODO(live-capture). The production seal is wired after that - * capture; tests seal deterministically. + * The Cpim wire tail — the 92 bytes after `[u32 BE 5][iv][u32 BE len][encout]` in the outer + * message (live capture, session 19f: `cap_wiravec.bin`). Shape: `[X 32][u32 BE 52][u32 BE 4] + * [Y 52]`, where the `[52][4][Y52]` group == the Spim's trailing[16:76] relayed verbatim + * (server material) and **X (32 B) is not yet source-pinned** — TODO(live-capture) with a + * second wire sample. The earlier §6 reading (two 256-bit sig/sha fields) is falsified: the + * `spim_map_signature` output and the SHA-256 digest appear nowhere in the captured wire. + * Tests seal deterministically; production wires in once X is named. */ fun interface CpimTailSeal { /** - * Returns the pair of 32-byte Cpim tail fields. [sessionKey] is the 16-byte session key, - * [encryptedPayload] the `encrypt(session, iv, payload)` output, [signatureOfZeroBlock] - * the `spimMapSignature(ByteArray(16))` output, [sha256OfEncrypted] the SHA-256 digest of - * [encryptedPayload] — everything the asm shows feeding the region. + * Returns the 92-byte wire tail. [spimTrailing] is the Spim's trailing byte block (the + * relayed `[52][4][Y52]` group comes from its bytes 16..76), [encryptedPayload] the + * `encrypt(sessionCtx, iv, plaintext)` output and [iv] the encrypt IV. */ - fun seal( - sessionKey: ByteArray, - encryptedPayload: ByteArray, - signatureOfZeroBlock: ByteArray, - sha256OfEncrypted: ByteArray, - ): Pair + fun seal(spimTrailing: ByteArray, encryptedPayload: ByteArray, iv: ByteArray): ByteArray } /** - * The Cpim wire bytes (`new()`'s deku Writer output, provisioning.md §6 items 1–9): + * The Cpim wire message (`request.cpim`, base64) — live-pinned from the session-19f capture + * (`cap_wiravec.bin`, 276 B; the earlier §6 208-B reading is superseded): * - * 1. `00 00 00 20` u32 BE (17efe9b) — the Vec length prefix of item 2 - * 2. `encsec(draw2)` (Vec#1, 32 B — encsec call 17efd5e on the second RNG draw) - * 3. `00 00 00 20` u32 BE — length prefix of item 4 - * 4. `draw1` raw (Vec#2, 32 B — the first RNG draw copied untouched, 17efc95) - * 5. the 60-byte blob: the caller's ≤ 60-byte `extra` slice, padded with the third RNG draw - * (the extra memcpy 17efb21 lands on the draw-3 slot, then the struct copy 17efcb5 takes - * exactly 60 bytes into the Cpim struct) - * 6. [gsaArg] as u64 BE — the omnisette reference passes −2 ("GSA"): `FF FF FF FF FF FF FF FE` - * 7. u32 BE — `bswap` of the little-endian dword at draw3[8..12] (17efc3c → struct +0x7c) - * 8–9. the two 256-bit [CpimTailSeal] fields — TODO(live-capture) + * `[u32 BE 5][iv 16][u32 BE 160][encout 160][CpimTailSeal 92]` + * + * `encout = encrypt(sessionCtx, iv, plaintext)` (17f090e), plaintext 160 B = + * `[u32 BE 32][vec1 32][u32 BE 32][vec2 32][blob 60][gsaArg u64 BE][ts u32 BE][u32 BE 1][tail 12]`: + * - vec1 = [ClearAdiProvision.encsec] of draw2 (the sealed client secret) + * - vec2 = draw1 raw — the 32-B gate material echoed back by the server's PTM + * - blob = the caller's ≤ 60-byte `extra` padded with draw3 (draw3 = the machine `mid`) + * - gsaArg = −2 (omnisette reference), wire `FF FF FF FF FF FF FF FE` — CONFIRMED live + * - ts = unix seconds (live: 0x6ac64ac3 ≈ provisioning wall clock) + * - the trailing `u32 BE 1` + 12 bytes are live-pinned in position; the 12 bytes' source is + * TODO(live-capture) (modeled as RNG here) + * - iv = 16 RNG bytes (17f0782–17f0888 draw them from the thread RNG pool — the earlier + * "IV assembled from the Spim's trailing fields" reading was the RNG pool, not the Spim) */ +data class CpimBuilt( + /** The full wire message — base64 it into `request.cpim`. */ + val wire: ByteArray, + /** RNG draw 1 — the 32-B gate material the server's PTM must echo (decrypt out[408:440]). */ + val draw1: ByteArray, + /** RNG draw 2 — becomes `client_secret`. */ + val draw2: ByteArray, + /** RNG draw 3 (60 B) — becomes the machine `mid`. */ + val draw3: ByteArray, + /** The 16-B encrypt IV, echoed in the clear at wire[4:20]. */ + val iv: ByteArray, +) + object Cpim { /** `extra` must fit the fixed 60-byte blob slot (`cmp rdx,0x3d; jae` → error, 17efb17). */ @@ -122,46 +130,44 @@ object Cpim { rng: ClearAdiRng, extra: ByteArray, gsaArg: Long, + nowSeconds: Long, sessionCtx: ByteArray, payload240: ByteArray, - tail: CpimTailSeal?, - ): ByteArray { + spimTrailing: ByteArray, + tail: CpimTailSeal, + ): CpimBuilt { require(extra.size <= MAX_EXTRA) { "extra blob of ${extra.size} B exceeds the 60 B Cpim slot (17efb17)" } val draw1 = rng.draw(32) val draw2 = rng.draw(32) val draw3 = rng.draw(60) + val iv = rng.draw(16) val vec1 = ClearAdiProvision.encsec(draw2) val blob = extra + draw3.copyOfRange(extra.size, BLOB_LEN) - val item7 = leU32(draw3, 8) - val out = ByteArray(4 + 32 + 4 + 32 + BLOB_LEN + 8 + 4 + 32 + 32) + val plain = ByteArray(4 + 32 + 4 + 32 + BLOB_LEN + 8 + 4 + 4 + 12) var p = 0 - p = putBeU32(out, p, 32) - System.arraycopy(vec1, 0, out, p, vec1.size); p += vec1.size - p = putBeU32(out, p, 32) - System.arraycopy(draw1, 0, out, p, draw1.size); p += draw1.size - System.arraycopy(blob, 0, out, p, blob.size); p += blob.size - p = putBeU64(out, p, gsaArg) - p = putBeU32(out, p, item7) - if (tail == null) { - throw IllegalStateException( - "Cpim tail fields (§6 items 8–9) need the live capture — no CpimTailSeal supplied (TODO(live-capture))", - ) - } - val iv16 = ByteArray(16) // IV assembled from the Spim's trailing fields — TODO(live-capture) - val encrypted = ClearAdiProvision.encrypt(sessionCtx, iv16, payload240) - val digest = java.security.MessageDigest.getInstance("SHA-256").digest(encrypted) - val signature = ClearAdiProvision.spimMapSignature(ByteArray(16)) - val (field8, field9) = tail.seal( - ClearAdiProvision.sessionKey(sessionCtx), - encrypted, - signature, - digest, - ) - require(field8.size == 32 && field9.size == 32) { "Cpim tail fields must be 32 B each" } - System.arraycopy(field8, 0, out, p, 32); p += 32 - System.arraycopy(field9, 0, out, p, 32); p += 32 - return out.copyOf(p) + p = putBeU32(plain, p, 32) + System.arraycopy(vec1, 0, plain, p, vec1.size); p += vec1.size + p = putBeU32(plain, p, 32) + System.arraycopy(draw1, 0, plain, p, draw1.size); p += draw1.size + System.arraycopy(blob, 0, plain, p, blob.size); p += blob.size + p = putBeU64(plain, p, gsaArg) + p = putBeU32(plain, p, (nowSeconds and 0xffffffffL).toInt()) + p = putBeU32(plain, p, 1) + System.arraycopy(rng.draw(12), 0, plain, p, 12); p += 12 + require(p == plain.size) + + val encrypted = ClearAdiProvision.encrypt(sessionCtx, iv, plain) + require(encrypted.size == plain.size) + val wire = ByteArray(4 + 16 + 4 + encrypted.size + 92) + var q = 0 + q = putBeU32(wire, q, 5) + System.arraycopy(iv, 0, wire, q, iv.size); q += iv.size + q = putBeU32(wire, q, encrypted.size) + System.arraycopy(encrypted, 0, wire, q, encrypted.size); q += encrypted.size + System.arraycopy(tail.seal(spimTrailing, encrypted, iv), 0, wire, q, 92); q += 92 + require(q == wire.size) + return CpimBuilt(wire, draw1, draw2, draw3, iv) } private fun putBeU32(b: ByteArray, off: Int, v: Int): Int { @@ -174,77 +180,84 @@ object Cpim { for (i in 0 until 8) b[off + i] = (v ushr (56 - 8 * i)).toByte() return off + 8 } - - private fun leU32(b: ByteArray, off: Int): Int = - (b[off].toInt() and 0xff) or ((b[off + 1].toInt() and 0xff) shl 8) or - ((b[off + 2].toInt() and 0xff) shl 16) or ((b[off + 3].toInt() and 0xff) shl 24) } /** - * The parsed Ptm (finish-provisioning `Response.ptm`, base64 on the wire). Same reader family - * as [Spim] (finish.asm 17f1f1e–17f2294): `field1` u32 BE first, then a lenient Vec (every - * candidate shape accepted by the live binary, error 0) whose bytes feed - * [ClearAdiProvision.decryptPtm] with `count = payload.size / 16` (17f2548). - * The live binary accepts every candidate shape thrown at it (u32 alone through 128 B tails, - * all error 0) — the trailing [T;N]/read_bytes_const/u32 reads (17f1f6a/17f1fd4/17f20e5) are - * lenient; their exact semantics TODO(live-capture). + * The parsed Ptm (finish-provisioning `Response.ptm`, base64 on the wire) — live-pinned from + * the session-19f capture (`cap_ptm.bin`, 503 B): the finish reader's u32/[T;N]/const-bytes/ + * u32/Vec sequence (17f1f1e–17f2294) resolves to + * `[u32 BE field1][seed 16][u32 BE payloadLen][payload payloadLen][trailing]` — live sample: + * field1=4, seed=ptm[4:20], payloadLen=448, payload=ptm[24:472]. The [seed] is the + * `decryptPTM` chain seed (finish 17f2557's rcx — confirmed live = ptm[4:20]) and [payload] + * feeds [ClearAdiProvision.decryptPtm] with `count = payload.size / 16` (17f2548). */ data class Ptm( val field1: Int, + val seed: ByteArray, val payload: ByteArray, + val trailing: ByteArray, ) { companion object { - /** Minimum reader requirement (finish `cmp …,0x3; ja` 17f1eec → 17f1f1e). */ - private const val MIN_LEN = 4 + /** u32 + seed + u32 (the live sample's field1 = 4; role of the leading u32 unpinned). */ + private const val HEADER = 24 fun parse(bytes: ByteArray): Ptm { - if (bytes.size < MIN_LEN) throw SpimFormatException("ptm shorter than the first u32 (${bytes.size} B)") - val field1 = ((bytes[0].toInt() and 0xff) shl 24) or ((bytes[1].toInt() and 0xff) shl 16) or - ((bytes[2].toInt() and 0xff) shl 8) or (bytes[3].toInt() and 0xff) - // Lenient Vec: takes the whole tail after the first u32 — the live binary accepts - // every candidate shape (u32 alone through 128 B tails, all error 0), so field1 - // is not enforced as a length (its role is TODO(live-capture)). - return Ptm(field1, bytes.copyOfRange(4, bytes.size)) + if (bytes.size < HEADER) throw SpimFormatException("ptm shorter than its 24-B header (${bytes.size} B)") + val field1 = beU32At(bytes, 0) + val seed = bytes.copyOfRange(4, 20) + val payloadLength = beU32At(bytes, 20) + if (24 + payloadLength > bytes.size) { + throw SpimFormatException("ptm payload Vec needs $payloadLength B, ${bytes.size - 24} available") + } + val payload = bytes.copyOfRange(24, 24 + payloadLength) + val trailing = bytes.copyOfRange(24 + payloadLength, bytes.size) + return Ptm(field1, seed, payload, trailing) } + + private fun beU32At(b: ByteArray, off: Int): Int = + ((b[off].toInt() and 0xff) shl 24) or ((b[off + 1].toInt() and 0xff) shl 16) or + ((b[off + 2].toInt() and 0xff) shl 8) or (b[off + 3].toInt() and 0xff) } } /** - * `finish()`'s return — the raw 0x78-byte ProvisionedData struct (17f2dec–17f2e56): - * +0x00 16 B, +0x10 u64, +0x18 u64, +0x20 16 B, +0x30..0x70 64 B (metadata blob), +0x70 u32, - * +0x74 u8 (flavor). Which bytes are `mid` vs `client_secret` is NOT provable from the dump - * (an OpenBubbles provisioning run under gdb is pending) — the cross-check against the - * OTP-path report's ProvisionedMachine (client_info 400/208 B, mid 32 B, secret 60 B, - * flavor) is likewise unconfirmed: TODO(live-capture) on the field mapping. Named accessors - * only; nothing is invented. + * `finish()`'s ProvisionedData — live-pinned from the session-19f capture (`cap_provisioned.bin`, + * 0x78 B at 17f2e5a): a Rust struct `{metadata: Vec, client_secret: [u8;32], mid: [u8;60], + * flavor: u8}` — NOT the decrypt output itself: + * - metadata = the decrypt output's Vec content (length = the u32 BE at out[0]; 400 B live) + * - client_secret = the session's RNG draw 2 (ctx+0x2910; live byte-verified) + * - mid = the session's RNG draw 3 (ctx+0x2930; live byte-verified) + * - the struct's flavor byte (ctx+0x296c = 0x01 in the live Mac run) uses its own numbering — + * distinct from the ClearAdiOtp flavor convention, so it is carried raw here. + * The finish gate (17f2cd7–17f2d0d, decrypt staging vs ctx+0x28f0) is the decrypt output's + * **out[408:440] == draw1** echo — the earlier "out[0:32]" reading was wrong. */ -data class ProvisionedData(val raw: ByteArray) { - init { - require(raw.size >= 0x78) { "ProvisionedData must be 0x78 bytes, got ${raw.size}" } - } +data class ProvisionedData( + val metadata: ByteArray, + val clientSecret: ByteArray, + val mid: ByteArray, +) { + companion object { + /** Gate echo position in the decrypt output (out[408:440] == RNG draw 1). */ + const val GATE_AT = 408 - /** +0x00, 16 B (role unproven). */ - val block00: ByteArray get() = raw.copyOfRange(0x00, 0x10) - /** +0x10, u64 LE (role unproven). */ - val word10: Long get() = le64(raw, 0x10) - /** +0x18, u64 LE (role unproven). */ - val word18: Long get() = le64(raw, 0x18) - /** +0x20, 16 B (role unproven). */ - val block20: ByteArray get() = raw.copyOfRange(0x20, 0x30) - /** +0x30..0x70, 64 B — the metadata blob per the OTP-path report's layout cross-check. */ - val metadata64: ByteArray get() = raw.copyOfRange(0x30, 0x70) - /** +0x70, u32 LE. */ - val word70: Int get() = le32(raw, 0x70) - /** +0x74, u8 — the flavor byte (Mac = 0, IOS = 1, per the ClearAdiOtp flavor convention). */ - val flavor: Int get() = raw[0x74].toInt() and 0xff + /** + * Assembles ProvisionedData from the [ClearAdiProvision.decryptPtm] output. [out] must + * carry `u32 BE metaLen` at 0, the metadata at 4, and the draw-1 echo at [GATE_AT] — + * a mismatch throws (the server's PTM did not echo our gate material). + */ + fun fromDecrypt(out: ByteArray, draw1: ByteArray, draw2: ByteArray, draw3: ByteArray): ProvisionedData { + require(out.size >= GATE_AT + 32 + 4) { "decryptPTM output too short: ${out.size} B" } + val metaLen = beU32(out, 0) + require(4 + metaLen <= out.size) { "decryptPTM metadata length $metaLen exceeds the output" } + require(out.copyOfRange(GATE_AT, GATE_AT + 32).contentEquals(draw1)) { + "PTM gate mismatch: the decrypted PTM does not echo the client gate material (17f2cd7)" + } + return ProvisionedData(out.copyOfRange(4, 4 + metaLen), draw2, draw3) + } - private fun le32(b: ByteArray, off: Int): Int = - (b[off].toInt() and 0xff) or ((b[off + 1].toInt() and 0xff) shl 8) or - ((b[off + 2].toInt() and 0xff) shl 16) or ((b[off + 3].toInt() and 0xff) shl 24) - - private fun le64(b: ByteArray, off: Int): Long { - var v = 0L - for (i in 7 downTo 0) v = (v shl 8) or (b[off + i].toLong() and 0xff) - return v + private fun beU32(b: ByteArray, off: Int): Int = + ((b[off].toInt() and 0xff) shl 24) or ((b[off + 1].toInt() and 0xff) shl 16) or + ((b[off + 2].toInt() and 0xff) shl 8) or (b[off + 3].toInt() and 0xff) } } diff --git a/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAnisetteProvider.kt b/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAnisetteProvider.kt index 6a51356..71af6f8 100644 --- a/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAnisetteProvider.kt +++ b/registration/src/main/kotlin/dev/fenn/imessage/registration/ClearAnisetteProvider.kt @@ -43,31 +43,11 @@ data class ClearAdiLoginInfo( ) /** - * The provisioning outcome: `finish()`'s raw ProvisionedData plus the response's routing info. - * The ProvisionedData→{client_secret, mid, metadata} mapping is TODO(live-capture) (one - * OpenBubbles provisioning run under gdb) — the raw 0x78 bytes are carried as-is and the - * mapping lives nowhere until then. + * The provisioning outcome: the machine tokens + the server's routing info, ready for + * [ClearAdiAnisette] and persistence. Live-pinned mapping (session 19f): client_secret = + * RNG draw 2, mid = RNG draw 3 (60 B), metadata = the PTM decrypt output's Vec (400 B live). */ -data class ClearAdiProvisionResult( - val provisionedData: ProvisionedData, - val rinfo: String, -) - -/** State JSON of [ClearAdiProvisionResult] (hex ProvisionedData + rinfo; mapping TODO). */ -fun resultToJson(result: ClearAdiProvisionResult): String = buildJsonObject { - put("provisioned_data", result.provisionedData.raw.joinToString("") { "%02x".format(it) }) - put("rinfo", result.rinfo) -}.toString() - -fun resultFromJson(json: String): ClearAdiProvisionResult { - val o = Json.parseToJsonElement(json).jsonObject - val hexData = o["provisioned_data"]!!.jsonPrimitive.content - require(hexData.length >= 0x78 * 2) { "stored provisioned_data shorter than 0x78 bytes" } - val raw = ByteArray(hexData.length / 2) { i -> - ((Character.digit(hexData[i * 2], 16) shl 4) or Character.digit(hexData[i * 2 + 1], 16)).toByte() - } - return ClearAdiProvisionResult(ProvisionedData(raw), o["rinfo"]!!.jsonPrimitive.content) -} +typealias ClearAdiProvisionResult = ProvisionedAnisette /** * The header factory over a provisioned machine: the real OTP derivation @@ -179,15 +159,12 @@ data class ProvisionedAnisette( * `Content-Type: application/x-www-form-urlencoded` — "not a bug, it's how you *think * different*", reference) → `Response.spim` (base64) * 3. [Spim.parse] the spim → `payload240` → [ClearAdiProvision.initSession] → session context - * 4. [Cpim.build] with 124 RNG bytes (32/32/60), an empty extra blob and the GSA arg −2 → - * base64 into `request.cpim` → `POST ` → `Response.{ptm, tk, X-Apple-I-MD-RINFO}` + * 4. [Cpim.build] with 124 RNG bytes (32/32/60) + the 16-B IV, an empty extra blob and the GSA + * arg −2 → base64 into `request.cpim` → `POST ` → `Response.{ptm, tk, X-Apple-I-MD-RINFO}` * 5. `tk` must be exactly 16 B (finish 17f1e31, error 0x8); key16 = `encrypt(session, NULL iv, - * tk)` (17f1eac); [ClearAdiProvision.decryptPtm] over the parsed Ptm payload → raw - * [ProvisionedData] - * - * The ProvisionedData→token-field mapping and the Cpim tail fields are the two remaining - * TODO(live-capture) items; they are modeled honestly in [ClearAdiFraming] and surfaced here - * as [cpimTailSeal] (required) and [finishSeed] (defaults to zero bytes). + * tk)` (17f1eac); [ClearAdiProvision.decryptPtm] over the Ptm payload with the Ptm-embedded + * seed (live-pinned: seed = ptm[4:20]) → [ProvisionedData.fromDecrypt] (the draw-1 gate echo + * at out[408:440]) → [ProvisionedAnisette] */ class ClearAnisetteProvider( private val http: IdsHttp, @@ -195,8 +172,6 @@ class ClearAnisetteProvider( private val loginInfo: ClearAdiLoginInfo, private val rng: ClearAdiRng, private val cpimTailSeal: CpimTailSeal, - /** Seed passed to decryptPTM (finish 17f2557's rcx) — zero bytes pending the live capture. */ - private val finishSeed: ByteArray = ByteArray(16), /** 16 random bytes identifying this client (reference `keychain_identifier`). */ private val keychainIdentifier: ByteArray = rng.draw(16), private val clock: () -> Long = { System.currentTimeMillis() }, @@ -204,9 +179,9 @@ class ClearAnisetteProvider( /** Load the stored state, or provision it through the GSA round trips. */ suspend fun ensureProvisioned(): ClearAdiProvisionResult { - stateStore.load()?.let { return resultFromJson(it) } + stateStore.load()?.let { return ProvisionedAnisette.fromJson(it) } val result = provision() - stateStore.save(resultToJson(result)) + stateStore.save(result.toJson()) return result } @@ -221,11 +196,20 @@ class ClearAnisetteProvider( val sessionCtx = ClearAdiProvision.initSession(spim.payload) // omnisette passes extra = [] and the GSA arg −2 (reference provision()). - val cpim = Cpim.build(rng, extra = ByteArray(0), gsaArg = GSA_ARG, sessionCtx, spim.payload, cpimTailSeal) + val built = Cpim.build( + rng, + extra = ByteArray(0), + gsaArg = GSA_ARG, + nowSeconds = now() / 1000, + sessionCtx = sessionCtx, + payload240 = spim.payload, + spimTrailing = spim.trailing, + tail = cpimTailSeal, + ) val finishBody = XmlPlist.encode( mapOf( "header" to emptyMap(), - "request" to mapOf("cpim" to Base64.getEncoder().encodeToString(cpim)), + "request" to mapOf("cpim" to Base64.getEncoder().encodeToString(built.wire)), ), ) val finishResponse = exchange("finish-provisioning", urls.finishProvisioning, finishBody) @@ -239,11 +223,18 @@ class ClearAnisetteProvider( } val parsed = Ptm.parse(ptm) val key16 = ClearAdiProvision.encrypt(sessionCtx, null, tk) - // count = payload.size / 16 (finish 17f2548); ProvisionedData is the first 0x78 B of - // the decrypted output (17f2dec–17f2e56) - val blocks = parsed.payload.size / 16 - val data = ClearAdiProvision.decryptPtm(key16, parsed.payload.copyOfRange(0, blocks * 16), finishSeed) - return ClearAdiProvisionResult(ProvisionedData(data.copyOf(0x78)), rinfo) + // count = payload.size / 16 (finish 17f2548); the seed rides in the PTM itself + val out = ClearAdiProvision.decryptPtm(key16, parsed.payload, parsed.seed) + val provisioned = ProvisionedData.fromDecrypt(out, built.draw1, built.draw2, built.draw3) + return ProvisionedAnisette( + clientSecret = built.draw2, + mid = built.draw3, + metadata = provisioned.metadata, + rinfo = rinfo, + // the OTP flavor keeps the ClearAdiOtp convention (loginInfo decision); the PTM + // struct's own flavor byte (0x01 on a live Mac run) uses a different numbering + flavor = flavor, + ) } /** The reference's fixed request-header set (`build_apple_request`, transcribed 1:1). */ diff --git a/registration/src/test/kotlin/dev/fenn/imessage/registration/ClearAnisetteProviderTest.kt b/registration/src/test/kotlin/dev/fenn/imessage/registration/ClearAnisetteProviderTest.kt index 3162d51..c77bc91 100644 --- a/registration/src/test/kotlin/dev/fenn/imessage/registration/ClearAnisetteProviderTest.kt +++ b/registration/src/test/kotlin/dev/fenn/imessage/registration/ClearAnisetteProviderTest.kt @@ -13,9 +13,10 @@ import kotlin.test.assertTrue /** * Tests for the wire-level ClearADI provider (ClearAdiFraming + ClearAnisetteProvider). - * The framing facts come from new.asm/finish.asm plus live-oracle probes of - * librust_lib_bluebubbles (see WORKLOG session 19e); everything the probes could not pin is - * TODO(live-capture) in the code and asserted only at the shape level here. + * Framing facts: new.asm/finish.asm plus the session-19f LIVE captures (WIRE-NOTES.md, + * dev/clearadi/captures/). The scripted HTTP flow exercises everything up to the PTM gate — + * faking a gate-valid PTM would need Apple's PTM encryptor, so the full-gate path is asserted + * at the data level ([ProvisionedData.fromDecrypt]) instead. */ class ClearAnisetteProviderTest { @@ -29,18 +30,18 @@ class ClearAnisetteProviderTest { @Test fun spimParseExtractsFields() { val payload = ByteArray(240) { it.toByte() } - val spim = Spim.parse(spimBytes(field1 = 0x01020304, payload = payload, tail = ByteArray(23))) + val tail = ByteArray(99) { 0x55.toByte() } // the live Spim carries 99 trailing bytes + val spim = Spim.parse(spimBytes(field1 = 0x01020304, payload = payload, tail = tail)) assertEquals(0x01020304, spim.field1) assertEquals(240, spim.payloadLength) assertContentEquals(payload, spim.payload) + assertContentEquals(tail, spim.trailing) } @Test - fun spimParseRejectsShortPayloadAndBadLeftover() { + fun spimParseRejectsShortPayloadAndTruncation() { // payload ≤ 59 B rejected (17efb33) assertFailsWith { Spim.parse(spimBytes(payload = ByteArray(59))) } - // > 23 trailing bytes rejected (live-pinned error 0x41) - assertFailsWith { Spim.parse(spimBytes(tail = ByteArray(24))) } // truncated header assertFailsWith { Spim.parse(ByteArray(3)) } // payload length beyond the buffer @@ -51,60 +52,86 @@ class ClearAnisetteProviderTest { // ---- Cpim build ---------------------------------------------------------- - /** Deterministic RNG: draws are 0x10, 0x20, 0x30-filled blocks (draw1, draw2, draw3). */ + /** Deterministic RNG: draws are 0x10/0x20/0x30/0x40/0x50-filled (draw1, draw2, draw3, iv, tail12). */ private class FixedRng : ClearAdiRng { var n = 0 override fun draw(n: Int): ByteArray { - val fill = when (++this.n) { 1 -> 0x10; 2 -> 0x20; else -> 0x30 }.toByte() + val fill = when (++this.n) { 1 -> 0x10; 2 -> 0x20; 3 -> 0x30; 4 -> 0x40; else -> 0x50 }.toByte() return ByteArray(n) { fill } } } - private val seal = CpimTailSeal { _, _, sig, digest -> - // the two 256-bit fields: shape-only seal so the test is deterministic - sig.copyOf(32) to digest + /** + * The live wire tail: `[X 32][u32 BE 52][u32 BE 4][Y 52]` — the 60-byte + * `[52][4][Y52]` group == the Spim's trailing[16:76] relayed verbatim (server material) + * and **X (32 B) is the one TODO(live-capture) byte group**, zero-filled here. + */ + private val seal = CpimTailSeal { spimTrailing, _, _ -> + ByteArray(32) + spimTrailing.copyOfRange(16, 76) } @Test - fun cpimBuildMatchesAsmLayout() { + fun cpimBuildMatchesLiveWireLayout() { val extra = ByteArray(10) { 0xAB.toByte() } + val trailing = ByteArray(99) { (it + 1).toByte() } val session = ClearAdiProvision.initSession(ByteArray(240)) - val cpim = Cpim.build(FixedRng(), extra, -2L, session, ByteArray(240), seal) - // 4+32+4+32+60+8+4+32+32 - assertEquals(208, cpim.size) - assertEquals(32, readBeU32(cpim, 0)) - // vec1 = encsec(draw2) — encsec is deterministic, so byte-exact here - assertContentEquals(ClearAdiProvision.encsec(ByteArray(32) { 0x20.toByte() }), cpim.copyOfRange(4, 36)) - assertEquals(32, readBeU32(cpim, 36)) - assertContentEquals(ByteArray(32) { 0x10.toByte() }, cpim.copyOfRange(40, 72)) // draw1 raw - // blob = extra ‖ draw3 padding - assertContentEquals(extra, cpim.copyOfRange(72, 82)) - assertContentEquals(ByteArray(50) { 0x30.toByte() }, cpim.copyOfRange(82, 132)) - // arg9 = −2 as u64 BE (omnisette "GSA") - assertEquals(-2L, readBeU64(cpim, 132)) - assertContentEquals(hex("FFFFFFFFFFFFFFFE"), cpim.copyOfRange(132, 140)) - // item 7 = BE u32 of the LE dword at draw3[8..12] — draw3 is 0x30-filled ⇒ 0x30303030 - assertEquals(0x30303030, readBeU32(cpim, 140)) - // tail fields present, 32 B each (items 8–9 = 64 B after the 144 B prefix) - assertEquals(64, cpim.size - 144) + val built = Cpim.build(FixedRng(), extra, -2L, 1_700_000_000L, session, ByteArray(240), trailing, seal) + // outer: [u32 BE 5][iv 16][u32 BE 160][encout 160][tail 92] + assertEquals(276, built.wire.size) + assertEquals(5, readBe(built.wire, 0)) + assertContentEquals(ByteArray(16) { 0x40.toByte() }, built.iv) + assertContentEquals(built.iv, built.wire.copyOfRange(4, 20)) + assertEquals(160, readBe(built.wire, 20)) + // plaintext rebuilt from the same draws → encout must match the real cipher + val vec1 = ClearAdiProvision.encsec(ByteArray(32) { 0x20.toByte() }) + val plain = ByteArray(160) + var p = 0 + p = putBe(plain, p, 32, 4) + System.arraycopy(vec1, 0, plain, p, 32); p += 32 + p = putBe(plain, p, 32, 4) + System.arraycopy(ByteArray(32) { 0x10.toByte() }, 0, plain, p, 32); p += 32 // draw1 = vec2 raw + System.arraycopy(extra, 0, plain, p, extra.size); p += extra.size + System.arraycopy(ByteArray(50) { 0x30.toByte() }, 0, plain, p, 50); p += 50 // blob = extra ‖ draw3 + p = putBe(plain, p, -2L, 8) + p = putBe(plain, p, 1_700_000_000L, 4) + p = putBe(plain, p, 1, 4) + System.arraycopy(ByteArray(12) { 0x50.toByte() }, 0, plain, p, 12); p += 12 + val encout = ClearAdiProvision.encrypt(session, built.iv, plain) + assertContentEquals(encout, built.wire.copyOfRange(24, 184)) + // the seal's relayed group comes from the Spim trailing + assertContentEquals(trailing.copyOfRange(16, 76), built.wire.copyOfRange(216, 276)) + // draws surface for the finish side + assertContentEquals(ByteArray(32) { 0x10.toByte() }, built.draw1) + assertContentEquals(ByteArray(32) { 0x20.toByte() }, built.draw2) + assertContentEquals(ByteArray(60) { 0x30.toByte() }, built.draw3) } @Test fun cpimBuildRejectsOversizeExtra() { assertFailsWith { - Cpim.build(FixedRng(), ByteArray(61), -2L, ByteArray(0x28F0), ByteArray(240), seal) + Cpim.build(FixedRng(), ByteArray(61), -2L, 0L, ByteArray(0x28F0), ByteArray(240), ByteArray(0), seal) } } - // ---- Ptm parse / tk check ------------------------------------------------ + // ---- Ptm parse (live-pinned layout) --------------------------------------- @Test - fun ptmParseTakesField1AndPayload() { - val ptm = Ptm.parse(hex("00000001") + ByteArray(32)) - assertEquals(1, ptm.field1) - assertEquals(32, ptm.payload.size) - // lenient Vec: field1 clipped to what the buffer holds (live-pinned) - assertEquals(4, Ptm.parse(hex("FFFFFFFF") + ByteArray(4)).payload.size) + fun ptmParsePinnedLayout() { + val seed = ByteArray(16) { 0x11.toByte() } + val payload = ByteArray(448) { 0x22.toByte() } + val trailing = ByteArray(31) { 0x33.toByte() } + val ptm = Ptm.parse( + beBytes(4, 4) + seed + beBytes(448, 4) + payload + trailing, + ) + assertEquals(4, ptm.field1) + assertContentEquals(seed, ptm.seed) + assertContentEquals(payload, ptm.payload) + assertContentEquals(trailing, ptm.trailing) + // payload length beyond the buffer + val short = beBytes(4, 4) + seed + beBytes(999, 4) + assertFailsWith { Ptm.parse(short) } + // truncated header + assertFailsWith { Ptm.parse(ByteArray(23)) } } @Test @@ -115,17 +142,43 @@ class ClearAnisetteProviderTest { assertTrue(ex.message!!.contains("tk not exactly 16 bytes")) } - // ---- ProvisionedData raw layout ------------------------------------------ + // ---- ProvisionedData assembly + gate --------------------------------------- + + private fun syntheticDecryptOut(draw1: ByteArray): ByteArray { + val out = ByteArray(448) + putBe(out, 0, 400, 4) + for (i in 4 until 404) out[i] = i.toByte() + draw1.copyInto(out, 408) + return out + } @Test - fun provisionedDataExposesRawLayout() { - val raw = ByteArray(0x78) { (it + 1).toByte() } - val data = ProvisionedData(raw) - assertEquals(16, data.block00.size) - // little-endian u64 of bytes 0x11..0x18 - assertEquals(0x1817161514131211UL.toLong(), data.word10) - assertEquals(64, data.metadata64.size) - assertEquals(0x75, data.flavor) // raw[0x74] = (0x74 + 1) + fun provisionedDataAssemblesFromDecryptOutput() { + val draw1 = ByteArray(32) { 0x10.toByte() } + val draw2 = ByteArray(32) { 0x20.toByte() } + val draw3 = ByteArray(60) { 0x30.toByte() } + val out = syntheticDecryptOut(draw1) + val data = ProvisionedData.fromDecrypt(out, draw1, draw2, draw3) + assertEquals(400, data.metadata.size) + assertEquals(25.toByte(), data.metadata[21]) + assertContentEquals(draw2, data.clientSecret) + assertContentEquals(draw3, data.mid) + } + + @Test + fun provisionedDataRejectsGateMismatchAndShortOutput() { + val draw1 = ByteArray(32) { 0x10.toByte() } + val draw2 = ByteArray(32) { 0x20.toByte() } + val draw3 = ByteArray(60) { 0x30.toByte() } + val wrongEcho = syntheticDecryptOut(ByteArray(32) { 0xEE.toByte() }) + val ex = assertFailsWith { + ProvisionedData.fromDecrypt(wrongEcho, draw1, draw2, draw3) + } + assertTrue(ex.message!!.contains("PTM gate mismatch")) + val ex2 = assertFailsWith { + ProvisionedData.fromDecrypt(ByteArray(100), draw1, draw2, draw3) + } + assertTrue(ex2.message!!.contains("too short")) } // ---- header assembly ----------------------------------------------------- @@ -175,28 +228,50 @@ class ClearAnisetteProviderTest { // ---- state round-trip + full scripted flow -------------------------------- @Test - fun provisionResultJsonRoundTrip() { - val raw = ByteArray(0x78) { it.toByte() } - val result = ClearAdiProvisionResult(ProvisionedData(raw), "1710631289") - val restored = resultFromJson(resultToJson(result)) - assertContentEquals(raw, restored.provisionedData.raw) - assertEquals("1710631289", restored.rinfo) + fun provisionedAnisetteJsonRoundTrip() { + val anisette = ProvisionedAnisette( + clientSecret = ByteArray(32) { 1 }, + mid = ByteArray(60) { 2 }, + metadata = ByteArray(400) { 3 }, + rinfo = "1710631289", + flavor = ClearAnisetteProvider.FLAVOR_MAC, + ) + val restored = ProvisionedAnisette.fromJson(anisette.toJson()) + assertContentEquals(anisette.clientSecret, restored.clientSecret) + assertContentEquals(anisette.mid, restored.mid) + assertContentEquals(anisette.metadata, restored.metadata) + assertEquals(anisette.rinfo, restored.rinfo) + assertEquals(anisette.flavor, restored.flavor) } @Test - fun provisionFlowsThroughScriptedHttp() = kotlinx.coroutines.runBlocking { + fun provisionFlowsThroughScriptedHttpToTheGate() = kotlinx.coroutines.runBlocking { val http = ScriptedIdsHttp() + val provider = provider(http = http) + // the scripted PTM cannot echo the gate (faking that needs Apple's PTM encryptor), + // so the live-shaped flow must fail exactly at the gate check + val ex = assertFailsWith { provider.provision() } + assertTrue(ex.message!!.contains("PTM gate mismatch")) + assertEquals(3, http.calls) // lookup + start + finish + } + + @Test + fun secondEnsureCallIsServedFromStore() = kotlinx.coroutines.runBlocking { val store = MemoryStateStore() - val provider = provider(http = http, store = store) - val result = provider.ensureProvisioned() - assertEquals(0x78, result.provisionedData.raw.size) - assertEquals("1710631289", result.rinfo) - // second call is served from the store without new HTTP traffic - val callsAfterFirst = http.calls + store.save( + ProvisionedAnisette( + clientSecret = ByteArray(32) { 1 }, + mid = ByteArray(60) { 2 }, + metadata = ByteArray(400) { 3 }, + rinfo = "1710631289", + flavor = ClearAnisetteProvider.FLAVOR_MAC, + ).toJson(), + ) + val provider = provider(http = ScriptedIdsHttp(), store = store) + val first = provider.ensureProvisioned() + assertEquals("1710631289", first.rinfo) val second = provider.ensureProvisioned() - assertEquals(callsAfterFirst, http.calls) - assertContentEquals(result.provisionedData.raw, second.provisionedData.raw) - assertTrue(store.saved!!.contains("provisioned_data")) + assertContentEquals(first.clientSecret, second.clientSecret) } private fun provider(http: ScriptedIdsHttp, store: ClearAdiStateStore = MemoryStateStore()): ClearAnisetteProvider = @@ -206,7 +281,6 @@ class ClearAnisetteProviderTest { loginInfo = ClearAdiLoginInfo("akd-user-agent/1", "com.apple.mme/1 (MacBookPro18,1)"), rng = FixedRng(), cpimTailSeal = seal, - finishSeed = ByteArray(16), keychainIdentifier = hex("000102030405060708090a0b0c0d0e0f"), clock = { 1_700_000_000_000L }, ) @@ -250,7 +324,7 @@ class ClearAnisetteProviderTest { 200, emptyMap(), XmlPlist.encode( - mapOf("Response" to mapOf("spim" to Base64.getEncoder().encodeToString(buildSpim()))), + mapOf("Response" to mapOf("spim" to Base64.getEncoder().encodeToString(buildSpim(tail = ByteArray(99))))), ), ) } else { @@ -258,14 +332,20 @@ class ClearAnisetteProviderTest { @Suppress("UNCHECKED_CAST") val request = (bodyPlist as Map)["request"] as Map assertTrue(request.containsKey("cpim")) + val cpim = Base64.getDecoder().decode(request["cpim"]!!) + // outer shape: [u32 BE 5][iv 16][u32 BE 160][encout 160][tail 88] + assertEquals(276, cpim.size) + assertEquals(160, readBe(cpim, 20)) IdsHttpResponse( 200, emptyMap(), XmlPlist.encode( mapOf( "Response" to mapOf( - // 240 B ptm ⇒ 15 decrypt blocks ⇒ ≥ 0x78 ProvisionedData source - "ptm" to Base64.getEncoder().encodeToString(ByteArray(240)), + // live-shaped: [u32 BE 4][seed 16][u32 BE 448][payload 448][tail 31] + "ptm" to Base64.getEncoder().encodeToString( + beBytes(4, 4) + ByteArray(16) + beBytes(448, 4) + ByteArray(448) + ByteArray(31), + ), "tk" to Base64.getEncoder().encodeToString(ByteArray(tkSize)), "X-Apple-I-MD-RINFO" to "1710631289", ), @@ -282,15 +362,17 @@ class ClearAnisetteProviderTest { } } - private fun readBeU32(b: ByteArray, off: Int): Int = - ((b[off].toInt() and 0xff) shl 24) or ((b[off + 1].toInt() and 0xff) shl 16) or - ((b[off + 2].toInt() and 0xff) shl 8) or (b[off + 3].toInt() and 0xff) +} - private fun readBeU64(b: ByteArray, off: Int): Long { - var v = 0L - for (i in 0 until 8) v = (v shl 8) or (b[off + i].toLong() and 0xff) - return v - } +private fun readBe(b: ByteArray, off: Int): Int = + ((b[off].toInt() and 0xff) shl 24) or ((b[off + 1].toInt() and 0xff) shl 16) or + ((b[off + 2].toInt() and 0xff) shl 8) or (b[off + 3].toInt() and 0xff) + +private fun beBytes(v: Long, n: Int): ByteArray = ByteArray(n) { (v ushr (8 * (n - 1 - it))).toByte() } + +private fun putBe(dst: ByteArray, off: Int, v: Long, n: Int): Int { + for (i in 0 until n) dst[off + i] = (v ushr (8 * (n - 1 - i))).toByte() + return off + n } private fun buildSpim(field1: Int = 0, payload: ByteArray = ByteArray(240), tail: ByteArray = ByteArray(0)): ByteArray {