registration: the register body is capture-exact (rev 51) — four services, id-get-handles, one prekey mint

The trapped reference register (frame-026) is now matched field for field.

- RegisterServiceCatalog: the reference's four services in wire order
  (madrid + multiplex1 + facetime.multi + ess), each row a constant from its
  own source, with per-service capabilities/sub-services/client-data and a
  per-service ec-version (real 1.0 on madrid, integer 1 on the other three
  — the plist types differ).
- IdsHandleFetcher: id-get-handles, the bodiless signed GET the reference
  calls per user before building the body; the register echoes the server's
  handle list on every service instead of inventing one from the username.
- One prekey mint and one kt-loggable-data blob for the whole body: the
  reference mints outside the service loop and clones, and the capture's
  blobs are byte-identical across the four services.
- MadridIdentity.clientData now builds the six identity keys only; the
  per-service capability keys come from the catalog row. Key order matches
  the capture (top-level and per-user).
- RegisterServiceCatalogTest + IdsHandleFetcherTest pin the shape against a
  PII-free projection of the capture
  (registration/src/test/resources/imessage/register-reference-shape.plist).

Nothing here is verified against Apple yet — the next live register is the
first one whose body matches the oracle's.
This commit is contained in:
fenleon committed 2026-10-16 21:43:31 +02:00
1 parent d0dd6d057e
commit 28e0432f09
9 files changed
+856 -24

No files matched your search

+10 -5
View File
@@ -1,6 +1,6 @@
# iMessage protocol spec (clean-room)
**Status:** draft — revision 50 (2026-10-16). Revision history at the bottom. Rev 49 corrected rev 48's "register #2": the empty-services register is the client's DEREGISTER (the failure-path `reset_state`), not a register step — and the reference's own register is confirmed status 0 (§1.1). Rev 50 closes the reference survey: no implementation available on this machine can confirm a registration, so the reference's captured bytes are the only oracle (see the rev 50 entry in the revision history).
**Status:** draft — revision 51 (2026-10-16). Revision history at the bottom. Rev 49 corrected rev 48's "register #2": the empty-services register is the client's DEREGISTER (the failure-path `reset_state`), not a register step — and the reference's own register is confirmed status 0 (§1.1). Rev 50 closed the reference survey: no implementation available on this machine can confirm a registration, so the reference's captured bytes are the only oracle. Rev 51 ports that oracle's remaining request-shape facts into code: the four-service catalog (§1.4) and the `id-get-handles` handle fetch (§1.4).
**How to read this document.** Each section opens with a one-line plain-English summary, then the facts. Facts carry citations (legend below). Where two reference implementations disagree on a wire detail, the disagreement is stated and tagged `TODO(capture):` — running mitmproxy captures arbitrate. All open capture items are collected in §7.
@@ -20,6 +20,7 @@
- [CF] — Apple's publicly documented binary property-list container format (Core Foundation; public Apple material)
- [CAP-REG] / [CAP-COURIER] — own mitmproxy captures of a registered iPhone 14,4, iOS 26.1 (2026-09-10; captures on file, not published) — ground truth, post-PQ3
- [CAP-IDSTLS] / [CAP-BAGCT] — direct unauthenticated probes from this machine (2026-09-10: openssl TLS captures; byte-exact bag bodies with response headers)
- [CAP-REG42] — the shipped reference's own `id-register` request body, trapped from its process on 2026-10-14 (`captures/c56k/ob-trap-20261014-201103/frame-026.bin`; gzip → 13301 B XML plist; shape-only projection checked in at `registration/src/test/resources/imessage/register-reference-shape.plist`) — the only working oracle's bytes (§rev 50)
- [RP:file] — rustpush source (SSPL, facts-only); bare [RP] = fact from that repo, source file not recorded in the extraction notes
- [APA:file] — apple-private-apis source (MPL 2.0, facts-only)
- [IG:file] — imessagego source (AGPL, facts-only)
@@ -111,7 +112,7 @@ A binary blob proving the client is a genuine Apple device; every registration s
- `cpim`: an OUTER 276-B message `[u32be(5)][iv 16][u32be(160)][encout 160][X 32][u32be(52)][u32be(4)][Y 52]`; the 160-B plaintext = `u32be(32) ‖ encsec(rng2) ‖ u32be(32) ‖ rng1 ‖ 60 B blob (caller-extra ‖ rng3 pad) ‖ u64be(−2) ‖ u32be(unix-seconds) ‖ u32be(1) ‖ 12 fresh RNG bytes`. The IV is 16 plain RNG bytes; X = `SHA-256(encout ‖ spim_map_signature(trailing[0:16]) ‖ fixed 24-byte .rodata filler)` — written to the wire byte-exact (rev 29 closes the X32/12-B-tail/trailing-head TODOs; the earlier "constant sig block" and "sig16 not on the wire" readings are corrected).
- `ptm`: `[u32be(4)][seed 16][u32be(payloadLen)][payload][tail]`; key16 = `encrypt(session, NULL iv, tk)`; block count = payloadLen/16; the finish gate is the decrypt output's `out[408:440] == rng1` echo. ProvisionedData = `{metadata: Vec (400 B live = decrypt out[4:404], length u32be at out[0]), client_secret = rng2 (32 B), mid = rng3 (60 B), flavor u8}`.
**Live-mint status (as of rev 49):** the full chain runs live on-device — export-code parse, hardware serialization byte-identical to capture, Apple-ID login with 2FA, fresh-blob mint accepted at step 2 through the rustls transport, courier connected, Albert activation accepted. **Confirmed on the reference (rev 49): its own §1.4 register returns status 0** — forced re-registration rotates every service's `id_keypair` certificate to `notAfter = register_time + 45 d` in id.plist, and rustpush aborts before recording any registration on a non-zero status, so 6004 is not a step in a healthy register. The blockers are therefore ours alone: the §1.4 register answers **6004 = `ActionRetryWithNewAbsintheContext`** (the response's SED message is a server-side VEN session record, not a stamp echo), and the §1.2 authenticate with the stored delegate credentials answers **6008** (stale delegate token — dies within hours). Body parity (rev 48) stands — full NGM identity `client-data` + gzip request body; the remaining request-shape divergence is that the reference sends all four services with both handles while ours sends madrid with one. Rev 49 removed the register's bogus first step (rev 48's "register #2" is the client's deregister).
**Live-mint status (as of rev 51):** the full chain runs live on-device — export-code parse, hardware serialization byte-identical to capture, Apple-ID login with 2FA, fresh-blob mint accepted at step 2 through the rustls transport, courier connected, Albert activation accepted. **Confirmed on the reference (rev 49): its own §1.4 register returns status 0** — forced re-registration rotates every service's `id_keypair` certificate to `notAfter = register_time + 45 d` in id.plist, and rustpush aborts before recording any registration on a non-zero status, so 6004 is not a step in a healthy register. The blockers are therefore ours alone: the §1.4 register answers **6004 = `ActionRetryWithNewAbsintheContext`** (the response's SED message is a server-side VEN session record, not a stamp echo), and the §1.2 authenticate with the stored delegate credentials answers **6008** (stale delegate token — dies within hours). Body parity (rev 48) stands — full NGM identity `client-data` + gzip request body. The last two request-shape divergences are now closed in code (rev 51: the four-service catalog + the `id-get-handles` handle fetch), so the next live register is the first attempt whose body matches the oracle's field for field. Rev 49 removed the register's bogus first step (rev 48's "register #2" is the client's deregister).
### 1.2 Activation endpoints and certificates
@@ -178,13 +179,16 @@ A binary blob proving the client is a genuine Apple device; every registration s
**Request (POST to the `id-register` bag URL [RP:ids/user.rs]; pinned URL [IG:register])**
- Body: XML plist in both implementations; disagreement is compression only — one gzips with `content-encoding: gzip` + `accept-encoding: gzip` [RP:ids/user.rs:1112-1115], the other sends raw XML with no content-type and no content-encoding header at all [IG:register.go:366-375]. One source records `Content-Type: application/x-apple-plist`, the other none [RP:ids/user.rs:1112], [IG:register.go:370-375]. → C3: does the live server accept both?
- Exactly eight top-level keys (agreed): `device-name`, `hardware-version`, `language`, `os-version`, `software-version`, `private-device-data` (dict), `services` (array), `validation-data` (data) [RP:ids/user.rs], [IG:register].
- Body: XML plist in both implementations; disagreement is compression only — one gzips with `content-encoding: gzip` + `accept-encoding: gzip` [RP:ids/user.rs:1112-1115], the other sends raw XML with no content-type and no content-encoding header at all [IG:register.go:366-375]. One source records `Content-Type: application/x-apple-plist`, the other none [RP:ids/user.rs:1112], [IG:register.go:370-375]. C3 CLOSED by capture (rev 51): the shipped reference gzips the body and sets BOTH `content-encoding: gzip` and `accept-encoding: gzip` [CAP-REG42], [RP:ids/user.rs:1112-1115].
- Exactly eight top-level keys (agreed): `device-name`, `hardware-version`, `language`, `os-version`, `software-version`, `private-device-data` (dict), `services` (array), `validation-data` (data) [RP:ids/user.rs], [IG:register]. The captured body writes them in this order: `device-name`, `hardware-version`, `language`, `os-version`, `private-device-data`, `services`, `software-version`, `validation-data` [CAP-REG42] — a plist dict is unordered, so this is cosmetic, but the writer now emits it.
- Headers: `x-push-token` (base64 of the 32-byte APNs connect token) [RP:ids/user.rs]; protocol version "1640" hardcoded [IG:ids/user.go:44] vs from config with "1660" in test fixtures [RP:test.rs:303] → C2; user agent `com.apple.invitation-registration [<name>,<version>,<build>,<hardware>]` (both) [RP:macos.rs:77-79], [IG:ids/config.go:89-92].
- **`private-device-data`** (agreed): `ap` "0"; `d` = seconds since the Apple epoch (2001-01-01) as a decimal string; `dt` = integer 1; `gt` "0"; `h` "1"; `m` "0"; `p` "0"; `pb` = OS build string; `pn` = platform name; `pv` = OS version; `s` "0"; `t` "0"; `u` = device UUID (uppercase string); `v` "1" — all strings except `dt` [RP:macos.rs:157-178], [IG:register.go:325-344]. The relay/phone profile sets `m` and `s` to "1" [RP:relay.rs:189-225]. Both sources also model four further keys (`c`, `ec`, `ktf`, `ktv`) but deliberately omit them from the built request.
- **`services`**: registers `com.apple.madrid` with a `sub-services` list and one capability record named "Messenger" with version 1. Flags DISAGREE: 17 [RP:imessage/aps_client.rs:74] vs 1 [IG:register.go:347]. Sub-service lists DISAGREE: four topics (`com.apple.private.alloy.sms`, `.gelato`, `.biz`, `.gamecenter.imessage`) [RP:imessage/aps_client.rs:19-24] vs seven (those four plus `.safetymonitor`, `.safetymonitor.ownaccount`, `.askto`) [IG:register.go:352-360], [IG:apns/topic.go:24-33] → C3. One source additionally registers FaceTime under `com.apple.ess` + `com.apple.private.alloy.facetime.*` (capability "Invitation") and Find My under `com.apple.private.alloy.multiplex1` (capability "com.apple.private.alloy") [RP:facetime.rs:26-56], [RP:findmy.rs:26-45].
- **`services` (C3 CLOSED by capture, rev 51):** the shipped reference registers exactly **four** services, in this order — `com.apple.madrid`, `com.apple.private.alloy.multiplex1`, `com.apple.private.alloy.facetime.multi`, `com.apple.ess` [CAP-REG42]. The list is the `id_services` argument at the one healthy call site [RP:api.rs:580], and each entry's row is a constant: madrid = flags **17** / "Messenger"; the other three = flags **1** / "com.apple.private.alloy" (multiplex1), "Invitation" (facetime.multi, ess) [RP:imessage/aps_client.rs:17], [RP:findmy.rs:25], [RP:facetime.rs:24], [RP:facetime.rs:40]. Sub-services: madrid 4 (`.sms`, `.gelato`, `.biz`, `.gamecenter.imessage`), facetime.multi 0, ess 3 (`.facetime.lp`, `.mw`, `.video`), multiplex1 5 (`.fmf`, `.fmd`, `.status.keysharing`, `.status.personal`, `.findmy.itemsharing-crossaccount` — the vendored checkout carries 4, the shipped binary 5, capture wins). The four-topic vs seven-topic disagreement [IG:register.go:352-360] is a *different* Apple-ID flow (no anisette, §rev 50) and not evidence of a second madrid shape.
- **Per-service `client-data` (capture):** every service carries the same six identity keys (the same NGM identity, byte-identical across services) plus its own extras — madrid 46 (`is-c2k-equipment`, `optionally-receive-typing-indicators`, `show-peer-errors`, the `supports-*` set, `nicknames-version`, `prefers-sdr` false), multiplex1 5 (`supports-fmd-v2`, `supports-incoming-fmd-v1`, `supports-findmy-plugin-messages`, `supports-beacon-sharing-v2`, `-v3`), facetime.multi 7 (`show-peer-errors`, `supports-avless`, `-co`, `-gft-calls`, `-gft-errors`, `-modern-gft`, `-self-one-to-one-invites`), ess 3 (`show-peer-errors`, `supports-gft-errors`, `supports-live-delivery`) [CAP-REG42]. `ec-version` is a **real** `1.0` on madrid and the **integer** `1` on the other three — the plist types differ, and this is the only value difference on the shared keys.
- Services-entry keys: `capabilities` (array), `service`, `sub-services`, `users`; capability-record keys: `flags` (int), `name` (string), `version` (int; both set 1).
- **Per-user entry keys (C53 — agreed)**: `client-data` (dict), `uris` (array), `user-id`, `kt-loggable-data` (data; optional in Go, always set in Rust when publishing keys), plus `tag` only for phone-number users — value `SIM` for the first phone user and `SIM2`, `SIM3`, … for subsequent ones [RP:ids/user.rs:1078-1081]; Go tags any phone-profile user "SIM" [IG:register.go:233-236]. `uris` entries are single-key dictionaries with key `uri` holding handle URIs (`tel:` E.164 or `mailto:`) — and neither implementation constructs them locally: both fetch the registered handle list from the server first and echo it back [IG:util/uri/uri.go:61-64,177-191], [RP:ids/user.rs:839-843,1040-1042].
- **Wire order, and one blob, not four:** the captured user entries write `client-data`, `kt-loggable-data`, `uris`, `user-id` (a `tag`, when present, last) [CAP-REG42] — the writer follows. All four services carry the **byte-identical** prekey and `kt-loggable-data` blobs, i.e. the reference mints each once outside the service loop and clones it [RP:ids/user.rs:982 `register`] — minting per service would put four different ECDSA signatures in one body [CAP-REG42].
- **The handle fetch is `id-get-handles`** [RP:ids/user.rs:791 `get_possible_handles`, called per user from `register` at :987 before the body is built]: a **bodiless GET** to the bag URL, headers `x-push-token` + `x-protocol-version` + an **unindexed** `x-auth-user-id`, plus the six signature headers from `base_request` (:712 — `sign(..., Auth, aps, None)` → no `-<N>` postfix; then the push signature with the same tuple). Response: `status` (0 = ok; the reference maps non-zero to `AuthInvalid(status)`) + `handles` = array of `{uri}`. The captured register's `uris` are the same two handles (`tel:` + `mailto:`) on **every** service — the reference fetches once per USER and reuses the list across services [CAP-REG42].
- `client-data` base wire keys (Rust): `public-message-identity-key`, `public-message-identity-version`, `ec-version`, `public-message-identity-ngm-version`, `public-message-ngm-device-prekey-data-key`, `kt-version`, plus per-service extras; Go adds the same six plus the boolean `supports-*` feature flags and `nicknames-version`/`optionally-receive-typing-indicators`/`prefers-sdr`/`show-peer-errors` [RP:ids/user.rs], [IG:register].
- **Capability booleans** (wire keys, union across both sources, all true unless noted): `optionally-receive-typing-indicators`, `show-peer-errors`, `supports-ack-v1`, `supports-audio-messaging-v2`, `supports-autoloopvideo-v1`, `supports-be-v1`, `supports-ca-v1`, `supports-cross-platform-sharing`, `supports-fsm-v1/v2/v3`, `supports-ii-v1`, `supports-impact-v1`, `supports-inline-attachments`, `supports-keep-receipts`, `supports-location-sharing`, `supports-media-v2`, `supports-original-timestamp-v1`, `supports-people-request-messages` (+ `-v2`, `-v3`), `supports-photos-extension-v1/v2`, `supports-protobuf-payload-data-v2`, `supports-rem`, `supports-sa-v1`, `supports-st-v1`, `supports-update-attachments-v1`, `supports-hdr`, `supports-heif`, `supports-dq-nr`, `prefers-sdr` (false), `nicknames-version` 1 [RP:imessage/aps_client.rs:25-73], [IG:register.go:240-299]. First source only: `is-c2k-equipment`, `supports-activity-sharing-v1`, `supports-shared-exp`, `supports-family-invite-message-bubble`, `supports-live-delivery`, `supports-stick-moji-backs`, `supports-emoji-tapbacks`, `supports-send-later-messages`, `supports-certified-delivery-v1`, `supports-transcript-backgrounds`, `supports-gti`, `supports-polls`. Second source only: `supports-askto`, `supports-emoji-stickers`, `supports-findmy-plugin-messages`, `supports-sticker-editing`, `supports-recurring-payment-bubbles`. Type discrepancy: one encodes versions as plist reals, the other as integers → C57.
- **Identity keys**: an RSA-1280 encryption key plus a P-256 signing key for the legacy scheme, plus a P-256 device key and a P-256 pre-key for the newer EC scheme [RP:ids/user.rs].
@@ -562,6 +566,7 @@ Each item names its arbitration path. Closed residue is noted inline.
## Revision history
- **Rev 51 (2026-10-16, session 37):** the §1.4 **request body is now capture-exact** — the two remaining divergences from the only working oracle are closed on paper and in code. (1) **Four services, not one:** the reference registers `[madrid, multiplex1, facetime.multi, ess]` in that order, each row a constant from its own source (`api.rs:580` + the four `IDSService` constants), with per-service capabilities/sub-services/`client-data` and a per-service `ec-version` that is a real `1.0` on madrid and the integer `1` elsewhere [CAP-REG42]. Our register sent madrid alone. Ported as `RegisterServiceCatalog` (four rows; one NGM identity and one auth certificate shared by all four, as the capture shows). (2) **The `uris` are server-fetched, not derived:** the reference calls `id-get-handles` per user before building the body and puts the returned list on every service [RP:ids/user.rs:791,987], [CAP-REG42] — our register invented one handle from the login username. Ported as `IdsHandleFetcher` (bodiless signed GET, unindexed `x-auth-user-id`); the login-derived handle survives only as a loud fallback. Also recorded: the register sets `accept-encoding: gzip` alongside `content-encoding: gzip` [CAP-REG42] (C3's last open half). Structural parity is pinned by `RegisterServiceCatalogTest` against a shape-only fixture extracted from the captured body (`registration/src/test/resources/imessage/register-reference-shape.plist`, source sha256 in the fixture; the reference user's identifying values are not recorded). Nothing here is verified against Apple yet — the 6004 read is still pending.
- **Rev 50 (2026-10-16, session 36):** reference survey closed — a negative result, recorded so it is not re-run. **No implementation available on this machine can confirm a registration**, so the reference's captured bytes are the only oracle. `open-absinthe` is the official **stub** (every body `todo!()`; "closed source ... no actual functionality"), so it cross-checks nothing — the [OA] legend entry is load-bearing rather than merely descriptive. `imessagego` (snapshot 2024-01-16) **requires an external Mac / jailbroken-iPhone registration provider plus a relay** and never registers standalone, so it evidences request *shape* only. Its back half matches ours: identical `authenticateDS` body shape (`{authentication-data.auth-token, csr, realm-user-id}`), the same eight register keys, **one service `com.apple.madrid`**, and its own degenerate deregister (`DeRegisterIDS` — register with nil handles/cert pairs) confirming independently that the empty-services body is a deregister, not a register step. Its front half is a different Apple-ID world: **no anisette** (zero `X-Apple-I-MD` hits), no `GsService2`, no `spd`/PET, no delegate — login is a single call to `profile.ess.apple.com/…/authenticateUser` (plist `{username, password}`, 2FA code appended to the password), and validation data comes from an external NAC server. Consequence for our 6004: anisette cannot be its cause. Also recorded: the shipped reference binary is **ClearADI-built** (1026 `clearadi` / 0 `RemoteAnisetteProviderV3` / 78 `open_absinthe` symbol hits), so a checkout whose `Cargo.toml` enables `remote-anisette-v3` is not the shipped build — trust the binary.
- **Rev 49 (2026-10-16 late evening):** rev 48's "register #2" is NOT a register — an empty-services body with no `x-auth-*` headers is the client's DEREGISTER. rustpush emits an empty-services register in exactly one place: `register(config, aps, &[], &mut [], identity)` inside `reset_state(state, reset_hw, logout)` (`/tmp/openbubbles/rust/src/api/api.rs:2424`, "try deregistering from iMessage, but if it fails we don't really care"), reached from `rustpush_service.dart:348 markFailedToLogin()` → `:4621 Future reset()` → `api.resetState`; both healthy register paths (`api.rs:580 register_ids`, `identity_manager.rs:344`) always send the four services. A captured FULL→EMPTY pair therefore means *register succeeded, a later step failed, the app reset*. The step is removed from `idsRegister`. Also settled by measurement rather than by response capture: **the reference's own register returns status 0** — forcing a re-register (host-side id.plist edit: `registered_at_s = now − 7200` **plus** `heartbeat_interval_s = 600`, a key the app deletes on write) rotates every service `id_keypair` certificate to `notAfter = register_time + 45 d` and renews the `env_hash`/pseudonym set, which `register()` can only do after a zero status (it returns `Err` before recording on any non-zero status). A register carries no anisette headers — `SignedRequest::sign` (`rustpush/src/auth.rs:395+`) appends only `x-{auth|push|id}-nonce|sig|cert[-i]` — so the absinthe context of a register is exactly its `validation-data`. Capture limits worth recording: the register rides HTTP/2 (PRI preface + HPACK), so its *headers* are not recoverable from a plaintext write trap, and its `plist::from_bytes::<Value>` response parse is LTO-inlined into `register`, so no breakpoint fires for the *response* body (needs a rustls read-path hook, e.g. `ConnectionCore::process_new_packets`). Reference body shape re-verified from frame-016: four services (`com.apple.madrid` 17 flags/"Messenger" + sub-services sms/gelato/biz/gamecenter.imessage, `com.apple.private.alloy.multiplex1` 1/"com.apple.private.alloy", `com.apple.private.alloy.facetime.multi` 1/"Invitation", `com.apple.ess` 1/"Invitation"), one user each, `uris` = tel + mailto on every service, `client-data` 52/11/13/9 keys, the 14 `private-device-data` keys, 517 B stamp. Our register still sends madrid only, with one handle.