From 2c0c0515ddad7c308b37ef4ea9be7eaffb1eb744 Mon Sep 17 00:00:00 2001 From: James Dansereau <106631954+jkdansereau@users.noreply.github.com> Date: Sun, 23 Aug 2026 10:22:09 -0400 Subject: [PATCH] =?UTF-8?q?PC=20port:=20phase=202=20=E2=80=94=20handoff=20?= =?UTF-8?q?for=20D51=20session:=20findings=20D50/D51,=20rewritten=20HANDOF?= =?UTF-8?q?F=20+=20research=20=C2=A7G/=C2=A7H?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - §F/D50: Plan B execution record (emit pass, plumbing, lang/font fixups, legal-screen UB, texCopyGdls byte proof) - §F/D51: current blocker — import_texture_i8 via main-DL G_TEXRECT lazy tile-0 upload; GE opcode space facts (non-F3DEX numbering, G_TEXRECT=0xE4) - HANDOFF.md rewritten: Task 1 = live-breakpoint gdb recipe for D51, Task 2 = pixel assert + soak, Task 3 = Tbg_*_stanZ backgrounds - AGENTS.md phase status updated (D43/Plan B closed, D51 open) --- AGENTS.md | 2 +- docs/HANDOFF.md | 603 +++++++++++++++++++---------------------- docs/PCPortResearch.md | 257 ++++++++++-------- 3 files changed, 423 insertions(+), 439 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0d85509c..e007d03f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,6 @@ Run `/linkcheck` for this sweep. Record new findings in `docs/PCPortResearch.md` - **Compile+link milestone:** all ~235 TUs compile and `ge007.x86_64.exe` links clean (`ninja ge007 -k 0`, 236/236). Findings D7–D17 in `docs/PCPortResearch.md` §11; asset stubs in `port/src/assetstubs.c`, fast3d no-ops in `port/src/gfxstub.c` (delete when real fast3d lands). - **Phase 1 (boot to window):** done — ROM loads/validates/maps at cart base 0x10000000 (`romdata.c`), SDL2 window + GL clear loop (`video.c`), absolute asset symbols from `scripts/gen_romassets.py` → `port/src/romassets_.s` (findings D18–D23). - **Phase 1.5 (boot to first frame):** DONE — boot chain complete AND real frames render (`[NOTE] frame N rendered in … us`). D34–D42 closed the blockers on top of D31–D33: ANIM_DATA lvalues, music seq-table ABI+endianness, music-heap sizing, libaudio bank-tree re-layout, implicit-declaration prototype shim (D38), Globalimagetable rebasing + true segment size (D39), ModelHitEntry pool sizing (D40), cross-thread GL context release (D41), rsp task-settings toggle (D42). ROM dump verified against the No-Intro good dump (byte-identical, header CRCs re-checked — `tools_pc/romverify.c`). -- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + first frames render; current blocker is **D43** (model-file loading ABI mismatch: N64-layout ROM model files read as 8-byte-pointer PC structs → D37-style re-layout). **D43 status:** all 512 model files validated; buffer sizing finalized by the D46 overlap-safety analysis; step-2 mechanical game-code edits APPLIED + committed (ecefffdb); **D47 closed every open converter question** — compaction state values, G_VTX encoding, opcode-aware remap rule, vertex formats — spec implemented + validated 512/512 by `tools_pc/d43_convert.py` (max D_PC = 0xB7E0, ratio ≤ 1.31). **D48 review re-plan:** the old "port d43_convert.py 1:1 to C" task is discarded — that script does not emit bytes, and **Plan B (offline pre-conversion)** is now the default: Python emit pass → `data/pcmodels-/` RZ sidecars served via a port-layer `hw_address`/`rom_size` table patch (no C converter; full mechanics in §F/D48). **D49 review PASSED** — all 8 D48 checklist items CONFIRMED against code + ROM (§F/D49): RZ format, memcpy DMA path, table chokepoint, obInit timing, buffer fit (worst-case P_final 0x16F74 vs 0x24C400 bank), reload hazard + pending one-line reset, per-region sidecars (EU=465 files; no JP ROM candidate yet), emission spec complete (fresh C probe: no record-size drift; added the bswap16-record-scalars rule). Next session EXECUTES Plan B (HANDOFF Task 2); fallback Plan A = D47.13 C converter with the corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`). All 23 d43_*.py scripts moved from gitignored build-pc/ to tracked `tools_pc/`. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48). +- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + real frames render. **Plan B EXECUTED (D50, committed):** all 512 NTSC model files converted offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + manifest.csv (single concatenated RZ image; regenerate per region with the script — sidecars are gitignored), served through the existing load path via a port-layer table patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot `pcmodelsPatchTable()` + poolRemaining reset hook in `load_object_fill_header`); runtime C fixups for language-bank BE offset tables + font re-layout (D50.3/D50.4); legal-screen UB seed (D50.5); **D50.6 fixed the first model-render crash** — `texCopyGdls` copied only w0 of each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes), leaving stale mempool w1s in the compacted GDLs; byte-proven against PlegalpageZ, fixed with a full-slot copy under #ifdef PORT. Model GDLs now execute to completion (PlegalpageZ sub-DLs). **Current blocker D51:** SIGSEGV in `import_texture_i8` via a main-DL G_TEXRECT lazy tile-0 upload (frame 5) — GE opcode space decoded (G_TEXRECT=0xE4/0xE5, non-F3DEX numbering; see §F/D51); investigation recipe in HANDOFF Task 1. Next: D51 → pixel-assert soak (PPM dump + pixcount still unbuilt) → next asset type (Tbg_*_stanZ backgrounds, same offline pattern; PD `preprocess/filebg.c` reference). Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48; full-struct Gfx copies on PC — D50.6). - **Phase 3 (audio + input):** libaudio → SDL — largely copy-and-adapt from PD (`mixer.c` 722 lines vs our 31-line stub; `input.c` SDL_GameController backend; §2.4); decide the ASP strategy (C2: `aspMain*` dummies in `ucode.c`). - **Phase 4 (saves + polish):** file-backed EEPROM — build on PD's `fs.c` (file-backed save dir); PFS/motor already no-op stubs. diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index ac213102..025620f5 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -1,374 +1,321 @@ -# Handoff brief — GoldenEye 007 PC port (Phase 2: Plan B review PASSED (D49) — next session executes the offline pre-conversion) +# Handoff brief — GoldenEye 007 PC port (Phase 2: Plan B executed (D50) — next session diagnoses D51, the main-DL texture-upload crash) _Paste-ready briefing. Authoritative detail lives in `AGENTS.md` and -`docs/PCPortResearch.md` (§F/D31–D48 findings, §H handoff); this is the summary -+ the immediate tasks._ +`docs/PCPortResearch.md` (§F/D43–D51 findings); this is the summary + the +immediate tasks._ ## Your job -- ~~Session A: independent review of Plan B (D48)~~ — **DONE (D49, committed):** - all 8 checklist items CONFIRMED against code + ROM; Plan B green-lit. One - spec amendment recorded there: bswap16 every u16/s16 record scalar on emit - (see Verified facts). -- **Session B (next): EXECUTE Plan B** per §Task 2 — Python emit pass + - sidecar generation, ~60 lines of port-layer plumbing, one-line hook, - build + pixel-assert soak. Commit at each working checkpoint - (`PC port: phase 2 — `). The goal: get stage geometry on screen. +- **Session C (next): diagnose + fix D51** — SIGSEGV in `import_texture_i8` + via a main-DL G_TEXRECT lazy tile-0 upload (frame 5). Recipe in Task 1. + Then build the pixel assert (Task 2), run a soak, commit at each milestone + (`PC port: phase 2 — `). The goal: get past frame 5 with actual model + geometry on screen, provable by pixel count. Work agentically — fix → build (~5 s) → verify → commit at each milestone. ## Read first (authoritative, in order) 1. `AGENTS.md` — non-negotiables, critical files, build, verification ritual. -2. `docs/PCPortResearch.md` **§F/D49** (review verdict + quantified fit - numbers), **D48** (the plan being executed — read fully), then **D47** - (converter contract — still ground truth for the emit pass), - **D46** (overlap safety + buffer sizing), **D45** (buffer constants), - **D43** (the model-file task), §2.4 (PD-port copy-candidate audit). -3. The code paths D48 cites: `port/src/rzdecomp.c`, `port/src/libultra.c` - (`piServiceDma`), `port/src/romdata.c` (`romdataInit`, - `romdataCartAddrValid`), `src/ramrom.c`, `src/game/ob.c` (`load_resource`, - `fileIndexLoadToBank`, `fileSetSize`, `obInit`), `src/game/objecthandler_2.c` - (`load_object_fill_header`). +2. `docs/PCPortResearch.md` **§F/D50** (what the last session did, incl. the + texCopyGdls byte proof), **D51** (current blocker + GE opcode facts), then + D49/D48/D47 (sidecar contract — still ground truth for regeneration). +3. The crash-chain code: `port/fast3d/gfx_pc.cpp` (`gfx_run_dl` G_TEXRECT case + ~line 2457, `import_texture*` ~600–950, `seg_addr` ~2282, + `gfx_sp_moveword` ~2290s), `src/game/tex.c` (texCopyGdls + texLoadFromGdl). ## Non-negotiables (the ones that bite) - N64 build files (`Makefile`, `tools/`, `rsp/`, `ld/`) — never touch. - **Game logic is unmodified.** Hardware deps go in `port/`. Narrow exception: mechanical, semantics-preserving ABI/layout fixes forced by the 32→64-bit transition (embedded pointers → u32 + cast at use; PC-guarded pool sizing / - idiom branches), each documented as a D3x finding. No behavior changes. - (Plan B's only game-code change is the PORT-guarded hook block in - `load_object_fill_header` — bookkeeping reset + one-shot patch call, no logic.) + idiom branches; UB-exposure seeds), each documented as a D3x/D50 finding. + No behavior changes. The N64 path stays verbatim under `#else`. - Region macros in `CMakeLists.txt` must mirror the N64 Makefile exactly. ## Current state -- **D49 (this handoff's predecessor, committed): Plan B review PASSED** — all 8 - D48 checklist items CONFIRMED with evidence (§F/D49). Quantified: Σ round8(C) - = 1,277,088 B; worst-case post-compaction P_final = 0x16F74 vs 0x24C400 stage - bank (25× headroom); all dst!=0 buffers pass against current constants; - fresh C probe re-verified every PC record size + promotion offset (no drift). - Notes: romdataInit has no JP ROM candidate (JP sidecars moot for now); - EU region has 465 model files vs 512 (generator maps what exists); the - poolRemaining=0 reset is still pending (Task 2 step 3). -- **D43 status (latest review session, §F/D48):** investigation complete; - converter spec finalized (D47) and validated 512/512 (`tools_pc/d43_convert.py`, - max D_PC = 0xB7E0, ratio ≤ 1.31). **The remaining task was re-planned:** the - old plan ("port d43_convert.py 1:1 to C as `romdataFixupModelFile`") is - discarded because (a) d43_convert.py does not emit bytes — the emission code - is unimplemented in any language, and (b) **Plan B (offline pre-conversion)** - reuses the existing ROM load chain and eliminates the C converter entirely: - a Python emit pass writes 512 PC-layout RZ sidecars to - `data/pcmodels-/`; the port layer serves them by patching - `file_resource_table[i].hw_address` + `resource_lookup_data_array[i].rom_size` - (full mechanics in D48.2). Plan A (D47.13 C converter) remains the fallback, - with a corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`, not - `D_PC ≤ avail`). -- DONE + committed (ecefffdb, pushed): all step-2 mechanical game-code edits - (bondtypes.h Vertex u32 union; model.c PROMOTE32; chr.c casts; - objecthandler_2.c Textures offset; tex.c Gfx-size/dispatch/byte-check/ - copy-count fixes), D45/D46 buffer constants (gun.c/front.c/initmenus.c), - fast3d seg_addr seg-5 raw-VMA case (port/fast3d/gfx_pc.cpp). -- DONE this review session: **all 23 `d43_*.py` scripts moved from gitignored - `build-pc/` to tracked `tools_pc/` and committed** — the spec is now - versioned. Converter re-run from new location: ALL CLEAN. -- Verified environment facts (do not rediscover): frames 1–4 render then FATAL - at `modelPromoteNodeOffsetsToPointers()` (model.c:5688) = D43; black screen - is clear color only (no stage geometry loaded yet). ROM dump byte-identical - to the No-Intro good dump (`tools_pc/romverify.exe`). Crash log with real - backtraces works (D44). Build clean 237/237; full rebuild ~5 s. +- **Plan B is EXECUTED (D50, committed):** all 512 NTSC model files converted + offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + + `manifest.csv` (single concatenated RZ image, 16-aligned offsets, decimal + manifest); port plumbing in `port/src/pcmodels.c` (+ `port/include/pcmodels.h`, + romdata.c/h cart-extension); one-shot `pcmodelsPatchTable()` + poolRemaining=0 + reset hook in `load_object_fill_header` (objecthandler_2.c, #ifdef PORT). + Boot log: `[INFO] pcmodels: table patched (512 model entries)`. +- **Runtime C fixups landed (D50.3/D50.4):** language-bank BE offset tables + (`romdataFixupLangBank`, called from language.c after each load + lazily in + the langGetJpnCharPixels paths) and font re-layout + (`romdataFontPcSize`/`romdataFixupFont` in textrelated.c). These are + load-time C fixups, NOT offline sidecars — small blobs on different load + paths; fine to keep runtime. +- **Legal-screen UB seed (D50.5):** front.c `constructor_menu00_legalscreen` + read an uninitialized pointer (N64 survived by luck); PORT branch seeds it + with `legalpage_text_array`. Identical lookat. +- **First model-render crash fixed (D50.6):** `texCopyGdls` copied only w0 of + each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes), + so the compacted GDLs kept stale mempool w1s → garbage G_SETTIMG → OOB in + `import_texture_rgba16`. Byte-proven against PlegalpageZ (sidecar w1 + 0x050012C8 vs RAM garbage 0xACAAB819 at file offset 0x2488). Fixed with a + full-slot copy under #ifdef PORT. **Model GDLs now execute to completion.** +- **Current blocker — D51:** SIGSEGV on frame 5 in `import_texture_i8` + (gfx_pc.cpp:782) via `G_TEXRECT` (main DL offset +0x260) → lazy upload of + RDP tile 0 with an invalid/stale source. See Task 1. Frames 1–4 render + successfully in every run (deterministic). -## Task 1 — Plan B review checklist (Session A) — DONE (D49: 8/8 CONFIRMED) +## Task 1 — D51 diagnosis (the immediate task) -Verify each claim. For each: state CONFIRMED or REFUTED in D49 with the -evidence you used (file:line, command output). If a falsification criterion -fires, stop and record it — do not paper over it. +The failing blit: 32×48 px screen rect (236,92)–(268,140), tile 0, i8 format. +G_TEXRECT carries no source address — it uses RDP texture-memory tiles; the +fault is in `import_texture(tile=0)` reading `rdp.textures[0]`'s stored source. +Work the suspects in order: -**R1. RZ format is reproducible.** Claim: every model ROM region is -`[0x11 0x72][raw deflate]`; `decompressdata()` (port/src/rzdecomp.c) skips -exactly those 2 bytes and inflates raw (`inflateInit2(-15)`) with a generous -avail_in bound, so a sidecar file in that format works through -`load_resource` unmodified. Verify: read rzdecomp.c; confirm the 2-byte prefix -is constant across all 512 (one-liner over `scripts/filelist.u.csv` + ROM — -`tools_pc/d43_convert.py` already decompresses all 512 with `[2:], -15`, re-run -it). **Falsified if:** the prefix varies per file, or decompressdata depends on -header state beyond 2 bytes. +1. **Dump live (post-mortem dumps FAIL — 0x70052e80 reads "Cannot access + memory" after the crash).** gdb launch mode: + ``` + gdb -batch \ + -ex "handle SIGSEGV stop" \ + -ex "break import_texture_i8" \ + -ex run \ + -ex "print rdp.textures[0]" \ + -ex "up 4" \ + -ex "print cmd" -ex "print dListStart" \ + -ex "x/256wx " \ + ./build-pc/ge007.x86_64.exe + ``` + (Adjust `up` depth to land in `gfx_run_dl`; the DL start may not be + 0x70052e80 every run — it's a mempool bump allocation. Alternatively break + at `osSpTaskStartGo` and read `t->t.data_ptr` / `data_size`.) +2. **Find the G_SETTIMG (w0 top byte 0xFD) slots in the dumped main DL** and + decode each w1 through the seg_addr rules (raw 0x05xxxxxx → seg5+low24; + marked LSB → segmentPointers[seg]+addr; unmarked <0x10000000 → + segmentPointers[seg]+low24; <0x800000 → +0x80000000; else passthrough). + Also dump `segmentPointers[]` at the breakpoint. If a SETTIMG w1 resolves + to an OOB/invalid pointer → that's the bug (converter remap miss, or a new + address class seg_addr doesn't handle). +3. **If no main-DL SETTIMG sets tile 0 before the G_TEXRECT:** the tile is + stale/uninitialized — check whether the game expects a tile set by an + earlier frame's DL (fast3d `rdp` persistence across frames) or by a sub-DL + that never ran. +4. **If the source pointer is valid but points at wrong-format/wrong-size + data:** the game may point it at a ROM/ramrom image needing a port fixup + (D50.3/D50.4 class) — check what asset lives there and how N64 PD handles + the analogue. +5. **Find who emits the G_TEXRECT** in game code: grep `src/game/` for + `gSPTileRect\|G_TEXRECT\|gSPTextureRectangle` (likely image.c / front.c / + title.c — legal-screen or logo 2D blit). Reading the emitter tells you + which texture it means and where its source should come from. -**R2. The DMA path is a plain host memcpy.** Claim: `romCopy` (src/ramrom.c) → -`osPiStartDma` → port shim `piServiceDma` (port/src/libultra.c) = `memcpy` from -`(void*)srcPA`, gated ONLY by `romdataCartAddrValid()` (port/src/romdata.c:212, -checks `addr ≥ CART_BASE && off+size ≤ romSize`). The ROM is VirtualAlloc'd at -CART_BASE 0x10000000 sized exactly romSize — so extending the reservation by -the sidecar total and extending the validity bound serves sidecars with zero -other changes. Verify: read both shims end-to-end; confirm no other gating -(alignment, hardcoded 0x10C00000 limits elsewhere, cache effects). Note -`romCopyAligned` exists but is NOT used by this load path. **Falsified if:** -any additional gate rejects non-ROM host pointers or sizes we can't satisfy. +Record findings as **D51.x** in §F; fix in `port/` where possible (seg_addr / +tile setup), PORT-guarded game-code only if ABI/layout-forced. Rebuild, +confirm the crash moves past frame 5, then Task 2. -**R3. The resource table is the single choke point and is writable.** Claim: -`file_resource_table` is a plain (non-const) global defined via include in -src/game/ob.c:22; ALL 512 model loads go through `load_object_fill_header` -(dst==0 → `_fileNameLoadToBank`, custom-buffer callers in front.c/gun.c/ -bondview2.c/initmenus.c → `_fileNameLoadToAddr`), both reading `hw_address` via -`load_resource`; no game code dereferences C*/G*/P*Z asset symbols directly -outside the table. Verify: grep callers of `fileLoad` / -`load_object_fill_header`; grep every model symbol (e.g. `CcamguardZ`, -`PflagZ`) outside `assets/obseg/file_resource_table.inc.c`, -`port/src/romassets_*.s`, and N64-only files (`obseg.h` decls, `ob_seg.s`). -**Falsified if:** any compiled game code reads model bytes straight from the -cart (it would bypass the sidecar and crash on N64 layout). +## Task 2 — pixel assert + soak (build it now — dev-loop item #1) -**R4. Patch timing: after obInit, before first model load.** Claim: `obInit()` -(src/boss.c:179) runs before the first model load and computes `rom_size` from -adjacent-entry hw_address DELTAS (ob.c:122) — so the patch must run after it -and set `rom_size` explicitly; nothing between obInit and the first model load -consumes model entries' rom_size/hw_address. Verify: trace the init chain -(bossInitMainthreadData → … → first stage object load; the D43 crash stack -shows the order); grep `rom_size` uses (ob.c:122, 253, 291; -`obLoadBGFileBytesAtOffset` is BG-only). **Falsified if:** anything in that -window reads model entries' rom_size/hw_address, or obInit can run after a -model load. +Env-gated framebuffer PPM dump every N frames in `port/video.c` +(e.g. `GE_PCDUMP=1` → `build-pc/frame_%05d.ppm`) + `tools_pc/pixcount.py` +(counts non-clear-color pixels; diffs two captures). Milestone: pixel count +rises above a small threshold and stays up through a 30 s+ soak. Screenshot +diffs against Mupen64Plus (verified No-Intro dump) are the ground truth for +"looks right". Commit: `PC port: phase 2 — D51 …` then `… pixel assert + +soak`. -**R5. Load-flow fit for pre-converted data.** Claim: with sidecars, a fresh -dst==0 load allocates S_bank = whole remaining stage bank; the compressed -source placement `round8(compressed)+8 ≤ S_bank` holds (max single-file -compressed ≪ ~2.3 MB bank); `load_resource` sets poolRemaining := D_PC -automatically; compaction headroom R = S_bank satisfies D46's strict bound with -P_conv = D_PC (D46 already proved the buffers against these sizes). ADDITIONALLY: -for every dst!=0 caller, the patched compressed size + 8 ≤ the caller's buffer -size (gun.c 0x18000/0xBD70/0x17000, front.c wallet 0x17000 / cast screen -0x25000, initmenus.c logo 0x85000, bondview2.c weaponbuf `size0`). Verify: have -the emit pass (or a quick Python pass) print per-file compressed sizes and -cross-check against the D45/D46 constants; re-run `tools_pc/d43_chainbound.py`. -**Falsified if:** any buffer/chain is undersized for its file. +## Task 3 — next asset type (after D51 + soak) -**R6. Reload hazard + the one-line reset.** Claim: `fileSetSize` (ob.c:346-347) -leaves poolRemaining = rom_remaining = post-compaction size; without a reset, a -same-stage reload allocates S_bank' = P_old which can be < round8(compressed)+8 -→ `load_resource` hits the `source − ptrdata < 8` branch → poolRemaining = 0 → -silent load failure. A PORT-guarded `poolRemaining = 0` reset immediately before -`_fileNameLoadToBank` restores fresh-bank semantics every load with identical -steady-state usage. Verify: read `fileSetSize`, `fileIndexLoadToBank`, -`load_resource`; grep consumers of `get_pc_remaining_buffer_for_index` (should -be compaction only) to confirm the reset clobbers nothing. **Falsified if:** -the reset breaks a consumer, or the hazard cannot actually occur (then drop the -reset and say so in D49). +Tbg_*_stanZ stage backgrounds (different format/path than models). SAME +offline pattern (D48): pin the format with Python over ROM ground truth, emit ++ validate offline, serve through the existing load path. PD's +`port/src/preprocess/filebg.c` is the per-field reference (§2.4) — but same +family ≠ identical format; validate every field against GE headers + ROM. -**R7. Region handling.** Claim: sidecars are derived from a specific region ROM; -the generator takes the region, reads `data/ge007..z64`, writes -`data/pcmodels-/`; romdataInit picks the directory matching the ROM it -loaded (same token logic as ROM discovery in `romdataInit`). Verify: read -`build-pc.sh` + `romdataInit`'s ROM discovery; confirm name sets are identical -across regions is NOT required (the generator maps whatever files exist). -**Falsified if:** the build flow can't regenerate sidecars per region, or -romdataInit can't tell which directory matches. +## Verified facts (sidecar spec — ground truth; do NOT re-derive) -**R8. Emission spec completeness.** Claim: the "Verified facts" section below + -D47.13 cover every field of every opcode present in ROM (1,2,4,8,9,10,12,13, -15,18,21,22,23,24), and the round-trip validator (§Task 2 step 1) catches any -miss. Verify: walk each `ModelRoData_*Record` in src/bondtypes.h and confirm an -emission rule exists for every pointer/float/int field; spot-check 2–3 PC -record sizes against a fresh C probe (the Python `PC_REC` table was -probe-verified once — re-confirm it hasn't drifted). **Falsified if:** any -field has no rule, or a PC record size is wrong. - -**Review output:** D49 in §F (8 items adjudicated + evidence), HANDOFF status -updated, committed + pushed. If all CONFIRMED → Session B executes §Task 2. - -## Task 2 — Execution checklist (Session B; only after D49 confirms) - -1. **Emit pass in Python** (`tools_pc/d43_emit.py`, importing/reusing - `d43_convert.py`'s walk + region map). For each of the 512 files, emit the - PC image per D47.13 + Verified facts below: switches (NS×8B LE) → - texconfigs (NT×12B; bswap TextureID u32) → nodes+records in DFS preorder - (PC layout mirroring bondtypes.h; every promoted pointer field = - zero-extended u64 of `0x05|new_off`; Primary/Secondary GDL ptrs remapped but - NOT promoted; BaseAddr emitted 0) → vertex arrays immediately after their - record (normal: bswap s16 x,y,z,index,s,t, bytes @C–F raw; collision: bswap - s16 x,y,z,index + remap LinkedTo u32@8 + bswap CollisionRelatedIndex s16@C / - reserved s16@E; nv==0 with non-null ptr → size up to next object offset, - D47.10) → PointUsage (2×numVertices s16, after op24 collision verts) → - **GDLs LAST, contiguous, visit order, 16B LE slots**: w0' = bswap32(w0); - w1' = bswap32(raw) EXCEPT seg-5 of {G_VTX=0x04, G_SETTIMG=0xFD, - G_LOADBLOCK=0xF3} → remap low24 via the region map; **no LSB set**; emit up - to and including ENDDL. Write each file as `[0x11 0x72] + raw deflate(PC - image)` (Python: `zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, - zlib.DEFLATED, -15)`) into `data/pcmodels-/` plus a - `manifest.csv` (`name,offset,size`) with offsets assigned contiguously in - `file_resource_table` order (parse the quoted names from - `assets/obseg/file_resource_table.inc.c`). - **Round-trip validator (mandatory before trusting output):** re-parse each - emitted image as PC layout — 48B node stride, `PC_REC` record sizes, LE Gfx - slots; assert every promoted pointer equals the expected new offset, every - seg-5 GDL w1 target resolves inside the image, D_PC matches the pass-1 size. - Report per-file compressed sizes (feed R5's cross-check). -2. **Port-layer plumbing in `port/src/romdata.c`** (~60 lines): - - `romdataInit`: if `data/pcmodels-/manifest.csv` exists for the - loaded ROM, reserve the extended region — VirtualAlloc at CART_BASE sized - `romSize + sidecarTotal` (aligned), memcpy the ROM as today, then load the - sidecar blob(s) at `[CART_BASE + romSize, …)` per manifest offsets. Log - count + total bytes. Missing directory → warn and continue (ROM-only - mode = current behavior). - - Extend `romdataCartAddrValid` to accept up to - `CART_BASE + romSize + sidecarTotal`. - - New `void romdataPatchModelSidecars(void)` — one-shot (static flag): for - each `file_resource_table[i]` whose name is in the manifest, set - `hw_address = (u8*)(CART_BASE + romSize + off)` and - `resource_lookup_data_array[i].rom_size = size`. `extern` both arrays from - ob.c (same pattern as the existing `resource_lookup_data_array` extern). -3. **Hook in `load_object_fill_header`** (src/game/objecthandler_2.c, ONE - `#if defined(PORT)` block, D48-class bookkeeping — document as D50): - (a) call `romdataPatchModelSidecars()` (lazy one-shot; obInit has run by - first model load — R4); (b) reset - `resource_lookup_data_array[fileGetIndex(name)].poolRemaining = 0` - immediately before the `_fileNameLoadToBank`/`_fileNameLoadToAddr` call - (R6). No other game-code changes. -4. **Build, run, verify + pixel assert (build it NOW — dev-loop item #1):** - env-gated framebuffer PPM dump every N frames in `port/video.c` - (e.g. `GE_PCDUMP=1` → `build-pc/frame_%05d.ppm`) + `tools_pc/pixcount.py` - (counts non-clear-color pixels; diffs two captures). Milestone: no SIGSEGV - in `modelPromoteNodeOffsetsToPointers`; pixel count rises above a small - threshold and stays up through a 30 s+ soak. If geometry is wrong/missing, - the converted sidecars are inspectable OFFLINE (Python) — that's the point - of Plan B; don't gdb the game first. -5. **Commit checkpoints** (`PC port: phase 2 — …`); record findings D49/D50+ - in §F; update AGENTS.md phase status + this file. Then move on to the next - asset type (Tbg_*_stanZ backgrounds) with the SAME offline pattern — PD's - `port/src/preprocess/filebg.c` is the per-field reference (§2.4). - -## Task 3 — Plan A fallback (only if D49 refutes Plan B) - -D47.13 C converter in `port/src/romdata.c`: two-pass walk+layout then emit, -staged at `F + R − D_PC`, memmove to F; hook after the filedata assignment and -before Switches/Textures/RootNode setup; set `poolRemaining = D_PC` exactly -(never rom_remaining); poolRemaining=0 reset before `_fileNameLoadToBank`. -**Use the corrected guard (D48.4): fail if `D_N64 + D_PC > avail`** — the -staging region overlaps the live N64 image otherwise. The Python emit pass from -Task 2 step 1 is still worth writing first: it becomes the byte-level oracle to -diff the C output against (convert one file in C, compare with Python output). - -## Verified facts (converter spec — ground truth for the emit pass; do NOT re-derive) +**Sidecar serving (D50):** +- Sidecars live in `data/pcmodels-/` = **one concatenated + `pcmodels.bin` + `manifest.csv` (`name,offset,size`, DECIMAL — C parser + uses strtol base 10; rows in file_resource_table.inc.c order)**. NOT + per-file .bin. Regenerate: `python tools_pc/d43_emit.py [ntsc-final| + pal-final|jpn-final|--check-only]` (needs the region ROM in data/). +- RZ format = `[0x11 0x72][raw deflate]`. Python decompress for inspection: + `zlib.decompress(raw[2:], -15)` after checking the 2-byte header. +- pcmodels.c copies the whole blob to `[CART_BASE+romSize, …)` (cart base + 0x10000000, romdataInit reserves romSize+sidecarTotal; + romdataCartAddrValid accepts the extension) and patches + `file_resource_table[i].hw_address = cartBase+romSize+off` + + `resource_lookup_data_array[i].rom_size = C_pc` per manifest row — one-shot, + from `load_object_fill_header` (after obInit has run). The existing + romCopy/decompressdata path then serves PC bytes unmodified and sets + poolRemaining = D_PC. +- **Inspecting a loaded file offline:** find the name in manifest.csv → + decompress that slice → it is exactly what lands at the file base in RAM + (verified byte-for-byte for PlegalpageZ). GDLs are at the tail of the image + in visit order; `x/Nwx ` in gdb compares directly. **Format / layout:** -- All 512 C*/G*/P*Z model files (Tbg_*_stanZ backgrounds are a different format/path — later) decompress fine; each file starts directly with the switches array (no in-file header — NS/NT come from the exe `ModelFileHeader` record via `objheader->numSwitches/numtextures`). -- File layout: `[switches NS×4B][textures NT×12B][root node @ 4NS+12NT]…`; all embedded pointers are vma `0x05xxxxxx` (mask 0xFFFFFF). -- **Tiling invariant verified, 0 failures in all 512 files** (`tools_pc/d43_invariants.py`): `[0,D)` is exactly tiled by {switches, textures, nodes(24B), records(RSZ per opcode), vertex arrays(nv×16B), collision arrays(nc×16B), PointUsage (exactly numVertices s16 entries — +2 cases are pure alignment padding), GDLs (each extends to the next GDL start or D; 8-byte aligned, inter-GDL gaps copied as NOOP)}. Records INTERLEAVE with vertex arrays in character bodies — do not assume "all records before bulk". -- **ModelNode = 24B N64** (u16 op + 2 pad + 5×4B ptrs: Data@4, Parent@8, Next@0xC, Prev@0x10, Child@0x14) / **48B PC**. Older "20B/40B" notes are wrong — verified against raw hex stride 0x18. -- **Opcodes present: 1,2,4,8,9,10,12,13,15,18,21,22,23,24** (HEADER, GROUP, DL, LOD, BSP, BOX, GUNFIRE, SHADOW, INTERLINK, SWITCH, GROUPSIMPLE, DLPRIMARY, HEADPH, DLCOLLISION). op22 N64: numVertices s32@0, Vertices@4, Primary@8, BaseAddr@0xC (size 0x10); only 5 files have it. -- Record sizes N64/PC: OP01 0x10/0x18, OP02 0x1C/0x28, OP04 0x14/0x28, OP08 0x10/0x18, OP09 0x24/0x30, OP10 0x1C/0x1C, OP12 0x28/0x30, OP13 0x20/0x30, OP15 0x1C/0x1C, OP17 0x20/0x28, OP18 0x08/0x10, OP21 0x14/0x14, OP22 0x10/0x20, OP23 0x02/0x02, OP24 0x20/0x40 (compiler-verified). Field layouts in `src/bondtypes.h` (ModelRoData_*Record); read N64 fields from packed offsets, emit via the real PC layout — don't hand-compute PC offsets. -- **f32 fields are big-endian in ROM** — bswap all f32 (record origins, BoundingVolumeRadius, Scale, Min/MaxDistance) and every s16 in vertex arrays; also bswap32 the TextureID u32 in the texture table (texLoad reads `*updateword & 0xffff` as the texture number). -- **u16/s16 record scalars are big-endian too (D49 amendment)** — bswap16 every u16/s16 scalar field in every record (AnimPart, MatrixIndex, JointID, MatrixIDs, Group1/2, RwDataIndex, op4 numVertices@0x10, op24 nv/ncv/ModelType/RwDataIndex, …); padding/reserved fields may be zeroed. Data-verified: these hold small BE values (JointID 1–11, nv ≤ 73) that raw emission would corrupt ×256 on an LE read. -- **Vertex arrays must stay 16B-stride** (GDL gSPVertex hardcodes 16B/vertex; propobj.c copies with sizeof(Vertex); model.c:4360 allocates numVertices×4 words). bswap every s16 field BE→LE. Main Vertices array is pure data; **CollisionVertices carry LinkedTo@8 = vma of a ModelNode** — remap to the new node offset (PROMOTE promotes them after rebase). -- **GDLs:** 8B BE word pairs → 16B LE slots: `w0=bswap32(rom_w0); w1=bswap32(rom_w1)` — **leave w1 raw, NO LSB set** (ROM convention; fast3d's extended seg_addr case handles unmarked segmented addresses — D47.13). GE uses STANDARD libultra GBI numbering (include/PR/gbi.h); opcodes live in the TOP byte of w0. **Opcode-aware remap: only {G_VTX=0x04, G_SETTIMG=0xFD, G_LOADBLOCK=0xF3} carry addresses in w1** (seg-5 → remap low24 via region map). All other opcodes get bswap-only — G_TRI4 (GE extension 0xB1) w1 is 4-bit vertex-index data, NOT an address (D47.6); same for TRI1/TEXTURE/SET*MODE/CLEARGEO/SETGEO/syncs. **G_VTX w0 is GBI1-style** (`04<<24 | dst<<16 | 16·n`, dst=((n−1)<<4)|v0; all 2826 have v0=0, n≤16) — bswap only, no field remap (D47.5). ROM scan: all 2805 seg-5 VTX refs have even offsets → lossless; no nested G_DL in model files. -- **Marker format:** top byte of w0 = 0xC0; type = `w0 & 7` (only 0/2/3/4 exist); texnum = `w1 & 0xfff`; min = top byte of w1. texLoadFromGdl dispatches on the top byte — PC fix: `(u8)(in->words.w0 >> 24)`. Type-0/1 markers with a valid tex are SKIPPED by the game (no expansion); the rest expand per D45's K table, which D46 verified as a true worst-case bound (no LUT textures exist; 99% of textures have maxlod=0 → TileLods emits nothing; state dedup resets per GDL via sub_GAME_7F0CC4C8). -- **seg-5 VTX offsets must be remapped** — all 2805 point into op4/op22 vertex arrays that move during re-layout (PexplosionbitZ: op4 with numVertices=0 but GDL loads 16 verts from offset 0x98 — the GDL's VTX count is authoritative, not the record field). -- PROMOTE contract: emit every promoted pointer as zero-extended u32 (low word of the 8B slot, high word 0); `PROMOTE` (`(u32)var + diff`, vma 0x5000000) then works unmodified. Fields promoted per opcode (model.c switch): all nodes Data/Parent/Next/Prev/Child; OP01 FirstGroup; OP02 ChildGroup; OP04 Vertices; OP08 Affects; OP09 left/rightChild; OP18 Controls; OP24 Vertices/CollisionVertices/PointUsage/CollisionVertices[i].LinkedTo. **NOT promoted:** GDL Primary/Secondary (stay raw 0x05xxxxxx), BaseAddr (game overwrites with fileramaddr). +- All 512 C*/G*/P*Z model files decompress fine; each file starts directly with + the switches array (no in-file header — NS/NT come from the exe + `ModelFileHeader` record via `objheader->numSwitches/numtextures`). +- PC file layout: `[switches NS×8B][texconfigs NT×12B][root node @ 8NS+12NT]…`; + all embedded pointers are vma `0x05xxxxxx` (mask 0xFFFFFF). N64 was + `[NS×4][NT×12][root @ 4NS+12NT]`. +- **Tiling invariant verified, 0 failures in all 512 files** + (`tools_pc/d43_invariants.py`): the image is exactly tiled by {switches, + textures, nodes(48B PC), records(PC_REC per opcode), vertex arrays(nv×16B), + collision arrays(nc×16B), PointUsage (numVertices s16 entries), GDLs (each + extends to the next GDL start or D)}. Records INTERLEAVE with vertex arrays + in character bodies — do not assume "all records before bulk". +- **ModelNode = 24B N64 / 48B PC** (u16 op + 2 pad + 5 ptrs: Data@4/8, + Parent, Next, Prev, Child). Record sizes N64/PC: OP01 0x10/0x18, OP02 + 0x1C/0x28, OP04 0x14/0x28, OP08 0x10/0x18, OP09 0x24/0x30, OP10 0x1C/0x1C, + OP12 0x28/0x30, OP13 0x20/0x30, OP15 0x1C/0x1C, OP17 0x20/0x28, OP18 + 0x08/0x10, OP21 0x14/0x14, OP22 0x10/0x20, OP23 0x02/0x02, OP24 0x20/0x40 + (compiler-verified by `tools_pc/d43_layoutprobe.c`). Opcodes present: + 1,2,4,8,9,10,12,13,15,18,21,22,23,24. +- **Endianness:** f32 fields BE in ROM → bswap; every u16/s16 record scalar + BE → bswap16; vertex-array s16s BE → bswap16 (x,y,z,index,s,t); TextureID + u32 → bswap32. Padding/reserved zeroed. +- **Vertex arrays stay 16B-stride** (GDL gSPVertex hardcodes it). Collision + vertices carry LinkedTo@8 = vma of a ModelNode — remapped. +- **GDLs:** N64 8B BE word pairs → PC 16B LE slots: `w0=bswap32(rom_w0); + w1=bswap32(rom_w1)`; **no LSB set** (fast3d's seg_addr handles unmarked + + raw-0x05). Opcode-aware remap: only {G_VTX=0x04, G_SETTIMG=0xFD, + G_LOADBLOCK=0xF3} carry addresses in w1; seg-5 low24 remapped via the region + map. **0xB1 (G_TRI4) w1 is packed 8-bit vertex indices — NEVER an address.** + G_VTX w0 is GBI1-style (`04<<24 | dst<<16 | 16·n`) — bswap only. +- **Marker format:** top byte of w0 = 0xC0; type = `w0 & 7`; texnum = + `w1 & 0xfff`. texLoadFromGdl expands markers into RDP commands (that's the + compaction growth); default case copies the full 16B slot as-is — **the + sidecar must contain final correct w1 values** (no runtime remap in + texLoadFromGdl). +- **PROMOTE contract:** promoted pointers emitted as zero-extended u64 with + low32 = VMA `0x05xxxxxx`; `PROMOTE` (`(u32)var + diff`, vma 0x5000000) works + unmodified. GDL Primary/Secondary NOT promoted (raw VMAs); BaseAddr=0 (game + overwrites with fileramaddr). +- **texCopyGdls contract (D50.6):** on PC it must copy the FULL 16-byte slot + (`*arg1 = *arg0` under #ifdef PORT) — the N64 `long long` idiom copies only + w0 on x86-64 and poisons the compaction scratch. texLoadFromGdl then reads + the scratch and writes back at the original GDL offsets; final fileSetSize + shrinks the bank chunk. -**Load path (single hook point):** -- ALL model loads go through `load_object_fill_header()` (objecthandler_2.c:97): dst≠0 → `_fileNameLoadToAddr(name,0,dst,size)` (region = caller's bytes — see D45 for the grown per-region sizes); else `_fileNameLoadToBank(name,0,0x100,4)`. **Fresh-load semantics (D47.2-4):** `fileIndexLoadToBank` takes ALL remaining MEMPOOL_STAGE space (`S_bank`) only when poolRemaining==0; then F := alloc(S_bank), R := S_bank, load_resource decompresses D into F and sets P := decompressed size (D_N64 from ROM; **D_PC directly for sidecars — no fixup needed**). At compaction: **delta = R − P** (positive on fresh loads — the GDL mirror moves FORWARD). `fileSetSize(reallocate=1)` → `mempAddEntryOfSizeToBank` rewinds the bank cursor to the post-compaction size, giving the tail back. -- **Reload hazard + PC fix (D47.4, needed by BOTH plans):** fileSetSize leaves P=R=post-compaction size, so a same-stage reload would alloc(P_old) which can be too small for the compressed source placement → `load_resource` sets poolRemaining=0 → silent failure. PORT-guarded reset of poolRemaining=0 before the load call forces fresh-bank semantics every load (staging space = whole remaining bank; steady-state usage identical). -- **Plan A only (fallback):** the fixup must set ONLY poolRemaining (= D_PC exactly), never rom_remaining; stage at F+R−D_PC with guard `D_N64 + D_PC ≤ avail` (D48.4); hook after filedata assignment, before Switches/Textures/RootNode setup + sub_GAME_7F075A90 (PROMOTE walks PC ModelNode stride — the image must already be re-laid-out). +**GE opcode space (include/PR/gbi.h, non-F3DEX numbering — verified D51):** +- DMA: G_SPNOOP=0, G_MTX=1, G_MOVEMEM=3, G_VTX=4, G_DL=6, G_SPRITE2D_BASE=9. +- IMM (G_IMMFIRST=−65): TRI1=0xBF, CULLDL=0xBE, POPMTX=0xBD, MOVEWORD=0xBC, + TEXTURE=0xBB, SETOTHERMODE_H=0xBA, SETOTHERMODE_L=0xB9, ENDDL=0xB8, + SETGEOMETRYMODE=0xB7, CLEARGEOMETRYMODE=0xB6, LINE3D=0xB5, RDPHALF_1=0xB4, + RDPHALF_2=0xB3, TRI2=0xB2, **G_TRI4(GE)=0xB1** (gbi_extension.h). +- RDP pass-through: G_SETTIMG=0xFD, G_LOADTXC=0xFE, G_SETCIMG=0xFF, + **G_TEXRECT=0xE4 / G_TEXRECTFLIP=0xE5** (GE-specific values — NOT libultra's + 0x46/0x45), plus VI2D=0xF5 etc. +- `gSPSegment(pkt,seg,base)` = `gMoveWd(pkt,G_MW_SEGMENT,seg*4,base)` → + w0=(0xBC<<24)|(seg*4<<8)|6, w1=data. fast3d gfx_sp_moveword stores data as-is + when ≥0x800000 else +0x80000000. +- Main DL is game-built per frame with gSP* macros (16B slots) — NOT from a + sidecar. Segment table: seg0=0, seg1=0x7029F800-ish, seg3=render_pos, + seg4=runtime vtx buffer, seg5=file base (set per object). -**Compaction contract (sub_GAME_7F0762E0, objecthandler_2.c:24-83) — the tightest constraint (applies to both plans):** -- It mirrors the GDL region [first_gdl_off, P_conv) into the caller-region's tail headroom (texCopyGdls to [A+R−P_conv+g, A+R)), then rewrites each GDL in place via texLoadFromGdl (expands texture markers into RDP commands), writing back starting at the original first-GDL offset; final fileSetSize = end of last rewritten GDL (= P_conv + 16·M_actual). **D46: fit and no-clobber are the SAME condition — R ≥ P_conv + 16×Σ(K_t−1) worst case — and all buffers satisfy it (see §F/D46 for the strict bound + the two cast-screen corrections).** -- Per-GDL `count` = byte distance between CONSECUTIVELY VISITED gdls → **GDLs must be packed contiguously at the tail of the PC image, in modelIterateDisplayLists visit order.** -- `modelIterateDisplayLists` (model.c:6254) **MUTATES the tree while walking**: LOD → `node->Child = LOD.Affects`; SWITCH → `node->Child = Switch.Controls`; BSP → `modelApplyReorderRelationsByArg(node,TRUE)` splices siblings (leftChild group before rightChild). Visit order per DL node: Primary, then Secondary (if non-null and ≠ previous), then DFS Child/Next/Parent. -- `tools_pc/d43_gdlorder.py` implements this exact simulation to verify visit-order adjacency — all 512 files pass (total GDLs 2301). -- Primary/Secondary are NOT promoted (raw 0x05xxxxxx), so modelNodeReplaceGdl's `Primary == find` comparisons work unchanged; after compaction Primary = replacementgdl (raw value, arbitrary parity). - -**Render-time GDL resolution:** -- The game emits `gSPDisplayList(gdl, rwdata->gdl)` with w1 = raw 0x05xxxxxx; segment 5 (SPSEGMENT_MODEL_COL1) is set to BaseAddr (= file base) by the preceding gSPSegment. fast3d's `seg_addr` (port/fast3d/gfx_pc.cpp:2282) resolves raw 0x05xxxxxx via `segmentPointers[5] + (w1 & 0xFFFFFF)` — **DONE, committed.** -- Segments 3/4 (MTX table / vertex buffer) are set by the game per-render via gSPSegment ✓. BG segments (13/14/15) belong to the background path — later task. +**Load path / compaction (unchanged by D50 — see D47/D46/D48 for full detail):** +- ALL model loads go through `load_object_fill_header()`; dst≠0 → + `_fileNameLoadToAddr` (caller's buffer), else `_fileNameLoadToBank` (fresh + bank when poolRemaining==0 — the hook resets it every time). +- Compaction: texCopyGdls mirrors [G,D) to tail scratch [B−D+G,B); + texLoadFromGdl rewrites each GDL in place with marker expansion; final size + = D + 16·Σ(K_t−1) worst case (P_est model — the emit pass's cross-checks + encode it for every dst!=0 buffer and chain). ## Helper scripts (tools_pc/, tracked; run from repo root) -- `d43_convert.py` — **reference layout + validator (D47):** DFS walk with LOD/SWITCH rewiring, region map, per-opcode record/vertex/GDL analysis, remap checks; runs on all 512 ROM files (ALL CLEAN). Lays the foundation for the emit pass (Task 2 step 1) — import/reuse its walk + region map. -- `d43_invariants.py` — full ownership tiling check (0 failures / 512). -- `d43_gdlorder.py` — exact visit-order simulation; all 512 pass. -- `d43_sizes.py` / `d43_sizes2.py` — decompress all 512, report N64/PC sizes + per-file worst-case P_final. -- `d43_walk.py` / `d43_fullwalk.py` / `d43_tree_dump.py` — tree walks, opcode histogram. (Header names lack the C/G/P prefix and trailing Z: file `CcamguardZ` → header `camguard`; all header files are literally named `modelFileHeader.inc.c` in per-model dirs.) -- `d43_gdldump.py` / `d43_gdlhist.py` / `d43_gdlseq.py` — GDL command dumps/histograms/sequences. -- `d43_seg5.py` / `d43_seg5ops.py` / `d43_seg5vtx.py` / `d43_seg5vtx2.py` — seg-5 VTX reference analysis (all 2805 point into op4/op22 vertex arrays → must remap). -- `d43_cover.py` — verifies every seg-5 VTX + SETTIMG target falls inside a modeled object (2804/2805; the one exception is PexplosionbitZ's orphan array at 0x98 — handled by the zero-vtx rule). -- `d43_lutscan.py` — proves NO LUT textures (formats 9-12) and maxlod distribution across all 512 files (D46). -- `d43_chainbound.py` — computes D46's strict bound for every buffer/chain vs the PC region sizes. -- `d43_decode.py` / `d43_layout*.py` / `d43_pointusage.py` / `d43_vtxfmt.py` — earlier investigation (format pinning). -- Gotcha: MODELFILEHEADER's name is field a[0] of the macro args (NOT the filename). +- `d43_emit.py` — **THE emit pass (D50.1):** converts all 512 files, validates + round-trip + tiling + buffer fits, writes pcmodels.bin + manifest.csv. + `--check-only` runs validation without writing. First stop for any + "sidecar looks wrong" question. +- `d43_layoutprobe.c` — compiler probe that re-verifies every PC record/node/ + header size (run with the exact CMake flags; compare against d43_emit.py's + tables after any bondtypes.h change). +- `d43_convert.py` — reference layout + validator (D47); foundation of the + emit pass. +- `d43_invariants.py` / `d43_gdlorder.py` / `d43_sizes*.py` / `d43_chainbound.py` + / `d43_cover.py` / `d43_lutscan.py` / `d43_seg5*.py` / `d43_gdl*.py` — the + investigation suite (all pass on NTSC). +- Gotcha: MODELFILEHEADER's name is field a[0] of the macro args (NOT the + filename); nsnt lookup keys by header symbol (`armourguard`), not file base + name (`CarmourguardZ`). ## Dev-loop speedups (priority order) -1. **Frame capture + pixel assert** — NOW (Task 2 step 4): env-gated PPM dump every N frames in `port/video.c` + `tools_pc/pixcount.py` (non-clear-color pixel count / diff two captures). Turns "the scene contains actual model geometry" into an assert, makes the 30 s soak scriptable, and enables screenshot diffs against Mupen64Plus with the verified No-Intro dump as ground truth. -2. **`tools_pc/sym.sh`** — tiny wrapper: takes a PC or rel offset, adds image base `0x140000000`, runs `addr2line -e build-pc/ge007.x86_64.exe -f -C`. One command per crash; re-verify the image base with `info address` after any link-layout change. -3. **`tools_pc/romdump.py`** — Python ROM/asset inspector: parse ModelFileHeader/ModelNode trees using the bondtypes.h layout, print opcode sequences / node links / Data records for a given cart address; also re-parse SIDECAR files as PC layout (post-Plan B this is the first stop for "geometry looks wrong"). -4. D44 (automatic backtraces) — DONE, committed. +1. **Frame capture + pixel assert** — NOW (Task 2): env-gated PPM dump every N + frames in `port/video.c` + `tools_pc/pixcount.py`. Turns "the scene contains + actual model geometry" into an assert and makes the 30 s soak scriptable. +2. **`tools_pc/sym.sh`** — tiny wrapper: PC or rel offset → add image base + 0x140000000 → `addr2line -f -C`. Re-verify the image base with `info + address` after any link-layout change. +3. Sidecar inspection is now trivial offline (manifest + zlib, see Verified + facts) — don't gdb the game for "geometry looks wrong" in a model file. +4. D44 automatic backtraces — DONE; `ge007.crash.log` is the first stop. ## PD port as standing reference (always available) -`PD_PORT_CHECKOUT` is the Perfect Dark PC port — same Rare engine -family, and the **standing reference/guidance source** for this project (noted in +`PD_PORT_CHECKOUT` is the Perfect Dark PC port — same Rare +engine family, the **standing reference** for this project (noted in AGENTS.md). Consult it whenever a remaining work item has a PD analogue: -- **Phase 2 asset conversion:** `port/src/preprocess/` (~4,100 lines) — per-ROM-segment - N64→PC layout converters hooked from their `romdata.c`. `filemodel.c` = D43 near-analogue - (same vma 0x5000000; PD converts at LOAD time — we convert OFFLINE for speed, same field - rules apply); `filebg.c` (expected follow-on), `segaudio.c` (cross-check for our D37 - bank-tree fixup), `filelang.c`, `filetiles.c`, `filepads.c`, `filesetup.c`, `gbi.c`, - `segfonts.c`. -- **Phase 3/4 (copy-and-adapt; our stubs were scaffolded for this):** `port/src/mixer.c` - (722 lines vs our 31 — libaudio→SDL mixer), `input.c` (1,551 vs 48 — SDL_GameController - backend with hotplug/rumble/keyboard; remap buttons to GE's scheme), `fs.c` (294 vs 81 — - file-backed save dir for Phase 4 EEPROM/PFS). -- **Caveats:** copy port-layer files only (their game code was modified for PC); same - family ≠ identical format — validate every conversion per-field against GE headers + ROM - ground truth. Full audit: `docs/PCPortResearch.md` §2.4. +- **Phase 2 asset conversion:** `port/src/preprocess/` (~4,100 lines) — + per-ROM-segment N64→PC layout converters hooked from their `romdata.c`. + `filemodel.c` = D43 near-analogue (PD converts at LOAD time — we convert + OFFLINE for speed, same field rules apply); `filebg.c` (expected follow-on), + `segaudio.c`, `filelang.c`, `filetiles.c`, `filepads.c`, `filesetup.c`, + `gbi.c`, `segfonts.c`. +- **Phase 3/4 (copy-and-adapt; our stubs were scaffolded for this):** + `port/src/mixer.c` (722 lines vs our 31), `input.c` (1,551 vs 48 — + SDL_GameController backend; remap buttons to GE's scheme), `fs.c` (294 vs 81 + — file-backed save dir). +- **Caveats:** copy port-layer files only (their game code was modified for + PC); same family ≠ identical format — validate every conversion per-field + against GE headers + ROM ground truth. Full audit: §2.4. ## Standing procedure (you will hit more of these) -Every ROM-serialized struct with a pointer field faults the same way once you reach more -asset loading. The D32 fix procedure (doc §H): at the fault, `ptype /o `; if a -pointer field's offset/size diverges from its N64 offset comment, change it to `u32` + cast -at the use sites; verify the load-time rebase yields valid V1 addresses (< 0x80000000); -**also check the ROM bytes' endianness per-field (D33 rule: u32 bswap32 / u16 bswap16 / u8 -identity) and any fixup loop's pointer stride on x86-64**; for packed struct trees use the -D37 re-layout pattern; for address-arithmetic idioms over exe symbols (D39/D42 class) guard -a PC branch that reproduces the N64 32-bit value exactly. Rebuild; confirm boot advances. -Log each as D3x in §F and note it in AGENTS.md phase status. **For asset FORMAT work, -prefer the offline pattern (D48): pin the format with Python over the ROM ground truth, -emit + validate offline, serve through the existing load path — runtime C conversion only -as fallback.** +Every ROM-serialized struct with a pointer field faults the same way once you +reach more asset loading. The D32 fix procedure (doc §H): at the fault, +`ptype /o `; if a pointer field's offset/size diverges from its N64 +offset comment, change it to `u32` + cast at the use sites; verify the +load-time rebase yields valid V1 addresses (< 0x80000000); **also check the +ROM bytes' endianness per-field (D33 rule: u32 bswap32 / u16 bswap16 / u8 +identity) and any fixup loop's pointer stride on x86-64**; for packed struct +trees use the D37 re-layout pattern; for address-arithmetic idioms over exe +symbols (D39/D42 class) guard a PC branch that reproduces the N64 32-bit value +exactly. Rebuild; confirm boot advances. Log each as D5x in §F and note it in +AGENTS.md phase status. **For asset FORMAT work, prefer the offline pattern +(D48): pin the format with Python over the ROM ground truth, emit + validate +offline, serve through the existing load path — runtime C conversion only as +fallback.** Also: any code that copies Gfx slots must use full-struct copies on +PC (D50.6) — never the `long long` union member. ## Environment (do not rediscover these) -- MSYS2/MinGW tools are in `/c/msys64/mingw64/bin/` — **not** on PATH. Prefix every shell: - `export PATH="/c/msys64/mingw64/bin:$PATH"`. -- Build: `./build-pc.sh ntsc-final` (incremental; full rebuild only when `port/shim/` - changes; a *clean* full build is ~5 s on the 16-core dev box). ROM is at - `data/ge007.ntsc-final.z64` (= `baserom.u.z64`, byte-identical). -- The ROM is **mapped at cart base 0x10000000** on PC (`romdataInit`), so `&Symbol` - values (e.g. `_sfxctlSegmentRomStart = 0x102EBDE0`) are directly dereferenceable host - pointers into the ROM image; `romCopy()` copies from those cart addresses. All game RAM - lives in the V1 view at 0x70xxxxxx (< 4 GiB) — s32 truncation of game-RAM pointers is - lossless (this is why D39 needed no changes outside image_bank.c). -- ROM provenance (verified 2026-08): byte-identical to the No-Intro good dump - `GoldenEye 007 (U) [!].z64` (kept in `data/`). MD5 - `70c525880240c1e838b8b1be35666c3b`, SHA-256 - `2cdcec8a9f0cb6e36337f3ee39d8ad105dc8afa6ba1c02d466e8f5b771f9a162`, 0xC00000 bytes, - country 'E' (US). Header CRCs DCBC50D1/09FD1AA3 re-verify with `tools_pc/romverify.exe`. -- gdb: **launch** mode only (`gdb -batch -ex "handle SIGSEGV stop" -ex run …`). Symbolicate - offline with `addr2line -e build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base - is `0x140000000` (re-verify with `info address` after a rebuild). Debug scripts live in - `build-pc/` (`d37_tree.py`, `d39_*.gdb`, `d41_probe.gdb`, …); the D43 investigation suite - is in tracked `tools_pc/`. -- **Capstone gotcha (D47.1):** the ROM is BIG-ENDIAN MIPS — always disassemble with - `CS_MODE_MIPS32 | CS_MODE_BIG_ENDIAN`. Default (LE) mode produces garbage that looks - plausible; an earlier session's disassembly was all LE and had to be redone. -- Crash log: `ge007.crash.log` (repo root) now holds Phase 1 + a `BACKTRACE:` - section (**D44** fixed). First stop for any fault; symbolicate with - `addr2line -e build-pc/ge007.x86_64.exe -f -C `. Frames past the - true chain may be stale — sanity-check they fall inside `.text`. -- `objdump -d --disassemble=` / `objdump -s -j .data` on build-pc/ge007.x86_64.exe is a - fast way to check what the compiler actually emitted. -- Many init functions use a fake RBP — compute stack offsets from the **entry RSP**, not RBP. -- Python on this box: no f-strings with nested quotes (< 3.12) — use `%` formatting. +- MSYS2/MinGW tools are in `/c/msys64/mingw64/bin/` — **not** on PATH. Prefix + every shell: `export PATH="/c/msys64/mingw64/bin:$PATH"` (build-pc.sh needs + it for cmake too). +- Build: `./build-pc.sh ntsc-final` (incremental; full rebuild ~5 s on the + 16-core dev box). ROM at `data/ge007.ntsc-final.z64` (= `baserom.u.z64`, + byte-identical to the No-Intro good dump, MD5 + `70c525880240c1e838b8b1be35666c3b`). +- **Sidecars are gitignored (data/).** If `data/pcmodels-ntsc-final/` is + missing on a fresh checkout, regenerate: `python tools_pc/d43_emit.py + ntsc-final`. The game boots without them (warn + ROM-only mode) but model + loads fail. +- The ROM is **mapped at cart base 0x10000000** on PC; sidecar image follows + at [CART_BASE+romSize, …). All game RAM lives in the V1 view at 0x70xxxxxx + (< 4 GiB) — s32 truncation of game-RAM pointers is lossless. +- gdb: **launch** mode only (`gdb -batch -ex "handle SIGSEGV stop" -ex run …` + — attach fails with error 87). Symbolicate offline with `addr2line -e + build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base + 0x140000000. **Post-mortem memory dumps of game RAM can fail** ("Cannot + access memory") — dump at a live breakpoint instead (hit this with the main + DL in D51). +- Many init functions use a fake RBP — compute stack offsets from the **entry + RSP**, not RBP. +- Python on this box: no f-strings with nested quotes (< 3.12) — use `%` + formatting. +- Untracked scratch at repo root (all.txt, b3.c, btest*.c, buildtest.txt, + err.txt, gcout.txt, preproc.txt, vsize.*) + `tools_pc/__pycache__/` are + session scratch — do NOT commit them; use explicit pathspecs with git add. ## Definition of done -- ~~**Session A:** D49 committed~~ — **DONE**: 8/8 CONFIRMED with evidence, - HANDOFF updated, committed. -- **Session B (the milestone proper):** stage objects load without faulting and the - rendered scene contains actual model geometry per the pixel assert (not just the clear - color / background), stable under a 30 s+ soak. Committed + pushed to `origin/master` - (`PC port: phase 2 — `). After that, Phase 2 makes it look right (fast3d CC/RM - correctness vs `gmain.s` — GE-specific, no PD analogue) and Phase 3 adds sound + input, - largely copy-and-adapt from the PD port. +- **Session C:** D51 diagnosed + fixed (finding in §F, crash moves past frame + 5), pixel assert built, 30 s+ soak clean with real geometry on screen. + Committed + pushed (`PC port: phase 2 — `). After that: Phase 2 makes + it look right (fast3d CC/RM correctness vs `rsp/graphics/gmain.s` — + GE-specific ground truth, no PD analogue), next asset type (Tbg_*_stanZ + backgrounds), and Phase 3 adds sound + input (copy-and-adapt from the PD + port). diff --git a/docs/PCPortResearch.md b/docs/PCPortResearch.md index 576c3776..b456fe29 100644 --- a/docs/PCPortResearch.md +++ b/docs/PCPortResearch.md @@ -1664,6 +1664,115 @@ missing rule — but Task 2's emit pass must bswap16 every u16/s16 record scalar (padding/reserved may be zeroed), and the HANDOFF Verified facts should be amended to say so. +**D50 (RESOLVED — Plan B executed: offline sidecars + port plumbing; boot advances to first model GDL execution)** + +Execution session ran HANDOFF Task 2 end-to-end. All 512 NTSC model files +converted offline and served through the existing load path; frames render +and model display lists execute. Sub-items (all `#ifdef PORT`-guarded or +port-layer only; N64 build untouched): + +1. **D50.1 Emit pass (`tools_pc/d43_emit.py`, tracked).** Per file: +decompress the N64 image from the ROM (filelist row); build the node map; +run the EXACT `modelIterateDisplayLists` visit simulation (LOD/SWITCH +rewire, BSP splice — d43_gdlorder logic) → gdl_seq; layout `[switches +NS×8][texconfigs NT×12][DFS nodes 48B + records PC_REC + vertex arrays] +…[GDLs packed contiguously in gdl_seq order, 16B per N64 slot]`; byte-exact +emit (bswap32/16; promoted pointers → zero-extended u64 `0x05xxxxxx` VMAs; +GDL Primary/Secondary raw VMAs, NOT promoted; BaseAddr=0); **round-trip +re-parse validation of every field against the N64 source + region tiling +(no gaps)** — 512/512 pass. Compression: `0x11 0x72` + raw deflate level 6 +(`zlib.compressobj(6, DEFLATED, -15)`); escalate to 9 only on a dst!=0 fit +violation (none occurred). Cross-checks all pass: per-file dst!=0 buffers +(`round8(C)+8 ≤ buf` AND `D+round8(C) ≤ buf` — inflate overlap safety), G* +hand-weapon worst case vs 0xF000, cast/title/bondview chain cumulative +P_final bounds, totals (Σ round8(C) = 1,277,088 B, matches D49). Output: +**single concatenated image** `data/pcmodels-/pcmodels.bin` +(sidecars at 16-aligned offsets) + `manifest.csv` (`name,offset,size`, +decimal — the C parser uses strtol base 10; file_resource_table.inc.c +order). Deviation from D48's per-file-sidecar sketch: one blob + manifest; +pcmodels.c copies the whole image to `[CART_BASE+romSize, …)` and patches +`hw_address = cartBase+romSize+off`. Regenerate: +`python tools_pc/d43_emit.py [ntsc-final|pal-final|jpn-final|--check-only]` +(needs the region ROM in data/; sidecars are gitignored with data/). +2. **D50.2 Port plumbing (`port/src/pcmodels.c` + `port/include/pcmodels.h`, +new; romdata.c/h extended).** `pcmodelsReserveSize(romImg)` derives +`data/pcmodels-/` from the ROM country byte (+0x3E), parses the +manifest, returns total bytes (0 → warn, ROM-only mode); romdataInit +reserves `romSize + sidecarTotal` at CART_BASE and +`romdataCartAddrValid` accepts the extension; `pcmodelsLoadSidecars( +cartBase, romSize)` copies the blob; `pcmodelsPatchTable()` — one-shot, +called from `load_object_fill_header` (hook block, objecthandler_2.c) after +obInit has run — redirects every manifest row's +`file_resource_table[i].hw_address` + sets +`resource_lookup_data_array[i].rom_size` to the PC compressed size. The same +hook block resets `poolRemaining = 0` for dst==0 loads (closes the D48.3/R6 +reload hazard). Boot log: `[INFO] pcmodels: table patched (512 model +entries)`. +3. **D50.3 Language banks (runtime C fixup).** Banks carry a big-endian +offset table; `romdataFixupLangBank(blob, decompressedSize)` decodes it in +place. Called from language.c after each of the 7 langInit loads and lazily +per-id in the `langGetJpnCharPixels` paths (idempotent via poolRemaining +check). +4. **D50.4 Fonts (runtime C fixup).** `load_font_tables` PC branch: +allocate `romdataFontPcSize()` (PC C layout of struct font: kerning[169] + +chars[94] + glyph pixel data), romCopy the N64 size, then +`romdataFixupFont` re-lays out, shifting the pixel block below the expanded +char array; `pixeldata` fields left as relative offsets — the existing +`pixeldata += base` loop promotes them. +5. **D50.5 Legal-screen UB exposure (front.c).** +`constructor_menu00_legalscreen` reads an uninitialized pointer at a lookat +call — on N64 the register happens to hold a readable address and the +result is zeroed by `* 0.0f` anyway; the x86-64 compiler folds the UB read +to NULL → fault. PORT branch seeds it with `legalpage_text_array` (the +value assigned a few lines later); identical lookat. +6. **D50.6 `texCopyGdls` copies only w0 on PC — first model-render crash +(RESOLVED).** `Gfx` is a union whose last member is `long long int +force_structure_alignment`: 8 bytes on N64 (= the whole slot), but on x86-64 +the slot is 16 bytes, so `arg1->force_structure_alignment = arg0->…` +copies only the low half. Compaction flow (sub_GAME_7F0762E0): +texCopyGdls mirrors the GDL block `[G,D)` to tail scratch `[B−D+G,B)`, then +texLoadFromGdl reads the scratch and writes expanded output back at +`[G,…)` via full-slot `*(out++)=*(in++)` — propagating the scratch's stale +w1s into the final GDLs. **Byte proof:** PlegalpageZ (NS=0, NT=5, +D=0x2638) loaded at 0x7012EA38; first 64 RAM bytes match the decompressed +sidecar exactly; the executed GDL at file offset 0x2488 has w0s matching +the sidecar exactly (0xB10000BA/0xE7000000/0xFD900000/0xE6000000) but RAM +w1s = 0x50362B58/0x66D73339/0xACAAB819/0x55BDF769 (stale mempool contents; +pad2 dwords also nonzero) vs sidecar w1s 0x0000A898/0/0x050012C8/0 — the +garbage G_SETTIMG w1 → `seg_addr` → OOB read in `import_texture_rgba16`. +Fix: `*arg1 = *arg0;` under `#ifdef PORT` (tex.c). Audit: tex.c is the only +`force_structure_alignment` use in game code (model.c:1510 is an unrelated +local 8-byte union); every other Gfx copy is full-struct. Post-fix: +PlegalpageZ's sub-DLs execute to completion; crash moves on (D51). + +**D51 (OPEN — current blocker: `import_texture_i8` SIGSEGV via main-DL G_TEXRECT lazy tile upload)** + +Post-D50 run: frames 1–4 log, then SIGSEGV on frame 5 in +`import_texture_i8` (gfx_pc.cpp:782, `addr[i]`) ← `import_texture(i=0, +tile=0)` ← `gfx_sp_tri1(128,129,131,is_rect=true)` ← +`gfx_draw_rectangle(236,92,268,140)` ← `gfx_dp_texture_rectangle` ← +`G_TEXRECT` case (gfx_run_dl ~line 2472), main DL start 0x70052e80, +failing opcode slot at +0x260 (0x700530e0). G_TEXRECT carries no source +address — it blits from RDP texture-memory tiles; the fault is the **lazy +upload of tile 0** whose stored source pointer is invalid (or the tile was +never set up this frame → stale `rdp.textures[0]`). Suspects: (a) a +main-DL G_SETTIMG (0xFD) earlier than +0x260 whose w1 seg_addr +mis-resolves; (b) tile referenced before setup / across frames; (c) the +game points the tile source at a ROM/ramrom image that needs a port fixup +(D50.3/D50.4 class). **Investigation not started.** Note: after the crash, +0x70052e80 is unreadable from gdb ("Cannot access memory") — dump live at a +breakpoint, not post-mortem. GE opcode facts verified this session +(include/PR/gbi.h, non-F3DEX numbering): G_IMMFIRST=−65; DMA G_MTX=1 / +G_MOVEMEM=3 / G_VTX=4 / G_DL=6; IMM TRI1=0xBF … ENDDL=0xB8; +GE extension G_TRI4=0xB1 (gbi_extension.h, 8-bit packed indices — NOT an +address carrier); RDP pass-throughs include G_SETTIMG=0xFD, +G_SETCIMG=0xFF, **G_TEXRECT=0xE4 / G_TEXRECTFLIP=0xE5** (GE-specific +values, not libultra's 0x46/0x45). The main DL is game-built per frame with +gSP* macros (16B slots); the segment table is set via +`gMoveWd(G_MW_SEGMENT, seg*4, base)` → w0=(0xBC<<24)|(seg*4<<8)|6, +w1=data; fast3d's gfx_sp_moveword stores data as-is when ≥ 0x800000 else ++0x80000000. + Environment notes for Task 2: standalone gcc needs `/c/msys64/mingw64/bin:/c/msys64/usr/bin` on PATH (cc1 fails SILENTLY without it — exit 1, zero diagnostics), and `-std=c11` is required (the CMake flag; @@ -1673,118 +1782,47 @@ game TUs. ### G. Phase 2 status (current) -Done: PD fast3d integrated (`port/fast3d/`, replaces the deleted gfxstub.c); -GE's real `src/sched.c` compiled (shedThread runs `__scMain`); real pthread -kernel (D24); dual-mapped DRAM + address shims (D25/D26); ASLR off with startup -sanity check (D27); PI completion messages (D29); SEH-free crash handler with -thread dumps (D30). The process now boots, maps the ROM, opens the window, -runs `mainproc()` → `bossInitMainthreadData` through controller init / timers / -mempool / VI / rsp / stan / game init. The D31 langInit file-load SIGSEGV is -**fixed** (real-zlib decompression, §F/D31): mainThread now loads all language -files and proceeds into `bossInitMainthreadData()` → `alloc_load_expand_ani_table` -→ `init_weapon_animation_groups_maybe` (boss.c:233), where it SIGSEGVs on the -**D32** ROM-struct pointer-width issue. D32 is decomposed: **Part A** (struct -layout — u32 pointer fields) is DONE & proven; **Part B** (the load-time pointer -rebase in `expand_ani_table_entries`) is OPEN and is the current blocker. No -frames rendered yet. See §H for the handoff + plan. - -Reference — the file-load chain (for D31 debugging): -`langInit` (language.c:230) loads 7 text banks via -`_fileNameLoadToBank(LnameX_lookuptable[bank][j_text_trigger], …, 0x100, -MEMPOOL_PERMANENT)` → `fileGetIndex` (strcmp over file_resource_table) → -`fileIndexLoadToBank(index, …)` (ob.c:216): if poolRemaining==0 take -`mempGetBankSizeLeft(bank)`; `ptrdata = mempAllocBytesInBank(…)`; hw_address -nonzero → `load_resource(ptrdata, poolRemaining, &file_resource_table[index], -&resource_lookup_data_array[index])` (ob.c:37): `source = (ptrdata + bytes) − -((rom_size+7)&−8)`; `romCopy(source, hw_address, rom_size)`; -`decompressdata(source, ptrdata, buffer)` with `u8 buffer[0x2100]` on the -stack → sets rz_inbuf=source+2 / rz_outbuf=ptrdata / rz_hlist=buffer → -`zlib_inflate()` (src/game/zlib.c:668). rom_size for index 670 = 0x720, -decompressed size = 3872 (0xF20). `file_resource_table` is compiled -(`assets/obseg/file_resource_table.inc.c`) with hw_address = cart addresses; -rom_size = adjacent-marker differences computed in obInit. - -Debugging techniques that worked: gdb **launch** mode (`gdb -batch -ex -"handle SIGSEGV stop" -ex run …` — attach fails with error 87); breakpoint -chains through init functions to bracket the crash; `finish` to let a suspect -function complete and inspect its return; hardware watchpoints on exact stack -slots (compute offsets from entry RSP, not RBP — see D31); `addr2line` for -offline symbolication; objdump + `gcc -E` with exact ninja flags to verify -shim expansion in compiled code. +Done through **D50**: PD fast3d integrated (`port/fast3d/`); GE's real +`src/sched.c` + pthread kernel; dual-mapped DRAM; ROM mapped at cart base; +SDL2 window; full boot chain (D31–D42); **Plan B executed (D50)** — all 512 +NTSC model files offline-converted to PC-layout RZ sidecars +(`tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + +manifest.csv), served through the existing load path via a port-layer table +patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot +`pcmodelsPatchTable()` + poolRemaining reset hook in +`load_object_fill_header`); runtime C fixups for language-bank BE offset +tables (D50.3) and font re-layout (D50.4); legal-screen UB seed (D50.5); +`texCopyGdls` w1 partial-copy bug fixed and byte-proven (D50.6). Frames 1–4 +render; PlegalpageZ model GDLs execute to completion. **Current blocker: +D51** — SIGSEGV in `import_texture_i8` via a main-DL G_TEXRECT lazy tile-0 +upload (frame 5); investigation not started, recipe in docs/HANDOFF.md. +After D51: pixel-assert soak (env-gated PPM dump + pixcount still unbuilt), +then the next asset type (Tbg_*_stanZ backgrounds) with the same offline +pattern. ### H. Handoff & plan (current session) -**State.** Boot is complete AND the first frames render: mainThread runs through -all of `bossInitMainthreadData()` + `bossEntry()`, enters `bossMainloop()`, -loads the stage, and fast3d renders real display lists — `[NOTE] frame N rendered -in … us` logs appear (frame 1 in ~70 ms). Findings **D34–D42** (§F) closed out -the boot + first-render blockers on top of D31–D33: -- **D34** `ANIM_DATA_*` → address-only lvalues at PE-image-base + ROM offset. -- **D35** music seq table: `RareALSeqData.address` → `u32` + BE fixup. -- **D36** music heap / PERMANENT pool grown for x86-64 libaudio struct bloat. -- **D37** libaudio bank trees re-laid out to PC-native form (`romdataFixupAudioBank`). -- **D38** implicit-declaration pointer truncation → `port/include/pc_protos.h` - prototype shim (398 decls, generator in build-pc/); build now warning-clean. -- **D39** Globalimagetable rebasing: per-symbol offsets + `GIMG_OFF()` macro - (== N64 `(u32)&sym`, incl. the 0x02000000 base) in image_bank.c; segment end - markers extended to the true 0x13F8 size (CSV truncation). -- **D40** ModelHitEntry BSS pool sized for the 40-byte PC stride (was 12 KB - overflow clobbering `is_ramrom_flag`). -- **D41** GL context released from the host main thread after init so the game - thread can bind it (WGL single-thread currency; SDL2 MakeCurrent return value - is an inverted ABI artifact — don't branch on it). -- **D42** rsp.c `g_gfxTaskSettingsList` XOR toggle → explicit toggle on x86-64. +Full paste-ready brief: **docs/HANDOFF.md** (rewritten this session). Summary: -**ROM provenance locked:** the working ROM is byte-identical to the No-Intro -good dump `GoldenEye 007 (U) [!].z64` (MD5 `70c525880240c1e838b8b1be35666c3b`, -0xC00000 bytes, country 'E'); embedded header CRCs DCBC50D1/09FD1AA3 -re-computed and matched via `tools/n64cksum.c`'s CIC-6102 routine — see -`tools_pc/romverify.c` (one-shot re-check) and docs/HANDOFF.md Environment. +**State.** Plan B is EXECUTED (D50): offline sidecars for all 512 NTSC model +files + port-layer plumbing + lang/font runtime fixups + legal-screen UB seed ++ the texCopyGdls w1 fix (D50.6, byte-proven against PlegalpageZ). Boot runs +to the main loop; frames 1–4 render; model GDLs execute (PlegalpageZ +sub-DLs complete). -**Current blocker — D43: model-file loading ABI mismatch.** After a few frames, -stage object load faults in `modelPromoteNodeOffsetsToPointers()` -(model.c:5688). Model files are ROM-serialized with N64 layout (ModelFileHeader -24 B / ModelNode 20 B, 4-byte pointer fields) but read on PC as 8-byte-pointer -structs; `load_object_fill_header()` (objecthandler_2.c) derives RootNode from -`numtextures` at the wrong offset before any rebase. Plan: a D37-style two-pass -re-layout of the model file image in `port/src/romdata.c` — header, switches -array, texture table, ModelNode tree (walked via Child/Next/Parent), and the -per-opcode ModelRoData records — with every embedded pointer rewritten as a -zero-extended offset from the image start so the existing -`PROMOTE`/`diff = fileramaddr - vma` rebase works unmodified. Map the record -types (bondtypes.h `ModelRoData` union, opcodes in bondconstants.h) — but do -NOT design from scratch: PD port's `port/src/preprocess/filemodel.c` (§2.4) is -a Rare-validated near-analogue (same PROMOTE idiom, same vma 0x5000000); adapt -its node walk / placement / record handling and validate every GE record type -per-field against bondtypes.h + ROM ground truth. Expect more D32-class ABI -fixes as other per-level asset types are first touched — standing sub-task, -not a one-off; PD's `preprocess/` module already solves most of them -(`filebg.c`, `filetiles.c`, …). +**Current blocker — D51.** SIGSEGV on frame 5 in `import_texture_i8` via a +main-DL G_TEXRECT lazy tile-0 upload (gfx_pc.cpp:782; failing opcode slot at +DL offset +0x260). G_TEXRECT blits from RDP texture-memory tiles, so the fault +is an invalid/stale tile source — either a mis-resolved main-DL G_SETTIMG, +a tile used before setup, or a ROM/ramrom image needing a port fixup. +Investigation not started; the gdb recipe (break live at +`import_texture_i8`, dump `rdp.textures[0]` + the full main DL — post-mortem +dumps of 0x70052e80 fail with "Cannot access memory") is in HANDOFF Task 1. -**Dev-loop measurements + speedups (session of 2026-08-22).** Measured -iteration costs on the dev box (16 cores): full clean rebuild ≈ 5 s wall -(239 ninja steps) — the build is NOT a bottleneck, ccache would be wasted -effort; launch → D43 crash ≈ 3.7 s — runtime wait is negligible. The cost is -in *diagnosis*: gdb launch-only mode + manual `addr2line` image-base arithmetic -per fault. Speedups, in priority order: (1) the D44 one-line fix → automatic -backtrace on every crash; (2) a `tools_pc/sym.sh` wrapper (PC or rel offset → -add image base 0x140000000 → `addr2line -f -C`); (3) a Python ROM/asset -inspector (`tools_pc/romdump.py`) that parses ModelFileHeader/ModelNode trees -straight from the .z64 using the bondtypes.h layout (opcode sequences, node -links, Data records) — accelerates D43 and every follow-on asset type, and can -validate re-layout output against ROM ground truth without running the game; -(4) env-gated framebuffer PPM dumps every N frames + a non-clear-color pixel -count/diff script — turns "the scene contains actual model geometry" into an -assert, makes the 30 s soak scriptable, and enables screenshot diffs against -an emulator (Mupen64Plus with the verified No-Intro dump). NOT worth it -(measured): ccache, boot fast-forward/skip-to-mainloop hooks. - -**Audio (Phase 3 prep).** Bank trees are correctly laid out at boot and the -libaudio manager + audio thread run without crashing; actual sound output still -needs an SDL backend for `alDriver`, and the ASP strategy question (C2) remains -open. No work required until rendering is stable. The SDL backend is largely -copy-and-adapt from PD: `port/src/mixer.c` (722 lines vs our 31-line stub, -which was scaffolded for it) plus `input.c`/`fs.c` for Phase 3/4 — see §2.4. +**Next after D51:** build the pixel assert (env-gated PPM dump every N frames +in port/video.c + `tools_pc/pixcount.py`) and run a 30 s+ soak; then move to +the next asset type (Tbg_*_stanZ backgrounds) with the same offline pattern — +PD's `port/src/preprocess/filebg.c` is the per-field reference (§2.4). **D32 repeatable fix procedure** (apply to any ROM-serialized struct that faults on a pointer-field read): @@ -1821,7 +1859,6 @@ prefix `export PATH=…`). Build: `./build-pc.sh ntsc-final`. gdb **launch** mod only (attach fails, error 87); symbolicate offline with `addr2line -e build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base 0x140000000. `load_resource`/many init fns use a fake RBP — compute stack offsets from entry -RSP. The D30 crash handler writes Phase 1 of `ge007.crash.log` (raw PC/ -registers) but its Phase 2 backtrace self-faults before printing (**D44**, -one-line fix pending) — until fixed, attribute crashes via gdb; after the fix -the crash log is the first stop. +RSP. The D30 crash handler writes `ge007.crash.log` with a working Phase-2 +backtrace (**D44** fixed) — first stop for any fault; frames past the true chain +may be stale, sanity-check they fall inside `.text`.