PC port: handoff for D43 session — dev-loop measurements, D44 crash-handler finding

docs/HANDOFF.md

@@ -1,23 +1,28 @@
 # Handoff brief — GoldenEye 007 PC port (Phase 1.5 done, Phase 2: first frames render)
 
 _Paste-ready briefing for the next agent session. The authoritative detail lives in
-`AGENTS.md` and `docs/PCPortResearch.md` (§H handoff, §F/D31–D43 findings); this is
-the summary + the immediate task._
+`AGENTS.md` and `docs/PCPortResearch.md` (§H handoff, §F/D31–D44 findings); this is
+the summary + the immediate tasks._
 
 ## Your job
-Boot is complete AND **real frames render** (`[NOTE] frame N rendered in … us`). The
-current blocker is **D43**: model-file loading ABI mismatch — stage object load faults
-in `modelPromoteNodeOffsetsToPointers()` (src/game/model.c:5688) because ROM model files
-are serialized with N64 layout (4-byte pointer fields) but read on PC as 8-byte-pointer
-structs. Fix it with a D37-style re-layout in `port/src/romdata.c`, then keep pushing
-rendering forward (more asset types will fault the same way). Work agentically — fix →
-build → verify under gdb → commit at each working milestone.
+Two tasks, in order:
+1. **D44 (quick win, ~15 min):** one-line fix to the crash handler so every crash
+   auto-logs a backtrace — see Immediate task 1. This shrinks the D43 debug loop
+   from "gdb launch session per fault" to "read ge007.crash.log".
+2. **D43 (main task):** model-file loading ABI mismatch — stage object load faults
+   in `modelPromoteNodeOffsetsToPointers()` (src/game/model.c:5688) because ROM model
+   files are serialized with N64 layout (4-byte pointer fields) but read on PC as
+   8-byte-pointer structs. Fix it with a D37-style re-layout in `port/src/romdata.c`,
+   then keep pushing rendering forward (more asset types will fault the same way).
+
+Work agentically — fix → build (~5 s) → verify → commit at each working milestone.
+Commit message style: `PC port: <phase> — <what>`.
 
 ## Read first (authoritative, in order)
 1. `AGENTS.md` — non-negotiables, critical files, build, verification ritual, phase status.
-2. `docs/PCPortResearch.md` §H (handoff + plan), then **§F/D43** (the open model-file
-   task) and **D37** (the libaudio bank-tree re-layout — the pattern to copy), D39–D42
-   for the first-render fixes, D31–D38 for background.
+2. `docs/PCPortResearch.md` §H (handoff + plan), then **§F/D44** (the quick win),
+   **D43** (the open model-file task), **D37** (the libaudio bank-tree re-layout —
+   the pattern to copy), D39–D42 for the first-render fixes, D31–D38 for background.
 
 ## Non-negotiables (the ones that bite)
 - N64 build files (`Makefile`, `tools/`, `rsp/`, `ld/`) — never touch.
@@ -27,35 +32,47 @@ build → verify under gdb → commit at each working milestone.
   as a D3x finding. No behavior changes.
 - Region macros in `CMakeLists.txt` must mirror the N64 Makefile exactly.
 
-## Current state (verified this session)
+## Current state (verified live this session, 2026-08-22)
+- **Re-ran the committed build:** frames 1–4 render (`[NOTE] frame N rendered in … us`,
+  frame 1 ~21 ms), then FATAL `0xc0000005` at PC `0x14007a31a` =
+  `modelPromoteNodeOffsetsToPointers()` model.c:5688 — exactly D43. The black screen you
+  see is the clear color only: display lists execute but no stage geometry has loaded yet.
+- **Dev loop measured (16-core box):** full clean rebuild ≈ **5 s** wall; launch → crash
+  ≈ **3.7 s**. Neither is a bottleneck — *diagnosis* is (gdb launch-only mode + manual
+  addr2line arithmetic per fault). ccache and boot fast-forward are NOT worth it.
+- **D44 identified + root-caused (fix pending):** `crashStackTraceSym()` in
+  `port/src/crash.c` calls `GetCurrentThreadStackLimits(low, high)` — passing NULL pointer
+  *values* instead of `&low, &high`. The API writes to address 0x0 *inside the SEH filter*
+  → the process dies before printing the backtrace. That's why `ge007.crash.log` contains
+  only Phase 1 (EXCEPTION/PC/MODULE lines) and no `BACKTRACE:` section, and why every D3x
+  fault so far cost a gdb session. The build already uses `-fno-omit-frame-pointer`
+  (CMakeLists.txt:193), so the EBP-chain walk will work once the call is fixed.
 - **ROM dump verified:** byte-identical to the No-Intro good dump, header CRCs re-compute
   clean (`tools_pc/romverify.c`). Fingerprint + provenance in Environment below.
-- **Frames render.** The chain of fixes that got us here (all documented in §F):
-  - **D31–D37:** boot chain — real-zlib file decompression, animation-table ABI+endianness,
-    ANIM_DATA lvalues, music seq-table fixup, music-heap sizing, libaudio bank-tree
-    re-layout (`romdataFixupAudioBank`).
-  - **D38:** implicit function declarations truncated pointer returns on x86-64 →
-    `port/include/pc_protos.h` prototype shim (398 decls; generator `scripts/gen_pcprotos.py`;
-    anchored in `port/shim/PR/ucode.h`; C-only). Build is now warning-clean.
-  - **D39:** Globalimagetable rebasing — `texReset()`'s `globalbank_rdram_offset + (u32)&sym`
-    idiom needs the N64 `(u32)&sym == 0x02000000 + off` value; PC branch uses a per-symbol
-    offset enum + `GIMG_OFF()` in `src/game/image_bank.c`. Segment end markers extended to
-    the true 0x13F8 size (the CSV's `Globalimagetable.bin` entry is truncated by 0x930).
-  - **D40:** `g_ModelHitEntries` BSS pool was N64-sized (12 KB for 600×20 B entries); the
-    40-byte PC stride overflowed .bss and clobbered `is_ramrom_flag`. PC branch declares a
-    `600 * sizeof(ModelHitEntry)` pool.
-  - **D41:** WGL contexts are current on one thread only; the host main thread created +
-    held the context while the game thread renders. `videoInit()` now calls
-    `gfx_sdl_release_context()` after init; the game thread re-binds per frame. NOTE: this
-    SDL2 build's `SDL_GL_MakeCurrent` return value is an inverted ABI artifact (0 = success,
-    -1/255 = failure) — do NOT branch on it.
-  - **D42:** rsp.c's XOR toggle of `g_gfxTaskSettingsList` truncated exe pointers to u32 →
-    explicit toggle under `#if defined(__x86_64__)`.
+- Frames render because of D31–D42 (all documented in §F): boot chain (real-zlib
+  decompression, animation-table ABI+endianness, ANIM_DATA lvalues, music seq-table fixup,
+  music-heap sizing, libaudio bank-tree re-layout), D38 prototype shim (warning-clean
+  build), D39 Globalimagetable rebasing (`GIMG_OFF()` idiom), D40 ModelHitEntry pool
+  sizing, D41 WGL single-thread context currency (`gfx_sdl_release_context()`), D42
+  rsp.c pointer-toggle truncation.
 
-## Immediate task — D43: model-file loading ABI mismatch
+## Immediate task 1 — D44: crash-handler one-liner (do this FIRST)
+1. In `port/src/crash.c`, function `crashStackTraceSym()`: change
+   `GetCurrentThreadStackLimits(low, high)` → `GetCurrentThreadStackLimits(&low, &high)`.
+   (Phase 1's `crashStackTraceRaw` already passes `&low, &high` — mirror it.)
+2. Rebuild (`./build-pc.sh ntsc-final`, ~5 s), run the exe unmodified. Expect a
+   `BACKTRACE:` section on stdout AND in `ge007.crash.log` after the Phase 1 lines.
+3. Confirm the logged backtrace shows the D43 chain: #00 at model.c:5688, then
+   `load_object_fill_header` / objecthandler frames above it. That proves the whole
+   diagnosis loop works without gdb. (If a frame's symbol is `?`, that's fine —
+   addr2line offline still resolves it; see `sym` tip below.)
+4. Commit: `PC port: fix crash-handler Phase 2 self-fault, auto-backtrace on crash (D44)`.
+
+## Immediate task 2 — D43: model-file loading ABI mismatch
 1. Reproduce: run; after a few rendered frames it SIGSEGVs in
    `modelPromoteNodeOffsetsToPointers()` (model.c:5688) during stage object load via
-   `load_object_fill_header()` (src/game/objecthandler_2.c:89).
+   `load_object_fill_header()` (src/game/objecthandler_2.c:89). With D44 fixed, the
+   crash log gives you the call chain for free.
 2. Understand the file format from `bondtypes.h` (`ModelFileHeader`, `ModelNode`,
    `ModelRoData` union + per-opcode records) and `bondconstants.h`
    (`MODELNODE_OPCODE_*`). N64 layout: header 24 B, node 20 B, all pointers 4 bytes.
@@ -65,19 +82,32 @@ build → verify under gdb → commit at each working milestone.
    existing `PROMOTE` rebase (`diff = fileramaddr - vma`, vma = 0x5000000) works
    unmodified. Hook it between the romCopy in `load_resource()` and first use (or at the
    `load_object_fill_header` call sites — choose the narrowest correct hook).
-4. Validate one small model against ROM ground truth under gdb before generalizing
-   (opcode sequence, node links, a few Data records).
+4. Validate one small model against ROM ground truth before generalizing (opcode
+   sequence, node links, a few Data records). gdb works; or write the optional
+   `tools_pc/romdump.py` inspector (below) and validate statically.
 5. Expect follow-on faults in other asset types (BG/stan/propobj tables) — same procedure.
-6. D40-class audit (done this session, negative): the dangerous pattern is a raw
-   `char[N]`/`u8[N]` BSS pool sized for N64 that gets cast to a pointer-containing
-   struct and written with PC stride. Only 3 raw byte arrays ≥1 KiB exist in game code
-   (crash/debug buffers — not pools); the 362 `dword_CODE_bss_*` placeholders are
-   mostly scalars or self-sizing typed arrays (e.g. bg.c `s_bound_info[204]`). Re-audit
-   per asset type as loading progresses.
+
+## Dev-loop speedups (build these when they pay off, not before)
+Priority order from the last session's measurements:
+1. **D44** (above) — automatic backtraces; biggest win per diagnosis.
+2. **`tools_pc/sym.sh`** — tiny wrapper: takes a PC or rel offset, adds image base
+   `0x140000000`, runs `addr2line -e build-pc/ge007.x86_64.exe -f -C`. Kills the manual
+   arithmetic; pairs with the crash log (one command per crash). Re-verify the image base
+   with `info address` after any link-layout change.
+3. **`tools_pc/romdump.py`** — Python ROM/asset inspector: load `data/ge007.ntsc-final.z64`,
+   parse ModelFileHeader/ModelNode trees using the bondtypes.h layout, print opcode
+   sequences / node links / Data records for a given cart address. Accelerates D43 and
+   every follow-on asset type; validates re-layout output against ROM ground truth without
+   running the game. (Python here: no f-strings with nested quotes — use `%` formatting.)
+4. **Frame capture + pixel assert** — env-gated framebuffer PPM dump every N frames in
+   `port/`, plus a script that counts non-clear-color pixels / diffs two captures. Turns
+   "the scene contains actual model geometry" into an assert, makes the 30 s soak
+   scriptable, and enables screenshot diffs against an emulator (Mupen64Plus with the
+   verified No-Intro dump as ground truth).
 
 ## Standing procedure (you will hit more of these)
 Every ROM-serialized struct with a pointer field faults the same way once you reach more
-asset loading. The D32 fix procedure (doc §H): at the fault run `ptype /o <Struct>`; if a
+asset loading. The D32 fix procedure (doc §H): at the fault, `ptype /o <Struct>`; if a
 pointer field's offset/size diverges from its N64 offset comment, change it to `u32` + cast
 at the use sites; verify the load-time rebase yields valid V1 addresses (< 0x80000000);
 **also check the ROM bytes' endianness per-field (D33 rule: u32 bswap32 / u16 bswap16 / u8
@@ -90,7 +120,8 @@ Log each as D3x in §F and note it in AGENTS.md phase status.
 - MSYS2/MinGW tools are in `/c/msys64/mingw64/bin/` — **not** on PATH. Prefix every shell:
   `export PATH="/c/msys64/mingw64/bin:$PATH"`.
 - Build: `./build-pc.sh ntsc-final` (incremental; full rebuild only when `port/shim/`
-  changes). ROM is at `data/ge007.ntsc-final.z64` (= `baserom.u.z64`, byte-identical).
+  changes; a *clean* full build is ~5 s on the 16-core dev box). ROM is at
+  `data/ge007.ntsc-final.z64` (= `baserom.u.z64`, byte-identical).
 - The ROM is **mapped at cart base 0x10000000** on PC (`romdataInit`), so `&<segment>Symbol`
   values (e.g. `_sfxctlSegmentRomStart = 0x102EBDE0`) are directly dereferenceable host
   pointers into the ROM image; `romCopy()` copies from those cart addresses. All game RAM
@@ -105,16 +136,18 @@ Log each as D3x in §F and note it in AGENTS.md phase status.
   offline with `addr2line -e build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base
   is `0x140000000` (re-verify with `info address` after a rebuild). Debug scripts live in
   `build-pc/` (`d37_tree.py`, `d39_*.gdb`, `d41_probe.gdb`, …).
+- Crash log: `ge007.crash.log` (repo root) currently holds Phase 1 only — the backtrace
+  phase self-faults (**D44**, one-line fix, Immediate task 1). Until then attribute crashes
+  via gdb; after the fix the crash log is the first stop for any fault.
 - `objdump -d --disassemble=<fn>` / `objdump -s -j .data` on build-pc/ge007.x86_64.exe is a
   fast way to check what the compiler actually emitted.
 - Many init functions use a fake RBP — compute stack offsets from the **entry RSP**, not RBP.
-- The D30 crash handler did **not** write `ge007.crash.log` for game-thread faults;
-  attribute crashes via gdb (the in-process dump does print PC/registers to stdout).
 - Python on this box: no f-strings with nested quotes (< 3.12) — use `%` formatting.
 
 ## Definition of done (this milestone)
-Stage objects load without faulting and the rendered scene contains actual model geometry
-(not just the clear color / background), stable under a 30 s+ soak. After that, Phase 2
-makes it look right (fast3d CC/RM correctness vs `gmain.s`) and Phase 3 adds sound
-(libaudio → SDL). Commit + push to `origin/master` at each working checkpoint using the
-message style `PC port: <phase> — <what>`.
+Checkpoint A: D44 committed — crash log shows a real backtrace for the D43 fault.
+Then the milestone proper: stage objects load without faulting and the rendered scene
+contains actual model geometry (not just the clear color / background), stable under a
+30 s+ soak. After that, Phase 2 makes it look right (fast3d CC/RM correctness vs
+`gmain.s`) and Phase 3 adds sound (libaudio → SDL). Commit + push to `origin/master` at
+each working checkpoint using the message style `PC port: <phase> — <what>`.

docs/PCPortResearch.md

@@ -1238,6 +1238,22 @@ operates on misaligned fields. Needs a D37-style re-layout of the model
 file image (header + switches array + texture table + node tree +
 ModelRoData records). Phase-2 project; see §H.
 
+**D44 (OPEN)** Crash-handler Phase 2 backtrace self-faults before printing.
+`crashStackTraceSym()` in `port/src/crash.c` calls
+`GetCurrentThreadStackLimits(low, high)` passing the NULL pointer *values*
+rather than `&low, &high` (Phase 1's `crashStackTraceRaw` passes them
+correctly). The API writes the stack limits to address 0x0 → access violation
+*inside the SEH filter* → Windows terminates the process without a second
+dispatch. Symptom: `ge007.crash.log` contains only Phase 1 (EXCEPTION/PC/
+MxCsr/thread-stack/MODULE lines) and no `BACKTRACE:` section — verified twice
+live; this is why every D3x fault so far cost a full gdb launch-mode session.
+Fix: one line, `GetCurrentThreadStackLimits(&low, &high)`. The build already
+compiles with `-fno-omit-frame-pointer` (CMakeLists.txt:193), so after the fix
+every crash auto-logs an EBP-chain backtrace to console + ge007.crash.log,
+symbolicable offline with addr2line. Verify: run, confirm `BACKTRACE:` with
+#00..#NN frames appears in the log and the D43 chain (modelPromoteNodeOffsets-
+ToPointers ← load_object_fill_header ← …) is visible without gdb.
+
 ### G. Phase 2 status (current)
 
 Done: PD fast3d integrated (`port/fast3d/`, replaces the deleted gfxstub.c);
@@ -1324,6 +1340,24 @@ validate one small model against ROM ground truth under gdb before generalizing.
 Expect more D32-class ABI fixes as other per-level asset types are first
 touched — standing sub-task, not a one-off.
 
+**Dev-loop measurements + speedups (session of 2026-08-22).** Measured
+iteration costs on the dev box (16 cores): full clean rebuild ≈ 5 s wall
+(239 ninja steps) — the build is NOT a bottleneck, ccache would be wasted
+effort; launch → D43 crash ≈ 3.7 s — runtime wait is negligible. The cost is
+in *diagnosis*: gdb launch-only mode + manual `addr2line` image-base arithmetic
+per fault. Speedups, in priority order: (1) the D44 one-line fix → automatic
+backtrace on every crash; (2) a `tools_pc/sym.sh` wrapper (PC or rel offset →
+add image base 0x140000000 → `addr2line -f -C`); (3) a Python ROM/asset
+inspector (`tools_pc/romdump.py`) that parses ModelFileHeader/ModelNode trees
+straight from the .z64 using the bondtypes.h layout (opcode sequences, node
+links, Data records) — accelerates D43 and every follow-on asset type, and can
+validate re-layout output against ROM ground truth without running the game;
+(4) env-gated framebuffer PPM dumps every N frames + a non-clear-color pixel
+count/diff script — turns "the scene contains actual model geometry" into an
+assert, makes the 30 s soak scriptable, and enables screenshot diffs against
+an emulator (Mupen64Plus with the verified No-Intro dump). NOT worth it
+(measured): ccache, boot fast-forward/skip-to-mainloop hooks.
+
 **Audio (Phase 3 prep).** Bank trees are correctly laid out at boot and the
 libaudio manager + audio thread run without crashing; actual sound output still
 needs an SDL backend for `alDriver`, and the ASP strategy question (C2) remains
@@ -1364,5 +1398,7 @@ prefix `export PATH=…`). Build: `./build-pc.sh ntsc-final`. gdb **launch** mod
 only (attach fails, error 87); symbolicate offline with `addr2line -e
 build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base 0x140000000.
 `load_resource`/many init fns use a fake RBP — compute stack offsets from entry
-RSP. The D30 crash handler did **not** write `ge007.crash.log` for the game-thread
-SIGSEGV this session (attribute via gdb instead; worth a separate look).
+RSP. The D30 crash handler writes Phase 1 of `ge007.crash.log` (raw PC/
+registers) but its Phase 2 backtrace self-faults before printing (**D44**,
+one-line fix pending) — until fixed, attribute crashes via gdb; after the fix
+the crash log is the first stop.