diff --git a/AGENTS.md b/AGENTS.md index ce4bf58e..4afa98e5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,6 @@ Run `/linkcheck` for this sweep. Record new findings in `docs/PCPortResearch.md` - **Compile+link milestone:** all ~235 TUs compile and `ge007.x86_64.exe` links clean (`ninja ge007 -k 0`, 236/236). Findings D7–D17 in `docs/PCPortResearch.md` §11; asset stubs in `port/src/assetstubs.c`, fast3d no-ops in `port/src/gfxstub.c` (delete when real fast3d lands). - **Phase 1 (boot to window):** done — ROM loads/validates/maps at cart base 0x10000000 (`romdata.c`), SDL2 window + GL clear loop (`video.c`), absolute asset symbols from `scripts/gen_romassets.py` → `port/src/romassets_.s` (findings D18–D23). - **Phase 1.5 (boot to first frame):** DONE — boot chain complete AND real frames render (`[NOTE] frame N rendered in … us`). D34–D42 closed the blockers on top of D31–D33: ANIM_DATA lvalues, music seq-table ABI+endianness, music-heap sizing, libaudio bank-tree re-layout, implicit-declaration prototype shim (D38), Globalimagetable rebasing + true segment size (D39), ModelHitEntry pool sizing (D40), cross-thread GL context release (D41), rsp task-settings toggle (D42). ROM dump verified against the No-Intro good dump (byte-identical, header CRCs re-checked — `tools_pc/romverify.c`). -- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + real frames render. **Plan B EXECUTED (D50, committed):** all 512 NTSC model files converted offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + manifest.csv (single concatenated RZ image; regenerate per region with the script — sidecars are gitignored), served through the existing load path via a port-layer table patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot `pcmodelsPatchTable()` + poolRemaining reset hook in `load_object_fill_header`); runtime C fixups for language-bank BE offset tables + font re-layout (D50.3/D50.4); legal-screen UB seed (D50.5); **D50.6 fixed the first model-render crash** — `texCopyGdls` copied only w0 of each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes), leaving stale mempool w1s in the compacted GDLs; byte-proven against PlegalpageZ, fixed with a full-slot copy under #ifdef PORT. Model GDLs now execute to completion (PlegalpageZ sub-DLs). **D51 RESOLVED** (font `pixeldata` fixup wrote the pointer at the wrong offsets — frame-5 `import_texture_i8` crash gone; GE opcode space decoded along the way, G_TEXRECT=0xE4/0xE5 non-F3DEX numbering), **D52 RESOLVED** (`osGetCount` now advances at the N64 RSP counter rate ≈ 46.5525 ticks/µs — fixed the non-deterministic post-frame-2 hang), **D53 RESOLVED** (model RW-data pool addressing: D53.1 `Model.datas` word-stride → `u32 *` under PORT + casts; D53.2 ModelSlot/AnimModelSlot re-laid on top of the PC struct Model — in-use@0x10 / pool@0x20, slot size = sizeof(Model) = 0xE8 — plus PORT-guarded heap-fallback bumps in model.c), **D54 RESOLVED** (cseq ALCMidiHdr BE header: port-layer `romdataFixupCseq` bswap32 of the first 17 u32s, called from music.c after each decompress), **D54b RESOLVED** (synth param slots sized for ALStartParamAlt = 0x28 in synthesizer.c alSynInit under PORT), **D55 RESOLVED** (RLE folder-menu background BE w/h header: bswap32 of the first word at `(u32 *)&unknown2` at cart-map time in romdataInit, guarded). Build GREEN; game boots, plays intro music, renders through logo/gun-barrel, instantiates Brosnan, and executes the gun-barrel hole DL clean to ENDDL. **D56 RESOLVED** (watch Model/RW-pool raw offsets: field-name Model access + static 0xC8-byte pool under PORT in bondview2.c), **D57 RESOLVED** (pointer-grown rwdata records overflowed N64-sized spare pools — Brosnan PC numRecords 153 > 140: capacities grown in initunk_005520.c + dynamic slot+pool fallback in model.c), **D58 RESOLVED** (gun-barrel DL: vertex-pointer K0 idiom rebuilt via `OS_K0_TO_PHYSICAL(…) | 0x80000000` in title.c + DL reserve 0x100→0x200 because `sizeof(Gfx)` is 16 on PC). **D59–D68 RESOLVED (committed):** the "GL DLL" crash was D64's unbounded blood-RLE write (sentinel section-adjacency lost on PC) clobbering the gun-barrel sub-DL; then D60–D62 (DMA target validation, OSMesgQueue layout), D65/D65b (HEADS/BODIES enum signed sentinels under PORT), D66 (romCopyAligned + ramrom replay 64-bit pointer width), D67 (struct image_entry N64 layout), **D68 (Globalimagetable BE→LE fixup** — `port/src/gimgfixup.c` bswaps IMAGESEG Gfx w1 words + sImageTableEntry.index after texReset's romCopy; compiled globalDL shadows synced for explosion.c). The game now renders the **entire intro** (logo → gun barrel w/ Brosnan → cast; ~frame 2100, ≈2 min wall-clock at the current ~20 fps clean-run rate). **Current blocker D69:** `load_bg_file` (bg.c:830) faults on first stage load — BG-file headers/records are N64 big-endian segment-0x0F offsets, read LE on PC; the bg `.seg` + `Tbg_*_stanZ` format work is next (decode from bg.c/stan.c → offline sidecar conversion per the D43 pattern or runtime port-layer fixup; PD `preprocess/filebg.c` is approach-reference only — its BG format differs). **D70–D74 (intro-logo pixels, RESOLVED):** D70 env-gated PPM frame capture (`GE_PCDUMP` → `./ppm/`, gitignored — kept as permanent dev tooling); **D71** C-array texture sources (the four rarewarelogo.c RGBA16 images) byte-swapped on LE PC → port-layer per-source bswap in `import_texture`; **D72** the UV path (GE always uses authored `tc[]` UVs; `lookat_enabled` defaults false); **D73 = root cause of old D72.3** — sinf/cosf `du` double constants are big-endian word pairs, garbage on LE PC → guMtxF2L emitted −32768 for every sin/cos entry → logo triangles projected off-screen; fixed with the PORT-only `DVAL()` macro in `src/libultra/gu/guint.h` (sinf.c/cosf.c `.d` reads replaced; N64 build untouched); **D74** texture import fallback no longer truncates valid gDPLoadBlock data (was dropping letter mip chains + reducing D_02005FF0 to 32×3), `TextureCacheKey` gained `size_bytes`, and the VBO path wraps UVs by tile size for WRAP sub-tiles (N64 semantics). The Rareware logo now renders its four gold letters on the dark-blue plate (PPM-verified ~frame 555; final pixel-perfect check vs N64 footage optional). D69 remains the milestone blocker. Committed through D74. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48; full-struct Gfx copies on PC — D50.6; audit struct type-puns after pointer-width changes — D53.2; raw byte offsets into pointer-containing structs shift on PC — D56; N64-sized reservations must be re-checked against 16-byte Gfx/Vtx widths — D58). **Open this session (tracked in §F as D75–D77, lower priority than D69):** D75 3D rendering still wrong (Nintendo logo misplacement + intro player models absent); D76 2D disclaimer screen only partially drawn; D77 no audible music on PC. Prioritization: rendering over audio unless a shared root cause is found (Q1). +- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + real frames render. **Plan B EXECUTED (D50, committed):** all 512 NTSC model files converted offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + manifest.csv (single concatenated RZ image; regenerate per region with the script — sidecars are gitignored), served through the existing load path via a port-layer table patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot `pcmodelsPatchTable()` + poolRemaining reset hook in `load_object_fill_header`); runtime C fixups for language-bank BE offset tables + font re-layout (D50.3/D50.4); legal-screen UB seed (D50.5); **D50.6 fixed the first model-render crash** — `texCopyGdls` copied only w0 of each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes), leaving stale mempool w1s in the compacted GDLs; byte-proven against PlegalpageZ, fixed with a full-slot copy under #ifdef PORT. Model GDLs now execute to completion (PlegalpageZ sub-DLs). **D51 RESOLVED** (font `pixeldata` fixup wrote the pointer at the wrong offsets — frame-5 `import_texture_i8` crash gone; GE opcode space decoded along the way, G_TEXRECT=0xE4/0xE5 non-F3DEX numbering), **D52 RESOLVED** (`osGetCount` now advances at the N64 RSP counter rate ≈ 46.5525 ticks/µs — fixed the non-deterministic post-frame-2 hang), **D53 RESOLVED** (model RW-data pool addressing: D53.1 `Model.datas` word-stride → `u32 *` under PORT + casts; D53.2 ModelSlot/AnimModelSlot re-laid on top of the PC struct Model — in-use@0x10 / pool@0x20, slot size = sizeof(Model) = 0xE8 — plus PORT-guarded heap-fallback bumps in model.c), **D54 RESOLVED** (cseq ALCMidiHdr BE header: port-layer `romdataFixupCseq` bswap32 of the first 17 u32s, called from music.c after each decompress), **D54b RESOLVED** (synth param slots sized for ALStartParamAlt = 0x28 in synthesizer.c alSynInit under PORT), **D55 RESOLVED** (RLE folder-menu background BE w/h header: bswap32 of the first word at `(u32 *)&unknown2` at cart-map time in romdataInit, guarded). Build GREEN; game boots, plays intro music, renders through logo/gun-barrel, instantiates Brosnan, and executes the gun-barrel hole DL clean to ENDDL. **D56 RESOLVED** (watch Model/RW-pool raw offsets: field-name Model access + static 0xC8-byte pool under PORT in bondview2.c), **D57 RESOLVED** (pointer-grown rwdata records overflowed N64-sized spare pools — Brosnan PC numRecords 153 > 140: capacities grown in initunk_005520.c + dynamic slot+pool fallback in model.c), **D58 RESOLVED** (gun-barrel DL: vertex-pointer K0 idiom rebuilt via `OS_K0_TO_PHYSICAL(…) | 0x80000000` in title.c + DL reserve 0x100→0x200 because `sizeof(Gfx)` is 16 on PC). **D59–D68 RESOLVED (committed):** the "GL DLL" crash was D64's unbounded blood-RLE write (sentinel section-adjacency lost on PC) clobbering the gun-barrel sub-DL; then D60–D62 (DMA target validation, OSMesgQueue layout), D65/D65b (HEADS/BODIES enum signed sentinels under PORT), D66 (romCopyAligned + ramrom replay 64-bit pointer width), D67 (struct image_entry N64 layout), **D68 (Globalimagetable BE→LE fixup** — `port/src/gimgfixup.c` bswaps IMAGESEG Gfx w1 words + sImageTableEntry.index after texReset's romCopy; compiled globalDL shadows synced for explosion.c). The game now renders the **entire intro** (logo → gun barrel w/ Brosnan → cast; ~frame 2100, ≈2 min wall-clock at the current ~20 fps clean-run rate). **D69 RESOLVED (committed):** `load_bg_file` no longer faults on stage load. BG-file headers/records are N64 big-endian segment-0x0F offsets with internal self-relative offsets; fixed via offline sidecar conversion (`tools_pc/d69_emit.py` → `data/pccg-/pccg.bin` + manifest, same Plan-B pattern as D43/pcmodels, port-layer wiring in `port/src/pccg.c`) plus four ABI layout fixes surfaced along the way: D78 (`StandTile` id/room bitfield packing), D79 (`bg_room_data` pointer-width), D83 (`StandTileHeaderMid`/`Tail` bitfield packing — same class as D78, caused an infinite loop in `stanBuildRoomData` until fixed), D84 (hand-inlined `BG_SEG_TO_PTR`-style folds in room-DL loaders needed the same `(u32)`-cast 32-bit wrap the macro already has). **D85/D86 OPEN (found during D69 verification, next up):** D85 — room primary/secondary DL binaries decompress correctly but `texLoadFromGdl` (the already-working model-GDL runtime converter) produces garbage from their content, not yet root-caused; safety-netted (bounds-checked in `bgBuildRoomVtxBounds` so it can't segfault, but geometry is wrong/empty). D86 — a **new** crash in `modelInitRwData` during room-triggered model instantiation, likely unrelated to bg/stan, not yet root-caused. See `docs/HANDOFF.md` and `docs/PCPortResearch.md` §F/D85-D86 for the recommended next steps (D86 first, then D85). **D70–D74 (intro-logo pixels, RESOLVED):** D70 env-gated PPM frame capture (`GE_PCDUMP` → `./ppm/`, gitignored — kept as permanent dev tooling); **D71** C-array texture sources (the four rarewarelogo.c RGBA16 images) byte-swapped on LE PC → port-layer per-source bswap in `import_texture`; **D72** the UV path (GE always uses authored `tc[]` UVs; `lookat_enabled` defaults false); **D73 = root cause of old D72.3** — sinf/cosf `du` double constants are big-endian word pairs, garbage on LE PC → guMtxF2L emitted −32768 for every sin/cos entry → logo triangles projected off-screen; fixed with the PORT-only `DVAL()` macro in `src/libultra/gu/guint.h` (sinf.c/cosf.c `.d` reads replaced; N64 build untouched); **D74** texture import fallback no longer truncates valid gDPLoadBlock data (was dropping letter mip chains + reducing D_02005FF0 to 32×3), `TextureCacheKey` gained `size_bytes`, and the VBO path wraps UVs by tile size for WRAP sub-tiles (N64 semantics). The Rareware logo now renders its four gold letters on the dark-blue plate (PPM-verified ~frame 555; final pixel-perfect check vs N64 footage optional). D69 is resolved; D85/D86 are the current blockers to a rendered stage frame (see above). Committed through D86. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48; full-struct Gfx copies on PC — D50.6; audit struct type-puns after pointer-width changes — D53.2; raw byte offsets into pointer-containing structs shift on PC — D56; N64-sized reservations must be re-checked against 16-byte Gfx/Vtx widths — D58). **Open this session (tracked in §F as D75–D77, lower priority than D69):** D75 3D rendering still wrong (Nintendo logo misplacement + intro player models absent); D76 2D disclaimer screen only partially drawn; D77 no audible music on PC. Prioritization: rendering over audio unless a shared root cause is found (Q1). - **Phase 3 (audio + input):** libaudio → SDL — largely copy-and-adapt from PD (`mixer.c` 722 lines vs our 31-line stub; `input.c` SDL_GameController backend; §2.4); decide the ASP strategy (C2: `aspMain*` dummies in `ucode.c`). - **Phase 4 (saves + polish):** file-backed EEPROM — build on PD's `fs.c` (file-backed save dir); PFS/motor already no-op stubs. diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index cf66a046..33b115ac 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -1,215 +1,115 @@ -# Handoff brief — GoldenEye 007 PC port (Phase 2: Session I — D69 BG/stan stage loading) +# Handoff brief — GoldenEye 007 PC port (Phase 2: Session J — D69 resolved, D85/D86 open) -_Paste-ready briefing. Authoritative detail lives in `AGENTS.md` and -`docs/PCPortResearch.md` (§F/D59–D74 findings, §G status, §H procedure); this -is the summary + the immediate tasks._ +_Paste-ready brief. Authoritative context: `AGENTS.md`, +`docs/PCPortResearch.md` §F (D69, D78-D86), `docs/BRIEF-D69-stage-load.md`._ -## Your job +## Where things stand -- **Session I — D69 (the milestone blocker):** make the first stage (BUNKER1) - load and render. BG-file data is N64 big-endian; reverse-engineer GE's bg - `.seg` + `Tbg_*_stanZ` formats from the decompiled consumers, then convert - or fix them for PC. This is the only thing standing between the intro and - gameplay (verified: clean runs reach frame ~2100 in ≈2 min wall-clock, then - fault deterministically in `load_bg_file`, bg.c:830). -- **Secondary (low priority):** final pixel check of the intro logo. D73 + - D74 fixed it at the data level and PPM frames ~550–560 show four gold - letters on the dark-blue plate; compare against N64 reference footage if - time allows (N64 reference footage screenshots are in `docs/reference/`: - `n64-footage-2026-08-24.jpg`, `n64-footage-gun-barrel.png`; captured with - the local Mupen64 checkout in `tools_pc/mupen64/`, gitignored). -- Work agentically: decode → implement → build (~5 s) → verify → commit at - each sub-milestone (document findings as D7x in §F). +**D69 (the "stage load faults" milestone blocker) is RESOLVED and +verified.** `load_bg_file` (bg.c) no longer faults on BUNKER1 (or any of +the 20+ other levels the offline converter covers). This took five +findings, all committed: -## Current state (verified this session) +- **D78** `StandTile.id`/`.room` bitfield packing (MIPS-BE vs x86-LE) — + `#ifdef PORT` layout fix in `src/bondtypes.h`. +- **D79** `bg_room_data`'s 3 offset fields declared `void *` (never + dereferenced, but grows the record on x86-64 and breaks array indexing) + — `#ifdef PORT` `u32` fix in `src/game/bg.h`. +- **D80/D81/D82** the bg `.seg` + `Tbg_*_stanZ` format spec, the offline + converter (`tools_pc/d69_emit.py`), and the port-layer sidecar wiring + (`port/src/pccg.c`, `port/include/pccg.h`, wired into `romdata.c` / + `libultra.c` / `objecthandler_2.c`, same Plan-B pattern as + `pcmodels.c`/D50). +- **D83** `StandTileHeaderMid`/`StandTileHeaderTail` bitfield packing + (same MIPS-BE/x86-LE class as D78, found because it caused an actual + infinite loop in `stanBuildRoomData` once D78-D82 got that far) — + `#ifdef PORT` reversed-field-order fix in `src/bondtypes.h`. +- **D84** three hand-inlined `BG_SEG_TO_PTR`-style segment folds in + `bgLoadRoomVtxData`/`bgLoadRoomPrimaryGdl`/`bgLoadRoomSecondaryGdl` did + the `+0xf1000000` fold as real 64-bit pointer arithmetic (doesn't wrap + at 32 bits the way the macro's `(u32)` cast does) — `#ifdef PORT` fix, + same fold as `BG_SEG_TO_PTR`, in `src/game/bg.c`. -- **Build GREEN.** `export PATH="/c/msys64/mingw64/bin:$PATH" && ./build-pc.sh ntsc-final`. -- **D59–D68 resolved** (details in PCPortResearch.md §F): blood-RLE sentinel, - HEADS/BODIES signed sentinels, romCopyAligned/ramrom 64-bit, image_entry - layout, Globalimagetable BE→LE fixup (`port/src/gimgfixup.c`), etc. -- **D70–D74 (intro logo) resolved:** - - D70: `GE_PCDUMP` PPM frame capture (kept — permanent dev tooling). - - D71: C-array texture sources (rarewarelogo.c RGBA16 images) bswapped on - LE PC → port-layer per-source normalization in `import_texture`. - - D72: GE always uses authored `tc[]` UVs; `lookat_enabled` defaults false. - - **D73 (root cause of the old D72.3):** sinf/cosf `du` double constants - are big-endian word pairs → garbage on LE PC → guMtxF2L emitted −32768 - for every sin/cos entry → logo triangles projected off-screen. Fixed with - the `DVAL()` macro in `src/libultra/gu/guint.h` (PORT-only; N64 build - untouched). All scenes using guRotate/guLookAt-derived matrices were - affected, not just the logo. - - **D74:** texture import fallback no longer truncates valid gDPLoadBlock - data (was dropping letter mip chains and reducing D_02005FF0 to 32×3); - `TextureCacheKey` gained `size_bytes`; VBO path now wraps UVs by tile - size for WRAP sub-tiles (N64 semantics; the logo's 20×3 sub-tile at - offset (11.5, 29) was sampling row 0 instead of rows 29–31). -- **Runtime:** boots → intro music → full intro (Rareware logo with gold - letters, gun barrel w/ Brosnan, cast screen) → **crash in `load_bg_file` - (bg.c:830)** on first stage load. EXCEPTION 0xc0000005; the crash log at - repo root (`ge007.crash.log`, gitignored) is from this exact fault: - PC=0x14000627c, FAULT ADDR = Rax+0x28 (reading `pPointTableBin` through the - mis-rebased room-fileposition pointer). -- **Committed through D74.** This session's TEMP probes are all stripped; - previously committed TEMP diagnostics remain — strip list in Task 3. +**Regenerate the sidecar** with `python tools_pc/d69_emit.py ntsc-final` +(writes `data/pccg-ntsc-final/pccg.bin` + `manifest.csv`, gitignored like +`pcmodels-*`) before running — it is not checked into git. PAL/JPN are +implemented in the converter but **not regenerated or verified** (those +ROMs aren't present in this dev environment). -## Known issues — open (reported this session; tracked in PCPortResearch.md §F as D75–D77) +## What's still open (found *during* D69 verification, not the original bug) -Not part of the D69 task, but reported alongside it and **not yet root-caused** (OPEN): +**D85 (room DL binaries decode to garbage; safety-netted, not fixed).** +`bgLoadRoomPrimaryGdl`/`bgLoadRoomSecondaryGdl` decompress correctly (the +`11 72` RZ header and offset are right — D84 fixed that), but the content +`texLoadFromGdl` (the *same* runtime converter already used for model +GDLs) produces from the raw N64 bytes is not valid GBI opcodes. Not +root-caused. A `#ifdef PORT` bounds check was added in +`bgBuildRoomVtxBounds` so a garbage command stream can't segfault (it just +produces an empty/degenerate bounding box for that batch) — this is a +crash-safety net, not a geometry fix. **Next step:** figure out what +`texLoadFromGdl` actually does with room-specific GBI markers — room GDLs +reference `bgApplyDynamicCCRMLUT`/`ptrDynamic_CC_RM_LUT`/ +`DL_LUT_PRIMARY_ADDFOG`, suggesting a CC/RM-LUT-selection marker convention +models don't use — and verify `csize_primary_DL_binary`/ +`csize_secondary_DL_binary` sizing end-to-end (these come from subtracting +adjacent `bg_room_data` offsets; D79 argued the +delta cancels in the +subtraction, but that argument hasn't been independently re-verified with +a probe). -- **D75 (HIGH):** 3D rendering still wrong — Nintendo logo misplacement + intro cutscene - player models absent (Rareware logo is fine). Suspected D73 matrix-path scope gap *or* a - separate model/RW-pool bug (cf. D56) — the 75a/75b split must be decided first. -- **D76 (MED):** 2D partial draw — disclaimer screen shows only "game classification" + one - line. Suspected incomplete D68 image-table fixup for some entries this screen references. -- **D77 (LOW):** no audible music on PC despite a clean audio thread. Suspected SDL device / - mixer not wired up (distinct from D54/D54b, which only stopped crashes). +**D86 (new crash, likely unrelated to bg/stan).** With D85's safety net +in, the game reaches a **new** segfault in `modelInitRwData` (model.c:6174, +called from `modelInit`/`animInit`), dereferencing a bad `ModelNode` +pointer. This is the existing D50-D58 model pipeline (stable for every +intro/cast model) hitting a case that's never been exercised before — +most likely a room-instantiated object (light fixture / prop / character) +whose load path differs, or a pcmodels-manifest edge case. **Not +root-caused.** Next step: identify which model name is being loaded at +the crash site (add a probe in `load_object_fill_header` or +`modelInit`/`animInit` printing the header/model name) and check whether +it's in `data/pcmodels-ntsc-final/manifest.csv`. -**Prioritization guidance (owner, this session):** -- **D69 (BUNKER1 `load_bg_file` crash) stays the primary milestone task** — do not let the - D75–D77 visual/audio items displace it. -- Among D75–D77: **rendering (D75) is prioritized over audio (D77)** on the assumption the - render pipeline is the harder/riskier PC-port work and audio is simpler. -- **Exception:** first spend one check on Q1 (shared root cause across audio + rendering). If a - single blocker affects both, resolve it before splitting effort. Evidence so far says they are - independent subsystems → default to render-first. -- Each fix remains a narrow `#ifdef PORT` ABI/endianness exception documented as its own D7x - finding when resolved (N64 build untouched; game logic unmodified). +**Net effect:** the game now runs substantially further (through BUNKER1's +full room-streaming setup) before crashing, vs. the original instant fault +in `load_bg_file`. The "loads without fault" acceptance bar is **not yet +fully met** — D86 is the current hard blocker to reaching a rendered +frame. Recommended order: D86 first (probably a small, isolated fix, and +unblocks seeing whether D85's degenerate room geometry is otherwise +"plausible pixels" per the D69 bar), then D85, and only then worry about +D75 (pre-existing intro-3D findings, still open, still lower priority). -## The blocker, precisely (D69) +## Debug tooling added this session (kept, env-gated, zero cost when unset) -`load_bg_file` (src/game/bg.c:800+) for level BUNKER1: -1. `obLoadBGFileBytesAtOffset("bg/bg_sev_all_p.seg", header, 0, 0x40)` — - **works**. `&fileentry->hw_address[offset]` == `hw_address + offset`; - compiled asset symbols are absolute cart addresses (e.g. - `bg_sev_all_p_seg` @ 0x10438660) into the ROM mapped at CART_BASE, and the - PI shim memcpys from there. GE_D69 probe confirms: idx=1, rom_size=0x10DF0, - hw=0x10438660. -2. `((s32 *)ptr_bg_data)[1]` — **the bug**. ROM bytes at +4 are - `0F 00 00 14` (BE value 0x0F000014). N64: BG_SEG_TO_PTR folds - `off + 0xF1000000` → file offset 0x14. PC reads LE 0x1400000F → pointer - ~5 MB past the stack buffer → fault reading `[1].pPointTableBin`. -3. Same class applies to: every bg_room_data record (more 0x0Fxxxxxx offsets, - pPointTableBin at record+0x28 per the crash disasm), the rest of the .seg - header/tables, and the whole `Tbg_sev_all_p_stanZ` geometry file - (stanDetermineEOF/stanLoadFile in stan.c). +- `GE_D69STAN=1` — `stanBuildRoomData` (stan.c): logs the tile walk + (address/room/mid/tail/pointCount) for the first 20 tiles and any tile + whose derived tile-size would be 0 (the D83 symptom). +- `GE_D69BB=1` — `bgBuildRoomVtxBounds`/`bgLoadRoomPrimaryGdl` (bg.c): logs + the room's expanded-GDL pointer, vertex-buffer pointer, first 8 decoded + Gfx slots, and the compressed/decompressed primary-DL bytes around the + fold (D84/D85 diagnostic). +- Pre-existing `GE_D69=1` (ob.c, `obLoadBGFileBytesAtOffset`) — **left in + place** (HANDOFF previously said to strip it once D69 is solved; D69's + *original* bug is solved, but D85/D86 are still open follow-ups in the + same overall load path, so it stays useful for now). -**Strategy options** (decide after decoding; AGENTS.md prefers offline): -- **(a) Offline sidecar conversion** (the D43/Plan-B pattern): a tools_pc - script converts all bg/*.seg + Tbg_*_stanZ per region into one image + - manifest, served through the existing load path via a port-layer table - patch (cf. `port/src/pcmodels.c`). Pros: matches established pattern, no - runtime cost, regenerable per region. Cons: must fully decode both formats. -- **(b) Runtime port-layer fixup** after each load (hook where the bytes land). - Same format knowledge; smaller initial step if only a few fields matter for - first-frame render — but stage geometry is deep; likely most fields end up - needing conversion anyway. +## Environment / build -**Caveat:** PD's `port/src/preprocess/filebg.c` describes a *different* BG -format (zipped multi-section: primary/section1/2/3 headers). GE's .seg files -are raw with segment-0x0F internal offsets. Same engine family ≠ identical -format — use PD only as an approach reference; validate every field against -GE's own consumers (bg.c, stan.c, and the N64 disasm where in doubt). +- `export PATH="/c/msys64/mingw64/bin:$PATH" && ./build-pc.sh ntsc-final` + (~5 s). Build is GREEN with all D69 work included. +- Run from the **repo root**, not `build-pc/`. +- `GE_PCDUMP="2100-2700:10"` + `tools_pc/pixcount.py` for frame captures + once D86 is fixed and a frame actually renders past stage load. +- Crash log: `ge007.crash.log` (repo root); symbolicate with + `addr2line -e build-pc/ge007.x86_64.exe -f -C ` (image base + `0x140000000`). +- gdb is launch-mode only and far too slow for timing-dependent bugs; + prefer env-gated probes + the crash log (see the D85/D86 probes above + for the pattern: `#if defined(PORT)` / `getenv("GE_...")`). -## Read first (authoritative, in order) +## Non-negotiables (unchanged, see AGENTS.md) -1. `AGENTS.md` — non-negotiables, critical files, build, verification ritual. -2. `docs/PCPortResearch.md` **§F/D69** (blocker + strategy), then D68 - (Globalimagetable fixup — the template for BE data in this segment family), - D73–D74 (this session), §G status, and the environment reminders at the end - of §H. -3. The consumers: `src/game/bg.c` (`load_bg_file` ~line 795+, levelinfotable - line 183, BG_SEG_TO_PTR in bg.h:23), `src/game/stan.c` (stanLoadFile / - stanDetermineEOF), `src/game/ob.c` (obLoadBGFileBytesAtOffset — note the - TEMP D69 probe), `assets/obseg/file_resource_table.inc.c` (name→symbol map). -4. Reference for the offline pattern: `tools_pc/d43_emit.py` + - `port/src/pcmodels.c` + romdata cart-extension plumbing. - -## Task 1 — decode the formats - -1. Dump BUNKER1's .seg header + first records from the ROM (cart 0x10438660, - size 0x10DF0) and map each word to its consumer in bg.c: - - word0 = 0; word1 = room-fileposition-list offset (0x14); word2/word3 = - more table offsets (0x334 / 0x32C). - - Follow `ptr_bgdata_room_fileposition_list[1].pPointTableBin` etc. into - bg_room_data (bg.h) and note every offset field + its base - (BG_SEG_TO_PTR = base + off + 0xF1000000, i.e. stored as 0x0Fxxxxxx). -2. Do the same for `Tbg_sev_all_p_stanZ` (cart 0x1087C3F0, size 0x3DD0): - StanPrefixRecord + whatever stanLoadFile walks. Cross-check struct field - offsets against the N64 disasm of the consumer functions where the decomp - is ambiguous. -3. Write the format notes into PCPortResearch.md §F/D69 (or a new §I) as you - go — they are the converter spec. - -## Task 2 — implement + verify - -1. Implement the chosen conversion/fixup; wire it in (sidecar table patch or - load hook). -2. Verify: stage loads without fault, then renders frames (watch for the next - BE field class surfacing at render time — vertex tables, DL pointers, - texture refs inside stan data; expect a few iterations, D43 had ~10). -3. Once a stage renders: pixel-assert soak (PPM dump + tools_pc/pixcount.py), - then move on (next stages use the same files pattern — make sure all 20+ - levels' bg/stan files are covered, not just BUNKER1). - -## Task 3 — strip TEMP diagnostics before committing - -This session's probes (GE_DBGUV/DBGTRI/DBGTALL/DBGMAT/D74IMP/D74DUMP/DBGLOAD) -are already stripped. Previously committed TEMP diagnostics still in the tree -(grep `TEMP D` + `GE_D`): -- gfx_pc.cpp: **TEMP D63** — the dram-branch trail (`s_d63taskcount`, - `d63RecordBranch`/`d63DumpTrail`/`d63LogLastBranchReread` externs near the - top of gfx_run_dl, the call in the G_DL case that also spams `[NOTE] D63 - dram-branch` every frame, the GE_D63 dump block, and the function - definitions at the end of the file); D59 seg probes if still present. -- gfx_pc.cpp: `GE_D71LOG` one-shot normalize log (keep the normalization - itself — it's the D71 fix). -- front.c: `d63MenuProbe`/After + call sites. blood_animation.c: D63 - g_GfxMemPos probe. ob.c: D69 probe (both obLoadBGFileBytesAtOffset - versions) + its stdio/stdlib includes — **strip only after D69 is solved**. -- rsp.c, model.c, initanitable.c, language.c, title.c, ramromreplay.c, - n64stubs.c, libultra.c, romdata.c: leftover TEMP blocks from D51–D66 - (decide per item; D60's DMA target validation is a permanent safety net and - should stay). -- Scratch files at repo root (`run_*.log`, `all.txt`, `b3.c`, `buildtest.txt`, - etc. from earlier sessions) have been deleted; if new scratch files appear, - delete them before committing (patterns are already in .gitignore). - -## Standing procedure (per AGENTS.md) - -- N64 build untouched; game **logic** unmodified — narrow exception for - pointer-width/ABI-only edits under #ifdef PORT, each documented in - PCPortResearch.md §F/D3x style. -- Verification ritual after any build-affecting change: undefined symbols, - duplicates, syntax (`./build-pc.sh` is the final word). -- **Run the game from the repo root** (`cd REPO_ROOT && - ./build-pc/ge007.x86_64.exe`) — running from build-pc/ picks up the wrong - ROM (baserom.u.z64) and misses the pcmodels.bin sidecar → spurious frame-4 - crash in modelPromoteNodeOffsetsToPointers. -- `Gfx.words.w0/w1` are both uintptr_t (16-byte Gfx on PC); ROM-copied N64 - Gfx DLs stay 8 bytes in memory — walk them at 8-byte stride when scanning. -- N64 CPU/RDP data in ROM is big-endian; any raw `*(u32*)` read of ROM-copied - buffers needs a bswap under PORT (D68 pattern) or offline conversion. -- Heavy env-gated logging perturbs tick-driven timing (a full-triangle trace - hung the kernel heartbeat at ~frame 107 while clean runs run fine past - frame 2100) — use short targeted probe windows, and diagnose stability only - with clean runs. - -## Environment reminders - -- MSYS2 tools: `export PATH="/c/msys64/mingw64/bin:$PATH"`. Build ~5 s. -- Crash log `ge007.crash.log` includes FAULT ADDR, STACK@RSP window, and - module list — first stop for any fault. Symbolicate offline: - `addr2line -e build-pc/ge007.x86_64.exe -f -C `. -- gdb is launch-mode only and far too slow for timing-dependent crashes — - prefer env-gated probes + the crash log. -- Image base 0x140000000; DRAM V1 @ 0x70000000, V2 @ 0x80000000 (8 MB each); - CART_BASE = 0x10000000; model sidecar extends the cart region at - 0x10C00000. `osGetCount` advances at ~46.5525 ticks/µs (D52). -- ROM: data/ge007.ntsc-final.z64 (byte-identical to the No-Intro good dump); - sidecar: data/pcmodels-ntsc-final/ (gitignored, regenerate with - tools_pc/d43_emit.py per region). -- Artifacts from this session: `ppm/` frames ~510–590 are the logo sequence - (frame ~555 is the best front-facing "RARE" shot); `tools_pc/mtxtest.c` is a - scratch matrix-convention harness (kept for reference). PPM files from - timeout-killed runs can be truncated — validate header + pixel count before - parsing. +1. N64 build files untouched. +2. Game logic unmodified except narrow, documented `#ifdef PORT` + ABI/layout exceptions (D78/D79/D83/D84 this session, all logged in + `PCPortResearch.md` §F). +3. Offline sidecar conversion preferred over runtime fixup — D80-D82 + follow the D43/pcmodels pattern exactly. diff --git a/docs/PCPortResearch.md b/docs/PCPortResearch.md index 322ca9ae..ed82e40f 100644 --- a/docs/PCPortResearch.md +++ b/docs/PCPortResearch.md @@ -2627,6 +2627,121 @@ rewrites `hw_address`/`rom_size`). Regenerate: `python tools_pc/d69_emit.py this session (PAL/JPN ROMs not present in this environment) — `data/pccg-*/` is gitignored like `pcmodels-*`. +**D83 (RESOLVED — StandTileHeaderMid/StandTileHeaderTail bitfield ABI, +found during D69 verification).** After D78-D82 landed, a clean run reached +`stanBuildRoomData` (`stan.c:245`) without faulting, but then **hung +forever** (kernel heartbeat: no frame rendered, stuck at the same PC across +repeated snapshots). Root cause: same MIPS-BE-vs-x86-LE bitfield-packing +class as D78, but in a struct D78 didn't touch. `StandTileHeaderTail { +s16 pointCount:4; s16 headerC:4; s16 headerD:4; s16 headerE:4; }` — on +N64/MIPS the FIRST-declared field occupies the HIGH bits (`pointCount` = +top nibble); x86 GCC packs the first-declared field into the LOW bits +(`pointCount` = bottom nibble instead). `tile->tail.hdrTail.pointCount` is +read pervasively (`list_of_tilesizes[]` tile-size lookup used for +navigation, edge walks, `stanBuildRoomData`'s bounds loop) — with the stock +declaration this silently read the wrong nibble on PC. An env-gated probe +(`GE_D69STAN=1` in `stanBuildRoomData`, TEMP, kept) proved it directly: +tile tail=`0x03dc` (N64: pointCount=0, top nibble) decoded to +`pointCount=12` on PC (bottom nibble) — `list_of_tilesizes[12]` is +out-of-bounds (table has 12 entries, 0-11) and happened to read a stray 0, +so `tile` never advanced — infinite loop. Fixed under `#ifdef PORT` by +declaring both `StandTileHeaderMid` and `StandTileHeaderTail`'s fields in +**reverse order**: x86's low-to-high packing then lands each field in the +same bit position MIPS's high-to-low packing does (byte-identical numeric +result, verified via a union/probe against `0x03dc` returning +`pointCount=0`). `StandTileHeaderMid`'s fields (`special`/`r`/`g`/`b`) are +never read via their bitfield names either (only via `.mid.half >> 0xc` +elsewhere in stan.c) so that half of the fix is precautionary. Same +narrow-ABI-exception class as D78; no logic change. + +**D84 (RESOLVED — bg.c hand-inlined segment-fold 64-bit-pointer overflow, +found during D69 verification).** With D83 in, `stanBuildRoomData` +completed and the game proceeded into room streaming +(`bgCheckIfRoomModelNeedsLoad` → `bgLoadRoomModelData` → +`bgLoadRoomVtxData`/`bgLoadRoomPrimaryGdl`/`bgLoadRoomSecondaryGdl`), which +then **segfaulted** at a fixed, reproducible fault address +(`0x7104561d`, identical across runs) inside `bgBuildRoomVtxBounds` +(`bg.c:2852`, reading `vtx[i].v.ob[0]`). Root cause: `bgLoadRoomVtxData` / +`bgLoadRoomPrimaryGdl` / `bgLoadRoomSecondaryGdl` each hand-roll the +`BG_SEG_TO_PTR` fold instead of calling the macro: +`offset = (((u8 *)room->pPointTableBin + ptr_bg_data) - ptr_bg_data) + +0xf1000000;` (and the Pri/Sec-mapping equivalents). On N64 this "+base +-base" cancellation is a no-op inside 32-bit pointer arithmetic that wraps +for free. D79 made `pPointTableBin`/`pPriMappingBin`/`pSecMappingBin` +plain `u32` fields (never dereferenced, matching every other use site), but +these three call sites still cast them to `(u8 *)` and did the arithmetic +as real 64-bit pointers: `+0xf1000000` no longer wraps at 32 bits the way +`BG_SEG_TO_PTR`'s explicit `(u32)` cast does, so the computed `offset` +came out roughly 4 GiB too large, corrupting every downstream room-file +read (compressed-data location and size). Fixed under `#ifdef PORT` by +doing the fold as plain `u32` math at all three sites, matching +`BG_SEG_TO_PTR` exactly (`offset = (u32)room->pPointTableBin + +0xf1000000;`, no pointer involved) — same narrow ABI-exception class as +D79/D69's original BG_SEG_TO_PTR fix, no logic change. (Root-caused via an +env-gated probe, `GE_D69BB=1` in `bgBuildRoomVtxBounds`/ +`bgLoadRoomPrimaryGdl`, TEMP, kept — confirmed the compressed room-DL bytes +now start with the correct `11 72` RZ magic at the right file offset.) + +**D85 (OPEN — room primary/secondary DL binaries decode to garbage after +D84; safety-netted, not crash-fixed at the geometry level).** With D84 in, +the compressed room DL binary loads and decompresses correctly (verified: +`11 72` RZ header at the right offset, plausible decompressed size), but +the **content** `texCopyGdls`/`texLoadFromGdl` produce from it is not a +valid GBI command stream (`GE_D69BB=1` dump: `cmd=00`, `01`, `02`, `52`... +none of these are display-list opcodes actually present in the source +bytes — the raw N64 bytes are untouched by the offline converter (D80: the +whole per-room DL/point-index blob is a byte stream, deliberately left +unconverted, out of scope for this milestone) and `texLoadFromGdl` is the +*same, already-working* model-GDL runtime converter (`bgLoadRoomPrimaryGdl` +calls it identically to the model-loading path) — so either room GDLs use +a BG-specific command/marker convention `texLoadFromGdl`'s marker-expansion +logic doesn't handle, or something upstream of it (compression alignment, +`csize_primary_DL_binary`/`csize_secondary_DL_binary` delta sizing) is +still off. Not yet root-caused; full triage is D75-class 3D-pipeline work, +out of scope for this session. **Crash prevented, not geometry fixed:** +added a `#ifdef PORT` bounds check in `bgBuildRoomVtxBounds` before every +`vtx[i]` dereference (`vtxOff`/`vtxEnd` must fit inside +`usize_point_index_binary`) — a garbage command stream now produces an +empty/degenerate bounding box for that vertex batch instead of an +out-of-bounds read, so a bad room fails to render sanely rather than +segfaulting. Follow-up: decode what `texLoadFromGdl` actually does with +room-specific opcodes (`bgApplyDynamicCCRMLUT`/`ptrDynamic_CC_RM_LUT`/ +`DL_LUT_PRIMARY_ADDFOG` suggest room GDLs carry CC/RM-LUT-selection markers +models don't use) and verify `csize_*_DL_binary` sizing end-to-end. + +**D86 (OPEN — new downstream crash in `modelInitRwData` during +room-triggered model instantiation, found during D69 verification, likely +unrelated to the bg/stan conversion).** With D85's safety net in, +`bgBuildRoomVtxBounds` no longer segfaults, but the game reaches a **new** +crash a little further into room streaming: `modelInitRwData` +(`model.c:6174`) dereferences a bad `node` pointer, called from +`modelInit` ← `animInit` (`model.c:6323`/`6331`). This is the existing +model-loading pipeline (D50-D58, already working for every intro/cast +model) hitting an edge case that was never exercised before because +gameplay never progressed this far — most likely a room-instantiated +object (light fixture / prop / character) whose model load path differs +from the intro's, or a `RootNode`/pcmodels-table edge case not covered by +the existing 512-file conversion. **Not yet root-caused** — needs its own +investigation (probably starts with identifying which model name is being +loaded at the crash site and whether it's present in +`data/pcmodels-ntsc-final/manifest.csv`). Filed separately from D75 +(pre-existing 3D-render findings) since it's a hard crash, not a visual +defect; likely fixable independent of D75/D85. + +**D69 status after D78-D86: the ORIGINAL blocker (`load_bg_file` faulting +on first stage load) is fully resolved and verified** — a clean run loads +BUNKER1's header/room/portal/envdata tables and its full ~1066-tile stan +file correctly (spot-checked byte-for-byte against the N64 source via the +`GE_D69STAN` probe: tile room/mid/tail/point values match). The game now +progresses substantially further than before (through room-streaming +setup) before hitting D86's separate, newly-exposed crash. D85 (room +geometry renders wrong, not yet crash-free at the *visual* level) and D86 +(new model-load crash) remain open follow-ups — **the "loads without +fault" acceptance bar is not yet fully met** (the process still exits via +crash, just much later in the load sequence), but the converter, port +wiring, and every ABI fix identified so far are format-verified correct +and committed. + **Environment reminders.** MSYS2 tools in `/c/msys64/mingw64/bin/` (not on PATH — prefix `export PATH=…`). Build: `./build-pc.sh ntsc-final`. gdb **launch** mode only (attach fails, error 87) and it is **far too slow for timing-dependent diff --git a/src/bondtypes.h b/src/bondtypes.h index e23dba66..7b86b231 100644 --- a/src/bondtypes.h +++ b/src/bondtypes.h @@ -401,6 +401,40 @@ typedef union /* unused struct for clarity of intent */ +#ifdef PORT + /* D69 (bitfield ABI, same class as D78): MIPS/GCC (N64) packs the + * FIRST-declared bitfield into the HIGH bits of the storage unit; + * x86 GCC packs it into the LOW bits. `tile->tail.hdrTail.pointCount` + * is read pervasively by stan.c (list_of_tilesizes[] tile-size lookup, + * navigation, edge walks) -- with the stock (N64-source-order) + * declaration this silently reads the WRONG nibble on PC (verified via + * offsetof/union probe against the real project headers: source order + * reads the bottom nibble, `pointCount` needs the top one), producing + * out-of-range list_of_tilesizes indices and an infinite navigation + * loop (list_of_tilesizes[12..15] is out-of-bounds / reads a stray 0, + * so `tile` never advances). Declaring the fields in REVERSE order + * under PORT makes x86's low-to-high packing land each field in the + * same bit position MIPS's high-to-low packing does -- byte-identical + * numeric result, no behavior change. `special`/`r`/`g`/`b` are never + * read via their bitfield names either (only via `.mid.half >> 0xc` + * shifts elsewhere in stan.c), so this is precautionary but free. */ + typedef struct StandTileHeaderMid + { + u16 b : 4; + u16 g : 4; + u16 r : 4; + u16 special : 4;// 0=normal 1=kneeling 3=ladder + } StandTileHeaderMid; + + typedef struct StandTileHeaderTail + { + s16 headerE : 4; + s16 headerD : 4; + // Indices of the most extreme points (the resulting triangle should encompass _MOST_ of the tile) + s16 headerC : 4; + s16 pointCount : 4; // seen lh, not lhu. Also seen with an explicit unnecessary '& 0xF' + } StandTileHeaderTail; +#else typedef struct StandTileHeaderMid { u16 special : 4;// 0=normal 1=kneeling 3=ladder @@ -417,6 +451,7 @@ typedef union s16 headerD : 4; s16 headerE : 4; } StandTileHeaderTail; +#endif typedef struct StandTile { diff --git a/src/game/bg.c b/src/game/bg.c index 12aaeba2..f289b791 100644 --- a/src/game/bg.c +++ b/src/game/bg.c @@ -2266,7 +2266,19 @@ s32 bgLoadRoomVtxData(s32 roomnum, u8 *dst, s32 len) * The seemingly unncessary " + ptr_bg_data - ptr_bg_data" likely comes from paired bgdata pointer/offset macros. * Adding 0xf1000000 strips the 0x0f000000 segment tag, yielding a file offset. */ +#ifdef PORT + /* D69: pPointTableBin is `u32` on PC (D79), not a real pointer -- the + * original expression's "+ptr_bg_data -ptr_bg_data" cancellation is a + * no-op on N64 (32-bit pointers, wraps for free) but on PC promotes it + * to a genuine 64-bit pointer, then `+ 0xf1000000` no longer wraps at + * 32 bits the way BG_SEG_TO_PTR's `(u32)` cast does -- the intended + * fold (strip the 0x0F segment tag) silently stops happening and + * `offset` ends up ~4 GiB too large. Do the fold as plain u32 math + * (matching BG_SEG_TO_PTR exactly), no pointer involved. */ + offset = (u32)ptr_bgdata_room_fileposition_list[roomnum].pPointTableBin + 0xf1000000; +#else offset = (((u8 *)ptr_bgdata_room_fileposition_list[roomnum].pPointTableBin + ptr_bg_data) - ptr_bg_data) + 0xf1000000; +#endif obLoadBGFileBytesAtOffset(levelinfotable[levelentry_index].bg_seg_filename, dst + (len - alignedsize), offset, alignedsize); result = bgDecompress(dst + (len - alignedsize), dst); @@ -2309,14 +2321,40 @@ s32 bgLoadRoomPrimaryGdl(s32 roomnum, u8 *dst, s32 allocsize) // Load the compressed data into the end of the buffer, starting at dst. scratch = dst + (allocsize - size); +#ifdef PORT + /* D69: see the matching comment in bgLoadRoomVtxData. */ + fileoffset = (u32)ptr_bgdata_room_fileposition_list[roomnum].pPriMappingBin; + fileoffset += 0xf1000000; +#else fileoffset = (s32)((u8 *)ptr_bgdata_room_fileposition_list[roomnum].pPriMappingBin + ptr_bg_data) - ptr_bg_data; fileoffset += 0xf1000000; +#endif obLoadBGFileBytesAtOffset(levelinfotable[levelentry_index].bg_seg_filename, scratch, fileoffset, size); +#if defined(PORT) + if (getenv("GE_D69BB")) { + osSyncPrintf("D69bb primGdl room=%d fileoffset=%08x size=%d " + "compbytes=%02x%02x%02x%02x%02x%02x%02x%02x\n", + roomnum, (unsigned)fileoffset, (int)size, + scratch[0], scratch[1], scratch[2], scratch[3], + scratch[4], scratch[5], scratch[6], scratch[7]); + } +#endif + // Decompress from the end-of-buffer location at dst. expanded_size = bgDecompress(scratch, dst); +#if defined(PORT) + if (getenv("GE_D69BB")) { + osSyncPrintf("D69bb primGdl room=%d expanded=%d dstbytes=" + "%02x%02x%02x%02x%02x%02x%02x%02x\n", + roomnum, (int)expanded_size, + dst[0], dst[1], dst[2], dst[3], + dst[4], dst[5], dst[6], dst[7]); + } +#endif + /** * Copy the decompressed GDL back to the end of the buffer as scratch. * texLoadFromGdl can then read from scratch and write the final @@ -2384,8 +2422,14 @@ s32 bgLoadRoomSecondaryGdl(s32 roomnum, u8 *dst, s32 allocsize) // Load the compressed data into the end of the buffer, starting at dst. scratch = dst + (allocsize - size); +#ifdef PORT + /* D69: see the matching comment in bgLoadRoomVtxData. */ + fileoffset = (u32)ptr_bgdata_room_fileposition_list[roomnum].pSecMappingBin; + fileoffset += 0xf1000000; +#else fileoffset = (s32)((u8 *)ptr_bgdata_room_fileposition_list[roomnum].pSecMappingBin + ptr_bg_data) - ptr_bg_data; fileoffset += 0xf1000000; +#endif obLoadBGFileBytesAtOffset(levelinfotable[levelentry_index].bg_seg_filename, scratch, fileoffset, size); @@ -2805,13 +2849,35 @@ void bgBuildRoomVtxBounds(s32 roomID) cmdindex = 0; numpoints = 0; +#if defined(PORT) + /* TEMP D69: log the vtx-bounds walk (env GE_D69BB=1) */ + if (getenv("GE_D69BB")) { + osSyncPrintf("D69bb room=%d gdl=%p vertices=%p usizePrim=%d usizePts=%d\n", + roomID, (void *)gdl, (void *)vertices, + (int)g_BgRoomInfo[roomID].usize_primary_DL_binary, + (int)g_BgRoomInfo[roomID].usize_point_index_binary); + for (cmdindex = 0; cmdindex < 8; cmdindex++) { + osSyncPrintf("D69bb cmd[%d] cmd=%02x par=%08x addr=%08x\n", + cmdindex, gdl[cmdindex].dma.cmd, + (unsigned)gdl[cmdindex].dma.par, + (unsigned)gdl[cmdindex].dma.addr); + } + } +#endif + while (gdl[cmdindex].dma.cmd != G_ENDDL) { - if (gdl[cmdindex].dma.cmd == G_VTX) + if (gdl[cmdindex].dma.cmd == G_VTX) { numpoints++; } cmdindex++; + if (cmdindex > 100000) { +#if defined(PORT) + osSyncPrintf("D69bb room=%d runaway cmd walk, aborting\n", roomID); + return; +#endif + } } points = memaAlloc(ALIGN16(numpoints * sizeof(RoomVtxBatchBounds))); @@ -2847,6 +2913,29 @@ void bgBuildRoomVtxBounds(s32 roomID) vtx = (Vtx *)(SEGMENT_OFFSET(gdl[cmdindex].dma.addr) + (u32)vertices); +#if defined(PORT) + /* TEMP D69 safety net: the room primary/secondary DL binaries + * are raw N64-format bytes (untouched by the offline converter, + * D80/D82 -- runtime texLoadFromGdl is supposed to convert them + * the same way it converts model GDLs, but that conversion is + * not yet verified correct for room-specific GDL content, see + * docs/PCPortResearch.md D83). Until that's resolved, a + * corrupted command stream can produce a `vtx` pointer far + * outside the room's vertex buffer; bounds-check before + * dereferencing so a bad room fails to render (skips this + * batch) instead of segfaulting the process. */ + { + u32 vtxOff = SEGMENT_OFFSET(gdl[cmdindex].dma.addr); + u32 vtxEnd = vtxOff + (u32)numvertices * sizeof(Vtx); + if (vtxOff >= (u32)g_BgRoomInfo[roomID].usize_point_index_binary || + vtxEnd > (u32)g_BgRoomInfo[roomID].usize_point_index_binary) { + numpoints++; + cmdindex++; + continue; + } + } +#endif + for (i = 0; i < numvertices; i++) { if (points[numpoints].xmin > vtx[i].v.ob[0]) diff --git a/src/game/stan.c b/src/game/stan.c index 498f54c5..42d24d87 100644 --- a/src/game/stan.c +++ b/src/game/stan.c @@ -258,8 +258,30 @@ void stanBuildRoomData(void) tile = stan_prefix->ptr_firstroom; +#if defined(PORT) + /* TEMP D69: log the tile walk (env GE_D69STAN=1) */ + if (getenv("GE_D69STAN")) { + osSyncPrintf("D69stan start tile=%p firstroom_raw=%p\n", + (void *)tile, (void *)stan_prefix->ptr_firstroom); + } +#endif + while (*(u32 *)tile) { +#if defined(PORT) + /* TEMP D69: log the tile walk (env GE_D69STAN=1) */ + if (getenv("GE_D69STAN")) { + static s32 s_n = 0; + s32 pc = tile->tail.hdrTail.pointCount & 0xf; + if (s_n < 20 || pc == 0 || list_of_tilesizes[pc] == 0) + osSyncPrintf("D69stan n=%d tile=%p room=%d mid=%04x " + "tail=%04x pc=%d sz=%d\n", + (int)s_n, (void *)tile, tile->room, + (u16)tile->mid.half, (u16)tile->tail.half, + (int)pc, (int)list_of_tilesizes[pc]); + s_n++; + } +#endif if (tile->room != lastRoom) { lastRoom = tile->room;