forked from Ports/goldeneye-pc-port
PC port: phase 2 — handoff for D51 session: findings D50/D51, rewritten HANDOFF + research §G/§H
- §F/D50: Plan B execution record (emit pass, plumbing, lang/font fixups, legal-screen UB, texCopyGdls byte proof) - §F/D51: current blocker — import_texture_i8 via main-DL G_TEXRECT lazy tile-0 upload; GE opcode space facts (non-F3DEX numbering, G_TEXRECT=0xE4) - HANDOFF.md rewritten: Task 1 = live-breakpoint gdb recipe for D51, Task 2 = pixel assert + soak, Task 3 = Tbg_*_stanZ backgrounds - AGENTS.md phase status updated (D43/Plan B closed, D51 open)
This commit is contained in:
James Dansereau committed
1 parent
330b2bae46
commit
2c0c0515dd
3 files changed
+423
-439
No files matched your search
@@ -48,6 +48,6 @@ Run `/linkcheck` for this sweep. Record new findings in `docs/PCPortResearch.md`
|
||||
- **Compile+link milestone:** all ~235 TUs compile and `ge007.x86_64.exe` links clean (`ninja ge007 -k 0`, 236/236). Findings D7–D17 in `docs/PCPortResearch.md` §11; asset stubs in `port/src/assetstubs.c`, fast3d no-ops in `port/src/gfxstub.c` (delete when real fast3d lands).
|
||||
- **Phase 1 (boot to window):** done — ROM loads/validates/maps at cart base 0x10000000 (`romdata.c`), SDL2 window + GL clear loop (`video.c`), absolute asset symbols from `scripts/gen_romassets.py` → `port/src/romassets_<r>.s` (findings D18–D23).
|
||||
- **Phase 1.5 (boot to first frame):** DONE — boot chain complete AND real frames render (`[NOTE] frame N rendered in … us`). D34–D42 closed the blockers on top of D31–D33: ANIM_DATA lvalues, music seq-table ABI+endianness, music-heap sizing, libaudio bank-tree re-layout, implicit-declaration prototype shim (D38), Globalimagetable rebasing + true segment size (D39), ModelHitEntry pool sizing (D40), cross-thread GL context release (D41), rsp task-settings toggle (D42). ROM dump verified against the No-Intro good dump (byte-identical, header CRCs re-checked — `tools_pc/romverify.c`).
|
||||
- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + first frames render; current blocker is **D43** (model-file loading ABI mismatch: N64-layout ROM model files read as 8-byte-pointer PC structs → D37-style re-layout). **D43 status:** all 512 model files validated; buffer sizing finalized by the D46 overlap-safety analysis; step-2 mechanical game-code edits APPLIED + committed (ecefffdb); **D47 closed every open converter question** — compaction state values, G_VTX encoding, opcode-aware remap rule, vertex formats — spec implemented + validated 512/512 by `tools_pc/d43_convert.py` (max D_PC = 0xB7E0, ratio ≤ 1.31). **D48 review re-plan:** the old "port d43_convert.py 1:1 to C" task is discarded — that script does not emit bytes, and **Plan B (offline pre-conversion)** is now the default: Python emit pass → `data/pcmodels-<region>/` RZ sidecars served via a port-layer `hw_address`/`rom_size` table patch (no C converter; full mechanics in §F/D48). **D49 review PASSED** — all 8 D48 checklist items CONFIRMED against code + ROM (§F/D49): RZ format, memcpy DMA path, table chokepoint, obInit timing, buffer fit (worst-case P_final 0x16F74 vs 0x24C400 bank), reload hazard + pending one-line reset, per-region sidecars (EU=465 files; no JP ROM candidate yet), emission spec complete (fresh C probe: no record-size drift; added the bswap16-record-scalars rule). Next session EXECUTES Plan B (HANDOFF Task 2); fallback Plan A = D47.13 C converter with the corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`). All 23 d43_*.py scripts moved from gitignored build-pc/ to tracked `tools_pc/`. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48).
|
||||
- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + real frames render. **Plan B EXECUTED (D50, committed):** all 512 NTSC model files converted offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` + manifest.csv (single concatenated RZ image; regenerate per region with the script — sidecars are gitignored), served through the existing load path via a port-layer table patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot `pcmodelsPatchTable()` + poolRemaining reset hook in `load_object_fill_header`); runtime C fixups for language-bank BE offset tables + font re-layout (D50.3/D50.4); legal-screen UB seed (D50.5); **D50.6 fixed the first model-render crash** — `texCopyGdls` copied only w0 of each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes), leaving stale mempool w1s in the compacted GDLs; byte-proven against PlegalpageZ, fixed with a full-slot copy under #ifdef PORT. Model GDLs now execute to completion (PlegalpageZ sub-DLs). **Current blocker D51:** SIGSEGV in `import_texture_i8` via a main-DL G_TEXRECT lazy tile-0 upload (frame 5) — GE opcode space decoded (G_TEXRECT=0xE4/0xE5, non-F3DEX numbering; see §F/D51); investigation recipe in HANDOFF Task 1. Next: D51 → pixel-assert soak (PPM dump + pixcount still unbuilt) → next asset type (Tbg_*_stanZ backgrounds, same offline pattern; PD `preprocess/filebg.c` reference). Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48; full-struct Gfx copies on PC — D50.6).
|
||||
- **Phase 3 (audio + input):** libaudio → SDL — largely copy-and-adapt from PD (`mixer.c` 722 lines vs our 31-line stub; `input.c` SDL_GameController backend; §2.4); decide the ASP strategy (C2: `aspMain*` dummies in `ucode.c`).
|
||||
- **Phase 4 (saves + polish):** file-backed EEPROM — build on PD's `fs.c` (file-backed save dir); PFS/motor already no-op stubs.
|
||||
+275
-328
@@ -1,374 +1,321 @@
|
||||
# Handoff brief — GoldenEye 007 PC port (Phase 2: Plan B review PASSED (D49) — next session executes the offline pre-conversion)
|
||||
# Handoff brief — GoldenEye 007 PC port (Phase 2: Plan B executed (D50) — next session diagnoses D51, the main-DL texture-upload crash)
|
||||
|
||||
_Paste-ready briefing. Authoritative detail lives in `AGENTS.md` and
|
||||
`docs/PCPortResearch.md` (§F/D31–D48 findings, §H handoff); this is the summary
|
||||
+ the immediate tasks._
|
||||
`docs/PCPortResearch.md` (§F/D43–D51 findings); this is the summary + the
|
||||
immediate tasks._
|
||||
|
||||
## Your job
|
||||
|
||||
- ~~Session A: independent review of Plan B (D48)~~ — **DONE (D49, committed):**
|
||||
all 8 checklist items CONFIRMED against code + ROM; Plan B green-lit. One
|
||||
spec amendment recorded there: bswap16 every u16/s16 record scalar on emit
|
||||
(see Verified facts).
|
||||
- **Session B (next): EXECUTE Plan B** per §Task 2 — Python emit pass +
|
||||
sidecar generation, ~60 lines of port-layer plumbing, one-line hook,
|
||||
build + pixel-assert soak. Commit at each working checkpoint
|
||||
(`PC port: phase 2 — <what>`). The goal: get stage geometry on screen.
|
||||
- **Session C (next): diagnose + fix D51** — SIGSEGV in `import_texture_i8`
|
||||
via a main-DL G_TEXRECT lazy tile-0 upload (frame 5). Recipe in Task 1.
|
||||
Then build the pixel assert (Task 2), run a soak, commit at each milestone
|
||||
(`PC port: phase 2 — <what>`). The goal: get past frame 5 with actual model
|
||||
geometry on screen, provable by pixel count.
|
||||
|
||||
Work agentically — fix → build (~5 s) → verify → commit at each milestone.
|
||||
|
||||
## Read first (authoritative, in order)
|
||||
1. `AGENTS.md` — non-negotiables, critical files, build, verification ritual.
|
||||
2. `docs/PCPortResearch.md` **§F/D49** (review verdict + quantified fit
|
||||
numbers), **D48** (the plan being executed — read fully), then **D47**
|
||||
(converter contract — still ground truth for the emit pass),
|
||||
**D46** (overlap safety + buffer sizing), **D45** (buffer constants),
|
||||
**D43** (the model-file task), §2.4 (PD-port copy-candidate audit).
|
||||
3. The code paths D48 cites: `port/src/rzdecomp.c`, `port/src/libultra.c`
|
||||
(`piServiceDma`), `port/src/romdata.c` (`romdataInit`,
|
||||
`romdataCartAddrValid`), `src/ramrom.c`, `src/game/ob.c` (`load_resource`,
|
||||
`fileIndexLoadToBank`, `fileSetSize`, `obInit`), `src/game/objecthandler_2.c`
|
||||
(`load_object_fill_header`).
|
||||
2. `docs/PCPortResearch.md` **§F/D50** (what the last session did, incl. the
|
||||
texCopyGdls byte proof), **D51** (current blocker + GE opcode facts), then
|
||||
D49/D48/D47 (sidecar contract — still ground truth for regeneration).
|
||||
3. The crash-chain code: `port/fast3d/gfx_pc.cpp` (`gfx_run_dl` G_TEXRECT case
|
||||
~line 2457, `import_texture*` ~600–950, `seg_addr` ~2282,
|
||||
`gfx_sp_moveword` ~2290s), `src/game/tex.c` (texCopyGdls + texLoadFromGdl).
|
||||
|
||||
## Non-negotiables (the ones that bite)
|
||||
- N64 build files (`Makefile`, `tools/`, `rsp/`, `ld/`) — never touch.
|
||||
- **Game logic is unmodified.** Hardware deps go in `port/`. Narrow exception:
|
||||
mechanical, semantics-preserving ABI/layout fixes forced by the 32→64-bit
|
||||
transition (embedded pointers → u32 + cast at use; PC-guarded pool sizing /
|
||||
idiom branches), each documented as a D3x finding. No behavior changes.
|
||||
(Plan B's only game-code change is the PORT-guarded hook block in
|
||||
`load_object_fill_header` — bookkeeping reset + one-shot patch call, no logic.)
|
||||
idiom branches; UB-exposure seeds), each documented as a D3x/D50 finding.
|
||||
No behavior changes. The N64 path stays verbatim under `#else`.
|
||||
- Region macros in `CMakeLists.txt` must mirror the N64 Makefile exactly.
|
||||
|
||||
## Current state
|
||||
- **D49 (this handoff's predecessor, committed): Plan B review PASSED** — all 8
|
||||
D48 checklist items CONFIRMED with evidence (§F/D49). Quantified: Σ round8(C)
|
||||
= 1,277,088 B; worst-case post-compaction P_final = 0x16F74 vs 0x24C400 stage
|
||||
bank (25× headroom); all dst!=0 buffers pass against current constants;
|
||||
fresh C probe re-verified every PC record size + promotion offset (no drift).
|
||||
Notes: romdataInit has no JP ROM candidate (JP sidecars moot for now);
|
||||
EU region has 465 model files vs 512 (generator maps what exists); the
|
||||
poolRemaining=0 reset is still pending (Task 2 step 3).
|
||||
- **D43 status (latest review session, §F/D48):** investigation complete;
|
||||
converter spec finalized (D47) and validated 512/512 (`tools_pc/d43_convert.py`,
|
||||
max D_PC = 0xB7E0, ratio ≤ 1.31). **The remaining task was re-planned:** the
|
||||
old plan ("port d43_convert.py 1:1 to C as `romdataFixupModelFile`") is
|
||||
discarded because (a) d43_convert.py does not emit bytes — the emission code
|
||||
is unimplemented in any language, and (b) **Plan B (offline pre-conversion)**
|
||||
reuses the existing ROM load chain and eliminates the C converter entirely:
|
||||
a Python emit pass writes 512 PC-layout RZ sidecars to
|
||||
`data/pcmodels-<region>/`; the port layer serves them by patching
|
||||
`file_resource_table[i].hw_address` + `resource_lookup_data_array[i].rom_size`
|
||||
(full mechanics in D48.2). Plan A (D47.13 C converter) remains the fallback,
|
||||
with a corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`, not
|
||||
`D_PC ≤ avail`).
|
||||
- DONE + committed (ecefffdb, pushed): all step-2 mechanical game-code edits
|
||||
(bondtypes.h Vertex u32 union; model.c PROMOTE32; chr.c casts;
|
||||
objecthandler_2.c Textures offset; tex.c Gfx-size/dispatch/byte-check/
|
||||
copy-count fixes), D45/D46 buffer constants (gun.c/front.c/initmenus.c),
|
||||
fast3d seg_addr seg-5 raw-VMA case (port/fast3d/gfx_pc.cpp).
|
||||
- DONE this review session: **all 23 `d43_*.py` scripts moved from gitignored
|
||||
`build-pc/` to tracked `tools_pc/` and committed** — the spec is now
|
||||
versioned. Converter re-run from new location: ALL CLEAN.
|
||||
- Verified environment facts (do not rediscover): frames 1–4 render then FATAL
|
||||
at `modelPromoteNodeOffsetsToPointers()` (model.c:5688) = D43; black screen
|
||||
is clear color only (no stage geometry loaded yet). ROM dump byte-identical
|
||||
to the No-Intro good dump (`tools_pc/romverify.exe`). Crash log with real
|
||||
backtraces works (D44). Build clean 237/237; full rebuild ~5 s.
|
||||
- **Plan B is EXECUTED (D50, committed):** all 512 NTSC model files converted
|
||||
offline by `tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin`
|
||||
+ `manifest.csv` (single concatenated RZ image, 16-aligned offsets, decimal
|
||||
manifest); port plumbing in `port/src/pcmodels.c` (+ `port/include/pcmodels.h`,
|
||||
romdata.c/h cart-extension); one-shot `pcmodelsPatchTable()` + poolRemaining=0
|
||||
reset hook in `load_object_fill_header` (objecthandler_2.c, #ifdef PORT).
|
||||
Boot log: `[INFO] pcmodels: table patched (512 model entries)`.
|
||||
- **Runtime C fixups landed (D50.3/D50.4):** language-bank BE offset tables
|
||||
(`romdataFixupLangBank`, called from language.c after each load + lazily in
|
||||
the langGetJpnCharPixels paths) and font re-layout
|
||||
(`romdataFontPcSize`/`romdataFixupFont` in textrelated.c). These are
|
||||
load-time C fixups, NOT offline sidecars — small blobs on different load
|
||||
paths; fine to keep runtime.
|
||||
- **Legal-screen UB seed (D50.5):** front.c `constructor_menu00_legalscreen`
|
||||
read an uninitialized pointer (N64 survived by luck); PORT branch seeds it
|
||||
with `legalpage_text_array`. Identical lookat.
|
||||
- **First model-render crash fixed (D50.6):** `texCopyGdls` copied only w0 of
|
||||
each 16-byte Gfx slot on PC (the union's trailing `long long` is 8 bytes),
|
||||
so the compacted GDLs kept stale mempool w1s → garbage G_SETTIMG → OOB in
|
||||
`import_texture_rgba16`. Byte-proven against PlegalpageZ (sidecar w1
|
||||
0x050012C8 vs RAM garbage 0xACAAB819 at file offset 0x2488). Fixed with a
|
||||
full-slot copy under #ifdef PORT. **Model GDLs now execute to completion.**
|
||||
- **Current blocker — D51:** SIGSEGV on frame 5 in `import_texture_i8`
|
||||
(gfx_pc.cpp:782) via `G_TEXRECT` (main DL offset +0x260) → lazy upload of
|
||||
RDP tile 0 with an invalid/stale source. See Task 1. Frames 1–4 render
|
||||
successfully in every run (deterministic).
|
||||
|
||||
## Task 1 — Plan B review checklist (Session A) — DONE (D49: 8/8 CONFIRMED)
|
||||
## Task 1 — D51 diagnosis (the immediate task)
|
||||
|
||||
Verify each claim. For each: state CONFIRMED or REFUTED in D49 with the
|
||||
evidence you used (file:line, command output). If a falsification criterion
|
||||
fires, stop and record it — do not paper over it.
|
||||
The failing blit: 32×48 px screen rect (236,92)–(268,140), tile 0, i8 format.
|
||||
G_TEXRECT carries no source address — it uses RDP texture-memory tiles; the
|
||||
fault is in `import_texture(tile=0)` reading `rdp.textures[0]`'s stored source.
|
||||
Work the suspects in order:
|
||||
|
||||
**R1. RZ format is reproducible.** Claim: every model ROM region is
|
||||
`[0x11 0x72][raw deflate]`; `decompressdata()` (port/src/rzdecomp.c) skips
|
||||
exactly those 2 bytes and inflates raw (`inflateInit2(-15)`) with a generous
|
||||
avail_in bound, so a sidecar file in that format works through
|
||||
`load_resource` unmodified. Verify: read rzdecomp.c; confirm the 2-byte prefix
|
||||
is constant across all 512 (one-liner over `scripts/filelist.u.csv` + ROM —
|
||||
`tools_pc/d43_convert.py` already decompresses all 512 with `[2:], -15`, re-run
|
||||
it). **Falsified if:** the prefix varies per file, or decompressdata depends on
|
||||
header state beyond 2 bytes.
|
||||
1. **Dump live (post-mortem dumps FAIL — 0x70052e80 reads "Cannot access
|
||||
memory" after the crash).** gdb launch mode:
|
||||
```
|
||||
gdb -batch \
|
||||
-ex "handle SIGSEGV stop" \
|
||||
-ex "break import_texture_i8" \
|
||||
-ex run \
|
||||
-ex "print rdp.textures[0]" \
|
||||
-ex "up 4" \
|
||||
-ex "print cmd" -ex "print dListStart" \
|
||||
-ex "x/256wx <dListStart>" \
|
||||
./build-pc/ge007.x86_64.exe
|
||||
```
|
||||
(Adjust `up` depth to land in `gfx_run_dl`; the DL start may not be
|
||||
0x70052e80 every run — it's a mempool bump allocation. Alternatively break
|
||||
at `osSpTaskStartGo` and read `t->t.data_ptr` / `data_size`.)
|
||||
2. **Find the G_SETTIMG (w0 top byte 0xFD) slots in the dumped main DL** and
|
||||
decode each w1 through the seg_addr rules (raw 0x05xxxxxx → seg5+low24;
|
||||
marked LSB → segmentPointers[seg]+addr; unmarked <0x10000000 →
|
||||
segmentPointers[seg]+low24; <0x800000 → +0x80000000; else passthrough).
|
||||
Also dump `segmentPointers[]` at the breakpoint. If a SETTIMG w1 resolves
|
||||
to an OOB/invalid pointer → that's the bug (converter remap miss, or a new
|
||||
address class seg_addr doesn't handle).
|
||||
3. **If no main-DL SETTIMG sets tile 0 before the G_TEXRECT:** the tile is
|
||||
stale/uninitialized — check whether the game expects a tile set by an
|
||||
earlier frame's DL (fast3d `rdp` persistence across frames) or by a sub-DL
|
||||
that never ran.
|
||||
4. **If the source pointer is valid but points at wrong-format/wrong-size
|
||||
data:** the game may point it at a ROM/ramrom image needing a port fixup
|
||||
(D50.3/D50.4 class) — check what asset lives there and how N64 PD handles
|
||||
the analogue.
|
||||
5. **Find who emits the G_TEXRECT** in game code: grep `src/game/` for
|
||||
`gSPTileRect\|G_TEXRECT\|gSPTextureRectangle` (likely image.c / front.c /
|
||||
title.c — legal-screen or logo 2D blit). Reading the emitter tells you
|
||||
which texture it means and where its source should come from.
|
||||
|
||||
**R2. The DMA path is a plain host memcpy.** Claim: `romCopy` (src/ramrom.c) →
|
||||
`osPiStartDma` → port shim `piServiceDma` (port/src/libultra.c) = `memcpy` from
|
||||
`(void*)srcPA`, gated ONLY by `romdataCartAddrValid()` (port/src/romdata.c:212,
|
||||
checks `addr ≥ CART_BASE && off+size ≤ romSize`). The ROM is VirtualAlloc'd at
|
||||
CART_BASE 0x10000000 sized exactly romSize — so extending the reservation by
|
||||
the sidecar total and extending the validity bound serves sidecars with zero
|
||||
other changes. Verify: read both shims end-to-end; confirm no other gating
|
||||
(alignment, hardcoded 0x10C00000 limits elsewhere, cache effects). Note
|
||||
`romCopyAligned` exists but is NOT used by this load path. **Falsified if:**
|
||||
any additional gate rejects non-ROM host pointers or sizes we can't satisfy.
|
||||
Record findings as **D51.x** in §F; fix in `port/` where possible (seg_addr /
|
||||
tile setup), PORT-guarded game-code only if ABI/layout-forced. Rebuild,
|
||||
confirm the crash moves past frame 5, then Task 2.
|
||||
|
||||
**R3. The resource table is the single choke point and is writable.** Claim:
|
||||
`file_resource_table` is a plain (non-const) global defined via include in
|
||||
src/game/ob.c:22; ALL 512 model loads go through `load_object_fill_header`
|
||||
(dst==0 → `_fileNameLoadToBank`, custom-buffer callers in front.c/gun.c/
|
||||
bondview2.c/initmenus.c → `_fileNameLoadToAddr`), both reading `hw_address` via
|
||||
`load_resource`; no game code dereferences C*/G*/P*Z asset symbols directly
|
||||
outside the table. Verify: grep callers of `fileLoad` /
|
||||
`load_object_fill_header`; grep every model symbol (e.g. `CcamguardZ`,
|
||||
`PflagZ`) outside `assets/obseg/file_resource_table.inc.c`,
|
||||
`port/src/romassets_*.s`, and N64-only files (`obseg.h` decls, `ob_seg.s`).
|
||||
**Falsified if:** any compiled game code reads model bytes straight from the
|
||||
cart (it would bypass the sidecar and crash on N64 layout).
|
||||
## Task 2 — pixel assert + soak (build it now — dev-loop item #1)
|
||||
|
||||
**R4. Patch timing: after obInit, before first model load.** Claim: `obInit()`
|
||||
(src/boss.c:179) runs before the first model load and computes `rom_size` from
|
||||
adjacent-entry hw_address DELTAS (ob.c:122) — so the patch must run after it
|
||||
and set `rom_size` explicitly; nothing between obInit and the first model load
|
||||
consumes model entries' rom_size/hw_address. Verify: trace the init chain
|
||||
(bossInitMainthreadData → … → first stage object load; the D43 crash stack
|
||||
shows the order); grep `rom_size` uses (ob.c:122, 253, 291;
|
||||
`obLoadBGFileBytesAtOffset` is BG-only). **Falsified if:** anything in that
|
||||
window reads model entries' rom_size/hw_address, or obInit can run after a
|
||||
model load.
|
||||
Env-gated framebuffer PPM dump every N frames in `port/video.c`
|
||||
(e.g. `GE_PCDUMP=1` → `build-pc/frame_%05d.ppm`) + `tools_pc/pixcount.py`
|
||||
(counts non-clear-color pixels; diffs two captures). Milestone: pixel count
|
||||
rises above a small threshold and stays up through a 30 s+ soak. Screenshot
|
||||
diffs against Mupen64Plus (verified No-Intro dump) are the ground truth for
|
||||
"looks right". Commit: `PC port: phase 2 — D51 …` then `… pixel assert +
|
||||
soak`.
|
||||
|
||||
**R5. Load-flow fit for pre-converted data.** Claim: with sidecars, a fresh
|
||||
dst==0 load allocates S_bank = whole remaining stage bank; the compressed
|
||||
source placement `round8(compressed)+8 ≤ S_bank` holds (max single-file
|
||||
compressed ≪ ~2.3 MB bank); `load_resource` sets poolRemaining := D_PC
|
||||
automatically; compaction headroom R = S_bank satisfies D46's strict bound with
|
||||
P_conv = D_PC (D46 already proved the buffers against these sizes). ADDITIONALLY:
|
||||
for every dst!=0 caller, the patched compressed size + 8 ≤ the caller's buffer
|
||||
size (gun.c 0x18000/0xBD70/0x17000, front.c wallet 0x17000 / cast screen
|
||||
0x25000, initmenus.c logo 0x85000, bondview2.c weaponbuf `size0`). Verify: have
|
||||
the emit pass (or a quick Python pass) print per-file compressed sizes and
|
||||
cross-check against the D45/D46 constants; re-run `tools_pc/d43_chainbound.py`.
|
||||
**Falsified if:** any buffer/chain is undersized for its file.
|
||||
## Task 3 — next asset type (after D51 + soak)
|
||||
|
||||
**R6. Reload hazard + the one-line reset.** Claim: `fileSetSize` (ob.c:346-347)
|
||||
leaves poolRemaining = rom_remaining = post-compaction size; without a reset, a
|
||||
same-stage reload allocates S_bank' = P_old which can be < round8(compressed)+8
|
||||
→ `load_resource` hits the `source − ptrdata < 8` branch → poolRemaining = 0 →
|
||||
silent load failure. A PORT-guarded `poolRemaining = 0` reset immediately before
|
||||
`_fileNameLoadToBank` restores fresh-bank semantics every load with identical
|
||||
steady-state usage. Verify: read `fileSetSize`, `fileIndexLoadToBank`,
|
||||
`load_resource`; grep consumers of `get_pc_remaining_buffer_for_index` (should
|
||||
be compaction only) to confirm the reset clobbers nothing. **Falsified if:**
|
||||
the reset breaks a consumer, or the hazard cannot actually occur (then drop the
|
||||
reset and say so in D49).
|
||||
Tbg_*_stanZ stage backgrounds (different format/path than models). SAME
|
||||
offline pattern (D48): pin the format with Python over ROM ground truth, emit
|
||||
+ validate offline, serve through the existing load path. PD's
|
||||
`port/src/preprocess/filebg.c` is the per-field reference (§2.4) — but same
|
||||
family ≠ identical format; validate every field against GE headers + ROM.
|
||||
|
||||
**R7. Region handling.** Claim: sidecars are derived from a specific region ROM;
|
||||
the generator takes the region, reads `data/ge007.<region>.z64`, writes
|
||||
`data/pcmodels-<region>/`; romdataInit picks the directory matching the ROM it
|
||||
loaded (same token logic as ROM discovery in `romdataInit`). Verify: read
|
||||
`build-pc.sh` + `romdataInit`'s ROM discovery; confirm name sets are identical
|
||||
across regions is NOT required (the generator maps whatever files exist).
|
||||
**Falsified if:** the build flow can't regenerate sidecars per region, or
|
||||
romdataInit can't tell which directory matches.
|
||||
## Verified facts (sidecar spec — ground truth; do NOT re-derive)
|
||||
|
||||
**R8. Emission spec completeness.** Claim: the "Verified facts" section below +
|
||||
D47.13 cover every field of every opcode present in ROM (1,2,4,8,9,10,12,13,
|
||||
15,18,21,22,23,24), and the round-trip validator (§Task 2 step 1) catches any
|
||||
miss. Verify: walk each `ModelRoData_*Record` in src/bondtypes.h and confirm an
|
||||
emission rule exists for every pointer/float/int field; spot-check 2–3 PC
|
||||
record sizes against a fresh C probe (the Python `PC_REC` table was
|
||||
probe-verified once — re-confirm it hasn't drifted). **Falsified if:** any
|
||||
field has no rule, or a PC record size is wrong.
|
||||
|
||||
**Review output:** D49 in §F (8 items adjudicated + evidence), HANDOFF status
|
||||
updated, committed + pushed. If all CONFIRMED → Session B executes §Task 2.
|
||||
|
||||
## Task 2 — Execution checklist (Session B; only after D49 confirms)
|
||||
|
||||
1. **Emit pass in Python** (`tools_pc/d43_emit.py`, importing/reusing
|
||||
`d43_convert.py`'s walk + region map). For each of the 512 files, emit the
|
||||
PC image per D47.13 + Verified facts below: switches (NS×8B LE) →
|
||||
texconfigs (NT×12B; bswap TextureID u32) → nodes+records in DFS preorder
|
||||
(PC layout mirroring bondtypes.h; every promoted pointer field =
|
||||
zero-extended u64 of `0x05|new_off`; Primary/Secondary GDL ptrs remapped but
|
||||
NOT promoted; BaseAddr emitted 0) → vertex arrays immediately after their
|
||||
record (normal: bswap s16 x,y,z,index,s,t, bytes @C–F raw; collision: bswap
|
||||
s16 x,y,z,index + remap LinkedTo u32@8 + bswap CollisionRelatedIndex s16@C /
|
||||
reserved s16@E; nv==0 with non-null ptr → size up to next object offset,
|
||||
D47.10) → PointUsage (2×numVertices s16, after op24 collision verts) →
|
||||
**GDLs LAST, contiguous, visit order, 16B LE slots**: w0' = bswap32(w0);
|
||||
w1' = bswap32(raw) EXCEPT seg-5 of {G_VTX=0x04, G_SETTIMG=0xFD,
|
||||
G_LOADBLOCK=0xF3} → remap low24 via the region map; **no LSB set**; emit up
|
||||
to and including ENDDL. Write each file as `[0x11 0x72] + raw deflate(PC
|
||||
image)` (Python: `zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
|
||||
zlib.DEFLATED, -15)`) into `data/pcmodels-<region>/` plus a
|
||||
`manifest.csv` (`name,offset,size`) with offsets assigned contiguously in
|
||||
`file_resource_table` order (parse the quoted names from
|
||||
`assets/obseg/file_resource_table.inc.c`).
|
||||
**Round-trip validator (mandatory before trusting output):** re-parse each
|
||||
emitted image as PC layout — 48B node stride, `PC_REC` record sizes, LE Gfx
|
||||
slots; assert every promoted pointer equals the expected new offset, every
|
||||
seg-5 GDL w1 target resolves inside the image, D_PC matches the pass-1 size.
|
||||
Report per-file compressed sizes (feed R5's cross-check).
|
||||
2. **Port-layer plumbing in `port/src/romdata.c`** (~60 lines):
|
||||
- `romdataInit`: if `data/pcmodels-<region>/manifest.csv` exists for the
|
||||
loaded ROM, reserve the extended region — VirtualAlloc at CART_BASE sized
|
||||
`romSize + sidecarTotal` (aligned), memcpy the ROM as today, then load the
|
||||
sidecar blob(s) at `[CART_BASE + romSize, …)` per manifest offsets. Log
|
||||
count + total bytes. Missing directory → warn and continue (ROM-only
|
||||
mode = current behavior).
|
||||
- Extend `romdataCartAddrValid` to accept up to
|
||||
`CART_BASE + romSize + sidecarTotal`.
|
||||
- New `void romdataPatchModelSidecars(void)` — one-shot (static flag): for
|
||||
each `file_resource_table[i]` whose name is in the manifest, set
|
||||
`hw_address = (u8*)(CART_BASE + romSize + off)` and
|
||||
`resource_lookup_data_array[i].rom_size = size`. `extern` both arrays from
|
||||
ob.c (same pattern as the existing `resource_lookup_data_array` extern).
|
||||
3. **Hook in `load_object_fill_header`** (src/game/objecthandler_2.c, ONE
|
||||
`#if defined(PORT)` block, D48-class bookkeeping — document as D50):
|
||||
(a) call `romdataPatchModelSidecars()` (lazy one-shot; obInit has run by
|
||||
first model load — R4); (b) reset
|
||||
`resource_lookup_data_array[fileGetIndex(name)].poolRemaining = 0`
|
||||
immediately before the `_fileNameLoadToBank`/`_fileNameLoadToAddr` call
|
||||
(R6). No other game-code changes.
|
||||
4. **Build, run, verify + pixel assert (build it NOW — dev-loop item #1):**
|
||||
env-gated framebuffer PPM dump every N frames in `port/video.c`
|
||||
(e.g. `GE_PCDUMP=1` → `build-pc/frame_%05d.ppm`) + `tools_pc/pixcount.py`
|
||||
(counts non-clear-color pixels; diffs two captures). Milestone: no SIGSEGV
|
||||
in `modelPromoteNodeOffsetsToPointers`; pixel count rises above a small
|
||||
threshold and stays up through a 30 s+ soak. If geometry is wrong/missing,
|
||||
the converted sidecars are inspectable OFFLINE (Python) — that's the point
|
||||
of Plan B; don't gdb the game first.
|
||||
5. **Commit checkpoints** (`PC port: phase 2 — …`); record findings D49/D50+
|
||||
in §F; update AGENTS.md phase status + this file. Then move on to the next
|
||||
asset type (Tbg_*_stanZ backgrounds) with the SAME offline pattern — PD's
|
||||
`port/src/preprocess/filebg.c` is the per-field reference (§2.4).
|
||||
|
||||
## Task 3 — Plan A fallback (only if D49 refutes Plan B)
|
||||
|
||||
D47.13 C converter in `port/src/romdata.c`: two-pass walk+layout then emit,
|
||||
staged at `F + R − D_PC`, memmove to F; hook after the filedata assignment and
|
||||
before Switches/Textures/RootNode setup; set `poolRemaining = D_PC` exactly
|
||||
(never rom_remaining); poolRemaining=0 reset before `_fileNameLoadToBank`.
|
||||
**Use the corrected guard (D48.4): fail if `D_N64 + D_PC > avail`** — the
|
||||
staging region overlaps the live N64 image otherwise. The Python emit pass from
|
||||
Task 2 step 1 is still worth writing first: it becomes the byte-level oracle to
|
||||
diff the C output against (convert one file in C, compare with Python output).
|
||||
|
||||
## Verified facts (converter spec — ground truth for the emit pass; do NOT re-derive)
|
||||
**Sidecar serving (D50):**
|
||||
- Sidecars live in `data/pcmodels-<region>/` = **one concatenated
|
||||
`pcmodels.bin` + `manifest.csv` (`name,offset,size`, DECIMAL — C parser
|
||||
uses strtol base 10; rows in file_resource_table.inc.c order)**. NOT
|
||||
per-file .bin. Regenerate: `python tools_pc/d43_emit.py [ntsc-final|
|
||||
pal-final|jpn-final|--check-only]` (needs the region ROM in data/).
|
||||
- RZ format = `[0x11 0x72][raw deflate]`. Python decompress for inspection:
|
||||
`zlib.decompress(raw[2:], -15)` after checking the 2-byte header.
|
||||
- pcmodels.c copies the whole blob to `[CART_BASE+romSize, …)` (cart base
|
||||
0x10000000, romdataInit reserves romSize+sidecarTotal;
|
||||
romdataCartAddrValid accepts the extension) and patches
|
||||
`file_resource_table[i].hw_address = cartBase+romSize+off` +
|
||||
`resource_lookup_data_array[i].rom_size = C_pc` per manifest row — one-shot,
|
||||
from `load_object_fill_header` (after obInit has run). The existing
|
||||
romCopy/decompressdata path then serves PC bytes unmodified and sets
|
||||
poolRemaining = D_PC.
|
||||
- **Inspecting a loaded file offline:** find the name in manifest.csv →
|
||||
decompress that slice → it is exactly what lands at the file base in RAM
|
||||
(verified byte-for-byte for PlegalpageZ). GDLs are at the tail of the image
|
||||
in visit order; `x/Nwx <filebase+off>` in gdb compares directly.
|
||||
|
||||
**Format / layout:**
|
||||
- All 512 C*/G*/P*Z model files (Tbg_*_stanZ backgrounds are a different format/path — later) decompress fine; each file starts directly with the switches array (no in-file header — NS/NT come from the exe `ModelFileHeader` record via `objheader->numSwitches/numtextures`).
|
||||
- File layout: `[switches NS×4B][textures NT×12B][root node @ 4NS+12NT]…`; all embedded pointers are vma `0x05xxxxxx` (mask 0xFFFFFF).
|
||||
- **Tiling invariant verified, 0 failures in all 512 files** (`tools_pc/d43_invariants.py`): `[0,D)` is exactly tiled by {switches, textures, nodes(24B), records(RSZ per opcode), vertex arrays(nv×16B), collision arrays(nc×16B), PointUsage (exactly numVertices s16 entries — +2 cases are pure alignment padding), GDLs (each extends to the next GDL start or D; 8-byte aligned, inter-GDL gaps copied as NOOP)}. Records INTERLEAVE with vertex arrays in character bodies — do not assume "all records before bulk".
|
||||
- **ModelNode = 24B N64** (u16 op + 2 pad + 5×4B ptrs: Data@4, Parent@8, Next@0xC, Prev@0x10, Child@0x14) / **48B PC**. Older "20B/40B" notes are wrong — verified against raw hex stride 0x18.
|
||||
- **Opcodes present: 1,2,4,8,9,10,12,13,15,18,21,22,23,24** (HEADER, GROUP, DL, LOD, BSP, BOX, GUNFIRE, SHADOW, INTERLINK, SWITCH, GROUPSIMPLE, DLPRIMARY, HEADPH, DLCOLLISION). op22 N64: numVertices s32@0, Vertices@4, Primary@8, BaseAddr@0xC (size 0x10); only 5 files have it.
|
||||
- Record sizes N64/PC: OP01 0x10/0x18, OP02 0x1C/0x28, OP04 0x14/0x28, OP08 0x10/0x18, OP09 0x24/0x30, OP10 0x1C/0x1C, OP12 0x28/0x30, OP13 0x20/0x30, OP15 0x1C/0x1C, OP17 0x20/0x28, OP18 0x08/0x10, OP21 0x14/0x14, OP22 0x10/0x20, OP23 0x02/0x02, OP24 0x20/0x40 (compiler-verified). Field layouts in `src/bondtypes.h` (ModelRoData_*Record); read N64 fields from packed offsets, emit via the real PC layout — don't hand-compute PC offsets.
|
||||
- **f32 fields are big-endian in ROM** — bswap all f32 (record origins, BoundingVolumeRadius, Scale, Min/MaxDistance) and every s16 in vertex arrays; also bswap32 the TextureID u32 in the texture table (texLoad reads `*updateword & 0xffff` as the texture number).
|
||||
- **u16/s16 record scalars are big-endian too (D49 amendment)** — bswap16 every u16/s16 scalar field in every record (AnimPart, MatrixIndex, JointID, MatrixIDs, Group1/2, RwDataIndex, op4 numVertices@0x10, op24 nv/ncv/ModelType/RwDataIndex, …); padding/reserved fields may be zeroed. Data-verified: these hold small BE values (JointID 1–11, nv ≤ 73) that raw emission would corrupt ×256 on an LE read.
|
||||
- **Vertex arrays must stay 16B-stride** (GDL gSPVertex hardcodes 16B/vertex; propobj.c copies with sizeof(Vertex); model.c:4360 allocates numVertices×4 words). bswap every s16 field BE→LE. Main Vertices array is pure data; **CollisionVertices carry LinkedTo@8 = vma of a ModelNode** — remap to the new node offset (PROMOTE promotes them after rebase).
|
||||
- **GDLs:** 8B BE word pairs → 16B LE slots: `w0=bswap32(rom_w0); w1=bswap32(rom_w1)` — **leave w1 raw, NO LSB set** (ROM convention; fast3d's extended seg_addr case handles unmarked segmented addresses — D47.13). GE uses STANDARD libultra GBI numbering (include/PR/gbi.h); opcodes live in the TOP byte of w0. **Opcode-aware remap: only {G_VTX=0x04, G_SETTIMG=0xFD, G_LOADBLOCK=0xF3} carry addresses in w1** (seg-5 → remap low24 via region map). All other opcodes get bswap-only — G_TRI4 (GE extension 0xB1) w1 is 4-bit vertex-index data, NOT an address (D47.6); same for TRI1/TEXTURE/SET*MODE/CLEARGEO/SETGEO/syncs. **G_VTX w0 is GBI1-style** (`04<<24 | dst<<16 | 16·n`, dst=((n−1)<<4)|v0; all 2826 have v0=0, n≤16) — bswap only, no field remap (D47.5). ROM scan: all 2805 seg-5 VTX refs have even offsets → lossless; no nested G_DL in model files.
|
||||
- **Marker format:** top byte of w0 = 0xC0; type = `w0 & 7` (only 0/2/3/4 exist); texnum = `w1 & 0xfff`; min = top byte of w1. texLoadFromGdl dispatches on the top byte — PC fix: `(u8)(in->words.w0 >> 24)`. Type-0/1 markers with a valid tex are SKIPPED by the game (no expansion); the rest expand per D45's K table, which D46 verified as a true worst-case bound (no LUT textures exist; 99% of textures have maxlod=0 → TileLods emits nothing; state dedup resets per GDL via sub_GAME_7F0CC4C8).
|
||||
- **seg-5 VTX offsets must be remapped** — all 2805 point into op4/op22 vertex arrays that move during re-layout (PexplosionbitZ: op4 with numVertices=0 but GDL loads 16 verts from offset 0x98 — the GDL's VTX count is authoritative, not the record field).
|
||||
- PROMOTE contract: emit every promoted pointer as zero-extended u32 (low word of the 8B slot, high word 0); `PROMOTE` (`(u32)var + diff`, vma 0x5000000) then works unmodified. Fields promoted per opcode (model.c switch): all nodes Data/Parent/Next/Prev/Child; OP01 FirstGroup; OP02 ChildGroup; OP04 Vertices; OP08 Affects; OP09 left/rightChild; OP18 Controls; OP24 Vertices/CollisionVertices/PointUsage/CollisionVertices[i].LinkedTo. **NOT promoted:** GDL Primary/Secondary (stay raw 0x05xxxxxx), BaseAddr (game overwrites with fileramaddr).
|
||||
- All 512 C*/G*/P*Z model files decompress fine; each file starts directly with
|
||||
the switches array (no in-file header — NS/NT come from the exe
|
||||
`ModelFileHeader` record via `objheader->numSwitches/numtextures`).
|
||||
- PC file layout: `[switches NS×8B][texconfigs NT×12B][root node @ 8NS+12NT]…`;
|
||||
all embedded pointers are vma `0x05xxxxxx` (mask 0xFFFFFF). N64 was
|
||||
`[NS×4][NT×12][root @ 4NS+12NT]`.
|
||||
- **Tiling invariant verified, 0 failures in all 512 files**
|
||||
(`tools_pc/d43_invariants.py`): the image is exactly tiled by {switches,
|
||||
textures, nodes(48B PC), records(PC_REC per opcode), vertex arrays(nv×16B),
|
||||
collision arrays(nc×16B), PointUsage (numVertices s16 entries), GDLs (each
|
||||
extends to the next GDL start or D)}. Records INTERLEAVE with vertex arrays
|
||||
in character bodies — do not assume "all records before bulk".
|
||||
- **ModelNode = 24B N64 / 48B PC** (u16 op + 2 pad + 5 ptrs: Data@4/8,
|
||||
Parent, Next, Prev, Child). Record sizes N64/PC: OP01 0x10/0x18, OP02
|
||||
0x1C/0x28, OP04 0x14/0x28, OP08 0x10/0x18, OP09 0x24/0x30, OP10 0x1C/0x1C,
|
||||
OP12 0x28/0x30, OP13 0x20/0x30, OP15 0x1C/0x1C, OP17 0x20/0x28, OP18
|
||||
0x08/0x10, OP21 0x14/0x14, OP22 0x10/0x20, OP23 0x02/0x02, OP24 0x20/0x40
|
||||
(compiler-verified by `tools_pc/d43_layoutprobe.c`). Opcodes present:
|
||||
1,2,4,8,9,10,12,13,15,18,21,22,23,24.
|
||||
- **Endianness:** f32 fields BE in ROM → bswap; every u16/s16 record scalar
|
||||
BE → bswap16; vertex-array s16s BE → bswap16 (x,y,z,index,s,t); TextureID
|
||||
u32 → bswap32. Padding/reserved zeroed.
|
||||
- **Vertex arrays stay 16B-stride** (GDL gSPVertex hardcodes it). Collision
|
||||
vertices carry LinkedTo@8 = vma of a ModelNode — remapped.
|
||||
- **GDLs:** N64 8B BE word pairs → PC 16B LE slots: `w0=bswap32(rom_w0);
|
||||
w1=bswap32(rom_w1)`; **no LSB set** (fast3d's seg_addr handles unmarked +
|
||||
raw-0x05). Opcode-aware remap: only {G_VTX=0x04, G_SETTIMG=0xFD,
|
||||
G_LOADBLOCK=0xF3} carry addresses in w1; seg-5 low24 remapped via the region
|
||||
map. **0xB1 (G_TRI4) w1 is packed 8-bit vertex indices — NEVER an address.**
|
||||
G_VTX w0 is GBI1-style (`04<<24 | dst<<16 | 16·n`) — bswap only.
|
||||
- **Marker format:** top byte of w0 = 0xC0; type = `w0 & 7`; texnum =
|
||||
`w1 & 0xfff`. texLoadFromGdl expands markers into RDP commands (that's the
|
||||
compaction growth); default case copies the full 16B slot as-is — **the
|
||||
sidecar must contain final correct w1 values** (no runtime remap in
|
||||
texLoadFromGdl).
|
||||
- **PROMOTE contract:** promoted pointers emitted as zero-extended u64 with
|
||||
low32 = VMA `0x05xxxxxx`; `PROMOTE` (`(u32)var + diff`, vma 0x5000000) works
|
||||
unmodified. GDL Primary/Secondary NOT promoted (raw VMAs); BaseAddr=0 (game
|
||||
overwrites with fileramaddr).
|
||||
- **texCopyGdls contract (D50.6):** on PC it must copy the FULL 16-byte slot
|
||||
(`*arg1 = *arg0` under #ifdef PORT) — the N64 `long long` idiom copies only
|
||||
w0 on x86-64 and poisons the compaction scratch. texLoadFromGdl then reads
|
||||
the scratch and writes back at the original GDL offsets; final fileSetSize
|
||||
shrinks the bank chunk.
|
||||
|
||||
**Load path (single hook point):**
|
||||
- ALL model loads go through `load_object_fill_header()` (objecthandler_2.c:97): dst≠0 → `_fileNameLoadToAddr(name,0,dst,size)` (region = caller's bytes — see D45 for the grown per-region sizes); else `_fileNameLoadToBank(name,0,0x100,4)`. **Fresh-load semantics (D47.2-4):** `fileIndexLoadToBank` takes ALL remaining MEMPOOL_STAGE space (`S_bank`) only when poolRemaining==0; then F := alloc(S_bank), R := S_bank, load_resource decompresses D into F and sets P := decompressed size (D_N64 from ROM; **D_PC directly for sidecars — no fixup needed**). At compaction: **delta = R − P** (positive on fresh loads — the GDL mirror moves FORWARD). `fileSetSize(reallocate=1)` → `mempAddEntryOfSizeToBank` rewinds the bank cursor to the post-compaction size, giving the tail back.
|
||||
- **Reload hazard + PC fix (D47.4, needed by BOTH plans):** fileSetSize leaves P=R=post-compaction size, so a same-stage reload would alloc(P_old) which can be too small for the compressed source placement → `load_resource` sets poolRemaining=0 → silent failure. PORT-guarded reset of poolRemaining=0 before the load call forces fresh-bank semantics every load (staging space = whole remaining bank; steady-state usage identical).
|
||||
- **Plan A only (fallback):** the fixup must set ONLY poolRemaining (= D_PC exactly), never rom_remaining; stage at F+R−D_PC with guard `D_N64 + D_PC ≤ avail` (D48.4); hook after filedata assignment, before Switches/Textures/RootNode setup + sub_GAME_7F075A90 (PROMOTE walks PC ModelNode stride — the image must already be re-laid-out).
|
||||
**GE opcode space (include/PR/gbi.h, non-F3DEX numbering — verified D51):**
|
||||
- DMA: G_SPNOOP=0, G_MTX=1, G_MOVEMEM=3, G_VTX=4, G_DL=6, G_SPRITE2D_BASE=9.
|
||||
- IMM (G_IMMFIRST=−65): TRI1=0xBF, CULLDL=0xBE, POPMTX=0xBD, MOVEWORD=0xBC,
|
||||
TEXTURE=0xBB, SETOTHERMODE_H=0xBA, SETOTHERMODE_L=0xB9, ENDDL=0xB8,
|
||||
SETGEOMETRYMODE=0xB7, CLEARGEOMETRYMODE=0xB6, LINE3D=0xB5, RDPHALF_1=0xB4,
|
||||
RDPHALF_2=0xB3, TRI2=0xB2, **G_TRI4(GE)=0xB1** (gbi_extension.h).
|
||||
- RDP pass-through: G_SETTIMG=0xFD, G_LOADTXC=0xFE, G_SETCIMG=0xFF,
|
||||
**G_TEXRECT=0xE4 / G_TEXRECTFLIP=0xE5** (GE-specific values — NOT libultra's
|
||||
0x46/0x45), plus VI2D=0xF5 etc.
|
||||
- `gSPSegment(pkt,seg,base)` = `gMoveWd(pkt,G_MW_SEGMENT,seg*4,base)` →
|
||||
w0=(0xBC<<24)|(seg*4<<8)|6, w1=data. fast3d gfx_sp_moveword stores data as-is
|
||||
when ≥0x800000 else +0x80000000.
|
||||
- Main DL is game-built per frame with gSP* macros (16B slots) — NOT from a
|
||||
sidecar. Segment table: seg0=0, seg1=0x7029F800-ish, seg3=render_pos,
|
||||
seg4=runtime vtx buffer, seg5=file base (set per object).
|
||||
|
||||
**Compaction contract (sub_GAME_7F0762E0, objecthandler_2.c:24-83) — the tightest constraint (applies to both plans):**
|
||||
- It mirrors the GDL region [first_gdl_off, P_conv) into the caller-region's tail headroom (texCopyGdls to [A+R−P_conv+g, A+R)), then rewrites each GDL in place via texLoadFromGdl (expands texture markers into RDP commands), writing back starting at the original first-GDL offset; final fileSetSize = end of last rewritten GDL (= P_conv + 16·M_actual). **D46: fit and no-clobber are the SAME condition — R ≥ P_conv + 16×Σ(K_t−1) worst case — and all buffers satisfy it (see §F/D46 for the strict bound + the two cast-screen corrections).**
|
||||
- Per-GDL `count` = byte distance between CONSECUTIVELY VISITED gdls → **GDLs must be packed contiguously at the tail of the PC image, in modelIterateDisplayLists visit order.**
|
||||
- `modelIterateDisplayLists` (model.c:6254) **MUTATES the tree while walking**: LOD → `node->Child = LOD.Affects`; SWITCH → `node->Child = Switch.Controls`; BSP → `modelApplyReorderRelationsByArg(node,TRUE)` splices siblings (leftChild group before rightChild). Visit order per DL node: Primary, then Secondary (if non-null and ≠ previous), then DFS Child/Next/Parent.
|
||||
- `tools_pc/d43_gdlorder.py` implements this exact simulation to verify visit-order adjacency — all 512 files pass (total GDLs 2301).
|
||||
- Primary/Secondary are NOT promoted (raw 0x05xxxxxx), so modelNodeReplaceGdl's `Primary == find` comparisons work unchanged; after compaction Primary = replacementgdl (raw value, arbitrary parity).
|
||||
|
||||
**Render-time GDL resolution:**
|
||||
- The game emits `gSPDisplayList(gdl, rwdata->gdl)` with w1 = raw 0x05xxxxxx; segment 5 (SPSEGMENT_MODEL_COL1) is set to BaseAddr (= file base) by the preceding gSPSegment. fast3d's `seg_addr` (port/fast3d/gfx_pc.cpp:2282) resolves raw 0x05xxxxxx via `segmentPointers[5] + (w1 & 0xFFFFFF)` — **DONE, committed.**
|
||||
- Segments 3/4 (MTX table / vertex buffer) are set by the game per-render via gSPSegment ✓. BG segments (13/14/15) belong to the background path — later task.
|
||||
**Load path / compaction (unchanged by D50 — see D47/D46/D48 for full detail):**
|
||||
- ALL model loads go through `load_object_fill_header()`; dst≠0 →
|
||||
`_fileNameLoadToAddr` (caller's buffer), else `_fileNameLoadToBank` (fresh
|
||||
bank when poolRemaining==0 — the hook resets it every time).
|
||||
- Compaction: texCopyGdls mirrors [G,D) to tail scratch [B−D+G,B);
|
||||
texLoadFromGdl rewrites each GDL in place with marker expansion; final size
|
||||
= D + 16·Σ(K_t−1) worst case (P_est model — the emit pass's cross-checks
|
||||
encode it for every dst!=0 buffer and chain).
|
||||
|
||||
## Helper scripts (tools_pc/, tracked; run from repo root)
|
||||
- `d43_convert.py` — **reference layout + validator (D47):** DFS walk with LOD/SWITCH rewiring, region map, per-opcode record/vertex/GDL analysis, remap checks; runs on all 512 ROM files (ALL CLEAN). Lays the foundation for the emit pass (Task 2 step 1) — import/reuse its walk + region map.
|
||||
- `d43_invariants.py` — full ownership tiling check (0 failures / 512).
|
||||
- `d43_gdlorder.py` — exact visit-order simulation; all 512 pass.
|
||||
- `d43_sizes.py` / `d43_sizes2.py` — decompress all 512, report N64/PC sizes + per-file worst-case P_final.
|
||||
- `d43_walk.py` / `d43_fullwalk.py` / `d43_tree_dump.py` — tree walks, opcode histogram. (Header names lack the C/G/P prefix and trailing Z: file `CcamguardZ` → header `camguard`; all header files are literally named `modelFileHeader.inc.c` in per-model dirs.)
|
||||
- `d43_gdldump.py` / `d43_gdlhist.py` / `d43_gdlseq.py` — GDL command dumps/histograms/sequences.
|
||||
- `d43_seg5.py` / `d43_seg5ops.py` / `d43_seg5vtx.py` / `d43_seg5vtx2.py` — seg-5 VTX reference analysis (all 2805 point into op4/op22 vertex arrays → must remap).
|
||||
- `d43_cover.py` — verifies every seg-5 VTX + SETTIMG target falls inside a modeled object (2804/2805; the one exception is PexplosionbitZ's orphan array at 0x98 — handled by the zero-vtx rule).
|
||||
- `d43_lutscan.py` — proves NO LUT textures (formats 9-12) and maxlod distribution across all 512 files (D46).
|
||||
- `d43_chainbound.py` — computes D46's strict bound for every buffer/chain vs the PC region sizes.
|
||||
- `d43_decode.py` / `d43_layout*.py` / `d43_pointusage.py` / `d43_vtxfmt.py` — earlier investigation (format pinning).
|
||||
- Gotcha: MODELFILEHEADER's name is field a[0] of the macro args (NOT the filename).
|
||||
- `d43_emit.py` — **THE emit pass (D50.1):** converts all 512 files, validates
|
||||
round-trip + tiling + buffer fits, writes pcmodels.bin + manifest.csv.
|
||||
`--check-only` runs validation without writing. First stop for any
|
||||
"sidecar looks wrong" question.
|
||||
- `d43_layoutprobe.c` — compiler probe that re-verifies every PC record/node/
|
||||
header size (run with the exact CMake flags; compare against d43_emit.py's
|
||||
tables after any bondtypes.h change).
|
||||
- `d43_convert.py` — reference layout + validator (D47); foundation of the
|
||||
emit pass.
|
||||
- `d43_invariants.py` / `d43_gdlorder.py` / `d43_sizes*.py` / `d43_chainbound.py`
|
||||
/ `d43_cover.py` / `d43_lutscan.py` / `d43_seg5*.py` / `d43_gdl*.py` — the
|
||||
investigation suite (all pass on NTSC).
|
||||
- Gotcha: MODELFILEHEADER's name is field a[0] of the macro args (NOT the
|
||||
filename); nsnt lookup keys by header symbol (`armourguard`), not file base
|
||||
name (`CarmourguardZ`).
|
||||
|
||||
## Dev-loop speedups (priority order)
|
||||
1. **Frame capture + pixel assert** — NOW (Task 2 step 4): env-gated PPM dump every N frames in `port/video.c` + `tools_pc/pixcount.py` (non-clear-color pixel count / diff two captures). Turns "the scene contains actual model geometry" into an assert, makes the 30 s soak scriptable, and enables screenshot diffs against Mupen64Plus with the verified No-Intro dump as ground truth.
|
||||
2. **`tools_pc/sym.sh`** — tiny wrapper: takes a PC or rel offset, adds image base `0x140000000`, runs `addr2line -e build-pc/ge007.x86_64.exe -f -C`. One command per crash; re-verify the image base with `info address` after any link-layout change.
|
||||
3. **`tools_pc/romdump.py`** — Python ROM/asset inspector: parse ModelFileHeader/ModelNode trees using the bondtypes.h layout, print opcode sequences / node links / Data records for a given cart address; also re-parse SIDECAR files as PC layout (post-Plan B this is the first stop for "geometry looks wrong").
|
||||
4. D44 (automatic backtraces) — DONE, committed.
|
||||
1. **Frame capture + pixel assert** — NOW (Task 2): env-gated PPM dump every N
|
||||
frames in `port/video.c` + `tools_pc/pixcount.py`. Turns "the scene contains
|
||||
actual model geometry" into an assert and makes the 30 s soak scriptable.
|
||||
2. **`tools_pc/sym.sh`** — tiny wrapper: PC or rel offset → add image base
|
||||
0x140000000 → `addr2line -f -C`. Re-verify the image base with `info
|
||||
address` after any link-layout change.
|
||||
3. Sidecar inspection is now trivial offline (manifest + zlib, see Verified
|
||||
facts) — don't gdb the game for "geometry looks wrong" in a model file.
|
||||
4. D44 automatic backtraces — DONE; `ge007.crash.log` is the first stop.
|
||||
|
||||
## PD port as standing reference (always available)
|
||||
`PD_PORT_CHECKOUT` is the Perfect Dark PC port — same Rare engine
|
||||
family, and the **standing reference/guidance source** for this project (noted in
|
||||
`PD_PORT_CHECKOUT` is the Perfect Dark PC port — same Rare
|
||||
engine family, the **standing reference** for this project (noted in
|
||||
AGENTS.md). Consult it whenever a remaining work item has a PD analogue:
|
||||
- **Phase 2 asset conversion:** `port/src/preprocess/` (~4,100 lines) — per-ROM-segment
|
||||
N64→PC layout converters hooked from their `romdata.c`. `filemodel.c` = D43 near-analogue
|
||||
(same vma 0x5000000; PD converts at LOAD time — we convert OFFLINE for speed, same field
|
||||
rules apply); `filebg.c` (expected follow-on), `segaudio.c` (cross-check for our D37
|
||||
bank-tree fixup), `filelang.c`, `filetiles.c`, `filepads.c`, `filesetup.c`, `gbi.c`,
|
||||
`segfonts.c`.
|
||||
- **Phase 3/4 (copy-and-adapt; our stubs were scaffolded for this):** `port/src/mixer.c`
|
||||
(722 lines vs our 31 — libaudio→SDL mixer), `input.c` (1,551 vs 48 — SDL_GameController
|
||||
backend with hotplug/rumble/keyboard; remap buttons to GE's scheme), `fs.c` (294 vs 81 —
|
||||
file-backed save dir for Phase 4 EEPROM/PFS).
|
||||
- **Caveats:** copy port-layer files only (their game code was modified for PC); same
|
||||
family ≠ identical format — validate every conversion per-field against GE headers + ROM
|
||||
ground truth. Full audit: `docs/PCPortResearch.md` §2.4.
|
||||
- **Phase 2 asset conversion:** `port/src/preprocess/` (~4,100 lines) —
|
||||
per-ROM-segment N64→PC layout converters hooked from their `romdata.c`.
|
||||
`filemodel.c` = D43 near-analogue (PD converts at LOAD time — we convert
|
||||
OFFLINE for speed, same field rules apply); `filebg.c` (expected follow-on),
|
||||
`segaudio.c`, `filelang.c`, `filetiles.c`, `filepads.c`, `filesetup.c`,
|
||||
`gbi.c`, `segfonts.c`.
|
||||
- **Phase 3/4 (copy-and-adapt; our stubs were scaffolded for this):**
|
||||
`port/src/mixer.c` (722 lines vs our 31), `input.c` (1,551 vs 48 —
|
||||
SDL_GameController backend; remap buttons to GE's scheme), `fs.c` (294 vs 81
|
||||
— file-backed save dir).
|
||||
- **Caveats:** copy port-layer files only (their game code was modified for
|
||||
PC); same family ≠ identical format — validate every conversion per-field
|
||||
against GE headers + ROM ground truth. Full audit: §2.4.
|
||||
|
||||
## Standing procedure (you will hit more of these)
|
||||
Every ROM-serialized struct with a pointer field faults the same way once you reach more
|
||||
asset loading. The D32 fix procedure (doc §H): at the fault, `ptype /o <Struct>`; if a
|
||||
pointer field's offset/size diverges from its N64 offset comment, change it to `u32` + cast
|
||||
at the use sites; verify the load-time rebase yields valid V1 addresses (< 0x80000000);
|
||||
**also check the ROM bytes' endianness per-field (D33 rule: u32 bswap32 / u16 bswap16 / u8
|
||||
identity) and any fixup loop's pointer stride on x86-64**; for packed struct trees use the
|
||||
D37 re-layout pattern; for address-arithmetic idioms over exe symbols (D39/D42 class) guard
|
||||
a PC branch that reproduces the N64 32-bit value exactly. Rebuild; confirm boot advances.
|
||||
Log each as D3x in §F and note it in AGENTS.md phase status. **For asset FORMAT work,
|
||||
prefer the offline pattern (D48): pin the format with Python over the ROM ground truth,
|
||||
emit + validate offline, serve through the existing load path — runtime C conversion only
|
||||
as fallback.**
|
||||
Every ROM-serialized struct with a pointer field faults the same way once you
|
||||
reach more asset loading. The D32 fix procedure (doc §H): at the fault,
|
||||
`ptype /o <Struct>`; if a pointer field's offset/size diverges from its N64
|
||||
offset comment, change it to `u32` + cast at the use sites; verify the
|
||||
load-time rebase yields valid V1 addresses (< 0x80000000); **also check the
|
||||
ROM bytes' endianness per-field (D33 rule: u32 bswap32 / u16 bswap16 / u8
|
||||
identity) and any fixup loop's pointer stride on x86-64**; for packed struct
|
||||
trees use the D37 re-layout pattern; for address-arithmetic idioms over exe
|
||||
symbols (D39/D42 class) guard a PC branch that reproduces the N64 32-bit value
|
||||
exactly. Rebuild; confirm boot advances. Log each as D5x in §F and note it in
|
||||
AGENTS.md phase status. **For asset FORMAT work, prefer the offline pattern
|
||||
(D48): pin the format with Python over the ROM ground truth, emit + validate
|
||||
offline, serve through the existing load path — runtime C conversion only as
|
||||
fallback.** Also: any code that copies Gfx slots must use full-struct copies on
|
||||
PC (D50.6) — never the `long long` union member.
|
||||
|
||||
## Environment (do not rediscover these)
|
||||
- MSYS2/MinGW tools are in `/c/msys64/mingw64/bin/` — **not** on PATH. Prefix every shell:
|
||||
`export PATH="/c/msys64/mingw64/bin:$PATH"`.
|
||||
- Build: `./build-pc.sh ntsc-final` (incremental; full rebuild only when `port/shim/`
|
||||
changes; a *clean* full build is ~5 s on the 16-core dev box). ROM is at
|
||||
`data/ge007.ntsc-final.z64` (= `baserom.u.z64`, byte-identical).
|
||||
- The ROM is **mapped at cart base 0x10000000** on PC (`romdataInit`), so `&<segment>Symbol`
|
||||
values (e.g. `_sfxctlSegmentRomStart = 0x102EBDE0`) are directly dereferenceable host
|
||||
pointers into the ROM image; `romCopy()` copies from those cart addresses. All game RAM
|
||||
lives in the V1 view at 0x70xxxxxx (< 4 GiB) — s32 truncation of game-RAM pointers is
|
||||
lossless (this is why D39 needed no changes outside image_bank.c).
|
||||
- ROM provenance (verified 2026-08): byte-identical to the No-Intro good dump
|
||||
`GoldenEye 007 (U) [!].z64` (kept in `data/`). MD5
|
||||
`70c525880240c1e838b8b1be35666c3b`, SHA-256
|
||||
`2cdcec8a9f0cb6e36337f3ee39d8ad105dc8afa6ba1c02d466e8f5b771f9a162`, 0xC00000 bytes,
|
||||
country 'E' (US). Header CRCs DCBC50D1/09FD1AA3 re-verify with `tools_pc/romverify.exe`.
|
||||
- gdb: **launch** mode only (`gdb -batch -ex "handle SIGSEGV stop" -ex run …`). Symbolicate
|
||||
offline with `addr2line -e build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base
|
||||
is `0x140000000` (re-verify with `info address` after a rebuild). Debug scripts live in
|
||||
`build-pc/` (`d37_tree.py`, `d39_*.gdb`, `d41_probe.gdb`, …); the D43 investigation suite
|
||||
is in tracked `tools_pc/`.
|
||||
- **Capstone gotcha (D47.1):** the ROM is BIG-ENDIAN MIPS — always disassemble with
|
||||
`CS_MODE_MIPS32 | CS_MODE_BIG_ENDIAN`. Default (LE) mode produces garbage that looks
|
||||
plausible; an earlier session's disassembly was all LE and had to be redone.
|
||||
- Crash log: `ge007.crash.log` (repo root) now holds Phase 1 + a `BACKTRACE:`
|
||||
section (**D44** fixed). First stop for any fault; symbolicate with
|
||||
`addr2line -e build-pc/ge007.x86_64.exe -f -C <abs-addr>`. Frames past the
|
||||
true chain may be stale — sanity-check they fall inside `.text`.
|
||||
- `objdump -d --disassemble=<fn>` / `objdump -s -j .data` on build-pc/ge007.x86_64.exe is a
|
||||
fast way to check what the compiler actually emitted.
|
||||
- Many init functions use a fake RBP — compute stack offsets from the **entry RSP**, not RBP.
|
||||
- Python on this box: no f-strings with nested quotes (< 3.12) — use `%` formatting.
|
||||
- MSYS2/MinGW tools are in `/c/msys64/mingw64/bin/` — **not** on PATH. Prefix
|
||||
every shell: `export PATH="/c/msys64/mingw64/bin:$PATH"` (build-pc.sh needs
|
||||
it for cmake too).
|
||||
- Build: `./build-pc.sh ntsc-final` (incremental; full rebuild ~5 s on the
|
||||
16-core dev box). ROM at `data/ge007.ntsc-final.z64` (= `baserom.u.z64`,
|
||||
byte-identical to the No-Intro good dump, MD5
|
||||
`70c525880240c1e838b8b1be35666c3b`).
|
||||
- **Sidecars are gitignored (data/).** If `data/pcmodels-ntsc-final/` is
|
||||
missing on a fresh checkout, regenerate: `python tools_pc/d43_emit.py
|
||||
ntsc-final`. The game boots without them (warn + ROM-only mode) but model
|
||||
loads fail.
|
||||
- The ROM is **mapped at cart base 0x10000000** on PC; sidecar image follows
|
||||
at [CART_BASE+romSize, …). All game RAM lives in the V1 view at 0x70xxxxxx
|
||||
(< 4 GiB) — s32 truncation of game-RAM pointers is lossless.
|
||||
- gdb: **launch** mode only (`gdb -batch -ex "handle SIGSEGV stop" -ex run …`
|
||||
— attach fails with error 87). Symbolicate offline with `addr2line -e
|
||||
build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base
|
||||
0x140000000. **Post-mortem memory dumps of game RAM can fail** ("Cannot
|
||||
access memory") — dump at a live breakpoint instead (hit this with the main
|
||||
DL in D51).
|
||||
- Many init functions use a fake RBP — compute stack offsets from the **entry
|
||||
RSP**, not RBP.
|
||||
- Python on this box: no f-strings with nested quotes (< 3.12) — use `%`
|
||||
formatting.
|
||||
- Untracked scratch at repo root (all.txt, b3.c, btest*.c, buildtest.txt,
|
||||
err.txt, gcout.txt, preproc.txt, vsize.*) + `tools_pc/__pycache__/` are
|
||||
session scratch — do NOT commit them; use explicit pathspecs with git add.
|
||||
|
||||
## Definition of done
|
||||
- ~~**Session A:** D49 committed~~ — **DONE**: 8/8 CONFIRMED with evidence,
|
||||
HANDOFF updated, committed.
|
||||
- **Session B (the milestone proper):** stage objects load without faulting and the
|
||||
rendered scene contains actual model geometry per the pixel assert (not just the clear
|
||||
color / background), stable under a 30 s+ soak. Committed + pushed to `origin/master`
|
||||
(`PC port: phase 2 — <what>`). After that, Phase 2 makes it look right (fast3d CC/RM
|
||||
correctness vs `gmain.s` — GE-specific, no PD analogue) and Phase 3 adds sound + input,
|
||||
largely copy-and-adapt from the PD port.
|
||||
- **Session C:** D51 diagnosed + fixed (finding in §F, crash moves past frame
|
||||
5), pixel assert built, 30 s+ soak clean with real geometry on screen.
|
||||
Committed + pushed (`PC port: phase 2 — <what>`). After that: Phase 2 makes
|
||||
it look right (fast3d CC/RM correctness vs `rsp/graphics/gmain.s` —
|
||||
GE-specific ground truth, no PD analogue), next asset type (Tbg_*_stanZ
|
||||
backgrounds), and Phase 3 adds sound + input (copy-and-adapt from the PD
|
||||
port).
|
||||
+147
-110
@@ -1664,6 +1664,115 @@ missing rule — but Task 2's emit pass must bswap16 every u16/s16 record
|
||||
scalar (padding/reserved may be zeroed), and the HANDOFF Verified facts
|
||||
should be amended to say so.
|
||||
|
||||
**D50 (RESOLVED — Plan B executed: offline sidecars + port plumbing; boot advances to first model GDL execution)**
|
||||
|
||||
Execution session ran HANDOFF Task 2 end-to-end. All 512 NTSC model files
|
||||
converted offline and served through the existing load path; frames render
|
||||
and model display lists execute. Sub-items (all `#ifdef PORT`-guarded or
|
||||
port-layer only; N64 build untouched):
|
||||
|
||||
1. **D50.1 Emit pass (`tools_pc/d43_emit.py`, tracked).** Per file:
|
||||
decompress the N64 image from the ROM (filelist row); build the node map;
|
||||
run the EXACT `modelIterateDisplayLists` visit simulation (LOD/SWITCH
|
||||
rewire, BSP splice — d43_gdlorder logic) → gdl_seq; layout `[switches
|
||||
NS×8][texconfigs NT×12][DFS nodes 48B + records PC_REC + vertex arrays]
|
||||
…[GDLs packed contiguously in gdl_seq order, 16B per N64 slot]`; byte-exact
|
||||
emit (bswap32/16; promoted pointers → zero-extended u64 `0x05xxxxxx` VMAs;
|
||||
GDL Primary/Secondary raw VMAs, NOT promoted; BaseAddr=0); **round-trip
|
||||
re-parse validation of every field against the N64 source + region tiling
|
||||
(no gaps)** — 512/512 pass. Compression: `0x11 0x72` + raw deflate level 6
|
||||
(`zlib.compressobj(6, DEFLATED, -15)`); escalate to 9 only on a dst!=0 fit
|
||||
violation (none occurred). Cross-checks all pass: per-file dst!=0 buffers
|
||||
(`round8(C)+8 ≤ buf` AND `D+round8(C) ≤ buf` — inflate overlap safety), G*
|
||||
hand-weapon worst case vs 0xF000, cast/title/bondview chain cumulative
|
||||
P_final bounds, totals (Σ round8(C) = 1,277,088 B, matches D49). Output:
|
||||
**single concatenated image** `data/pcmodels-<region>/pcmodels.bin`
|
||||
(sidecars at 16-aligned offsets) + `manifest.csv` (`name,offset,size`,
|
||||
decimal — the C parser uses strtol base 10; file_resource_table.inc.c
|
||||
order). Deviation from D48's per-file-sidecar sketch: one blob + manifest;
|
||||
pcmodels.c copies the whole image to `[CART_BASE+romSize, …)` and patches
|
||||
`hw_address = cartBase+romSize+off`. Regenerate:
|
||||
`python tools_pc/d43_emit.py [ntsc-final|pal-final|jpn-final|--check-only]`
|
||||
(needs the region ROM in data/; sidecars are gitignored with data/).
|
||||
2. **D50.2 Port plumbing (`port/src/pcmodels.c` + `port/include/pcmodels.h`,
|
||||
new; romdata.c/h extended).** `pcmodelsReserveSize(romImg)` derives
|
||||
`data/pcmodels-<region>/` from the ROM country byte (+0x3E), parses the
|
||||
manifest, returns total bytes (0 → warn, ROM-only mode); romdataInit
|
||||
reserves `romSize + sidecarTotal` at CART_BASE and
|
||||
`romdataCartAddrValid` accepts the extension; `pcmodelsLoadSidecars(
|
||||
cartBase, romSize)` copies the blob; `pcmodelsPatchTable()` — one-shot,
|
||||
called from `load_object_fill_header` (hook block, objecthandler_2.c) after
|
||||
obInit has run — redirects every manifest row's
|
||||
`file_resource_table[i].hw_address` + sets
|
||||
`resource_lookup_data_array[i].rom_size` to the PC compressed size. The same
|
||||
hook block resets `poolRemaining = 0` for dst==0 loads (closes the D48.3/R6
|
||||
reload hazard). Boot log: `[INFO] pcmodels: table patched (512 model
|
||||
entries)`.
|
||||
3. **D50.3 Language banks (runtime C fixup).** Banks carry a big-endian
|
||||
offset table; `romdataFixupLangBank(blob, decompressedSize)` decodes it in
|
||||
place. Called from language.c after each of the 7 langInit loads and lazily
|
||||
per-id in the `langGetJpnCharPixels` paths (idempotent via poolRemaining
|
||||
check).
|
||||
4. **D50.4 Fonts (runtime C fixup).** `load_font_tables` PC branch:
|
||||
allocate `romdataFontPcSize()` (PC C layout of struct font: kerning[169] +
|
||||
chars[94] + glyph pixel data), romCopy the N64 size, then
|
||||
`romdataFixupFont` re-lays out, shifting the pixel block below the expanded
|
||||
char array; `pixeldata` fields left as relative offsets — the existing
|
||||
`pixeldata += base` loop promotes them.
|
||||
5. **D50.5 Legal-screen UB exposure (front.c).**
|
||||
`constructor_menu00_legalscreen` reads an uninitialized pointer at a lookat
|
||||
call — on N64 the register happens to hold a readable address and the
|
||||
result is zeroed by `* 0.0f` anyway; the x86-64 compiler folds the UB read
|
||||
to NULL → fault. PORT branch seeds it with `legalpage_text_array` (the
|
||||
value assigned a few lines later); identical lookat.
|
||||
6. **D50.6 `texCopyGdls` copies only w0 on PC — first model-render crash
|
||||
(RESOLVED).** `Gfx` is a union whose last member is `long long int
|
||||
force_structure_alignment`: 8 bytes on N64 (= the whole slot), but on x86-64
|
||||
the slot is 16 bytes, so `arg1->force_structure_alignment = arg0->…`
|
||||
copies only the low half. Compaction flow (sub_GAME_7F0762E0):
|
||||
texCopyGdls mirrors the GDL block `[G,D)` to tail scratch `[B−D+G,B)`, then
|
||||
texLoadFromGdl reads the scratch and writes expanded output back at
|
||||
`[G,…)` via full-slot `*(out++)=*(in++)` — propagating the scratch's stale
|
||||
w1s into the final GDLs. **Byte proof:** PlegalpageZ (NS=0, NT=5,
|
||||
D=0x2638) loaded at 0x7012EA38; first 64 RAM bytes match the decompressed
|
||||
sidecar exactly; the executed GDL at file offset 0x2488 has w0s matching
|
||||
the sidecar exactly (0xB10000BA/0xE7000000/0xFD900000/0xE6000000) but RAM
|
||||
w1s = 0x50362B58/0x66D73339/0xACAAB819/0x55BDF769 (stale mempool contents;
|
||||
pad2 dwords also nonzero) vs sidecar w1s 0x0000A898/0/0x050012C8/0 — the
|
||||
garbage G_SETTIMG w1 → `seg_addr` → OOB read in `import_texture_rgba16`.
|
||||
Fix: `*arg1 = *arg0;` under `#ifdef PORT` (tex.c). Audit: tex.c is the only
|
||||
`force_structure_alignment` use in game code (model.c:1510 is an unrelated
|
||||
local 8-byte union); every other Gfx copy is full-struct. Post-fix:
|
||||
PlegalpageZ's sub-DLs execute to completion; crash moves on (D51).
|
||||
|
||||
**D51 (OPEN — current blocker: `import_texture_i8` SIGSEGV via main-DL G_TEXRECT lazy tile upload)**
|
||||
|
||||
Post-D50 run: frames 1–4 log, then SIGSEGV on frame 5 in
|
||||
`import_texture_i8` (gfx_pc.cpp:782, `addr[i]`) ← `import_texture(i=0,
|
||||
tile=0)` ← `gfx_sp_tri1(128,129,131,is_rect=true)` ←
|
||||
`gfx_draw_rectangle(236,92,268,140)` ← `gfx_dp_texture_rectangle` ←
|
||||
`G_TEXRECT` case (gfx_run_dl ~line 2472), main DL start 0x70052e80,
|
||||
failing opcode slot at +0x260 (0x700530e0). G_TEXRECT carries no source
|
||||
address — it blits from RDP texture-memory tiles; the fault is the **lazy
|
||||
upload of tile 0** whose stored source pointer is invalid (or the tile was
|
||||
never set up this frame → stale `rdp.textures[0]`). Suspects: (a) a
|
||||
main-DL G_SETTIMG (0xFD) earlier than +0x260 whose w1 seg_addr
|
||||
mis-resolves; (b) tile referenced before setup / across frames; (c) the
|
||||
game points the tile source at a ROM/ramrom image that needs a port fixup
|
||||
(D50.3/D50.4 class). **Investigation not started.** Note: after the crash,
|
||||
0x70052e80 is unreadable from gdb ("Cannot access memory") — dump live at a
|
||||
breakpoint, not post-mortem. GE opcode facts verified this session
|
||||
(include/PR/gbi.h, non-F3DEX numbering): G_IMMFIRST=−65; DMA G_MTX=1 /
|
||||
G_MOVEMEM=3 / G_VTX=4 / G_DL=6; IMM TRI1=0xBF … ENDDL=0xB8;
|
||||
GE extension G_TRI4=0xB1 (gbi_extension.h, 8-bit packed indices — NOT an
|
||||
address carrier); RDP pass-throughs include G_SETTIMG=0xFD,
|
||||
G_SETCIMG=0xFF, **G_TEXRECT=0xE4 / G_TEXRECTFLIP=0xE5** (GE-specific
|
||||
values, not libultra's 0x46/0x45). The main DL is game-built per frame with
|
||||
gSP* macros (16B slots); the segment table is set via
|
||||
`gMoveWd(G_MW_SEGMENT, seg*4, base)` → w0=(0xBC<<24)|(seg*4<<8)|6,
|
||||
w1=data; fast3d's gfx_sp_moveword stores data as-is when ≥ 0x800000 else
|
||||
+0x80000000.
|
||||
|
||||
Environment notes for Task 2: standalone gcc needs
|
||||
`/c/msys64/mingw64/bin:/c/msys64/usr/bin` on PATH (cc1 fails SILENTLY without
|
||||
it — exit 1, zero diagnostics), and `-std=c11` is required (the CMake flag;
|
||||
@@ -1673,118 +1782,47 @@ game TUs.
|
||||
|
||||
### G. Phase 2 status (current)
|
||||
|
||||
Done: PD fast3d integrated (`port/fast3d/`, replaces the deleted gfxstub.c);
|
||||
GE's real `src/sched.c` compiled (shedThread runs `__scMain`); real pthread
|
||||
kernel (D24); dual-mapped DRAM + address shims (D25/D26); ASLR off with startup
|
||||
sanity check (D27); PI completion messages (D29); SEH-free crash handler with
|
||||
thread dumps (D30). The process now boots, maps the ROM, opens the window,
|
||||
runs `mainproc()` → `bossInitMainthreadData` through controller init / timers /
|
||||
mempool / VI / rsp / stan / game init. The D31 langInit file-load SIGSEGV is
|
||||
**fixed** (real-zlib decompression, §F/D31): mainThread now loads all language
|
||||
files and proceeds into `bossInitMainthreadData()` → `alloc_load_expand_ani_table`
|
||||
→ `init_weapon_animation_groups_maybe` (boss.c:233), where it SIGSEGVs on the
|
||||
**D32** ROM-struct pointer-width issue. D32 is decomposed: **Part A** (struct
|
||||
layout — u32 pointer fields) is DONE & proven; **Part B** (the load-time pointer
|
||||
rebase in `expand_ani_table_entries`) is OPEN and is the current blocker. No
|
||||
frames rendered yet. See §H for the handoff + plan.
|
||||
|
||||
Reference — the file-load chain (for D31 debugging):
|
||||
`langInit` (language.c:230) loads 7 text banks via
|
||||
`_fileNameLoadToBank(LnameX_lookuptable[bank][j_text_trigger], …, 0x100,
|
||||
MEMPOOL_PERMANENT)` → `fileGetIndex` (strcmp over file_resource_table) →
|
||||
`fileIndexLoadToBank(index, …)` (ob.c:216): if poolRemaining==0 take
|
||||
`mempGetBankSizeLeft(bank)`; `ptrdata = mempAllocBytesInBank(…)`; hw_address
|
||||
nonzero → `load_resource(ptrdata, poolRemaining, &file_resource_table[index],
|
||||
&resource_lookup_data_array[index])` (ob.c:37): `source = (ptrdata + bytes) −
|
||||
((rom_size+7)&−8)`; `romCopy(source, hw_address, rom_size)`;
|
||||
`decompressdata(source, ptrdata, buffer)` with `u8 buffer[0x2100]` on the
|
||||
stack → sets rz_inbuf=source+2 / rz_outbuf=ptrdata / rz_hlist=buffer →
|
||||
`zlib_inflate()` (src/game/zlib.c:668). rom_size for index 670 = 0x720,
|
||||
decompressed size = 3872 (0xF20). `file_resource_table` is compiled
|
||||
(`assets/obseg/file_resource_table.inc.c`) with hw_address = cart addresses;
|
||||
rom_size = adjacent-marker differences computed in obInit.
|
||||
|
||||
Debugging techniques that worked: gdb **launch** mode (`gdb -batch -ex
|
||||
"handle SIGSEGV stop" -ex run …` — attach fails with error 87); breakpoint
|
||||
chains through init functions to bracket the crash; `finish` to let a suspect
|
||||
function complete and inspect its return; hardware watchpoints on exact stack
|
||||
slots (compute offsets from entry RSP, not RBP — see D31); `addr2line` for
|
||||
offline symbolication; objdump + `gcc -E` with exact ninja flags to verify
|
||||
shim expansion in compiled code.
|
||||
Done through **D50**: PD fast3d integrated (`port/fast3d/`); GE's real
|
||||
`src/sched.c` + pthread kernel; dual-mapped DRAM; ROM mapped at cart base;
|
||||
SDL2 window; full boot chain (D31–D42); **Plan B executed (D50)** — all 512
|
||||
NTSC model files offline-converted to PC-layout RZ sidecars
|
||||
(`tools_pc/d43_emit.py` → `data/pcmodels-ntsc-final/pcmodels.bin` +
|
||||
manifest.csv), served through the existing load path via a port-layer table
|
||||
patch (`port/src/pcmodels.c`, romdata cart-extension, one-shot
|
||||
`pcmodelsPatchTable()` + poolRemaining reset hook in
|
||||
`load_object_fill_header`); runtime C fixups for language-bank BE offset
|
||||
tables (D50.3) and font re-layout (D50.4); legal-screen UB seed (D50.5);
|
||||
`texCopyGdls` w1 partial-copy bug fixed and byte-proven (D50.6). Frames 1–4
|
||||
render; PlegalpageZ model GDLs execute to completion. **Current blocker:
|
||||
D51** — SIGSEGV in `import_texture_i8` via a main-DL G_TEXRECT lazy tile-0
|
||||
upload (frame 5); investigation not started, recipe in docs/HANDOFF.md.
|
||||
After D51: pixel-assert soak (env-gated PPM dump + pixcount still unbuilt),
|
||||
then the next asset type (Tbg_*_stanZ backgrounds) with the same offline
|
||||
pattern.
|
||||
|
||||
### H. Handoff & plan (current session)
|
||||
|
||||
**State.** Boot is complete AND the first frames render: mainThread runs through
|
||||
all of `bossInitMainthreadData()` + `bossEntry()`, enters `bossMainloop()`,
|
||||
loads the stage, and fast3d renders real display lists — `[NOTE] frame N rendered
|
||||
in … us` logs appear (frame 1 in ~70 ms). Findings **D34–D42** (§F) closed out
|
||||
the boot + first-render blockers on top of D31–D33:
|
||||
- **D34** `ANIM_DATA_*` → address-only lvalues at PE-image-base + ROM offset.
|
||||
- **D35** music seq table: `RareALSeqData.address` → `u32` + BE fixup.
|
||||
- **D36** music heap / PERMANENT pool grown for x86-64 libaudio struct bloat.
|
||||
- **D37** libaudio bank trees re-laid out to PC-native form (`romdataFixupAudioBank`).
|
||||
- **D38** implicit-declaration pointer truncation → `port/include/pc_protos.h`
|
||||
prototype shim (398 decls, generator in build-pc/); build now warning-clean.
|
||||
- **D39** Globalimagetable rebasing: per-symbol offsets + `GIMG_OFF()` macro
|
||||
(== N64 `(u32)&sym`, incl. the 0x02000000 base) in image_bank.c; segment end
|
||||
markers extended to the true 0x13F8 size (CSV truncation).
|
||||
- **D40** ModelHitEntry BSS pool sized for the 40-byte PC stride (was 12 KB
|
||||
overflow clobbering `is_ramrom_flag`).
|
||||
- **D41** GL context released from the host main thread after init so the game
|
||||
thread can bind it (WGL single-thread currency; SDL2 MakeCurrent return value
|
||||
is an inverted ABI artifact — don't branch on it).
|
||||
- **D42** rsp.c `g_gfxTaskSettingsList` XOR toggle → explicit toggle on x86-64.
|
||||
Full paste-ready brief: **docs/HANDOFF.md** (rewritten this session). Summary:
|
||||
|
||||
**ROM provenance locked:** the working ROM is byte-identical to the No-Intro
|
||||
good dump `GoldenEye 007 (U) [!].z64` (MD5 `70c525880240c1e838b8b1be35666c3b`,
|
||||
0xC00000 bytes, country 'E'); embedded header CRCs DCBC50D1/09FD1AA3
|
||||
re-computed and matched via `tools/n64cksum.c`'s CIC-6102 routine — see
|
||||
`tools_pc/romverify.c` (one-shot re-check) and docs/HANDOFF.md Environment.
|
||||
**State.** Plan B is EXECUTED (D50): offline sidecars for all 512 NTSC model
|
||||
files + port-layer plumbing + lang/font runtime fixups + legal-screen UB seed
|
||||
+ the texCopyGdls w1 fix (D50.6, byte-proven against PlegalpageZ). Boot runs
|
||||
to the main loop; frames 1–4 render; model GDLs execute (PlegalpageZ
|
||||
sub-DLs complete).
|
||||
|
||||
**Current blocker — D43: model-file loading ABI mismatch.** After a few frames,
|
||||
stage object load faults in `modelPromoteNodeOffsetsToPointers()`
|
||||
(model.c:5688). Model files are ROM-serialized with N64 layout (ModelFileHeader
|
||||
24 B / ModelNode 20 B, 4-byte pointer fields) but read on PC as 8-byte-pointer
|
||||
structs; `load_object_fill_header()` (objecthandler_2.c) derives RootNode from
|
||||
`numtextures` at the wrong offset before any rebase. Plan: a D37-style two-pass
|
||||
re-layout of the model file image in `port/src/romdata.c` — header, switches
|
||||
array, texture table, ModelNode tree (walked via Child/Next/Parent), and the
|
||||
per-opcode ModelRoData records — with every embedded pointer rewritten as a
|
||||
zero-extended offset from the image start so the existing
|
||||
`PROMOTE`/`diff = fileramaddr - vma` rebase works unmodified. Map the record
|
||||
types (bondtypes.h `ModelRoData` union, opcodes in bondconstants.h) — but do
|
||||
NOT design from scratch: PD port's `port/src/preprocess/filemodel.c` (§2.4) is
|
||||
a Rare-validated near-analogue (same PROMOTE idiom, same vma 0x5000000); adapt
|
||||
its node walk / placement / record handling and validate every GE record type
|
||||
per-field against bondtypes.h + ROM ground truth. Expect more D32-class ABI
|
||||
fixes as other per-level asset types are first touched — standing sub-task,
|
||||
not a one-off; PD's `preprocess/` module already solves most of them
|
||||
(`filebg.c`, `filetiles.c`, …).
|
||||
**Current blocker — D51.** SIGSEGV on frame 5 in `import_texture_i8` via a
|
||||
main-DL G_TEXRECT lazy tile-0 upload (gfx_pc.cpp:782; failing opcode slot at
|
||||
DL offset +0x260). G_TEXRECT blits from RDP texture-memory tiles, so the fault
|
||||
is an invalid/stale tile source — either a mis-resolved main-DL G_SETTIMG,
|
||||
a tile used before setup, or a ROM/ramrom image needing a port fixup.
|
||||
Investigation not started; the gdb recipe (break live at
|
||||
`import_texture_i8`, dump `rdp.textures[0]` + the full main DL — post-mortem
|
||||
dumps of 0x70052e80 fail with "Cannot access memory") is in HANDOFF Task 1.
|
||||
|
||||
**Dev-loop measurements + speedups (session of 2026-08-22).** Measured
|
||||
iteration costs on the dev box (16 cores): full clean rebuild ≈ 5 s wall
|
||||
(239 ninja steps) — the build is NOT a bottleneck, ccache would be wasted
|
||||
effort; launch → D43 crash ≈ 3.7 s — runtime wait is negligible. The cost is
|
||||
in *diagnosis*: gdb launch-only mode + manual `addr2line` image-base arithmetic
|
||||
per fault. Speedups, in priority order: (1) the D44 one-line fix → automatic
|
||||
backtrace on every crash; (2) a `tools_pc/sym.sh` wrapper (PC or rel offset →
|
||||
add image base 0x140000000 → `addr2line -f -C`); (3) a Python ROM/asset
|
||||
inspector (`tools_pc/romdump.py`) that parses ModelFileHeader/ModelNode trees
|
||||
straight from the .z64 using the bondtypes.h layout (opcode sequences, node
|
||||
links, Data records) — accelerates D43 and every follow-on asset type, and can
|
||||
validate re-layout output against ROM ground truth without running the game;
|
||||
(4) env-gated framebuffer PPM dumps every N frames + a non-clear-color pixel
|
||||
count/diff script — turns "the scene contains actual model geometry" into an
|
||||
assert, makes the 30 s soak scriptable, and enables screenshot diffs against
|
||||
an emulator (Mupen64Plus with the verified No-Intro dump). NOT worth it
|
||||
(measured): ccache, boot fast-forward/skip-to-mainloop hooks.
|
||||
|
||||
**Audio (Phase 3 prep).** Bank trees are correctly laid out at boot and the
|
||||
libaudio manager + audio thread run without crashing; actual sound output still
|
||||
needs an SDL backend for `alDriver`, and the ASP strategy question (C2) remains
|
||||
open. No work required until rendering is stable. The SDL backend is largely
|
||||
copy-and-adapt from PD: `port/src/mixer.c` (722 lines vs our 31-line stub,
|
||||
which was scaffolded for it) plus `input.c`/`fs.c` for Phase 3/4 — see §2.4.
|
||||
**Next after D51:** build the pixel assert (env-gated PPM dump every N frames
|
||||
in port/video.c + `tools_pc/pixcount.py`) and run a 30 s+ soak; then move to
|
||||
the next asset type (Tbg_*_stanZ backgrounds) with the same offline pattern —
|
||||
PD's `port/src/preprocess/filebg.c` is the per-field reference (§2.4).
|
||||
|
||||
**D32 repeatable fix procedure** (apply to any ROM-serialized struct that faults
|
||||
on a pointer-field read):
|
||||
@@ -1821,7 +1859,6 @@ prefix `export PATH=…`). Build: `./build-pc.sh ntsc-final`. gdb **launch** mod
|
||||
only (attach fails, error 87); symbolicate offline with `addr2line -e
|
||||
build-pc/ge007.x86_64.exe -f -C <0x140000000+rel>`. Image base 0x140000000.
|
||||
`load_resource`/many init fns use a fake RBP — compute stack offsets from entry
|
||||
RSP. The D30 crash handler writes Phase 1 of `ge007.crash.log` (raw PC/
|
||||
registers) but its Phase 2 backtrace self-faults before printing (**D44**,
|
||||
one-line fix pending) — until fixed, attribute crashes via gdb; after the fix
|
||||
the crash log is the first stop.
|
||||
RSP. The D30 crash handler writes `ge007.crash.log` with a working Phase-2
|
||||
backtrace (**D44** fixed) — first stop for any fault; frames past the true chain
|
||||
may be stale, sanity-check they fall inside `.text`.
|
||||
Reference in new issue
Block a user