diff --git a/AGENTS.md b/AGENTS.md index 367c5936..0d85509c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,6 @@ Run `/linkcheck` for this sweep. Record new findings in `docs/PCPortResearch.md` - **Compile+link milestone:** all ~235 TUs compile and `ge007.x86_64.exe` links clean (`ninja ge007 -k 0`, 236/236). Findings D7–D17 in `docs/PCPortResearch.md` §11; asset stubs in `port/src/assetstubs.c`, fast3d no-ops in `port/src/gfxstub.c` (delete when real fast3d lands). - **Phase 1 (boot to window):** done — ROM loads/validates/maps at cart base 0x10000000 (`romdata.c`), SDL2 window + GL clear loop (`video.c`), absolute asset symbols from `scripts/gen_romassets.py` → `port/src/romassets_.s` (findings D18–D23). - **Phase 1.5 (boot to first frame):** DONE — boot chain complete AND real frames render (`[NOTE] frame N rendered in … us`). D34–D42 closed the blockers on top of D31–D33: ANIM_DATA lvalues, music seq-table ABI+endianness, music-heap sizing, libaudio bank-tree re-layout, implicit-declaration prototype shim (D38), Globalimagetable rebasing + true segment size (D39), ModelHitEntry pool sizing (D40), cross-thread GL context release (D41), rsp task-settings toggle (D42). ROM dump verified against the No-Intro good dump (byte-identical, header CRCs re-checked — `tools_pc/romverify.c`). -- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + first frames render; current blocker is **D43** (model-file loading ABI mismatch: N64-layout ROM model files read as 8-byte-pointer PC structs → D37-style re-layout). **D43 status:** all 512 model files validated; buffer sizing finalized by the D46 overlap-safety analysis; step-2 mechanical game-code edits APPLIED + committed (ecefffdb); **D47 closed every open converter question** — compaction state values, G_VTX encoding, opcode-aware remap rule, vertex formats — spec implemented + validated 512/512 by `tools_pc/d43_convert.py` (max D_PC = 0xB7E0, ratio ≤ 1.31). **D48 review re-plan:** the old "port d43_convert.py 1:1 to C" task is discarded — that script does not emit bytes, and **Plan B (offline pre-conversion)** is now the default: Python emit pass → `data/pcmodels-/` RZ sidecars served via a port-layer `hw_address`/`rom_size` table patch (no C converter; full mechanics in §F/D48). Next session independently reviews Plan B (HANDOFF Task 1 → finding D49), then executes (Task 2); fallback Plan A = D47.13 C converter with the corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`). All 23 d43_*.py scripts moved from gitignored build-pc/ to tracked `tools_pc/`. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48). +- **Phase 2 (rendering):** IN PROGRESS — fast3d integrated + first frames render; current blocker is **D43** (model-file loading ABI mismatch: N64-layout ROM model files read as 8-byte-pointer PC structs → D37-style re-layout). **D43 status:** all 512 model files validated; buffer sizing finalized by the D46 overlap-safety analysis; step-2 mechanical game-code edits APPLIED + committed (ecefffdb); **D47 closed every open converter question** — compaction state values, G_VTX encoding, opcode-aware remap rule, vertex formats — spec implemented + validated 512/512 by `tools_pc/d43_convert.py` (max D_PC = 0xB7E0, ratio ≤ 1.31). **D48 review re-plan:** the old "port d43_convert.py 1:1 to C" task is discarded — that script does not emit bytes, and **Plan B (offline pre-conversion)** is now the default: Python emit pass → `data/pcmodels-/` RZ sidecars served via a port-layer `hw_address`/`rom_size` table patch (no C converter; full mechanics in §F/D48). **D49 review PASSED** — all 8 D48 checklist items CONFIRMED against code + ROM (§F/D49): RZ format, memcpy DMA path, table chokepoint, obInit timing, buffer fit (worst-case P_final 0x16F74 vs 0x24C400 bank), reload hazard + pending one-line reset, per-region sidecars (EU=465 files; no JP ROM candidate yet), emission spec complete (fresh C probe: no record-size drift; added the bswap16-record-scalars rule). Next session EXECUTES Plan B (HANDOFF Task 2); fallback Plan A = D47.13 C converter with the corrected staging guard (D48.4: `D_N64 + D_PC ≤ avail`). All 23 d43_*.py scripts moved from gitignored build-pc/ to tracked `tools_pc/`. Recurring sub-task: ABI/layout fixes per the D32 procedure (§H) as more asset types are first touched (PD's `preprocess/` module already solves most of them; prefer the offline pattern for format work — D48). - **Phase 3 (audio + input):** libaudio → SDL — largely copy-and-adapt from PD (`mixer.c` 722 lines vs our 31-line stub; `input.c` SDL_GameController backend; §2.4); decide the ASP strategy (C2: `aspMain*` dummies in `ucode.c`). - **Phase 4 (saves + polish):** file-backed EEPROM — build on PD's `fs.c` (file-backed save dir); PFS/motor already no-op stubs. diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 83f364a6..ac213102 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -1,30 +1,27 @@ -# Handoff brief — GoldenEye 007 PC port (Phase 2: first frames render; D43 re-planned as offline pre-conversion) +# Handoff brief — GoldenEye 007 PC port (Phase 2: Plan B review PASSED (D49) — next session executes the offline pre-conversion) _Paste-ready briefing. Authoritative detail lives in `AGENTS.md` and `docs/PCPortResearch.md` (§F/D31–D48 findings, §H handoff); this is the summary + the immediate tasks._ -## Your job — TWO sessions, in order +## Your job -- **Session A (next): INDEPENDENT REVIEW of Plan B (finding D48).** Do NOT - implement anything. Verify the 8 claims in §Task 1 against code + ROM, each - with its falsification criterion. Record results as finding **D49** in - `docs/PCPortResearch.md` §F (per-item CONFIRMED/REFUTED + evidence), update - this file's status, commit. If every item is confirmed, Session B is green-lit. - If any is refuted: record why, then either adjust Plan B per the finding or - fall back to Plan A (§Task 3). The goal does not change: get stage geometry - on screen. -- **Session B (after D49 confirms): EXECUTE Plan B** per §Task 2 — Python emit - pass + sidecar generation, ~60 lines of port-layer plumbing, one-line hook, +- ~~Session A: independent review of Plan B (D48)~~ — **DONE (D49, committed):** + all 8 checklist items CONFIRMED against code + ROM; Plan B green-lit. One + spec amendment recorded there: bswap16 every u16/s16 record scalar on emit + (see Verified facts). +- **Session B (next): EXECUTE Plan B** per §Task 2 — Python emit pass + + sidecar generation, ~60 lines of port-layer plumbing, one-line hook, build + pixel-assert soak. Commit at each working checkpoint - (`PC port: phase 2 — `). + (`PC port: phase 2 — `). The goal: get stage geometry on screen. Work agentically — fix → build (~5 s) → verify → commit at each milestone. ## Read first (authoritative, in order) 1. `AGENTS.md` — non-negotiables, critical files, build, verification ritual. -2. `docs/PCPortResearch.md` **§F/D48** (the plan under review — read fully), - then **D47** (converter contract — still ground truth for the emit pass), +2. `docs/PCPortResearch.md` **§F/D49** (review verdict + quantified fit + numbers), **D48** (the plan being executed — read fully), then **D47** + (converter contract — still ground truth for the emit pass), **D46** (overlap safety + buffer sizing), **D45** (buffer constants), **D43** (the model-file task), §2.4 (PD-port copy-candidate audit). 3. The code paths D48 cites: `port/src/rzdecomp.c`, `port/src/libultra.c` @@ -44,6 +41,14 @@ Work agentically — fix → build (~5 s) → verify → commit at each mileston - Region macros in `CMakeLists.txt` must mirror the N64 Makefile exactly. ## Current state +- **D49 (this handoff's predecessor, committed): Plan B review PASSED** — all 8 + D48 checklist items CONFIRMED with evidence (§F/D49). Quantified: Σ round8(C) + = 1,277,088 B; worst-case post-compaction P_final = 0x16F74 vs 0x24C400 stage + bank (25× headroom); all dst!=0 buffers pass against current constants; + fresh C probe re-verified every PC record size + promotion offset (no drift). + Notes: romdataInit has no JP ROM candidate (JP sidecars moot for now); + EU region has 465 model files vs 512 (generator maps what exists); the + poolRemaining=0 reset is still pending (Task 2 step 3). - **D43 status (latest review session, §F/D48):** investigation complete; converter spec finalized (D47) and validated 512/512 (`tools_pc/d43_convert.py`, max D_PC = 0xB7E0, ratio ≤ 1.31). **The remaining task was re-planned:** the @@ -71,7 +76,7 @@ Work agentically — fix → build (~5 s) → verify → commit at each mileston to the No-Intro good dump (`tools_pc/romverify.exe`). Crash log with real backtraces works (D44). Build clean 237/237; full rebuild ~5 s. -## Task 1 — Plan B review checklist (Session A) +## Task 1 — Plan B review checklist (Session A) — DONE (D49: 8/8 CONFIRMED) Verify each claim. For each: state CONFIRMED or REFUTED in D49 with the evidence you used (file:line, command output). If a falsification criterion @@ -250,6 +255,7 @@ diff the C output against (convert one file in C, compare with Python output). - **Opcodes present: 1,2,4,8,9,10,12,13,15,18,21,22,23,24** (HEADER, GROUP, DL, LOD, BSP, BOX, GUNFIRE, SHADOW, INTERLINK, SWITCH, GROUPSIMPLE, DLPRIMARY, HEADPH, DLCOLLISION). op22 N64: numVertices s32@0, Vertices@4, Primary@8, BaseAddr@0xC (size 0x10); only 5 files have it. - Record sizes N64/PC: OP01 0x10/0x18, OP02 0x1C/0x28, OP04 0x14/0x28, OP08 0x10/0x18, OP09 0x24/0x30, OP10 0x1C/0x1C, OP12 0x28/0x30, OP13 0x20/0x30, OP15 0x1C/0x1C, OP17 0x20/0x28, OP18 0x08/0x10, OP21 0x14/0x14, OP22 0x10/0x20, OP23 0x02/0x02, OP24 0x20/0x40 (compiler-verified). Field layouts in `src/bondtypes.h` (ModelRoData_*Record); read N64 fields from packed offsets, emit via the real PC layout — don't hand-compute PC offsets. - **f32 fields are big-endian in ROM** — bswap all f32 (record origins, BoundingVolumeRadius, Scale, Min/MaxDistance) and every s16 in vertex arrays; also bswap32 the TextureID u32 in the texture table (texLoad reads `*updateword & 0xffff` as the texture number). +- **u16/s16 record scalars are big-endian too (D49 amendment)** — bswap16 every u16/s16 scalar field in every record (AnimPart, MatrixIndex, JointID, MatrixIDs, Group1/2, RwDataIndex, op4 numVertices@0x10, op24 nv/ncv/ModelType/RwDataIndex, …); padding/reserved fields may be zeroed. Data-verified: these hold small BE values (JointID 1–11, nv ≤ 73) that raw emission would corrupt ×256 on an LE read. - **Vertex arrays must stay 16B-stride** (GDL gSPVertex hardcodes 16B/vertex; propobj.c copies with sizeof(Vertex); model.c:4360 allocates numVertices×4 words). bswap every s16 field BE→LE. Main Vertices array is pure data; **CollisionVertices carry LinkedTo@8 = vma of a ModelNode** — remap to the new node offset (PROMOTE promotes them after rebase). - **GDLs:** 8B BE word pairs → 16B LE slots: `w0=bswap32(rom_w0); w1=bswap32(rom_w1)` — **leave w1 raw, NO LSB set** (ROM convention; fast3d's extended seg_addr case handles unmarked segmented addresses — D47.13). GE uses STANDARD libultra GBI numbering (include/PR/gbi.h); opcodes live in the TOP byte of w0. **Opcode-aware remap: only {G_VTX=0x04, G_SETTIMG=0xFD, G_LOADBLOCK=0xF3} carry addresses in w1** (seg-5 → remap low24 via region map). All other opcodes get bswap-only — G_TRI4 (GE extension 0xB1) w1 is 4-bit vertex-index data, NOT an address (D47.6); same for TRI1/TEXTURE/SET*MODE/CLEARGEO/SETGEO/syncs. **G_VTX w0 is GBI1-style** (`04<<24 | dst<<16 | 16·n`, dst=((n−1)<<4)|v0; all 2826 have v0=0, n≤16) — bswap only, no field remap (D47.5). ROM scan: all 2805 seg-5 VTX refs have even offsets → lossless; no nested G_DL in model files. - **Marker format:** top byte of w0 = 0xC0; type = `w0 & 7` (only 0/2/3/4 exist); texnum = `w1 & 0xfff`; min = top byte of w1. texLoadFromGdl dispatches on the top byte — PC fix: `(u8)(in->words.w0 >> 24)`. Type-0/1 markers with a valid tex are SKIPPED by the game (no expansion); the rest expand per D45's K table, which D46 verified as a true worst-case bound (no LUT textures exist; 99% of textures have maxlod=0 → TileLods emits nothing; state dedup resets per GDL via sub_GAME_7F0CC4C8). @@ -358,8 +364,8 @@ as fallback.** - Python on this box: no f-strings with nested quotes (< 3.12) — use `%` formatting. ## Definition of done -- **Session A:** D49 committed — all 8 checklist items adjudicated with evidence; - HANDOFF status updated; if any item refuted, the fallback/adjustment is spelled out. +- ~~**Session A:** D49 committed~~ — **DONE**: 8/8 CONFIRMED with evidence, + HANDOFF updated, committed. - **Session B (the milestone proper):** stage objects load without faulting and the rendered scene contains actual model geometry per the pixel assert (not just the clear color / background), stable under a 30 s+ soak. Committed + pushed to `origin/master` diff --git a/docs/PCPortResearch.md b/docs/PCPortResearch.md index fd49b08a..576c3776 100644 --- a/docs/PCPortResearch.md +++ b/docs/PCPortResearch.md @@ -1569,6 +1569,108 @@ ROM before executing Plan B; record results as D49): see docs/HANDOFF.md §Task 1 — eight claims, each with a falsification criterion. If any fires, fallback to Plan A with the corrected guard from item 4. +**D49 (REVIEW — Plan B verification: D48 checklist R1–R8 all CONFIRMED)** +Independent review session executed the D48.6 checklist against code + ROM +(NTSC `data/ge007.ntsc-final.z64`). **Verdict: 8/8 confirmed — Plan B is +cleared for execution (HANDOFF Task 2); no fallback to Plan A.** Per item: + +1. **R1 RZ format — CONFIRMED.** `decompressdata()` (port/src/rzdecomp.c) +skips the 2-byte header and raw-deflates (`inflateInit2(-15)`, avail_in ≤ +0x400000). All 512 model files (C*/G*/P*Z in +`assets/obseg/file_resource_table.inc.c`; table↔`scripts/filelist.u.csv` +set identity exact, 512=512) start `0x11 0x72` and inflate cleanly. +Σ round8(compressed) = **1,277,088 B — exactly D48's claimed number**; max +single-file compressed 0x421E; max decompressed/compressed ratio 4.75. +2. **R2 romCopy is a host memcpy with no 0x10C00000 limit — CONFIRMED.** +`romCopy` → `doRomCopy` → `osInvalDCache` (no-op shim) + `osPiStartDma` → +`piServiceDma` (port/src/libultra.c): the only gate is +`romdataCartAddrValid()` (port/src/romdata.c:212), then plain memcpy. The ROM +is VirtualAlloc'd at CART_BASE 0x10000000 sized exactly romSize +(romdata.c:156-159); nothing in the port layer bounds the region, so placing +sidecars at [CART_BASE+romSize, …) needs only a reservation-size extension + +validity-check extension in romdataInit. +3. **R3 the table is the single chokepoint — CONFIRMED.** All 512 model loads +funnel through `load_object_fill_header` (src/game/objecthandler_2.c:89) → +`_fileNameLoadToBank`/`_FileNameLoadToAddr` → `fileIndexLoad*` → +`load_resource`, which read `hw_address`/`rom_size` from +`file_resource_table` + `resource_lookup_data_array`. The C*/G*/P*Z symbols +are `.set` markers in `port/src/romassets_.s` (gen'd by +`scripts/gen_romassets.py`); no game code dereferences them outside the table +(grep: all hits are string literals/comments). Callers audited: bondview2.c, +chr_b.c, ejectedcartridges.c, front.c, gun.c, loadobjectmodel.c. +4. **R4 obInit runs once, before any model load — CONFIRMED.** `obInit()` +is called exactly once at src/boss.c:179 inside one-shot +`bossInitMainthreadData()` (before the infinite main loop); it computes +`rom_size` from adjacent-entry `hw_address` DELTAS (ob.c:122). `rom_size`/ +`hw_address` are touched only in ob.c. The D43 crash stack was post-obInit. +A lazy one-shot table patch at the port hook site in +`load_object_fill_header` is therefore safe. +5. **R5 buffers fit — CONFIRMED (quantified).** Fresh dst==0 loads: max +round8(compressed)+8 = 0x4228 ≪ STAGE bank (max 0x24C400 = poolArea 0x2A4400 +− me 352 KiB NTSC; ≥ ~1.3 MB even after conservative pre-model stage usage — +BG stan ≤ 0xA3E0 per stage). Worst-case post-compaction size across all 512 +files (D_PC + 16·Σ(K_t−1) with K_BOUND {0:38,1:46,2:36,3:18,4:15}) = +**0x16F74** (Csuit_lf_handZ) — 25× headroom vs a fresh bank. dst!=0 callers: +every buffer ≥ 0xF000 > 0x4228; `tools_pc/d43_chainbound.py` re-run against +the CURRENT constants all pass: suit 0x1002C≤0x18000, trigger/watchlaser +0xED20≤0x17000, wallet 0x105BC≤0x17000, worst weapon GmapZ 0xE91C≤0xF000, +bondview chain 0x1CE5C≤0x23000, cast rifle/pistol 0x240DC/0x23F24 ≤ 0x25000 +(front.c:7795), logo 0x85000 (initmenus.c:38). `poolRemaining := D_PC` is +automatic (`load_resource` ← `decompressdata` return, ob.c:61). +6. **R6 poolRemaining hazard + reset — CONFIRMED.** Mechanism verified in +current code: `fileIndexLoadToBank` (ob.c:219-247) allocates +`poolRemaining` when non-zero (the post-compaction P left by fileSetSize), +and `load_resource` (ob.c:49-53) needs round8(rom_size)+8 ≤ bytes or it sets +poolRemaining=0 → **silent load failure**. The data CAN trigger it (e.g. +PlegalpageZ: round8(C)+8 = 0xFC8 > D_PC = 0x5F0; its own path is a custom +buffer so it is safe, but the condition exists for any file reloaded in the +same stage). Hazard window = same-stage reloads only: poolRemaining is zeroed +at stage entry/exit (boss.c:415-417 / 639-641), and all consumers of +`get_pc_remaining_buffer_for_index` / `get_pc_buffer_remaining_value` run +post-load+compaction; the PROMOTE walk (sub_GAME_7F075A90) does not touch +lookup data. The addr path is unaffected (bytes = caller's fixed buffer). +The one-line poolRemaining=0 reset before `_fileNameLoadToBank`/ +`_FileNameLoadToAddr` in `load_object_fill_header` is **pending** (not yet in +tree) and closes the hazard for both plans. +7. **R7 per-region sidecars + directory matching — CONFIRMED (with notes).** +Region is build-time-fixed: `build-pc.sh` ROMID → CMake ASSET_REGION {u,e,j} +→ `romassets_.s`; the loaded ROM token (`ge007..z64` / +`baserom..z64`) uniquely determines the region and romHeaderValid enforces +the country byte, so romdataInit can derive `data/pcmodels-/` with the +same token logic (baserom.u → ntsc-final, baserom.e → pal-final). Name sets +are NOT identical across regions (filelist: u=512, **e=465**, j=512) — the +generator must map whatever files exist, and the patch loop only patches names +present in the manifest. Notes: (a) romdataInit has **no JP candidate** +(GE007_IS_PAL comes from versioninfo.h.in) — PC cannot boot a JP ROM today, +so JP sidecars are moot until that is added; (b) missing sidecar directory → +warn + continue in ROM-only mode (Task 2 step 2). +8. **R8 emission-spec completeness — CONFIRMED (with one explicitness gap).** +Walked every `ModelRoData_*Record` for the 14 opcodes present in ROM +(1,2,4,8,9,10,12,13,15,18,21,22,23,24) in src/bondtypes.h: every field is +covered by Verified facts + D47.13 (pointer-promotion list D47.5; f32 bswap; +TextureID bswap32; vertex arrays D47.8; GDL slots D47.13). A fresh C probe +(mingw gcc with the EXACT flags from build-pc/compile_commands.json) +re-verified all 15 PC record sizes + ModelNode (0x30) + ModelFileHeader +(0x38) + Vertex (0x10) and every pointer-promotion offset — **no drift** vs +d43_convert.py's PC_REC / the spec. Gap: Verified facts enumerate bswap for +f32, vertex-array s16s and the TextureID u32, but do not explicitly enumerate +the **u16/s16 record scalars** (AnimPart, MatrixIndex, JointID, MatrixIDs, +Group1/2, RwDataIndex, op4 numVertices@0x10, op24 nv/ncv/ModelType/ +RwDataIndex). Data check: these hold small BE values (JointID 1–11, nv ≤ +73) that raw emission would corrupt (LE read → ×256); all research tooling +reads them BE (`bu16`/`be16`). The rule follows from the D33 per-field +endianness convention (u16 → bswap16), so this is an explicitness gap, not a +missing rule — but Task 2's emit pass must bswap16 every u16/s16 record +scalar (padding/reserved may be zeroed), and the HANDOFF Verified facts +should be amended to say so. + +Environment notes for Task 2: standalone gcc needs +`/c/msys64/mingw64/bin:/c/msys64/usr/bin` on PATH (cc1 fails SILENTLY without +it — exit 1, zero diagnostics), and `-std=c11` is required (the CMake flag; +under the default gnu23 `typedef s32 bool` in bondtypes.h breaks). Also: +`include/stddef.h`'s body is `#if 0`'d — offsetof/size_t are unavailable in +game TUs. + ### G. Phase 2 status (current) Done: PD fast3d integrated (`port/fast3d/`, replaces the deleted gfxstub.c);