diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 175ae7c8..478d5d14 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -108,18 +108,36 @@ Full workstream breakdown (WS1–WS6) in `docs/PLAN-linear-level-sweep.md`. end-credits crash; **Cuba itself loads + renders 300+ frames fine** and the credits work through the real front-end flow. -## Next task — C2 / C2-GDL model-GDL relocation (Runway + Facility + Jungle) +## Done this session (M-18) — 1 fix, 18→20/21 PASS -The last 3 crashes are **one class**: the model / prop GDL relocation -(`texLoadFromGdl` / `sub_GAME_7F0762E0`, `objecthandler_2.c`) writes -non-16-aligned `dst` because it mixes N64 8-byte `Gfx` and PC 16-byte -stride in the `replacementgdl` / `name` offset math → garbage `G_SETTIMG` -w1 → `import_texture_i8` fault (Runway, Facility). Jungle renders ~300 -frames then hits a related explosion-DL `G_MTX` (`gfx_sp_matrix`). -**Brief: `docs/BRIEF-C2gdl-model-reloc.md` — its own focused session** -(D80/D82/D83 area). Files: `src/game/objecthandler_2.c`, `src/game/tex.c` -(`texLoadFromGdl`), `src/game/model.c` (`modelNodeReplaceGdl`), -possibly `tools_pc/d43_emit.py` model texture-blob offsets. +- **D130** (`port/src/romdata.c`, uncommitted) — Facility `-level_34` + + Runway `-level_35` C2 crash (`import_texture_i8` AV on wild ptr + `0x72181ee8`). Root cause was NOT the model-GDL relocation the + D124-Facility addendum / `BRIEF-C2gdl-model-reloc.md` suspected — + disproved: `texLoadFromGdl` never copies a `G_SETTIMG` on these levels, + `texWriteLoadToTmem*` is never called, and `gdl` in `sub_GAME_7F0762E0` + is still a segmented `0x05xxxxxx` value so its `& 0x00ffffff` masks are + correct. Real cause: **`romdataFixupFont` corrupts BankGothic/ZurichBold + glyph indices 0/1/2** — the in-place N64 24B → PC 32B `fontchar` relayout + aliases `dst`/`src` for low glyph indices (stride grew 8B < the 24B field + read span), so a field write clobbers a later field's source mid-loop + (glyph 1 `width` = `bswap(index)`, glyph 2 `pixeldata` ≈ `0x020002e8` → + `+= font_base` → wild). Facility's title "Chemical Warfare Facility #2" + renders `#` → `gDPLoadTextureBlock(gdl, curchar->pixeldata=wild, …)` → + fast3d AV. Fix: stage all 6 N64 fields in a local `f[6]` before writing. + §F/§H **D130**; PORT-LEARNINGS §A. Verified: -level_34 0/~14, -level_35 + 0/4; -level_09/20/24 unregressed. + +## Next task — Jungle `-level_37` (C2m, the last crash) + +Jungle renders **~300 frames** then faults in `gfx_sp_matrix` +(`gfx_pc.cpp:1046`, fault `0x401c68e0`) — an explosion / particle DL +`G_MTX` with a bad matrix pointer. D75 / D124-Jungle matrix-family +territory (compiled/global DL, NOT the per-level model-GDL path — that +premise is dead, see D130). `docs/BRIEF-C2gdl-model-reloc.md` is now +mostly obsolete. Likely near `port/src/gimgfixup.c` (D124-Jungle) / +compiled `globalDL_0xNNN` matrix refs, or fast3d's `seg_addr` for a +`G_MTX` w1. Then all 21 → hand `docs/LEVEL-PLAYTEST.md` to the user (WS6). Everything else: hand `docs/LEVEL-PLAYTEST.md` to the user for the WS6 completion pass (real input, per-level objective checklist). diff --git a/docs/LEVEL-STATUS.md b/docs/LEVEL-STATUS.md index 6360513d..07d3eab1 100644 --- a/docs/LEVEL-STATUS.md +++ b/docs/LEVEL-STATUS.md @@ -24,9 +24,9 @@ Build = `f2beae4b` + `tools_pc/d88_propdefs.py` (D123) + `port/src/gimgfixup.c` | Statue | 22 | **PASS** 80.1% | — (C1 fixed, D123) | — | | Streets | 29 | **PASS** 91.6% | — (C1 fixed, D123; no C2 crash this run — timing) | — | | Cradle | 41 | **PASS** 55.4% (low — partial render, no crash) | — (C1 fixed, D123) | — | -| Runway | 35 | CRASH (intermittent, ~3/4 runs; `0x1400d1d83`) | `import_texture_i8` — same as Facility | **C2** | -| Facility | 34 | CRASH | `import_texture_i8` gfx_pc.cpp:821 (fault 0x72181ee8) — model-GDL relocation align bug, D124, NOT fixed | **C2** | -| Jungle | 37 | CRASH | `gfx_sp_matrix` gfx_pc.cpp:1046 (fault 0x401c68e0) — C2 texture crash fixed (D124), now explosion-DL `G_MTX` (D75/matrix family) | **C2m** | +| Runway | 35 | **PASS** | — (D130: `romdataFixupFont` glyph-relayout aliasing) | — | +| Facility | 34 | **PASS** | — (D130: `romdataFixupFont` corrupted glyph `#`/`"` pixeldata → fast3d AV) | — | +| Jungle | 37 | CRASH | `gfx_sp_matrix` gfx_pc.cpp:1046 (fault 0x401c68e0) — renders ~300 frames then explosion-DL `G_MTX` (D75/matrix family); separate from D130 | **C2m** | | Aztec | 28 | **PASS** 90.8% | — (C3 fixed, D125) | — | | Bunker2 | 27 | **PASS** 91.5% | — (C3r fixed, D126) | — | | Depot | 30 | **PASS** 79.8% | — (C4 fixed, D126) | — | @@ -34,7 +34,13 @@ Build = `f2beae4b` + `tools_pc/d88_propdefs.py` (D123) + `port/src/gimgfixup.c` | Surface2 | 43 | **PASS** 70.4% | — (C6 fixed, D126) | — | | Surface1 | 36 | **PASS** 77.5% | — (C7 guarded, D127) | — | -**18 / 21 PASS.** D123 cleared C1 on all 6. D124 cleared Jungle's texture +**20 / 21 PASS** (M-18). D130 cleared C2 Facility + Runway — the crash was +`romdataFixupFont` corrupting fontchar glyphs 0/1/2 (in-place 24→32B relayout +aliases for low indices), NOT the model-GDL relocation the D124-Facility +addendum suspected. Only Jungle (C2m, explosion-DL `G_MTX`, ~frame 300) +remains. + +**(older) 18 / 21 PASS.** D123 cleared C1 on all 6. D124 cleared Jungle's texture crash (now hits an explosion-DL matrix crash). D125 cleared C3-Aztec. **D126** (objective sub-record `->next` pointer widens 4→8B → 8-byte store clobbers the next propdef record's header → walk desync → wrong command @@ -167,15 +173,13 @@ id from a propDef record). Fault addr 0x0. Files: `tools_pc/d88_propdefs.py`, crash on load, not silence. May be dodge-able with a narrow guard until audio lands. Files: `src/snd.c`, `port/src/` audio stubs. -## Next (3 crashes, 1 class) -1. **C2 / C2-GDL** Runway + Facility — model/prop GDL relocation writes - non-16-aligned `dst` (N64 8B `Gfx` vs PC 16B stride) in the - `texLoadFromGdl` / `sub_GAME_7F0762E0` path → garbage `G_SETTIMG` w1 → - `import_texture_i8` fault. `docs/BRIEF-C2gdl-model-reloc.md` — its own - focused session (D80/D82/D83 area). -2. **C2m** Jungle — renders ~300 frames then an explosion-DL `G_MTX` - (`gfx_sp_matrix`), D75/matrix family. May be downstream of the same - GDL-reloc infra. +## Next (1 crash) +1. **C2m** Jungle `-level_37` — renders ~300 frames then an explosion-DL + `G_MTX` (`gfx_sp_matrix` gfx_pc.cpp:1046, fault 0x401c68e0), D75/matrix + family. NOT related to D130 (Facility/Runway were a font-relayout bug, now + fixed). `docs/BRIEF-C2gdl-model-reloc.md` is now largely obsolete — its + model-GDL-relocation premise was disproven (D130); the Jungle crash is a + compiled/explosion-DL matrix pointer, closer to D75/D124-Jungle territory. Everything else loads + renders + survives the no-input window. Hand `docs/LEVEL-PLAYTEST.md` to the user for the WS6 completion pass. diff --git a/docs/PCPortResearch.md b/docs/PCPortResearch.md index 5254191b..f7427760 100644 --- a/docs/PCPortResearch.md +++ b/docs/PCPortResearch.md @@ -820,7 +820,8 @@ covers D24–D69; the log continues in §H (D32 procedure, D70–D121). | D123 | crash class C1: `chrIsNotDeadOrShot` NULL deref on 6 levels (Dam/Runway/Frigate/Statue/Streets/Cradle). D122's `OBJ_TAIL_DESC` zeroed the widened `VehichleRecord/AircraftRecord.ailist` slot (w32), but `prop.c:1764/1786` reads a pre-populated int AI-list id there → `ailistFindById(0)` → `GAILIST_AIM_AT_BOND` → `ai()` runs a CHR aim list with `ChrEntityp==NULL` | converter fixed (`OBJ_ID_WORDS`); C1 cleared on all 6, residual crashes are fast3d (C2) | | D125 | crash classes C3+C6 (Aztec/Bunker2/Surface2): **root cause found (M-16)** — `tools_pc/d88_emit.py:374` assigns a 4-byte literal to an 8-byte slice (`out[dst_o+0x30:dst_o+0x38] = b"\x00…"`, the stan zero-fill in `emit_pad`); every pad/boundpad record silently shrinks the output bytearray by 4 B, and once the tail falls below the boundpad plink string blob, later verbatim leaf writes hit Python out-of-range slice semantics (insert at current end, not relocated offset) → boundpad names drift/truncate (Aztec pad33 `p138d2`→`8d2`) → `stanPackId()` reject → `getposstan()` NULL stan → `setupDoor` leaves door `model=NULL` → crash `propobj.c:13601`. M-14's "propDefs zeros in RAM" was a misread; sidecar propdefs were always correct (M-15) and post-load RAM matches the sidecar byte-for-byte (M-16) | **resolved (M-16b)** — line 374 → 8 NULs; all 21 sidecars regen'd; **Aztec `-level_28` now PASSES** (was C3 CRASH). Bunker2 falls through to a separate DOOR-tail `linkedDoor` layout bug (C3 residual) | | D126 | crash classes C3r/C4/C6 (Bunker2 `-level_27` `door7F054FB4` propobj.c:13523, Depot `-level_30` prop.c:902, Surface2 `-level_43` loadobjectmodel.c:393): the objective sub-records `criteria_picture` (30), `criteria_roomentered` (32), `criteria_deposit` (33), `setup_objective_text` (35) each end in a `T *next` list pointer that the setup walk (`set_parent_cur_obj_*` / `setup_briefing_text_entry_parent`) writes unconditionally. On PC that pointer widens 4→8B and lands 8-aligned at offset 16 → struct is 24B/6w (N64 16/20). `d88_propdefs.py` emitted them at N64 size via the generic arm → the runtime 8-byte `->next` write clobbered the *next* record's header → propdef walk desynced, command indices drifted ~100, `linkedDoorOffset + arg2` resolved to the wrong record → door `linkedDoor` chain walked into garbage. | resolved — `d88_propdefs.py` PROPDEF_PC_BYTES[30/32/33/35]=24 + typed handler; `loadobjectmodel.c` sizepropdef PORT returns 6. Bunker2/Depot/Surface2 now PASS (13→16/21) | -| D124 | crash class C2: fast3d bad texture pointer. **Jungle** (`0xabcd0824`): `gimgSyncCompiledGlobalDLs()` slot-detect keyed on the post-fixup marker, which `texLoad()` had already overwritten → compiled `globalDL_0xNNN` explosion DLs kept link-time `IMAGESEG` words → latent on every level, tripped by the first explosion-DL draw. **Facility** (`0x72181ee8`): separate — model/prop GDL from the `texLoadFromGdl()`/`sub_GAME_7F0762E0` relocation path writes non-16-aligned `dst` (N64 8B vs PC 16B `Gfx` stride mix in `objecthandler_2.c`), open D80/D82/D83 area | Jungle fixed (`port/src/gimgfixup.c`); Facility diagnosed, not fixed | +| D124 | crash class C2: fast3d bad texture pointer. **Jungle** (`0xabcd0824`): `gimgSyncCompiledGlobalDLs()` slot-detect keyed on the post-fixup marker, which `texLoad()` had already overwritten → compiled `globalDL_0xNNN` explosion DLs kept link-time `IMAGESEG` words → latent on every level, tripped by the first explosion-DL draw. **Facility** (`0x72181ee8`): see **D130** — the model-GDL-relocation hypothesis (M-14 addendum) was WRONG; real cause was `romdataFixupFont` | Jungle fixed (`port/src/gimgfixup.c`); Facility → D130 | +| D130 | crash class C2 (Facility `-level_34`, Runway `-level_35`): `import_texture_i8` AV on a wild texture pointer (`0x72181ee8`), baked into the HUD glyph DL by `gDPLoadTextureBlock(gdl, curchar->pixeldata, …)` when the level-title string ("Chemical Warfare Facility #2") renders `#`/`"`. Root cause: `romdataFixupFont` (`port/src/romdata.c`) converts the N64 24B fontchar array to the PC 32B array **in place**, forward-field, backward-glyph. PC−N64 stride = 8, so for glyphs 0/1/2 `d` overlaps `s` by less than the 24B read span and an early field write clobbers a later field's source mid-`for k<5` loop → glyph 1's `width` = `bswap(index)` = `0x01000000`, glyph 2's `pixeldata` = `pcPixOff+(index−pixStart)` ≈ `0x020002e8` → `+= font_base` → wild → fast3d AV. On N64 the array is read-only rodata, never re-laid-out, so no bug there. NOT the model-GDL relocation (`sub_GAME_7F0762E0` / `texLoadFromGdl`) the D124-Facility addendum suspected — `gdl` there is still a segmented `0x05xxxxxx` value so the `& 0x00ffffff` masks are correct, and `texLoadFromGdl` never copies a `G_SETTIMG` for these levels. | resolved — stage all 6 N64 fields in a local `f[6]` before writing any. Facility + Runway now PASS (16→18/21). Probes reverted. | Phase 2 replaced the Phase-1 demo loop with the real `mainproc()` on real OS threads, compiled GE's real `src/sched.c`, and brought in PD's fast3d software @@ -4389,7 +4390,58 @@ unregressed. Sweep 13 → 16/21. Generalisable quirk appended to align), C2m Jungle (`G_MTX`), C5 Control (BG portal), C7 Surface1 (`sndSetupSound`). +**D130 (session M-18) — Facility + Runway C2 crash was `romdataFixupFont`, +NOT the model-GDL relocation.** `-level_34` (and `-level_35`) fault in +`import_texture_i8` (`gfx_pc.cpp:821`) on `loaded_texture.addr = 0x72181ee8`, +an un-mapped wild pointer. gdb showed the fault DL is a 2D `G_TEXRECT` block +(`gfx_dp_texture_rectangle` → `gfx_sp_tri1(is_rect)`), with the wild pointer +baked into a preceding `G_SETTIMG` w1. Traced back through `GE_C2GDL` probes: + +- `texLoadFromGdl` **never copies a `G_SETTIMG` (0xFD)** on these levels, and + `texWriteLoadToTmemAddr/Zero` are **never called** — so the M-14 addendum's + "model-GDL relocation writes non-16-aligned dst" hypothesis is wrong. A + probe in `sub_GAME_7F0762E0` also confirmed `gdl` from + `modelIterateDisplayLists` is still a **segmented `0x05xxxxxx` value** + (`modelPromoteNodeOffsetsToPointers` does NOT promote DL Primary/Secondary), + so the `Switches + (x & 0x00ffffff)` idiom is correct as written. +- The wild `G_SETTIMG` is emitted by `gDPLoadTextureBlock(gdl, + curchar->pixeldata, G_IM_FMT_I, G_IM_SIZ_8b, …)` in + `textRenderGlyphOutlined` (`textrelated.c`) while rendering the level-title + string "Chemical Warfare Facility #2" — specifically glyph `#` (and `"`). +- `curchar->pixeldata` for BankGothic/ZurichBold glyph indices **0, 1, 2** is + corrupt after `load_font_tables`: glyph 1 `.width = 0x01000000`, glyph 2 + `.pixeldata = 0x020002e8` (→ `+= font_base` → wild). + +Root cause: `romdataFixupFont` (`port/src/romdata.c`) re-lays-out the N64 24B +`fontchar` array into the PC 32B array **in place** — backward over glyphs, +forward over the 5 leading `u32` fields per glyph. PC−N64 stride = 8, so +`d = s + 8*i`; for `i ∈ {0,1,2}` `d` overlaps the 24-byte source read span +and an early `*(u32*)(d+4k) = bswap(*(u32*)(s+4k))` write clobbers a later +`k`'s source. Glyph 1 (`Δ=12`): k=3/k=4 read bytes k=0/k=1 already +overwrote → `width = bswap(bswap(index)) = index = 0x01000000`. Glyph 2 +(`Δ=20`): k=0 writes `s+20`; line 507 then reads `s+20` for the pixeldata +offset → `o = index = 0x02000000` → `o >= pixStart` → remapped to +`pcPixOff + (0x02000000 − pixStart) ≈ 0x020002e8`. Confirmed exactly against +a raw-bytes dump. On N64 the array is read-only rodata — never re-laid-out — +so no bug there; this is purely a PC-relayout aliasing bug. + +Fix: stage all six N64 fields into a local `u32 f[6]` before writing any +field of `d`. (A previous iteration's write can never touch `s(i)` — for +`j > i`, `d(j)` starts past `s(i)`'s end — so a full pre-read is safe.) +`GLYPH_IDX` clamp in `textrelated.c` was tried and reverted: the crashing +string has no control byte, and a negative-index guard is a separate latent +concern (logged in PORT-LEARNINGS §A). + +**Verification:** `-level_34` 0 crashes / ~14 runs (was ~50–100%); `-level_35` +0 crashes / 4 runs (was ~3/4). `-level_09`/`-level_20`/`-level_24` unregressed +(91.3% / 91.7% / 90.8% non-clear). `-level_37` (Jungle) unchanged — still the +separate C2m `gfx_sp_matrix` explosion-DL `G_MTX` crash at ~frame 300 (D75 +family). Sweep **16 → 18/21**. All probes reverted; only +`port/src/romdata.c` (11 lines) changed. Confidence: **high** (exact +byte-level match, deterministic value, 2 levels fixed, no regression). + **D124-Facility addendum (session M-14 — partial, NOT fixed, out of time).** +*(Superseded by D130 — the model-GDL hypothesis below was wrong.)* Re-instrumented with a `GE_C2GDL` probe in `sub_GAME_7F0762E0` (`objecthandler_2.c`) + a bad-`G_SETTIMG` catcher in fast3d (`gfx_pc.cpp` G_SETTIMG case). Both probes **reverted** — tree is clean. diff --git a/docs/PORT-LEARNINGS.md b/docs/PORT-LEARNINGS.md index ae77c058..8f0cf1bd 100644 --- a/docs/PORT-LEARNINGS.md +++ b/docs/PORT-LEARNINGS.md @@ -40,6 +40,28 @@ state. 22 (`TagObjectRecord.NextTag`, already handled). Check every record type the walk can see for a `set_parent_*` / `*_entry_parent` writer. +- **D130 corollary — in-place N64→PC struct-array re-layout aliases when the + stride grew less than the read span.** A ROM struct array widened on PC + (`fontchar` 24→32B, `romdataFixupFont`) is often re-laid-out *in place*: + for glyph `i`, `dst = base + PCstride*i`, `src = base + N64stride*i`, so + `dst − src = (PCstride−N64stride)*i`. For small `i` that delta is *less + than the per-element field-copy read span*, so `dst` overlaps `src`, and a + forward field-by-field `*dst_k = f(*src_k)` loop overwrites a not-yet-read + `src_j` (`j > k`). Result: later fields of the first few elements come out + as `f(f(earlier field))` — e.g. glyph 1's `width` became `bswap(bswap(index))`. + Iterating elements backward does NOT help (the overlap is *within* one + element). Fix: read every source field of the element into locals first, + then write. Grep every `romdataFixup*` / in-place relayout for a + read-write loop whose `dst`/`src` can alias for low indices. +- **A negative/OOB index into a widened struct array is fatal on PC, benign + on N64.** `chars[*text - 0x21]` with a control byte (`*text < 0x21`) reads + before `chars[0]`. On N64 that hits the adjacent 4-byte-field kerning table + (small values → a garbage-but-TMEM-valid glyph); on PC the struct's 8-byte + `pixeldata` reads a wild pointer → fast3d AV. Latent in `textrelated.c` + (`textRender*`/`textMeasure` ASCII paths) — a `GLYPH_IDX` clamp fixes it if + a level actually feeds a control byte to the HUD text (none do yet; D130 + was a font-relayout bug, not this). + ## B. 16-byte PC `Gfx` / `Vtx` vs 8-byte N64 Any buffer reservation, `memcpy` size, slot stride, or pool budget diff --git a/port/src/romdata.c b/port/src/romdata.c index 15aa4122..df919050 100644 --- a/port/src/romdata.c +++ b/port/src/romdata.c @@ -501,10 +501,18 @@ void romdataFixupFont(u8 *blob, u32 n64Size) for (i = n; i-- > 0;) { const u8 *s = blob + N64_COFF + N64_CSTRIDE * i; u8 *d = blob + ROMDATA_OFFSEXP(struct font, chars) + sizeof(struct fontchar) * i; - u32 k, o; + u32 f[6], k, o; + /* D130: PC stride (32) - N64 stride (24) = 8, so for the first few + * glyphs `d` overlaps `s` by less than the 24-byte read span and a + * field write clobbers a later field's source mid-loop (glyph 1's + * width came out as bswap(index), glyph 2's pixeldata as + * pcPixOff+(index-pixStart) -> wild pointer -> fast3d AV rendering + * '#'/'"'). Stage all six N64 fields before writing any. */ + for (k = 0; k < 6; k++) + f[k] = romdataBswap32(*(const u32 *)(s + 4 * k)); for (k = 0; k < 5; k++) - *(u32 *)(d + 4 * k) = romdataBswap32(*(const u32 *)(s + 4 * k)); - o = romdataBswap32(*(const u32 *)(s + 20)); + *(u32 *)(d + 4 * k) = f[k]; + o = f[5]; if (o != 0 && pixStart != 0 && o >= pixStart) o = pcPixOff + (o - pixStart); /* D51: on PC pixeldata is a u64 at d+24 (fontchar needs 8-align,