mirror of
https://github.com/jkdansereau/goldeneye-pc-port.git
synced 2026-10-23 15:52:42 +02:00
D236 (M-198): fix Surface 1/2 tree cards; root cause was a DETAIL tile binding
GE's texLoadFromGdl picks a handler per G_NOOP texture reference from w0 & 7. TEXTURETYPE_LOD/MIPMAP (texHandleType0/Type2) issue ONE load block at TMEM 0, so tiles 0..n are LOD levels of that one image. TEXTURETYPE_DETAIL (texHandleType1, src/game/tex.c) issues TWO: the detail texture at TMEM 0 and the real base image at TMEM texGetSizeInBytes(detail,0) on tiles 1+. gfx_lod_tile_offset folded every G_TL_LOD draw to tile 0 (D107 -- correct for the first two, wrong for DETAIL), so the port drew the detail texture and never imported the base image at all. All four Surface 1 treeline cards (tex1198-tex1201, RGBA5551 64x17 cut-outs) are DETAIL bindings sharing one detail texture, tex2465: an opaque 32x32 IA8 noise tile. That is the same texture D267/D276 measured at 1023/1024 texels opaque and called the mystery "noise painter" -- hence 20+ passes chasing decode, alpha, mip and geometry theories for what was a binding bug. Fix (port layer only): under G_TL_LOD, if tile fi+1 differs from tile fi in BOTH TMEM address and fmt/siz, sample tile 1. Asymmetric on purpose -- LOD levels of one image always share a format, so a mip chain can never trip it, and a detail pair that happened to share a format just keeps the old behaviour. Config Video.DetailBaseTile, default on. Verified: Surface 1 idle spawn goes from an opaque tiled noise wall to discrete snow-covered pines with ground/snow unchanged; Surface 2 shows a discrete night treeline; Depot renders clean; bunker1 golden framediff is bit-identical with the fix on and off. Offline scan of every level's bg: DETAIL bindings exist only in sevx (84), depo (39), sevb (4), sev (4), so every other level is provably untouched. User play-test confirmed. Also refuted and removed pass 25's fix direction: cut-out mip suppression changed nothing, and its alpha census showed the wall is painted by a fully opaque texture -- retiring the mip / 1-bit-alpha / XLU-smear branch. The GE_D236P25 probe is gone with it. tools_pc/gen_env_probes.py: stop counting a "GE_FOO[" log-line TAG in the doc as a documented env var. GE_TEXDUMP_ADDR prints "GE_TEXA[...]", which is not the name of any getenv(), so the check reported a permanent phantom GONE. Generalised as docs/porting-notes.md D12; full trail in findings.md D236 pass 26. Carries the pending D75/M-197 sidecar-converter fix and the env-gated D236/D75 probes, which the doc gates require to land alongside their rows.
This commit is contained in:
1 parent
327e6fe1a7
commit
425237f93f
17 files changed
+1732
-12
No files matched your search
@@ -3,7 +3,7 @@
|
||||
Every `getenv("GE_…")` in the tree. The table prose is hand-curated;
|
||||
**`tools_pc/gen_env_probes.py`** re-greps the live sites and reports drift
|
||||
(NEW / GONE vars + a fresh file:line map) — run it before trusting the
|
||||
File:line cells. Last reconciled 2026-09-21 (GE_D235 added, D235 fix wired; GE_D288 refined per M-155 follow-up; GE_D75V added for the Nintendo-logo vertex-slot investigation, M-195; GE_D322 pool-telemetry probe added; GE_D75D + D75DCC added for the M-196 draw-path investigation that refuted slot-overwrite). All are **env-gated**: unset = zero behavior
|
||||
File:line cells. Last reconciled 2026-09-21 (GE_D235 added, D235 fix wired; GE_D288 refined per M-155 follow-up; GE_D75V added for the Nintendo-logo vertex-slot investigation, M-195; GE_D322 pool-telemetry probe added; GE_D75D + D75DCC added for the M-196 draw-path investigation that refuted slot-overwrite; GE_TEXDUMP_ADDR added + GE_D75D gained logo-mode `lg_*` fields for the M-197 root cause). All are **env-gated**: unset = zero behavior
|
||||
change, so they are safe to leave in a release build (they only cost a
|
||||
`getenv` on the guarded path). "Dead" below means *the finding it was built
|
||||
for is closed* — the probe is strip-candidate scaffolding, not that it does
|
||||
@@ -41,6 +41,7 @@ Two classes:
|
||||
| `GE_MIXERTRACE` | `port/src/mixer.c`, `src/audi.c` (`amDmaCallback`), `src/libultra/audio/load.c` (`_decodeChunk`, `alLoadParam`) | Per-opcode mixer trace to `mixertrace.log` (DMEM addresses, ADPCM books, decoded frames, `[DMAREQ]`/`[DMAHIT]`/`[DMAMISS]`/`[BINDTABLE]`). **WARNING: unbuffered per-opcode `fprintf`, ~25 MB/min. It slows the process enough to starve the audio thread — M-63 measured "audio at 13 % of real time" that was entirely this probe's own artifact. NEVER use it for timing-sensitive measurement; use `GE_D204`.** Grep it, never read it whole. | **live** (D202) — timing-unsafe |
|
||||
| `GE_TEXPITCH` | `port/fast3d/gfx_pc.cpp:924` (`gfx_tex_pitch_fix`, cached) | `=0` disables the D183 defensive de-stride in `import_texture` (default on). RC2/RC3 texture-shear A/B knob. | **live** (RC3 not finished) |
|
||||
| `GE_TEXRAW` | `port/fast3d/gfx_pc.cpp:1079` | With `GE_TEXDUMP`, also dumps the raw pre-import source bytes (`texdump/rNNN_f…_s…_WxH.bin`) so a decode bug can be told from a source-data bug offline (D183). | **live** (RC3 not finished) |
|
||||
| `GE_TEXDUMP_ADDR` | `port/fast3d/gfx_pc.cpp` (`import_texture`, cached env check) | D75/M-197: with value `<hex>`, dumps the raw source bytes of **every** import from that exact address (bypassing the 400-dump cap) to `texdump/aNNNNN_WxH.bin` + a `GE_TEXA[…]` log line with fmt/siz/size/min/max/nonzero byte stats. Distinguished "never loaded" from "loaded zeros" for the Nintendo-logo texture buffer (root-caused to the D50 sidecar converter dropping embedded blobs). | **live** (D75) |
|
||||
| `GE_D219RAW` | `src/game/image.c` (`texReadUncompressed`/`texChannelsToPixels`, `#ifdef PORT`) | Diagnostic-only, print-only (no behavior change): logs the runtime texture decompressor's bitstream header (texnum/format/width/height/compmethod) for every non-zlib wide-pixel image, plus a dump of the first row of decoded texels exactly as stored in the pool (post-`PORT_PIXEL16`, i.e. byte-swapped for the 16-bit family) for RGBA16/RGB15/IA16. No texnum filter — lets any suspect texture be cross-referenced against the offline ROM census (`scratch/d219_census.txt`) without narrowing by hand. | **live** (D219, closed — kept for future runtime-decoder triage) |
|
||||
| `GE_WRAPFIX` | `port/fast3d/gfx_pc.cpp:3166` (`gfx_set_wrap_fix`) | RC3 test override for the `Video.WrapFix` D74/D167 wrap-block path (`atoi != 0` wins over the ini). | **live** (RC3 not finished) |
|
||||
| `GE_SAVELOG` | `src/game/file.c:46`, `src/game/file2.c:16` (`SAVELOG` macro), `src/boss.c:749`, `src/game/objective_status.c:297,343` | Traces the campaign save/unlock chain (objective completion → `end_of_mission_briefing` → `fileWriteSave` → EEPROM). Used to confirm D157. `g_savelogObjOnce` gates the per-criterion dump to the `bossReturnTitleStage` call. | **live** (strip once a few more level boundaries are playtested) |
|
||||
@@ -68,12 +69,13 @@ Two classes:
|
||||
| `GE_D236ALPHA` | `port/fast3d/gfx_pc.cpp:1896` (`gfx_sp_tri1`, cached `s_d236alpha`) | D236 pass-14 diag: for the `0x0c184b50` tree-class render mode, logs the combine/alpha state of up to 4000 matching triangles (deduped on combine change) so the captured draw can be cross-checked against the source. Zero cost unset; remove once D236 concludes. | **live** (D236 OPEN) |
|
||||
| `GE_D236ZFIX` | `port/fast3d/gfx_pc.cpp:1970` (`gfx_sp_tri1`, cached `s_d236zfix`) | D236 pass-12 test override: forces `depth_update=true` for the `0x0c184b50` tree-class render mode — a cheap mechanism test before deciding what a real fix looks like. Off by default; remove once D236 concludes. | **live, test-only** (D236 OPEN) |
|
||||
| `GE_D236RAW` | `src/game/bg.c:2437` (`#ifdef PORT`) | D236 pass-16 diag: one-shot census of the raw vertex table's alpha-byte distribution (5 buckets + min/max), camera-independent, to check whether any vertex in the whole table exceeds the low ceiling. Zero cost unset; remove once D236 concludes. | **live** (D236 OPEN) |
|
||||
| `GE_D236BT` | `src/game/chr.c` (`get_ptr_allocated_block_for_vertices`, `#ifdef PORT`) + `src/game/model.c` (`doshadow`) + `port/fast3d/gfx_pc.cpp` (`port_d236bt_dump()`) | D236 pass-17 diag: host stack backtrace at vertex-arena allocation time (pinpoints the CPU-side quad builder — proved it's `doshadow`) plus a one-shot live ShadowRecord/tconfig dump in `doshadow` (writes `d236sh.bin` to CWD on first hit). Zero cost unset; remove once D236 concludes. | **live, TEMP** (D236 OPEN) |
|
||||
| `GE_D245V` | `src/game/sky.c:962` (`skyPortRenderPoly`-adjacent, `#ifdef PORT`) | D245 water-seam diag: per-call min/max of the recomputed S/T texture-coordinate ranges for the water fan vertices (rate-limited), to check whether the seam tracks a tc-shift boundary. Inert unset. | **live** (D245 OPEN) |
|
||||
| `GE_D245_FIXEDSHIFT` | `src/game/sky.c:946` (`#ifdef PORT`) | D245 test override: pins the water tc shift `k` to the given integer (0..SKY_TC_MAX_SHIFT) instead of the computed per-vertex value, to A/B the allowShift=TRUE behavior. Off by default (unset = current behavior). | **live, test-only** (D245 OPEN) |
|
||||
| `GE_D252` | `port/fast3d/gfx_pc.cpp:733` (`import_texture`, cached `s_d252`) | D252 rainbow-particle diag: one-shot per (address,size) import log of the tile state (tmem/fmt/siz/addr/size/line) for small textures so a suspect particle tile can be checked against its source bytes by hand. Remove once D252 is resolved. | **live** (D252 OPEN — non-reproducible) |
|
||||
| `GE_D235` | `src/boss.c` (per-stage `while (!done)` block, after `memaReset`; `#ifdef PORT`) | D235 evidence probe: logs how many GL-texture-cache entries the per-stage `videoResetTextureCache()` call purges at each level load. A cold boot into a level should log 0; a death/restart (or any transition from a level that uploaded textures) logs >0 — direct evidence of the stale-entry mechanism. The clear itself is **not** gated on this var (it's the fix, see below); the var only controls the log line. | **live** (D235) |
|
||||
| `GE_D75V` | `port/fast3d/gfx_pc.cpp` (`gfx_sp_vertex`, end of the copy/transform loop; cached env check) | D75 (Nintendo logo) per-batch vertex probe: for each `G_VTX` batch in frames `lo..hi`, logs frame#, source pointer, n_vertices, original dest_index, raw ob[0..2] of the first vertex, min/max of the transformed x/y/z/w across the batch, and the MP translation row. Used to prove the transform is correct (M-195) and that every batch encodes v0=0 (slot-overwrite hypothesis). Value form `"lo-hi"` (e.g. `300-900` = the logo screen); unset = off. | **live** (D75) |
|
||||
| `GE_D75D` | `port/fast3d/gfx_pc.cpp` (`gfx_sp_vertex` G_VTX case + `gfx_sp_tri1` rejection/emit gates; cached env check) | D75 (Nintendo logo) per-frame draw-path probe (M-196): for frames `lo..hi`, logs one line/frame with the V/T token stream (`seq`), max triangle vertex index, cumulative vertices loaded, and per-triangle accounting — trivial-reject / backface-cull counts, how many tris reach GL, how many of those are on-screen (NDC bbox ⊂ [−1,1]²), invisible-flag count, vertex-color min/max, first oml word, depth z/w range + NaN count, texture-sampler presence (`utex` = GL shader, `cutex` = combiner), combine mode, prim/env color. Used to REFUTE the slot-overwrite hypothesis (DL is streamed, max index 15) and narrow the bug to a texture-modulate material rendering black. Value form `"lo-hi"` (logo window ≈ `244-320` on a bare front-end boot); unset = off. Companion one-shot `D75DCC:` log in `gfx_generate_cc` fires when `combine_mode==0x009ffe4f19ffe4f1` to decode the logo's combine slots. | **live** (D75) |
|
||||
| `GE_D75D` | `port/fast3d/gfx_pc.cpp` (`gfx_sp_vertex` G_VTX case + `gfx_sp_tri1` rejection/emit gates; cached env check) | D75 (Nintendo logo) per-frame draw-path probe (M-196): for frames `lo..hi`, logs one line/frame with the V/T token stream (`seq`), max triangle vertex index, cumulative vertices loaded, and per-triangle accounting — trivial-reject / backface-cull counts, how many tris reach GL, how many of those are on-screen (NDC bbox ⊂ [−1,1]²), invisible-flag count, vertex-color min/max, first oml word, depth z/w range + NaN count, texture-sampler presence (`utex` = GL shader, `cutex` = combiner), combine mode, prim/env color. Used to REFUTE the slot-overwrite hypothesis (DL is streamed, max index 15) and narrow the bug to a texture-modulate material rendering black. M-197 added logo-mode `lg_*` fields (sampled only on tris with the logo combine word — the frame-level `utex`/`cutex` are first-tri-wins and were the source of the "combiner claims 0" false anomaly): `lg_utex`/`lg_tex` + tile-0 descriptor + texture buffer address/id/size for the logo material. Value form `"lo-hi"` (logo window ≈ `244-320` on a bare front-end boot); unset = off. Companion one-shot `D75DCC:` log in `gfx_generate_cc` fires when `combine_mode==0x009ffe4f19ffe4f1` to decode the logo's combine slots. | **live** (D75) |
|
||||
|
||||
## Dxx diagnostic probes — finding closed unless noted
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ once the game is playable start-to-finish without crashes.
|
||||
|----|---------|----------------|
|
||||
| ~~**D114/D116**~~ | ~~HUD text + ammo digits render each glyph individually X-mirrored~~ | **CLOSED — NOT A BUG (M-33, §F D168).** The `GE_PCDUMP` PPM writer never reversed `glReadPixels` rows, so every capture was upside-down; "mirrored" was a misread of inverted asymmetric content. The game renders correctly on hardware (developer-confirmed). Writer fixed (`gfx_opengl.cpp`). "Every stage verified non-mirrored" was correct — there was no mirror. |
|
||||
| ~~**D74 wrap-block**~~ | ~~`gfx_pc.cpp:1546/1551` PORT block is dead code (guard `cms & G_TX_WRAP` always false; indexes per-texunit arrays with the vertex index → OOB if reached).~~ | **RESOLVED (M-30, RC3/D167).** The in-vertex-loop wrap block was reworked and hoisted to a per-texunit pre-wrap window (`gfx_pc.cpp` ~1574, now `cms`-correct and indexed by texunit not vertex), behind the `Video.WrapFix` knob (default OFF). No longer dead code or OOB. See §F "RC3 · D167". |
|
||||
| **D75** | ~~Nintendo logo mispositioned; gun-barrel Bond model absent~~; cast-roll character models — status unknown. (Rareware logo OK.) | **Shrunk (M-33, upright captures after D168).** The "overlaps D114 shared mirror" note is withdrawn. **Gun-barrel Bond RENDERS correctly** (walk + fire, upright) — the "absent" reports were the flipped capture. **Nintendo logo IS still broken** — renders as two overlapping white ellipsoids shifted left, no wordmark/geometry (a real `logoinst` model transform/geometry bug, not a flip). Cast roll not re-captured — recheck with `GE_PCDUMP="1400-2200:25"`. `findings.md` §F "D75 Bug 2 — M-33 UPDATE". |
|
||||
| **D75** | ~~Nintendo logo mispositioned; gun-barrel Bond model absent~~; cast-roll character models — status unknown. (Rareware logo OK.) | **Nintendo logo + GoldenEye logo + legal page FIXED (M-197, user-confirmed live 2026-09-21).** Root cause: the D50 sidecar converter (`tools_pc/d43_emit.py`) reserved + remapped the model's embedded texture-blob span but never copied the pixel bytes into the emitted RZ image → TEXEL0 sampled a zeroed buffer → the texture-modulate logo rendered black (M-195/M-196 had proven emission/transform/combiner all correct). Same latent bug hit `PgoldeneyelogoZ` + `PlegalpageZ`. Fix = tooling-only (blob raw-copy + round-trip check); ntsc sidecars regenerated; logo window now renders 23k px (7.59%) vs 0–33 pre-fix. Gun-barrel Bond renders correctly (M-33). Cast roll still unverified — recheck with `GE_PCDUMP="1400-2200:25"`. `findings.md` §F D75 (M-195/M-196/M-197 updates) + M-197 at file tail. |
|
||||
| **D76** | 2D disclaimer screen only partially drawn. | §F D76. |
|
||||
| ~~**D77**~~ | ~~No audible music on PC.~~ | **RESOLVED (M-104, PR #42) — in-level music now plays.** Root cause was a D6-class missing `return` in `sub_GAME_7F0C0BF0` (`mp_music.c`) that zeroed the seqp volume on the only in-level trigger path; menu/intro music bypassed it and was always fine. The remaining audio-*quality* gap (wrong-sounding instruments on some tracks — elevator named specifically — plus occasional garbling) is tracked as **D230**. §F D77/D230. |
|
||||
| ~~**D116 (HUD)**~~ | ~~proportional-font strings + ammo counter mirrored~~ | **CLOSED — NOT A BUG (M-33, §F D168).** Same capture-orientation artifact as D114. |
|
||||
@@ -25,7 +25,7 @@ once the game is playable start-to-finish without crashes.
|
||||
| ~~**D233**~~ | ~~All levels (intermittent): near doors — especially doors you can't see through to the inside — the indoor level geometry vanishes and you can see straight through it into the room, while world objects (computer terminals, furniture) in that space still draw.~~ | **FIXED + VISUALLY VERIFIED (M-107 fix, M-108 verify).** `port/fast3d/gfx_pc.cpp` `gfx_sp_tri1` was trivial-rejecting visible triangles that had a behind-camera (`w<0`) vertex. Status cell here was stale at "not investigated" — corrected M-119. §F D233. |
|
||||
| ~~**D234**~~ | ~~Dam: the guard towers along the actual dam structure are invisible except for the first one.~~ | **FIXED + VISUALLY VERIFIED (M-108).** Same underlying mechanism as D233. Status cell here was stale at "not investigated" — corrected M-119. §F D234. |
|
||||
| **D235** | Wrong textures load after dying/restarting a level — clarified: not Facility-specific, seen on multiple levels; trigger is death/restart (level reload), not a general build regression. | **Observed, not investigated** (M-106, user QA report, clarified same session). This is a state-leak on level re-entry, not a one-level texture-decode bug — prime suspect is the texture pool / CI-cache (D217/D228 neighbourhood) not fully resetting between a level teardown and reload (stale arena base, a palette-cache entry keyed by a reused address, or a texpool index not rewound). Needs: confirm it does NOT repro on a fresh level boot (only after death/restart); which surfaces; a screenshot. Next: instrument `texInitPool`/the CI-cache clear path for what does/doesn't reset on reload vs cold boot. |
|
||||
| **D236** | Surface 1: the 2D billboard trees that appear at the start of the level and around the map don't draw as discrete tree sprites — they render as a wall/sheet of tree texture instead of the proper 2D tree objects. | **OPEN — second pass active (D266–D269); first-pass decode fix was reverted after a negative user re-test (D265)** (M-106, user QA report). The earlier IA16 framing was a misidentification: the cards are flat quads by design (no billboard opcode; byte-matched ROM emission) and bind an opaque IA8 64×16 strip plus an alpha-blended **IA4 128×16** strip (texdump r117) that is a row of ~13 faint conifer silhouettes. The wall came from `import_texture_ia4` decoding the RDP's I2:A2 nibble as I3:A1 — A2=1 canopy pixels went fully opaque, A2=2 dithering vanished (bug inherited verbatim from the PD port). **Second pass (D266–D269) re-framed the bug: the visible wall band is CI8 32×32 gradient/blob cards, not IA at all** — two draw classes per room stream (OPA + XLU), BILERP/TEXTLOD/2-cycle/**FOG-on**; texel-alpha blending proven correct by force-alpha A/B (D268). Current ranked suspects: fog range/application (D218 family) > TEXTLOD/mip sampling > draw density/culling > palette. The I2:A2 decode + AC-aware edge fixes are re-applied in-tree (uncommitted) and still owe a game-wide IA4 regression sweep before commit. |
|
||||
| **D236** | Surface 1: the 2D billboard trees that appear at the start of the level and around the map don't draw as discrete tree sprites — they render as a wall/sheet of tree texture instead of the proper 2D tree objects. | **FIXED (M-198, pass 26) — awaiting user play-test sign-off.** Root cause was a texture-BINDING bug, not decode/alpha/mip/geometry: GE binds the four Surface 1 treeline cards (`tex1198`-`tex1201`, RGBA5551 64x17 cut-outs) as `TEXTURETYPE_DETAIL`, where `texHandleType1` (`tex.c`) loads the *detail* texture at TMEM 0 and the real base image at TMEM `texGetSizeInBytes(detail,0)` on tiles 1+. fast3d's `gfx_lod_tile_offset` folded every `G_TL_LOD` draw to tile 0 (D107, correct for TEXTURETYPE_LOD/MIPMAP), so the port drew the shared detail texture -- `tex2465`, an opaque 32x32 IA8 noise tile, which is exactly the 'noise painter' D267/D276 measured at 1023/1024 texels opaque -- and never imported the cards at all. Fix: when a tile pair differs in both TMEM address and `fmt`/`siz` under `G_TL_LOD`, sample the base image (tile 1). `Video.DetailBaseTile`, default on. Live-verified on Surface 1 (discrete snow-covered pines) and Surface 2; Depot clean; bunker1 golden bit-identical with the fix on/off; `type=1` bindings exist only in sevx/depo/sevb/sev, so every other level is provably untouched. Full trail: `findings.md` D236 pass 26; generalised as `porting-notes.md` §D12.
|
||||
| **D249** | Streets and Egyptian (the two biggest/most-open levels) show culling problems on far-away geometry at *default* FOV — distant geometry drops/pops incorrectly. Also a QoL ask: improve LOD in general, but gate it behind a toggle so N64-accurate ("old school") LOD stays selectable. | **Observed, not investigated** (M-119, user QA report). Distinct from D222 (high-FOV near-edge culling) and D218 (fog/far-clip tint). Two candidate mechanisms: (a) portal/room-visibility cutting off too aggressively at range on large multi-room layouts (D233/D234 neighbourhood — check for a far-plane/depth-range sibling to D233's w<0 trivial-reject fix), or (b) GE's own LOD/detail-swap system not ported faithfully (mip-tile / simple-model substitution at distance, `gfx_lod_tile_offset` family). First checks: `GE_PCDUMP` a long sightline on Streets/Egyptian where geometry visibly pops; determine which of (a)/(b) is firing before designing the fix or the requested toggle. §F D249. |
|
||||
| **D217** | 1P weapon viewmodel + character-model CI textures render with a wrong palette on many levels / multiple weapons — PP7 grip + hand render flat green on Facility & Depot, "weird or missing" gun textures elsewhere; Bond's watch face too. Surfaced by D215 (viewmodel now draws). Clean on Bunker1. | **Defect ① FIXED (M-90), pending human playtest eyeball — full detail in `findings.md` §H "D217 — M-90" and §F D217.** Root cause: weapon texpool arena bases land %8==4 on PC (memp bump drift from pointer bloat); `texAlignIndices` absolute 8-byte row padding then shifts every palette 4 bytes early while model-DL `G_LOADTLUT` reads the N64 offset (grip count=1 load lands on next tenant's header slot → `0x1A00` green; larger loads read a 2-entry zero pad — the M-86 tell). Fix: `texInitPool` rounds the base up to 8 under `#ifdef PORT` (no-op on N64); 21/21 sweep PASS. Two defects: ① **visible green = misaligned texpool arena** (fixed as above). ② **CI texture-cache collision** (no palette-content key) — proven, **fix landed on branch `fix/d217-model-texture-palette` `f6fc0399`** (palette FNV-1a hash in the CI cache key), owes a 21-level `verify.sh sweep`; does not fix ①. |
|
||||
| **D174** | "No blood effect" — user reports no visible blood when shooting guards / into surfaces. | **User QA report; likely D120, unverified.** Decals: the D120 `PointUsage[]` converter fix (M-30) was never interactively verified and only takes effect after a full sidecar regen (`d43 && d69 && d88 --regen`) — check whether the playtest build had regenerated sidecars before concluding the fix failed. Spray: D172 observed the particle spray *drawing* (wrong color, magenta/cyan) on Silo — if the user sees no spray either, that's new info (different weapon/range, or the D172 decode is worse than "wrong color"). First step: BUNKER1 firefight with regenerated sidecars; if decals still absent, instrument the `chr.c:3328` decal walk. |
|
||||
|
||||
@@ -132,7 +132,7 @@ D164,FIXED,"Disclaimer / legal screen only draws the first line (D76 root cause,
|
||||
D165,FIXED,"Front-end menu cursor doesn't feel like a mouse pointer (M-31, user bug B5 follow-up).","FIXED (port/src/input.c; Input.MenuPointerMode, Input.HipfirePitchSpeed added)."
|
||||
D166,FIXED,Hipfire mouse pitch is digital while yaw is analog —,FIXED (port/src/input.c).…
|
||||
D235,OPEN,Wrong textures load after dying/restarting a level —,OPEN — needs: does it repro on a *fresh* level boot (rules out reload-state) vs only after death/restart (confirms it); which surfaces; a screenshot.…
|
||||
D236,OPEN,"Surface 1: 2D billboard trees render as a wall/sheet of tree texture instead of discrete camera-facing tree sprites (M-106, user QA report).","OPEN, sixteenth pass (2026-09-18) —"
|
||||
D236,OPEN,"Surface 1: 2D billboard trees render as a wall/sheet of tree texture instead of discrete camera-facing tree sprites (M-106, user QA report).","OPEN — pass 24 (2026-09-21, static) DECODED THE TREE TEXTURE (new tools_pc/texdecode.py ports image.c's LOOKUP/HUFFLOOKUP/RLELOOKUP codecs)."
|
||||
D237,OPEN,"QoL ask (M-106, user QA):",OPEN — not designed; port-only
|
||||
D238,OPEN,"QoL ask (M-106, user QA):",OPEN — not designed; port-only (port/src/input.c + config.c)
|
||||
D239,CLOSED,"QoL ask (M-106, user QA):",CLOSED — executed in the v0.2.0 prep pass (port-only)…
|
||||
|
||||
|
+338
-2
@@ -539,7 +539,7 @@ covers D24–D69; the log continues in §H (D32 procedure, D70–D121).
|
||||
| D233 | **All levels (intermittent): indoor level geometry invisible near doors — you can see through walls/floors into rooms you can't see into, while world objects…** — full `## D233` entry at file tail | **FIXED + VISUALLY VERIFIED (M-107 fix, M-108 verify).** See the other D233 row for full detail. |
|
||||
| D234 | **Dam: guard towers along the dam structure invisible except the first one (M-106, user QA report).** — full `## D234` entry at file tail | **VISUALLY VERIFIED FIXED (M-108), mechanism unexplained.** See the other D234 row for detail. |
|
||||
| D235 | **Wrong textures load after dying/restarting a level — not Facility-specific, seen on multiple levels (M-106, user QA report; clarified same session).** — full `## D235` entry at file tail | OPEN — needs: does it repro on a *fresh* level boot (rules out reload-state) vs only after death/restart (confirms it); which surfaces; a screenshot.… |
|
||||
| D236 | **Surface 1: 2D billboard trees render as a wall/sheet of tree texture instead of discrete camera-facing tree sprites (M-106, user QA report).** — full `## D236` entry at file tail | OPEN, sixteenth pass (2026-09-18) — new `GE_D236RAW` headless dump PROVES every room-background static vertex (all ~40 rooms sampled) is alpha=255 on load, ruling out "static ROM-authored low alpha" outright; a `vtxseg` extension to `GE_D236ALPHA` shows the tree class's vertices come from a raw/unsegmented pointer (not `SPSEGMENT_MODEL_VTX`/`BG_VTX`), consistent with a dynamically-built quad — but all 4 known `model.c` dynamic-quad builders (gunfire/shadow/rottex/tube) and propobj.c's screen-sprite builder were read in full and ruled out by content/addressing mismatch. The actual write site remains genuinely unidentified after a real search; next step is a stack-backtrace-style probe, not more signature-matching. Not v0.3.0-blocking, user is continuing anyway. |
|
||||
| D236 | **Surface 1: 2D billboard trees render as a wall/sheet of tree texture instead of discrete camera-facing tree sprites (M-106, user QA report).** — full `## D236` entry at file tail | OPEN — **pass 24 (2026-09-21, static) DECODED THE TREE TEXTURE** (new `tools_pc/texdecode.py` ports image.c's LOOKUP/HUFFLOOKUP/RLELOOKUP codecs). **`tex1198` LOD 0 = 64×17 RGBA5551, 44.2% opaque, and its 1-bit alpha renders as an unmistakable treeline silhouette** — pass 23's identification confirmed by decode, not inference. **Rare authored a six-level LOD pyramid that PRESERVES the silhouette and its alpha coverage (44.2% → 43.1% → 38.8%), each level a separately-authored image with its own compmethod.** Since `XLU_SURF` has no alpha test (pass 22), averaging that 0/1 alpha into fractional mip alpha would smear the minified treeline into exactly the reported band — so 'authored LOD chain replaced by driver-generated mips' is now the leading mechanism, with hard ground truth behind it. Corrects pass 23: only `tex1198` is a treeline (`tex1199` 89.4% opaque denser foliage; `tex1200`/`tex1201` 100% opaque detail textures sharing the class). Two decoder bugs found while porting (missing per-image byte-align, `image.c:1096`; a GUESSED rather than ported RLE fudge) — both produced confident wrong pictures. **Pass 25 (live) answered all three questions: (1) YES 64×17 RGBA16 intact, pool ≡ offline decode; (2) YES alpha survives (44.2% opaque); (3) NO — no code path uploads authored chains as GL mips.** New: `tex1198`–`1201` are **never drawn from an idle spawn** (zero RGBA16 imports in 4800 frames) — the spawn wall is painted by other classes (cf D276/D280); the drawn tree-class card is **`tex149` IA8 32×64 hard cutout, byte-exact import, no authored chain**. Mip mechanism located: `texGetDepthAndSize` sizes LOADBLOCKs from the header `lods` field even for single-image textures (live A/B: 2808 B vs 2048 B real data), and the port's RC2 clip + `glGenerateMipmap`/trilinear (observed live) box-filters hard 0/F alpha into fractional mips → XLU smear. Pass 26 fix directions in entry (cheapest: no mip gen for no-chain textures). |
|
||||
| D237 | **QoL ask (M-106, user QA): F10 options overlay — make it scrollable with the mouse wheel, expose more settings into it (e.g. screen-flash / no-hit-flash…** — full `## D237` entry at file tail | OPEN — not designed; port-only |
|
||||
| D238 | **QoL ask (M-106, user QA): mouse sensitivity rework toward normal PC-shooter feel.** — full `## D238` entry at file tail | OPEN — not designed; port-only (`port/src/input.c` + `config.c`) |
|
||||
| D239 | **QoL ask (M-106, user QA): click-to-lock vs always-grab parity.** — full `## D239` entry at file tail | CLOSED — executed in the v0.2.0 prep pass (port-only)… |
|
||||
@@ -2430,6 +2430,8 @@ Confidence overall: **medium** (triage solid, no runtime confirmation).
|
||||
|
||||
> **M-196 UPDATE (v0.4.0 Phase 1, 2026-09-21) — slot-overwrite hypothesis REFUTED; bug narrowed to the color-combiner/texture path.** New `GE_D75D="lo-hi"` probe in `port/fast3d/gfx_pc.cpp`: per-frame G_VTX/G_TRI token stream + per-triangle rejection/emission accounting (trivial-reject, backface-cull, NDC bbox, invisible-flag, vertex-color range, oml, depth z/w, combine mode, prim/env color). Front-end boot (no `-level`), logo window = frames ~244+ where a large model appears (nv=93 G_VTX batches, **nt=1018** tris/frame; the earlier frames 5–243 are the title-screen textured quads, nv=2/nt=12, which render fine at 8.36% coverage). **(1) Slot-overwrite is refuted.** The logo model's DL is **streamed**: `seq` = `VTTTT…VTTT…` (each G_VTX batch immediately followed by its own triangles), and `maxtri=15` — no triangle ever references a vertex index beyond 15, i.e. within a single 16-slot batch. M-195's "all batches write to slot 0, only the last survives" mechanism **cannot apply** to this model; there is no cross-batch clobbering. **(2) Rejection gates are not the cause.** Of the 1018 tris/frame: `rej_triv=0`, `rej_cull≈520` (a normal ~half for a closed mesh), **~500 reach OpenGL**; and all ~500 are on-screen (`emit_on≈500, emit_off=0`, NDC bbox ⊂ [−1,1]²). Not the invisible flag (`invis=0`). Vertex colors span `[0,255]` (not black). Depth `z/w ∈ [−1, 0.97]`, no NaN. **(3) Symptom quantified.** PPM pixel analysis of the logo window: **0–33 content px/frame** (a ~19×2px fragment right of centre) vs the title quads' 8.36% — this is M-195's "tiny orbiting fragment", now measured. **(4) Narrowed cause = texture-modulate material rendering black.** The model's combine mode is `0x009ffe4f19ffe4f1` (`is_2cyc=0`). A one-shot decode in `gfx_generate_cc` (new `D75DCC:` log, gated to this exact word) shows it parses to **rgb a=`G_CCMUX_TEXEL0`(1), c=`G_CCMUX_SHADE`(4)** — a texture-modulate material — and `gfx_generate_cc` sets `used_textures[0]=1`. `env=(0,0,0,0)`, `prim=(255,255,255,255)`. Intended output = TEXEL0 × SHADE; if TEXEL0 samples black at draw time (no/empty texture bound), the whole model is black-on-black → invisible except AA-edge fragments. **Anomaly to close next:** at emit time BOTH `gfx_rapi->shader_get_info(prg)` and `comb->used_textures[0]` read **0**, even though `gfx_generate_cc` set it to 1 for this exact combine_mode — so the TEXEL0 input is being lost between combiner build and draw (suspects: `shader_id0`→`gfx_lookup_or_create_shader_program`, the `prg[tm]` variant selection at gfx_pc.cpp:~2503, or the GL backend's sampler binding / texture upload for this model's tile). If the TEXEL0 sampler is dropped, `SHADER_TEXEL0` defaults to 0 → MODULATE → black. Port-layer fast3d only; no rule-2 sign-off needed. Probes (`GE_D75D`, `D75DCC`) registered in `GE-ENV-PROBES.md`; both env-gated/inert without the var.
|
||||
|
||||
> **M-197 UPDATE (v0.4.0 Phase 1, 2026-09-21) — ROOT CAUSE FOUND + FIXED: the D50 sidecar converter reserved and remapped the model's embedded texture-blob span but never copied the pixel bytes into the emitted image, so TEXEL0 sampled a zeroed buffer.** The M-196 "utex=0 anomaly" was first closed as a probe artifact: the frame-level `utex`/`cutex` fields were sampled on the *first* emitted triangle of the frame (a title quad, utex=0) while `comb` held the *last* triangle's mode; new logo-mode-specific fields (`lg_utex`/`lg_tex`, gated to combine word `0x009ffe4f19ffe4f1`) show both = 1 for every logo triangle — the combiner and GL shader correctly claim a texture. The real defect is upstream in the asset pipeline: the NINTENDO logo model (`PnintendologoZ`, loaded via `front.c` `init_menu00_nintendoscreen` → `load_object_fill_header(dst=ptr_logo_and_walletbond_DL, 0x3c000)`) carries its texture pixels **embedded in the model file** — GDL `G_SETTIMG` (GE opcode `0xFD`) slots with seg-5 (file-base-relative) targets. The N64 original has a 1024-byte white-wordmark IA8 blob at file offset `0x408`. The D50 offline converter (`tools_pc/d43_emit.py`) detected the blob span (`[0x408, 0x808)`), reserved it in the PC layout (new offset `0x6428`), and remapped all 23 SETTIMG slots to `w1=0x05006428` — **but the emit pass never copied `src[0x408:0x808]` into the output buffer**, so the sidecar's RZ image carried a zero-filled blob region. At runtime `seg_addr(0x05006428)` = file base `0x70157a98` (M-195's BaseAddr) + `0x6428` = **`0x7015dec0`** — exactly the all-zero texture buffer the `GE_TEXDUMP_ADDR` probe measured (both imports, dls=242 and dls=4444: `nonzero=0/1024`). TEXEL0 × SHADE over black pixels = the whole 1018-tri model renders black-on-black. **Three files are affected** (the only model files with seg-5 SETTIMG targets, found by scanning all 512): `PnintendologoZ` (`[0x408,0x808)`), `PgoldeneyelogoZ` (`[0x68,0xb30)`, 2 images), `PlegalpageZ` (`[0x90,0x2070)`, 5 images). **Fix (tooling-only, no `src/game` change, no rule-2 sign-off):** `d43_emit.py` now tracks `blob_regions` through layout, raw-copies each span in the emit pass (`buf[n:n+osz] = src[o:o+osz]` — the RDP reads these bytes as-is, no transform), and round-trip-validates the blob bytes so this cannot silently regress. Sidecars regenerated (`ALL CHECKS PASSED`, incl. the new blob check); offline re-decode confirms all 8 blob targets now carry real pixels (nintendologo: `ffffffff…` white wordmark, 713/1024 nonzero). **Live-verified:** bare front-end boot, `GE_PCDUMP="240-330:10"` — logo window now renders the animated 3D logo: frame 320 = **23,302 non-black px (7.59%)**, wide wordmark bbox `(94,190)-(542,290)`, vs M-196's 0–33 px; frames 340–560 show the logo rotating through its full animation at 3–10% coverage. No placed object offset falls inside any blob span (checked for all three files), so the old-space overlap between a PC-footprint node region and the nintendologo blob start is a bookkeeping artifact with no pointer ambiguity. **User-confirmed live (2026-09-21):** "the images and legal page and Nintendo logo are good now." **Remaining:** `pal-final`/`jpn-final` sidecars must be regenerated on machines holding those ROMs (`python tools_pc/d43_emit.py <region>`). Probes added this pass: `GE_TEXDUMP_ADDR=<hex>` (raw source-byte dump + stats for every import from one address, bypassing the 400-cap) and the D75D `lg_*` logo-mode fields; both env-gated.
|
||||
|
||||
**D75 Bug 2 — RUNTIME PROBE (session M-32b, `GE_D75=1` in `title.c`
|
||||
`sub_GAME_7F007F30`, kept env-gated).** Booted the bare front end
|
||||
(`GE_D75=1 GE_PCDUMP="1-1200:120"`, no `-level`) and captured through the
|
||||
@@ -10738,7 +10740,7 @@ D243 (cutscene shake/duplication, its own CONSOLIDATED NEXT STEPS block appears
|
||||
|
||||
**Surface 1: 2D billboard trees render as a wall/sheet of tree texture instead of discrete camera-facing tree sprites (M-106, user QA report).** Eight passes so far (D265, D266–D270, D271, D272, D273, D274, D275–D276). Ruled out: IA4/IA8 decode (as originally scoped), the blend pipeline, fog combine (as originally scoped), portal-culling over-draw (D271, real bug, fixed, not the cause), converter/UV-conversion bugs (D272, code-read clean), the CI palette bit-layout (D273/D274, tested twice against real content), "mips absent" (D274, confirmed present/enabled), "identical UV window" *for the CI8 classes* (D272). **Current leading picture (D276, M-134): the actual visible weave at confirmed weave pixels is painted by a different, previously-dismissed class — an IA8 blocky-noise texture (`oml=0xc81049d8`/`fmt=3`) sampled with a fixed, byte-identical 2×-wrapped UV window and low fog, reproducing D265's original "same window" observation exactly (D269's CI8-classes framing had reframed away from that lead prematurely).** A third, unattributed CI8 class (`0x0c184b50`) may be the reference's actual discrete-tree layer and shows an erratic z/w suggesting its own occlusion/billboard bug — D236 may split into two root causes. **Ninth pass (D280, M-135): fog math verified N64-faithful (±2 truncation over 2505 verts), the IA8 noise class visually confirmed as the weave's painter (HF 14.8→2.9 when collapsed), `0xc8102078` ruled out (terrain, 1.4M tris), and the depth-race read inverted — `0x0c184b50` tree cards are IN FRONT of the noise quad (z/w smaller=closer) yet still don't visibly win; under deterministic `-level_36` boot they barely instantiate at all (~4 tris vs 720 in a free run). No quick win: remaining work is CPU-side instantiation/room-streaming, not a port fix.** **Tenth pass (M-155): that CPU-side-instantiation framing itself is REFUTED — three independent deterministic re-runs of the same `-level_36` boot (one with sustained camera rotation) proved the class's per-frame count is byte-identical, not run-to-run dependent; the earlier "~4 vs 720" comparison was an outdoor-vs-indoor room mismatch (the boot's scripted intro walks the camera out of the tree-bearing room at a fixed frame), not nondeterminism. No fix; the original wall-vs-discrete-sprites symptom is unresolved and untouched by this pass.** Distinct from D176(b)'s painted tree-line backdrop. Logged in `GRAPHICS-BACKLOG.md`.
|
||||
|
||||
**Status (from §F table):** OPEN — tenth pass refuted the "instantiation is nondeterministic" theory (ordinary room-based occlusion, not a bug); original wall-vs-sprites symptom still unresolved; see D276/D280/M-155
|
||||
**Status (from §F table):** OPEN — **pass 24 (2026-09-21, static) DECODED THE TREE TEXTURE** (new `tools_pc/texdecode.py` ports image.c's LOOKUP/HUFFLOOKUP/RLELOOKUP codecs). **`tex1198` LOD 0 = 64×17 RGBA5551, 44.2% opaque, and its 1-bit alpha renders as an unmistakable treeline silhouette** — pass 23's identification confirmed by decode, not inference. **Rare authored a six-level LOD pyramid that PRESERVES the silhouette and its alpha coverage (44.2% → 43.1% → 38.8%), each level a separately-authored image with its own compmethod.** Since `XLU_SURF` has no alpha test (pass 22), averaging that 0/1 alpha into fractional mip alpha would smear the minified treeline into exactly the reported band — so 'authored LOD chain replaced by driver-generated mips' is now the leading mechanism, with hard ground truth behind it. Corrects pass 23: only `tex1198` is a treeline (`tex1199` 89.4% opaque denser foliage; `tex1200`/`tex1201` 100% opaque detail textures sharing the class). Two decoder bugs found while porting (missing per-image byte-align, `image.c:1096`; a GUESSED rather than ported RLE fudge) — both produced confident wrong pictures. Next: one `-level_36` run filtered on **`fmt=0 siz=2`**, diffing the live import against this offline reference on (1) 64×17 RGBA5551 dims/format, (2) ~44% alpha transparency, (3) whether all six authored LODs reach GL or only LOD 0.
|
||||
|
||||
**Next pass framing (post rule-2-clarification):**
|
||||
|
||||
@@ -10983,6 +10985,322 @@ User played live with `GE_D236ORDER=1` set, walked toward Surface1's tree line,
|
||||
|
||||
**Not v0.3.0-blocking, user is continuing anyway.** `GE_D236RAW` and the `vtxseg` extension to `GE_D236ALPHA` are both in tree, env-gated, zero cost unset. Raw headless dump preserved at `scratch/d236_raw_headless.log`.
|
||||
|
||||
**Pass 17 (2026-09-21, fresh session) — WRITER IDENTIFIED: `doshadow`. Plus a false alarm resolved.**
|
||||
|
||||
1. **Vertex struct is 16 bytes, not 32**: GE's RSP uses a custom vertex format `{s16 ob[3]; u16 flag; s16 tc[2]; cn[4]}` (alpha at byte 0xF) — no screen-space sx/sy/st/ct. `vtxallocator(n)` = `dynAllocate(n<<4)` = n×16B; the observed +0x40 arena increments are doshadow/dogfnegx's 4-vertex (64B) allocations, not 2×32B Vtx.
|
||||
2. **Decisive probe**: env-gated backtrace at allocation time (`GE_D236BT`, `get_ptr_allocated_block_for_vertices` in `chr.c`, calling a new C-linkage `port_d236bt_dump()` extracted from the RSP-side unwind logic in `gfx_pc.cpp`). All allocator hits resolve to **`doshadow`** (`model.c:5397`) via `drawjointlist → chrRenderProp → chrpropRender → chrpropsRenderPass → lvlRender`. Zero dotube activity. First allocation's physical address exactly matches the RSP-side rawvtx capture. Pass 16's exclusion of doshadow was based on a wrong claim ("dotube/doshadow use segment-tagged addressing") — only `dorottex` tags G_VTX; both use raw pointers.
|
||||
3. **doshadow alpha mechanism**: `cn[3] = (envcolour.a * D_800363F0)/255` for CHR/SMOKE props, else `= D_800363F0` — per-frame global shadow intensity, explains the varying low alphas (observed 1→2→12→13→14; probe saw 13 and 80 alternating). All 4 vertices are copies of a constant template (`D_800363F8`). Textured quad via `shadow->image` → `texSelect`.
|
||||
4. **FALSE ALARM (pass 17c) — the d43_emit.py op-13 "ShadowRecord bug" does not exist.** A ROM scan of all 42 op-13 records (all in C*Z character files, none in P*Z props) initially suggested the converter writes HeaderNode to r+24 / Scale to r+32 / omits BaseAddr — misaligned vs the C struct's *commented* offsets (image@0x10, HeaderNode@0x14, Scale@0x18, BaseAddr@0x1C). But a runtime dump of the live record (`GE_D236BT` probe in `doshadow`) shows **HeaderNode@0x14 = 0 and the valid pointer at 0x18** — because on PC (64-bit) `image` widens to 8 bytes, shifting HeaderNode→0x18, Scale→0x20, BaseAddr→0x28. The converter's r+16/r+24/r+32 layout **matches the PC struct exactly**; the C comments describe the N64 32-bit layout (the D52/A1 pointer-widening class). Live values all sane: image→valid tconfig (32×32, fmt=0), HeaderNode valid, height≈110, size≈(389,413), BaseAddr set by the loader's promotion pass (`modelPromoteNodeOffsetsToPointers` SHADOW case overwrites it with fileramaddr anyway). **Do not "fix" d43_emit.py op-13.**
|
||||
5. **Conclusion so far**: the faint textured sheets ARE `doshadow`'s drop-shadow quads — large (≈390×413 world units) flat ground quads sampling a tiny 32×32 texture at low per-frame alpha. N64 draws the identical quads (same code, same data). **The user-visible bug is therefore almost certainly that the trees' actual sprite geometry is missing/broken on PC**, leaving only the shadow sheets visible — not a shadow bug.
|
||||
|
||||
**Next steps (fresh session):** (a) look at the dumped frames (`ppm/frame_0003xx.ppm`, converted PNGs in /tmp) to characterize what's actually on screen; (b) identify which prop/character model owns the shadow node (node 0x7020ef44, file image base 0x7020eda0 — all 42 op-13 records are in C*Z character files, so check which characters are present on Surface 1 and whether *their* main geometry renders); (c) if the tree sprites are a separate prop class, verify that file's GDL/vertex emission through `d43_emit.py` (D75 blob-bug class is the standing suspect for missing geometry). **TEMP probes in tree (env-gated, zero cost unset):** `GE_D236BT` backtrace probe in `chr.c` + live-record dump in `model.c:doshadow` (writes `d236sh.bin` to CWD on first hit — delete both when D236 concludes), `port_d236bt_dump()` helper in `gfx_pc.cpp`. Untracked scan script: `tools_pc/scan_op13.py`.
|
||||
|
||||
**Pass 18 (2026-09-21, static ROM analysis session) — the tree-class RM word is a runtime LUT rewrite; exact source pair identified.**
|
||||
|
||||
1. **All tree/plant prop files carry zero renderable geometry — confirmed with the real node parser, not just an opcode histogram.** `PpalmtreeZ`, `PpalmZ`, `Pjungle3_treeZ`, `Pjungle5_treeZ`, `Pplant1`–`Pplant11Z`, `Pgas_plant_*Z`: node set is only {Group(2), BBox(10), DLCollision(24)} — **no op-4 DisplayListRecord anywhere**. Same on N64. The visible tree sprites are therefore drawn by the room background (`bg_sevx_all_p.seg`, ROM 0x66F700, 116176 B; Surface1=level_36, Surface2=level_43), not by prop models.
|
||||
2. **The measured tree-class RM word `0x0c184b50` exists in no ROM data.** Scanned all 25 non-empty bg files (decompressed RZ GDLs) and all 512 model GDLs: zero hits. The only `0x0c18xxxx` word in ROM is `0x0c182048` (menu logo bg). It is generated at runtime.
|
||||
3. **Exact source found — `DL_LUT_CLOUD`.** Compiled GE's LUT pairs (`scratch/d236/lutdump.c`, output in `scratch/d236/lutvals.txt`): CLOUD = from `G_RM_PASS|G_RM_AA_ZB_XLU_SURF2` = **`0x0c1849d8`** → to `G_RM_PASS|G_RM_ZB_CLD_SURF2` = **`0x0c184b50`** — exact match with the measured tree-class word. The "from" word is one of the six bg RM words present in ROM (×24 in sevx). `G_RM_PASS = 0x0c080000` explains the high bits: `0x0c08 | 0x0010 (ZB_CLD_SURF2 hi) = 0x0c18`. So: room GDLs contain `SETOTHERMODE_L 0x0c1849d8`, and a runtime `bgApplyDynamicCCRMLUT` pass rewrites them to `0x0c184b50` before the RSP sees them. Note `DL_LUT_BILLBOARD` shares the same "from" word (`0x0c1849d8`) but maps to `0x0c193078` (TEX_EDGE2) — which LUT wins depends on the applied index.
|
||||
4. **Pass 17's doshadow finding stands but is a different sheet.** doshadow emits `G_RM_ZB_CLD_SURF|G_RM_ZB_CLD_SURF2 = 0x00504b50` (no PASS bits) — a *different* word from the measured tree class. The faint character drop-shadow quads (doshadow, 32×32 shadow texture) and the tree-texture billboards (bg CLOUD-rewritten quads) are two separate draw sources; the user-visible "wall of tree texture" is the latter.
|
||||
5. **N64 cutout mechanism (hypothesis to verify):** `0x0c184b50` = ZMODE_XLU + FORCE_BL + CVG_WRAP/FULL + IM_RD, alphacompare bits 0–2 = 000 (NONE), no CVG_X_ALPHA. No alpha *test* — the silhouette must come from plain alpha **blending** of the IA texture (`out = src*A + dst*(1-A)`; transparent texels have A=0). The port's `use_alpha` gate in `gfx_pc.cpp` (~line 2386: requires `(oml & (3<<20)) == G_BL_CLR_MEM<<20 && (oml & (3<<16)) == G_BL_1MA<<16`) must be checked against this word's blender bits (bits 16–17 = 00, bits 20–21 = 01) — if the gate rejects it, fast3d draws the quad opaque → exactly the reported "wall/sheet" symptom.
|
||||
|
||||
**Next steps (fresh session):** (a) read `bgApplyDynamicCCRMLUT`'s implementation + the `enum CCRMLUT` in `src/game/bg.h`; map the literal indices 1/5/6/7 used at `bg.c:2797–2819` and `CCRMLUT_WALLETBOND` (`front.c:2177`) to tables, and find which index applies `DL_LUT_CLOUD` plus **all** call sites of that index (the room-load path uses 1/5/6/7 only — the CLOUD application may live in a second call path); (b) verify the port's `use_alpha`/blend handling in `port/fast3d/gfx_pc.cpp` for RM `0x0c184b50` (it is already special-cased at ~lines 2166/2217/2259/2279/2356 — D236 probes + zfix; review what they do); (c) if the gate rejects this RM, fix in `port/fast3d` (port layer — rule 2 allows it); (d) when concluded, remove TEMP probes listed above. Scratch tools: `scratch/d236/{lutdump.c,lutbuild.bat,lutvals.txt,scan_bg_rm.py,rmvals.txt}`, `tools_pc/scan_op13.py`.
|
||||
|
||||
**Pass 19 (2026-09-22, static session) — both pass-18 leads REFUTED; complete runtime-emitter map for `0x0c184b50` established; the discrete-sprite mechanism identified as `dogfnegx` (op-12 GUNFIRE node).**
|
||||
|
||||
1. **`CCRMLUT_CLOUD` (index 4) has ZERO call sites — pass 18's "runtime LUT rewrite" mechanism is dead.** Complete call-site map for `bgApplyDynamicCCRMLUT` (`src/game/bgapply.c:77`): `bg.c:2797/2804/2812/2819` (room load; literal indices 1=`CCRMLUT_PRIMARY_ADDFOG`, 5=`CCRMLUT_SECONDARY_ADDFOG` when `g_FogSkyIsEnabled`, else 6=`CCRMLUT_PRIMARY`, 7=`CCRMLUT_SECONDARY`) and `front.c:2177` (`CCRMLUT_WALLETBOND`). `CCRMLUT_BILLBOARD` (2), `CCRMLUT_WATER` (3) and `CCRMLUT_CLOUD` (4) are defined (`bondconstants.h:2931-2942`) but never applied anywhere in the compiled set — the `DL_LUT_CLOUD` table itself exists (`bg.c:2174`, comment: "Transparent to Cloud (Saves AA - Stops Jaggies from appearing behind BillBoard)") mapping exactly `0x0c1849d8 → 0x0c184b50`, but its enum index is never passed to `bgApplyDynamicCCRMLUT`. Corroborating: the ADDFOG LUTs that ARE applied map the same "from" word `0x0c1849d8` → `G_RM_FOG_SHADE_A|G_RM_AA_ZB_XLU_SURF2` = **`0xc81049d8`** (`bg.c:2039-2040`, `2061-2062`) — exactly D276's confirmed noise-weave class. So the sevx room-bg quads that carry `0x0c1849d8` in ROM end up as `0xc81049d8` at runtime (fog enabled), NOT as tree cards.
|
||||
2. **The only code that emits `0x0c184b50` is `texSetRenderMode` mode 4 ("cloud"), 2-cycle, zbuffer-on** (`othermodemicrocode.c:300`: `gDPSetRenderMode(G_RM_PASS, G_RM_ZB_CLD_SURF2)` = `0x0c184b50` exactly), reached from its callers via `sub_GAME_7F073038` (`model.c:4940-4942`) → `texSelect(..., arg2=4, ...)` → `texSetRenderMode`. Mode-4 `texSelect` call sites: **`model.c:5369` `dogfnegx`** (op-12 `MODELNODE_OPCODE_GUNFIRE`), **`model.c:5545` `doshadow`** (op-13 `MODELNODE_OPCODE_SHADOW`), `glass2.c:732` bullet sparks, `gunfire.c:1512/1518/1524` flares, `explosion.c:1890` smoke (zbuf=1), `front.c` menu/crosshair (zbuf=0 → emits `0x0c184340` instead). Note the 1-cycle variant (emitted when `tconfig==NULL` or `tconfig->level==0`, `othermodemicrocode.c:415`) emits `G_RM_ZB_CLD_SURF|G_RM_ZB_CLD_SURF2` = `0x00504b50` with zbuffer on (`0x00504340` without) — so pass 18's "doshadow emits `0x00504b50`" holds only for the 1-cycle case; with a mipmapped texture + zbuffer on, **both doshadow and dogfnegx emit exactly `0x0c184b50`** (2-cycle combine: `G_CC_TRILERP`/`G_CC_MODULATEIA2` for RGBA/IA → A = texel.a × shade.a). Pass 17's backtrace attributed the tree class's vertex allocations to `doshadow`, but `dogfnegx` is a co-emitter through the identical `drawjointlist → chrRenderProp` path and the same `vtxallocator(4)` + raw-pointer pattern — the attribution may have resolved only the more frequent of the two.
|
||||
3. **`dogfnegx` IS the discrete camera-facing sprite mechanism** (`model.c:5225-5380`): it computes the view direction from `rodata->Offset` through the node's model matrix, builds a rotation aligning the quad to face the camera, applies per-frame random scale (0.75–1.25) and random UV rotation around the texture centre, and draws a `rodata->Size`-sized textured quad with mode-4 cloud RM. Gated on `(renderdata->flags & 2) && rwdata->Gunfire.visible`. `ModelRoData_GunfireRecord` (`bondtypes.h:1151`): Offset@0x0, Size@0xC, Image@0x18, Scale@0x1C, RwDataIndex@0x20. This is almost certainly what N64 shows as the "discrete tree sprites".
|
||||
4. **The port's `use_alpha` gate ACCEPTS `0x0c184b50` — pass 18's prime suspect is refuted by static decode.** Against this repo's gbi.h blender layout (`include/PR/gbi.h:720-723`: `GBL_c2` puts c2-modifier at bits 16-17, c2-secondary-colour at bits 20-21 of other_mode_l): the gate (`gfx_pc.cpp` ~line 2386) requires bits 16-17 == `G_BL_1MA`(0) and bits 20-21 == `G_BL_CLR_MEM`(1); the word has byte2=`0x18` → bits 16-17=0 ✓, bits 20-21=1 ✓ → `use_alpha=true`. Full decode: ALPHACOMPARE=NONE, no CVG_X_ALPHA/ALPHA_CVG_SEL, FORCE_BL set, c1=(IN, A=0), c2=(IN, A_IN over MEM, 1MA) = standard src-over alpha blend with A from the combiner (`texSelect` mode-4 2-cycle sets `G_CC_TRILERP`/`G_CC_MODULATEIA2` for IA → A = texel.a × shade.a). No port-side opaque-draw gate explains the wall.
|
||||
5. **Implication / reframed picture:** with the tree class alpha-blended correctly in the port, the consistent story across D276 (visible weave painted by the `0xc81049d8` room-bg noise class), D280 (tree cards in front yet not visibly winning) and pass 15 (tree-class vertex alpha capped ≤36/255 ≈ 14%, bimodal 25/255) is that the tree cards blend at ~10% opacity over the opaque noise wall — nearly invisible, exactly "the wall shows through". The remaining open questions are (a) whether that low alpha is ROM-faithful (doshadow's `cn[3] = envcolour.a × D_800363F0/255`, `D_800363F0` set per-chr from shade alpha at `chr.c:3319`, init 0x50; dogfnegx uses template `D_800363E0`) or a port-fed wrong input, and (b) whether the discrete billboards (`dogfnegx`) actually instantiate on PC at all — its gate inputs (`renderdata->flags & 2`, `rwdata->Gunfire.visible`) are unverified.
|
||||
|
||||
**Next steps (fresh session):** (a) ROM-holding session: run new `tools_pc/scan_op12.py` (added this pass; counts op-12/op-13 nodes per model file, dumps GunfireRecord Offset/Size/Image) to identify which models own Surface 1's tree billboards/shadows and cross-ref against pass 17's D236SH live-record data (32×32 fmt=0 shadow texture); (b) runtime: `GE_D236ALPHA` + `GE_D236RM` on `-level_36`, correlate the tree class's per-vertex alpha with `D_800363F0`/envcolour.a, and compare against N64 reference footage (faint vs solid sprites decides faithful-faint vs port bug); (c) verify `dogfnegx`'s gate inputs on PC (`renderdata->flags & 2`, `rwdata->Gunfire.visible`) — if billboards never fire, the discrete sprites are missing entirely (rule-2 bucket (a)/(b) question); (d) on conclusion remove TEMP probes: `GE_D236RM/ORDER/ALPHA/ZFIX/BT` in `gfx_pc.cpp`, `GE_D236BT` + D236SH dump in `model.c:doshadow`, `port_d236bt_dump()`, `tools_pc/scan_op1{2,3}.py`. Scratch tools: `scratch/d236/{lutdump.c,lutbuild.bat,lutvals.txt,scan_bg_rm.py,rmvals.txt,rmdump.c,rmbuild.bat,rmnames.txt}`.
|
||||
|
||||
**Pass 20 (2026-09-22, ROM-holding session) — pass-19 step (a) executed: full 512-file `scan_op12.py` run; the `dogfnegx`=trees identification is REFUTED by the asset data.**
|
||||
|
||||
1. **Verified scan result** (re-ran `tools_pc/scan_op12.py ntsc-final` this pass). *Note: the script's final line "total files with op12: 62" is a reporting artifact — that counter accumulates every file with EITHER opcode, so it equals the op13 file count; the real per-opcode totals are below.* Across all 512 converted model files:
|
||||
- **op-12 (`MODELNODE_OPCODE_GUNFIRE` / `dogfnegx`) — 21 nodes total, ALL in weapon/gun/tank models.** 16 one-node weapon files (`PchrautoshotZ`, `Pchrfnp90Z`, `PchrgoldenZ`, `PchrgrenadelaunchZ`, `PchrkalashZ`, `Pchrm16Z`, `Pchrmp5kZ`, `Pchrmp5ksilZ`, `PchrrugerZ`, `PchrshotgunZ`, `PchrskorpionZ`, `PchrspectreZ`, `Pchrtt33Z`, `PchruziZ`, `PchrwppkZ`, `PchrwppksilZ`) + `PgroundgunZ`(2), `Pgun_runway1Z`(1), `ProofgunZ`(1), `PtankZ`(1). The dumped records confirm muzzle-flash semantics: barrel-relative offsets (e.g. shotgun `(-375,0,16)`), modest sizes, small embedded-image indices (`img` 0x18–0x6c, `vma?False`). `PtankZ`'s single node is a large smoke/flash quad (`(0,5,0)`, size 2143×1026).
|
||||
- **op-13 (`MODELNODE_OPCODE_SHADOW` / `doshadow`) — 42 nodes total, ALL in C*Z character files** (one per character; matches pass 17's "42 op-13 records, all in C*Z").
|
||||
- **No tree/plant model carries either node.** The 16 foliage models present (`Pjungle3_treeZ`, `Pjungle5_treeZ`, `PpalmZ`, `PpalmtreeZ`, `Pplant1Z/2Z/2bZ/3Z/11Z`, `Pgas_plant_*Z`, `Pgasplant_clear_doorZ`) contain **only** `{GROUP(2), BBOX(10), DLCOLLISION(24)}` nodes — static geometry + collision, zero billboard/shadow nodes.
|
||||
|
||||
2. **Refutation of pass 19's central identification.** Pass 19 concluded "`dogfnegx` (op-12 GUNFIRE) is the discrete camera-facing billboard [that draws the trees]" and that the `0x0c184b50` class is emitted by dogfnegx/doshadow. But `dogfnegx` has **zero tree instances** in the entire asset set — every one of its 21 nodes is a weapon muzzle flash or tank effect. So the `0x0c184b50` class that passes 17/19 chased (pass 17's backtrace attributed its vertex allocations to `doshadow`) is, on this level set, most plausibly **character drop-shadows + weapon muzzle flashes**, *not* the trees. Passes 17–19 appear to have been instrumenting the wrong draw class for the reported symptom.
|
||||
|
||||
3. **Redirection: the visible trees are static geometry, not dynamic billboards.** Given (1), the Surface 1 trees must be drawn from static data — either (a) the foliage PROP models (`PpalmZ`/`Pjungle3_treeZ`/etc., which DO load and contain GROUP/BBOX/DLCOLLISION nodes) or (b) the level background file `bg_sevx_all_p.bin`. Both are static; neither uses the dogfnegx/doshadow path. **Honest caveat — not yet disambiguated:** my offline decode of `bg_sevx_all_p`'s room secondary-DL streams found billboard-shaped cards (G_VTX+G_TRI4 with a 0xC0/G_SETTEX texture-select), but their OML words (`0x0c1849d8`/`0x0c184dd8`) do *not* match the runtime `0x0c184b50`, and pass 19 established those exact bg quads become the **noise wall** (`0xc81049d8`) at runtime (fog-enabled ADDFOG LUT). So it is NOT yet confirmed whether the visible tree sprites come from the foliage prop models or the bg file — that needs a runtime attribution.
|
||||
|
||||
4. **Next step, concrete (supersedes pass 19's b/c):** a single runtime probe that attributes the on-screen tree-region triangles to their draw source — tag `gfx_sp_tri*` triangles by whether their vertices came from a **model-node GDL** (prop/model path) vs a **bg room DL** (`g_BgRoomInfo[].ptr_*_expanded_mapping_info`, SPSEGMENT_BG_VTX=14 / the bg-widened arena), and for the visible tree pixels log the active RM + texture/tile. That one capture decides (a)-vs-(b) and ends the "which class is actually the trees" question 19 passes have been circling. If it's the foliage prop models, the standing D75 blob-conversion suspect (missing/mis-converted mesh or texture in `d43_emit.py`) becomes the prime fix target; if it's the bg file, the D85/D276 bg-DL + noise-wall thread resumes.
|
||||
|
||||
**TEMP probes still in tree (env-gated, zero cost unset) — remove when D236 concludes:** `GE_D236RM/ORDER/ALPHA/ZFIX/BT` (`gfx_pc.cpp`), `GE_D236BT` backtrace + D236SH dump (`model.c:doshadow`, writes `d236sh.bin`), `port_d236bt_dump()` helper, `tools_pc/scan_op1{2,3}.py`.
|
||||
|
||||
|
||||
**Pass 21 (2026-09-21, static session) — pass-20's step (a)-vs-(b) ANSWERED STATICALLY, no runtime probe needed: the Surface 1 trees cannot be prop models, because the level places none. Prop/`d43_emit.py` conversion is retired as a D236 fix target; the bg-file branch is the only one left.**
|
||||
|
||||
Pass 20 closed by proposing a runtime draw-source attribution probe (tag `gfx_sp_tri*` by model-node GDL vs bg room DL) to decide whether the visible trees are (a) foliage prop models or (b) static `bg_sevx_all_p` geometry. That probe is **not necessary** — the setup data answers it, and two independent static lines of evidence agree.
|
||||
|
||||
1. **No Severnaya-type tree model exists in the asset set at all.** All 16 foliage models in `data/pcmodels-ntsc-final/manifest.csv` are jungle-themed (`Pjungle3_treeZ`, `Pjungle5_treeZ`, `PpalmZ`, `PpalmtreeZ`, `Pplant1Z/2Z/2bZ/3Z/11Z`, `Pgas_plant_*Z`), and the `PROP` enum in `src/bondconstants.h` describes each one as "Jungle …" (`PROP_JUNGLE3_TREE` = "Jungle Large Tree", `PROP_PALM` = "Jungle Palm Tree", `PROP_PLANT2B` = "Jungle Plant, Low Shrub", …). There is no pine/fir/snow-tree prop anywhere in the 512-model set.
|
||||
|
||||
2. **Surface 1's setup places zero foliage props — and neither does any other setup in the repo.** New tool `tools_pc/scan_setup_props.py` parses the `PROP` enum out of `src/bondconstants.h` and the object records out of `assets/obseg/setup/Usetup*Z.c`, then reports which props each setup instantiates. For `UsetupsevxZ.c` (Surface 1 — `bg.c:201` maps `LEVELID_SURFACE` → `bg/bg_sevx_all_p.seg` + this setup) it matches **219 object records**, of which **all 83 `StandardProp` records are individually accounted for** (`PROP_WOOD_SM_CRATE4`×31, `PROP_BOOK1`×16, `PROP_OIL_DRUM7`×12, `PROP_WOODEN_TABLE1`×4, `PROP_BOOKSHELF1`×4, `PROP_PADLOCK`×4, `PROP_DESK_LAMP2`×2, `PROP_METAL_CHAIR1`×2, `PROP_JERRY_CAN1`×2, plus one each of `PROP_MOTORBIKE`, `PROP_CARBMW`, `PROP_STOOL1`, `PROP_HATPEAKED`, `PROP_KEYBOARD1`, `PROP_CHRMAP`) — **not one foliage prop**. Same result for all 14 setups present in the repo (`arch ark azt cave control cryp dam depo pete run sevbunker sevb sevxb sevx`), including Dam/Runway/Streets, which also have visible outdoor greenery. GE's outdoor trees are never props.
|
||||
|
||||
*Positive control (the zero is meaningful).* A "0 foliage" count is only evidence if the detector can fire. The tool now self-tests and prints it: `PROP enum: 341 entries; foliage props recognised (9): PROP_JUNGLE3_TREE, PROP_JUNGLE5_TREE, PROP_PALM, PROP_PALMTREE, PROP_PLANT1, PROP_PLANT11, PROP_PLANT2, PROP_PLANT2B, PROP_PLANT3`. The matcher resolves every foliage name in the enum and still reports none placed. The record parser is independently sanity-checked by the distinctive props it *does* find in Surface 1 — motorbike, safe + safe door, the map objective item, the Yale and bolt keys, `PROP_HATCHSEVX`, `PROP_SEVDISH` — all of which really are in that level.
|
||||
|
||||
*Tooling bug caught during this pass, worth recording:* the first version of `load_prop_enum()` located the enum with `l.strip().startswith("typedef enum PROP")`, which matches `typedef enum PROPFLAG` (`bondconstants.h:294`) hundreds of lines earlier. That yields an enum containing **no `PROP_*` members at all**, so every `obj` resolved to `?` and every foliage count was a meaningless 0. The run *looked* exactly like a clean negative result. Fixed to an exact `l.strip() == "typedef enum PROP"` match, and the self-test above exists specifically so this failure mode cannot masquerade as a finding again. This is the D219 probe lesson in a different costume (**a diagnostic must be shown capable of producing the positive before its negative is trusted**).
|
||||
|
||||
3. **What this settles.** Pass 20's alternative (a) is dead: **the visible Surface 1 trees are static background geometry in `bg_sevx_all_p.seg`**, reached through `bgLoadRoomPrimaryGdl` / `bgLoadRoomSecondaryGdl`. The standing D75 blob-conversion suspect (`tools_pc/d43_emit.py` mis-converting a prop mesh or texture) is therefore **not** a D236 fix target and should not be pursued for this bug. This independently corroborates D280's ninth-pass reframing ("the remaining work is CPU-side instantiation/room-streaming, not a port fix") and explains why 19 passes of model-node instrumentation (op-12/op-13, `dogfnegx`/`doshadow`) found nothing tree-shaped: **there was never a tree model to find.**
|
||||
|
||||
4. **Render-mode path re-checked while here (no bug found; recorded so pass 22 does not re-derive it).** The bg-authored foliage/billboard cards carry `G_RM_AA_ZB_XLU_SURF` (`0x…49d8`), and `DL_LUT_BILLBOARD` (`bg.c:2151`, `CCRMLUT_BILLBOARD` = index 2) exists precisely to rewrite `XLU_SURF` → `G_RM_AA_ZB_TEX_EDGE` ("Transparent 1Cycle to BillBoard 1Cycle") — the alpha-cutout mode that would give discrete silhouettes instead of a blended sheet. **But index 2 is never applied**, on N64 or PC: the only call sites are `bg.c:2797/2804/2812/2819` (room load, literals 1/5/6/7 = `PRIMARY_ADDFOG`/`SECONDARY_ADDFOG`/`PRIMARY`/`SECONDARY` by `g_FogSkyIsEnabled`) and `front.c:2177` (`WALLETBOND`). So `BILLBOARD`, `WATER` and `CLOUD` are all dead LUTs in the original game too — this is **not** the PC/N64 divergence, and "apply the BILLBOARD LUT" is not a legitimate fix (it would be a game-logic change, rule 2).
|
||||
|
||||
Also checked and **cleared**: `bgApplyDynamicCCRMLUT`'s scan bound. `Gfx` is 16 bytes on PC (`include/PR/gbi.h`: `Gwords { uintptr_t w0, w1; }`), and `end` is computed as `ptr + usize_primary_DL_binary` in *bytes* — but `usize_*` is assigned from `expanded_size` **after** `bgWidenRoomGdl()` has already rewritten it to the widened PC byte count (`bg.c:2526`/`2559`), so the scan covers the whole room DL, not half of it. No 32→64-bit truncation here. The LUT tables themselves are layout-safe too: their scalar initializers (`DL_LUT_WATER`'s raw `0xB900031D, 0x00552078, …`) brace-elide into `Gfx.words.w0/w1` and zero-extend correctly at either width.
|
||||
|
||||
5. **Next step for pass 22 (bg branch only).** The question is now narrow: *within `bg_sevx_all_p`'s room display lists, which draw class carries the tree cards, and why does it not reach the screen?* D280 established that the `0x0c184b50` class sits in front of the noise quad, has CPU-baked bimodal fog (25/255) and "barely draws at all" under a deterministic `-level_36` boot — but pass 20 proved that word is only ever emitted at runtime by `texSetRenderMode` mode 4 (shadows/muzzle flashes/sparks), so it is **not** the bg tree class either. The remaining candidates are the bg-authored classes D269/D272 catalogued (`oml=0xc4112078` OPA and `0xc41049d8` XLU, both CI8) versus D276's `0xc81049d8` IA8 noise painter. Concretely: dump the Surface 1 room GDLs with the existing `GE_D85DUMP` probe (no new code) and census the distinct `oml`/texture pairs per room, then match against a room the user can stand in front of trees in. That is a static-plus-one-run step, not a new instrumentation build.
|
||||
|
||||
**Tooling added this pass:** `tools_pc/scan_setup_props.py` (read-only, no ROM needed, runs against the repo's own sources). Unlike the `GE_D236*` env probes it costs nothing at runtime and is generally useful — keep it rather than removing it when D236 concludes.
|
||||
|
||||
|
||||
**Pass 22 (2026-09-21, static session) — pass-21's owed step done OFFLINE, no run: complete render-mode census of Surface 1's bg room DLs, straight out of the ROM. D276's wall painter `0xc81049d8` is now tied to its authored source (bg *secondary* DL, `G_RM_AA_ZB_XLU_SURF`, 24 of 42 rooms). No bg file in any level authors the TEX_EDGE "billboard cut-out" class, so the tree cards are plain alpha-blended quads and the remaining suspicion falls on their texture's alpha, not on geometry, culling, fog or billboard orientation.**
|
||||
|
||||
Pass 21 proposed dumping the Surface 1 room GDLs with `GE_D85DUMP` and censusing the `oml`/texture classes — a build plus one run. That turned out to be unnecessary too: the bg `.seg` files are stored **uncompressed** in the ROM (they have to be — `obLoadBGFileBytesAtOffset` random-accesses them by byte offset), so the whole structure is readable offline.
|
||||
|
||||
**Two new tools, both read-only, no ROM-dependent build, no C compiler:**
|
||||
|
||||
* `tools_pc/bg_gdl_census.py` — decodes `bg_<level>_all_p` out of the ROM, walks every room's primary and secondary GDL, and censuses the distinct render modes per room. It reproduces `bg.c`'s own structure exactly: file word[1] → the 24-byte `bg_room_data` array (`bg.c:865`); a room's compressed blob size = distance to the next non-zero mapping-bin offset with primary and secondary treated as one interleaved stream (`bg.c:944-990` + `getPri/SecMappingBinCount`); each blob raw-deflate behind its own 2-byte header (`bgDecompress` → `decompressdata`, which is just "skip 2 bytes, inflate").
|
||||
*Validation:* **every** blob in every level tested inflates cleanly with zero failures. Since the sizes are derived, not stored, a single off-by-one in that derivation would corrupt the stream and fail inflate — so a clean sweep is a real check on the derivation, not a vacuous one.
|
||||
* `tools_pc/rendermode.py` — evaluates the `G_RM_*`/`RM_*(clk)`/`GBL_c1`/`GBL_c2` macros out of `include/PR/gbi.h` and the `DL_LUT_*` tables out of `src/game/bg.c` in pure Python (93 `G_RM_*` composites resolve). It decodes any `oml` word into flags + blender equation, and produces the (from → to) rewrite maps for the LUTs `bg.c` actually applies. *Cross-check:* it independently reproduces pass 19's `0x0c1849d8 → 0xc81049d8` ADDFOG mapping, which pass 19 derived by hand.
|
||||
|
||||
1. **Surface 1 (`bg_sevx_all_p`, 42 rooms) authors exactly six render-mode classes** — the complete set, with their fog-on LUT images:
|
||||
|
||||
| authored | fog-on (LUT 1 pri / 5 sec) | setups | rooms | mode |
|
||||
|---|---|---|---|---|
|
||||
| `0x0c182078` | `0xc8102078` | 39 | 39 | `AA_ZB_OPA_SURF` (primary DLs) |
|
||||
| `0x0c1849d8` | `0xc81049d8` | 24 | 24 | `AA_ZB_XLU_SURF` (secondary DLs) |
|
||||
| `0x0c184dd8` | `0xc8104dd8` | 15 | 15 | `AA_ZB_XLU_DECAL` |
|
||||
| `0x0c192d58` | `0xc8112d58` | 7 | 7 | `AA_ZB_OPA_DECAL` |
|
||||
| `0x00502078` | (unchanged) | 2 | 2 | OPA_SURF, no-fog blender |
|
||||
| `0x005049d8` | (unchanged) | 1 | 1 | XLU_SURF, no-fog blender |
|
||||
|
||||
2. **D276's wall painter is now attributed to authored data.** `0xc81049d8` — the class D276 identified as the thing actually painting the visible weave — is **exactly** the LUT image of the bg *secondary* DL class `0x0c1849d8` (`G_RM_AA_ZB_XLU_SURF`), present in 24 of Surface 1's 42 rooms. Nineteen passes have logged that word live; this is the first time it has been traced back to a specific authored source in a specific file. It is ordinary bg transparent-surface geometry, and there is a lot of it.
|
||||
|
||||
3. **`LEVELID_SURFACE` == 36**, so the `-level_36` deterministic boot used by D269/D272/D276/D280 really was Surface 1 (`LEVELID_STATUE = 22` + 14 sequential entries; `bg.c:201` then maps it to `bg/bg_sevx_all_p.seg`). Worth stating explicitly — several passes' conclusions rest on it and none of them checked.
|
||||
|
||||
4. **No bg file in ANY level authors the TEX_EDGE "billboard cut-out" class `0x0c193078`.** Censused `sevx sevb sev dam jun run depo azt` — the union of their authored render modes is `{0c182078, 0c182048, 0c1849d8, 0c184dd8, 0c184e50, 0c192d58, 00502078, 005049d8, 00504dd8, 00552d58}`, and `0x0c193078` appears in none of them, **including Jungle and Depot**. That matters because `DL_LUT_PRIMARY_ADDFOG` carries a dedicated rewrite pair for it with Rare's own comment: *"Billboard Cut-out to FOG Billboard Cut-out - eg, Mario Tree or Depot lamp"*. So the engine's TEX_EDGE billboard path is real but is **never fed from bg geometry** — it belongs to the prop/model path (which is where the jungle foliage props live, per pass 21). Combined with pass 21, Surface 1 has neither tree props nor TEX_EDGE bg cards.
|
||||
|
||||
5. **Consequence — the mechanism is narrowed to texture alpha.** Surface 1's tree cards can only be plain `XLU_SURF` textured quads in the secondary DLs. `XLU_SURF` is `GBL_c2(CLR_IN, A_IN, CLR_MEM, 1MA)` — a straight src-over blend with **no alpha test and no coverage-from-alpha**. Under that mode a card is discrete only if its texture's alpha is transparent outside the silhouette. So "sheet instead of discrete trees" is, mechanically, "the alpha of this class's texture is opaque where it should be transparent" — which is precisely what D276 found and measured (*"fully opaque, alpha nibble mostly 0xf"*, 1023/1024 texels at A=F per D267). **This retires, for this class, the geometry/culling/fog/billboard-orientation branches** that D269→D280 cycled through: none of them can turn a correct-alpha XLU_SURF card into a sheet, and none of them are needed to explain it.
|
||||
|
||||
The open sub-question is no longer *"which draw class"* but *"is this texture's format/alpha being decoded correctly at all"*. D265 tried IA4 `I3:A1`→`I2:A2` (reverted), D267 dumped the IA8 set and dismissed it on a single-instance visual judgement, D273 ruled out the CI palette layout, D276 re-identified the IA8 class but never questioned whether `fmt=3 siz=1` is the *right* interpretation of those bytes. A 32×32 image that decodes as "high-frequency blocky gray static with opaque alpha" is exactly what a tree texture looks like when read at the **wrong format or wrong bit-split** — that specific possibility has never been tested.
|
||||
|
||||
6. **Anomaly to resolve before trusting the older logs.** D269 (M-128) recorded the two live Surface 1 classes as `oml=0xc4112078` (OPA) and `0xc41049d8` (XLU). The LUT images derived here are `0xc8112078` and `0xc81049d8`. The difference is exactly one field — cycle-1's fog alpha source, bits 26-27: `G_BL_A_SHADE` (2, what the LUT writes via `G_RM_FOG_SHADE_A`) vs `G_BL_A_FOG` (1, what D269 logged). D276 and D280 logged the `c8` (A_SHADE) form, matching the LUT. So either something rewrites the fog alpha source after `bgApplyDynamicCCRMLUT`, or D269's words were mis-transcribed and three passes of "two draw classes per room stream" rest on them. Decide this by decoding a live log with `python tools_pc/rendermode.py <word>` rather than by eye — it is a one-bit-field difference that is very easy to read past, and it changes the fog blend if real.
|
||||
|
||||
7. **Also recorded (fog-off path).** `DL_LUT_PRIMARY` (6) and `DL_LUT_SECONDARY` (7) contain **zero** render-mode rewrite pairs — their bodies are entirely `gsDPSetCombineMode`/`gsDPSetCombineLERP`. So when `g_FogSkyIsEnabled` is false, room render modes are left exactly as authored and only combiners are patched. (This is a correct reading of the tables, not a parser gap: the tool finds 8 pairs in LUT 1 and 3 in LUT 5 from the same code path.)
|
||||
|
||||
**Next step for pass 23 (one run, or possibly none):** re-dump the Surface 1 textures for the `0xc81049d8` class with the existing `GE_TEXDUMP`/`GE_TEXRAW`/`GE_D270KEEPOML=1049d8` probe combination (the current `texdump/` in-tree is from a D75 logo capture, not a Surface run), then decode the dumped bytes offline under *every* plausible format hypothesis — IA8 as currently assumed, IA4 both bit-splits, I8, I4, CI4/CI8 with the room's TLUT, RGBA5551 — and look for the one that yields a tree silhouette with a sensible alpha channel. Cheap, offline, and it directly tests the one mechanism left standing. Resolve item 6's `c4`/`c8` discrepancy from the same run's log.
|
||||
|
||||
**Tooling added this pass (keep — read-only, no runtime cost):** `tools_pc/bg_gdl_census.py`, `tools_pc/rendermode.py`.
|
||||
|
||||
|
||||
**Pass 23 (2026-09-21, static session) — THE TREE CARDS ARE IDENTIFIED. Surface 1's `0x0c1849d8`/`0xc81049d8` class binds 13 distinct textures, and four of them — `tex1198`–`tex1201`, `RGBA16` (RGBA5551, ONE bit of alpha), 64×17, `HUFFLOOKUP`, six EXPLICIT LODs — are each bound in 21 of the level's 42 rooms, 3.5× the next most widespread. They are the only explicit-LOD textures in the entire Surface 1 bg. They would log as `fmt=0 siz=2`, which is why D265's IA hunt, D269/D272/D273/D274's CI8 hunt and D276's IA8 hunt all missed them: every probe filter in this investigation has been looking at minority members of the right render-mode class.**
|
||||
|
||||
Pass 22 left the census with `tex=0` — Surface 1's room DLs contain no `G_SETTIMG` at all. That turned out to be the key, not a dead end.
|
||||
|
||||
1. **bg room DLs reference textures by NUMBER, not address.** `texLoadFromGdl` (`tex.c:779`) has `case G_NOOP: texnum = in->words.w1 & 0xfff;` — opcode `0xC0`, 329 occurrences in Surface 1's room DLs — and resolves each into real texture commands at room load. So the texture identity of every bg draw is readable offline, by tracking the active render mode across the command stream and recording the `G_NOOP` texnums bound under it. `bg_gdl_census.py` now does this.
|
||||
|
||||
2. **Reading the texture headers offline.** A texture's ROM blob begins with a control byte (`image.c:2568`): bit 7 `hasExplicitLods`, bit 6 `iszlib`, bits 0-5 `lod` (image count); the bitstream starts at **byte 1**. `iszlib=0` → `texInflateNonZlib`'s 24-bit `ffff wwwwwwww hhhhhhhh cccc` header; `iszlib=1` → `texInflateZlib`'s `format u8, numcolours-1 u8, palette[], w/h per image`. New tool `tools_pc/texheader.py`; offsets/sizes come from the existing `imagelist.u.csv`.
|
||||
*Sanity check, and a near-miss worth recording:* the first attempt forgot the control byte. The resulting parse **still looked plausible** — it produced four "formats" and confident dimensions — but 1020 of 2699 entries were implausible and the format nibble only ever took the values 0/4/8/12, the tell that the field was reading across a boundary. With the byte-1 offset, **1** of 2699 is implausible and the dimension histogram is dominated by 64×32, 64×64, 32×64, 32×32. The tool prints that count every run so the offset can't silently drift again ([[porting-notes §E positive-control rule]] — same class as pass 21's enum bug).
|
||||
|
||||
3. **The full texture list for the wall class** (`oml=0x0c1849d8` → live `0xc81049d8`), by room coverage:
|
||||
|
||||
| texture | rooms | ROM format | dims | compmethod | explicit LODs | logs live as |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `tex1201` | **21** | `RGBA16` | 64×17 | HUFFLOOKUP | **yes (6)** | `fmt=0 siz=2` |
|
||||
| `tex1200` | **21** | `RGBA16` | 64×17 | HUFFLOOKUP | **yes (6)** | `fmt=0 siz=2` |
|
||||
| `tex1199` | **21** | `RGBA16` | 64×17 | HUFFLOOKUP | **yes (6)** | `fmt=0 siz=2` |
|
||||
| `tex1198` | **21** | `RGBA16` | 64×17 | HUFFLOOKUP | **yes (6)** | `fmt=0 siz=2` |
|
||||
| `tex149` | 6 | `IA8` | 32×64 | RLELOOKUP | no | `fmt=3 siz=1` |
|
||||
| `tex806` | 5 | `IA8` | 64×32 | HUFFLOOKUP | no | `fmt=3 siz=1` |
|
||||
| `tex844` | 4 | `RGBA16_CI8` | (zlib, 180 col) | — | no | `fmt=2 siz=1` |
|
||||
| `tex1373` | 3 | `IA8` | 64×32 | RLE | no | `fmt=3 siz=1` |
|
||||
| `tex239` | 2 | `IA8` | 64×32 | HUFFBLUR | no | `fmt=3 siz=1` |
|
||||
| `tex241`/`tex151` | 1 | `RGBA16_CI8` | (zlib) | — | no | `fmt=2 siz=1` |
|
||||
| `tex619` | 1 | `IA8` | 32×64 | RLELOOKUP | no | `fmt=3 siz=1` |
|
||||
| `tex238` | 1 | `IA8` | 64×32 | RLE | no | `fmt=3 siz=1` |
|
||||
|
||||
4. **This resolves the D269-vs-D276 contradiction.** Three passes argued about whether "the wall" is CI8 (D269/D272: `fmt=2 siz=1`) or IA8 (D276: `fmt=3 siz=1`). **Both are members of this one render-mode class, and both are minority members.** D269 sampled a CI8 card and generalised; D276 sampled an IA8 card and generalised and called D269's reframing premature. Neither was wrong about what it sampled; both were wrong to treat their sample as *the* class. The dominant members — four textures covering half the level — are `fmt=0 siz=2` **RGBA16**, a format nobody has filtered for or dumped in 22 passes.
|
||||
|
||||
5. **Why these four are the tree cards** (inference, stated as such — see the honesty note below). Four consecutive texture numbers; near-identical compressed sizes (1569/2035/1902/1964 bytes); identical geometry (64×17) and codec; each bound in exactly 21 of 42 rooms — i.e. every outdoor room, four variants apiece; drawn in the *secondary* (transparent) pass. That is precisely the signature of a small set of foliage/treeline billboard variants scattered across an outdoor level. Nothing else in the level looks like this.
|
||||
|
||||
6. **Mechanism hypothesis, now concrete for the first time.** `RGBA16` is RGBA**5551** — exactly **one bit** of alpha. A 64×17 strip is a wide, short card: a treeline band whose alpha bit is 0 for sky above the silhouette and 1 for tree. It is drawn under `G_RM_AA_ZB_XLU_SURF`, which pass 22 established is a straight src-over blend with **no alpha test and no coverage-from-alpha**. So the silhouette exists *only* in that one alpha bit, and the card is minified hard at distance — which is why Rare authored **six explicit LOD images** for it, the only explicit-LOD textures in this entire bg. Two ways that becomes a sheet on PC, both untested:
|
||||
* **the 1-bit alpha is decoded as always-opaque** → a solid rectangle of tree-coloured texture tiled across the horizon; or
|
||||
* **the authored LOD chain is dropped and GL generates mips by averaging** → the 0/1 alpha becomes fractional, and with no alpha test a minified treeline smears into a translucent band instead of crisp cutouts.
|
||||
|
||||
The second is the concrete, asset-grounded version of the "TEXTLOD/mip sampling" suspicion D269 raised, D272 reframed to, and D274 parked for lack of a mechanism. It now has one.
|
||||
|
||||
7. **Honesty note — what is NOT established.** The pixels have not been decoded: `HUFFLOOKUP` is one of Rare's huffman/RLE/lookup codecs and none of them are implemented offline, so "these are trees" is a strong inference from format, geometry, count and room coverage, **not** a decode. It should be confirmed before any fix is built on it. Equally, no claim is made here that either mechanism in (6) is the actual bug — both are hypotheses that now have a specific, cheap test.
|
||||
|
||||
**Next step for pass 24 — one run, tightly specified:** boot `-level_36` and dump the textures for this class with the probe filter set to **`fmt=0 siz=2`** (RGBA16) rather than the IA/CI filters every previous pass used — `GE_TEXDUMP=1 GE_TEXRAW=1 GE_D270KEEPOML=1049d8`, then match `GE_TEXI[N] addr=` against the dump by address as D275 did. Three things to read off that one capture: (a) do `tex1198`–`tex1201` arrive as 64×17 RGBA5551, or has the width/height/format been mangled on the way in (D265 already found one transposed-dimensions bug in this area); (b) is the alpha bit non-trivial, or opaque everywhere; (c) do all six explicit LOD images arrive, or only LOD 0. Also resolve pass 22's item 6 (`c4` vs `c8` fog alpha) from the same log using `python tools_pc/rendermode.py <word>`.
|
||||
|
||||
**Tooling added/extended this pass (keep — read-only, no runtime cost):** `tools_pc/texheader.py` (new); `tools_pc/bg_gdl_census.py` now tracks `G_NOOP` texnums and prints the decoded ROM header for every texture each render-mode class binds.
|
||||
|
||||
|
||||
**Pass 24 (2026-09-21, static session) — THE TREE TEXTURE IS DECODED. `tex1198` LOD 0 is a 64×17 RGBA5551 treeline strip, 44.2 % opaque, and its alpha channel renders as an unmistakable row of trees. Rare authored a six-level LOD pyramid for it that PRESERVES the silhouette and its alpha coverage all the way down (44.2 % → 43.1 % → 38.8 %). Pass 23's mechanism (b) — authored LOD chain replaced by averaged mips — now has hard ground truth behind it, and the reference image to diff a live capture against exists offline. Still no build, still no run.**
|
||||
|
||||
Pass 23 identified the candidate cards but could not decode them (`HUFFLOOKUP` unimplemented). That is now done: `tools_pc/texdecode.py` ports `texReadBits` (`image_bank.c:99`), `texInflateNonZlib`'s image loop, `texBuildLookup`, `texInflateHuffman`, `texInflateRle` and `texInflateLookupFromBuffer` from `src/game/image.c`, covering the `LOOKUP` / `HUFFLOOKUP` / `RLELOOKUP` compmethods.
|
||||
|
||||
1. **`tex1198` LOD 0 — 64×17 RGBA16 (RGBA5551), HUFFLOOKUP, 481/1088 texels opaque (44.2 %).** Its 1-bit alpha channel:
|
||||
|
||||
```
|
||||
###....###############...############...########....########.###
|
||||
###.....##############...############....#.#####.....######..###
|
||||
###.....#############....###########......######.....######...##
|
||||
##.....#############.....######.####.......#####....######.#..##
|
||||
###.....#############....###########.......###......#######...##
|
||||
###.....####.########....###########......######.....######...##
|
||||
#.......############.....###########.......#####.....#####.....#
|
||||
##.......###########.....#######..##.......####......#####.....#
|
||||
##......#######.####......#..###..#........####......#####.....#
|
||||
#.......########.###......#..###..#........####......####......#
|
||||
##.......###########........####...........#.#.......#####.....#
|
||||
##.......###########.........###...........####......#####.....#
|
||||
##.......##.###..###.........###...........####.......####.....#
|
||||
##........#.###..##..........###............#.#.......###......#
|
||||
.#...........#...##..........##.............##........###.......
|
||||
.................##...........#.............##..........#.......
|
||||
.............................................#..................
|
||||
```
|
||||
|
||||
Vertical trunks tapering to points, transparent sky between them, thinning to isolated tips at the bottom edge. **These are the trees.** Pass 23's identification is confirmed by decode, not inference. 522 of the 1088 texels are literally `0x0000` (black, alpha 0).
|
||||
|
||||
2. **The authored LOD pyramid preserves the silhouette** — this is the load-bearing new fact:
|
||||
|
||||
| LOD | dims | compmethod | alpha opaque |
|
||||
|---|---|---|---|
|
||||
| 0 | 64×17 | HUFFLOOKUP | 481/1088 = **44.2 %** |
|
||||
| 1 | 32×9 | RLELOOKUP | 124/288 = **43.1 %** |
|
||||
| 2 | 16×5 | LOOKUP | 31/80 = **38.8 %** |
|
||||
| 3 | 8×3 | RLE | (channel codec, not ported) |
|
||||
|
||||
LOD 1 and LOD 2 still read as recognisable tree silhouettes, and the opaque fraction holds near 40 % instead of collapsing. Each level uses a *different* compression method, i.e. these are six separately authored images, not a generated chain. **That is exactly what you do to keep a 1-bit-alpha cutout crisp under minification, and exactly what naive box-filter mip generation destroys** — averaging a 0/1 alpha yields fractional alpha, and under `XLU_SURF` (no alpha test, no coverage-from-alpha — pass 22) fractional alpha does not cut out, it blends. A minified treeline then smears into a translucent band across the horizon: the reported "wall/sheet of tree texture".
|
||||
|
||||
3. **Correction to pass 23.** Pass 23 called `tex1198`–`tex1201` "the tree cards". The decode refines that: only **`tex1198`** is unambiguously a treeline strip (44.2 % alpha). **`tex1199`** is the same geometry and codec but 89.4 % opaque with thin vertical gaps — a denser foliage/branch mass or a treed cliff face. **`tex1200` and `tex1201` are 100 % opaque** dense detail textures (138–162 distinct colours, no alpha structure at all) that merely share the render-mode class. The "four textures in 21 of 42 rooms each" observation stands; "all four are trees" does not.
|
||||
|
||||
4. **Two decoder bugs found and fixed while porting — both would have produced confident, wrong pictures.**
|
||||
* **Per-image byte alignment.** `image.c:1096` ends each LOD image with `if (img_bitcount == 0) img_curpos++; else img_bitcount = 0;` — discard the partial byte, and skip a whole byte when already aligned. Omitting it desynchronised every LOD after the first. It surfaced as LOD 1 parsing as `fmt=0 1x32 cm=0`; a scan of neighbouring bit offsets showed a constant **+8-bit** drift, which pointed straight at the align.
|
||||
* **`texInflateRle`'s fudge was guessed, not ported.** The first draft invented `fudge = ((1<<bt) + (1<<rl) + 1 + blocksize) // (blocksize+1)`. The real derivation (`image.c:1605`) is a loop: `cost = bt + rl + blocksize + 1; while (cost > 0) { cost -= blocksize + 1; fudge++; }`. The guess still decoded *something* — it gave `tex1198` LOD 1 as 66.7 % opaque and `tex149` as 48.3 % — and only the corrected port gives the self-consistent 43.1 % and 34.4 %. **Every RLELOOKUP figure in this entry is from the corrected port; any earlier session notes quoting 48.3 % for `tex149` are wrong.** (Same lesson as passes 21 and 23, third time in four passes: in this codebase a half-right bitstream parse produces plausible output, never an obvious failure. Port the derivation, don't reconstruct it.)
|
||||
|
||||
5. **What this gives pass 25.** There is now an offline, N64-correct reference for the exact texture at the centre of D236: format, dimensions, per-LOD alpha coverage, and the silhouette itself, reproducible with `python tools_pc/texdecode.py 1198`. A live capture can be diffed against it field by field instead of being interpreted on its own.
|
||||
|
||||
**Next step (pass 25) — one run, three questions, all now answerable by direct comparison:**
|
||||
1. Does `tex1198` arrive as **64×17 RGBA5551**, or are the dimensions/format mangled? (D265 already found one transposed-dimensions bug in this area.)
|
||||
2. Is the imported alpha still **~44 % transparent**, or opaque everywhere? If opaque, the bug is in the 5551 alpha-bit decode and mechanism (a) is confirmed.
|
||||
3. Do **all six authored LOD images** reach the GL texture, or only LOD 0 with driver-generated mips? If the latter, mechanism (b) is confirmed and the fix is to upload the authored chain (`glTexImage2D` per level with `GL_TEXTURE_MAX_LEVEL`) rather than letting the driver filter a 1-bit alpha.
|
||||
Capture with `GE_TEXDUMP=1 GE_TEXRAW=1 GE_D270KEEPOML=1049d8` on `-level_36`, filtering **`fmt=0 siz=2`**. Note `texSwapAltRowBytes` (`image.c:1088`) runs on explicit-LOD textures and swaps bytes on alternate rows — expect that in the raw dump; `texdecode.py` decodes logically and does not apply it.
|
||||
|
||||
**Tooling added this pass (keep — read-only, no runtime cost):** `tools_pc/texdecode.py` (`<texnum>`, `--lod N`, `--ppm out.ppm`; PPM renders transparent texels magenta).
|
||||
|
||||
**Pass 25 (2026-09-21, live session) — all three pass-24 questions answered by direct comparison; the mip mechanism is now located to a line. Q1 YES, Q2 YES, Q3 NO — and the "wall" turns out not to be painted by `tex1198` at all from an idle spawn.**
|
||||
|
||||
**Setup:** live runs of `-level_36` (idle boot, 4800 frames) with `GE_TEXDUMP=1 GE_TEXRAW=1 GE_D219RAW=1`, plus one A/B run flipping the in-tree config key `Video.FixMipTextures` (default 1 → 0) to observe unclipped LOADBLOCK sizes. `texdecode.py` was first verified function-by-function against `image.c`/`image_bank.c` (all pass-24 numbers reproduced: 44.2/43.1/38.8 %, 89.4 %, 100 %, tex149 34.375 %). One latent decoder bug fixed on the way: a `max(1, …)` clamp on `(ncol-1).bit_length()` in the plain-LOOKUP path that C's `texGetBitSize(1)=0` does not have (unreachable for GE assets; no output change).
|
||||
|
||||
**Q1 — YES, dimensions/format intact.** D219RAW shows all six `tex1198` LODs pool-decoded at level load with the exact offline dims/compmethods (64×17 HUFFLOOKUP, 32×9 RLELOOKUP, 16×5 LOOKUP, 8×3/4×2/2×2 RLE), format=1 (RGBA16) throughout. The live C decoder's first row of LOD 0 is **pixel-identical** to the offline decode after the documented `texSwapAltRowBytes` byte-swap — and that swap is a no-op on PORT (D159 early-return), so raw dumps are plain linear pixels.
|
||||
|
||||
**Q2 — YES, alpha survives intact.** Pool bytes ≡ ROM-decoded bytes ⇒ 44.2 % opaque at the pool stage. `import_texture_rgba16` (`gfx_pc.cpp:791`) maps bit0→alpha 255/0; `import_texture_ia8` maps upper nibble=intensity, lower=alpha — both preserve alpha by construction. Data check on the one drawn member of the class: `GE_TEXI[101]` (IA8 32×64, `tex149`) is **byte-for-byte identical** to the offline ROM decode (2048/2048).
|
||||
|
||||
**Q3 — NO, structurally.** `gfx_opengl_upload_texture` (`gfx_opengl.cpp:713`) does `glTexImage2D(level 0)` + optional `glGenerateMipmap`; there is no code path that uploads an authored chain as GL mip levels. Each LOADBLOCK becomes one GL texture; the driver box-filters level 0.
|
||||
|
||||
**New finding N1 — `tex1198`–`tex1201` are never drawn from an idle spawn.** Across all 4800 frames there is not a single `fmt=0 siz=2` (RGBA16) import while CI8/IA8 background draws continue every frame. The tree-like cover visible at Bond's spawn point is therefore painted by the *other* classes — consistent with D276/D280 (the IA8 noise class paints the weave). Pass 24's "leading mechanism" applies to `tex1198`'s cards, which only draw when the player reaches their rooms; the spawn-visible wall is a different painter.
|
||||
|
||||
**New finding N2 — the drawn tree-class texture is `tex149`: IA8 32×64 hard cutout, 34.4 % opaque** (alpha nibble F for opaque pixels, 0 for holes — no ramp). Its ROM blob is **329 bytes = exactly one image**; there is no authored mip chain.
|
||||
|
||||
**New finding N3 — the mip mechanism, located precisely.** `texGetDepthAndSize` (`tex.c:330`) sizes the LOADBLOCK from `maxlod`, and `texInflateNonZlib` sets `maxlod = forcenumimages` = the header's `lods` field (**7** for `tex149`) **even when `explicit_lods=0` and only one image was decoded**. So GE's LOADBLOCK is deliberately larger than the real pool data: live A/B with `Video.FixMipTextures=0` shows the same 32×64 cards at **size=2808** (base 2048 + 760 B "mip region") vs 2048 clipped. On N64 those extra bytes are whatever adjacent stage-pool memory follows the allocation — deterministic, likely neighbouring level textures. The port's RC2 clip (`g_fix_mip_textures`, `gfx_pc.cpp:1089/1142`, default ON) truncates to base height and `glGenerateMipmap` + trilinear minification replaces that region with driver box-filtered mips — **observed live on the 32×64 upload: `gen_mip=1 filter=LINEAR mipmap_filter_mode=MIPMAP_LINEAR`; 380/400 sampled uploads generate mips.** For a hard 0/F alpha cutout, box filtering turns every mip level into fractional alpha, and the XLU blend then smears minified silhouettes — the reported wall/sheet. (For `tex1198`-class cards with real six-level chains, the same clip+regenerate path replaces the authored pyramid — pass 24's mechanism (b), now confirmed at the line level.)
|
||||
|
||||
**Honesty note.** The N64 side of N3 is established for the *load* (block size, pool contents) but not for the *sample*: what the RDP actually read out of that 760 B mip region for minified cards (adjacent-pool bytes vs effectively level-0) cannot be resolved without hardware or an RDP sampler model. Both candidate port fixes below are therefore approximations to be judged by playtest A/B, not by derivation.
|
||||
|
||||
**Fix directions for pass 26 (port-only; no game logic):**
|
||||
1. **No-chain textures (`explicit_lods=0`, single-image blob — `tex149` class):** do not `glGenerateMipmap`; sample level 0 only (`GL_LINEAR`, non-mip min filter). Matches "RDP with no real mips" and kills the fractional-alpha smear at zero cost. This is the cheapest test of the whole hypothesis: if the spawn wall sharpens into discrete trees, mechanism confirmed end-to-end.
|
||||
2. **Chained textures (`tex1198` class):** upload the authored levels as real GL mip levels (needs per-LOD dims from the pool header; GE's authored dims are *not* exact halvings — 17→9→5→3→2 vs GL's required 17→8→4→2 — so levels must be padded/cropped to legal sizes, or uploaded into a custom layout). Bigger job; only after (1) is judged.
|
||||
|
||||
**Probe added this pass (TEMPORARY — remove once the mip question is closed):** `GE_D236P25` env-gated block in `gfx_opengl_upload_texture` logging width/height/gen_mipmaps/filter per upload (capped 400 lines). **Process note:** Git-Bash `kill` does not terminate the game here — use `taskkill //F //IM ge007.x86_64.exe` (two idle runs were left alive this session).
|
||||
|
||||
**Pass 26 (2026-09-21, live session) — D236 ROOT-CAUSED AND FIXED. The tree cards were never a decode, alpha or mip problem: the port was binding the wrong texture. Surface 1 now renders discrete snow-covered pines; verified live.**
|
||||
|
||||
**The pass-25 fix direction was tested first and REFUTED.** Implemented cut-out
|
||||
mip suppression (skip `glGenerateMipmap` + `GL_TEXTURE_MAX_LEVEL 0` for
|
||||
textures whose alpha is hard 0/255 with real holes), config-gated, and A/B'd it
|
||||
live on `-level_36` at the idle spawn. **The wall did not change.** Two things
|
||||
came out of that run and both mattered:
|
||||
|
||||
- The per-upload alpha census showed the spawn view is dominated by **248
|
||||
uploads of a fully-opaque 32×32 texture** (`clear=0 solid=1024 mid=0`) — the
|
||||
"wall" is opaque, so no alpha/blend mechanism could have produced it. That
|
||||
retires the whole mip / 1-bit-alpha / XLU-smear branch (passes 22–25).
|
||||
- Correction to pass 25's **N3**: the 760 B "mip region" beyond a no-chain
|
||||
texture's base image is **not adjacent pool garbage**. `texInflateNonZlib`
|
||||
(`image.c:1123-1153`) and `texInflateZlib` (`image.c:324-360`) *generate* the
|
||||
chain themselves via `texShrinkNonPaletted`/`texShrinkPaletted` when
|
||||
`explicit_lods == 0 && forcenumimages >= 2`. Those shrinkers also re-quantise
|
||||
alpha to the source's bit depth — for RGBA5551, `a = (a0+a1+a2+a3 + 2) >> 2`
|
||||
keeps every level a hard 0/1 cut-out, unlike a GL box filter. Worth knowing
|
||||
for any future mip work; irrelevant to this bug.
|
||||
|
||||
**Root cause (`gfx_lod_tile_offset`, `port/fast3d/gfx_pc.cpp`).** GE's
|
||||
`texLoadFromGdl` picks a handler per binding from `in->words.w0 & 7`:
|
||||
|
||||
- `TEXTURETYPE_LOD`(0) / `TEXTURETYPE_MIPMAP`(2) — `texHandleType0`/`Type2`:
|
||||
ONE load block at TMEM 0, tiles 0..n are LOD levels of that one image.
|
||||
- `TEXTURETYPE_DETAIL`(1) — `texHandleType1` (`tex.c`): **TWO** load blocks —
|
||||
the *detail* texture at TMEM 0 (`texWriteLoadToTmemZero(tex2)`) and the
|
||||
**base image** at TMEM `texGetSizeInBytes(tex2,0)` on tiles 1+
|
||||
(`texWriteLoadToTmemAddr(tex1,size)` + `texWriteTileLods(tex1,…,1,size)`).
|
||||
|
||||
D107 folded every `G_TL_LOD` draw to tile 0 on the premise "GE loads the whole
|
||||
mip chain at TMEM 0". True for types 0/2, **false for type 1** — there tile 0 is
|
||||
the detail texture. So the port drew the detail texture instead of the base
|
||||
image, and never imported the base image at all.
|
||||
|
||||
**Confirmed against the data, offline and live:**
|
||||
|
||||
1. `GE_D85TEX` on `-level_36`: `tex1198`–`tex1201` each resolve 28× with a
|
||||
non-NULL pool pointer and **`type=1`** — they were being bound all along.
|
||||
Pass 25's N1 ("zero `fmt=0 siz=2` imports in 4800 frames") was the symptom,
|
||||
not evidence the cards weren't referenced.
|
||||
2. Offline scan of every `G_NOOP` in `bg_sevx_all_p`: the only `type=1`
|
||||
bindings in Surface 1 are exactly the four tree cards, **21 rooms each, all
|
||||
sharing `texnum2 = 2465`**. Everything else in the level is `type=2` with
|
||||
`detail=0`.
|
||||
3. `tex2465` = **IA8 32×32 HUFFLOOKUP, `lods=0`, fully opaque** — precisely the
|
||||
248-upload opaque 32×32 class above, and the same texture D267/D276 measured
|
||||
at "1023/1024 texels at A=F" and called the noise painter. D276's painter was
|
||||
never a mystery texture; it is the tree cards' detail tile.
|
||||
|
||||
**Fix (port-layer only, `gfx_lod_tile_offset`).** When `rdp.tex_lod` is set and
|
||||
tile `fi+1` sits at a different TMEM address *and* declares a different
|
||||
`fmt`/`siz` than tile `fi`, return tile offset 1 (the base image) instead of 0.
|
||||
Deliberately asymmetric: LOD levels of one image always share a format, so a
|
||||
mip chain can never trip it, and a detail pair that happened to share a format
|
||||
would simply keep today's behaviour. Config `Video.DetailBaseTile`, default on.
|
||||
|
||||
**Two weaker discriminators were tried live and both failed — recorded so they
|
||||
are not retried:** (a) "tile fi+1 is at a different TMEM address" alone — mip
|
||||
levels are at different TMEM offsets too, so every ordinary surface started
|
||||
sampling its first mip (trees correct, all ground/snow textures wrecked; the
|
||||
user caught this in the same build); (b) additionally requiring
|
||||
`rdp.loaded_texture[tmem].addr` to be set — same failure, because
|
||||
`rdp.loaded_texture[]` is persistent state never cleared between binds, so a
|
||||
stale entry from an earlier type-1 draw satisfies it.
|
||||
|
||||
**Verification.**
|
||||
- Live `-level_36` idle spawn, frame 560: opaque tiled noise wall →
|
||||
**discrete snow-covered pines lining the valley**, ground/snow unchanged
|
||||
against the pre-fix capture. Surface 2 (`-level_43`) likewise shows a discrete
|
||||
night treeline; Depot (`-level_30`, the only other level with `type=1`
|
||||
bindings) renders clean.
|
||||
- `bunker1` golden framediff is **bit-identical with the fix on and off**
|
||||
(`worst_cell=189.741875` both ways — that delta is pre-existing, from the
|
||||
other uncommitted work in the tree, not from this change).
|
||||
- Offline blast-radius scan of every level's bg: `type=1` bindings exist **only**
|
||||
in `sevx` (84), `depo` (39), `sevb` (4), `sev` (4). Every other level has zero,
|
||||
so the change is provably inert on them.
|
||||
|
||||
**Generalised to `docs/porting-notes.md` §D12.**
|
||||
|
||||
**Housekeeping:** the pass-25 `GE_D236P25` probe and the pass-26 cut-out
|
||||
experiment are both removed from the tree (the mip question is closed).
|
||||
|
||||
## D237 — QoL ask (M-106, user QA): F10 options overlay — make it scrollable with the mouse wheel, expose more settings into it (e.g. screen-flash / no-hit-flash…
|
||||
|
||||
**QoL ask (M-106, user QA): F10 options overlay — make it scrollable with the mouse wheel, expose more settings into it (e.g. screen-flash / no-hit-flash `Game.NoHitFlash`, and other easily-exposed config keys), and organise rows by category (mouse settings together, video together, etc.).** The overlay (`port/src/optionsoverlay.c`) already renders over the registered `config.c` option list; what's missing is wheel-scroll nav, a broader row set, and grouping headers. Cross-ref `OPTIONS-MENU-PLAN.md` §3 (v1 minimum set) + QOL-INVENTORY "PD has, GE lacks".
|
||||
@@ -12247,3 +12565,21 @@ Full campaign (or at minimum Silo → Caverns/Cradle) with **`GE_D322=1 GE_D204=
|
||||
- All pools healthy while audio is bad → back to the mixer/reverb state classes (re-open M-65's ruled-out list with a long-session `GE_AUDIODUMP`).
|
||||
|
||||
**Status:** OPEN — static triage complete, probe shipped, awaiting campaign capture. Cross-ref: D202/M-65+M-66b (ownerless-loop leak class + expiration), D207 (8-cap starvation design notes), D305 (pool/list desync observation), D204 (pipeline pacing + `GE_D204` monitor), D248/D250 (frame pacing, fixed), issue #87.
|
||||
|
||||
### M-197 (v0.4.0 Phase 1, 2026-09-21): D75 root cause — the D50 model-sidecar converter dropped embedded texture blobs; fix + live verification
|
||||
|
||||
**Symptom (inherited from M-33/M-195/M-196):** on the intro's Nintendo-logo screen the 3D `logoinst` model rendered as a tiny orbiting fragment (0–33 content px/frame) — M-195 proved emission + vertex transform correct, M-196 refuted G_VTX slot-overwrite and narrowed to "texture-modulate material rendering black" with an open anomaly (`utex`/`comb->used_textures[0]` reading 0 at emit despite `gfx_generate_cc` setting it).
|
||||
|
||||
**Step 1 — the anomaly was a probe artifact.** The D75D frame-level fields are first-write-wins per frame: `utex`/`cutex` captured on the *first* emitted triangle (a title-screen quad with no texture) while `comb` held the *last* triangle's combine mode. Added logo-mode-specific sampling (`lg_*` fields, updated on every emitted tri whose `rdp.combine_mode == 0x009ffe4f19ffe4f1`): `lg_utex=1 lg_tex=1` on all 14k+ logo triangles, tile 0 = tmem 0 / fmt IA8 / 32×32, `lg_texaddr=0x7015dec0`. The combiner/GL-shader texture claim was never the bug.
|
||||
|
||||
**Step 2 — the texture buffer is zero because the sidecar never contained the pixels.** New `GE_TEXDUMP_ADDR=<hex>` probe (raw source-byte dump + min/max/nonzero stats for every import from one address, bypassing the 400-dump cap) showed the buffer at `0x7015dec0` all-zero on *every* import across a 90 s run. GE's ucode has no RSP `loadtxl` (checked `rsp/graphics/gmain.s`) — all texture loading is DP DMA from DRAM, so the zeroed DRAM is upstream of the RSP. Tracing the load path: `front.c` loads `PnintendologoZ` into `ptr_logo_and_walletbond_DL` via `load_object_fill_header` → `load_resource` → RZ-inflate of the **D50 PC sidecar** (`port/src/pcmodels.c`, produced by `tools_pc/d43_emit.py`). Offline decode of the shipped sidecar: all 23 GDL `G_SETTIMG` slots correctly remapped to `w1=0x05006428`, but the decompressed image at `0x6428` is **zero for ≥1024 bytes** and contains no trace of the source pixels (the N64 original's blob at file offset `0x408` is a 32×32 IA8 white wordmark, `ffffffff…`, 713/1024 nonzero). Address chain closes exactly: runtime file base `0x70157a98` (M-195) + `0x6428` = `0x7015dec0`.
|
||||
|
||||
**Root cause:** in `d43_emit.py`, the layout pass detects embedded image blobs (G_SETTIMG seg-5 targets — present only in P* logo/legal files), reserves the span via `add_region` (so pointers remap and tiling passes) — but **no emit-pass code ever copies the span's bytes into `buf`** (`bytearray(D_PC)` stays zero there). Round-trip validation checked every structured field and GDL slot (including `w1 == 0x05|remap(target)`, which *passes* because remap found the reserved region) but never the raw blob pixels, so the gap was invisible to `ALL CHECKS PASSED`.
|
||||
|
||||
**Blast radius:** a scan of all 512 model files finds exactly three with seg-5 SETTIMG targets: `PnintendologoZ` (1 image), `PgoldeneyelogoZ` (2), `PlegalpageZ` (5) — the intro's GoldenEye logo and legal page were black-textured too (less visible: dark screens). No placed object offset falls inside any blob span in any of the three files, so old/new-space remapping is unambiguous (the nintendologo node-region overlap at the blob start is a PC-footprint bookkeeping artifact — 12 bytes past the N64 node's true end, no pointer targets it).
|
||||
|
||||
**Fix (tooling-only):** `tools_pc/d43_emit.py` — (1) layout pass records `blob_regions = [(old, osz, new)]`; (2) emit pass raw-copies each span (`buf[n:n+osz] = src[o:o+osz]`, identity — the RDP reads blob bytes as-is on both platforms); (3) `validate()` gains a raw-identity round-trip check per blob span. Regenerated `data/pcmodels-ntsc-final/` (`ALL CHECKS PASSED`); offline re-decode confirms all 8 blob targets now carry their N64 pixels. No C code changed — the port's load path (`pcmodels.c` → `load_resource` → inflate) is correct and untouched; no rule-2 sign-off involved (no `src/game` edit).
|
||||
|
||||
**Live verification:** bare front-end boot (existing binary, new data only). `GE_PCDUMP="240-330:10"`: logo window goes 45 → 851 → 8,599 → **23,302 non-black px (7.59%)** over frames 260–320 with a wide wordmark bbox `(94,190)-(542,290)`; `GE_PCDUMP="340-560:20"` shows the logo continuing its rotation animation at 3–10% coverage (ASCII-art inspection: elliptical ring + shaded 3D lettering, grey/white palette consistent with the wordmark texture). Pre-fix baseline (M-196): 0–33 px/frame.
|
||||
|
||||
**Status:** **FIXED, user-confirmed live (2026-09-21)** — "the images and legal page and Nintendo logo are good now." Tooling fix + ntsc-final sidecars regenerated; headless-verified before the live check. Remaining: `pal-final`/`jpn-final` sidecar regeneration on machines holding those ROMs (`python tools_pc/d43_emit.py <region>` — the fix is in the tool, so any future regen is correct), and D75's last unverified scope: cast-roll models (status unknown since M-33; recheck with `GE_PCDUMP="1400-2200:25"`).
|
||||
@@ -28,6 +28,7 @@ good that you are looking at one of these.
|
||||
- [D3. GCC/mingw makes an all-non-negative `enum` UNSIGNED](#d3-gccmingw-makes-an-all-non-negative-enum-unsigned)
|
||||
- [D4. N64 "interrupts off" must be a real lock on PC](#d4-n64-interrupts-off-is-not-free-on-pc--it-must-be-a-real-lock)
|
||||
- [D5. Loop bounds that assume linker adjacency of two globals](#d5-loop-bounds-that-assume-linker-adjacency-of-two-file-scope-globals)
|
||||
- [D12. Collapsing an LOD binding to tile 0 breaks a DETAIL binding](#d12-collapse-an-lod-binding-to-tile-0-is-wrong-for-a-two-texture-detail-binding-d236)
|
||||
- [E. Process / method notes](#e-process--method-notes)
|
||||
|
||||
## A. Pointer-width struct growth (32→64): the dominant class
|
||||
@@ -865,6 +866,48 @@ through a converter or a runtime bswap fixup reads scrambled.
|
||||
(ginit.s is missing from this repo; the expansion lives there), not UV,
|
||||
shade, or the combiner.
|
||||
|
||||
## D12. "Collapse an LOD binding to tile 0" is wrong for a two-texture DETAIL binding (D236)
|
||||
|
||||
**M-198 (finding D236 pass 26).** GE's `texLoadFromGdl` resolves a room DL's
|
||||
`G_NOOP` texture reference through one of three handlers, chosen by
|
||||
`in->words.w0 & 7` (`src/game/tex.c`):
|
||||
|
||||
| type | handler | TMEM layout |
|
||||
|---|---|---|
|
||||
| 0 `TEXTURETYPE_LOD` | `texHandleType0` | ONE load block at TMEM 0; tiles 0..n are LOD levels **of that one image** |
|
||||
| 1 `TEXTURETYPE_DETAIL` | `texHandleType1` | TWO load blocks: the **detail** texture at TMEM 0, the **base** image at TMEM `texGetSizeInBytes(detail,0)` on tiles 1.. |
|
||||
| 2 `TEXTURETYPE_MIPMAP` | `texHandleType2` | as type 0 |
|
||||
|
||||
fast3d's `gfx_lod_tile_offset` (`port/fast3d/gfx_pc.cpp`) folded **every**
|
||||
`G_TL_LOD` draw to tile 0, on the reasoning (D107) that "GE loads the whole mip
|
||||
chain at TMEM 0". That is true for types 0 and 2 and **false for type 1**,
|
||||
where TMEM 0 holds the detail texture and the image you actually want is on
|
||||
tile 1. The port therefore drew the detail texture in place of the base image —
|
||||
and never even *imported* the base image, so every probe looking for it came
|
||||
back empty.
|
||||
|
||||
On Surface 1 this was the whole of D236: all four treeline cards
|
||||
(`tex1198`–`tex1201`, RGBA5551 64×17 cut-outs) are type-1 bindings sharing one
|
||||
detail texture, `tex2465` — an opaque 32×32 IA8 noise tile. Result: an opaque
|
||||
tiled noise wall where the treeline belongs, for 20+ investigation passes.
|
||||
|
||||
**Tells.** (a) A texture the game demonstrably decodes into the pool is never
|
||||
imported by the renderer — that is a *binding* bug, not a decode, alpha, mip or
|
||||
geometry bug, and no amount of work on the texture's contents will move it.
|
||||
(b) The wrong thing drawn is itself a real, correctly-decoded texture from the
|
||||
same draw — look for a second texture bound by the same command.
|
||||
|
||||
**Discriminating the two layouts at draw time.** Prefer an invariant that a mip
|
||||
chain can never satisfy. Tiles 1.. of a type-0/2 binding are LOD levels of tile
|
||||
0's image, so they necessarily carry the **same `fmt`/`siz`**; a type-1 pair is
|
||||
two unrelated textures and generally does not. Deliberately choose the
|
||||
asymmetric test — a mip chain can never trip it, and a detail pair that happens
|
||||
to share a format merely keeps the old behaviour. Two weaker discriminators
|
||||
were tried live and both misfire: "tile 1 sits at a different TMEM address"
|
||||
matches ordinary mip chains (their levels are at different TMEM offsets too),
|
||||
and "tile 1's TMEM slot has a registered load" matches them as well, because
|
||||
`rdp.loaded_texture[]` is persistent state that is never cleared between binds.
|
||||
|
||||
## D2. The HUD/model "X-mirror" (D114/D116): RESOLVED, it was an upside-down capture
|
||||
|
||||
**M-33 (finding D168).** There was no mirror. `gfx_opengl_dump_bound_fbo`
|
||||
@@ -1169,6 +1212,43 @@ produce the same visual symptom but need opposite fixes.
|
||||
writing anything new; same Rare engine family.
|
||||
- Don't re-investigate a closed §F finding or re-derive a format spec
|
||||
that already has a converter.
|
||||
- **When re-implementing a ROM bitstream/codec offline, PORT the derivation
|
||||
from the decomp; never reconstruct it from the comment.** A half-right
|
||||
bitstream parser in this codebase does not fail loudly — it returns
|
||||
confident, plausible, wrong output. Three hits in four D236 passes:
|
||||
(1) a texture-header parse that omitted the leading control byte
|
||||
(`image.c:2568`) still produced sensible-looking formats and dimensions,
|
||||
and was only caught by counting implausible entries (1020/2699 vs 1/2699
|
||||
when correct) and noticing the format nibble only ever took values
|
||||
0/4/8/12 — the tell that a field was straddling a boundary;
|
||||
(2) omitting the per-image byte-align at the end of each LOD
|
||||
(`image.c:1096`: discard the partial byte, and skip a whole byte when
|
||||
already aligned) silently desynchronised every LOD after the first;
|
||||
(3) `texInflateRle`'s run-length `fudge` was *guessed* from the doc comment
|
||||
instead of ported from the loop at `image.c:1605`, and still decoded
|
||||
images — just with the wrong alpha coverage (66.7% vs the correct 43.1%).
|
||||
Practical rules: copy the arithmetic statement-for-statement even when it
|
||||
looks like it could be closed-formed; build a cheap global plausibility
|
||||
counter into the tool and print it every run; and cross-check one decoded
|
||||
artifact against an independent expectation (here, that a LOD chain's
|
||||
dimensions halve exactly and its alpha coverage stays roughly constant)
|
||||
before trusting any number the tool emits.
|
||||
|
||||
- **A negative result from a diagnostic is worthless until the diagnostic has
|
||||
been shown capable of producing the positive.** Two independent hits: D219's
|
||||
"verify diagnostic caps exempt the real target signature before trusting
|
||||
'clean'", and D236 pass 21, where a static setup scanner located GE's `PROP`
|
||||
enum with `startswith("typedef enum PROP")` — which matches
|
||||
`typedef enum PROPFLAG` hundreds of lines earlier, yielding an enum with **no
|
||||
`PROP_*` members at all**. Every prop id then resolved to `?` and every
|
||||
"foliage props placed" count came back 0, which is exactly what a clean,
|
||||
meaningful negative looks like. Build the positive control **into** the probe:
|
||||
have it print what it is capable of matching (`foliage props recognised (9):
|
||||
…`) and fail loudly when that set is empty, so the failure mode cannot be
|
||||
read as a finding. Applies equally to grep-based audits over generated C,
|
||||
where near-miss identifier prefixes (`PROP` / `PROPFLAG` / `PROPDEF_TYPE`) are
|
||||
the norm.
|
||||
|
||||
- **`docs/dev/findings-index.csv` is generator-derived** (`tools_pc/gen_findings_index.py`): the status cell is normalised from the `## Dxx` entry's §F summary-table row in `findings.md`, so hand-editing the CSV is silently reverted on the next regen (hit live in the D219/D252 M-158 session). Update the table row, then run the generator.
|
||||
- **Every game launch rewrites `data/ge007.eep`** (the eeprom shim stores on
|
||||
every write; validation resets, cheat patches and normal saves all write).
|
||||
|
||||
@@ -47,6 +47,7 @@ void gfx_set_target_fps(int);
|
||||
void gfx_set_texture_filter(enum FilteringMode mode);
|
||||
void gfx_set_mipmap_filter(enum MipmapFilteringMode mode);
|
||||
void gfx_set_fix_mip_textures(int on);
|
||||
void gfx_set_detail_base_tile(int on); /* D236: sample the base image of a DETAIL binding */
|
||||
void gfx_set_wrap_fix(int on);
|
||||
void gfx_set_anisotropy_level(int level); /* 1 = off; clamped to GL max */
|
||||
void gfx_set_safe_area_crop(int on); /* crop the N64 TV-overscan safe-area margin instead of showing it as black bars */
|
||||
|
||||
+210
-2
@@ -4,6 +4,10 @@
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#ifdef _WIN32
|
||||
#define WIN32_LEAN_AND_MEAN
|
||||
#include <windows.h> /* D236 pass 17 GE_D236BT backtrace probe */
|
||||
#endif
|
||||
#include <cassert>
|
||||
#include <cstdio>
|
||||
|
||||
@@ -144,6 +148,74 @@ static struct RSP {
|
||||
* which segment the tree class's vertices actually came from. See the
|
||||
* G_VTX case comment below for why. Remove once D236 pass 16 concludes. */
|
||||
static uint8_t g_d236_last_vtx_seg = 0xFF;
|
||||
/* D236 pass 17 (TEMP): full raw w1 of the most recent G_VTX, so the
|
||||
* GE_D236BT backtrace probe can report WHICH buffer the tree class's
|
||||
* triangles read from alongside the C call site that submitted them.
|
||||
* Remove once D236 pass 17 concludes. */
|
||||
static uint32_t g_d236_last_vtx_raw = 0;
|
||||
|
||||
/* D236 pass 17b (TEMP): host stack backtrace dumper, C-linkage so the
|
||||
* allocator-side probe in src/game/chr.c can call it too. Windows-only:
|
||||
* RtlCaptureContext + RtlVirtualUnwind over the exe's own .pdata table
|
||||
* (this box's dbghelp.dll is stripped: no CaptureStackBackTrace,
|
||||
* StackWalk64 fails immediately; RtlCaptureContext must be called
|
||||
* directly -- GetProcAddress indirection zero-filled the context;
|
||||
* mingw-ld leaves DataDirectory[10] zero even when .pdata is present,
|
||||
* so find it by section name and trim the trailing zeroed entry; NULL
|
||||
* out-params crash RtlVirtualUnwind). Frames print as RVAs -- resolve
|
||||
* offline with addr2line against this exact exe. Remove once D236
|
||||
* pass 17 concludes. */
|
||||
#ifdef _WIN32
|
||||
extern "C" void port_d236bt_dump(const char* tag) {
|
||||
CONTEXT ctx;
|
||||
RtlCaptureContext(&ctx);
|
||||
const HMODULE imgbase_h = GetModuleHandleA(NULL);
|
||||
const ULONG64 imgbase = (ULONG64)(uintptr_t)imgbase_h;
|
||||
PIMAGE_DOS_HEADER dos = (PIMAGE_DOS_HEADER)imgbase_h;
|
||||
PIMAGE_NT_HEADERS nth = (PIMAGE_NT_HEADERS)((BYTE*)imgbase_h + dos->e_lfanew);
|
||||
PIMAGE_SECTION_HEADER shs = (PIMAGE_SECTION_HEADER)
|
||||
((BYTE*)&nth->OptionalHeader + nth->FileHeader.SizeOfOptionalHeader);
|
||||
typedef struct { DWORD BeginAddress; DWORD EndAddress; DWORD UnwindDataAddress; } D236_RF;
|
||||
D236_RF* rfs = NULL;
|
||||
int nvalid = 0;
|
||||
for (WORD si = 0; si < nth->FileHeader.NumberOfSections; si++) {
|
||||
if (_stricmp((const char*)shs[si].Name, ".pdata") == 0) {
|
||||
rfs = (D236_RF*)((BYTE*)imgbase_h + shs[si].VirtualAddress);
|
||||
nvalid = (int)(shs[si].SizeOfRawData / sizeof(D236_RF));
|
||||
while (nvalid > 0 && (rfs[nvalid - 1].BeginAddress == 0 ||
|
||||
rfs[nvalid - 1].EndAddress == 0))
|
||||
nvalid--;
|
||||
break;
|
||||
}
|
||||
}
|
||||
ULONG64 rip = (ULONG64)(uintptr_t)ctx.Rip;
|
||||
fprintf(stderr, "D236BT %s rip_rva=0x%llx nvalid=%d\n",
|
||||
tag ? tag : "?", (unsigned long long)(rip - imgbase), nvalid);
|
||||
for (int depth = 0; depth < 32 && rfs != NULL; depth++) {
|
||||
const ULONG64 rva = rip - imgbase;
|
||||
int lo = 0, hi = nvalid - 1;
|
||||
D236_RF* rf = NULL;
|
||||
while (lo <= hi) {
|
||||
const int mid = (lo + hi) / 2;
|
||||
if (rva < rfs[mid].BeginAddress) hi = mid - 1;
|
||||
else if (rva >= rfs[mid].EndAddress) lo = mid + 1;
|
||||
else { rf = &rfs[mid]; break; }
|
||||
}
|
||||
if (!rf) {
|
||||
fprintf(stderr, "D236BT %s [%d] rva=0x%llx (outside exe .pdata)\n", tag, depth,
|
||||
(unsigned long long)rva);
|
||||
break;
|
||||
}
|
||||
PVOID hd = NULL;
|
||||
DWORD64 ef = 0;
|
||||
RtlVirtualUnwind(0, imgbase, rip, (PRUNTIME_FUNCTION)rf, &ctx,
|
||||
&hd, &ef, NULL);
|
||||
rip = (ULONG64)(uintptr_t)ctx.Rip;
|
||||
fprintf(stderr, "D236BT %s [%d] rva=0x%llx\n", tag, depth,
|
||||
(unsigned long long)(rip - imgbase));
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
struct RawTexMetadata {
|
||||
uint16_t width, height;
|
||||
@@ -1020,6 +1092,11 @@ bool g_fix_mip_textures = true;
|
||||
* off. Video.WrapFix = 1. */
|
||||
bool g_wrap_fix = false;
|
||||
|
||||
/* D236 pass 26: for a GE TEXTURETYPE_DETAIL binding, sample the BASE image
|
||||
* (tile 1) rather than the detail texture sitting at TMEM 0. See
|
||||
* gfx_lod_tile_offset. Video.DetailBaseTile. */
|
||||
bool g_detail_base_tile = false;
|
||||
|
||||
/* D183 source-pitch de-stride (see import_texture). Default on;
|
||||
* GE_TEXPITCH=0 restores the old flat read for A/B. */
|
||||
static bool gfx_tex_pitch_fix(void) {
|
||||
@@ -1241,6 +1318,33 @@ static void import_texture(int i, int tile, bool importReplacement) {
|
||||
FILE* bf = fopen(nm, "wb");
|
||||
if (bf) { fwrite(loaded_texture.addr, 1, loaded_texture.size_bytes, bf); fclose(bf); }
|
||||
}
|
||||
/* D75 round 3: GE_TEXDUMP_ADDR=<hex> -- dump the raw source bytes of
|
||||
* EVERY import from that address (bypassing the 400-cap) with byte
|
||||
* stats, so a late re-import into a scratch-arena alias can be told
|
||||
* apart from a never-loaded buffer. */
|
||||
static uint64_t ge_texdump_addr = 0;
|
||||
static int ge_texdump_addr_init = 0;
|
||||
if (!ge_texdump_addr_init) {
|
||||
ge_texdump_addr_init = 1;
|
||||
const char* ea = getenv("GE_TEXDUMP_ADDR");
|
||||
if (ea && *ea) ge_texdump_addr = strtoull(ea, NULL, 16);
|
||||
}
|
||||
if (ge_texdump_addr && (uint64_t)(uintptr_t)orig_addr == ge_texdump_addr) {
|
||||
unsigned mn = 255, mx = 0, nz = 0;
|
||||
for (uint32_t i = 0; i < loaded_texture.size_bytes; i++) {
|
||||
uint8_t b = loaded_texture.addr[i];
|
||||
if (b < mn) mn = b; if (b > mx) mx = b; if (b) nz++;
|
||||
}
|
||||
char nm2[160];
|
||||
snprintf(nm2, sizeof nm2, "texdump/a%05d_%ux%u.bin", tdc - 1, tw, th);
|
||||
FILE* bf2 = fopen(nm2, "wb");
|
||||
if (bf2) { fwrite(loaded_texture.addr, 1, loaded_texture.size_bytes, bf2); fclose(bf2); }
|
||||
extern uint32_t num_dls; // same counter the D157I probe uses
|
||||
sysLogPrintf(LOG_NOTE,
|
||||
"GE_TEXA[%d] dls=%u addr=%p fmt=%u siz=%u size=%u min=%u max=%u nonzero=%u/%u file=%s",
|
||||
tdc - 1, num_dls, (void*)orig_addr, fmt, siz,
|
||||
loaded_texture.size_bytes, mn, mx, nz, loaded_texture.size_bytes, nm2);
|
||||
}
|
||||
}
|
||||
|
||||
/* D161: a CI-format tile drawn with the TLUT disabled (G_TT_NONE) must NOT
|
||||
@@ -1486,6 +1590,16 @@ static uint32_t d75d_oml = 0xFFFFFFFFu; static int d75d_oml_set = 0;
|
||||
static float d75d_zmin=1e9f,d75d_zmax=-1e9f; static uint32_t d75d_znan=0; static int d75d_use_tex=-1;
|
||||
static uint64_t d75d_comb=0; static int d75d_prim[4]={-1,-1,-1,-1}; static int d75d_env[4]={-1,-1,-1,-1};
|
||||
static int d75d_comb_utex=-1; // comb->used_textures[0] (combiner's own claim)
|
||||
// D75D round 2: logo-mode-only sampling (the frame-level fields above mix the
|
||||
// first emitted tri -- often a title quad -- with the LAST tri's combine mode,
|
||||
// which is what made 'comb claims 0' look like an anomaly). These are updated
|
||||
// on EVERY emitted tri whose rdp.combine_mode is the logo word, so comb and
|
||||
// utex always describe the same triangle.
|
||||
static int d75d_lg_utex=-1, d75d_lg_tex=-1; static uint32_t d75d_lg_cnt=0;
|
||||
static int d75d_lg_set=0; static uint32_t d75d_lg_tmem,d75d_lg_fmt,d75d_lg_siz;
|
||||
static int d75d_lg_shifts,d75d_lg_shifft,d75d_lg_uls,d75d_lg_ult,d75d_lg_lrs,d75d_lg_lrt;
|
||||
static const uint8_t* d75d_lg_texaddr=nullptr; static uint32_t d75d_lg_texid,d75d_lg_texbytes;
|
||||
static int d75d_lg_u0,d75d_lg_v0;
|
||||
|
||||
static void d75d_init(void) {
|
||||
if (d75d_init_done) return;
|
||||
@@ -1501,7 +1615,7 @@ static bool d75d_env_active(void) { d75d_init(); return d75d_lo >= 0; }
|
||||
static void d75d_flush(uint32_t f) {
|
||||
if (f == 0xFFFFFFFFu || d75d_seqlen == 0) return;
|
||||
fprintf(stderr,
|
||||
"D75D: f=%u nv=%u nt=%u maxtri=%u vtxloaded=%u batchhi=%u rej_triv=%u rej_cull=%u emitted=%u emit_on=%u emit_off=%u ndc_x=[%.3f,%.3f] ndc_y=[%.3f,%.3f] invis=%u col=[%d,%d] oml=0x%08x omset=%d z=[%.4f,%.4f] znan=%u utex=%d cutex=%d comb=0x%016llx prim=(%d,%d,%d,%d) env=(%d,%d,%d,%d) seq=%.8s%s\n",
|
||||
"D75D: f=%u nv=%u nt=%u maxtri=%u vtxloaded=%u batchhi=%u rej_triv=%u rej_cull=%u emitted=%u emit_on=%u emit_off=%u ndc_x=[%.3f,%.3f] ndc_y=[%.3f,%.3f] invis=%u col=[%d,%d] oml=0x%08x omset=%d z=[%.4f,%.4f] znan=%u utex=%d cutex=%d comb=0x%016llx prim=(%d,%d,%d,%d) env=(%d,%d,%d,%d) lg_cnt=%u lg_utex=%d lg_tex=%d lg_tile(tmem=%u fmt=%u siz=%u sh=%d st=%d uls=%d ult=%d lrs=%d lrt=%d) lg_texaddr=%p lg_texid=%u lg_texbytes=%u lg_uv0=(%d,%d) seq=%.8s%s\n",
|
||||
f, d75d_nv, d75d_nt, d75d_maxtri, d75d_vtxloaded, d75d_batchhi,
|
||||
d75d_rej_triv, d75d_rej_cull, d75d_emitted, d75d_emit_on, d75d_emit_off,
|
||||
d75d_ndc_minx, d75d_ndc_maxx, d75d_ndc_miny, d75d_ndc_maxy,
|
||||
@@ -1509,6 +1623,11 @@ static void d75d_flush(uint32_t f) {
|
||||
d75d_zmin, d75d_zmax, d75d_znan, d75d_use_tex, d75d_comb_utex,
|
||||
(unsigned long long)d75d_comb, d75d_prim[0],d75d_prim[1],d75d_prim[2],d75d_prim[3],
|
||||
d75d_env[0],d75d_env[1],d75d_env[2],d75d_env[3],
|
||||
d75d_lg_cnt, d75d_lg_utex, d75d_lg_tex,
|
||||
(unsigned)d75d_lg_tmem,(unsigned)d75d_lg_fmt,(unsigned)d75d_lg_siz,
|
||||
d75d_lg_shifts,d75d_lg_shifft,d75d_lg_uls,d75d_lg_ult,d75d_lg_lrs,d75d_lg_lrt,
|
||||
(const void*)d75d_lg_texaddr,(unsigned)d75d_lg_texid,(unsigned)d75d_lg_texbytes,
|
||||
d75d_lg_u0,d75d_lg_v0,
|
||||
d75d_seq, (d75d_seqlen >= 4096) ? "...[trunc]" : "");
|
||||
d75d_seqlen = 0; d75d_maxtri = 0; d75d_vtxloaded = 0; d75d_batchhi = 0; d75d_nv = 0; d75d_nt = 0;
|
||||
d75d_rej_triv = 0; d75d_rej_cull = 0; d75d_emitted = 0; d75d_emit_on = 0; d75d_emit_off = 0;
|
||||
@@ -1516,6 +1635,7 @@ static void d75d_flush(uint32_t f) {
|
||||
d75d_emit_invis = 0; d75d_cmin=999; d75d_cmax=-1; d75d_oml=0xFFFFFFFFu; d75d_oml_set=0;
|
||||
d75d_zmin=1e9f;d75d_zmax=-1e9f;d75d_znan=0;d75d_use_tex=-1;
|
||||
d75d_comb=0; for(int i=0;i<4;i++){d75d_prim[i]=-1;d75d_env[i]=-1;} d75d_comb_utex=-1;
|
||||
d75d_lg_utex=-1; d75d_lg_tex=-1;
|
||||
}
|
||||
static void d75d_frame_check(uint32_t f) {
|
||||
if (f != d75d_frame) { d75d_flush(f); d75d_frame = f; }
|
||||
@@ -1543,6 +1663,8 @@ static void d75d_note_rej(uint32_t f, int which) {
|
||||
d75d_frame_check(f);
|
||||
if (which == 0) d75d_rej_triv++; else d75d_rej_cull++;
|
||||
}
|
||||
static inline int gfx_lod_tile_offset(const int i); // fwd (defined below; D75D round 2)
|
||||
|
||||
static void d75d_note_emit(uint32_t f, struct LoadedVertex* const* v_arr) {
|
||||
d75d_init();
|
||||
if (d75d_lo < 0 || f < (uint32_t)d75d_lo || f > (uint32_t)d75d_hi) return;
|
||||
@@ -1585,6 +1707,24 @@ static void d75d_note_emit_z(uint32_t f, struct LoadedVertex* const* v_arr, int
|
||||
}
|
||||
if (d75d_use_tex < 0) d75d_use_tex = has_tex;
|
||||
if (d75d_comb_utex < 0) d75d_comb_utex = comb_utex;
|
||||
if (rdp.combine_mode == 0x009ffe4f19ffe4f1ULL) {
|
||||
d75d_lg_utex = comb_utex; d75d_lg_tex = has_tex; d75d_lg_cnt++;
|
||||
if (!d75d_lg_set && v_arr[0] && v_arr[0]->w != 0.f) {
|
||||
d75d_lg_set = 1;
|
||||
const uint32_t t0 = rdp.first_tile_index + gfx_lod_tile_offset(0);
|
||||
d75d_lg_tmem = rdp.texture_tile[t0].tmem; d75d_lg_fmt = rdp.texture_tile[t0].fmt;
|
||||
d75d_lg_siz = rdp.texture_tile[t0].siz;
|
||||
d75d_lg_shifts = rdp.texture_tile[t0].shifts; d75d_lg_shifft = rdp.texture_tile[t0].shiftt;
|
||||
d75d_lg_uls = rdp.texture_tile[t0].uls; d75d_lg_ult = rdp.texture_tile[t0].ult;
|
||||
d75d_lg_lrs = rdp.texture_tile[t0].lrs; d75d_lg_lrt = rdp.texture_tile[t0].lrt;
|
||||
if (rendering_state.textures[0]) {
|
||||
d75d_lg_texaddr = rendering_state.textures[0]->first.texture_addr;
|
||||
d75d_lg_texid = rendering_state.textures[0]->second.texture_id;
|
||||
d75d_lg_texbytes = rendering_state.textures[0]->first.size_bytes;
|
||||
}
|
||||
d75d_lg_u0 = (int)v_arr[0]->u; d75d_lg_v0 = (int)v_arr[0]->v;
|
||||
}
|
||||
}
|
||||
d75d_comb = rdp.combine_mode;
|
||||
d75d_prim[0]=rdp.prim_color.r; d75d_prim[1]=rdp.prim_color.g; d75d_prim[2]=rdp.prim_color.b; d75d_prim[3]=rdp.prim_color.a;
|
||||
d75d_env[0]=rdp.env_color.r; d75d_env[1]=rdp.env_color.g; d75d_env[2]=rdp.env_color.b; d75d_env[3]=rdp.env_color.a;
|
||||
@@ -1922,7 +2062,38 @@ static inline int gfx_lod_tile_offset(const int i) {
|
||||
// fire @ TMEM 0x188. Returning 0 here fed TEXEL1 the smoke texture too
|
||||
// (smoke * smoke) -> the magenta/cyan particle colour. Only fold to the
|
||||
// base tile when LOD is actually active.
|
||||
return rdp.tex_lod ? 0 : i;
|
||||
//
|
||||
// D236 pass 26: "GE loads the whole mip chain at TMEM 0" holds for the
|
||||
// TEXTURETYPE_LOD / TEXTURETYPE_MIPMAP bindings (texHandleType0 /
|
||||
// texHandleType2 both do texWriteLoadToTmemAddr(tex, 0)), but NOT for
|
||||
// TEXTURETYPE_DETAIL. texHandleType1 (tex.c) loads the DETAIL texture at
|
||||
// TMEM 0 and the real base image at TMEM offset texGetSizeInBytes(tex2,0)
|
||||
// on tiles 1+. Folding to tile 0 there samples the detail texture instead
|
||||
// of the base image. On Surface 1 that is literally the D236 bug: all four
|
||||
// treeline cards (tex1198-1201, RGBA5551 64x17 cut-outs) are type-1
|
||||
// bindings whose detail texture is tex2465, an opaque 32x32 IA8 noise
|
||||
// tile -- so the treeline draws as an opaque tiled noise wall and the
|
||||
// cards themselves are never even imported (pass 25's "zero fmt=0 siz=2
|
||||
// imports in 4800 frames").
|
||||
//
|
||||
// Discriminator: tile fi+1's declared FORMAT. Tiles 1.. of a type-0/type-2
|
||||
// binding are LOD levels of tile 0's image, so they necessarily carry the
|
||||
// same fmt/siz; a type-1 pair is two unrelated textures and generally does
|
||||
// not (Surface 1's cards: tile 0 = tex2465 IA8, tile 1 = tex1198 RGBA16).
|
||||
// Deliberately asymmetric -- a mip chain can never trip this, and a detail
|
||||
// pair that happens to share a format just keeps today's behaviour -- so
|
||||
// the D107 mip case is safe by construction.
|
||||
if (rdp.tex_lod) {
|
||||
const uint32_t fi = rdp.first_tile_index;
|
||||
if (g_detail_base_tile && fi + 1 < 8 &&
|
||||
rdp.texture_tile[fi + 1].tmem != rdp.texture_tile[fi].tmem &&
|
||||
(rdp.texture_tile[fi + 1].fmt != rdp.texture_tile[fi].fmt ||
|
||||
rdp.texture_tile[fi + 1].siz != rdp.texture_tile[fi].siz)) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
return i;
|
||||
}
|
||||
|
||||
static void gfx_sp_tri1(uint8_t vtx1_idx, uint8_t vtx2_idx, uint8_t vtx3_idx, bool is_rect) {
|
||||
@@ -2105,6 +2276,40 @@ static void gfx_sp_tri1(uint8_t vtx1_idx, uint8_t vtx2_idx, uint8_t vtx3_idx, bo
|
||||
* correlated against "close" vs "far" the same way pass 13's live
|
||||
* capture was read. Zero cost unset. Remove once D236 pass 14
|
||||
* concludes. */
|
||||
/* D236 pass 17 (TEMP): six passes of signature-matching (11-16) failed
|
||||
* to identify the CPU-side writer of the tree class's raw-pointer vertex
|
||||
* batches. Stop guessing: on the first triangle(s) of this class whose
|
||||
* vertices came from a raw (unsegmented, seg 0x70) G_VTX load, capture a
|
||||
* host stack backtrace -- the C call chain that submitted this display
|
||||
* list IS the writer's caller, no inference needed. Windows-only (this
|
||||
* build): RtlCaptureContext + RtlVirtualUnwind over the exe's own .pdata
|
||||
* table -- this box's dbghelp.dll is stripped (no CaptureStackBackTrace;
|
||||
* StackWalk64 fails immediately), so ntdll does the unwind and there is
|
||||
* no dbghelp dependency at all. Frames print as RVAs, resolved offline
|
||||
* with addr2line against this exact exe. Env-gated, fires at most once
|
||||
* per distinct raw pointer (cap 5), zero cost unset. Remove once D236
|
||||
* pass 17 concludes. */
|
||||
#ifdef _WIN32
|
||||
static int s_d236bt = -1;
|
||||
if (s_d236bt < 0) s_d236bt = getenv("GE_D236BT") != NULL;
|
||||
if (s_d236bt && rdp.other_mode_l == 0x0c184b50u && g_d236_last_vtx_seg == 0x70) {
|
||||
static uint32_t d236bt_seen[5] = {0, 0, 0, 0, 0};
|
||||
static int d236bt_n = 0;
|
||||
int dup = 0;
|
||||
for (int i = 0; i < d236bt_n; i++) {
|
||||
if (d236bt_seen[i] == g_d236_last_vtx_raw) { dup = 1; break; }
|
||||
}
|
||||
if (!dup && d236bt_n < 5) {
|
||||
d236bt_seen[d236bt_n++] = g_d236_last_vtx_raw;
|
||||
fprintf(stderr,
|
||||
"D236BT rawvtx=0x%08x a=(%u,%u,%u) invw1=%.4f\n", g_d236_last_vtx_raw,
|
||||
v1->color.a, v2->color.a, v3->color.a,
|
||||
(v1->w != 0.f) ? 1.f / v1->w : 0.f);
|
||||
port_d236bt_dump("rsp");
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
static int s_d236alpha = -1;
|
||||
if (s_d236alpha < 0) s_d236alpha = getenv("GE_D236ALPHA") != NULL;
|
||||
if (s_d236alpha && rdp.other_mode_l == 0x0c184b50u) {
|
||||
@@ -3609,6 +3814,7 @@ static void gfx_run_dl(Gfx* cmd) {
|
||||
* model/CPU-built-quad source) without guessing from source
|
||||
* review alone. Remove once D236 pass 16 concludes. */
|
||||
g_d236_last_vtx_seg = (uint8_t)(cmd->words.w1 >> 24);
|
||||
g_d236_last_vtx_raw = cmd->words.w1;
|
||||
gfx_sp_vertex(C0(0, 16) / sizeof(Vtx), C0(16, 4), (const Vtx*)seg_addr(cmd->words.w1));
|
||||
d75d_note_vtx(videoGetFrameCount(), C0(0, 16) / sizeof(Vtx), C0(16, 4));
|
||||
break;
|
||||
@@ -4102,6 +4308,8 @@ extern "C" void gfx_set_mipmap_filter(enum MipmapFilteringMode mode) {
|
||||
|
||||
extern "C" void gfx_set_fix_mip_textures(int on) { g_fix_mip_textures = !!on; }
|
||||
|
||||
extern "C" void gfx_set_detail_base_tile(int on) { g_detail_base_tile = !!on; }
|
||||
|
||||
/* D212: expose the (already-implemented) rendering-API anisotropy hook to the
|
||||
* port layer. Clamp to [1, GL max] so a stale ini value can't feed an invalid
|
||||
* GL_TEXTURE_MAX_ANISOTROPY. 1 = isotropic (driver default). */
|
||||
|
||||
@@ -58,6 +58,7 @@ static int cfgFpsCap = 60; /* frame cap in fps; 0 = uncapped (vsync); me
|
||||
static int cfgMSAA = 4; /* 1/2/4/8 samples; default 4 (modern ports ship AA on; snaps down to the highest supported level) */
|
||||
static int cfgTexFilter = 1; /* 0 = nearest, 1 = bilinear (default), 2 = N64 3-point + trilinear */
|
||||
static int cfgFixMipTex = 1; /* RC2: clip mip-contaminated texture uploads to base height */
|
||||
static int cfgDetailBaseTile = 1; /* D236: TEXTURETYPE_DETAIL -> sample the base image, not the detail tile */
|
||||
static int cfgWrapFix = 0; /* D74 sub-tile UV pre-wrap + RC3/D167 non-PoT mask-period wrap (opt-in; GE_WRAPFIX env overrides) */
|
||||
static int cfgFovScale = 100; /* D211: percent of the original vertical FOV; 100 = unchanged (byte-identical) */
|
||||
static int cfgWidescreenAuto = 1; /* WIDESCREEN-FOV-PLAN Phase 4: auto-scale vertical FOV by window aspect ratio; on by default, no-op at 4:3 */
|
||||
@@ -240,6 +241,7 @@ PD_CONSTRUCTOR static void videoConfigInit(void)
|
||||
configRegisterInt("Video.MSAA", &cfgMSAA, 1, 8);
|
||||
configRegisterInt("Video.TextureFilter", &cfgTexFilter, 0, 2);
|
||||
configRegisterInt("Video.FixMipTextures", &cfgFixMipTex, 0, 1);
|
||||
configRegisterInt("Video.DetailBaseTile", &cfgDetailBaseTile, 0, 1);
|
||||
configRegisterInt("Video.WrapFix", &cfgWrapFix, 0, 1);
|
||||
configRegisterInt("Video.FovScale", &cfgFovScale, 50, 150);
|
||||
configRegisterInt("Video.WidescreenAuto", &cfgWidescreenAuto, 0, 1);
|
||||
@@ -460,6 +462,7 @@ int videoInit(void)
|
||||
* NOT fix the Depot roof, see docs/BRIEF-B2-depot-textures.md). All keep
|
||||
* point-sampled tiles (HUD, G_TF_POINT) crisp via the per-tile flag. */
|
||||
gfx_set_fix_mip_textures(cfgFixMipTex);
|
||||
gfx_set_detail_base_tile(cfgDetailBaseTile);
|
||||
gfx_set_wrap_fix(cfgWrapFix);
|
||||
|
||||
videoApplyTexFilter();
|
||||
|
||||
@@ -1266,8 +1266,37 @@ s32 get_numguards(void)
|
||||
}
|
||||
|
||||
|
||||
/* D236 pass 17b (TEMP): env-gated backtrace at the vertex allocator --
|
||||
* every dynamic-quad builder in model.c (dotube/doshadow/dorottex/
|
||||
* dogfnegx) funnels through here, so a stack capture on this call names
|
||||
* the actual writer of a raw-pointer G_VTX batch. Correlate the printed
|
||||
* physical address with the RSP-side GE_D236BT rawvtx captures. Port-side
|
||||
* dumper: port/fast3d/gfx_pc.cpp (port_d236bt_dump). Zero cost unset.
|
||||
* Remove once D236 pass 17 concludes. */
|
||||
void get_ptr_allocated_block_for_vertices(int param_1)
|
||||
{
|
||||
#ifdef PORT
|
||||
static int s_d236bt = -1;
|
||||
if (s_d236bt < 0) s_d236bt = getenv("GE_D236BT") != NULL;
|
||||
if (s_d236bt) {
|
||||
extern void port_d236bt_dump(const char* tag);
|
||||
static int d236bt_allocs = 0;
|
||||
if (d236bt_allocs < 8) {
|
||||
d236bt_allocs++;
|
||||
Vtx* p = (Vtx*)dynAllocate(param_1 << 4);
|
||||
fprintf(stderr, "D236BT alloc n=%d bytes=0x%x phys=0x%08x\n",
|
||||
param_1, param_1 << 4, (unsigned)osVirtualToPhysical(p));
|
||||
port_d236bt_dump("alloc");
|
||||
/* CRITICAL: callers do `vtx = (Vtx*)vtxallocator(n)` on this
|
||||
* void function and read RAX (works on the unprobed path only
|
||||
* because nothing clobbers it between dynAllocate and ret).
|
||||
* Pin p in RAX so the probe's fprintf/unwind calls don't
|
||||
* hand the caller garbage. */
|
||||
__asm__ volatile("" : "+a" (p));
|
||||
return;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
dynAllocate(param_1 << 4);
|
||||
}
|
||||
|
||||
|
||||
@@ -5470,6 +5470,43 @@ void doshadow(ModelRenderData *renderdata, Model *model, ModelNode *node)
|
||||
}
|
||||
#endif
|
||||
|
||||
#ifdef PORT
|
||||
/* D236 pass 17c (TEMP): env-gated runtime dump of the live ShadowRecord +
|
||||
* tconfig, to settle which fields the PC loader actually populated (the
|
||||
* d43_emit.py op-13 emit and modelPromoteNodeOffsetsToPointers disagree
|
||||
* on paper; this shows ground truth). Zero cost unset. Remove once D236
|
||||
* pass 17 concludes. */
|
||||
{
|
||||
static int s_d236bt = -1;
|
||||
if (s_d236bt < 0) s_d236bt = getenv("GE_D236BT") != NULL;
|
||||
if (s_d236bt) {
|
||||
static int d236sh_n = 0;
|
||||
if (d236sh_n < 6) {
|
||||
d236sh_n++;
|
||||
if (d236sh_n == 1) {
|
||||
FILE *f = fopen("d236sh.bin", "wb");
|
||||
if (f) {
|
||||
fwrite(shadow, 1, 0x24, f); /* live ShadowRecord */
|
||||
fwrite(&node->Opcode, 1, 8, f); /* node opcode+pad */
|
||||
fwrite(node->Data, 1, 0x24, f); /* record via Data ptr */
|
||||
fwrite(shadow->BaseAddr, 1, 0x60, f); /* file header */
|
||||
fclose(f);
|
||||
}
|
||||
}
|
||||
sImageTableEntry *tc = (sImageTableEntry *)shadow->image;
|
||||
fprintf(stderr,
|
||||
"D236SH node=%p img=%p hdr=%p base=%p height=%.1f size=(%.1f,%.1f) a=%u\n"
|
||||
" tc: idx=0x%08x %ux%u lvl=%u fmt=%u depth=%u fs=%u ft=%u\n",
|
||||
(void*)node, (void*)shadow->image, (void*)shadow->HeaderNode,
|
||||
(void*)shadow->BaseAddr, height, sizex, sizey,
|
||||
(unsigned)vtxtemplate.v.cn[3],
|
||||
(unsigned)tc->index, tc->width, tc->height, tc->level,
|
||||
tc->format, tc->depth, tc->flagsS, tc->flagsT);
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
vtx = (Vtx *) vtxallocator(4);
|
||||
|
||||
vtx[0] = vtxtemplate;
|
||||
|
||||
@@ -0,0 +1,250 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D236 pass 22: offline census of a level's bg room display lists.
|
||||
|
||||
Decodes `bg/bg_<level>_all_p.seg` straight out of the ROM and walks every
|
||||
room's primary and secondary GDL, reporting the distinct render-mode
|
||||
(`G_SETOTHERMODE_L`, i.e. the `oml` words the live `GE_D2xx` probes log) and
|
||||
texture-image classes each room draws. This answers "which bg draw class
|
||||
carries the trees, and which room draws it" without building or running the
|
||||
game.
|
||||
|
||||
Pipeline (mirrors `bg.c` exactly):
|
||||
* the .seg file is stored UNCOMPRESSED in ROM (obLoadBGFileBytesAtOffset
|
||||
random-accesses it by byte offset); only the per-room blobs inside it are
|
||||
deflated;
|
||||
* word[1] of the decompressed file is the segment offset of the
|
||||
`bg_room_data` array (`bg.c:865`), 24 bytes/record:
|
||||
u32 pPointTableBin, u32 pPriMappingBin, u32 pSecMappingBin, coord3d pos;
|
||||
* a room's compressed DL size is the distance to the *next* non-zero
|
||||
mapping-bin offset, taking primary and secondary as one interleaved
|
||||
stream (`bg.c:944-990` + `getPri/SecMappingBinCount`);
|
||||
* each blob is itself raw-deflate behind its own 2-byte header
|
||||
(`bgDecompress` -> `decompressdata`).
|
||||
|
||||
The words reported are the ones **as authored in the file**, i.e. BEFORE
|
||||
`bgApplyDynamicCCRMLUT` rewrites them at room load. The `->` column shows
|
||||
what the applied LUT turns each one into, so the output can be compared
|
||||
directly against what the runtime probes log. Only LUTs 1/5/6/7 are ever
|
||||
applied (`bg.c:2797/2804/2812/2819`); see D236 pass 21.
|
||||
|
||||
Usage: python tools_pc/bg_gdl_census.py [sevx] [ntsc-final]
|
||||
"""
|
||||
import collections
|
||||
import contextlib
|
||||
import io
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import zlib
|
||||
|
||||
import rendermode
|
||||
import texheader
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, os.path.join(ROOT, "tools_pc"))
|
||||
|
||||
G_SETOTHERMODE_L = 0xB9
|
||||
G_SETTIMG = 0xFD
|
||||
G_NOOP = 0xC0
|
||||
G_ENDDL = 0xB8
|
||||
G_DL = 0xDE
|
||||
# gsDPSetRenderMode's w0: SETOTHERMODE_L, shift 3, length 29 -> the render-mode
|
||||
# field. Other 0xB9 commands (alpha compare, z source) use different shifts
|
||||
# and are counted separately.
|
||||
RENDERMODE_W0 = 0xB900031D
|
||||
|
||||
IM_FMT = {0: "RGBA", 1: "YUV", 2: "CI", 3: "IA", 4: "I"}
|
||||
IM_SIZ = {0: "4b", 1: "8b", 2: "16b", 3: "32b"}
|
||||
|
||||
|
||||
def load_rom_file(name, region):
|
||||
"""Decompress one ROM file table entry by name."""
|
||||
sys.argv = ["", region]
|
||||
buf = io.StringIO()
|
||||
with contextlib.redirect_stdout(buf):
|
||||
import d43_emit as m
|
||||
key, row = m.find_row(name)
|
||||
if not key:
|
||||
raise SystemExit("not in ROM file table: %s" % name)
|
||||
addr, size = row
|
||||
raw = m.rom[addr:addr + size]
|
||||
# Model files are raw-deflate behind a 2-byte 0x1172 header, but the bg
|
||||
# .seg files are stored UNCOMPRESSED in ROM -- they have to be, because
|
||||
# obLoadBGFileBytesAtOffset() random-accesses them by byte offset to pull
|
||||
# one room's blob at a time. Only the per-room blobs inside are deflated.
|
||||
try:
|
||||
return zlib.decompress(raw[2:], -15)
|
||||
except zlib.error:
|
||||
return raw
|
||||
|
||||
|
||||
def seg_off(v):
|
||||
"""Segment-0x0F address -> file offset (bg.c's BG_SEG_TO_PTR)."""
|
||||
return v & 0x00FFFFFF
|
||||
|
||||
|
||||
def read_rooms(data):
|
||||
"""Parse the bg_room_data array: [(point, pri, sec, pos), ...]."""
|
||||
list_off = seg_off(struct.unpack_from(">I", data, 4)[0])
|
||||
rooms = []
|
||||
off = list_off
|
||||
while off + 24 <= len(data):
|
||||
point, pri, sec = struct.unpack_from(">III", data, off)
|
||||
pos = struct.unpack_from(">fff", data, off + 12)
|
||||
rooms.append((point, pri, sec, pos))
|
||||
# The array is terminated by the sentinel record whose pPriMappingBin
|
||||
# is 0 AND which is not just an interior room with no primary DL, so
|
||||
# walk until all three offsets are zero.
|
||||
if len(rooms) > 1 and point == 0 and pri == 0 and sec == 0:
|
||||
break
|
||||
off += 24
|
||||
return rooms, list_off
|
||||
|
||||
|
||||
def next_nonzero(rooms, start, field):
|
||||
i = start
|
||||
while i < len(rooms) and rooms[i][field] == 0:
|
||||
i += 1
|
||||
return i if i < len(rooms) else None
|
||||
|
||||
|
||||
def blob_sizes(rooms, i):
|
||||
"""csize_primary / csize_secondary for room i, per bg.c:944-990."""
|
||||
_point, pri, sec, _pos = rooms[i]
|
||||
csz_pri = csz_sec = 0
|
||||
if pri:
|
||||
pidx = next_nonzero(rooms, i + 1, 1)
|
||||
sidx = next_nonzero(rooms, i, 2)
|
||||
if pidx is not None and (sidx is None or pidx <= sidx):
|
||||
csz_pri = seg_off(rooms[pidx][1]) - seg_off(pri)
|
||||
elif sidx is not None:
|
||||
csz_pri = seg_off(rooms[sidx][2]) - seg_off(pri)
|
||||
if sec:
|
||||
pidx = next_nonzero(rooms, i + 1, 1)
|
||||
sidx = next_nonzero(rooms, i + 1, 2)
|
||||
if pidx is not None and (sidx is None or pidx <= sidx):
|
||||
csz_sec = seg_off(rooms[pidx][1]) - seg_off(sec)
|
||||
elif sidx is not None:
|
||||
csz_sec = seg_off(rooms[sidx][2]) - seg_off(sec)
|
||||
return csz_pri, csz_sec
|
||||
|
||||
|
||||
def inflate_blob(data, off, size):
|
||||
if size <= 2:
|
||||
return None
|
||||
try:
|
||||
return zlib.decompress(data[off + 2:off + size], -15)
|
||||
except zlib.error as e:
|
||||
return ("ERR", str(e))
|
||||
|
||||
|
||||
def walk_gdl(gdl):
|
||||
"""Walk an 8-byte big-endian N64 Gfx stream.
|
||||
|
||||
Returns (rendermodes, textures, opcodes, ncmd) where rendermodes maps an
|
||||
oml word to how many draw-setup commands used it, and textures maps
|
||||
(fmt, siz) to the set of image addresses set while it was active.
|
||||
"""
|
||||
rms = collections.Counter()
|
||||
texs = collections.Counter()
|
||||
ops = collections.Counter()
|
||||
pairs = collections.Counter()
|
||||
cur_rm = None
|
||||
n = len(gdl) // 8
|
||||
for k in range(n):
|
||||
w0, w1 = struct.unpack_from(">II", gdl, k * 8)
|
||||
op = w0 >> 24
|
||||
ops[op] += 1
|
||||
if op == G_SETOTHERMODE_L and w0 == RENDERMODE_W0:
|
||||
cur_rm = w1
|
||||
rms[w1] += 1
|
||||
elif op == G_NOOP:
|
||||
# bg room DLs never carry G_SETTIMG -- they reference textures by
|
||||
# NUMBER in a G_NOOP, which texLoadFromGdl() resolves into real
|
||||
# texture commands at room load (tex.c:779, `case G_NOOP:
|
||||
# texnum = in->words.w1 & 0xfff`).
|
||||
texs[w1 & 0xFFF] += 1
|
||||
pairs[(cur_rm, w1 & 0xFFF)] += 1
|
||||
return rms, texs, ops, pairs, n
|
||||
|
||||
|
||||
def report(level, region):
|
||||
name = "bg_%s_all_p" % level # file-table basename (no dir/.seg)
|
||||
data = load_rom_file(name, region)
|
||||
print("%s: %d bytes decompressed" % (name, len(data)))
|
||||
rooms, list_off = read_rooms(data)
|
||||
print("room table at file offset 0x%x, %d records\n" % (list_off, len(rooms)))
|
||||
|
||||
all_rms = collections.Counter()
|
||||
all_pairs = collections.Counter()
|
||||
rm_rooms = collections.defaultdict(set)
|
||||
tex_rooms = collections.defaultdict(set)
|
||||
bad = 0
|
||||
for i in range(1, len(rooms)):
|
||||
csz_pri, csz_sec = blob_sizes(rooms, i)
|
||||
for label, off_raw, csz in (("pri", rooms[i][1], csz_pri),
|
||||
("sec", rooms[i][2], csz_sec)):
|
||||
if not off_raw or csz <= 0:
|
||||
continue
|
||||
out = inflate_blob(data, seg_off(off_raw), csz)
|
||||
if out is None:
|
||||
continue
|
||||
if isinstance(out, tuple):
|
||||
bad += 1
|
||||
print("room %3d %s: INFLATE FAILED at 0x%x size %d (%s)"
|
||||
% (i, label, seg_off(off_raw), csz, out[1]))
|
||||
continue
|
||||
rms, texs, ops, pairs, ncmd = walk_gdl(out)
|
||||
all_rms.update(rms)
|
||||
all_pairs.update(pairs)
|
||||
for w in rms:
|
||||
rm_rooms[w].add(i)
|
||||
for t in texs:
|
||||
tex_rooms[t].add(i)
|
||||
print("room %3d %s: csize=%-6d usize=%-6d cmds=%-5d rm=%d tex=%d %s"
|
||||
% (i, label, csz, len(out), ncmd, len(rms), sum(texs.values()),
|
||||
" ".join("0x%08x" % w for w in sorted(rms))))
|
||||
|
||||
print("\n%s" % ("=" * 78))
|
||||
rm = rendermode.RenderModes()
|
||||
luts = rendermode.applied_luts(rm)
|
||||
print("RENDER-MODE CENSUS (as authored in the file, pre-LUT)")
|
||||
print(" the '->' column is what bgApplyDynamicCCRMLUT rewrites the word to at")
|
||||
print(" room load when fog is on (LUT 1 primary / LUT 5 secondary) -- i.e. the")
|
||||
print(" value a live GE_D2xx probe would log. With fog OFF, LUTs 6/7 rewrite")
|
||||
print(" COMBINE modes only, so render modes stay exactly as authored.")
|
||||
print("=" * 78)
|
||||
for w, n in all_rms.most_common():
|
||||
rr = sorted(rm_rooms[w])
|
||||
to = luts.get(1, {}).get(w) or luts.get(5, {}).get(w)
|
||||
print(" 0x%08x -> %-12s %4d setups %3d rooms rooms=%s"
|
||||
% (w, ("0x%08x" % to) if to else "(unchanged)", n, len(rr),
|
||||
",".join(map(str, rr[:14])) + ("..." if len(rr) > 14 else "")))
|
||||
print(" %s" % " | ".join(rm.decode(w)["flags"]))
|
||||
|
||||
print("\n%s" % ("=" * 78))
|
||||
print("TEXTURES PER RENDER-MODE CLASS (texture numbers from G_NOOP w1)")
|
||||
print(" ROM header decoded offline; 'live' is what a runtime probe prints.")
|
||||
print("=" * 78)
|
||||
rom, trows = texheader.load(region)
|
||||
bytex = collections.defaultdict(collections.Counter)
|
||||
for (w, t), _n in all_pairs.items():
|
||||
bytex[w][t] = len(tex_rooms[t])
|
||||
for w, _n in all_rms.most_common():
|
||||
print(" rm=0x%08x" % w)
|
||||
for t, nrooms in bytex[w].most_common():
|
||||
try:
|
||||
desc = texheader.describe(t, rom, trows)
|
||||
except Exception:
|
||||
desc = "tex%-5d (not in image list)" % t
|
||||
print(" %2d rooms %s" % (nrooms, desc))
|
||||
|
||||
if bad:
|
||||
print("\n%d blob(s) failed to inflate -- size derivation may be off for those."
|
||||
% bad)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
lvl = sys.argv[1] if len(sys.argv) > 1 else "sevx"
|
||||
reg = sys.argv[2] if len(sys.argv) > 2 else "ntsc-final"
|
||||
report(lvl, reg)
|
||||
+19
-2
@@ -402,8 +402,11 @@ def process(name):
|
||||
merged[-1][1] = max(merged[-1][1], iv[1])
|
||||
else:
|
||||
merged.append(iv)
|
||||
blob_regions = [] # (old_off, old_size, new_off) raw pixel spans
|
||||
for (t, e) in merged:
|
||||
n = dstpos
|
||||
dstpos = add_region(t, e - t, dstpos)
|
||||
blob_regions.append((t, e - t, n))
|
||||
|
||||
# GDLs last, tight-packed 16B slots. Visited GDLs go in VISIT order (the
|
||||
# compaction span arithmetic requires it). Referenced-but-unvisited GDLs
|
||||
@@ -669,9 +672,17 @@ def process(name):
|
||||
j += 1
|
||||
o += 8
|
||||
|
||||
# Embedded image blobs: raw pixel copy (the RDP reads them as-is; no
|
||||
# transform). Layout reserves the span and remaps G_SETTIMG w1 to it, but
|
||||
# without this copy the buffer stays zeroed and the texture renders black
|
||||
# (M-197 / D75: PnintendologoZ white wordmark invisible on PC).
|
||||
for (o, osz, n) in blob_regions:
|
||||
buf[n:n + osz] = src[o:o + osz]
|
||||
|
||||
# ---- round-trip validation ----
|
||||
validate(name, src, D, NS, NT, nodes, R0, placed, all_gdls, buf, D_PC,
|
||||
regions, node_newoff, rec_newoff, gdl_newoff, remap, inv_remap)
|
||||
regions, blob_regions, node_newoff, rec_newoff, gdl_newoff,
|
||||
remap, inv_remap)
|
||||
|
||||
# ---- compress ----
|
||||
co = zlib.compressobj(6, zlib.DEFLATED, -15)
|
||||
@@ -693,9 +704,15 @@ op24_is_collision = {}
|
||||
op24_pointusage = {}
|
||||
|
||||
def validate(name, src, D, NS, NT, nodes, R0, placed, all_gdls, buf, D_PC,
|
||||
regions, node_newoff, rec_newoff, gdl_newoff, remap, inv_remap):
|
||||
regions, blob_regions, node_newoff, rec_newoff, gdl_newoff,
|
||||
remap, inv_remap):
|
||||
def err(m): errors.append(f"{name}: RT {m}")
|
||||
|
||||
# embedded image blobs: raw identity round-trip (M-197)
|
||||
for (o, osz, n) in blob_regions:
|
||||
if bytes(buf[n:n + osz]) != bytes(src[o:o + osz]):
|
||||
err(f"blob {o:#x}+{osz:#x}: pixel bytes differ at new {n:#x}")
|
||||
|
||||
# tiling: regions cover [0, D_PC) exactly
|
||||
rs = sorted(regions, key=lambda x: x[2])
|
||||
pos = 0
|
||||
|
||||
@@ -49,7 +49,13 @@ def doc_vars() -> tuple[set[str], set[str]]:
|
||||
return set(), set()
|
||||
named, tombstone = set(), set()
|
||||
for line in DOC.read_text(encoding="utf-8").splitlines():
|
||||
vs = re.findall(r"GE_[A-Z0-9_]+", line)
|
||||
# `GE_FOO[` in prose is a probe's LOG-LINE TAG, not an env var -- several
|
||||
# probes print a tag that is not the name of any getenv() (e.g.
|
||||
# GE_TEXDUMP_ADDR prints "GE_TEXA[...]"). Counting those as documented
|
||||
# vars reports a permanent phantom GONE. A row for a real var always
|
||||
# names it plainly in the table's first cell, so it is still picked up.
|
||||
vs = [m.group(1) for m in re.finditer(r"(GE_[A-Z0-9_]+)(\[?)", line)
|
||||
if not m.group(2)]
|
||||
named.update(vs)
|
||||
if "tombstone" in line.lower():
|
||||
tombstone.update(vs)
|
||||
|
||||
@@ -0,0 +1,225 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D236 pass 22: evaluate N64 render-mode words offline, straight from the repo.
|
||||
|
||||
Two jobs:
|
||||
|
||||
1. `decode(word)` -- turn a raw `G_SETOTHERMODE_L` value (the `oml` the live
|
||||
`GE_D2xx` probes print) into the flag names and blender equation it stands
|
||||
for, and name it if it matches a `G_RM_*` composite in `include/PR/gbi.h`.
|
||||
|
||||
2. `applied_luts()` -- evaluate the `DL_LUT_*` tables in `src/game/bg.c` into
|
||||
plain (from -> to) word maps, for the four LUT indices `bg.c` actually
|
||||
applies at room load (1/5/6/7). This is what lets a *static* census of a
|
||||
bg file's authored render modes be compared against the words a *live*
|
||||
probe logs, which are the post-rewrite ones.
|
||||
|
||||
Everything is parsed from the real headers/sources rather than hardcoded, so
|
||||
it cannot drift from the build. No C compiler required.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
GBI_H = os.path.join(ROOT, "include", "PR", "gbi.h")
|
||||
BG_C = os.path.join(ROOT, "src", "game", "bg.c")
|
||||
|
||||
# Order matters for readable decoding: mask-style fields first.
|
||||
_FIELDS = [
|
||||
("CVG_DST", 0x300, {0x000: "CVG_DST_CLAMP", 0x100: "CVG_DST_WRAP",
|
||||
0x200: "CVG_DST_FULL", 0x300: "CVG_DST_SAVE"}),
|
||||
("ZMODE", 0xC00, {0x000: "ZMODE_OPA", 0x400: "ZMODE_INTER",
|
||||
0x800: "ZMODE_XLU", 0xC00: "ZMODE_DEC"}),
|
||||
]
|
||||
_BITS = [(0x8, "AA_EN"), (0x10, "Z_CMP"), (0x20, "Z_UPD"), (0x40, "IM_RD"),
|
||||
(0x80, "CLR_ON_CVG"), (0x1000, "CVG_X_ALPHA"),
|
||||
(0x2000, "ALPHA_CVG_SEL"), (0x4000, "FORCE_BL")]
|
||||
|
||||
_BL_P = {0: "CLR_IN", 1: "CLR_MEM", 2: "CLR_BL", 3: "CLR_FOG"}
|
||||
_BL_A = {0: "A_IN/1MA", 1: "A_FOG/A_MEM", 2: "A_SHADE/1", 3: "0"}
|
||||
|
||||
|
||||
def _flag_defines():
|
||||
"""Simple `#define NAME <int literal>` pairs from gbi.h."""
|
||||
out = {}
|
||||
for line in open(GBI_H, encoding="utf-8", errors="replace"):
|
||||
m = re.match(r"#define\s+(\w+)\s+(0x[0-9a-fA-F]+|\d+)\s*(/\*.*)?$", line)
|
||||
if m:
|
||||
out[m.group(1)] = int(m.group(2), 0)
|
||||
return out
|
||||
|
||||
|
||||
def _macro_bodies():
|
||||
"""`#define NAME(args) body` and `#define NAME body` with continuations."""
|
||||
src = open(GBI_H, encoding="utf-8", errors="replace").read()
|
||||
src = re.sub(r"/\*.*?\*/", " ", src, flags=re.S)
|
||||
src = src.replace("\\\n", " ")
|
||||
fn, obj = {}, {}
|
||||
for line in src.split("\n"):
|
||||
m = re.match(r"#define\s+(\w+)\(([^)]*)\)\s+(.*)$", line)
|
||||
if m:
|
||||
fn[m.group(1)] = ([a.strip() for a in m.group(2).split(",")], m.group(3).strip())
|
||||
continue
|
||||
m = re.match(r"#define\s+(\w+)\s+(.+)$", line)
|
||||
if m:
|
||||
obj[m.group(1)] = m.group(2).strip()
|
||||
return fn, obj
|
||||
|
||||
|
||||
class RenderModes(object):
|
||||
def __init__(self):
|
||||
self.flags = _flag_defines()
|
||||
self.fn, self.obj = _macro_bodies()
|
||||
self.by_name = {}
|
||||
for name in self.obj:
|
||||
if not name.startswith("G_RM_"):
|
||||
continue
|
||||
try:
|
||||
v = self.eval_name(name)
|
||||
except Exception:
|
||||
continue
|
||||
if v is not None:
|
||||
self.by_name[name] = v & 0xFFFFFFFF
|
||||
self.by_value = {}
|
||||
for n, v in self.by_name.items():
|
||||
self.by_value.setdefault(v, []).append(n)
|
||||
|
||||
def _expand(self, text, depth=0):
|
||||
if depth > 12:
|
||||
raise ValueError("macro recursion")
|
||||
# GBL_c1/GBL_c2 are the only function-like macros we need.
|
||||
def gbl(m):
|
||||
which, args = m.group(1), [a.strip() for a in m.group(2).split(",")]
|
||||
vals = [self._value(a) for a in args]
|
||||
sh = (30, 26, 22, 18) if which == "1" else (28, 24, 20, 16)
|
||||
return "(%d)" % sum(v << s for v, s in zip(vals, sh))
|
||||
prev = None
|
||||
while prev != text:
|
||||
prev = text
|
||||
text = re.sub(r"GBL_c([12])\s*\(([^()]*)\)", gbl, text)
|
||||
return text
|
||||
|
||||
def _value(self, tok):
|
||||
tok = tok.strip()
|
||||
if re.match(r"^(0x[0-9a-fA-F]+|\d+)$", tok):
|
||||
return int(tok, 0)
|
||||
if tok in self.flags:
|
||||
return self.flags[tok]
|
||||
if tok in self.obj:
|
||||
return self.eval_name(tok)
|
||||
raise ValueError("unknown token %r" % tok)
|
||||
|
||||
def eval_name(self, name, _depth=0):
|
||||
if _depth > 12:
|
||||
raise ValueError("recursion")
|
||||
body = self.obj[name]
|
||||
# G_RM_FOO -> RM_FOO(1) / G_RM_FOO2 -> RM_FOO(2)
|
||||
m = re.match(r"^(RM_\w+)\((\d)\)$", body)
|
||||
if m:
|
||||
macro, clk = m.group(1), m.group(2)
|
||||
if macro not in self.fn:
|
||||
return None
|
||||
_args, mbody = self.fn[macro]
|
||||
mbody = mbody.replace("GBL_c##clk", "GBL_c%s" % clk)
|
||||
return self._eval_expr(mbody)
|
||||
return self._eval_expr(body)
|
||||
|
||||
def _eval_expr(self, expr):
|
||||
expr = self._expand(expr)
|
||||
expr = re.sub(r"\b([A-Za-z_]\w*)\b",
|
||||
lambda m: str(self._value(m.group(1))), expr)
|
||||
return eval(expr, {"__builtins__": {}}, {}) & 0xFFFFFFFF # noqa: S307
|
||||
|
||||
def decode(self, word):
|
||||
word &= 0xFFFFFFFF
|
||||
low = word & 0xFFFF
|
||||
parts = [n for b, n in _BITS if low & b]
|
||||
for _label, mask, table in _FIELDS:
|
||||
parts.append(table[low & mask])
|
||||
c1 = "(%s*%s + %s*%s)" % (_BL_P[(word >> 30) & 3], _BL_A[(word >> 26) & 3],
|
||||
_BL_P[(word >> 22) & 3], _BL_A[(word >> 18) & 3])
|
||||
c2 = "(%s*%s + %s*%s)" % (_BL_P[(word >> 28) & 3], _BL_A[(word >> 24) & 3],
|
||||
_BL_P[(word >> 20) & 3], _BL_A[(word >> 16) & 3])
|
||||
names = self.by_value.get(word, [])
|
||||
return {"word": word, "flags": parts, "cyc1": c1, "cyc2": c2, "names": names}
|
||||
|
||||
def name_of(self, word):
|
||||
n = self.by_value.get(word & 0xFFFFFFFF, [])
|
||||
return "/".join(n) if n else ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- bg.c LUTs
|
||||
|
||||
LUT_NAMES = {1: "DL_LUT_PRIMARY_ADDFOG", 5: "DL_LUT_SECONDARY_ADDFOG",
|
||||
6: "DL_LUT_PRIMARY", 7: "DL_LUT_SECONDARY"}
|
||||
|
||||
|
||||
def _lut_entries(body, rm):
|
||||
"""Evaluate a DL_LUT_* initializer body into a flat list of (w0, w1)."""
|
||||
body = re.sub(r"//[^\n]*", " ", body)
|
||||
body = re.sub(r"/\*.*?\*/", " ", body, flags=re.S)
|
||||
words = []
|
||||
for tok in re.finditer(r"gsDPSetRenderMode\s*\(([^,]+),([^)]+)\)|(0x[0-9a-fA-F]+|\b0\b)",
|
||||
body):
|
||||
if tok.group(1):
|
||||
a = rm._eval_expr(tok.group(1).strip())
|
||||
b = rm._eval_expr(tok.group(2).strip())
|
||||
words.append((0xB900031D, (a | b) & 0xFFFFFFFF))
|
||||
else:
|
||||
words.append(("RAW", int(tok.group(3), 0)))
|
||||
# Flatten raw scalar runs into (w0, w1) pairs the same way C brace-elision
|
||||
# fills Gfx.words.w0/w1.
|
||||
out, pend = [], []
|
||||
for kind, val in words:
|
||||
if kind == "RAW":
|
||||
pend.append(val)
|
||||
if len(pend) == 2:
|
||||
out.append((pend[0], pend[1]))
|
||||
pend = []
|
||||
else:
|
||||
if pend:
|
||||
out.append((pend[0], 0))
|
||||
pend = []
|
||||
out.append((kind, val))
|
||||
if pend:
|
||||
out.append((pend[0], 0))
|
||||
return out
|
||||
|
||||
|
||||
def applied_luts(rm=None):
|
||||
"""{lut_index: {from_word: to_word}} for the LUTs bg.c actually applies."""
|
||||
rm = rm or RenderModes()
|
||||
src = open(BG_C, encoding="utf-8", errors="replace").read()
|
||||
maps = {}
|
||||
for idx, name in LUT_NAMES.items():
|
||||
m = re.search(r"Gfx\s+%s\[\]\s*=\s*\{(.*?)\n\};" % re.escape(name), src, re.S)
|
||||
if not m:
|
||||
continue
|
||||
ents = _lut_entries(m.group(1), rm)
|
||||
pairs = {}
|
||||
# (match, replacement) pairs, terminated by an entry with w0 == 0.
|
||||
i = 0
|
||||
while i + 1 < len(ents) and ents[i][0] != 0:
|
||||
(aw0, aw1), (bw0, bw1) = ents[i], ents[i + 1]
|
||||
if aw0 == 0xB900031D and bw0 == 0xB900031D:
|
||||
pairs[aw1] = bw1
|
||||
i += 2
|
||||
maps[idx] = pairs
|
||||
return maps
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
rm = RenderModes()
|
||||
if len(sys.argv) > 1:
|
||||
for a in sys.argv[1:]:
|
||||
d = rm.decode(int(a, 16))
|
||||
print("0x%08x %s" % (d["word"], "/".join(d["names"]) or "(no G_RM_* match)"))
|
||||
print(" flags: %s" % " | ".join(d["flags"]))
|
||||
print(" cyc1 %s cyc2 %s" % (d["cyc1"], d["cyc2"]))
|
||||
else:
|
||||
print("known G_RM_* composites: %d" % len(rm.by_name))
|
||||
for idx, pairs in sorted(applied_luts(rm).items()):
|
||||
print("\n%s (index %d): %d rewrite pairs" % (LUT_NAMES[idx], idx, len(pairs)))
|
||||
for a, b in sorted(pairs.items()):
|
||||
print(" 0x%08x %-26s -> 0x%08x %s"
|
||||
% (a, rm.name_of(a), b, rm.name_of(b)))
|
||||
@@ -0,0 +1,96 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D236 pass 21: list the PROP_* models a level's setup actually places.
|
||||
|
||||
Parses the PROP enum out of src/bondconstants.h and the object records out of
|
||||
one or more assets/obseg/setup/Usetup*Z.c files, then reports which props each
|
||||
setup instantiates. Used to decide, statically, whether a given piece of level
|
||||
scenery can possibly be a prop model at all -- if the setup places none of the
|
||||
candidate models, the geometry must come from the level's bg file instead.
|
||||
|
||||
Note: only the record types that carry a real ObjectRecord header (StandardProp,
|
||||
Door, Collectable, ...) have a meaningful `obj` field; Guard/Tag rows reuse the
|
||||
same word layout for something else and are reported but should not be trusted
|
||||
as model placements.
|
||||
|
||||
Usage: python tools_pc/scan_setup_props.py [setup.c ...]
|
||||
(default: UsetupsevxZ.c UsetupsevxbZ.c -- Surface 1 / Surface 2)
|
||||
"""
|
||||
import collections
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
CONSTANTS = os.path.join(ROOT, "src", "bondconstants.h")
|
||||
SETUP_DIR = os.path.join(ROOT, "assets", "obseg", "setup")
|
||||
|
||||
# One object record: the PropDefHeaderRecord word, then _mkword(obj, pad).
|
||||
RECORD_RE = re.compile(
|
||||
r"/\* Type = (\w+); index = \d+ \*/\s*"
|
||||
r"_mkword\(\s*-?\d+,\s*_mkshort\(\s*-?\d+,\s*-?\d+\s*\)\s*\),\s*"
|
||||
r"_mkword\(\s*(-?\d+)\s*,")
|
||||
|
||||
FOLIAGE_RE = re.compile(r"PROP_(JUNGLE\d*_TREE|PALM|PALMTREE|PLANT\d*B?)$")
|
||||
|
||||
|
||||
def load_prop_enum(path=CONSTANTS):
|
||||
lines = open(path, encoding="utf-8", errors="replace").read().split("\n")
|
||||
# Exact match only: "typedef enum PROPFLAG" / "PROPDEF_TYPE" / "PROP_TYPE"
|
||||
# all share the prefix, and picking one of those silently yields an enum
|
||||
# with no PROP_* members at all (every obj then resolves to "?").
|
||||
i = next(n for n, l in enumerate(lines) if l.strip() == "typedef enum PROP")
|
||||
while "{" not in lines[i]:
|
||||
i += 1
|
||||
i += 1
|
||||
val, out = 0, {}
|
||||
while not lines[i].strip().startswith("}"):
|
||||
m = re.match(r"(PROP_[A-Z0-9_]+)\s*(?:=\s*(-?\w+))?\s*,?", lines[i].strip())
|
||||
if m:
|
||||
if m.group(2):
|
||||
val = int(m.group(2), 0)
|
||||
out[val] = m.group(1)
|
||||
val += 1
|
||||
i += 1
|
||||
return out
|
||||
|
||||
|
||||
def scan(path, enum):
|
||||
txt = open(path, encoding="utf-8", errors="replace").read().replace("\r", "")
|
||||
counts = collections.Counter()
|
||||
for m in RECORD_RE.finditer(txt):
|
||||
counts[(m.group(1), int(m.group(2)))] += 1
|
||||
declared = txt.count("/* Type = ")
|
||||
return counts, declared
|
||||
|
||||
|
||||
def main(argv):
|
||||
enum = load_prop_enum()
|
||||
# Self-test / positive control: a "0 foliage" result is only meaningful if
|
||||
# the detector can fire at all. Print the foliage props the enum defines
|
||||
# and that FOLIAGE_RE recognises, so a zero count below reads as "not
|
||||
# placed", never as "the matcher never matched anything".
|
||||
known = sorted(n for n in enum.values() if FOLIAGE_RE.match(n))
|
||||
print("PROP enum: %d entries; foliage props recognised (%d): %s"
|
||||
% (len(enum), len(known), ", ".join(known)))
|
||||
if not known:
|
||||
print("ERROR: foliage detector matches nothing -- results below are meaningless")
|
||||
return 1
|
||||
names = argv[1:] or ["UsetupsevxZ.c", "UsetupsevxbZ.c"]
|
||||
for name in names:
|
||||
path = name if os.path.exists(name) else os.path.join(SETUP_DIR, os.path.basename(name))
|
||||
counts, declared = scan(path, enum)
|
||||
print("=" * 70)
|
||||
print(os.path.basename(path), "-- %d object records matched of %d total records"
|
||||
% (sum(counts.values()), declared))
|
||||
foliage = 0
|
||||
for (typ, obj), n in sorted(counts.items(), key=lambda kv: -kv[1]):
|
||||
nm = enum.get(obj, "?")
|
||||
tag = " <== FOLIAGE" if FOLIAGE_RE.match(nm) else ""
|
||||
if tag:
|
||||
foliage += n
|
||||
print(" %4d %-22s obj=%4d %s%s" % (n, typ, obj, nm, tag))
|
||||
print(" FOLIAGE PROPS PLACED: %d" % foliage)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv)
|
||||
@@ -0,0 +1,308 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D236 pass 24: decode GE texture pixels offline, straight from the ROM.
|
||||
|
||||
Ports the subset of `src/game/image.c` needed to get real pixels out of a
|
||||
texture without building or running the game:
|
||||
|
||||
* `texReadBits` (image_bank.c:99) -- MSB-first bit reader
|
||||
* `texInflateNonZlib` (image.c:896) -- per-image header loop
|
||||
* `texBuildLookup` (image.c:1669) -- colour lookup table
|
||||
* `texInflateHuffman` (image.c:1434) -- huffman tree + decode
|
||||
* `texInflateRle` (image.c:1596) -- RLE directives
|
||||
* `texInflateLookupFromBuffer` (image.c:2090) -- indices -> pixels
|
||||
|
||||
That covers compmethods UNCOMP0/1 (trivially), LOOKUP, HUFFLOOKUP and
|
||||
RLELOOKUP. The channel-based methods (HUFFMAN, HUFFMANPERCHANNEL, RLE,
|
||||
*BLUR) need `texChannelsToPixels` and are not implemented -- `decode()`
|
||||
raises for those rather than returning something plausible-looking.
|
||||
|
||||
The point of this tool is the ALPHA channel: D236's tree cards are
|
||||
`TEXFORMAT_RGBA16` = RGBA5551, and `g_TexFormatHas1BitAlpha[RGBA16] == 1`
|
||||
(image.c:90), so their silhouette lives entirely in one bit per texel.
|
||||
|
||||
Usage:
|
||||
python tools_pc/texdecode.py 1198 # all LODs, alpha art
|
||||
python tools_pc/texdecode.py 1198 --lod 0 --ppm out.ppm
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, os.path.join(ROOT, "tools_pc"))
|
||||
import texheader # noqa: E402
|
||||
|
||||
# image.c per-format tables
|
||||
NUM_CHANNELS = [4, 3, 3, 3, 2, 2, 1, 1, 1, 1, 1, 1, 1]
|
||||
HAS_1BIT_ALPHA = [0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0]
|
||||
CHANNEL_SIZES = [0x100, 0x20, 0x100, 0x20, 0x100, 0x10, 8, 0x100, 0x10,
|
||||
0x100, 0x10, 0x100, 0x10]
|
||||
BITS_PER_PIXEL = [0x20, 0x10, 0x18, 0x0F, 0x10, 8, 4, 8, 4, 0x10, 0x10, 0x10, 0x10]
|
||||
|
||||
F_RGBA32, F_RGBA16, F_RGB24, F_RGB15, F_IA16, F_IA8, F_IA4, F_I8, F_I4 = range(9)
|
||||
LOOKUP_METHODS = {5: "LOOKUP", 6: "HUFFLOOKUP", 7: "RLELOOKUP"}
|
||||
|
||||
|
||||
class Bits(object):
|
||||
"""texSetBitstring / texReadBits."""
|
||||
|
||||
def __init__(self, data, pos=0):
|
||||
self.d = data
|
||||
self.p = pos
|
||||
self.acc = 0
|
||||
self.n = 0
|
||||
|
||||
def align_after_image(self):
|
||||
"""image.c:1096 -- at the end of each LOD image the bitstream is
|
||||
byte-aligned: any partial byte is discarded, and if the reader happens
|
||||
to sit exactly on a boundary a whole byte is skipped anyway. Omitting
|
||||
this desynchronises every LOD after the first (found as a constant
|
||||
+8-bit drift while walking tex1198)."""
|
||||
if self.n == 0:
|
||||
self.p += 1
|
||||
else:
|
||||
self.n = 0
|
||||
|
||||
def read(self, count):
|
||||
while self.n < count:
|
||||
self.acc = (self.d[self.p] | (self.acc << 8)) & 0xFFFFFFFFFFFF
|
||||
self.p += 1
|
||||
self.n += 8
|
||||
self.n -= count
|
||||
return (self.acc >> self.n) & ((1 << count) - 1)
|
||||
|
||||
|
||||
def build_lookup(bs, bitsperpixel):
|
||||
"""texBuildLookup -> (entries, count)."""
|
||||
n = bs.read(11)
|
||||
if bitsperpixel <= 24:
|
||||
return [bs.read(bitsperpixel) for _ in range(n)], n
|
||||
return [(bs.read(24) << 8) | bs.read(bitsperpixel - 24) for _ in range(n)], n
|
||||
|
||||
|
||||
def inflate_huffman(bs, numiterations, chansize):
|
||||
"""texInflateHuffman -- ported branch for branch."""
|
||||
NB = 2048
|
||||
freq = [0] * NB
|
||||
nodes = [[-1, -1] for _ in range(NB)]
|
||||
for i in range(chansize):
|
||||
freq[i] = bs.read(8)
|
||||
|
||||
def two_smallest():
|
||||
f1 = f2 = 9999
|
||||
i1 = i2 = -1
|
||||
for i in range(chansize):
|
||||
if freq[i] < f1:
|
||||
if f2 < f1:
|
||||
f1, i1 = freq[i], i
|
||||
else:
|
||||
f2, i2 = freq[i], i
|
||||
elif freq[i] < f2:
|
||||
f2, i2 = freq[i], i
|
||||
return f1, i1, f2, i2
|
||||
|
||||
f1, i1, f2, i2 = two_smallest()
|
||||
rootindex = 0
|
||||
while not (f1 == 9999 or f2 == 9999):
|
||||
s = freq[i1] + freq[i2] or 1
|
||||
freq[i1] = 9999
|
||||
freq[i2] = 9999
|
||||
if nodes[i1][0] < 0 and nodes[i1][1] < 0:
|
||||
nodes[i1][0] = i1 + 10000
|
||||
rootindex = i1
|
||||
freq[i1] = s
|
||||
nodes[i1][1] = (i2 + 10000) if (nodes[i2][0] < 0 and nodes[i2][1] < 0) else i2
|
||||
elif nodes[i2][0] < 0 and nodes[i2][1] < 0:
|
||||
nodes[i2][0] = i2 + 10000
|
||||
rootindex = i2
|
||||
freq[i2] = s
|
||||
nodes[i2][1] = (i1 + 10000) if (nodes[i1][0] < 0 and nodes[i1][1] < 0) else i1
|
||||
else:
|
||||
rootindex = 0
|
||||
while nodes[rootindex][0] >= 0 or nodes[rootindex][1] >= 0 or freq[rootindex] < 9999:
|
||||
rootindex += 1
|
||||
freq[rootindex] = s
|
||||
nodes[rootindex][0] = i1
|
||||
nodes[rootindex][1] = i2
|
||||
f1, i1, f2, i2 = two_smallest()
|
||||
|
||||
out = []
|
||||
for _ in range(numiterations):
|
||||
v = rootindex
|
||||
while v < 10000:
|
||||
v = nodes[v][bs.read(1)]
|
||||
out.append(v - 10000)
|
||||
return out
|
||||
|
||||
|
||||
def inflate_rle(bs, blockstotal):
|
||||
"""texInflateRle (image.c:1596) -- ported exactly.
|
||||
|
||||
Header: 3 bits backtrack-field size, 3 bits runlen-field size, 4 bits
|
||||
block size. Then directives: a 0 bit introduces a literal block; a 1 bit
|
||||
introduces a run (backtrack distance, run length), and a run is ALWAYS
|
||||
followed by one literal with no marker bit of its own.
|
||||
|
||||
The `fudge` added to every run length is derived, not guessed -- short
|
||||
runs cost more than literals, so they are never encoded and the field is
|
||||
biased by however many blocks the directive header itself costs.
|
||||
"""
|
||||
btfieldsize = bs.read(3)
|
||||
rlfieldsize = bs.read(3)
|
||||
blocksize = bs.read(4)
|
||||
cost = btfieldsize + rlfieldsize + blocksize + 1
|
||||
fudge = 0
|
||||
while cost > 0:
|
||||
cost -= blocksize + 1
|
||||
fudge += 1
|
||||
|
||||
out = []
|
||||
guard = blockstotal * 4 + 64
|
||||
while len(out) < blockstotal and guard > 0:
|
||||
guard -= 1
|
||||
if bs.read(1) == 0:
|
||||
out.append(bs.read(blocksize))
|
||||
else:
|
||||
start = len(out) - bs.read(btfieldsize) - 1
|
||||
runlen = bs.read(rlfieldsize) + fudge
|
||||
for k in range(runlen):
|
||||
out.append(out[start + k] if 0 <= start + k < len(out) else 0)
|
||||
out.append(bs.read(blocksize)) # a run is always followed by a literal
|
||||
return out[:blockstotal]
|
||||
|
||||
|
||||
def _stride16(width):
|
||||
return (width + 3) & 0xFFC
|
||||
|
||||
|
||||
def decode_image(bs, fmt, width, height, compmethod):
|
||||
"""Return (rows, kind) where rows is a list of per-row pixel lists."""
|
||||
if compmethod not in LOOKUP_METHODS:
|
||||
raise NotImplementedError(
|
||||
"compmethod %d needs texChannelsToPixels, not ported" % compmethod)
|
||||
lookup, ncol = build_lookup(bs, BITS_PER_PIXEL[fmt])
|
||||
npix = width * height
|
||||
if compmethod == 6:
|
||||
idx = inflate_huffman(bs, npix, ncol)
|
||||
elif compmethod == 7:
|
||||
idx = inflate_rle(bs, npix)
|
||||
else: # plain LOOKUP: indices are read straight off the bitstream
|
||||
# texGetBitSize(ncol) == (ncol-1).bit_length(), and is 0 for a
|
||||
# single-colour table -- do NOT clamp to 1, that desynchronises.
|
||||
nbits = (ncol - 1).bit_length()
|
||||
idx = [bs.read(nbits) for _ in range(npix)]
|
||||
rows = []
|
||||
for y in range(height):
|
||||
row = [lookup[i] if i < len(lookup) else 0
|
||||
for i in idx[y * width:(y + 1) * width]]
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def decode(num, region="ntsc-final", max_lods=None):
|
||||
"""Decode every LOD image of one texture. Returns (header, [image, ...])."""
|
||||
rom, rows = texheader.load(region)
|
||||
off, size = rows[num]
|
||||
blob = rom[off:off + size]
|
||||
hdr = texheader.parse(blob)
|
||||
if hdr["zlib"]:
|
||||
raise NotImplementedError("zlib/paletted textures not handled here")
|
||||
bs = Bits(blob, 1)
|
||||
numimages = hdr["lods"] if hdr["explicit_lods"] else 1
|
||||
if max_lods:
|
||||
numimages = min(numimages, max_lods)
|
||||
imgs = []
|
||||
for _ in range(numimages):
|
||||
fmt = bs.read(4)
|
||||
w = bs.read(8)
|
||||
h = bs.read(8)
|
||||
cm = bs.read(4)
|
||||
if w * h > 0x2000 or w == 0 or h == 0:
|
||||
break
|
||||
try:
|
||||
rows_ = decode_image(bs, fmt, w, h, cm)
|
||||
except NotImplementedError as e:
|
||||
# Later LODs often switch to a channel-based codec; stop cleanly
|
||||
# and report, rather than pretending the chain ended here.
|
||||
imgs.append({"format": fmt, "width": w, "height": h,
|
||||
"compmethod": cm, "rows": None, "error": str(e)})
|
||||
break
|
||||
imgs.append({"format": fmt, "width": w, "height": h, "compmethod": cm,
|
||||
"rows": rows_})
|
||||
bs.align_after_image()
|
||||
return hdr, imgs
|
||||
|
||||
|
||||
def alpha_art(img):
|
||||
"""ASCII art of the 1-bit alpha channel of an RGBA5551 image."""
|
||||
if img["format"] != F_RGBA16:
|
||||
return None
|
||||
return ["".join("#" if (p & 1) else "." for p in row) for row in img["rows"]]
|
||||
|
||||
|
||||
def rgb_art(img):
|
||||
"""Coarse luminance art, to sanity-check the colour channels."""
|
||||
ramp = " .:-=+*#%@"
|
||||
out = []
|
||||
for row in img["rows"]:
|
||||
line = ""
|
||||
for p in row:
|
||||
r, g, b = (p >> 11) & 31, (p >> 6) & 31, (p >> 1) & 31
|
||||
line += ramp[min(9, (r + g + b) * 10 // 96)]
|
||||
out.append(line)
|
||||
return out
|
||||
|
||||
|
||||
def write_ppm(img, path):
|
||||
w, h = img["width"], img["height"]
|
||||
with open(path, "wb") as f:
|
||||
f.write(b"P6\n%d %d\n255\n" % (w, h))
|
||||
for row in img["rows"]:
|
||||
for p in row:
|
||||
r, g, b, a = (p >> 11) & 31, (p >> 6) & 31, (p >> 1) & 31, p & 1
|
||||
# transparent texels rendered magenta so the silhouette is obvious
|
||||
f.write(bytes((r << 3, g << 3, b << 3)) if a else b"\xff\x00\xff")
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 2:
|
||||
print(__doc__)
|
||||
return 1
|
||||
num = int(argv[1], 0)
|
||||
only = None
|
||||
ppm = None
|
||||
if "--lod" in argv:
|
||||
only = int(argv[argv.index("--lod") + 1])
|
||||
if "--ppm" in argv:
|
||||
ppm = argv[argv.index("--ppm") + 1]
|
||||
hdr, imgs = decode(num)
|
||||
print("tex%d: explicit_lods=%d lods=%d -> decoded %d image(s)"
|
||||
% (num, hdr["explicit_lods"], hdr["lods"], len(imgs)))
|
||||
for i, img in enumerate(imgs):
|
||||
if only is not None and i != only:
|
||||
continue
|
||||
print("\n LOD %d: %s %dx%d compmethod=%s"
|
||||
% (i, texheader.TEXFORMAT.get(img["format"], img["format"]),
|
||||
img["width"], img["height"],
|
||||
texheader.COMPMETHOD.get(img["compmethod"], img["compmethod"])))
|
||||
if img["rows"] is None:
|
||||
print(" NOT DECODED: %s" % img["error"])
|
||||
continue
|
||||
a = alpha_art(img)
|
||||
if a:
|
||||
opaque = sum(r.count("#") for r in a)
|
||||
total = img["width"] * img["height"]
|
||||
print(" alpha: %d/%d opaque (%.1f%%) ['#'=opaque '.'=transparent]"
|
||||
% (opaque, total, 100.0 * opaque / total))
|
||||
for r in a:
|
||||
print(" " + r)
|
||||
else:
|
||||
for r in rgb_art(img):
|
||||
print(" " + r)
|
||||
if ppm and (only is None or i == only):
|
||||
write_ppm(img, ppm)
|
||||
print(" wrote %s" % ppm)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env python3
|
||||
"""D236 pass 23: read GE texture headers straight out of the ROM, offline.
|
||||
|
||||
A texture's ROM blob starts with a control byte (`image.c:2568`):
|
||||
|
||||
bit 7 hasExplicitLods bit 6 iszlib bits 0-5 lod (image count)
|
||||
|
||||
and the bitstream begins at byte 1. Forgetting that control byte shifts
|
||||
every subsequent field by 8 bits and yields a header that still *parses* --
|
||||
plausible-looking formats and dimensions -- so `census()` below reports how
|
||||
many entries come out implausible as a built-in sanity check (expect ~1 of
|
||||
2699 for U; a few hundred means the offset is wrong again).
|
||||
|
||||
Two payload layouts follow, per `image.c`:
|
||||
* iszlib=0 -> `texInflateNonZlib`: a 24-bit bitstream header per image,
|
||||
`ffff wwwwwwww hhhhhhhh cccc` (format / width / height / compmethod).
|
||||
* iszlib=1 -> `texInflateZlib`: format u8, numcolours-1 u8, then the
|
||||
palette, then per-image width u8 / height u8.
|
||||
|
||||
Only headers are decoded here -- the pixel payloads use Rare's huffman/RLE/
|
||||
lookup codecs, which are not implemented.
|
||||
|
||||
Usage: python tools_pc/texheader.py [texnum ...] (no args: census)
|
||||
"""
|
||||
import collections
|
||||
import os
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
TEXFORMAT = {0: "RGBA32", 1: "RGBA16", 2: "RGB24", 3: "RGB15", 4: "IA16",
|
||||
5: "IA8", 6: "IA4", 7: "I8", 8: "I4", 9: "RGBA16_CI8",
|
||||
10: "RGBA16_CI4", 11: "IA16_CI8", 12: "IA16_CI4"}
|
||||
COMPMETHOD = {0: "UNCOMP0", 1: "UNCOMP1", 2: "HUFFMAN", 3: "HUFFMANCHAN",
|
||||
4: "RLE", 5: "LOOKUP", 6: "HUFFLOOKUP", 7: "RLELOOKUP",
|
||||
8: "HUFFBLUR", 9: "RLEBLUR"}
|
||||
# image.c g_TexFormatGbiMappings / g_TexFormatDepths -- what the runtime
|
||||
# reports, i.e. what a live GE_TEXI/GE_D2xx probe prints as fmt=/siz=.
|
||||
GBI_FMT = {0: 0, 1: 0, 2: 0, 3: 0, 4: 3, 5: 3, 6: 3, 7: 4, 8: 4,
|
||||
9: 2, 10: 2, 11: 2, 12: 2}
|
||||
GBI_SIZ = {0: 3, 1: 2, 2: 3, 3: 2, 4: 2, 5: 1, 6: 0, 7: 1, 8: 0,
|
||||
9: 1, 10: 0, 11: 1, 12: 0}
|
||||
GBI_FMT_NAME = {0: "RGBA", 2: "CI", 3: "IA", 4: "I"}
|
||||
|
||||
|
||||
def load(region="ntsc-final"):
|
||||
"""(rom bytes, [(offset, size), ...] indexed by texture number)."""
|
||||
rom = open(os.path.join(ROOT, "data", "ge007.%s.z64" % region), "rb").read()
|
||||
rows = []
|
||||
csv = os.path.join(ROOT, "imagelist.%s.csv" % ("u" if region.startswith("ntsc") else region[0]))
|
||||
for line in open(csv):
|
||||
if not line.strip():
|
||||
continue
|
||||
f = line.split(",")
|
||||
rows.append((int(f[0]), int(f[1])))
|
||||
return rom, rows
|
||||
|
||||
|
||||
def parse(blob):
|
||||
ctrl = blob[0]
|
||||
d = {"lods": ctrl & 0x3F, "zlib": (ctrl >> 6) & 1, "explicit_lods": (ctrl >> 7) & 1}
|
||||
if d["zlib"]:
|
||||
d["format"] = blob[1]
|
||||
d["numcolours"] = blob[2] + 1
|
||||
else:
|
||||
v = (blob[1] << 16) | (blob[2] << 8) | blob[3]
|
||||
d["format"] = (v >> 20) & 0xF
|
||||
d["width"] = (v >> 12) & 0xFF
|
||||
d["height"] = (v >> 4) & 0xFF
|
||||
d["compmethod"] = v & 0xF
|
||||
f = d["format"]
|
||||
d["format_name"] = TEXFORMAT.get(f, "?%d" % f)
|
||||
d["gbi_fmt"] = GBI_FMT.get(f)
|
||||
d["gbi_siz"] = GBI_SIZ.get(f)
|
||||
return d
|
||||
|
||||
|
||||
def implausible(d):
|
||||
if d["format"] > 12:
|
||||
return True
|
||||
if d["zlib"]:
|
||||
return False
|
||||
return d["compmethod"] > 9 or d["width"] == 0 or d["height"] == 0
|
||||
|
||||
|
||||
def describe(num, rom, rows):
|
||||
off, size = rows[num]
|
||||
d = parse(rom[off:off + 5])
|
||||
live = "fmt=%d(%s) siz=%d" % (d["gbi_fmt"], GBI_FMT_NAME.get(d["gbi_fmt"], "?"), d["gbi_siz"])
|
||||
if d["zlib"]:
|
||||
body = "ZLIB %-11s ncol=%-4d" % (d["format_name"], d["numcolours"])
|
||||
else:
|
||||
body = "NONZLIB %-11s %3dx%-3d %-11s" % (d["format_name"], d["width"], d["height"],
|
||||
COMPMETHOD.get(d["compmethod"], d["compmethod"]))
|
||||
return "tex%-5d size=%-5d %s lods=%d explicit_lods=%d -> live %s" % (
|
||||
num, size, body, d["lods"], d["explicit_lods"], live)
|
||||
|
||||
|
||||
def census(rom, rows):
|
||||
fmts, bad = collections.Counter(), 0
|
||||
for num in range(len(rows)):
|
||||
off, size = rows[num]
|
||||
if size < 5:
|
||||
continue
|
||||
d = parse(rom[off:off + 5])
|
||||
fmts[("zlib " if d["zlib"] else "") + d["format_name"]] += 1
|
||||
if implausible(d):
|
||||
bad += 1
|
||||
return fmts, bad
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
rom, rows = load()
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
if args:
|
||||
for a in args:
|
||||
print(describe(int(a, 0), rom, rows))
|
||||
else:
|
||||
fmts, bad = census(rom, rows)
|
||||
print("%d textures, %d implausible headers (sanity check: expect ~1)" % (len(rows), bad))
|
||||
for name, n in fmts.most_common():
|
||||
print(" %-16s %5d" % (name, n))
|
||||
Reference in new issue
Block a user