D70-D72: PPM capture tool, C-array texture byte-order fix, tc[] UV restore

- D70: GE_PCDUMP env-gated PPM frame dump (video.c + gfx_opengl.cpp); ppm/ gitignored
- D71: per-source bswap of C-array texture sources in import_texture (the four
  rarewarelogo.c RGBA16 images are LE u32 arrays on PC -> pink/green logo)
- D72: removed PD-inherited normal/lookat UV override; lookat_enabled defaults
  false (GE always uses authored tc[] UVs; N64 boots with RSP mem zeroed)
- D72.3 OPEN: logo triangles still project off-screen; a flat dark-blue
  checkerboard triangle is rasterized instead - identity pending
  (GE_DBGUV/GE_DBGTALL env-gated traces added)
- Docs: PCPortResearch.md F/D70-D72, HANDOFF refreshed, D71 log gated
This commit is contained in:
James Dansereau committed 2026-08-24 19:12:40 -04:00
1 parent 8f631462f3
commit 8ed411d3f1
6 files changed
+258 -49

No files matched your search

+1
View File
@@ -124,6 +124,7 @@ _run.log
ge007.crash.log
.pi/
run_out.log
ppm/
# PC port run/scratch logs
run_*.log
+30 -9
View File
@@ -6,13 +6,21 @@ is the summary + the immediate tasks._
## Your job
- **Session H:** make the first stage (BUNKER1) load and render. The game now
plays the entire ~3.5-minute intro at ~59 fps and crashes in
`load_bg_file` because BG-file data is N64 big-endian (D69). This is the
"next asset type" milestone: reverse-engineer GE's bg `.seg` +
`Tbg_*_stanZ` formats from the decompiled consumers, then convert/fix them
for PC. Work agentically: decode → implement → build (~5 s) → verify →
commit at each sub-milestone (document findings as D6x in §F).
- **Session H:** two live threads, both verified reproducible:
1. **D72.3 (intro logo pixels):** after D71 (logo byte order) + D72 (tc[]
UVs), the Rareware logo still doesn't appear — all its triangles project
off-screen, and a flat dark-blue checkerboard triangle (broken
rasterization of something else) fills the lower screen instead. Tools
are in place: `GE_PCDUMP` frame capture + `GE_DBGUV`/`GE_DBGTALL`
traces in gfx_pc.cpp. Find which triangle draws the checkerboard, then
check MP matrix + viewport at logo time (title.c
load_display_rare_logo).
2. **D69 (milestone):** make the first stage (BUNKER1) load and render —
BG-file data is N64 big-endian; reverse-engineer GE's bg `.seg` +
`Tbg_*_stanZ` formats from the decompiled consumers, then convert/fix
for PC.
Work agentically: decode → implement → build (~5 s) → verify → commit at
each sub-milestone (document findings as D6x/D7x in §F).
## Current state (verified this session)
@@ -37,11 +45,18 @@ is the summary + the immediate tasks._
tables, counts from D39 layout) after texReset's romCopy; marker scan in
`texLoadFromDisplayList` moved to bytes 6..7 under PORT; compiled
globalDL shadows synced after texLoad (`gimgSyncCompiledGlobalDLs`).
- **D70–D72 (intro logo pixels, latest session):** D70 env-gated PPM frame
capture (`GE_PCDUMP` → `./ppm/`, gitignored); D71 RESOLVED — C-array
texture sources (the four rarewarelogo.c RGBA16 images) were byte-swapped
on LE PC → pink/green logo; port-layer per-source bswap normalizes them.
D72 RESOLVED the UV path (GE always uses authored tc[] UVs; lookat defaults
off) but **D72.3 OPEN**: all logo triangles project off-screen and a flat
dark-blue checkerboard triangle rasterizes instead — see Your job #1.
- **Runtime:** boots → intro music → full intro (logo, gun barrel w/ Brosnan,
cast screen) at ~59 fps → **crash in `load_bg_file` (bg.c:830)** on first
stage load. EXCEPTION 0xc0000005, FAULT ADDR ≈ header buffer + 0x500000F.
- **Committed at D68** (see git log). TEMP diagnostics from the D51–D69
sessions are still in the tree — strip list in Task 3.
- **Committed through D72** (see git log). TEMP diagnostics from the
D51–D72 sessions are still in the tree — strip list in Task 3.
## The blocker, precisely (D69)
@@ -125,6 +140,12 @@ Env-gated probes to remove (grep `GE_D` + `TEMP D`):
includes.
- image.c / image_bank.c: D66 texLoad probe if still present.
- gfx_pc.cpp: D59 seg probes, D63 dram-branch/modelRenderNodeGundl/rspGfxTaskStart.
- gfx_pc.cpp: D70/D71/D72 TEMP — `GE_DBGUV`/`GE_DBGTALL` UV+triangle loggers
(gfx_dbg_uv_enabled + the DBGTRI/DBGTALL blocks in gfx_sp_vertex/gfx_sp_tri1)
and the `GE_D71LOG` normalize log. Keep the D71 normalization itself — it's
the fix, not a probe.
- video.c + gfx_opengl.cpp: D70 `GE_PCDUMP` frame capture (videoEndFrame dump
hook, gfx_opengl_pcdump_* implementations).
- blood_animation.c: D63 g_GfxMemPos probe.
- front.c: D63 menu probes (d63MenuProbe/After + call sites).
- rsp.c, model.c, initanitable.c, language.c, title.c, ramromreplay.c,
+50
View File
@@ -2289,6 +2289,56 @@ on a pointer-field read):
safe when all access goes through a pointer stored in the sub-object at init
(modelInit → Model.datas) and only the one site references the raw offset.
**D70 (dev tooling, TEMP):** env-gated PPM frame capture for numerical visual
debugging: `GE_PCDUMP="first-last[:step]"` dumps the bound FBO to
`./ppm/frame_NNNNNN.ppm` from `videoEndFrame()` (`gfx_opengl_pcdump_enabled()`/
`gfx_opengl_dump_bound_fbo()` implemented in gfx_opengl.cpp). Established the
intro timeline numerically (legal text f20–100, Rareware logo f380–580,
iris/gun-barrel f900–1220, red region f1240–1360, cast f1520+) and confirmed
the user's "pink and green" report at f480 (left half RGB≈(135,78,129), right
half (82,104,0)). Strip per HANDOFF Task 3.
**D71 (RESOLVED):** the Rareware logo rendered as two flat colors — pink-red
and bright green — instead of gold lettering. Root cause: the four RGBA16 logo
images in assets/rarewarelogo.c (`imgRAre_0x0020` etc.) are `u32` C arrays
compiled into the exe `.data`; on LE PC each N64 texel pair is stored as a
little-endian u32, so `import_texture_rgba16`'s big-endian u16 read produced
byte-swapped texels (raw 0xED0F gold → 0x4FCC green + 0x0FED pink — exactly
the two observed colors). All other texture sources are raw N64 BE byte
streams (ROM cart map 0x10xxxxxx, model sidecar 0x10Cxxxxx, KSEG0/V1 buffers
0x70–0x90xxxxxx) and must not be touched. Fix (port/fast3d/gfx_pc.cpp):
`gfx_tex_source_is_c_array()` classifies by address range;
`gfx_tex_normalize_source()` bswaps each u32 once per source into a stable
cached buffer (cache key stays the original address) and `import_texture()`
decodes from it. Verified: exactly two sources normalized in a full run (the
logo image banks); logo now renders gold on dark blue. One-shot per-source log
gated by `GE_D71LOG`.
**D72 (RESOLVED + D72.3 OPEN):** the logo UV path.
- D72.1: removed the PD-inherited normal/lookat-based UV overwrite in
`gfx_sp_vertex()` — GE always uses authored per-vertex `tc[]` UVs: no GE
code sets G_TT_BASE/G_TT_CLAMP (only G_TT_NONE), and stage geometry has
gSPLookAt set every frame yet N64 textures are fixed to surfaces, so the
normal-derived path is wrong for GE in general.
- D72.2: `rsp.lookat_enabled` now defaults false — N64 boots with RSP memory
zeroed; no lookat exists until gSPLookAt writes one (the intro logo has
none). GL UV convention confirmed as U.5 (texel×32) from the LUT/rect path,
so DL_RAREWARETEXT's identity-scale corner UVs 0x0010..0x03F0 map to full
32×32 coverage with a half-texel inset.
- D72.3 (OPEN): after D71+D72 the logo still does not appear: DBGTRI traces
show ALL logo triangles (plate fan + letter quads) project off-screen
(clip x=y≈−1.6e11, w<0, screen ≈(32776,32776)), yet the frame shows a large
flat dark-blue (0,0,64) region — rows 150..479, cols 1..639 — filled as a
PERFECT checkerboard (50/50 pixel parity), i.e. broken rasterization of some
big triangle drawn after clear_framebuffer_black. Identity not yet
established (`GE_DBGTALL` all-triangle trace captured; analysis pending).
Ruled out: matrix-format mismatch — this codebase's guMtxF2L is Rare's
modified variant writing the interleaved hi16/lo16 s32 Q15.16 packed format
that gfx_sp_matrix decodes exactly (FTOFIX32=×65536); `D_8002A7D0` is a
zero-init u32 so `[D_8002A7D0]`==`[0]`; alloc_intro_matrices() runs from
initmenus. Next: find the checkerboard's triangle via GE_DBGTALL, then
verify MP matrix + viewport at logo time (title.c load_display_rare_logo).
**Non-negotiable #2 refinement (applied to AGENTS.md).** The original "game code
compiles unmodified / fix belongs in port/" is too absolute: pointer-width layout
cannot be isolated in `port/` (no hook between the romCopy and the first read).
+21
View File
@@ -1048,6 +1048,27 @@ static void gfx_opengl_end_frame(void) {
glFlush();
}
/* Frame capture (dev tool, env GE_PCDUMP) — see gfx_opengl.h. TEMP D70: read
* the default framebuffer (the window back buffer; the scene renders there
* directly at 1:1 window size) and write a P6 PPM. */
extern "C" bool gfx_opengl_pcdump_enabled(void) {
return getenv("GE_PCDUMP") != NULL;
}
extern "C" bool gfx_opengl_dump_bound_fbo(uint32_t width, uint32_t height, const char* path) {
size_t n = (size_t)width * height * 3;
uint8_t* pixels = (uint8_t*)malloc(n);
if (!pixels) return false;
glReadPixels(0, 0, (GLsizei)width, (GLsizei)height, GL_RGB, GL_UNSIGNED_BYTE, pixels);
FILE* f = fopen(path, "wb");
if (!f) { free(pixels); return false; }
fprintf(f, "P6\n%u %u\n255\n", width, height);
fwrite(pixels, 1, n, f);
fclose(f);
free(pixels);
return true;
}
static void gfx_opengl_finish_render(void) {
}
+125 -40
View File
@@ -609,6 +609,45 @@ void gfx_texture_cache_delete_range(const uint8_t* start, const uint8_t* end) {
}
}
// D71 (docs/PCPortResearch.md): texture sources arrive in two byte conventions.
// Raw N64 big-endian byte streams: ROM cart map (0x10xxxxxx), model-sidecar
// blobs (cart extension 0x10Cxxxxx), KSEG0 mirror (0x80xxxxxx) and V1
// dram/BSS/heap buffers (0x70xxxxxx, e.g. tex.c texture pool, rle_expand_8bit
// output). C-compiled u32 arrays in the exe image (.data/.rodata,
// 0x140xxxxxx on MinGW x64) instead store each N64 texel pair as a
// little-endian u32 — e.g. the rarewarelogo.c RGBA16 images — so the N64 byte
// order is recovered by bswap32 of every u32. Without this, the logo's gold
// texels (0xED0F...) decode from the swapped pairs (0x4FCC/0xCC4F) as bright
// green/pink — the garbled Rareware-logo pixels.
static bool gfx_tex_source_is_c_array(const uint8_t* addr) {
const uintptr_t a = (uintptr_t)addr;
if (a >= 0x10000000u && a < 0x20000000u) return false; // cart map + sidecar
if (a >= 0x70000000u && a < 0x90000000u) return false; // V1 dram + KSEG0 mirror
return true; // exe image: C-compiled array
}
static std::map<const uint8_t*, std::vector<uint8_t> > s_c_array_tex_norms;
// Returns a pointer to the source in N64 byte order (the original pointer for
// raw-stream sources, a stable per-source bswapped copy for C arrays).
// extent is the full image size incl. padded rows (ci8 reads up to it).
static const uint8_t* gfx_tex_normalize_source(const uint8_t* addr, uint32_t extent) {
if (!gfx_tex_source_is_c_array(addr)) return addr;
auto it = s_c_array_tex_norms.find(addr);
if (it != s_c_array_tex_norms.end()) return it->second.data();
const uint32_t n = (extent + 3u) & ~3u;
std::vector<uint8_t> buf(n);
const uint32_t* src = (const uint32_t*)addr;
uint32_t* dst = (uint32_t*)buf.data();
for (uint32_t i = 0; i < n / 4; i++)
dst[i] = PD_BE32(src[i]);
if (getenv("GE_D71LOG"))
fprintf(stderr, "[D71] normalized C-array texture source %p (%u bytes)\n",
(const void*)addr, extent);
return s_c_array_tex_norms.emplace(addr, std::move(buf)).first->second.data();
}
static void import_texture_rgba16(int tile, const LoadedTexture& loaded_texture, bool gen_mipmaps) {
const uint8_t* addr = loaded_texture.addr;
const uint32_t size_bytes = loaded_texture.size_bytes;
@@ -905,6 +944,14 @@ static void import_texture(int i, int tile, bool importReplacement) {
return;
}
// D71: importers read raw N64 byte streams; normalize C-array sources.
const uint8_t* saved_addr = loaded_texture.addr;
loaded_texture.addr =
gfx_tex_normalize_source(orig_addr,
loaded_texture.full_size_bytes > loaded_texture.size_bytes
? loaded_texture.full_size_bytes
: loaded_texture.size_bytes);
if (fmt == G_IM_FMT_RGBA) {
if (siz == G_IM_SIZ_16b) {
import_texture_rgba16(tile, loaded_texture, rdp.tex_lod);
@@ -942,6 +989,8 @@ static void import_texture(int i, int tile, bool importReplacement) {
} else {
sysFatalError("Bad texture format in tile %d: %02x %02x", tile, fmt, siz);
}
loaded_texture.addr = saved_addr;
}
static void gfx_normalize_vector(float v[3]) {
@@ -1122,43 +1171,19 @@ static void gfx_sp_vertex(size_t n_vertices, size_t dest_index, const Vtx* verti
d->color.g = g > 255 ? 255 : g;
d->color.b = b > 255 ? 255 : b;
if (rsp.geometry_mode & G_TEXTURE_GEN) {
float dotx = 0, doty = 0;
if (rsp.lookat_enabled) {
dotx += vcn->x * rsp.current_lookat_coeffs[0][0];
dotx += vcn->y * rsp.current_lookat_coeffs[0][1];
dotx += vcn->z * rsp.current_lookat_coeffs[0][2];
doty += vcn->x * rsp.current_lookat_coeffs[1][0];
doty += vcn->y * rsp.current_lookat_coeffs[1][1];
doty += vcn->z * rsp.current_lookat_coeffs[1][2];
dotx /= 127.0f;
doty /= 127.0f;
} else {
float tvcn[3];
calculate_normal_dir(vcn, tvcn);
dotx = tvcn[0];
doty = tvcn[1];
}
dotx = clampf(dotx, -1.0f, 1.0f);
doty = clampf(doty, -1.0f, 1.0f);
if (rsp.geometry_mode & G_TEXTURE_GEN_LINEAR) {
// Not sure exactly what formula we should use to get accurate values
/*dotx = (2.906921f * dotx * dotx + 1.36114f) * dotx;
doty = (2.906921f * doty * doty + 1.36114f) * doty;
dotx = (dotx + 1.0f) / 4.0f;
doty = (doty + 1.0f) / 4.0f;*/
dotx = acosf(-dotx) /* M_PI */ / 4.0f;
doty = acosf(-doty) /* M_PI */ / 4.0f;
} else {
dotx = (dotx + 1.0f) / 4.0f;
doty = (doty + 1.0f) / 4.0f;
}
U = (int32_t)(dotx * rsp.texture_scaling_factor.s);
V = (int32_t)(doty * rsp.texture_scaling_factor.t);
}
/* D72 (docs/PCPortResearch.md): unlike the PD port, GE never
* generates texture coordinates from vertex normals. This PD-
* inherited block overwrote the authored tc[] UVs on every
* lit+textured surface (lookat_enabled defaults to true here),
* which smeared the Rareware-logo quads into a diagonal gold
* wedge: their corner UVs (0x0010..0x03F0 = full 32x32 coverage)
* were replaced by normal-derived values. GE's ground truth:
* every DL that could env-map sets gsDPSetTextureLUT(G_TT_NONE),
* stage rendering always has a camera lookat yet stage textures
* are surface-fixed on hardware, and the logo vertex batches
* carry authored atlas-slice UVs. So U/V stay as computed from
* tc[] above. (gSPLookAt state is still tracked — it may matter
* for lighting fidelity later.) */
} else {
d->color.r = vcn->r;
d->color.g = vcn->g;
@@ -1230,12 +1255,57 @@ static inline int gfx_lod_tile_offset(const int i) {
return (rdp.tex_lod ? rdp.tex_detail : i);
}
// TEMP D72: env-gated UV pipeline logging (GE_DBGUV=1).
static bool gfx_dbg_uv_enabled(void) {
static int e = -1;
if (e < 0) e = getenv("GE_DBGUV") != nullptr;
return e;
}
static void gfx_sp_tri1(uint8_t vtx1_idx, uint8_t vtx2_idx, uint8_t vtx3_idx, bool is_rect) {
struct LoadedVertex* v1 = &rsp.loaded_vertices[vtx1_idx];
struct LoadedVertex* v2 = &rsp.loaded_vertices[vtx2_idx];
struct LoadedVertex* v3 = &rsp.loaded_vertices[vtx3_idx];
struct LoadedVertex* v_arr[3] = { v1, v2, v3 };
// TEMP D72: GE_DBGTRI=1 — trace triangles; GE_DBGTALL=1 logs all.
static int dbg_tri_count = 0;
if (gfx_dbg_uv_enabled() && getenv("GE_DBGTALL")) {
float sx[3] = { v1->x / (v1->w), v2->x / (v2->w), v3->x / (v3->w) };
float sy[3] = { v1->y / (v1->w), v2->y / (v2->w), v3->y / (v3->w) };
if (dbg_tri_count++ < 4000) {
fprintf(stderr, "[DBGTALL] n=%d scr=(%.0f,%.0f)(%.0f,%.0f)(%.0f,%.0f) col=(%d,%d,%d,%d) clip=(%x,%x,%x)\n",
dbg_tri_count, sx[0], sy[0], sx[1], sy[1], sx[2], sy[2],
v1->color.r, v1->color.g, v1->color.b, v1->color.a,
v1->clip_rej, v2->clip_rej, v3->clip_rej);
}
}
if (gfx_dbg_uv_enabled()) {
bool gold = false, blue = false;
for (int i = 0; i < 3; i++) {
if (v_arr[i]->color.b == 0 && v_arr[i]->color.r > 90) gold = true;
if (v_arr[i]->color.b > 100 && v_arr[i]->color.r < 50) blue = true;
}
if (gold || blue) {
float dx1 = v1->x / (v1->w) - v2->x / (v2->w);
float dy1 = v1->y / (v1->w) - v2->y / (v2->w);
float dx2 = v3->x / (v3->w) - v2->x / (v2->w);
float dy2 = v3->y / (v3->w) - v2->y / (v2->w);
if (dbg_tri_count++ < 64) {
fprintf(stderr, "[DBGTRI] %s cull=%d clip=(%x,%x,%x) cross=%.3f scr=(%.1f,%.1f)(%.1f,%.1f)(%.1f,%.1f) clip4=(%.0f,%.0f,%.0f,%.2f) col=(%d,%d,%d)\n",
gold ? "GOLD" : "BLUE",
rsp.geometry_mode & G_CULL_BOTH,
v1->clip_rej, v2->clip_rej, v3->clip_rej,
dx1 * dy2 - dy1 * dx2,
v1->x / (v1->w), v1->y / (v1->w),
v2->x / (v2->w), v2->y / (v2->w),
v3->x / (v3->w), v3->y / (v3->w),
v1->x, v1->y, v1->z, v1->w,
v1->color.r, v1->color.g, v1->color.b);
}
}
}
if ((rsp.extra_geometry_mode & G_NO_CLIPPING_EXT) == 0) {
if (v1->clip_rej & v2->clip_rej & v3->clip_rej) {
// The whole triangle lies outside the visible area
@@ -1513,6 +1583,22 @@ static void gfx_sp_tri1(uint8_t vtx1_idx, uint8_t vtx2_idx, uint8_t vtx3_idx, bo
buf_vbo[buf_vbo_len++] = u / tex_width[t];
buf_vbo[buf_vbo_len++] = v / tex_height[t];
// TEMP D72: GE_DBGUV=1 — log the UV pipeline per vertex, but
// only for C-array (logo) textures so the cap isn't exhausted.
if (gfx_dbg_uv_enabled() &&
gfx_tex_source_is_c_array(
rdp.loaded_texture[rdp.texture_tile[rdp.first_tile_index + tile].tmem].addr)) {
static int dbg_uv_count = 0;
if (dbg_uv_count++ < 30000) {
fprintf(stderr, "[DBGUV] v.u=%d v.v=%d shifts=%d shiftt=%d uls=%d ult=%d -> u=%.4f v=%.4f (tex %dx%d)\n",
v_arr[i]->u, v_arr[i]->v, shifts, shiftt,
rdp.texture_tile[rdp.first_tile_index + tile].uls,
rdp.texture_tile[rdp.first_tile_index + tile].ult,
u / tex_width[t], v / tex_height[t],
tex_width[t], tex_height[t]);
}
}
bool clampS = tm & (1 << 2 * t);
bool clampT = tm & (1 << (2 * t + 1));
@@ -2671,9 +2757,8 @@ extern "C" void gfx_init(const GfxInitSettings *settings) {
tex_upload_buffer = (uint8_t*)malloc(max_tex_size * max_tex_size * 4);
}
rsp.lookat[0].dir[0] = rsp.lookat[1].dir[1] = 0x7F;
rsp.current_lookat_coeffs[0][0] = rsp.current_lookat_coeffs[1][1] = 1.f;
rsp.lookat_enabled = true;
/* D72: N64 boots with RSP memory zeroed — no lookat until gSPLookAt. */
rsp.lookat_enabled = false;
}
extern "C" void gfx_destroy(void) {
+31
View File
@@ -12,6 +12,14 @@
#include <stdio.h>
#include <string.h>
#if defined(_WIN32)
#include <direct.h>
#define GE_MKDIR(p) _mkdir(p)
#else
#include <sys/stat.h>
#define GE_MKDIR(p) mkdir(p, 0777)
#endif
#include <PR/ultratypes.h>
#include <PR/gbi.h>
@@ -170,6 +178,29 @@ void videoEndFrame(void)
}
gfx_end_frame();
/* TEMP D70 (env-gated, strip per HANDOFF Task 3): GE_PCDUMP="first-last"
* or "first-last:step" dumps the presented frame as ./ppm/frame_NNNNNN.ppm
* (one frame behind real time — read happens after SwapWindow). Used to
* pixel-verify the intro's 3D content. */
if (gfx_opengl_pcdump_enabled()) {
static int lo = -1, hi = 0, step = 1;
if (lo < 0) {
const char *v = getenv("GE_PCDUMP");
lo = 1; hi = 0x7fffffff; step = 1;
sscanf(v, "%d-%d:%d", &lo, &hi, &step);
if (sscanf(v, "%d-%d", &lo, &hi) != 2)
hi = 0x7fffffff;
GE_MKDIR("ppm");
}
if ((int)frames >= lo && (int)frames <= hi &&
((int)frames - lo) % step == 0) {
char path[128];
snprintf(path, sizeof(path), "ppm/frame_%06d.ppm", (int)frames);
gfx_opengl_dump_bound_fbo((uint32_t)gfx_current_dimensions.width,
(uint32_t)gfx_current_dimensions.height, path);
}
}
++frames;
++fpsNumFrames;