Commit Graph
48 Commits
Author SHA1 Message Date
James Dansereau 884cfdb9b7 tools(verify): add verify.sh + crash_brief.py (speed-ups plan Step 2, 9)
verify.sh wraps the manual render-verify recipe (build-if-stale, 640x480
ini pin/restore, headless GE_PCDUMP capture, pixcount, framediff, crash
symbolication) into one command with a machine-readable PASS/CRASH/
NO-FRAMES/REGRESSION verdict + --json, for a single level or a sweep
subset. Live-tested on Windows: single-level and subset-sweep runs both
produce correct verdicts and leave the tree clean.

crash_brief.py turns a ge007.crash.log into a symbolicated, dispatch-ready
investigation brief: greps porting-notes.md/findings-index.csv for
sections covering the crashing function/file, and flags known
parked-crash signatures (D188 va_list, D189 stack overrun) so agents stop
re-investigating them. verify.sh calls it automatically on a CRASH
verdict. Live-tested against a synthetic-but-real crash log built from an
actual built symbol; caught and fixed 3 bugs before trusting it (addr2line
unresolvable under native Windows Python's PATH handling, a substring
keyword-match false-positive, a Windows drive-letter path-split bug).

Claude-Session: https://claude.ai/code/session_01P3WrUHAFQZUuhrHhiJgTUr
2026-09-04 22:09:58 -04:00
James Dansereau dbea5a30fe chore(agentic): speed-ups plan Steps 1, 4 (N2/R8/R9), 6
Static/tooling slice of docs/dev/notes/AGENTIC-SPEEDUPS-PLAN.md — no build or
runtime environment needed to verify any of it.

- Step 6 / N7: add a VERIFY: line to the investigation-brief template
  (docs/dev-process.md) and the CLAUDE.md dispatch-preflight checklist — the
  brief must now name the exact probe/verify invocation that proves the task
  done, or state that runtime verification is impossible here.
- Step 4 / N2: tools_pc/README.md — indexes the ~60 scripts with a
  living vs artifact marker so closed-finding artifacts can be archived.
- Step 4 / R8: tools_pc/gen_findings_index.py -> docs/dev/findings-index.csv
  (label, bucket, one_liner, status_raw) — grep-before-you-read aid for the
  ~200 KB finding log. Heuristic parser of the §F/§H tables.
- Step 4 / R9: tools_pc/gen_env_probes.py — re-greps live getenv("GE_*")
  sites and reports NEW/GONE drift vs docs/dev/GE-ENV-PROBES.md (keeps the
  curated prose, only checks the site map). Doc reconciled: 7 new probes
  added, GE_D54 kept as a tombstone.
- CI: validate job now runs both generators with --check.

Step 1 / N1 (HANDOFF preflight) applied in the local-only docs/HANDOFF.md.
Step 2 (verify.sh) and the HANDOFF/HANDOFF-ARCHIVE trim are left for a
session with a build environment.
2026-09-04 19:21:35 -04:00
James Dansereau bde5e0bdfa docs(release): drop fixed items from the alpha known-issues list (#21)
The HUD weapon/ammo-type icon (D187, PR #19) and blank briefing objective
text (D178, cd1ed574) are both fixed and on main. Remove them from the
release-notes and bundled-README known-issues blurbs so a v0.1.0 tag ships
an accurate list. Black sky and the front-end 3D models (Nintendo logo,
MISSION COMPLETE / mode-select) are still open and stay listed.
2026-09-03 16:37:10 -04:00
James Dansereau ed6e677d06 build(release): fall back to 7z / Compress-Archive when zip is absent (#20)
A stock MSYS2 MINGW64 shell ships neither `zip` nor `unzip`, so
`tools_pc/bundle-win.sh` failed at the packaging step for a local release
build (the M-45 handoff's documented v0.1.0 cut path). CI is unaffected
(it `pacman -S zip unzip`, so the `zip` branch is still taken).

Fallback order: zip -> 7z on PATH -> "C:\Program Files\7-Zip\7z.exe" ->
powershell Compress-Archive. Verified end-to-end: bundle builds, unpacks
to a clean dir, prepare-assets runs, ge007.x86_64.exe boots -level_09 to
frame 2100+ crash-free.
2026-09-03 16:36:42 -04:00
James Dansereau 174233f89f ci: mark bundle-linux.sh executable
Committed without the +x bit from a Windows checkout -> CI hit
"Permission denied" (exit 126) invoking tools_pc/bundle-linux.sh.

Claude-Session: https://claude.ai/code/session_01V4pK3pT83YnPsBp3Ey4RP2
2026-09-03 11:50:29 -04:00
James Dansereau a0611727e9 release: add Linux x86-64 bundle + dual-platform release notes
- tools_pc/bundle-linux.sh: package a Linux build (exe + README + licenses +
  prepare-assets) as a .tar.gz. Mirrors bundle-win.sh's ROM / size guards and
  prepare-assets assembly. No shared libs bundled — the generated README tells
  the user to install SDL2 / zlib / libGL from their distro.
- tools_pc/dist/README.md.in: made OS-neutral via @PLATFORM@ / @EXE@ / @DEPS@ /
  @LICENSE_EXTRA@ tokens; refreshed the known-issues list (dropped the stale
  "ladders don't work" line — D177 fixed M-36; added black sky + missing HUD
  weapon icon).
- .github/workflows/ci.yml: linux-build now packages + uploads the Linux
  bundle; the release job attaches both platforms to the draft pre-release.
- .github/release-notes.md: rewritten for the two-platform NTSC-U alpha with
  the prepare-assets step and current known issues.

Alpha scope: NTSC-U only, one-time prepare-assets step (Part A launcher and
Part B native converter from release-dropin-rom-plan.md remain follow-ups).

Claude-Session: https://claude.ai/code/session_01V4pK3pT83YnPsBp3Ey4RP2
2026-09-03 10:54:06 -04:00
James Dansereau 205004242c fix(release): prepare-assets must also run d88_emit.py (per-level stage setup)
d88_emit.py appends the 21 Usetup*Z stage-setup files (object placement, AI
scripts, pads/nav) to the same data/pccg-<region>/ sidecar d69 produces.
Without it the port has no setup data for any level and segfaults on load
before frame 1 (this was mis-filed as D185 'd69 regression' — it was a
missing pipeline step). Vendor d88_emit.py + its one local import
d88_propdefs.py; run d43 -> d69 -> d88 --regen.

Verified: fresh d43+d69+d88 from an assembled bundle produces pccg.bin
(3604378 B, 74 rows) and pcmodels.bin byte-identical to the known-good
sidecars; -level_09 and -level_20 boot crash-free against them.
2026-09-02 20:59:49 -04:00
James Dansereau dfab1876e4 feat(release): self-contained asset-prep tool for the alpha bundle
The Windows bundle from bundle-win.sh / CI was unrunnable on its own (D179):
the port loads two ROM-derived directories (data/pcmodels-<region>/ and
data/pccg-<region>/) that are gitignored and cannot be shipped.

Add tools_pc/dist/prepare-assets/prepare-assets.py — a stdlib-only wrapper
(no compiler, no N64 toolchain, no network) that:
  - auto-detects the ROM near the bundle, verifies SHA-1 -> region
    (NTSC-U / PAL / NTSC-J), rejects unrecognised / byte-swapped dumps
  - runs d43_emit.py + d69_emit.py against a vendored input tree
  - writes data/pcmodels-<region>/ and data/pccg-<region>/ next to the exe

bundle-win.sh assembles prepare-assets/ fresh at package time (emit scripts
from tools_pc/, filelist.u.csv + file_resource_table.inc.c + the 512
modelFileHeader.inc.c files as vendored inputs) so it can never drift from
the scripts. Vendored inputs are decomp layout/symbol metadata, not game
assets. bundled README gets a one-time "prepare-assets" setup step.

Verified end-to-end: from a bundle containing only exe + DLLs + a US ROM,
prepare-assets.py produces all four output files byte-identical to
d43_emit.py / d69_emit.py run from the repo root. Bundle 14.1 MB (< 60 MB
guard). Error paths (no ROM / unknown SHA-1) tested.

Resolves BACKLOG "Alpha release — asset-prep tool" steps 1-3; step 4
(clean-machine smoke test with the game actually running) still owed.
2026-09-02 20:43:55 -04:00
James Dansereau def0e0ac56 release: Windows alpha bundle pipeline
- tools_pc/bundle-win.sh: package a Windows build for distribution — exe +
  MinGW/SDL2/zlib runtime DLLs + README + license texts, into
  dist/goldeneye-pc-port-<ver>-win64/{,.zip,.zip.sha256}. Hard-fails if a
  ROM image or an oversized blob lands in the bundle; verifies the DLL
  closure with ldd.
- tools_pc/dist/README.md.in: the bundled README template (ROM-supply
  instructions, run steps, controls, legal), @VERSION@-substituted.
- CI: build tag trigger (v*); windows-build now packages the bundle every
  run and uploads it; new release job (tags only, contents: write scoped
  to it) attaches the bundle to a DRAFT pre-release for a human to
  smoke-test and publish.
- README: Download section + Legal reworded to cover pre-built binaries
  (engine only, no game data, permissive runtime libs bundled with their
  licenses).
- .gitignore: /dist/ (root bundle output; tools_pc/dist/ template stays
  tracked).

Cutting a release: tag vX.Y.Z-alpha, push; CI builds + drafts the
pre-release; smoke-test with a ROM, then Publish.

Claude-Session: https://claude.ai/code/session_01GULf6knbsqfsnW39tMr9WT
2026-09-02 00:44:54 -04:00
James Dansereau 5baa53eb61 tools_pc/debug.ps1: fix DLL-not-found on launch from a plain shell
The exe links MSYS2 mingw64 DLLs (SDL2, libgcc, libstdc++, libwinpthread,
zlib). Run from a normal PowerShell those aren't on PATH, so gdb reported
"During startup program exited with code 0xc0000135" and nothing ran.
Prepend the mingw64 bin dir (derived from -Gdb) to PATH before launching.

Also relax ErrorActionPreference around the gdb call so its stderr
progress lines don't raise NativeCommandError noise mid-run.

Claude-Session: https://claude.ai/code/session_01UpTM1AcBRJuTetoZNcCD5V
2026-09-01 20:28:13 -04:00
James Dansereau 142e9c8f26 D168 verified; goldens regenerated; D75 shrunk; D169 marked fixed
Built the migrated tree (MSYS2 mingw64 + ROM + sidecar regen) and ran
-level_09 with GE_PCDUMP: HUD text and the exit arrow now read
right-side-up. D114/D116 confirmed as capture artifacts.

- tools_pc/golden/*.png regenerated from a fresh correctly-oriented run
  (replaces the stopgap in-place flip); stray v*.png removed
- D75 shrunk: with upright captures the gun-barrel Bond RENDERS fine
  (walk + fire) -- the "absent" reports were the flipped capture. The
  Nintendo logo is genuinely broken (renders as 2 white blobs shifted
  left -- a real logoinst transform bug, not a flip). Cast roll not
  re-captured.
- D169 entry -> FIXED with the build/verify note

Claude-Session: https://claude.ai/code/session_01UpTM1AcBRJuTetoZNcCD5V
2026-09-01 20:22:32 -04:00
James Dansereau fa54f8fb90 docs: add operating manual, fix cross-references after doc restructure
- Commit CLAUDE.md + AGENTS.md so a fresh clone has the operating manual and
  the ABI/no-game-logic rules (trimmed of stale phase status; the personal
  pd_port path removed; points at the split docs).
- Sweep every stale doc reference across the tree after the restructure:
  PCPortResearch.md -> internals.md / dev/findings.md,
  PORT-LEARNINGS.md -> porting-notes.md,
  AGENT-WORKFLOW.md -> dev-process.md,
  docs/{GRAPHICS-BACKLOG,HANDOFF-ARCHIVE,GE-ENV-PROBES,LEVEL-*,TEXTURE-GLITCH-ANALYSIS,AUDIT-M6-*,AUDIO-PLAN}.md -> docs/dev/.
- Drop the superseded docs/PLAN-linear-level-sweep.md.
- docs/HANDOFF.md + docs/BACKLOG.md stay local-only (rolling working files);
  CLAUDE.md now handles their absence in a fresh clone.
2026-09-01 20:01:09 -04:00
James Dansereau 203549f57e D168: fix vertically-flipped GE_PCDUMP captures; close D114/D116
gfx_opengl_dump_bound_fbo wrote glReadPixels output (GL bottom-left
origin) straight into a top-row-first P6 PPM, so every GE_PCDUMP / F12
capture was upside-down. This is the entire source of the D114/D116
"HUD/text X-mirror" - four sessions chasing a fast3d mirror that never
existed, misreading inverted screenshots of asymmetric content. The
developer confirms the game renders correctly on real hardware.

- writer now emits rows top-to-bottom
- tools_pc/golden/*.png flipped in place to match + golden/README.md
- D114/D116 reclassified as capture-orientation artifacts (not deleted);
  D75 "logo mispositioned" / RC1 "photo 180 deg" flagged for re-judging
- annotations in findings.md, GRAPHICS-BACKLOG.md,
  TEXTURE-GLITCH-ANALYSIS.md, porting-notes.md D2, GE-ENV-PROBES.md

Not runtime-verified (migrated tree has no ROM/toolchain) - needs a
fresh capture to confirm text reads normally.

Claude-Session: https://claude.ai/code/session_01UpTM1AcBRJuTetoZNcCD5V
2026-09-01 19:44:39 -04:00
James Dansereau 52539d10de Prepare for public release
- README, LICENSE (MIT for the port layer), NOTICE, CONTRIBUTING.
- Restructure docs: internals.md, porting-notes.md, dev-process.md +
  dev/agentic-development.md (the research write-up), dev/ (raw record).
- Replace decomp status-badge CI with a PC-port validate workflow.
- Remove session/agent scaffolding and stale planning docs from the tree.
- Add .gitattributes (normalize line endings to LF); add screenshots.

The pre-2026-08 history is the GoldenEye 007 decompilation by Larry Ficken
(KholdFuzion) and contributors, preserved unchanged. The PC-port work from
2026-08-17 on is mine.
2026-09-01 19:28:59 -04:00
James Dansereau 1bc4928a55 d43_emit: validate PointUsage[] round-trip + index-range + non-zero (D120)
Confirms the D120 fix converter-side: 1469 PointUsage regions / 70177
s16 entries across the model set (every guard/character has several) now
byteswap-round-trip, every index < numVertices, and no region is
all-zero (the pre-M-30 bug). Guards against a future emit-gap
regression.
2026-08-31 11:13:53 -04:00
James Dansereau 742f2f820e D120: emit the opcode-0x18 PointUsage[] s16 chain (was zero-filled)
d43_emit.py reserved the PointUsage region (add_region) during layout but
the emit pass never wrote it -> all zeros -> chr.c's blood-decal walk
'index = PointUsage[index]' cycled 0->0 forever (capped by the #ifdef
PORT guard at chr.c:3328, so decals were missing/wrong, not a hang).
Now byteswaps each s16 index (numVertices entries, negative-terminated,
no remap). Full regen chain re-run, round-trip validation passes. The
chr.c guard stays as a safety net (SMALL-FIXES B3). Also: retract RC4
(palette off-by-one) in TEXTURE-GLITCH-ANALYSIS -- the current RGBA5551
decode is correct; the doc's 'spec' was ARGB1555.
2026-08-31 10:59:26 -04:00
James Dansereau 4cd8ccaeac D157: objective MinDificulty read at wrong byte on LE -> campaign never unlocks
ROOT CAUSE of 'complete Dam, Facility stays locked'. The type-23
objective record's MinDificulty is a big-endian s32 at word 3; on N64 the
value is its low byte == offset 0xF, and struct objective_entry reads
'difficulty' as s8 @0xF. d88_propdefs.py bswap32's word 3, so on
little-endian the value moves to offset 0xC -> objective_entry.difficulty
@0xF reads 0 for EVERY objective. get_difficulty_for_objective() then
returns Agent(0) for all, so objectiveIsAllComplete() on Agent evaluates
objectives that should be difficulty-gated out (Dam obj 0 'Neutralize all
alarms' is Secret Agent; obj 1/2 are 00 Agent) -> they're incomplete ->
never returns TRUE -> end_of_mission_briefing() / the unlock write never
runs. User was right: you don't touch an alarm on Agent Dam.

Fix (D151-class, #ifdef PORT): reorder struct objective_entry's word-3
tail so 'difficulty' reads @0xC (post-swap position); N64 layout kept
under #else. Also fixes dump_objectives.py (was reading byte 0xC = the
BE high byte = always 0; hence LEVEL-OBJECTIVES.md said 'Agent' for
everything). Dam now decodes obj0=Secret Agent, obj1/2=00 Agent,
obj3=Agent. -level_09 coverage identical (phash delta is the M-30 RC2
texture change, not this).
2026-08-31 10:25:16 -04:00
James Dansereau 290e85c50d M-29 docs/tools: SMALL-FIXES notes, LEVEL-OBJECTIVES decode, animgen dump helpers 2026-08-31 07:42:37 -04:00
James Dansereau 51212024e8 M-27: debug.ps1 builds by default (-NoBuild to skip) so playtests never run a stale exe 2026-08-30 15:30:32 -04:00
James Dansereau cea9ce0df9 M-27: D146 unknown-GBI-opcode -> end DL instead of abort(); tools_pc/debug.ps1
D146 (port/fast3d/gfx_pc.cpp): an unrecognised opcode in the DL walk called
sysFatalError() -> abort() -> whole process dies with no ge007.crash.log
(the crash handler does not cover abort). Seen on the real post-Dam
MISSION COMPLETE screen: "[FATAL] Unknown GBI opcode 0x40c03 at
0x70161a8c" - a front-end 3D model (dossier, D75/D144 family) whose display
list was never built, sitting at a valid DRAM address full of junk. D144
guarded the bad matrix pointer; the walk then hit more garbage. Now: log
loudly (rate-limited x20) and `return` to end the (sub-)DL so the frame
finishes. A real level-render desync is still visible in the log.

tools_pc/debug.ps1: launch the port under gdb from the repo root so a
playtest crash always leaves a backtrace (abort/fatal or silent exit).
-Menu <id> sets GE_STARTMENU; passes through -level_XX etc. Prints the
FATAL line + backtrace + crash log when the game exits.

Verified: GE_STARTMENU=13 mission-complete -> Next chain renders, no abort,
no crash (D75 model cosmetics aside).
2026-08-30 15:27:23 -04:00
James Dansereau 710d02f802 M-26: [B/C] level_sweep STALLED verdict, raw mouse input opt-in
- level_sweep.sh: frames rendered then frozen with process alive ->
  STALLED (froze at N/M frames) instead of a false PASS (the Silo
  -level_20 fly-down / D117-D134 load case). Retried once like NO-FRAMES.
- input.c: Input.MouseRawInput=1 (default 0) sets SDL relative
  system-scale + warp hints off in inputInit -> no OS pointer accel on aim.
- docs: M-26-QOL-REVIEW.md + HANDOFF section.
2026-08-30 12:55:12 -04:00
James Dansereau bc97cdab0a M-25: playtest.sh — accept level names, --list, auto-symbolicate crash PCs 2026-08-30 12:31:08 -04:00
James Dansereau 73820a0c24 M-23: WS6 Facility playtest — D135/D137/D138/D139 (firefight, aim, freeze, unload)
First real human WS6 playthrough. Each crash is a subsystem the 21-level
boot sweep never exercises.

- D135 (propobj.c) bgTestHitOnObj: unported N64 GBI parser — raw 8-byte-Gfx
  byte/word indices desync on the PC 16-byte Gfx stream, walk off the DL.
  Ported #ifdef PORT to gdl->words.w0/.w1 shifts (mirrors pd_port bg.c:3635).
  Sub-fix: objHit:9717 div-by-0 — texnum recovery not portable, force -1.
- D137 (gunfire.c) gunDrawSight: s32 sp54 holds a Gfx*; 4-byte slot splices
  adjacent stack into the pointer -> wild DL write on right-mouse aim.
  #ifdef PORT: Gfx *sp54.
- D138 (snd.c) sndCreatePostEvent: parked audio never drains evtq->allocList,
  per-objTick positional-sound posts stall the main thread ("1 room from the
  end" freeze). #ifdef PORT no-op until audio lands (D77).
- D139 (cleanup_objects.c) cleanupObjects: read the propDef type via
  (u8)obj[0] — correct only big-endian (type = low byte of the header word).
  On LE that's extrascale -> walk never hits PROPDEF_END -> runs off the blob
  -> frees garbage -> stage-unload crash. #ifdef PORT uses pdef->type.
  UNVERIFIED (fast teardown test hit D140 first).

Open: D140 pause-menu watch-model NULL matrix; D118c aim-mode mouse-down
-> crouch. New harness tools_pc/repro_gdb.sh + build-pc/d136_cmds.txt.

-level_09 unregressed (91.65%). Docs: PCPortResearch §F D135-D140,
PORT-LEARNINGS §B/§C, HANDOFF M-23 + M-24 next-task.
2026-08-30 11:28:50 -04:00
James Dansereau 78c0f72157 level_sweep.sh: early-exit watchdog; M-22 re-sweep 20/21 PASS
The 45s watchdog is now a ceiling, not a flat per-level cost: a level is
killed as soon as its whole GE_PCDUMP frame set is on disk (PASS) or the
ppm count has been frozen for STALL_SECS (default 18). EXPECT_FRAMES is
derived from SWEEP_DUMP (start-end:stride). Full 21-level sweep ~7min
(was ~16).

M-22 full re-sweep after D134: 20/21 PASS. Facility(34) + Jungle(37) --
the M-20 re-verify failures -- both PASS clean, confirming D134 was their
cause. Only Cuba(54) crashes: known D129 residual (bare -level_54 hits the
cast/credits scroll referencing an unloaded text bank), parked with D76.
2026-08-29 23:57:03 -04:00
James Dansereau 46d4f03e5a D134: task-done SP/DP events must not be dropped into a full sched interruptQ
The frame-2 boot hang blamed on D117 flakiness since M-13 is a real bug:
osSpTaskStartGo posted OS_EVENT_SP/DP with OS_MESG_NOBLOCK into the 8-slot
scheduler interruptQ that the 60 Hz VI pacemaker also fills. A gfx task runs
the whole frame synchronously on the sched thread, so a slow early frame lets
retraces fill the queue and the done event is silently discarded -> curRSPTask
never clears -> pendingGfx never clears -> permanent stall.

portPostEventForce() drops the oldest (stale retrace) message and retries;
portPostVIEvent reserves 2 slots. -level_09: 6/6 boots to frame 600, 0
heartbeats (pre-fix 1/3).

Also hardens tools_pc/level_sweep.sh: 45s watchdog, lockdir serialization,
stray-process kill, one retry on NO-FRAMES.
2026-08-29 23:32:17 -04:00
James Dansereau 5e9ea50add D132: fix propDef union-index slots for LINK/SWITCH/LOCK_DOOR/SAFE_ITEM
Types 14/19/38/44 declare their Index{1,2,3} fields as a union with a
pointer, so on PC each is an 8B/8-aligned slot (Index1@8, Index2@16,
...) and the record ends in a *next the proplvreset2 walk writes. The
converter was laying the N64 tight-4-byte index words at PC bytes 4/8/12,
putting Index1 where the compiler reads Index2 and leaving Index2 zero.
Non-crashing but silently broke switch-doors, dual weapons, padlocked
doors, and safe/safe-item links.

- tools_pc/d88_propdefs.py: PROPDEF_PC_BYTES 14/19->32, 44->40; handler
  emits Index{k} into low 4B of the 8-aligned slot at PC 8+8*k.
- loadobjectmodel.c sizepropdef() PORT: LINK/SWITCH/LOCK_DOOR->8,
  SAFE_ITEM->10, TAG stays 6.

Verified: -level_09 framediff 3/3 PASS; -level_20 identical to a
freshly-built pre-D132 baseline (frame_000320 delta is pre-existing
stale-golden/Silo slowdown, not a regression); Archives/Streets/Dam
load+render+no-crash.
2026-08-29 23:16:16 -04:00
James Dansereau 73dfa8d81f D126: objective sub-record ->next pointer growth desyncs propdef walk
criteria_picture (30) / criteria_roomentered (32) / criteria_deposit (33)
/ setup_objective_text (35) each end in a T*next list pointer that the
setup walk writes unconditionally (set_parent_cur_obj_* /
setup_briefing_text_entry_parent). On x86-64 the pointer widens 4->8B and
8-aligns at offset 16, making the struct 24B/6w (N64 16/20). d88_propdefs.py
emitted N64 size -> the runtime 8-byte ->next store clobbered the next
propdef record's header -> walk desync -> command indices drifted ~100 ->
setupDoor linkedDoor resolution landed on the wrong record -> crash.

Fixes C3r (Bunker2 -level_27), C4 (Depot -level_30), C6 (Surface2 -level_43)
in one change. Sweep 13 -> 16/21. Bunker1/Silo/Aztec/Archives/Egypt/Train
unregressed; d125_check.py 21/21 MATCH.

- tools_pc/d88_propdefs.py: PROPDEF_PC_BYTES[30/32/33/35]=24 + typed handler
- src/game/loadobjectmodel.c: sizepropdef PORT returns 6 for those types
2026-08-29 19:35:19 -04:00
James Dansereau 060a7252e5 D125: fix d88_emit.py:374 8-byte slice / 4-byte literal (Aztec C3 crash)
emit_pad()'s widened-stan zero-fill assigned a 4-byte literal to an
8-byte bytearray slice. Python slice-assign with a short RHS silently
deletes the surplus bytes, so `out` shrank 4 B per pad+boundpad; once
len(out) dropped below a later leaf write's target offset, the write
clamped to empty-at-end and inserted there, drifting/truncating the
boundpad plink-name blob (Aztec pad33 "p138d2" -> "8d2") -> stanPackId
reject -> NULL stan -> door model=NULL -> crash propobj.c:13601.

Fix: 8 NUL bytes (only width-mismatched out[ write in the file).
All 21 Usetup*Z sidecars regenerated. Aztec (-level_28) now PASSES
(was C3 CRASH); Bunker1/Silo unregressed; d125_check.py 21/21 MATCH.
Sweep 12 -> 13/21.

Bunker2 (-level_27) still crashes: separate DOOR-tail linkedDoor
layout bug in d88_propdefs.py (C3r) -- linkedDoor is resolved from
linkedDoorOffset (read-before-write int id, D123 pattern). Folded
into the converter write-audit.

M-16 probes reverted; d125_check.py kept as the offline regression guard.
Generalisable quirk added to PORT-LEARNINGS.md §C.
2026-08-29 19:12:52 -04:00
James Dansereau 710d0b12b5 docs: correct stale 'opaque passthrough' propDefs comment in d88_emit.py (D122/D123/D125) 2026-08-29 16:39:58 -04:00
James Dansereau 9d118e0b13 M-13: 21-level sweep matrix + D123 (C1 fix) + D124 (C2 Jungle fix)
WS4 level sweep driven to completion: 12/21 solo levels PASS
(load + render + no-crash). Full matrix in docs/LEVEL-STATUS.md,
reusable runner tools_pc/level_sweep.sh, WS6 checklist docs/LEVEL-PLAYTEST.md.

Fixes:
- D123 (tools_pc/d88_propdefs.py): crash class C1 on 6 levels
  (Dam/Runway/Frigate/Statue/Streets/Cradle). D122 fallout - the widened
  Vehichle/AircraftRecord.ailist slot was zeroed instead of carrying its
  read-before-write integer AI-list id, so prop.c -> ailistFindById(0) ->
  GAILIST_AIM_AT_BOND -> ai() ran a CHR aim list with a NULL chr ->
  chrIsNotDeadOrShot(NULL). New OBJ_ID_WORDS arm. 4 levels -> PASS,
  Runway falls through to C2. BUNKER1/Silo unregressed.
- D124 (port/src/gimgfixup.c): crash class C2 on Jungle. Compiled
  explosion-DL pointer-sync keyed on an IMAGESEG marker that texLoad()
  had already erased -> compiled globalDL_0xNNN arrays kept link-time
  IMAGESEG words -> 0xABCDxxxx into fast3d on first explosion draw.
  Latent on every level. Detect the slot from the compiled array instead.

Diagnosed, NOT fixed (write-ups in PCPortResearch.md SF):
- C2-GDL (D124-Facility addendum): Facility/Runway - model texture-pool
  path yields a garbage tex->data; every model file loads at an odd
  bank base (mempAllocBytesInBank does no alignment). Suspect
  d43_emit.py/pcmodels sidecar texture-blob offsets (N64 8B vs PC 16B Gfx).
- C3/C6 (D125): Aztec/Bunker2/Surface2 - converted propDefs blob in RAM
  doesn't match d88_propdefs.py offline output (garbage after record 0);
  runtime sizepropdef() walk drifts ~100 records -> wrong door/modelid.
  d88_propdefs.py convert_stream + sizepropdef() PORT strides verified
  correct; suspect a dest-offset/pd_end bug in d88_emit.py propdefs emit.

Remaining: 9 crashes / 6 classes (C2-GDL, C3/C6, C2m Jungle G_MTX,
C5 Control BG portal, C4 Depot BG tile/room, C7 Surface1 sndSetupSound).
Priority order in docs/LEVEL-STATUS.md.

Also folds in the in-progress docs restructure (HANDOFF/AGENTS/CLAUDE,
PCPortResearch D121/D122 SF/SH entries, PLAN-linear-level-sweep,
HANDOFF-ARCHIVE, BACKLOG).
2026-08-29 16:36:18 -04:00
James Dansereau a8b174a829 D122: fix propDef obj half-swap for ObjectRecord-derived types
tools_pc/d88_propdefs.py had a per-type handler table plus a generic
fallback. Six `inherits ObjectRecord` propDef types (47 TINTED_GLASS,
39 VEHICHLE, 40 AIRCRAFT, 45 TANK, then 13 AUTOGUN, 20 AMMO/MultiAmmoCrate)
had no handler and fell through to the generic arm, which bswap32's N64
word 1. For an ObjectRecord that word is [s16 obj][s16 pad] and needs a
half-swap, not a word-swap - so the model id `obj` landed in the wrong
16 bits, giving a garbage/OOB index into PitemZ_entries[] and a crash in
modelLoad (loadobjectmodel.c:393) / modelInitRwData (model.c:6249).

BUNKER1/Silo were unaffected because neither emits any of the six types.

Fix (converter + matching walk stride, no game logic):
- d88_propdefs.py: OBJ_TAIL_DESC handler - real 144B PC ObjectRecord
  prefix + typed tail (pointers widened 4->8B/8-aligned, [u16|u16] words
  half-swapped). PROPDEF_PC_BYTES 39/40 176/180->208, 45 224->248.
- loadobjectmodel.c sizepropdef() #ifdef PORT arm: VEHICHLE/AIRCRAFT
  44/45->52, TANK 56->62 words (N64 values in the #else sizeof switch).

Verified: -level_33/34/35 clear the model-load chain (they now crash
further in, in unrelated chr/fast3d code - separate blockers);
-level_09 BUNKER1 and -level_20 Silo unregressed. Sidecars regenerated
(d88_emit.py --regen, 21/21). Full write-up: PCPortResearch.md D122
(uncommitted, rides with the in-progress docs restructure).
2026-08-29 15:26:22 -04:00
James Dansereau ffc3112037 D117: framediff.py visual-regression tool; root-cause frame nondeterminism
Root cause (HIGH confidence): pure variable-timestep frame pacing, not
PRNG/uninit state. osGetCount() is wall-clock on PC (libultra.c:88), and
frametiming.c waitForNextFrame advances game logic by however many 60Hz
ticks of real time elapsed per render -> two runs diverge 32-69% of
pixels by frame 200-440. random.c seeding verified deterministic and
correct; the PRNG stream only diverges as a victim of the pacing jitter.

GE_DETERM fixed-tick mode: assessed NOT narrow (frame-coupled virtual
clock deadlocks the boss.c retrace gate / pre-first-frame waits;
robust version redesigns VI retrace + pacemaker semantics). Deferred
with a full design in PCPortResearch.md F/D117 per AGENTS.md.

tools_pc/framediff.py: structural/tolerant frame compare (16x12 grid
mean-colour delta + non-clear-% swing + 16x16 aHash), --mask for the
animated HUD, --exact for a future deterministic build, --update to
refresh tools_pc/golden/. Reads .ppm and .png (built-in PNG decoder).
Validated: two nondeterministic re-runs pass structural, --exact
reports the real 32-69%, a wrong-frame pair fails.

No game-logic or port-layer C changes; no probes left in tree.
2026-08-29 10:49:15 -04:00
James Dansereau 42eacc4aae tools_pc: D115 golden frame baseline for regression diffing (-level_09 BUNKER1) 2026-08-29 10:38:23 -04:00
James Dansereau 3a02ded079 D112: fix d43_emit put_f32 byte-swap off-by-one (skeletal models collapsed)
put_f32 reversed bytes doff+1..doff+4 instead of doff..doff+3, dropping
each float's MSB and pulling in a byte of the adjacent field. Corrupted
every f32 in converted model rodata (joint Origins, LOD distances, BSP
planes, bounding radii) for all sidecar-loaded models. Compiled-in
front-end models were unaffected, hence 'logo fine, every guard a line'.

render_pos[] joint matrices now coherent; BUNKER guards render as
humanoids and monitor screens draw content. Regen: python
tools_pc/d43_emit.py ntsc-final.
2026-08-29 09:21:12 -04:00
James Dansereau 2f3bc74417 D88.6: intro CAMERA lang1c is a u16 pair, not an s32
SetupIntroCamera.lang1c is union { u16 lang_index[2]; u32 lang_ptr; }
and bondview_r.c reads lang1c.lang_index[1]. The intro converter
bswap32'd the whole word, which swaps element 0 and element 1 -- the
consumer then read the wrong language slot id, indexed g_LangBanks out
of range and crashed in langGet on a NULL bank. Byte-swap each u16 of
lang1c in place instead; lang20 stays a genuine s32 bswap32. Requires
d88_emit.py --regen.
2026-08-28 09:58:22 -04:00
James Dansereau 5dbe8a48c2 D88.5: byte-swap the stan tile id-hi half in the converter
stanMatchTileName reads a tile's packed name id through a StandTilePoint
alias: (u16)tile->x == stanIdHi and *((u8*)&tile->y) == stanIdLo. D78
left the id/room word as a verbatim byte array ("id is provably dead"),
which missed this aliased scalar read -- on little-endian PC the u16
load of the big-endian id-hi bytes came back byte-swapped, so every
name lookup during pad setup missed (0/276). Swap bytes 0-1 of the
id/room word in d69_emit's stan converter; idLo (byte 2) and room
(byte 3) stay put. Verified: 273/273 matches, pads now get real stan
tiles. Requires sidecar regen (d69_emit.py + d88_emit.py --regen).
2026-08-28 09:58:14 -04:00
James Dansereau 695f42d383 D88.4: emit converted propDefs into the Usetup*Z sidecar
d88_emit.py runs d88_propdefs.convert_stream over the propDefs region and
feeds its growth into the cumulative delta so intro + every later table
shifts correctly.
2026-08-28 09:16:13 -04:00
James Dansereau f2281ed661 D88.4: propDefs record format spec + N64->PC converter (d88_propdefs.py)
The propDefs stream is a flat s32[]; each record's serialized N64 word count
is fixed across all 21 levels (verified: the getools C sources tile every
propDefs region byte-for-byte against the ROM). Every pointer member is 0 in
the file, so records containing pointers grow 4->8B per pointer on PC.

d88_propdefs.py rewrites each record to its native PC struct layout: header
word (u16+u8+u8) and _mkword half-pairs swapped independently, scalars
bswap32'd, pointer slots widened to 8 zero bytes, runtime areas zero-filled.
d88_layoutprobe.c (expanded) is the compiler-verified sizeof/offsetof source
for PROPDEF_PC_BYTES. Converts + walk-verifies all 21 levels clean.
2026-08-28 09:16:05 -04:00
James Dansereau f333a713fd D88.4: d88_emit.py --regen flag + PC layout probe for propDef records
--regen drops all Usetup*Z rows (manifest + bytes) and rebuilds the sidecar
from the surviving d69 rows before re-converting, for clean converter
iteration. d88_layoutprobe.c prints sizeof/offsetof for every propDef record
struct against the real port include chain.
2026-08-28 09:04:46 -04:00
James Dansereau fbfa2f8231 D88.4: add propDefs region scanner (analysis helper, WIP)
Read-only tool to enumerate PROPDEF_* record types per Usetup*Z file.
Walk currently drifts on the sizeof()-derived record types (DOOR/GUARD/
PROP/etc) because their exact N64 word sizes aren't yet confirmed -- that
confirmation is the core remaining D88.4 spec task.
2026-08-28 08:33:31 -04:00
James Dansereau cca7d0bfad D88.2: Usetup*Z stage-setup offline converter (tools_pc/d88_emit.py)
Converts the 21 per-level Usetup*Z stage-setup files to PC layout,
appending rows to the same pccg.bin/manifest sidecar as d69_emit.py.
Delta-relocates the 8 pointer-widened sub-tables (header 40->80B,
waypoint/waygroup/PathRecord/AIListRecord/PadRecord/BoundPadRecord/pname)
and byte-swaps the s32 ID/offset fields. propDefs left as an opaque
relocated passthrough for now (D88.4).
2026-08-28 08:30:56 -04:00
James DansereauandClaude Sonnet 5 2acb7f6a42 D69.2: bg/stan offline converter (tools_pc/d69_emit.py)
Implements the D80-D81 format spec: size-preserving bswap of bg .seg
header/room/envdata tables, growth+relocation of the portal_data_entry
table (8B N64 -> 16B PC, offset_portal stays a real dereferenced pointer)
and its point-data blob (relocated by portal_delta, content size
unchanged), and the Tbg_*_stanZ prefix room-offset array (4B N64 -> 8B PC
slots, matching StanPrefixRecord's PC-compiled pointer alignment) plus
per-tile bswap of mid/tail halves and point coordinates. Round-trip
validated against BUNKER1 (bg/bg_sev_all_p.seg, Tbg_sev_all_p_stanZ):
header offsets, MaxNumRooms walk, portal targets/connectedRoom bytes, and
all 1066 stan tiles match the N64 source exactly.

Output: data/pccg-<region>/pccg.bin + manifest.csv (gitignored, same shape
as pcmodels.bin/D43). Regenerate: python tools_pc/d69_emit.py ntsc-final.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C3xib6jjMV8o2iE4JfixAo
2026-08-27 21:02:02 -04:00
James Dansereau 0010270f1b Repo cleanup: reference footage to docs/reference/, ignore local artifacts
- Move the two N64 reference screenshots out of the repo root into
  docs/reference/ (n64-footage-2026-08-24.jpg, n64-footage-gun-barrel.png);
  HANDOFF.md now points there.
- Gitignore screenshots/ (session captures) and tools_pc/mupen64/ (local
  Mupen64 Plus checkout used for N64 reference footage; 8 MB of binaries).
- Commit tools_pc/mtxtest.c, the standalone fast3d matrix-convention
  harness that HANDOFF/D72.3 docs reference as kept-for-reference.
- Refresh PORTING.md: status was still 'scaffolding / building once
  implementation lands'; now reflects Phase 2 in progress and points at
  docs/HANDOFF.md for current session state.
2026-08-27 19:27:05 -04:00
James Dansereau 8f631462f3 PC port: phase 2 — D59-D68 milestone: full intro renders; blocker is D69 (BG-file BE headers)
D59/D64: the 'external GL DLL' crash was an unbounded blood-RLE write
(sentinel die_blood_image_end lost its N64 section adjacency on PC)
clobbering the gun-barrel sub-DL at ptr_logo_and_walletbond_DL+0x200.
D60: osPiStartDma DMA-target validation (VirtualQuery-based) in libultra.c.
D61/D63: TEMP env-gated ROM-read / VTX-pool / menu probes (strip list in
HANDOFF Task 3). D62: OSMesgQueue/OSScMsg PC layout verification.
D65/D65b: enum HEADS/BODIES signed sentinels under PORT (PC GCC picks
unsigned underlying types; N64 semantics were signed).
D66: romCopyAligned + ramrom replay 64-bit pointer width (s32 truncation
of .bss targets > 4 GiB). D67: struct image_entry N64 layout
reconstruction (dataoffset:24 first, all-u32 bitfields, sizeof==8).
D68: Globalimagetable BE->LE fixup (port/src/gimgfixup.c): bswap the
IMAGESEG-marked Gfx w1 words in the 17 ROM-copied DLs + sImageTableEntry.
index in the 32 tables after texReset's romCopy; marker scan moved to
bytes 6..7 under PORT; compiled globalDL shadows synced post-texLoad for
explosion.c's g_ExplosionDisplayLists.

Result: game boots, plays intro music, and renders the entire ~3.5-minute
intro (logo -> gun barrel with Brosnan -> cast screen) at ~59 fps. New
blocker D69: load_bg_file faults on first stage load - BG-file headers are
N64 big-endian segment-0x0F offsets read LE on PC. Full analysis + strategy
in PCPortResearch.md §F/D69; HANDOFF rewritten for the D69 session.
2026-08-24 03:27:29 -04:00
James Dansereau 330b2bae46 PC port: phase 2 — Plan B executed (D50): offline model sidecars + port plumbing; texCopyGdls w1 fix
- tools_pc/d43_emit.py: full emit pass, 512/512 round-trip validated;
  data/pcmodels-<region>/pcmodels.bin + manifest.csv (gitignored, regenerate)
- port/src/pcmodels.c(+h): cart-extension reservation, sidecar copy, one-shot
  file_resource_table patch; romdata.c/h extended
- objecthandler_2.c: #ifdef PORT hook — pcmodelsPatchTable() + poolRemaining=0
  reset (closes D48.3/R6 reload hazard)
- language.c/textrelated.c: runtime C fixups for lang-bank BE offset tables
  (romdataFixupLangBank) and font re-layout (romdataFontPcSize/FixupFont)
- front.c: legal-screen uninitialized-pointer UB seed (N64 survived by luck)
- tex.c (D50.6): texCopyGdls copied only w0 of each 16B Gfx slot on PC —
  compaction scratch kept stale mempool w1s -> garbage G_SETTIMG -> OOB in
  import_texture_rgba16. Full-slot copy under #ifdef PORT; byte-proven vs
  PlegalpageZ sidecar (w0s match, w1s stale). Model GDLs now execute.
2026-08-23 10:22:03 -04:00
James Dansereau 73ea03734d PC port: phase 2 — D48 review: re-plan D43 as offline pre-conversion (Plan B)
- D48 finding in PCPortResearch.md §F: d43_convert.py does not emit bytes
  (layout+remap validation only); Plan B = Python emit pass -> data/pcmodels-<region>/
  RZ sidecars served via port-layer hw_address/rom_size table patch (no C
  converter); Plan A kept as fallback with corrected staging guard (D_N64+D_PC<=avail)
- Moved all 23 d43_*.py investigation scripts from gitignored build-pc/ to tracked
  tools_pc/ (spec is now versioned); d43_cover.py path ref updated; converter
  re-run from new location: ALL CLEAN
- HANDOFF.md rewritten as two-session prompt: Session A = independent review of
  Plan B (8 claims, each with falsification criterion -> finding D49), Session B =
  execute (emit pass + round-trip validator, romdata plumbing, one-line hook,
  pixel-assert soak); verified-facts converter spec preserved
- AGENTS.md phase status updated to D48 re-plan
2026-08-22 18:43:04 -04:00
James Dansereau bd56cff073 PC port: phase 1.5 — ROM dump verification groundwork
No-Intro 'GoldenEye 007 (U) [!].z64' placed in data/ and verified
byte-identical (MD5 70c52588...) to the dev ROM copies
(data/ge007.ntsc-final.z64, baserom.u.z64). Header magic/title/country
check out; embedded CRCs DCBC50D1/09FD1AA3 re-computed and matched via
tools/n64cksum.c's CIC-6102 routine.

Adds tools_pc/romverify.c: one-shot integrity check (size, GE header
magic at 0x00, title, country byte, CRC1/CRC2) so any future ROM swap
is detectable in seconds. Fingerprint + provenance recorded in
docs/HANDOFF.md; gitignore the built romverify.exe.
2026-08-21 20:29:03 -04:00
James Dansereau 8482ba3ed1 PC port: Phase 1.5 — root-cause D32 Part B (D33): ROM big-endian
structured fields + x86-64 loop-stride bug; fix designed, handoff updated

- Finding D33 in docs/PCPortResearch.md: the .z64 stores multi-byte
  structured fields in big-endian byte order (bit-packed streams raw);
  per-field load-time transform rule (u32 bswap32 / u16 bswap16 / u8
  identity) validated over all 173 animation entries; full blob layout
  mapped; exact reproduction of the gdb-observed garbage values.
- Second root cause: expand_ani_table_entries iterates s32** (+8 bytes/iter
  on x86-64 vs +4 on N64), rebasing only even-indexed entries (verified in
  compiled binary). Fix design: romdataFixupAnimationData() in port/, one
  call between romCopy and expand, s32* loop stride fix.
- docs/HANDOFF.md rewritten with the implementation task; AGENTS.md phase
  status updated. tools_pc/disasm.py (BE ROM research tool) added.
2026-08-21 20:18:38 -04:00