structured fields + x86-64 loop-stride bug; fix designed, handoff updated
- Finding D33 in docs/PCPortResearch.md: the .z64 stores multi-byte
structured fields in big-endian byte order (bit-packed streams raw);
per-field load-time transform rule (u32 bswap32 / u16 bswap16 / u8
identity) validated over all 173 animation entries; full blob layout
mapped; exact reproduction of the gdb-observed garbage values.
- Second root cause: expand_ani_table_entries iterates s32** (+8 bytes/iter
on x86-64 vs +4 on N64), rebasing only even-indexed entries (verified in
compiled binary). Fix design: romdataFixupAnimationData() in port/, one
call between romCopy and expand, s32* loop stride fix.
- docs/HANDOFF.md rewritten with the implementation task; AGENTS.md phase
status updated. tools_pc/disasm.py (BE ROM research tool) added.
Summarizes Phase 1.5 state, the D32 Part B immediate task + gdb method, the
standing ABI/layout fix procedure, and environment gotchas for a fresh agent
session. AGENTS.md now points to it for new sessions.
- D31 (resolved): replace GE's gzip-1.2.4 inflate with real-zlib
decompressdata() in port/src/rzdecomp.c; exclude src/game/decompress.c +
zlib.c from the PC build. Root cause: struct huft is 8B on MIPS but 16B on
x86-64, overflowing load_resource's fixed 0x2100 buffer and clobbering its
frame/retaddr on the first file load. mainThread now boots past all language
files into bossInitMainthreadData().
- D32 (in progress): ROM-serialized structs with 32-bit pointer fields
misalign when read as 64-bit pointers. Part A done: ModelAnimation's
bitDescriptors/bitStream are now u32 (N64 layout preserved) with casts at the
two use sites in model.c. Part B (load-time rebase in expand_ani_table_entries)
is open — see docs §F/D32 + §H.
- Docs: D31 resolved, D32 recorded (A done / B open), Phase 1.5 added, new §H
handoff section. AGENTS.md: refined non-negotiable #2 to "game logic is
unmodified" with a narrow ABI/layout exception; refreshed phase status.
- Replace setjmp/longjmp green threads with real pthreads (8MB stacks);
PortQueue side table (mutex+condvar) for OSMesgQueue; dedicated tick
thread posts VI retrace + services timers (D24).
- Dual-mapped N64 DRAM: one 8MB store at V1 0x70000000 (s32-safe, all game
RAM symbols) and V2 0x80000000 (KSEG0 mirror for offset|0x80000000
rebuilds). Windows: CreateFileMappingW + MapViewOfFileEx; POSIX: memfd
+ MAP_FIXED. dram_syms.s pins cfb_16/_bssSegmentEnd/tlb block (D25).
- Address-width shims: PHYS_TO_K0 identity, OS_K0_TO_PHYSICAL = offset from
V1 base, OS_PHYSICAL_TO_K0 identity; fast3d seg_addr() remaps small
offsets into V2 (D26).
- ASLR off (-Wl,--disable-dynamicbase): absolute symbols referenced through
pointer-typed externs (.refptr + BASE reloc) must not rebase; startup
sanity check on image base 0x140000000 (D27).
- osPiStartDma posts its completion message on every path (fixes
romReceiveMesg deadlock) (D29).
- SEH-free crash handler: raw fault log first, validated manual EBP-chain
walk second (-fno-omit-frame-pointer), TerminateProcess; crashDumpThreads
via Toolhelp32 for heartbeat hang dumps (D30).
- PD fast3d integrated (port/fast3d/), replaces gfxstub.c; real src/sched.c
compiled; mainproc runs as a real pthread, host thread pumps SDL events.
- Docs: findings D24-D31 + Phase 2 status (section F/G).
Current state: boots through bossInitMainthreadData to langInit(), then
SIGSEGVs on the first file load — stack frame clobbered by a table of
truncated s32 pointers (D31, open). Next: attribute the write with
'set scheduler-locking on' + hardware watchpoint.
- scripts/gen_romassets.py: emit absolute cart-address asset symbols
(0x10000000 + rom_offset) from filelist.<r>.csv + the asset .s files,
in file_resource_table.inc.c order. US region (romassets_u.s).
- romdata.c: load/validate the GE-specific header (magic, name @0x20,
cart ID @0x3C, country @0x3E) and map the .z64 at cart base 0x10000000
so every &asset-symbol is a live host pointer.
- libultra.c: osPiStartDma/osPiRawStartDma now service reads as
bounds-checked memcpys from the mapped ROM (synchronous completion).
- system.c: implement sysResolvePath ($S data dir, $E exe dir).
- video.c: real SDL2 window + GL context + clear/swap frame loop with
FPS accounting; main.c runs a Phase 1 demo loop (ESC quits) until the
software RSP/scheduler can drive mainproc().
- CMakeLists: project(... ASM), wire romassets_<region>.s, drop
assetstubs.c (replaced by generated symbols); n64stubs.c keeps only
pure-RAM segment markers.
- docs: findings D18-D23 (header offsets, BE music table, .rz tracks,
scanner-prefixed manifest names, CMake ASM gotcha, absolute-symbol
approach).
Verified: baserom.u.z64 (12MB) passes validation and maps at 0x10000000;
window opens and renders until killed; asset offsets spot-checked against
ROM bytes (music bank seqCount=63 BE, first track offset 508).
All ~235 translation units now compile and the target links under MSYS2
GCC 16 + CMake/Ninja (ninja ge007 -k 0: 236/236, zero errors). N64 build
untouched; every src/ change is -DPORT-gated.
Portability fixes (findings D7-D17 in docs/PCPortResearch.md section 11):
- AI X-macro system: SWITCH() call sites activated to the author's
hand-written byte-identical references under PORT; port/shim/
bondaicommands.h + bondconstants.h + CPPLib.h re-emit the IDO paste
tricks (isSubroutine, file-record macros, generated CALL) GCC-clean.
- 64-bit pointer-in-word initializers: monAnim jump targets now go
through an index table resolved by the interpreter (chrai.h/propobj.c);
UsetuplenZ.c credits pointer PORT-gated to 0.
- IDO printf engine compiled for PC (xprintf/xlitob/xldtob) with a
shadow xstdio.h resolving the _Printf prototype clash.
- Host-libc gaps stubbed in port/: bcopy/bzero, __libm_qnan_f,
tlbmanage*, g_ViXScales/YScales; chrObjRandom PRNG ported from .s.
Asset data strategy (D16): single-file assets compiled for real
(animation tables, fonts, textures, rareware logo); obseg per-file .c
excluded (colliding file-local globals the N64 build isolates via
per-ELF linking); 758 ROM-derived table symbols + ramrom_* + segment
markers stubbed in port/src/assetstubs.c / n64stubs.c.
fast3d is a no-op placeholder (port/src/gfxstub.c) until Phase 2 brings
in the real software RSP. Binary not yet runnable end-to-end: ROM
loading in romdata.c is still a TODO.
James Dansereau