Commit Graph
19 Commits
Author SHA1 Message Date
James Dansereau 8c3ce1c688 Build provenance: origin URL + codename in --version and startup log; fork policy in CONTRIBUTING (#82)
- versioninfo gains GE007_VERSION_CODENAME (larkspur) and GE007_ORIGIN_URL,
  printed by --version (new 'origin' line) and the startup log. A build keeps
  pointing at this repository no matter who compiles or renames it.
- CONTRIBUTING: new 'Forks and derivative works' section — rebranding,
  localizing and extending are welcome without permission; attribution (with
  a link) and preserving/linking the docs/dev engineering record are the ask;
  stripping the attribution is what disqualifies a fork as a descendant.
2026-09-13 22:14:41 -04:00
James Dansereau 86440e3b58 fix(linux): survive the decomp's latent stack-buffer overruns (D189) (#24)
* fix(linux): survive the decomp's latent stack-buffer overruns (D189)

The decompiled game code has a class of fixed-size stack-buffer over/under-
runs that write a few words into adjacent stack scratch:

  * stan.c  sub_GAME_7F0B1DDC  — `StandTile *tileStack[39]` with a bail check
    of `if (cat >= 41)` that only runs once per outer iteration, so `cat`
    reaches 40 + pointCount and the append writes several slots past the end.
    BUNKER1 (-level_09) prop room-list setup legitimately visits ~40 tiles.
  * bondview2.c  bondviewCalcIntroSwirlCamera  — `f32 pointbuf[10]` indexed
    `&pointbuf[i*3]` for i in [-1..2] then written [0..5], i.e. [-3 .. 11].

On the N64 and on the MinGW/Windows build (neither uses a stack protector)
these are harmless in practice — the Windows build is stable over thousands
of frames. Only Ubuntu's gcc default (`-fstack-protector-strong`) turns them
into a fatal `__stack_chk_fail`, at BUNKER1 stage load and again at the intro
swirl camera (~frame 5).

Two-part fix:
  * `-fno-stack-protector` globally (CMakeLists) — matches the N64 and MinGW
    builds so the same code runs the same way. A hardened Linux build would
    need the whole overrun class swept first.
  * `StandTile *tileStack[64]` under `#ifdef PORT` (stan.c) — the one case
    that is cleanly understood and bounded (`pointCount` is 4-bit, so 40+15
    is the true worst case). Pure local scratch, never escapes, no behaviour
    change. N64 keeps `[39]` under `#else`.
  * stan.c:2225 link-resolve also switched to the D177 PORT_PTRADD macro for
    idiom consistency (the blob maps below 4 GiB so the bare cast is
    currently lossless; #else is byte-identical).

Stacked on the D188 xprintf fix. Verified: Windows -level_09 framediff 3/3
within golden threshold (all changes are #ifdef PORT or no-op on MinGW);
Linux -O2 -level_09 now runs to frame 2100+ with no crash.

* docs: D189 update (stack-protector / latent overrun class) + porting-notes D8

* docs: D190 — Linux -level_45 objective.c load crash (open); D189 sweep 5/6
2026-09-03 20:18:25 -04:00
James Dansereau 7549570bb5 build: keep release optimisation (-O2) instead of forcing -Og
The non-Debug branch unconditionally appended -Og, which (last -O wins)
overrode RelWithDebInfo's -O2 for every TU. -Og is a ~10% frame-time
regression on the software RSP/rasteriser hot path and is not what a
release/playtest build wants. Restrict -Og to explicit Debug builds.

Note: does NOT fix the ~1 fps issue under investigation — the software
renderer is ~950 ms/frame at both -Og and -O2 on this box; this is just
the correct default.
2026-09-03 00:18:45 -04:00
James Dansereau b49907ded9 build: opt-in ccache + BUILD_DIR override
CMakeLists.txt: CMAKE_{C,CXX}_COMPILER_LAUNCHER=ccache when found (no-op
otherwise); game sources are frozen so branch-switch rebuilds get near-free.
build-pc.sh: BUILD_DIR=${BUILD_DIR:-build-pc} so parallel agents/worktrees
can use separate build dirs.
2026-09-02 23:54:36 -04:00
James Dansereau cd6af6c5fd linux-runtime: image base 0x20000000 to clear the N64 address ranges
The no-PIE default base 0x400000 puts .data/.bss around 0x00500000-
0x00600000, colliding with the address space fast3d's seg_addr() decodes.
A game-built display list embeds &symbol (osVirtualToPhysical identity
shim); seg_addr sees w1 < 0x800000, treats it as a K0-physical DRAM
offset and remaps to 0x80xxxxxx (zeroed DRAM mirror) -> 'unknown GBI
opcode 0x00 at 0x80575d40' twice per frame -> DL walk aborts.

Move the image to 0x20000000 -- the gap between segmented addrs
(<0x10000000), the cart map (0x10000000), the D131 module range
([0x40000000,0x70000000)) and DRAM (>=0x70000000). seg_addr passes
0x20xxxxxx through untouched. Still <2GiB so -mcmodel=small relocations
are fine (0x140000000 needed mcmodel=large). ELF analogue of the Windows
0x140000000 base. D146 opcode-0x00 errors: 2/frame -> 0.
2026-09-02 23:16:53 -04:00
James Dansereau fe3d543647 linux-runtime: BUNKER1 boots + renders on Linux
First runtime bring-up on Linux (Ubuntu, gcc-14). -level_09 now renders
past frame 1200 crash-free with a clean shutdown, matching Windows.

- port/src/romdata.c: no-PIE loads the image at 0x400000, so the D34
  'clear low 32 bits of a symbol address' gives 0 -> false 'image base not
  4GiB-aligned' abort. The ANIM_DATA_* lvalues are only ever address-taken
  (never dereferenced), so fall back to a fixed 4GiB-aligned sentinel
  (0x100000000) for g_pc_animdata_base.
- port/src/main.c: the 'image must load at 0x140000000' assertion is
  PE-specific (.refptr base relocations); ELF no-PIE resolves dram_syms.s
  absolute symbols literally regardless of load address. Guard it to
  PLATFORM_WINDOWS.
- port/src/libultra.c: mmap game-thread stacks with MAP_32BIT so they sit
  in the low 2GiB. The decomp truncates stack-buffer pointers to u32 (e.g.
   in texLoad); glibc pthread stacks live above 4GiB and
  those truncations corrupt. One fix covers the whole idiom class. Windows
  pthread stacks are already low.
- src/game/image.c: belt-and-braces #ifdef PORT -- align compbuffer at
  uintptr_t width (identical result on N64).
- CMakeLists.txt: drop the -Ttext-segment experiment (needs -mcmodel=large;
  breaks R_X86_64_32 relocs) -- comment only.

Windows: build 240/241, -level_09 crash-free past frame 1440.
2026-09-02 22:50:56 -04:00
James Dansereau fe7a34c56e linux-build: port/shim/sched.h routes C++ TUs to the host <sched.h>
The decomp's src/sched.h (N64 OSSc* scheduler) is on the include path and
shadows the system header. libstdc++'s <thread>/<mutex> gthr headers in the
fast3d C++ TUs #include <sched.h> and need the real one (cpu_set_t,
sched_yield) -- with -std=gnu++20 libstdc++ defines _GNU_SOURCE so
<pthread.h> also references cpu_set_t. New shim: C++ -> generated
hostsched.h (absolute path, hoststring pattern); C -> #include_next so
src/sched.h still wins as before. Windows build verified green (240/241).
2026-09-02 22:31:33 -04:00
James Dansereau 8eb2efa6ca linux-build: resolve host <stddef.h> via -print-file-name
<stddef.h> is compiler-internal (lib/gcc/<triple>/<ver>/include), not in
bin/../include on a native Linux GCC, so the MinGW-layout guess produced
/usr/include/stddef.h which does not exist -> fatal error in every fast3d
C++ TU. Ask the compiler with -print-file-name=include/stddef.h and fall
back to the old path. MinGW resolves to its own lib/gcc internal copy;
Windows build verified green (240/241).
2026-09-02 22:16:20 -04:00
James Dansereau 7c3bb42687 linux-build: -std=gnu extensions + fix remaining snprintf decls
- CMakeLists: CMAKE_C_EXTENSIONS/CXX_EXTENSIONS ON. Strict -std=c11/c++20
  define __STRICT_ANSI__, which disables glibc _DEFAULT_SOURCE and hides
  clock_gettime(CLOCK_REALTIME) etc. that port/src/libultra.c needs.
  -std=gnu11/gnu++20 is GCC's default and what MinGW already used.
- port/src/{pccg,pcmodels,romdata}.c: local 'extern int snprintf(...,
  unsigned long long maxsize, ...)' decls conflict with glibc's size_t one
  (same bug already fixed in pc_protos.h in M-39); spell it size_t.
- libultra.c: drop the _GNU_SOURCE define (pulls cpu_set_t refs into
  <pthread.h> without <sched.h>); the D60 POSIX stack-bounds diagnostic
  now just prints n/a rather than needing pthread_getattr_np.

Windows: full rebuild 240/241, -level_09 crash-free past frame 1740.
2026-09-02 22:12:58 -04:00
James Dansereau 31a0d6c42d linux-build: -fpermissive for GCC 14 hard-error promotions
GCC 14 turned int<->pointer (known prototype), incompatible-pointer arg
types, and implicit int/function into hard errors. The WIP decomp still hits
these (append_text_picked_up in propobj.c passes AMMOTYPE/u32 to vestigial
u8* params). -fpermissive downgrades them to warnings — the C analogue of
the existing -Wno-error= lines, and does not touch game code. Non-MinGW C only.
2026-09-02 21:43:09 -04:00
James Dansereau 12c3941fd3 linux-build: -fplan9-extensions for the 'inherits' idiom; pin gcc-14
- src/bondtypes.h's `#define inherits struct` makes the base an unnamed
  tagged member and reads its fields directly. mingw32 GCC enables this via
  implicit -fms-extensions; native GCC does not -> "'ObjectRecord' has no
  member named 'type'". Add -fplan9-extensions (C only) for non-MinGW.
- Pin the CI job to ubuntu-24.04 + gcc-14 so the first Linux bring-up chases
  real portability gaps, not old-GCC pedantry (dev toolchain is GCC 16).
2026-09-02 21:36:15 -04:00
James Dansereau 01fd39a089 port: route C TUs to the host <stddef.h>; link no-PIE on Linux
Two GCC-on-Linux build breaks that MinGW masked (surfaced by the new
linux-build CI job):

- port/shim/stddef.h routed C TUs to the empty N64 include/stddef.h stub.
  MinGW's CRT headers leak size_t/ptrdiff_t transitively; glibc/GCC define
  them only in the compiler's own <stddef.h>, so include/PR/ultratypes.h's
  non-N64 path ("typedef ptrdiff_t ssize_t") failed with "unknown type name".
  Route C TUs to hoststddef.h too (C++ already was). Additive — the active
  N64 stub typedefs nothing.

- CMakeLists: dram_syms.s absolute symbols at 0x70000000 need a non-PIE image
  on Linux exactly as they need --disable-dynamicbase on Windows. Add an
  elseif(UNIX AND NOT APPLE) branch: -fno-pie / -no-pie.

Windows unaffected (host <stddef.h> is strictly more correct than the stub;
the MINGW linker branch is unchanged).
2026-09-02 21:19:46 -04:00
James Dansereau 52539d10de Prepare for public release
- README, LICENSE (MIT for the port layer), NOTICE, CONTRIBUTING.
- Restructure docs: internals.md, porting-notes.md, dev-process.md +
  dev/agentic-development.md (the research write-up), dev/ (raw record).
- Replace decomp status-badge CI with a PC-port validate workflow.
- Remove session/agent scaffolding and stale planning docs from the tree.
- Add .gitattributes (normalize line endings to LF); add screenshots.

The pre-2026-08 history is the GoldenEye 007 decompilation by Larry Ficken
(KholdFuzion) and contributors, preserved unchanged. The PC-port work from
2026-08-17 on is mine.
2026-09-01 19:28:59 -04:00
James Dansereau 98412c281c PC port: Phase 1.5 done, first frames render (D38-D42)
- D38: implicit function declarations truncated pointer returns on x86-64;
  added port/include/pc_protos.h prototype shim (398 decls, C-only, anchored
  in port/shim/PR/ucode.h) + pc_netorder.c for ntohl/ntohs; build warning-clean.
- D39: Globalimagetable rebasing - per-symbol offsets + GIMG_OFF() in
  image_bank.c (== N64 (u32)&sym incl. the 0x02000000 base); segment end
  markers extended to the true 0x13F8 size (CSV entry truncated by 0x930).
- D40: g_ModelHitEntries pool sized for the 40-byte PC ModelHitEntry stride
  (was a 12KB .bss overflow clobbering is_ramrom_flag -> demo-replay SIGSEGV).
- D41: release the GL context from the host main thread after videoInit so
  the game scheduler thread can bind it (WGL single-thread currency; this
  SDL2 build's MakeCurrent return value is an inverted ABI artifact).
- D42: rsp.c g_gfxTaskSettingsList XOR toggle -> explicit toggle on x86-64.

Boot now renders real frames ([NOTE] frame N rendered in ... us). Next
blocker D43 (model-file loading ABI mismatch) documented in PCPortResearch.md
section F/H; HANDOFF.md rewritten for it.
2026-08-22 04:07:07 -04:00
James Dansereau dbeee7353f PC port: Phase 1.5 — fix langInit decompression (D31), start D32 ABI layout
- D31 (resolved): replace GE's gzip-1.2.4 inflate with real-zlib
  decompressdata() in port/src/rzdecomp.c; exclude src/game/decompress.c +
  zlib.c from the PC build. Root cause: struct huft is 8B on MIPS but 16B on
  x86-64, overflowing load_resource's fixed 0x2100 buffer and clobbering its
  frame/retaddr on the first file load. mainThread now boots past all language
  files into bossInitMainthreadData().

- D32 (in progress): ROM-serialized structs with 32-bit pointer fields
  misalign when read as 64-bit pointers. Part A done: ModelAnimation's
  bitDescriptors/bitStream are now u32 (N64 layout preserved) with casts at the
  two use sites in model.c. Part B (load-time rebase in expand_ani_table_entries)
  is open — see docs §F/D32 + §H.

- Docs: D31 resolved, D32 recorded (A done / B open), Phase 1.5 added, new §H
  handoff section. AGENTS.md: refined non-negotiable #2 to "game logic is
  unmodified" with a narrow ABI/layout exception; refreshed phase status.
2026-08-21 16:01:49 -04:00
James Dansereau 13ca03a6db PC port: Phase 2 — real threads, dual-mapped DRAM, fast3d integration
- Replace setjmp/longjmp green threads with real pthreads (8MB stacks);
  PortQueue side table (mutex+condvar) for OSMesgQueue; dedicated tick
  thread posts VI retrace + services timers (D24).
- Dual-mapped N64 DRAM: one 8MB store at V1 0x70000000 (s32-safe, all game
  RAM symbols) and V2 0x80000000 (KSEG0 mirror for offset|0x80000000
  rebuilds). Windows: CreateFileMappingW + MapViewOfFileEx; POSIX: memfd
  + MAP_FIXED. dram_syms.s pins cfb_16/_bssSegmentEnd/tlb block (D25).
- Address-width shims: PHYS_TO_K0 identity, OS_K0_TO_PHYSICAL = offset from
  V1 base, OS_PHYSICAL_TO_K0 identity; fast3d seg_addr() remaps small
  offsets into V2 (D26).
- ASLR off (-Wl,--disable-dynamicbase): absolute symbols referenced through
  pointer-typed externs (.refptr + BASE reloc) must not rebase; startup
  sanity check on image base 0x140000000 (D27).
- osPiStartDma posts its completion message on every path (fixes
  romReceiveMesg deadlock) (D29).
- SEH-free crash handler: raw fault log first, validated manual EBP-chain
  walk second (-fno-omit-frame-pointer), TerminateProcess; crashDumpThreads
  via Toolhelp32 for heartbeat hang dumps (D30).
- PD fast3d integrated (port/fast3d/), replaces gfxstub.c; real src/sched.c
  compiled; mainproc runs as a real pthread, host thread pumps SDL events.
- Docs: findings D24-D31 + Phase 2 status (section F/G).

Current state: boots through bossInitMainthreadData to langInit(), then
SIGSEGVs on the first file load — stack frame clobbered by a table of
truncated s32 pointers (D31, open). Next: attribute the write with
'set scheduler-locking on' + hardware watchpoint.
2026-08-21 14:42:19 -04:00
James Dansereau 2aad171344 PC port: Phase 1 boot-to-window (ROM load + map + SDL2 clear loop)
- scripts/gen_romassets.py: emit absolute cart-address asset symbols
  (0x10000000 + rom_offset) from filelist.<r>.csv + the asset .s files,
  in file_resource_table.inc.c order. US region (romassets_u.s).
- romdata.c: load/validate the GE-specific header (magic, name @0x20,
  cart ID @0x3C, country @0x3E) and map the .z64 at cart base 0x10000000
  so every &asset-symbol is a live host pointer.
- libultra.c: osPiStartDma/osPiRawStartDma now service reads as
  bounds-checked memcpys from the mapped ROM (synchronous completion).
- system.c: implement sysResolvePath ($S data dir, $E exe dir).
- video.c: real SDL2 window + GL context + clear/swap frame loop with
  FPS accounting; main.c runs a Phase 1 demo loop (ESC quits) until the
  software RSP/scheduler can drive mainproc().
- CMakeLists: project(... ASM), wire romassets_<region>.s, drop
  assetstubs.c (replaced by generated symbols); n64stubs.c keeps only
  pure-RAM segment markers.
- docs: findings D18-D23 (header offsets, BE music table, .rz tracks,
  scanner-prefixed manifest names, CMake ASM gotcha, absolute-symbol
  approach).

Verified: baserom.u.z64 (12MB) passes validation and maps at 0x10000000;
window opens and renders until killed; asset offsets spot-checked against
ROM bytes (music bank seqCount=63 BE, first track offset 508).
2026-08-20 22:19:45 -04:00
James Dansereau 5ea46fabc1 PC port: full compile + link milestone (ge007.x86_64.exe builds clean)
All ~235 translation units now compile and the target links under MSYS2
GCC 16 + CMake/Ninja (ninja ge007 -k 0: 236/236, zero errors). N64 build
untouched; every src/ change is -DPORT-gated.

Portability fixes (findings D7-D17 in docs/PCPortResearch.md section 11):
- AI X-macro system: SWITCH() call sites activated to the author's
  hand-written byte-identical references under PORT; port/shim/
  bondaicommands.h + bondconstants.h + CPPLib.h re-emit the IDO paste
  tricks (isSubroutine, file-record macros, generated CALL) GCC-clean.
- 64-bit pointer-in-word initializers: monAnim jump targets now go
  through an index table resolved by the interpreter (chrai.h/propobj.c);
  UsetuplenZ.c credits pointer PORT-gated to 0.
- IDO printf engine compiled for PC (xprintf/xlitob/xldtob) with a
  shadow xstdio.h resolving the _Printf prototype clash.
- Host-libc gaps stubbed in port/: bcopy/bzero, __libm_qnan_f,
  tlbmanage*, g_ViXScales/YScales; chrObjRandom PRNG ported from .s.

Asset data strategy (D16): single-file assets compiled for real
(animation tables, fonts, textures, rareware logo); obseg per-file .c
excluded (colliding file-local globals the N64 build isolates via
per-ELF linking); 758 ROM-derived table symbols + ramrom_* + segment
markers stubbed in port/src/assetstubs.c / n64stubs.c.

fast3d is a no-op placeholder (port/src/gfxstub.c) until Phase 2 brings
in the real software RSP. Binary not yet runnable end-to-end: ROM
loading in romdata.c is still a TODO.
2026-08-20 20:27:06 -04:00
James Dansereau cd6cc158b6 Add PC port scaffolding (CMake build + port layer)
Initial scaffolding for a PC port of GoldenEye 007, modeled on the
Perfect Dark PC port (same Rare "Indy" engine). The N64 build
(Makefile + IDA) is untouched; the PC build is a parallel CMake build.

Build:
- CMakeLists.txt: SDL2 + zlib + OpenGL. Compiles src/game/*.c + curated
  engine files + libultra audio/gu + port/. Per-region macro sets
  (ROMID US/EU/JP) mirror the N64 Makefile's LCDEFS exactly.
- build-pc.sh, cmake/: configure+build helper and target/SDL2 modules.

Port layer (port/):
- video/audio/input/fs/romdata/config/system/mixer: stubs with
  TODO(Phase N) markers and correct interfaces.
- libultra.c: libultra OS shims (message queues, VI, PI, AI, EEPROM,
  controller, PFS/motor no-ops, region-gated osTvType).
- gesched.c: software-RSP scheduler replacing src/sched.c task submission.
- n64stubs.c: inert stubs for N64-boot + rmon symbols.
- random.c: PRNG ported verbatim from src/random.s (bit-exact).
- ucode.c: RSP/ASP/GSP microcode segment markers (dummies; Phase 3 TODO).

Docs:
- docs/PCPortResearch.md: architecture, GE-specific deltas, phased plan,
  and the scaffolding review (findings A1-A4/B1-B4/C1-C2 + resolutions).
- PORTING.md: top-level entry point.

See docs/PCPortResearch.md for the full plan.
2026-08-17 19:17:06 -04:00