From b01c7fa89c77504cb7e05f98e142fd6a599b3418 Mon Sep 17 00:00:00 2001 From: James Dansereau <106631954+jkdansereau@users.noreply.github.com> Date: Wed, 2 Sep 2026 20:54:20 -0400 Subject: [PATCH] port(romdata): POSIX fixed-address ROM map (mmap) for the Linux build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cart-base map at 0x10000000 (so absolute ROM-asset symbols are live host pointers) was Windows-only; the #else branch was a stub that logged "not implemented" and fell to the degraded heap-copy path. - Factor the post-map work (memcpy + pcmodels/pccg sidecar load + D55 RLE header fixup) into romdataFinishCartMap(), shared by both paths. - POSIX #else: anonymous mmap at CART_BASE with MAP_FIXED_NOREPLACE where available; on any failure munmap + fall through to the identical heap copy. - romdataDestroy() munmaps on POSIX (length tracked in mappedLen). Verified on Windows: builds 242/242, -level_09 boots crash-free (identical behaviour with/without the refactor). Linux compile + runtime verification owed — see docs/dev/LINUX-PORT.md. No behaviour change on Windows. --- docs/dev/LINUX-PORT.md | 91 +++++++++++++++++++++++++++++++ port/src/romdata.c | 121 ++++++++++++++++++++++++++--------------- 2 files changed, 168 insertions(+), 44 deletions(-) create mode 100644 docs/dev/LINUX-PORT.md diff --git a/docs/dev/LINUX-PORT.md b/docs/dev/LINUX-PORT.md new file mode 100644 index 00000000..9586411f --- /dev/null +++ b/docs/dev/LINUX-PORT.md @@ -0,0 +1,91 @@ +# Linux port — status & gap analysis + +Session M-38 (2026-09-02). Goal-2 on the roadmap (after the Windows alpha / +release artifact). This is a scoping + first-patch pass; **nothing here has run +on a Linux machine yet.** + +## Method + +Read `build-pc.sh`, `CMakeLists.txt`, `docs/building.md`, and grepped +`port/src`, `port/fast3d`, `port/include` for Windows-only headers / Win32 APIs +/ unguarded `_WIN32` branches. A local Qwen `triage` pass seeded the hunt; every +claim below was re-checked against the real files (the 27B was wrong on two of +three headline items — see "Ruled out"). + +## What is already Linux-ready (verified by reading) + +- **`port/include/platform.h`** — sets `PLATFORM_LINUX` / `PLATFORM_MACOS` / + `PLATFORM_WINDOWS` cleanly. +- **`CMakeLists.txt`** — `find_package(SDL2 REQUIRED)`, `find_package(ZLIB + REQUIRED)`, and `else()` branches already present for the non-Windows case: + `GL_LIBRARY = GL` (l.291), `EXTRA_LIBRARIES = stdc++ m dl pthread` (l.304), + `elseif(UNIX)` platform block (l.86). `WIN32_EXECUTABLE` and the MinGW zlib + fallback are correctly `if(WIN32)` / `if(MINGW)` guarded. +- **`build-pc.sh`** — already documents the Linux apt deps + (`cmake libsdl2-dev zlib1g-dev libgl1-mesa-dev`) and is a plain bash script, + no MSYS-isms in the executable body. +- **`port/src/system.c`** — `sysSleep()` has a `nanosleep()` `#else` branch + (l.76-82); `` is `PLATFORM_WINDOWS`-guarded. +- **`port/src/libultra.c`** — the OS-thread kernel has a real pthreads branch + next to the Win32 `_beginthreadex` one. +- **`port/src/crash.c`** — `MessageBoxW` / `SetThreadPriority` / `__debugbreak` + all `_WIN32`-guarded, with `PLATFORM_LINUX` counterparts. +- **`port/src/video.c`** — `GE_MKDIR` has both `_mkdir` and `mkdir(p,0777)`. +- No `__declspec`, `#pragma comment`, or hardcoded `\`-path / drive-letter + usage anywhere in `port/`. + +## Ruled out (Qwen triage false positives) + +- ~~`system.c:75` `Sleep()` has no POSIX branch~~ — it does (`nanosleep`). +- ~~CMake needs `else()` branches for zlib/pthread on Linux~~ — already there. + +## The one real code gap — FIXED this session (verification owed) + +**`port/src/romdata.c`** — the fixed-address cart map (`0x10000000`, so absolute +ROM-asset symbols are live host pointers) had a **Windows-only** implementation +(`VirtualAlloc`); the `#else` branch was a stub that logged "not implemented on +this platform" and fell straight to the heap-copy fallback. In heap-copy mode +anything that dereferences a cart address *directly* (not via +`romdataGetRom()` / the PI shims) reads wrong memory — so Linux would run, if at +all, in the same degraded mode Windows only hits when `VirtualAlloc` loses the +address race. + +**Change (M-38):** +- Factored the post-map work (memcpy + `pcmodelsLoadSidecars` + + `pccgLoadSidecars` + D55 RLE-header fixup) into `romdataFinishCartMap()`, + shared by both paths so they cannot drift. +- POSIX `#else` now does an anonymous `mmap((void*)CART_BASE, len, + PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED_NOREPLACE, -1, 0)`. + `MAP_FIXED_NOREPLACE` (Linux 4.17+) fails rather than clobbering an existing + mapping; where the macro is absent the plain hint is advisory and the + `at == CART_BASE` check catches a relocated result. On any failure it + `munmap`s and falls through to the identical heap-copy path as before. +- `romdataDestroy()` `munmap`s on POSIX (tracked length in `mappedLen`). + +**Owed verification (needs a Linux box):** +1. Compiles under GCC on Linux (`` resolves; no decomp-include-path + shadowing surprise). +2. `mmap` actually lands at `0x10000000` on a stock ASLR Linux — `0x10000000` + (256 MB) is low but usually free; if the kernel routinely refuses it, + consider linking the binary no-PIE or reserving the range via a linker + script so the loader keeps it clear. +3. `-level_09` boots crash-free with the mapped path taken (check the log says + "mapped at 0x10000000 (cart base)", not "using heap copy"). +4. Whether the heap-copy fallback is actually survivable on Linux if step 2 + fails — audit for direct `0x1xxxxxxx` derefs outside the PI shims. + +## Remaining Linux work after the romdata patch lands + +| # | Item | Size | Note | +|---|------|------|------| +| 1 | Run the Ubuntu **`validate`** CI job → upgrade it to a real **compile** job | S | `.github/workflows/ci.yml` already configures on Ubuntu; flipping it to `cmake --build` surfaces the real GCC-strictness gaps cheaply | +| 2 | First actual Linux runtime bring-up (`-level_09`) | M | expect a short tail of unguarded calls / struct-layout asserts GCC-on-Linux vs MinGW | +| 3 | `docs/building.md` — promote Linux from "untested" to "supported" once 1+2 pass | S | | +| 4 | SDL2 / GL context creation on X11 + Wayland | M | fast3d is GL 3.3 core; PD port runs on Linux so the path is known-good | +| 5 | Package: an AppImage or tarball equivalent of `bundle-win.sh` | M | reuse `prepare-assets.py` verbatim — it is already OS-agnostic | + +## Recommended next step + +Land the romdata patch behind CI (item 1) — turn the Ubuntu `validate` job into +a compile job in the same PR so the "does it build on Linux" question is +answered by the bots, then do the runtime bring-up on a real machine. diff --git a/port/src/romdata.c b/port/src/romdata.c index dac3561a..fa1596aa 100644 --- a/port/src/romdata.c +++ b/port/src/romdata.c @@ -32,6 +32,11 @@ extern int snprintf(char *str, unsigned long long maxsize, const char *format, . #define WIN32_LEAN_AND_MEAN #endif #include +#else + /* POSIX: mmap the ROM at the fixed cart address (mirrors the VirtualAlloc + * path). is a host header the decomp include path does not + * shadow. */ + #include #endif #include "system.h" @@ -50,6 +55,9 @@ extern int snprintf(char *str, unsigned long long maxsize, const char *format, . static u8 *rom = NULL; /* heap copy (fallback path) */ static u32 romSize = 0; static int mappedAtCartBase = 0; +#if !defined(PLATFORM_WINDOWS) +static unsigned long long mappedLen = 0; /* munmap() needs the length */ +#endif /* D34 (docs/internals.md): PE image base, low 32 bits zero. On N64 the * animation_data segment links at VMA 0, so game code computes record @@ -105,6 +113,35 @@ static int romHeaderValid(const u8 *h, char *err, size_t errsz) static u32 romdataBswap32(u32 v); /* defined below; used by the D55 fixup */ +/* The caller has placed a writable region of (romSize + sidecars) bytes at + * CART_BASE. Populate it, load the sidecars, apply the D55 RLE-header fixup, + * and mark the ROM live. Shared by the Windows (VirtualAlloc) and POSIX (mmap) + * fixed-address paths so the post-map logic can never drift between them. */ +static int romdataFinishCartMap(const char *tok, u8 *img, + u32 sideTotal, u32 cgTotal) +{ + memcpy((void *)(uintptr_t)CART_BASE, img, romSize); + free(img); + mappedAtCartBase = 1; + sysLogPrintf(LOG_INFO, "romdataInit: %s (%u bytes) mapped at 0x%08X " + "(cart base)%s%s", tok, romSize, CART_BASE, + sideTotal ? ", + model sidecars" : "", + cgTotal ? ", + bg/stan sidecars" : ""); + pcmodelsLoadSidecars(CART_BASE, romSize); + pccgLoadSidecars(CART_BASE + romSize + pcmodelsTotalSize()); + + /* D55: the RLE folder-menu background at `unknown2` has a big-endian w/h + * header that rle_expand_8bit() reads little-endian; match the N64 .data + * copy by swapping it in place (no-op if it already reads as LE). */ + { + extern void *unknown2; /* absolute cart address */ + u32 *hdr = (u32 *)&unknown2; + if ((u16)*hdr > 512 || (u16)(*hdr >> 16) > 512) + *hdr = romdataBswap32(*hdr); + } + return 0; +} + int romdataInit(void) { char err[128] = ""; @@ -168,51 +205,44 @@ int romdataInit(void) * sidecar extension. */ u32 cgTotal = pccgReserveSize(img); - /* Map at the cart address so absolute asset symbols are live. */ + /* Map at the cart address so absolute asset symbols are live. On + * failure both paths fall through to the heap copy below (degraded: + * anything that dereferences a cart address directly, rather than via + * romdataGetRom()/the PI shims, will read wrong memory). */ + { + u32 maplen = romSize + sideTotal + cgTotal; #if defined(PLATFORM_WINDOWS) - void *at = VirtualAlloc((LPVOID)CART_BASE, romSize + sideTotal + cgTotal, - MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); - if (at == (void *)(uintptr_t)CART_BASE) { - memcpy(at, img, romSize); - free(img); - mappedAtCartBase = 1; - sysLogPrintf(LOG_INFO, "romdataInit: %s (%u bytes) mapped at " - "0x%08X (cart base)%s%s", - tok, romSize, CART_BASE, - sideTotal ? ", + model sidecars" : "", - cgTotal ? ", + bg/stan sidecars" : ""); - pcmodelsLoadSidecars(CART_BASE, romSize); - pccgLoadSidecars(CART_BASE + romSize + pcmodelsTotalSize()); - - /* D55 (docs/internals.md): the RLE folder-menu background at - * `unknown2` (romassets_.s) is stored in the ROM with a - * big-endian w/h header (01 B8 01 2B = 440x299), but - * rle_expand_8bit() reads it little-endian. The N64 build embeds - * this asset into .data via assets/romfiles2.s from an extracted - * .bin whose header is byte-swapped (title2.c hardcodes 440x299 - * row rendering, and this is the only decodable 440x299 RLE - * stream in the ROM), so make the cart copy match it: swap the - * w/h word in place. The guard makes the fixup a no-op if a - * region's copy already reads as a plausible LE size. */ - { - extern void *unknown2; /* absolute cart address */ - u32 *hdr = (u32 *)&unknown2; - if ((u16)*hdr > 512 || (u16)(*hdr >> 16) > 512) - *hdr = romdataBswap32(*hdr); - } - return 0; - } - if (at) - VirtualFree(at, 0, MEM_RELEASE); - sysLogPrintf(LOG_WARNING, "romdataInit: could not reserve 0x%08X; " - "using heap copy — direct ROM reads will fail", - CART_BASE); + void *at = VirtualAlloc((LPVOID)CART_BASE, maplen, + MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); + if (at == (void *)(uintptr_t)CART_BASE) + return romdataFinishCartMap(tok, img, sideTotal, cgTotal); + if (at) + VirtualFree(at, 0, MEM_RELEASE); + sysLogPrintf(LOG_WARNING, "romdataInit: could not reserve 0x%08X; " + "using heap copy — direct ROM reads will fail", + CART_BASE); #else - /* Best effort: fixed-hint mmap. If the hint is refused we fall back - * to a heap copy (same limitation as above). */ - sysLogPrintf(LOG_WARNING, "romdataInit: fixed-address mapping not " - "implemented on this platform; using heap copy"); + /* POSIX: anonymous fixed-address mmap. MAP_FIXED_NOREPLACE (Linux + * 4.17+) fails instead of clobbering an existing mapping; where it + * is unavailable the plain hint is advisory and the == check below + * catches a relocated result. */ + int flags = MAP_PRIVATE | MAP_ANONYMOUS; +#ifdef MAP_FIXED_NOREPLACE + flags |= MAP_FIXED_NOREPLACE; #endif + void *at = mmap((void *)(uintptr_t)CART_BASE, maplen, + PROT_READ | PROT_WRITE, flags, -1, 0); + if (at != MAP_FAILED && at == (void *)(uintptr_t)CART_BASE) { + mappedLen = maplen; + return romdataFinishCartMap(tok, img, sideTotal, cgTotal); + } + if (at != MAP_FAILED) + munmap(at, maplen); + sysLogPrintf(LOG_WARNING, "romdataInit: could not map 0x%08X " + "(ASLR/kernel refused the fixed address); using heap " + "copy — direct ROM reads will fail", CART_BASE); +#endif + } rom = img; return 0; @@ -225,12 +255,15 @@ int romdataInit(void) void romdataDestroy(void) { -#if defined(PLATFORM_WINDOWS) if (mappedAtCartBase) { +#if defined(PLATFORM_WINDOWS) VirtualFree((LPVOID)CART_BASE, 0, MEM_RELEASE); +#else + munmap((void *)(uintptr_t)CART_BASE, (size_t)mappedLen); + mappedLen = 0; +#endif mappedAtCartBase = 0; } -#endif free(rom); rom = NULL; romSize = 0;