From 5071185f8c35b8f55750b53ec084d62b933b6838 Mon Sep 17 00:00:00 2001 From: James Dansereau <106631954+jkdansereau@users.noreply.github.com> Date: Wed, 16 Sep 2026 15:28:44 -0400 Subject: [PATCH] findings: D295 M-148/M-149 + D297 filed/closed (M-150); porting-notes D8/D9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - D295: full M-148 root-cause/repro record (difficultytext overflow -> high-bit byte -> langGetJpnCharPixels NULL deref; OOB theory retracted; pre-D284 save fingerprint identified), verdict FIXED (M-149). - D297: new entry — v0.3.0 wipes all pre-fix saves (D284 side effect); scope verified (5 slots only, block-0 is a factory seal), FIXED (M-150). - D284: correct the false 'savegames are unaffected' claim. - porting-notes: D8 instance (difficultytext[4]); new D9 — changing a PRNG/CRC implementation invalidates persisted derived state; sweep rewrites data/ge007.eep gotcha in E. --- docs/dev/findings-index.csv | 4 ++- docs/dev/findings.md | 68 +++++++++++++++++++++++++++++++++++-- docs/porting-notes.md | 33 ++++++++++++++++++ 3 files changed, 102 insertions(+), 3 deletions(-) diff --git a/docs/dev/findings-index.csv b/docs/dev/findings-index.csv index 77c554d3..16ab3e35 100644 --- a/docs/dev/findings-index.csv +++ b/docs/dev/findings-index.csv @@ -172,4 +172,6 @@ D291,OPEN,"Dam: texture glitches when the truck drives past (M-141, Steam Deck u D292,OPEN,Death animations are glitched —,"OPEN — same status as D173 (observed, not root-caused)." D293,FIXED,"QoL ask: no discoverable way to quit the game from the UI (M-141, Steam Deck user playtest —","FIXED (M-141) — ""Quit to desktop"" row added to the F10 overlay; build-verified Win+Linux, not yet live-confirmed.…" D294,OPEN,"Statue: ground geometry near the large CCCP/hammer-and-sickle monument briefly renders solid black over a wide area, then self-recovers 10-30 seconds later… —",OPEN — not investigated.… -D295,OPEN,"Campaign-halting crash at the end of Control Center on 00 Agent difficulty, then a crash on every subsequent relaunch/resume (GitHub #87, user campaign playtes…",OPEN — not investigated. +D295,FIXED,"Campaign-halting crash at the end of Control Center on 00 Agent difficulty, then a crash on every subsequent relaunch/resume (GitHub #87, user campaign playtes…",FIXED (M-149) — M-148 root-caused + reproduced on HEAD; fix landed: +D296,OPEN,"Control Center, 00 Agent:","OPEN — single live report, un-triaged; observed only under a gdb-attached debug-harness launch (tools_pc/debug.ps1), not the packaged release artifact —" +D297,FIXED,v0.3.0 silently wipes ALL pre-fix player saves on first launch (D284 side effect; found during D295/M-148).,FIXED (M-150) — migration shim landed: diff --git a/docs/dev/findings.md b/docs/dev/findings.md index 2691a685..5a98ec4f 100644 --- a/docs/dev/findings.md +++ b/docs/dev/findings.md @@ -569,7 +569,7 @@ covers D24–D69; the log continues in §H (D32 procedure, D70–D121). | D263 | **v0.2.1 (user report): Facility — "Ourumov not shooting Trevelyan"; the scripted opening beat never triggers. RESOLVED: not a port bug — the beat is…** — full `## D263` entry at file tail | RESOLVED (not a bug), but refined (v0.2.1 further user testing, 2026-09-15) —… | | D265 | **D236 investigation (port-layer, `port/fast3d/gfx_pc.cpp`): the tree cards are IA8/IA4, not IA16 — and `import_texture_ia4` decoded the RDP's I2:A2 texel as…** — full `## D265` entry at file tail | PARKED — reverted to baseline; evidence + suspect list above. | | D258 | **PAL/JP asset conversion is broken at the source-data level — `scripts/filelist.e.csv` and `filelist.j.csv` carry naming defects, so `gen_romassets.py e/j` (and…** — full `## D258` entry at file tail | OPEN — repair path verified, not applied (out of v0.2.0 scope). | -| D284 | **`port/src/random.c`'s MIPS64 `dsll32`/`dsrl32` shift helpers were backwards — PC PRNG desynced from the N64's from draw 1, in every release to date (M-138 review; fix M-139b).** — full `## D284` entry at file tail | FIXED, 21-level sweep clean, no crashes — RNG stream changes from every prior build; existing replays will diverge, savegames unaffected | +| D284 | **`port/src/random.c`'s MIPS64 `dsll32`/`dsrl32` shift helpers were backwards — PC PRNG desynced from the N64's from draw 1, in every release to date (M-138 review; fix M-139b).** — full `## D284` entry at file tail | FIXED, 21-level sweep clean, no crashes — RNG stream changes from every prior build; existing replays will diverge. **"Savegames unaffected" was FALSE (M-148): `fileGenerateCRC` runs off a local constant seed via `randomGetNextFrom`, so the fix invalidates every stored slot CRC — v0.3.0 silently wipes all pre-fix saves on first launch (release-blocker candidate, see D295 M-148)** | | D285 | **Steam Deck SIGSEGV during D255 live-testing = SEPARATE bug: the `sndHandleEvent` preempt-scan walks the active-sound list unlocked (M-140; fixed M-141).** — full `## D285` entry at file tail | **FIXED (M-141, `src/snd.c`)** — `osSetIntMask` lock added around the preempt-scan, matching `sndSetupSound`'s existing unconditional (no `#ifdef PORT`) pattern in the same file; build green, `-level_09`/`-level_20` unregressed (1400+ frames, no crash, no D152 lock-stuck warnings). Not yet gdb/live-confirmed against a real reproduced crash — see M-141 body. Do not conflate with D255 — different thread, different subsystem. | | D286 | **User-confirmed real ~8s frame freeze on Caverns, Steam Deck — self-recovered, not a crash (M-140).** — full `## D286` entry at file tail | OPEN — one confirmed occurrence, no repro trigger identified, not root-caused; distinct from D255/D285. Do not chase a repro without a better lead. | | D287 | **User-confirmed real, non-self-recovering freeze on Windows (M-141, 2026-09-15/16) after ~15-20 minutes with no input, triggered by clicking back into the…** — full `## D287` entry at file tail | OPEN — one confirmed occurrence, real (not self-recovering), not root-caused; `gfxFrameMsgQ` full is a lead, not a confirmed cause. | @@ -580,7 +580,9 @@ covers D24–D69; the log continues in §H (D32 procedure, D70–D121). | D292 | **Death animations are glitched — Bond is often sunk halfway into the floor (M-141, Steam Deck user playtest).** — full `## D292` entry at file tail | OPEN — same status as D173 (observed, not root-caused). Cross-ref D173/D148 as one investigation thread, not three separate bugs, next time someone has a live session to chase it. | | D293 | **QoL ask: no discoverable way to quit the game from the UI (M-141, Steam Deck user playtest — "couldn't see a way to exit the game. Maybe build this into the…** — full `## D293` entry at file tail | FIXED (M-141) — "Quit to desktop" row added to the F10 overlay; build-verified Win+Linux, not yet live-confirmed.… | | D294 | **Statue: ground geometry near the large CCCP/hammer-and-sickle monument briefly renders solid black over a wide area, then self-recovers 10-30 seconds later…** — full `## D294` entry at file tail | OPEN — not investigated.… | -| D295 | **Campaign-halting crash at the end of Control Center on 00 Agent difficulty, then a crash on every subsequent relaunch/resume (GitHub #87, user campaign playtest).** — full `## D295` entry at file tail | OPEN — not investigated. No crash log/repro steps captured yet; user also reports escalating audio corruption through Streets leading up to it — possibly related, not confirmed. | +| D295 | **Campaign-halting crash at the end of Control Center on 00 Agent difficulty, then a crash on every subsequent relaunch/resume (GitHub #87, user campaign playtest).** — full `## D295` entry at file tail | FIXED (M-149) — M-148 root-caused + reproduced on HEAD; fix landed: port-side JPN-cache seed (`port/src/main.c`) + `#ifdef PORT` buffer widen in `front.c` (D8 class). Verified: previously-crashing input now clean, 21/21 sweep PASS. Companion save-wipe regression → D297. History — M-148 fully root-caused + **reproduced on current HEAD**: `difficultytext[4]` (`front.c:2616`) overflow — `strcpy("00 Agent")`/`strcat("\n")` spills onto `textpos.p[0]` (y, ≥0x80 in normal geometries) → high-bit byte → `langGetJpnCharPixels` NULL deref (`language.c:320`, `g_JpnCacheCacheItems` NULL outside JPN mode — likewise unallocated on N64 non-JP builds, crash potential is decomp-native). The user's save is a *valid pre-D284* save (all 5 slot CRCs bit-exact under the old PRNG); post-fix builds wipe it at validation, which is why the unmodified artifact no longer triggers it — NOT a fix. No OOB exists (M-147 guess retracted: 007 difficulty has no stored times). Fix options flagged for sign-off (port-side JPN-cache init preferred; `difficultytext` enlarge = D8 class). Side finding: v0.3.0 silently wipes all pre-fix saves on first launch (D284 side effect — own D entry owed). Separate still-open issue found: unconditional `[EVTQ-DROP]` audio-pool-exhaustion logging, `src/libultra/audio/event.c:92`.… | +| D296 | **Control Center, 00 Agent: noticeable stutter, user's own read is "maybe from the drone guns" (M-146, same D295 investigation session).** — full `## D296` entry at file tail | OPEN — single live report, un-triaged; observed only under a gdb-attached debug-harness launch (`tools_pc/debug.ps1`), not the packaged release artifact — user flagged that harness overhead has caused false stutter before. Needs a plain-exe / real-release repro check before any perf tracing.… | +| D297 | **v0.3.0 silently wipes ALL pre-fix player saves on first launch (D284 side effect; found during D295/M-148).** — full `## D297` entry at file tail | FIXED (M-150) — migration shim landed: `port/src/legacycrc.c` (isolated, deliberately-wrong pre-D284 PRNG, reviewer-verified bit-exact against `0de99b4d^`) + eeprom-shim read-path hook in `libultra.c` (block 4 only; persists pristine+migrated bytes before the read-time-only AllUnlocked patch). Verified: user's #87 artifact → `D297-migrated 5`, all slots OK, idempotent on relaunch; native post-fix save untouched; corrupt slot left to the game's own wipe path; fresh eep untouched; 21/21 sweep PASS. Block-0 header needed no migration (factory seal, verified). | Phase 2 replaced the Phase-1 demo loop with the real `mainproc()` on real OS threads, compiled GE's real `src/sched.c`, and brought in PD's fast3d software @@ -10166,6 +10168,8 @@ static inline u64 dsrl32(u64 x, int n) { return x >> (n + 32); } **User-facing note:** this silently changes every RNG-derived sequence (loot placement/rolls, AI behavioural variance, anything seeded from `randomGetNext`/`randomGetNextFrom`) from the first draw, in every build going forward — savegames remain compatible (the seed is not part of save state), but **existing replays recorded against the old stream will diverge** (replay state persists `g_randomSeed` itself, so a replay recorded pre-fix and played back post-fix, or vice versa, will not reproduce the same run). Called out in the v0.3.0 release notes per M-139c. +**CORRECTION (M-148, 2026-09-16, via D295 re-verification): "savegames remain compatible" is FALSE.** `fileGenerateCRC` (`src/game/crc.c`) — the save-slot validation checksum in `fileValidateSaves` (`file2.c:544`) — runs `randomGetNextFrom` off its own **local constant seed `0x8F809F473108B3C1`**, not `g_randomSeed`. The stored slot CRCs are therefore a fingerprint of the PRNG implementation that wrote them. Verified with compiled C from both sides of this fix against a real v0.2.x save (GitHub #87's `ge007.eep`): pre-fix PRNG reproduces all five stored slot CRCs bit-exact; post-fix PRNG matches zero → `fileValidateSaves` resets every slot. **Net effect: the first launch of any post-fix build (i.e. v0.3.0) silently erases every existing player's entire campaign progress.** This is a release-blocker-class regression distinct from the replay divergence above — decision owed (dual-CRC acceptance / one-time migration / loud release note), own D entry per D295 M-148. + ## D285 — M-140: a real Steam Deck SIGSEGV during D255 live-testing turned out to be a SEPARATE, previously-undocumented bug — an unprotected shared-list read in `sndHandleEvent`'s preempt-scan (`src/snd.c:366`), classic D4-class main-thread-vs-audio-thread race, case identified but NOT fixed **Context.** While setting up a live Deck test session for D255 (SSH access, `GE_D255=1`), found a fresh `ge007.crash.log` on the user's Deck (mtime inside the user's active testing window, confirmed by the user as "any crashes from the last hour or two on that build is my testing") from the **published v0.2.1 binary**, which still had its own debug symbols (`with debug_info, not stripped`). Symbolized directly on-device with `addr2line` against the exact crashing binary — not a rebuild, not a guess: @@ -10981,3 +10985,63 @@ crash class C2 (Facility `-level_34`, Runway `-level_35`): `import_texture_i8` A **Also reported, same playtest, possibly related:** progressively worsening audio corruption starting around Streets and continuing through to the Control Center crash — user's own framing is "Idk if that contributed to it somehow." Streets → Depot → Control Center is roughly the mission order in this stretch; if audio state is shared/leaked across level transitions (cf. D204 tempo drift, D202 stuck door loop, D285 audio-thread SIGSEGV — all real, distinct bugs already found in the audio-thread class this session/prior sessions), a slow audio-side corruption building up over several levels and finally taking down Control Center is plausible, but **nothing here confirms causation** — could equally be an unrelated coincidence of timing. **Status (from §F table):** OPEN — not investigated. No crash log, `bt full`, or repro steps captured yet. **Next steps, in order:** (1) get the user's `ge007.crash.log` (or a `gdb bt full` under `debug.ps1`) from the actual crash, plus their `data/ge007.eep` if they're willing to share it — a corrupted-save reproduction would be the fastest path to a fix and would explain the "crashes on every relaunch" persistence; (2) if the save file also crashes cleanly on a fresh debug build, that confirms save-state corruption over a fresh code bug; (3) chase the audio-degradation report only if it turns out to correlate with the crash (e.g. an audio-thread crash address) — don't spend time on it as a separate lead until then. + +**M-146 (2026-09-16, same day): artifacts obtained + platform + version pinpointed; likely already fixed by v0.2.2, unconfirmed.** User supplied `ge007.crash.log`, `audiotrace.log`, and `data/ge007.eep` (via the GitHub issue + a G-Drive link for the save). + +- **Platform confirmed Windows, from the crash log format itself, not inference.** `port/src/crash.c` emits totally different output per platform (`#if PLATFORM_WINDOWS` vs `#elif PLATFORM_LINUX`). The user's log opens `EXCEPTION: 0xc0000005` (an NT `STATUS_ACCESS_VIOLATION` code) and includes `MxCsr=... x87CW=... x87SW=...` (the x86 `CONTEXT` FPU state) — both Windows-only; the Linux path prints `SIGNAL: 11` and never emits FPU control-word fields. `FAULT ADDR: 0000000000000000` — a genuine null-pointer dereference, PC `0x1400797c1` (module base `0x140000000`, RVA `0x797c1`, unsymbolicated — no matching build/PDB available to resolve it further). +- **Version pinpointed by ancestry, not the crash log (no version string is captured in `ge007.crash.log` today — worth adding one).** Issue #87 was filed 2026-09-16 01:13 UTC; `v0.2.2` was tagged 2026-09-16 13:13 UTC, ~12h *later* — v0.2.2 did not exist yet, so the user was on `v0.2.1` (the latest public release at report time). `git merge-base --is-ancestor` confirms both `1ba83ba4` (D255 fix) and `1970ab37` (D285 fix) are **absent from `v0.2.1`, present in `v0.2.2`**. +- **D255 is the leading candidate**, on shape + level content: D255 was a 32-bit-pointer-truncation bug (`Model*` read through a mistyped 4-byte `chrflags` alias) that on PC either dereferences garbage or, when the truncated low bits land on zero, crashes exactly like a real NULL — matching this log's `FAULT ADDR: 0`. Control Center's own briefing (`assets/obseg/brief/UbriefcontrolZ.c`) lists **"Destroy armored mainframes"** as an objective — an explosion-triggered object-destruction event, the same `vtxstore_fix_refs` code path D255 patched. Not proven (no symbols to confirm the crash PC actually lands in `vtxstore_fix_refs`), but a strong circumstantial fit. +- **D285 fits the reported audio-corruption symptom** (an unlocked shared-list read racing a locked writer in `sndHandleEvent`) but less specifically fits the null-fault shape. +- **New, separate, still-OPEN issue found while reading `audiotrace.log`:** it is nothing but a wall of `[EVTQ-DROP]` lines (1089 total) — `src/libultra/audio/event.c:92`'s D202-era diagnostic, which logs unconditionally (no env gate) whenever the audio event free-list (`evtq->freeList`) is exhausted. This fires continuously in the user's log, i.e. **real evidence the event pool was genuinely, sustainedly exhausted during their session** — distinct from D285 (a race, not a leak) and not fixed by either v0.2.2 patch. The probe itself is also a minor release-hygiene bug (D202 cleanup was supposed to remove it — see the D202 M-2x closeout note — and never did; it's still live on `release/v0.3.0`/HEAD today, silently writing an unbounded log file in shipped builds). +- **Repro attempt on current `release/v0.3.0` HEAD (already carries the v0.2.2 fixes) did NOT reproduce the crash**, but this is a partial result, not a clean disproof: + - Front-end/mission-select/file-select screens with the user's real `ge007.eep` loaded (`GE_STARTMENU=5/7`, `tools_pc/debug.ps1`): no crash, save data reads fine. + - **User live-played Control Center start-to-finish at the correct difficulty** (`-level_23 -hard2` — `src/boss.c:396-401`'s native N64 debug switch for `DIFFICULTY_00`, not a bare `-level_23` which silently defaults to Agent and would NOT have been a faithful repro) with the real save loaded, under `tools_pc/debug.ps1` (gdb-attached). Completed the level (with cheats enabled partway through) — **no crash**, consistent with "v0.2.2 already fixed it" but not a controlled A/B, since the branch tested already has both fixes applied. User separately reported drone-gun stutter during this playthrough — logged as **D296**, with the debug-harness caveat noted there. + - **Next, concrete: user is pulling the actual `v0.2.1` release artifact and replaying the same save file on it.** If it crashes there and not on current HEAD, that's the controlled confirmation this finding needs to close as FIXED-by-v0.2.2; if it does NOT crash on `v0.2.1` either, D255/D285 are not the cause and this stays open for a fresh investigation (get a build with matching symbols/PDB from whoever can reproduce it, to actually resolve RVA `0x797c1`). + +**M-147 (same day, 2026-09-16): RETRACTING the D255/D285 hypothesis above — real root cause found via symbols, and it's a different bug family entirely.** + +- User pulled the actual `v0.2.1` release artifact (`goldeneye-pc-port-0.2.1-win64`, which ships full DWARF debug info) and reproduced with the real save: **crashes deterministically right after the intro, same `EXCEPTION: 0xc0000005` / `FAULT ADDR: 0000000000000000` / same crash PC `0x1400797c1` as the original report** (confirmed by direct comparison — this is the exact same crash, not a new one). +- `addr2line -e ge007.x86_64.exe -f -C -p 0x1400797c1` resolves cleanly: **`langGetJpnCharPixels` at `src/game/language.c:320`.** A real, symbol-confirmed NULL dereference, not a guess. +- **Mechanism:** `g_JpnCacheCacheItems` (`language.c:29`) is a global pointer, allocated only inside `langInit()` when `j_text_trigger` is true (`language.c:247-249`, gated on Japanese-language mode). `langGetJpnCharPixels()` (`language.c:303`) has **no NULL check** before indexing it at line 320. It is called unconditionally — **not gated by `j_text_trigger` or the actual selected language** — from `textRenderGlyph`'s string-draw loop (`src/game/textrelated.c:475-499`) for *any* character byte with the high bit set (`*text >= 0x80`). So any rendered string containing a stray high-bit byte null-derefs immediately, on any region build, regardless of language setting. This is the same general shape as the already-documented `D50.5` "Legal-screen UB exposure" nearby in this file (a read that's harmless-by-luck on N64 hardware, fatal on PC) — a good candidate for the same style of minimal `#ifdef PORT` defensive guard, not a game-logic change. +- **Confirmed save-specific, not a stock v0.2.1 regression:** user re-ran the same v0.2.1 binary with a fresh (no-save) `data/` and it ran fine — the crash needs this specific save's corrupted content. +- **The wrinkle that keeps this open rather than closing it as a clean v0.2.1-only bug:** `src/game/language.c`, `textrelated.c`, `file2.c`, `file.c`, and `token.c` are **byte-identical** between the `v0.2.1` tag and current `release/v0.3.0` HEAD (`git diff v0.2.1 HEAD` on those five files shows nothing but two unrelated diagnostic-probe additions in `front.c`/`libultra.c`). Yet replaying the exact same save on a HEAD build through a full, real boot (no `GE_STARTMENU` shortcuts, 90 s window, `tools_pc/debug.ps1` gdb-attached) **did not crash.** Since the code path itself is unchanged, the most likely explanation is that the *actual* defect is upstream of `language.c` — almost certainly an out-of-bounds/garbage read while formatting save-derived display text (leading guess: the packed completion-time/percentage fields, `geSaveGetTime`-style bit-unpacking in `port/src/libultra.c`/`src/game/file2.c`, or a folder/mission-select stat string built from `unlocked_cheats_*`/`times[76]`) — and *which* garbage byte lands there is sensitive to the binary's memory layout, which differs between the two builds purely from the ~100+ unrelated intervening commits. **This means the underlying bug is plausibly still live on HEAD, just not triggered by this specific save on this specific build — not something v0.2.2 fixed on purpose.** D255/D285 are unrelated; both are real, correctly-identified, already-fixed bugs, they just aren't this one. +- **Next steps, in order:** (1) add a temporary `#ifdef PORT` probe at `language.c:319` logging `codepoint`/`i` plus the 8–16 bytes around the source string pointer whenever `g_JpnCacheCacheItems == NULL`, then replay the v0.2.1 repro (or try harder to trigger it on HEAD — a longer soak, or navigating to the specific folder/mission-select row that displays this save's Control Center/Caverns stats, since our 90 s window only proved the *default* landing screen doesn't trigger it) to catch the actual string content live; (2) once the upstream OOB read is identified, the real fix is there, not just a defensive NULL guard in `language.c` (though that guard is also cheap, safe, and worth adding regardless, matching the `D50.5` precedent, since a malformed/third-party-corrupted save should degap gracefully rather than crash even if the deeper cause isn't found yet); (3) do NOT close this finding as "fixed by v0.2.2" — that was this session's own wrong first guess, corrected here. + +**M-148 (2026-09-16, independent re-verification session): FULLY ROOT-CAUSED end-to-end; verdict OPEN — the bug is live on `release/v0.3.0` HEAD, reproduced there from scratch. The M-147 "upstream OOB/garbage read" guess is retracted: no OOB exists.** Every claim below was re-derived from primary evidence (crash logs, both binaries under gdb, the user's actual `ge007.eep`, compiled-C CRC reproduction) rather than trusted from prior write-ups. + +- **The user's save is NOT corrupted — it is a valid *pre-D284* save.** `fileValidateSaves` (`file2.c:544`) CRCs each slot over `&save->completion_bitflags .. save+1` (slot bytes [8..96), 88 bytes) via `fileGenerateCRC` (`src/game/crc.c`), which runs `randomGetNextFrom` off a **local constant seed `0x8F809F473108B3C1`** — *not* `g_randomSeed`. Reproduced that function in compiled C from both the pre-fix (`git show 0de99b4d^:port/src/random.c`) and current-HEAD `random.c`: **the pre-D284 (v0.2.x-shipped) PRNG reproduces all five stored slot CRCs bit-exact; the fixed PRNG matches zero.** (Pure-Python model `tools_pc/d295_crc.py` — gdb scripts/outputs in `scratch/d295/` — now matches the compiled C exactly — two transcription bugs found and fixed along the way: the seed is updated *in place* by `randomGetNextFrom(&polynormal)` each step, and `a2 = a2 >> 0x1f` / `a3 = a2 >> 0x14` are plain `dsrl`, not the `*32` helpers.) Consequence: on any post-fix build, first launch with this save wipes all five slots (confirmed live, `GE_SAVELOG=1`: every slot `BAD -> RESET (progress wiped)`), so **the reported crash does not reproduce on HEAD with the unmodified artifact — but that is a side effect of the wipe, not a fix.** +- **This disproves the D284 commit/entry claim "savegames are unaffected (seed isn't save state)"** — see the correction appended to `## D284`. `fileGenerateCRC`'s stream *is* persisted state: every stored slot CRC is a fingerprint of the PRNG implementation that wrote it. **Release-blocker-class regression for v0.3.0: every existing player's entire campaign progress is silently erased on first launch.** Needs its own D entry + a decision (dual-CRC acceptance / one-time migration / at minimum a loud release note) — flagged here, not unilaterally filed. +- **The crash mechanism (M-147) is confirmed end-to-end and is the whole story; there is no upstream OOB.** `front.c:2616` declares `char difficultytext[4]`; the file-select folder panel does `strcpy(difficultytext, "00 Agent")` (9 bytes incl. NUL) + `strcat("\n")`. gdb on **both** the v0.2.1 artifact and a HEAD debug build confirms the stack layout: `&textpos.p[0] == &difficultytext + 8`, so the `\n` lands on the y-coordinate's low byte and the NUL one past it; the subsequent `textpos.p[0] = floor(folderpos2d.f[1]) + 0x15` store then clobbers `\n` with y. Observed y values across window geometries: **128 and 143 — both ≥ 0x80**, i.e. the fatal case is the *normal* one here. `textRender` (`textrelated.c`, `-funsigned-char`) then scans a byte ≥ 0x80 → JPN glyph path → `langGetJpnCharPixels` (`language.c:320`) dereferences `g_JpnCacheCacheItems`, which is NULL outside Japanese mode on PC. Note: on N64 **non-JP builds this pointer is likewise unallocated** (`langInit` allocates it only when `j_text_trigger`, i.e. JP) — so the crash potential is decomp-native, not a port artifact; the reason it (apparently) never manifested on N64 is that at native geometry `y = floor(folderpos2d.f[1]) + 0x15` presumably stayed < 0x80 (unverified from here), whereas PC window geometry puts y at 128/143 in the observed cases. The overflow itself is decomp-faithful original behavior. `"007"` (4 bytes) and `"Agent"` (6) fit or stop before offset 8 → safe; only `"00 Agent"`/`"Secret Agent"` as a folder's highest completed difficulty reach the fatal byte. (`missiontext[18]` one line above has the same class: `"Mission Cuba.Control Center\n"` is 29 bytes — flagged for review, not separately instrumented.) +- **The M-147 "OOB read of save-derived stat text" guess is retracted — verified against source:** `fileGetSaveStageDifficultyTime` (`file2.c:230`) special-cases `DIFFICULTY_007` *before* any bit math — it returns constant `0x3FF` if `fileIs007ModeUnlocked(folder)` (the slot's `flag_007` bit, or all 20 levels completed on 00 Agent) else 0; the high bit region (offsets 600–790 of a 608-bit `times[]`) is **never read**, and the setter (`fileSetDifficultyStageTime`) excludes 007 (`difficulty < DIFFICULTY_007`), so it's never written either. No OOB, no build-layout sensitivity in the game code. The v0.2.1-vs-HEAD difference was entirely the CRC wipe above: v0.2.1 accepted the save (old PRNG) → folder 1's highest = 00 Agent (`flag_007`=0, 00-Agent 16/20 levels → not 007-unlocked) → `"00 Agent"` rendered → crash; HEAD wiped the save → nothing rendered → no crash. +- **DECISIVE HEAD REPRO (bug is live on v0.3.0):** recomputed the user's five slot CRCs with the *fixed* PRNG (`d295_crc.py`, data-only patch of the artifact — no code probes), placed it in `data/`, ran HEAD unattended 80 s: `GE_SAVELOG` shows all five slots **OK** (progress retained), then `EXCEPTION: 0xc0000005`, `FAULT ADDR: 0x0`, register state byte-identical to the user's v0.2.1 log (`Rax=2 Rcx=0 Rdx=1 Rsi=0xffffffff Rdi=0`); `addr2line` on the HEAD binary → **`langGetJpnCharPixels` at `src/game/language.c:320`**. Repro log saved as `ge007.crash.log.head-repro-M148` in the issue-artifacts folder. `data/ge007.eep` restored to the user's original artifact afterwards (md5 `76e18aa4…`). +- **M-149 (fix landed + verified):** both signed-off items applied and verified on `release/v0.3.0`: (a) `port/src/main.c` seeds `g_JpnCacheCacheItems`/`g_JpnCharCachePixels` from static port buffers before `osStartThread` (exact `langInit` construction, `#include "game/language.h"`, `_Static_assert` on the pixel-stub size; reviewer-caught bugs fixed pre-apply: the `0xC000` bitfield sentinel was wrong — eliminated in favour of field assignment — and a stale `sizeof(jpnItems)==0x100` assert would have hard-failed the build, dropped); (b) `src/game/front.c` widens `difficultytext[4]`→`[32]`, `missiontext[18]`→`[32]` under `#ifdef PORT` (D8 class; N64 build keeps decomp sizes — the decomp author's own comment at front.c:2549-2550 already flagged this buffer as "likely incorrect"). Verification: clean rebuild; **the exact previously-crashing input** (user's save re-stamped with post-fix CRCs via `tools_pc/d295_crc.py`) now runs 80 s unattended with all five slots OK, no crash log — vs the byte-identical register-state NULL deref before the fix; user's original artifact still boots cleanly (takes the D297 wipe path, no crash); 21/21-level `tools_pc/level_sweep.sh` PASS. `data/ge007.eep` restored to the original artifact (md5 `76e18aa4…`) — **note: every game launch rewrites `data/ge007.eep`, so the sweep clobbered the restoration that preceded it; the restore+hash check was re-run after the sweep and again at review time.** +- **Verdict: FIXED (M-149)** for the crash itself; the companion save-wipe regression is tracked separately as **D297** (release-blocker, migration shim pending). History — pre-fix, any v0.3.0 player who completed an 00-Agent (or Secret-Agent) level — their save then validates fine on the post-fix PRNG — would crash at file-select on next launch in any window geometry where the folder y ≥ 0x80 (the common case observed here). Fix plan, **signed off by user 2026-09-16** (both items go; review order: crash-fix bundle first, migration shim separately): (a) *port-side defense-in-depth, rule-2-clean:* seed `g_JpnCacheCacheItems`/`g_JpnCharCachePixels` from static port buffers before `mainproc` starts (non-JP mode only) so a stray high-bit byte takes the normal cache-miss path (romCopy from the JP font segment → garbage glyph) instead of NULL-deref — design validated: `langTick` is `j_text_trigger`-guarded, `langInit` non-JP never touches these, no race; (b) *game-side, D8-class layout correction:* enlarge `difficultytext[4]` → `[16]`, `missiontext[18]` → `[32]` under `#ifdef PORT`. Both are small. Scope note for the migration shim: `fileValidateSaves`'s block-0 `smallSave` CRC (file2.c:512-535) needs **no** migration — `SAVEFLAGS_SET(FOLDER3, SAVESLOT1, BOND_CONNERY, FALSE)` = 0x42 and any in-use save has a different header byte (user's: 0x57), so it takes the "customised file" branch and is factory-reset on every load by design (N64-faithful); only the 5 slots carry user data. The separate v0.3.0 save-wipe regression (D284 side effect, above) is a second, independent release-blocker candidate and should be filed/decided on its own. + +## D296 - Control Center, 00 Agent: noticeable stutter, user's own read is "maybe from the drone guns" (M-146, 2026-09-16, same D295 investigation session). + +**Reported while live-playing Control Center on 00 Agent (`-level_23 -hard2`) to investigate D295** (that crash did not reproduce; this is a separate, secondary observation from the same playthrough). User: "I am getting pretty good stuttering maybe from the drone guns on 00 agent on that level." + +**Caveat that must be resolved before trusting this as a real perf bug — flagged proactively by the user, not yet checked by us:** this playthrough ran under `tools_pc/debug.ps1`, i.e. **gdb-attached**, not the packaged release artifact. The user has previously seen debug-harness overhead produce stutter that doesn't exist in a genuine release build. **Not yet investigated** whether: +- (a) the stutter reproduces on a plain `build-pc/ge007.x86_64.exe` launch (no gdb attached, no `-NoBuild`/dev-tree quirks) — this is the first thing to rule out, cheap (~2 min repro attempt), before any perf tracing; +- (b) it reproduces on the actual packaged/zipped release artifact a real user would download (further rules out any local-tree-only difference, e.g. debug symbols, unstripped binary, dev `data/` sidecar state); +- (c) if it survives both of those, whether it is specifically tied to drone-gun turret enemies (their fire pattern, tracking, or projectile spawn — worth an env-gated per-frame timing probe around whatever `ai_*`/`gunfire.c` path drone turrets use) or just general Control Center scene load on 00 Agent (higher enemy density than Agent/Secret Agent difficulty). + +**Status:** OPEN — single live report, un-triaged, un-reproduced outside the debug harness. **Do not start perf-tracing `port/fast3d/` or AI code over this until step (a)/(b) above rule out harness overhead** — that's the cheap, disqualifying check and it hasn't been run yet. + +## D297 - v0.3.0 silently wipes ALL pre-fix player saves on first launch (D284 side effect; release-blocker, found during D295/M-148). + +**What:** every save written by a v0.2.x build fails `fileValidateSaves` CRC on any post-D284 build and is reset to factory (`BAD -> RESET (progress wiped)`, confirmed live via `GE_SAVELOG=1` on the user's #87 artifact — all five slots). Players upgrading to v0.3.0 lose their entire campaign with no warning. + +**Root cause (full evidence chain in `## D295` M-148):** slot CRCs are computed by `fileGenerateCRC` (`src/game/crc.c`) off a *local constant seed* through `randomGetNextFrom` — so each stored CRC is a fingerprint of the PRNG implementation that wrote it. D284 (`0de99b4d`, port-side `dsll32/dsrl32` semantics fix) changed the PRNG output without changing any persisted state, so every pre-fix CRC now mismatches. The D284 commit's "savegames are unaffected" claim is corrected in `## D284`. + +**Scope of any migration (verified, not assumed):** +- Only the **5 slots** (file bytes 32–607) carry user data; each needs its two CRC words recomputed with the *new* PRNG over slot bytes [8..96). +- The block-0 `smallSave` header check (`file2.c:512-535`) needs **no** migration: `SAVEFLAGS_SET(FOLDER3, SAVESLOT1, BOND_CONNERY, FALSE)` = 0x42 and any in-use save has a different header byte (user's artifact: 0x57), so it takes the "customised file" branch and is factory-reset on every load by design (N64-faithful). It is a pristine-cart seal, not progress. + +**Fix plan (signed off in principle; reviewed 2026-09-16 and landed as M-150):** `port/src/legacycrc.c` — an isolated, clearly-dated port file carrying the *old* (deliberately-wrong) PRNG + CRC math, called from the eeprom shim's read path: if a slot fails the new-CRC check but validates under the legacy PRNG, re-stamp `chksum1/chksum2` with the new-PRNG values before handing the region to the game. Zero game-code changes; rule-2-clean (port layer satisfies a hardware-format dependency). The old PRNG must stay in-tree until no pre-D284 saves are expected in the wild — the file header must say so loudly. Reference implementation: `tools_pc/d295_crc.py` (bit-exact against compiled C for both PRNGs). + +**Alternatives rejected:** dual-CRC acceptance inside `fileValidateSaves` (game-code change, rule 2); "loud release note only" (ships known total data loss — not acceptable for a release). + +- **M-150 (fix landed + verified):** independent review of the draft passed with no corrections — reviewer diffed `geLegacyRandomGetNextFrom` line-by-line against `git show 0de99b4d^:port/src/random.c` (bit-exact, operation for operation), and separately confirmed the `save_data` layout, the hook site in `geEepromRW`'s read branch, that `geEepromPatchAllCheats` is genuinely read-time-only (never persists — so the persist-before-patch ordering claim holds), the one-past-the-end boundary pattern (same as `fileValidateSaves` itself, file2.c:544), and that fresh zero slots fail *both* CRC checks and fall through untouched. Applied: new `port/src/legacycrc.c` (isolated, loud dated header explaining why a deliberately-wrong PRNG lives in the tree and when it may be deleted) + a hook in `port/src/libultra.c` `geEepromRW` read branch — fires only on `block == 4 && nbytes == sizeof(ge_save_slot)*5`, runs **before** `geEepromPatchAllCheats` so only pristine+migrated bytes are ever persisted, persists once via `s_eeprom` + `geEepromStore()` with an INFO log. No CMakeLists change (`SRC_PORT` is `GLOB_RECURSE`); zero game-file edits. Verification on `release/v0.3.0` HEAD (ntsc-final): **(1)** user's original #87 artifact → `D297-migrated 5`, all five slots OK, no crash; stored CRCs now exactly the new-PRNG values (`tools_pc/d295_crc.py`: "new PRNG matches all 5 stored CRCs: True"); relaunch → **no** migration (idempotent), still all OK. **(2)** native post-fix save (CRCs pre-stamped via `d295_crc.py`) → zero migrations, all five OK. **(3a)** corrupted slot (one byte flipped inside slot 0's CRC region) → only 4 migrated; the corrupt slot fails both checks, is left alone, and takes the game's own `BAD -> RESET` wipe path as designed. **(3b)** fresh zeroed eep → no migration, clean boot. Then 21/21-level `tools_pc/level_sweep.sh` PASS. `data/ge007.eep` restored to the original artifact and hash-verified **at report time** (md5 `76e18aa4…`; per the porting-notes §E gotcha, after the sweep, not before). Operational note: `fileValidateSaves` runs from `init_menu00_legalscreen` (`front.c:1450`) — the legal screen appears ~36 s / frame ~1800 into an unattended boot, so verification runs must be ≥ 60 s to observe it. + +**Status:** **FIXED (M-150)** — migration shim landed and verified; release-blocker cleared. Crash-side companion D295 is FIXED (M-149). diff --git a/docs/porting-notes.md b/docs/porting-notes.md index b9607ff6..66bcb0e4 100644 --- a/docs/porting-notes.md +++ b/docs/porting-notes.md @@ -980,6 +980,15 @@ and turns every such overrun into a fatal `*** stack smashing detected ***` (bail is `if (cat >= 41)`, checked once per outer loop → writes `[40..]`); `bondview2.c` `bondviewCalcIntroSwirlCamera` `f32 pointbuf[10]` written `[-3 .. 11]`. +- Instance (D295/M-148): `front.c:2616` `char difficultytext[4]` receives + `strcpy("00 Agent")`/`strcat("\n")` — the NUL lands exactly on the adjacent + `textpos.p[0]` (y-coord), which is then re-stored; when y ≥ 0x80 the string + carries a high-bit byte into `textRender` → JPN-glyph path → NULL deref of + `g_JpnCacheCacheItems` (`language.c:320`). Same class, different fatality: + no canary involved — the clobbered *neighbour* changes control flow. On N64 + the identical overflow is cosmetic (the JPN-cache pointer is valid memory in + any language); the crash severity is a PC artifact of that global being NULL + outside Japanese mode. - Fix policy: **`-fno-stack-protector` globally** (`CMakeLists.txt`) so Linux matches the N64 and MinGW builds; the code is byte-matched to the ROM and cannot be "fixed" per site without diverging. Where a case is cleanly @@ -992,6 +1001,23 @@ and turns every such overrun into a fatal `*** stack smashing detected ***` - Distinct from D7 (`va_list` UB) and D6 (missing `return`) but the same "works on N64 + MinGW, fatal on hardened Linux" shape. +## D9. Changing a PRNG/CRC implementation invalidates persisted derived state — save checksums are de facto file format + +D284 (M-139b) fixed `port/src/random.c`'s `dsll32`/`dsrl32` semantics and the +entry claimed "savegames unaffected (seed isn't save state)". It was wrong: +`fileGenerateCRC` (`src/game/crc.c`) — the per-slot validation checksum checked +by `fileValidateSaves` (`file2.c:544`) — runs `randomGetNextFrom` off its own +**local constant seed**, so every stored slot CRC is a fingerprint of the PRNG +implementation that wrote it. Verified against a real v0.2.x save (GitHub #87): +pre-fix PRNG reproduces all five stored CRCs bit-exact, post-fix matches zero → +first launch of any post-fix build **silently wipes every existing player's +entire campaign** (D295 M-148). Generalizable rule for this port: before +changing anything that feeds persisted derived state (PRNG, CRC/hash routines, +endian/byte-order helpers used by serializers), audit *all* consumers whose +output is stored on disk — not just the in-memory streams. A "port-only" fix to +a checksum primitive is a save-file-format break unless old and new outputs are +made mutually acceptable (dual validation / one-time migration). + ## E. Process / method notes - Investigation loop is: reproduce → env-gated capped probe → root-cause @@ -1003,6 +1029,13 @@ and turns every such overrun into a fatal `*** stack smashing detected ***` writing anything new; same Rare engine family. - Don't re-investigate a closed §F finding or re-derive a format spec that already has a converter. +- **Every game launch rewrites `data/ge007.eep`** (the eeprom shim stores on + every write; validation resets, cheat patches and normal saves all write). + When restoring a specific save artifact for a controlled test (e.g. an issue + reproduction), restore it *after* any sweep/verify run — or re-hash after + each run — not before: `tools_pc/level_sweep.sh` alone boots the game 21 + times against `data/` and will clobber a pre-run restore (caught live in the + D295 M-149 session; "verified" must mean checked at report time). - **The port is NOT frame-deterministic** (D117): `osGetCount()` is wall-clock on PC, and GE is a variable-timestep sim (`frametiming.c waitForNextFrame` → `deltaFrames` = 60 Hz ticks of real