mirror of
https://github.com/jkdansereau/goldeneye-pc-port.git
synced 2026-10-23 15:52:42 +02:00
chore(agentic): speed-ups plan Steps 1, 4 (N2/R8/R9), 6
Static/tooling slice of docs/dev/notes/AGENTIC-SPEEDUPS-PLAN.md — no build or
runtime environment needed to verify any of it.
- Step 6 / N7: add a VERIFY: line to the investigation-brief template
(docs/dev-process.md) and the CLAUDE.md dispatch-preflight checklist — the
brief must now name the exact probe/verify invocation that proves the task
done, or state that runtime verification is impossible here.
- Step 4 / N2: tools_pc/README.md — indexes the ~60 scripts with a
living vs artifact marker so closed-finding artifacts can be archived.
- Step 4 / R8: tools_pc/gen_findings_index.py -> docs/dev/findings-index.csv
(label, bucket, one_liner, status_raw) — grep-before-you-read aid for the
~200 KB finding log. Heuristic parser of the §F/§H tables.
- Step 4 / R9: tools_pc/gen_env_probes.py — re-greps live getenv("GE_*")
sites and reports NEW/GONE drift vs docs/dev/GE-ENV-PROBES.md (keeps the
curated prose, only checks the site map). Doc reconciled: 7 new probes
added, GE_D54 kept as a tombstone.
- CI: validate job now runs both generators with --check.
Step 1 / N1 (HANDOFF preflight) applied in the local-only docs/HANDOFF.md.
Step 2 (verify.sh) and the HANDOFF/HANDOFF-ARCHIVE trim are left for a
session with a build environment.
This commit is contained in:
1 parent
5c3944582b
commit
dbea5a30fe
8 files changed
+427
-3
No files matched your search
@@ -0,0 +1,73 @@
|
||||
# `tools_pc/` — PC-port dev & test scripts
|
||||
|
||||
Index for the ~60 scripts here (speed-ups plan Step 4 / N2). None of these are
|
||||
part of the build; they are converters, verification tooling, and
|
||||
investigation artifacts.
|
||||
|
||||
**Marker:**
|
||||
- **`living`** — kept and re-run; part of the verification ritual, the release
|
||||
flow, or a converter the build depends on (via the sidecars it emits).
|
||||
- **`artifact`** — added to chase one finding. Safe to move to
|
||||
`tools_pc/archive/<dxx>/` once that finding is closed and the technique is
|
||||
captured in `docs/dev/findings.md` / `docs/porting-notes.md`.
|
||||
|
||||
## Living — converters the sidecars depend on
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `d43_emit.py` | Offline N64→PC model-file converter → RZ sidecar + `manifest.csv` (D50 / Plan B). Regen after any model-format change. |
|
||||
| `d43_convert.py` | Reference single-file model converter + full-512 layout/pointer validator (D43). |
|
||||
| `d69_emit.py` | Offline converter for stage `bg/*.seg` + `Tbg_*_stanZ` → concatenated sidecar (D69/D78–D82). |
|
||||
| `d88_emit.py` | Offline converter for per-level `Usetup*Z` stage-setup files → appended to the `pccg.bin` sidecar (D88). |
|
||||
| `d88_propdefs.py` | The `propDefs` polymorphic-record stream N64→PC converter used by `d88_emit.py` (D88.4). |
|
||||
| `d69_emit.py` / `d88_emit.py` / `d43_emit.py` | Run all three (+ `d88_propdefs`) to rebuild `data/` sidecars from the ROM. |
|
||||
|
||||
## Living — verification & release
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `framediff.py` | Visual regression: candidate `GE_PCDUMP` frames vs `tools_pc/golden/`, per-region divergence. Structural/tolerant by default; `--exact` after `GE_DETERM`. |
|
||||
| `pixcount.py` | Count non-black pixels in a PPM dump — "did the scene render anything" as a number. |
|
||||
| `level_sweep.sh` | Bare `-level_XX` boot of all 21 solo levels → PASS / NO-FRAMES / CRASH. (Step 2 folds this into `verify.sh sweep`.) |
|
||||
| `playtest.sh` | Launch a level for `docs/dev/LEVEL-PLAYTEST.md` human validation (WS6). |
|
||||
| `debug.ps1` / `repro_gdb.sh` / `attach_animgen.sh` | Launch (or attach to) the game under gdb so a crash always leaves a backtrace. |
|
||||
| `bundle-win.sh` / `bundle-linux.sh` | Package a built tree as a distributable archive (exe + licenses + `prepare-assets/`). |
|
||||
| `romverify.c` | One-shot `.z64` integrity check against the repo's ground truths. |
|
||||
| `dump_objectives.py` | Dump per-level objectives + win/fail criteria from the ROM (playtest validation; Step 10 / N8 acceptance oracle). |
|
||||
| `disasm.py` | Minimal MIPS disassembler for the BE ROM (RAM-addr → file offset). |
|
||||
| `ppm2bmp.py` | PPM → 24-bit BMP, no deps — eyeball `GE_PCDUMP` frames without PIL. |
|
||||
| `gen_findings_index.py` | Regenerate `docs/dev/findings-index.csv` (grep-before-you-read aid for the 200 KB finding log). `--check` in CI-style use. |
|
||||
| `gen_env_probes.py` | Drift check for `docs/dev/GE-ENV-PROBES.md` — re-greps live `getenv("GE_*")` sites, reports NEW/GONE. |
|
||||
|
||||
## Living — compiler-verified layout probes (kept: re-run when structs change)
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `d43_layoutprobe.c` | Prints `sizeof`/`offsetof` for every struct `d43_emit.py` must reproduce, built with the port toolchain. |
|
||||
| `d88_layoutprobe.c` | Same, for the `PROPDEF_*` record structs `d88_emit.py` converts. |
|
||||
| `mtxtest.c` | Standalone numerical test of the fast3d matrix pipeline conventions (GE swapped-perspective, packing, MUL order). |
|
||||
|
||||
## Investigation artifacts — D43 model-file format (finding closed; archive candidates)
|
||||
|
||||
`d43_chainbound.py`, `d43_cover.py`, `d43_decode.py`, `d43_fullwalk.py`,
|
||||
`d43_gdldump.py`, `d43_gdlhist.py`, `d43_gdlorder.py`, `d43_gdlseq.py`,
|
||||
`d43_invariants.py`, `d43_layout.py`, `d43_layout1.py`, `d43_lutscan.py`,
|
||||
`d43_pointusage.py`, `d43_seg5.py`, `d43_seg5ops.py`, `d43_seg5vtx.py`,
|
||||
`d43_seg5vtx2.py`, `d43_sizes.py`, `d43_sizes2.py`, `d43_tree_dump.py`,
|
||||
`d43_vtxfmt.py`, `d43_walk.py` — layout/pointer/GDL-order analysis passes that
|
||||
derived the D43 converter spec. `d43_sizes.py` is still `exec()`'d by a couple
|
||||
of the others.
|
||||
|
||||
## Investigation artifacts — other closed/parked findings
|
||||
|
||||
| Script | Finding |
|
||||
|---|---|
|
||||
| `d125_check.py` | D125 — is the emitted `propDefs` blob byte-identical to `d88_propdefs.convert_stream()`? |
|
||||
| `d88_propdef_scan.py` | D88.4 — histogram of `PROPDEF_*` record types across shipped levels. |
|
||||
| `d51_gdb.py` / `dump_animgen.cmd` | D51 texture-tile crash capture / Facility-outro-hang anim-state dump (gdb scripts). |
|
||||
|
||||
## Other
|
||||
|
||||
- `dist/` — staged bundle README template (`README.md.in`) + tokens.
|
||||
- `golden/` — per-level golden frames for `framediff.py` (Step 3 fills this out).
|
||||
- `__pycache__/` — ignored.
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Drift check for docs/dev/GE-ENV-PROBES.md (speed-ups plan Step 4 / R9).
|
||||
|
||||
That doc is a hand-curated table of every `GE_*` env probe -- var, file:line,
|
||||
what it does, live/dead. It was last snapshotted by a manual grep and drifts
|
||||
as probes are added/removed. This script does NOT regenerate the prose (the
|
||||
curation is the value); it re-greps the live `getenv("GE_...")` sites and
|
||||
reports:
|
||||
* NEW -- a var with live sites that the doc never names
|
||||
* GONE -- a var the doc names that has no live site any more
|
||||
* a fresh var -> file:line site map (paste into the doc's File:line cells)
|
||||
|
||||
Usage:
|
||||
python tools_pc/gen_env_probes.py # print the drift report
|
||||
python tools_pc/gen_env_probes.py --check # exit 1 if NEW or GONE
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DOC = ROOT / "docs" / "dev" / "GE-ENV-PROBES.md"
|
||||
SCAN_DIRS = ["src", "port"]
|
||||
GETENV_RE = re.compile(r'getenv\(\s*"(GE_[A-Z0-9_]+)"\s*\)')
|
||||
|
||||
|
||||
def live_sites() -> dict[str, list[str]]:
|
||||
sites: dict[str, list[str]] = {}
|
||||
for d in SCAN_DIRS:
|
||||
for path in sorted((ROOT / d).rglob("*")):
|
||||
if path.suffix not in {".c", ".cpp", ".h", ".def", ".inc"}:
|
||||
continue
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
except OSError:
|
||||
continue
|
||||
for n, line in enumerate(lines, 1):
|
||||
for m in GETENV_RE.finditer(line):
|
||||
rel = path.relative_to(ROOT).as_posix()
|
||||
sites.setdefault(m.group(1), []).append(f"{rel}:{n}")
|
||||
return sites
|
||||
|
||||
|
||||
def doc_vars() -> tuple[set[str], set[str]]:
|
||||
"""Return (all vars named in the doc, vars whose row is an intentional tombstone)."""
|
||||
if not DOC.exists():
|
||||
return set(), set()
|
||||
named, tombstone = set(), set()
|
||||
for line in DOC.read_text(encoding="utf-8").splitlines():
|
||||
vs = re.findall(r"GE_[A-Z0-9_]+", line)
|
||||
named.update(vs)
|
||||
if "tombstone" in line.lower():
|
||||
tombstone.update(vs)
|
||||
return named, tombstone
|
||||
|
||||
|
||||
def main() -> int:
|
||||
check = "--check" in sys.argv[1:]
|
||||
sites = live_sites()
|
||||
documented, tombstoned = doc_vars()
|
||||
live = set(sites)
|
||||
|
||||
new = sorted(live - documented)
|
||||
gone = sorted(v for v in documented - live if v not in ({"GE_DETERM"} | tombstoned))
|
||||
|
||||
out = []
|
||||
out.append(f"# GE_* probe drift vs {DOC.relative_to(ROOT).as_posix()}")
|
||||
out.append(f"# {len(live)} vars with live sites, {len(documented)} named in the doc\n")
|
||||
if new:
|
||||
out.append("## NEW -- live sites, not in the doc (add a row):")
|
||||
for v in new:
|
||||
out.append(f" {v}")
|
||||
for s in sites[v]:
|
||||
out.append(f" {s}")
|
||||
if gone:
|
||||
out.append("\n## GONE -- named in the doc, no live site (mark removed):")
|
||||
out += [f" {v}" for v in gone]
|
||||
if not new and not gone:
|
||||
out.append("doc is in sync with live getenv sites (site line numbers still drift -- see map below)")
|
||||
|
||||
out.append("\n## full live site map:")
|
||||
for v in sorted(sites):
|
||||
out.append(f" {v}: " + ", ".join(sites[v]))
|
||||
|
||||
print("\n".join(out))
|
||||
if check and (new or gone):
|
||||
sys.stderr.write(f"\nGE-ENV-PROBES.md drift: {len(new)} new, {len(gone)} gone\n")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,157 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate docs/dev/findings-index.csv from the §F/§H tables in findings.md.
|
||||
|
||||
Agentic speed-ups plan Step 4 / R8: findings.md is ~200 KB. Agents should grep
|
||||
this CSV (label, status, one-liner) to decide *which* Dxx entries to actually
|
||||
read, instead of linear-reading the log.
|
||||
|
||||
Heuristic parser: every markdown table row whose first cell matches a Dxx
|
||||
label (`D12`, `D88.4`, `D75 · D77`, `D24-D30`, ...). The topic cell is
|
||||
truncated to a one-liner; the status cell is normalised to a short bucket
|
||||
(RESOLVED / OPEN / FIXED / PARTIAL / PROPOSED / ?) with the raw text kept.
|
||||
|
||||
Usage:
|
||||
python tools_pc/gen_findings_index.py # write the CSV
|
||||
python tools_pc/gen_findings_index.py --check # exit 1 if stale
|
||||
python tools_pc/gen_findings_index.py --stdout # print, don't write
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import io
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SRC = ROOT / "docs" / "dev" / "findings.md"
|
||||
OUT = ROOT / "docs" / "dev" / "findings-index.csv"
|
||||
|
||||
LABEL_RE = re.compile(r"^D\d+(?:\.\d+)?(?:[-–]D\d+)?(?:\s*[·,]\s*D\d+(?:\.\d+)?)*\+?$")
|
||||
|
||||
STATUS_BUCKETS = [
|
||||
("FIXED", re.compile(r"\bfixed\b", re.I)),
|
||||
("RESOLVED", re.compile(r"\bresolved\b", re.I)),
|
||||
("PARTIAL", re.compile(r"\bpartial|clamp landed|landed\b|safety-net", re.I)),
|
||||
("PROPOSED", re.compile(r"\bproposed|not applied|design\b", re.I)),
|
||||
("OPEN", re.compile(r"\bopen\b|not started|awaiting|owed|pending", re.I)),
|
||||
]
|
||||
|
||||
|
||||
def one_liner(cell: str, limit: int = 160) -> str:
|
||||
"""Strip markdown emphasis/code and collapse to a single short line."""
|
||||
txt = cell.replace("**", "").replace("`", "")
|
||||
txt = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", txt) # [t](u) -> t
|
||||
txt = re.sub(r"\s+", " ", txt).strip()
|
||||
# first sentence-ish
|
||||
m = re.match(r"(.{20,}?[.—:])\s", txt)
|
||||
if m and len(m.group(1)) <= limit:
|
||||
txt = m.group(1)
|
||||
if len(txt) > limit:
|
||||
txt = txt[: limit - 1].rstrip() + "…"
|
||||
return txt
|
||||
|
||||
|
||||
LEAD_RE = re.compile(r"^\**\s*(OPEN|RESOLVED|FIXED|PROPOSED|PARTIAL|LANDED|CLAMP|DISPROVEN)", re.I)
|
||||
|
||||
|
||||
def bucket(status_cell: str) -> str:
|
||||
# An explicit leading verdict token wins over prose keywords deeper in the
|
||||
# cell (e.g. "OPEN (parked). ... D168 fixed ..." is OPEN, not FIXED).
|
||||
lead = LEAD_RE.match(status_cell.strip())
|
||||
if lead:
|
||||
tok = lead.group(1).upper()
|
||||
return {"LANDED": "PARTIAL", "CLAMP": "PARTIAL", "DISPROVEN": "RESOLVED"}.get(tok, tok)
|
||||
for name, rx in STATUS_BUCKETS:
|
||||
if rx.search(status_cell):
|
||||
return name
|
||||
return "?"
|
||||
|
||||
|
||||
def split_row(line: str) -> list[str] | None:
|
||||
line = line.strip()
|
||||
if not line.startswith("|"):
|
||||
return None
|
||||
# split on '|' that is not inside backticks
|
||||
cells, buf, tick = [], [], False
|
||||
for ch in line:
|
||||
if ch == "`":
|
||||
tick = not tick
|
||||
buf.append(ch)
|
||||
elif ch == "|" and not tick:
|
||||
cells.append("".join(buf))
|
||||
buf = []
|
||||
else:
|
||||
buf.append(ch)
|
||||
cells.append("".join(buf))
|
||||
cells = [c.strip() for c in cells]
|
||||
# leading/trailing empty from outer pipes
|
||||
if cells and cells[0] == "":
|
||||
cells = cells[1:]
|
||||
if cells and cells[-1] == "":
|
||||
cells = cells[:-1]
|
||||
return cells or None
|
||||
|
||||
|
||||
def rows():
|
||||
section = None
|
||||
for raw in SRC.read_text(encoding="utf-8").splitlines():
|
||||
h = re.match(r"^#+\s+([A-Z])\.\s", raw) or re.match(r"^#+\s.*§?([FH])\b", raw)
|
||||
if raw.startswith("### ") and re.search(r"\b([FH])\.", raw):
|
||||
section = re.search(r"\b([FH])\.", raw).group(1)
|
||||
cells = split_row(raw)
|
||||
if not cells or len(cells) < 2:
|
||||
continue
|
||||
label = cells[0].replace("–", "-")
|
||||
if not LABEL_RE.match(cells[0]):
|
||||
continue
|
||||
topic = cells[1]
|
||||
status = cells[2] if len(cells) > 2 else ""
|
||||
yield {
|
||||
"label": label,
|
||||
"bucket": bucket(status),
|
||||
"one_liner": one_liner(topic),
|
||||
"status_raw": one_liner(status, 200) if status else "",
|
||||
}
|
||||
|
||||
|
||||
def build_csv() -> str:
|
||||
buf = io.StringIO()
|
||||
w = csv.DictWriter(
|
||||
buf, fieldnames=["label", "bucket", "one_liner", "status_raw"], lineterminator="\n"
|
||||
)
|
||||
w.writeheader()
|
||||
seen = set()
|
||||
for r in rows():
|
||||
key = r["label"]
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
w.writerow(r)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = set(sys.argv[1:])
|
||||
text = (
|
||||
"# GENERATED by tools_pc/gen_findings_index.py -- do not hand-edit.\n"
|
||||
"# Grep this to pick which findings.md entries to read; it is not authoritative.\n"
|
||||
+ build_csv()
|
||||
)
|
||||
if "--stdout" in args:
|
||||
sys.stdout.write(text)
|
||||
return 0
|
||||
want = text.encode("utf-8")
|
||||
if "--check" in args:
|
||||
cur = OUT.read_bytes() if OUT.exists() else b""
|
||||
if cur != want:
|
||||
sys.stderr.write("findings-index.csv is stale: re-run gen_findings_index.py\n")
|
||||
return 1
|
||||
return 0
|
||||
OUT.write_bytes(want)
|
||||
print(f"wrote {OUT.relative_to(ROOT)} ({text.count(chr(10))} lines)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in new issue
Block a user