Artifact 20279089301ade287f87f8657e7560346aa5bc81b709bb12cc41ae3d9163fa1c:
- File
scripts/soak.README
— part of check-in
[fa78377389]
at
2026-09-18 12:31:12
on branch trunk
— soak: a nightly deep DST and fuzz run
scripts/soak.sh passed the optimize flag this Zig rejects, so it has been unrunnable since the toolchain moved; it takes --release=safe now. A scheduled workflow runs it at 03:00 UTC on the x64 runner with the script's nightly budgets, overridable from the dispatch form, keeps the log and any reproduce line as an artifact, and is pinned to the ci toolchain by ci_test like the other workflows. (user: dmestas, size: 2908) [annotate] [blame] [check-ins using]
soak.sh — out-of-band deep DST + fuzz soak runner ================================================= What it is ---------- The pre-commit gate and CI run the DST and fuzz harnesses SHALLOW, for regression detection only: `zig build dst` sweeps a few seeds x a few thousand steps, and CI fuzzes one seed x 100 rounds ("regression detection, not a bug hunt", in CI's own words). soak.sh is the bug hunt — the SAME harnesses at nightly depth on a ReleaseSafe build (asserts ON). It adds machine-hours, not new invariants: every "depth belongs to an out-of-band runner" comment in the tree refers to this script. It never auto-files and never auto-fixes. On the first failure it appends the harness's own `reproduce with seed=...` line (reshaped into a runnable command) to a dated log, drops a flag file, and exits non-zero. Running it ---------- Full nightly (multi-hour): ./scripts/soak.sh Bounded smoke (minutes): DST_SEEDS=3 DST_STEPS=500 \ FUZZ_TARGET_BUDGET=6 FUZZ_SEEDS=3 ./scripts/soak.sh Config env vars and their nightly defaults are documented in the script header (DST_SEEDS=200, DST_STEPS=10000, FUZZ_TARGET_BUDGET=300s/target, FUZZ_SEEDS=20, ZIT_SOAK_DIR=$HOME/.zit-soak). Logs and the flag file live under ZIT_SOAK_DIR, outside the repo, so a soak run never shows up in `fossil status`. Depth by time, not by flat rounds: a flat --rounds would run the cheapest fuzz target in seconds and the heaviest (~276x per-round cost, src/fuzz_tests.zig: 63-78) for hours. Each target instead gets an equal wall-clock budget and its --rounds is grown from measured speed to fill each slice, so the per-target cost ratios honor themselves with no second table to drift. The target list is enumerated from the runner's own usage output (the ci.yml:146-159 pattern), so that cannot drift either. Operator responsibilities (NOT done by the script) -------------------------------------------------- 1. Install the schedule. The script only RUNS the soak; installing a launchd/cron entry is an operator action. Copy-paste-ready launchd plist and cron line are in the script header. The self-hosted runner being down does not block a manual `./scripts/soak.sh`. 2. Run the first full nightly and retain its log. 3. Triage findings. A raised flag ($ZIT_SOAK_DIR/SOAK-FAILED) is a P6 GO/NO-GO input, not a footnote. Reproduce with the logged line, disposition, then delete the flag. A clean run never clears a flag the operator has not yet seen. Reproduce lines --------------- dst: zig build dst --release=safe -- --seed N --steps <steps> fuzz: zig build fuzz --release=safe -- <target> --seed 0xSEED --rounds R Both replay the exact failure: one PRNG stream per run, the seed is the whole universe. The dst line carries --steps (the harness's own line omits it); the fuzz line carries the target and the --rounds the soak had grown to.