Exit codes and storage
| Exit code | Meaning |
|---|---|
0 | every case passed (or --min-pass-rate was met) |
1 | at least one case failed, was flaky, or errored (a broken pipeline is never green); for compare --fail-on-regression, the gate failed |
2 | usage or configuration error; nothing was run (invalid suite, missing env var, bad flag, a judge that does not work) |
130 | interrupted (Ctrl+C); attempts finished so far are saved and the run is marked interrupted |
- Errored vs failed: failed means the pipeline answered and a scorer said no. Errored means BehavTest couldn't get a verdict (pipeline down, timeout, judge unavailable). The console and the JSON report keep them apart.
- Where results are stored: one SQLite file,
.behavtest/results.db(or--db <path>), with tablesruns,results(one row per attempt, with snapshots of the input and expected values),scores,tracesandlabels(your pass/fail labels on judge verdicts). Runs are written incrementally, so a crash keeps what completed, and older databases upgrade automatically. - Nothing is written anywhere else unless you ask for a file (
--json,--md,--export,report --out).