Deal with flaky, non-deterministic outputs
Run each case several times with --repeat 5. A case that passes on some attempts and fails on others is labelled flaky. compare then decides whether a change in pass rate is bigger than the noise. See repeats.