AI Output Is Nondeterministic. Here's How to Test It Anyway.
Why pass or fail testing breaks for AI applications, and the statistics bug that shaped how BehavTest tells a real regression from noise.
Blog
Articles on behavioral regression testing for AI applications, and on the decisions behind BehavTest, newest first. Follow along with the RSS feed.
My RAG Eval Kept Failing the Cheap Model. The Model Wasn't the Problem.
A real debugging story: why a cheap LLM judge scored 7 out of 12 on a RAG faithfulness check, and why the fix was the prompt, not the model.