Evals for the Applied AI Engineer's Brief

Does the brief tell the truth?

I ran the pipeline on 15 Stockholm startups, read every failure, and changed it twice. Each version was scored on the same startups.

Results

What each metric means

Limits

15 startups is small, and each version ran once: a difference of one or two facts moves a rate a lot. The checker is a model too, so its verdicts can be wrong. The reference one-liners are mine. Treat these as signals for where to look, not as a benchmark.

Every brief