N1Bench
The first open benchmark for causal reasoning over personal health data. We publish our own scores next to every frontier model — a system that tells you what’s working should be able to prove it doesn’t tell you things that aren’t.
Abstain and Overall columns are shown on wider screens.
What we measure
How close the recovered effect size sits to ground truth on records where a real effect was planted. Scored on absolute error, normalised by the outcome's own night-to-night variation.
Whether a stated 95% interval contains the truth 95% of the time. Penalises overconfidence and uselessly wide intervals equally — an interval that always covers by being enormous scores no better than one that never covers.
On records with no detectable effect, how often the system declines to conclude instead of inventing one. This is the metric most systems fail, and the reason a leaderboard is worth publishing at all.
What each part is worth
The same model with one component removed at a time, scored on the same held-out records.
The held-out records and the scoring harness are open. Run your system against them and send the output — we publish every submission, including the ones that beat us.
Read the v0.4 write-up →