Independent research imprint

Oakmont Lab

Reproducible reanalyses of what AI actually does in learning and assessment.

I take a published claim about AI in learning or assessment, reanalyse it from the data the authors released, and report how much of it holds up. Every reanalysis is pre-registered before I look and reproducible from the raw files, and none of it is for sale. When a result survives, I say so. When it does not, I say that too.

Experiments

July 2026 What is a reported 0.88 grading correlation worth? A reanalysis of the BeSTraP short-answer dataset A pre-registered reanalysis of the publicly released BeSTraP dataset, which reports a fine-tuned GPT-2 grader at Pearson 0.88 with human short-answer grades. Using only the released data, we ask what that number is worth: it holds up on every mechanism the data lets us check, sits below human consistency, and cannot be recomputed from the release.

↑