Benchmarks
Four controlled comparisons. Each one holds everything constant except the single thing being compared — the encoder, the model size, the kernel — and reports the spread rather than a winner, because the spread is usually the part that changes a decision.
Where several things are compared at once the p-values are corrected for it, and where a result depends on an arbitrary choice — which clips were drawn, which batch size the model actually runs at — that dependence is stated instead of averaged away. Two of these panels reversed the author’s own prediction, and one of them is most useful for the cell it refuses to quote.
4 articles in this section · all 25 articles, newest first →
Speech
One panel, held constant in the words spoken so that only the delivery and the model change.
Fourteen Encoders Heard the Same Voice. Their Error Rates Differed Five-Fold.
August 2026Eight speakers each recorded the same 1,360 sentences in six phonation states, so the words are held constant and only the delivery changes. Over one frozen list of 11,935 trials, equal error rate ran from 0.047 to 0.233 — produced by nothing but the choice of encoder, with 29 of 91 pairwise comparisons surviving Holm-Bonferroni correction. CAMPPlus is last for every speaker and every condition, and the mechanism is impostor-score compression. Parameter count does not order the panel. Ten claims were retracted along the way.
Models and language
What changes when the model gets bigger, and what does not change at all.
The Size Where a Model Stops Saying "I Don’t Know"
August 2026Five local models against one fact their source text does not contain. Scaling recovered none of it. Compiling the fact worked — above a threshold, with a regression in the middle where the model stopped abstaining and started agreeing.
The Less the Text Said, the More Sure It Got
August 2026On the standard benchmark for contradictory biomedical claims, 77.1% of the flagged pairs never state what a verdict would need. Seven of eight model runs asserted contradiction more often on exactly those pairs, and instructing the model to be conservative raised its refusal rate by forty points without changing the direction at all. 5,824 judgements, all local, with an audit script that recomputes every number.
Compute
A comparison whose useful answer was the crossover point, not the winner.
I Wrote the Kernel. Then I Measured It, and at My Model's Shape It Was 1.67x Slower Than Doing Nothing.
September 2026A hand-written Triton LayerNorm against PyTorch eager and torch.compile on one GPU, at six batch sizes. Above roughly 4,096 rows the kernel wins on time and wins harder on memory — 1.36x faster at 65,536 rows using half the peak memory. Below that crossover it loses, and the model in question trains at 128 rows, where it is 1.67x slower than doing nothing. Includes the two defects the harness found in itself and one cell too unstable to quote.