Research
Fourteen studies. Some reproduce a result with a known answer to find out what a fresh implementation gets wrong; some ablate a system the author built to find out which stage is actually costing him; some are audits of the instruments themselves, which is where several of the more uncomfortable findings came from.
What they have in common is the reporting standard rather than the subject. Each one states what was measured, over how many trials, with the correction applied where multiple comparisons were made — and each one records the claims that did not survive. Several headline results on this page are smaller than the first version of themselves, and two were withdrawn outright. Those are left visible on purpose: a page that only shows the claims that held is not a research record, it is a highlight reel.
14 articles in this section · all 28 articles, newest first →
The published record — 13 papers with DOIs
Open access on Zenodo, single-author, every one with released code or data. The DOI given is the concept DOI, which always resolves to the newest version — several of these have been corrected in place after publication, and the correction is part of the record rather than a reason to hide the earlier version.
Reconstruction History, Not Synthesis: Benign Processing Moves Frozen Detectors on Genuine Human Speech
17 Sep 2026 · DOI 10.5281/zenodo.22819222Thirteen frozen synthetic-speech detectors, declared and hashed before any was scored. A neural-codec encode-decode that generates no speech tokens moved 9 of 13 past Holm correction on genuine studio recordings; Griffin-Lim, from 1984 and with no neural network in it, moved 12 of 13. Thirteen of his own claims were withdrawn under three rounds of independent adversarial audit. Write-up →
What a Streaming Decoder Costs: A Five-Gate Benchmark of Twenty Neural Audio Decoders, and a Metric That Failed Its Own Validity Check
17 Sep 2026 · DOI 10.5281/zenodo.22798415Only three of twenty arms truly stream, and the negative controls through the identical code path sit at exactly 1, which is what makes the positives mean anything. The sole pre-registered quality metric was beaten by Griffin-Lim, which has zero trained parameters — reported as a finding about the metric because the pre-registration declared in advance that it would be. Write-up →
Continuous Vocal-Attribute Traversals Fall Below the Path Resolution of Current Speech Representations
6 Sep 2026 · DOI 10.5281/zenodo.22538235A pre-registered null. 58 recorded traversals — a trained singer sliding continuously from clean phonation into rasp, growl or breathiness — so every intermediate point on the path is a real larynx rather than a model's guess. The validity gate failed in both tiers across six representations, SNR 0.53 to 1.85 against a required 3.0. The two hypotheses are unreachable with these instruments, NOT refuted. Write-up →
Finite-Sample Decision Risk in Unseeded Quantum Compiler Benchmarking
19 Sep 2026 (Version 4) · DOI 10.5281/zenodo.22310059The real change between two compiler versions is +5.37% (95% CI +4.27 to +6.50), yet the suite's own three-run protocol reports it as a regression of 10% or more 24.4% of the time, reproduced at 22.6% on a disjoint 200-seed sample. Over the 26 eligible circuits, 12 have a decision-risk interval excluding zero, 7 carry at least 5% risk and 4 at least 10%. Version 4 is the audited release: five independent adversarial audits, four of his own claims withdrawn, and every defect they found in his own apparatus documented rather than removed. Write-up →
Structure Substitutes for Scale in Cross-Document Temporal Reasoning
30 Aug 2026 · DOI 10.5281/zenodo.22169643Thirty-eight questions whose answer no passage states. Every model scored 0 of 38 on the prose and correctly refused; an 8B model given the identical facts as a structured chronology scored 28 of 38, matching a 14B model on prose. An audit of his own scorer afterwards cut the headline from +32 points to +15.8 — corrected in place, not retracted. Write-up →
Speaker Encoders Disagree About Who You Are When You Shout: A Content-Matched Benchmark of Fourteen Speaker Encoders Under Expressive Phonation
29 Aug 2026 · DOI 10.5281/zenodo.22158030Fourteen speaker-verification encoders over one frozen list of 11,935 trials, built on eight speakers recording the same 1,360 sentences in six phonation states. Equal error rate spans 0.047 to 0.233 on exactly the same audio; CAMPPlus has the highest error of the fourteen, all 13 of its comparisons surviving Holm–Bonferroni. Parameter count does not order the panel. Ten retracted claims are published with the measurement that killed each one. Write-up →
Intra-Speaker Vocal Variation and Speaker-Embedding Displacement — A Matched-Content Replication Across Three Encoder Architectures
Aug 2026 · DOI 10.5281/zenodo.21921958Four speakers, matched sentences, three encoders. One speaker's voice sat 0.7 semitones below normal — imperceptible — and his verification score fell 0.238 across all eight sentences. A p = 0.013 result he was entitled to report was withdrawn because two defensible corrections gave opposite directions. Superseded in scale by the fourteen-encoder benchmark; the two studies are never combined. Write-up →
The Loss Curve Is Not a Sufficient Statistic — Silent Objective Failures from Sentinel–Class Collisions in Neural Codec Language Models
Aug 2026 · DOI 10.5281/zenodo.21864659A sentinel–class collision makes masked cross-entropy silently delete an entire class — usually the end-of-sequence token — while the loss curve still descends normally. The reported loss is a correct measurement of a corrupted objective, so no curve-based diagnostic can catch it. Reproduced in 40 lines on CPU; both detectors are released in trainproof. Write-up →
A 2D Linear-Elasticity Finite Element Solver Written from First Principles, Validated Against the Kirsch Stress Concentration Factor
Aug 2026 · DOI 10.5281/zenodo.21892064A solver built from nothing — hand-written Delaunay mesher, isoparametric elements, conjugate-gradient solve, no FEA library and no scipy. It recovers the analytic Kirsch factor as 3.00002 against an exact 3 the code never receives. Ten gates were pre-registered; nine pass, and the failure is published rather than hidden. Write-up →
An Automated Failure-Mode QA Framework for Neural Text-to-Speech Systems
Jun 2026 · DOI 10.5281/zenodo.20757553130 edge cases across 3 voices, 390 samples, zero structural audio defects. Blinded human validation of the 42 ASR-uncertain cases split 45/55 between real failures and ASR false negatives, which is the argument for quarantining uncertain cases instead of auto-judging them. The methodology behind ttsproof. Write-up →
Diagnosing Hierarchical Retrieval Failure in Long-Document RAG
Jun 2026 · DOI 10.5281/zenodo.20692451A stage-level ablation isolating where long-document retrieval actually fails: first-stage chapter-selection error compounding in 27 of 80 questions and neighbour-expansion dilution in 32 of 80. Oracle-chapter routing recovers recall to 0.7844, which quantifies how much of the loss is routing rather than ranking. Write-up →
LongBook Verifier — Evidence-Grounded Evaluation of Long-Document Retrieval and AI-Generated Answers
2026 · DOI 10.5281/zenodo.20513116Retrieval baselines, model-output scoring and a 240,767-word stress-test benchmark, released as a complete research package. The research behind the live BookProof product.
BedVibe-TTS — An Engineering Report on a Multilingual Neural Codec Language Model with Multi-Axis Conditioning
Apr 2026 · DOI 10.5281/zenodo.19781414Architecture, data pipeline, training diagnostics and ablations for a from-scratch multilingual neural codec language model with speaker, emotion and trait conditioning, plus a reproducibility finding on attention-backend × precision interactions across GPU generations.
Speech and training runs
What a model does to a voice, and what a loss curve fails to say about the run that produced it.
I Recorded the Path a Real Voice Takes. Six Representations Could Not See It, So My Two Hypotheses Were Never Tested.
September 2026A validity gate written before any embedding existed, with the power to stop the study — and it did. 58 recorded traversals, a trained singer sliding continuously from clean phonation into rasp, so every intermediate point on the path is a real larynx rather than a model’s guess. Six representations, four window settings, zero passes: SNR 0.53 to 1.85 against a required 3.0. The instrument is verified working and the obvious artifact excluded, so the two hypotheses are recorded as unreachable with these instruments rather than refuted. Neither was ever computed.
How Little Your Voice Has to Change Before a Machine Stops Recognising It
August 2026Four speakers, eight matched sentences, three encoder architectures. One speaker moved his pitch by less than a semitone and lost a quarter of his identity score. Includes a control that could have killed the result and a pre-registered prediction withdrawn because two defensible ways of cleaning the data gave opposite answers. Superseded in scale by the fourteen-encoder benchmark; the two studies are separate and their numbers are never combined.
One Integer Deleted the Stop Token From My Loss. The Curve Never Noticed.
August 2026A text-to-speech model trained for weeks and never learned to stop talking. Two constants happened to be equal, the stop token fell out of the loss, and no loss curve anywhere could have shown it.
Why Corrupted Training Data Doesn't Show Up as High Loss
August 2026Bad samples are assumed to announce themselves as high loss. Two failures from production say the opposite: noise is learnable, so it hides.
Retrieval and agents
Systems whose correctness depends on what they did not read. Every study here measures the part that is missing rather than the part that was returned.
I Assumed My Retriever Failed at Stage One. The Bigger Failure Was at Stage Three.
August 2026A stage-level ablation over 80 questions and 400 scored rows. The stage blamed for the loss was not the stage responsible: disabling neighbour expansion raised context recall from 0.3365 to 0.4771, level with the baseline it was losing to, while hit@k did not move at all.
My Agent Answers From 0.6% of Its Corpus and Reports It Like a Full Read
August 2026An agent holding 1,003 chunks answers from six of them, and the answer reads exactly like one built from reading all 1,003. Faithfulness cannot catch it: an answer perfectly grounded in what was retrieved can still be wrong about what was not.
My Headline Was +32 Points. Then I Audited My Own Scorer.
August 2026Thirty-eight questions whose answer no passage states. Every model scored 0 of 38 on the prose and correctly refused; an 8B model given the same facts as a structured chronology scored 28 of 38. Then an audit of the scorer cut the headline from +32 points to +15.8 — corrected in place, not retracted.
I Abandoned a Study Because the Index Already Answered It. Then I Checked the Index.
September 2026A study abandoned because a code-intelligence index was said to answer the question exactly. Checking the index first found it has no function-level nodes at all, so it does not answer that question. Two bugs filed, both confirmed by the maintainer at source level.
Instruments, search and indexing
Measurements of the measuring apparatus. Four cases where the instrument, not the system under test, was the thing that was wrong.
Two of My Instruments Were Wrong, in Opposite Directions
August 2026An audit of 38 live pages where the two findings worth keeping both came from the tooling being broken: one script reported fixed pages as broken, and one date parser had thrown on every crawl-statistics read since it was written, so the report answering "is the crawler arriving at all" had never once returned a number.
Requesting Indexing Did Nothing, and I Have the Control Group
August 2026One host with 0 of 38 URLs in Google while four sibling hosts on the same domain, IPs and certificate indexed normally. The standard advice — submit the URLs — was run against a held-out control: treatment 32%, control 33%. Fifteen causes eliminated with measurements, two of them the author’s own hypotheses withdrawn.
The Compiler Got 5% Slower. The Benchmark Called It a 10% Regression a Quarter of the Time.
September 2026A benchmark suite pins the seeds that build its circuits and passes no seed to the code that compiles them. The real change between two compiler versions is +5.37% [+4.27, +6.50], yet the suite’s own three-run protocol calls it a regression of 10% or more 24.4% of the time. A pre-registered replication over 39 circuits, four claims withdrawn on the record, and six defects an independent audit found in the measuring apparatus itself.
Six Things Photopea's Scripting API Does That Nobody Wrote Down
August 2026Six undocumented behaviours in a scripting API, each measured with the workaround for it. If your automation reports success and changes nothing, one of these is probably why.
Physics reproductions
Two results with a known answer, rebuilt from nothing to find out what the rebuild would get wrong.
Ten Tests, Written Before the Code. The One That Failed Taught Me the Most
August 2026A finite element solver written from scratch and pointed at an answer known since 1898. It landed on 3.00002. Ten tests were written before the code; the one that failed taught the most, and a number the internet repeats confidently failed to reproduce.
Geometric Optics Lies: Reproducing a 1976 Black-Hole Calculation from Scratch
August 2026A 1976 black-hole emission calculation reproduced from scratch. The standard geometric-optics shortcut overstates photon emission by 4.16x. Six validation gates, two analytic limits the code was never tuned to, and one number retracted.