BedVibe Studios — Engineering

Splits structural audio defects from pronunciation errors, canonicalizes both expected text and ASR transcript to spoken form for equivalence-aware WER/CER, and quarantines ASR-uncertain short utterances for human review instead of guessing.

• Built-in benchmark corpus: 817 curated cases across 39 categories, versioned independently of the software
• Equivalence-aware WER/CER with diacritic folding (Reykjavik == Reykjavík)
• ASR-uncertainty quarantine — sets aside cases too short to auto-judge instead of guessing
• One-command engine benchmarking, self-contained HTML reports, and a CI regression gate
• Backed by a citable 390-sample study (DOI 10.5281/zenodo.20757553, CC-BY-4.0)

Deterministic PASS/WARN/FAIL verdicts across a training run's whole life — before it starts (dataset, tokenizer, and an environment preflight that catches a stack which will not import, an unreadable checkpoint, or too little system RAM), a live guardian during training, a post-run autopsy after, and a baseline compare. Reads five log formats including TensorBoard event files, decoded from the wire format with no torch, tensorflow or tensorboard installed. No ML judging ML: every verdict is a rule that fires or doesn't, every finding cites its exact numbers, and a check that could not run is reported as NOT-CHECKED rather than as a pass.

• trainproof doctor . — zero-config autopsy: discovers every training log in a directory, judges them, prints a triage-sorted summary
• Live guardian: a one-line Hugging Face callback; opt-in stop_on_fail aborted a diverging Qwen2.5-3B QLoRA at step 20 of 300 in a real demo
• Preflight: lints dataset + tokenizer before a single GPU-second (malformed JSONL, empty rows, duplicates, missing eos/pad, over-length)
• compare catches shuffled-labels corruption that single-run loss rules cannot — the relative loss floor sits ~6× a known-good baseline
• Stable rule IDs (TP-DIVERGE, TP-OVERFIT, TP-DEAD-RUN, …) documented in RULES.md, plus --json output built for CI and AI coding agents
• CONTRACTS.md pins the promises — exit codes, JSON schema policy, rule-ID stability and a verdict-stability guarantee; SARIF 2.1.0 output turns findings into GitHub PR annotations
• A PASS names every check that ran and every check it skipped, with the reason — a skipped check is never reported as a passed one
• Grounded in an 18-run fault-injection study (Qwen2.5-3B QLoRA, RTX 5080) — six configurations at three seeds each, every real log shipped in-repo — plus a 9.8-hour Coqui XTTS fine-tune that diverged on its own, the first shipped failure nobody injected
• Reads five log formats including TensorBoard event files, decoded from the wire format with no tensorflow, tensorboard, protobuf, torch or numpy — validated byte-exact against tensorboard's own EventAccumulator, and the rules are exercised on real runs from three frameworks: HuggingFace, Coqui XTTS and PyTorch Lightning