SubQ Snap, a hosted decision model, got every tool choice right on a 250-case tool-selection test and cost a fraction of a cent. On the same 100 hand-graded answers we use for every grader, it agreed with the human labels...
ReadWe ran OpenAI's new Decisions API through the same 100 hand-graded answers we used to test Jev and its open-source clones. It agreed with the human labels 83% of the time, against 90% Jev and 89% for JevK5-4B. Median response...
ReadWe compared six open decision models against a Jev run on 100 hand-graded answers. JevK5-4B agreed with the human labels 89% of the time against Jev's 90%, with a median response time of about 162 ms on a Mac. It...
ReadHow we put a cheap first-pass grader in front of Claude Opus, cut answer-grading cost per eval from about $43.60 to about $11, and checked that the verdicts still agree with Opus about 99 times in 100.
ReadNotes from testing TypeSafe's Jev as a replacement for a Claude-based answer grader. It matched Claude without source text, did worse with it, and its confidence score turned out to be the useful part.
ReadHow differences in starting accuracy made a training-data effect look convincing, and what retention and recovery revealed.
ReadOur GPU training image took thirty minutes to build on every merge, docs-only merges included. Claude Code traced 24 of those minutes to one chown and one missing cache flag. Code-only builds now finish in about four.
ReadUpgrading our eval grader model moved recall 6.4 points on identical responses. Claude Sonnet/Opus 4.5 -> 5 led to significant gains without changing anything else, including the prompt.
Read