ROC-AUC at 0.97, Precision at 9%: The Base-Rate Math
Work through a 1%-positive detector, turn thresholds into ROC and precision-recall points, and choose an operating point for a 30-alert review budget.
Tag
#model-evaluation
9articleswith this tag.
GPT-5.6 Terra's 82% Kiro Cost Reduction Needs a Larger Denominator
OpenAI and AWS report that GPT-5.6 Terra completed successful Terminal-Bench tasks in Kiro at roughly 82% lower cost, but repository work adds review and repair.
Anthropic's August Risk Report Is a Disclosure, Not a Safety Certificate
Read Anthropic's August 2026 Risk Report by separating its coverage date, company ratings, disclosed failures, redactions, and external review.
Visual Studio BYOM Connects Models Easily. Agent Reliability Still Needs Testing
Visual Studio 18.10 Insiders can connect local and hosted models to Agent mode, but one hands-on test shows that endpoint availability does not imply reliable coding work.
Thomson Reuters Built a Legal AI Model. What Does It Prove?
Thomson Reuters released a proprietary model for CoCounsel and open weights for a smaller version. Here is what buyers can verify—and what remains a vendor claim.
Three Million Hugging Face Models Make Popularity a Poor Selector
Hugging Face has passed three million models, but likes and downloads reveal attention rather than license fit, provenance, hardware cost, or task quality.
Gemini 3.7 Flash Doubles in Price, but Higher Thinking Can Still Cost Less
Gemini 3.7 Flash prices rise in January 2027, while the cheapest thinking level still depends on acceptance, retries, latency, and fallback cost.
DeepSeek's V4 Flash Vision API Is Useful—and Still Experimental
DeepSeek has added image input to its fast API line, but the preview does not yet establish reliable performance across screenshots, charts, and documents.
Grok 4.6's Low Token Price Does Not Make It the Cheapest Model
Grok 4.6 combines low headline pricing with strong reported benchmarks, but retries, failed tasks, latency, and tool charges can reverse the apparent advantage.