ajay6601/clinicalbench

ClinicalBench helps you figure out whether you can trust AI to automatically catch dangerous mistakes in medical AI outputs, without a human reviewer checking every response. If you're building a clinical AI product and want to use an AI "judge" to approve or block answers before they reach patients, this project tests how reliable that judge actually is. The framework runs medical questions through several AI models, then has automated judges score each answer on four dimensions: factuality, safety, hallucination, and completeness. It validates those judge scores against known-correct answers to see whether the judges give high marks to good answers and low marks to bad ones. You can also run a second judge model as an independent reference to check agreement, and inject fake medical claims into correct answers to see if the judge catches them. A healthcare startup founder or product manager building a clinical AI feature would use this to set up automated quality gates, rules that block unsafe outputs or flag borderline ones for human review. For example, you might configure stricter thresholds for emergency medicine responses than for general health Q&A. The project includes calibrated thresholds based on real testing, so you know what tradeoff you're making between catching errors and false alarms. The results are surprisingly encouraging: the factuality judge catches nearly all wrong answers while rarely flagging correct ones, and a cheap model works almost as well as an expensive one. The hallucination judge is the weakest link, the author recommends running two models and taking the stricter score to improve it. The whole evaluation pipeline costs about $10-14 in API calls. The author openly notes the limitations: the test data is clean multiple-choice questions rather than messy real-world clinical text, and there are no physician-reviewed annotations. The framework and thresholds transfer, but you'd want to retest with data that matches your actual deployment context before trusting it in production.