Ground Truth for a Breast Cancer Detection AI

Challenge

An AI developer required a large, independent test dataset to support a standalone performance study of a breast cancer detection algorithm. FDA expectations for such studies are exacting: the reference standard must be established independently of the algorithm, truth determinations must be defensible at the lesion level, and the dataset must remain scrupulously sequestered from any use in algorithm training. Assembling thousands of cases meeting these criteria demanded both scale and methodological discipline.

Our Solution

Intrinsic Imaging engaged fellowship-trained breast radiologists to perform lesion-level annotation of digital mammography and tomosynthesis examinations within our Part 11 compliant reading environment. Pathology correlation guided every reference-standard determination, and expert consensus resolved disagreements, with each decision documented in a complete audit trail. The resulting dataset was maintained under strict sequestration from the algorithm's development, preserving its independence as a test set.

Results

• Reference standards established for 5000+ cases with complete audit trails
• Dataset sequestered from algorithm training
• Standalone performance study completed on schedule