Pivotal MRMC Trial of AI Software Detecting Musculoskeletal Pathology

Challenge

An emerging AI company developing software to detect musculoskeletal pathology on radiographs required a pivotal clinical trial to support FDA and European submissions. Musculoskeletal findings such as fractures are often subtle, multiple and anatomically distributed, so the study had to score detection at the finding level with localization, not simply at the image level, and the reference standard had to establish the location of every true finding.

Readers in the intended-use population range from radiologists to emergency and orthopedic physicians, and the sponsor needed the reader panel to reflect that range while keeping the MRMC design statistically clean and the reading sessions free of recall bias.

Intrinsic Imaging’s Solution

Intrinsic Imaging assembled the case set across anatomical regions and finding types with a reference standard established by musculoskeletal radiologists using follow-up imaging and clinical records, recording the location of every true finding so that reader marks could be scored for localization accuracy under a predefined matching rule.

A reader panel representing the intended-use population was sourced from the company's reviewer network, trained on the marking tool and the matching rule, and read the full case set with and without AI output in a fully crossed, randomized design with a washout period between sessions. The reading platform captured mark coordinates, confidence scores and reading time per case.

Quality control monitored marking completeness and session compliance, the matching algorithm was validated before scoring, and the dataset was delivered with per-finding localization scoring for the sponsor's MRMC analysis.

Intrinsic Imaging’s Results

• Approximately 1,000 musculoskeletal radiographic cases scored at the finding level with localization under a validated matching rule
• Reader panel of 20 spanning radiology, emergency medicine and orthopedics completed roughly 40,000 case interpretations with and without AI
• Reference standard with documented location for every true finding, supporting localization-based figure-of-merit analysis
• Full regulatory documentation delivered under an ISO 13485 certified quality system for FDA and European submissions