Intrinsic Imaging
Intrinsic Imaging

Pivotal MRMC Trial of AI Software Detecting Musculoskeletal Pathology

Case Study · AI/CAD

  • Musculoskeletal radiographs
  • Multi-reader multi-case
  • Lesion-level detection with localization
Download the PDF

Challenge

Intrinsic Imaging supported this study’s imaging activities, addressing the scientific and operational considerations that inform the work of an imaging CRO and imaging core lab. An emerging AI company developing software to detect musculoskeletal pathology on radiographs required a pivotal clinical trial to support FDA and European submissions. Musculoskeletal findings such as fractures are often subtle, multiple and anatomically distributed, so the study had to score detection at the finding level with localization, not simply at the image level, and the reference standard had to establish the location of every true finding.

Readers in the intended-use population range from radiologists to emergency and orthopedic physicians, and the sponsor needed the reader panel to reflect that range while keeping the MRMC design statistically clean and the reading sessions free of recall bias. An image containing several abnormalities can be classified correctly overall while clinically relevant findings remain undetected. Localization therefore matters to the interpretation of assistance: a reader must identify the relevant anatomical finding, not simply indicate that an examination is abnormal, for a detection claim to be meaningfully assessed.

Intrinsic Imaging’s Solution

Intrinsic Imaging assembled the case set across anatomical regions and finding types with a reference standard established by musculoskeletal radiologists using follow-up imaging and clinical records, recording the location of every true finding so that reader marks could be scored for localization accuracy under a predefined matching rule. Recording reference locations connects the clinical definition of a true abnormality to the spatial rules used in scoring. Follow-up imaging and clinical records provide supporting context, although the resulting reference standard still depends on the completeness of available evidence and expert interpretation of anatomically complex or subtle findings.

A reader panel representing the intended-use population was sourced from the company's reviewer network, trained on the marking tool and the matching rule, and read the full case set with and without AI output in a fully crossed, randomized design with a washout period between sessions. The reading platform captured mark coordinates, confidence scores and reading time per case. Including different clinical specialties addresses the range of intended users without assuming that their baseline performance or response to assistance is identical. Confidence and timing observations add context to localization results, but changes in these measures should not be equated automatically with greater diagnostic accuracy or improved patient management.

Quality control monitored marking completeness and session compliance, the matching algorithm was validated before scoring, and the dataset was delivered with per-finding localization scoring for the sponsor's MRMC analysis. A predefined matching rule limits discretionary decisions about whether a reader mark corresponds to a reference finding. Its tolerance remains part of the endpoint definition: overly permissive matching can credit imprecise marks, whereas overly restrictive matching can penalize clinically reasonable localization, making the documented rule essential to interpretation.

Intrinsic Imaging’s Results

  • Approximately 1,000 musculoskeletal radiographic cases scored at the finding level with localization under a validated matching rule
  • Reader panel of 20 spanning radiology, emergency medicine and orthopedics completed roughly 40,000 case interpretations with and without AI
  • Reference standard with documented location for every true finding, supporting localization-based figure-of-merit analysis
  • Full regulatory documentation delivered under an ISO 13485 certified quality system for FDA and European submissions
View all case studies

About Intrinsic Imaging, LLC

Intrinsic Imaging is an ISO certified medical imaging CRO and imaging core lab providing comprehensive services in support of Phase I-IV clinical trials and Class I, II, III medical device trials. Intrinsic Imaging is also a world leader in conducting Artificial Intelligence studies.

With its team of more than 650 board-certified and fellowship-trained physicians, supporting over 4000 imaging centers in 64 countries, Intrinsic Imaging’s team has sub-specialization in all therapeutic areas, including Cardiovascular & Thoracic, Central Nervous System, Gastroenterology & Metabolic, Musculoskeletal, Oncology and Ophthalmology.

Certified by the British Standards Institution (BSI), Intrinsic Imaging is the only company in the world to hold five ISO certifications specifically for providing imaging core lab services on clinical trials. For more information, please visit IntrinsicImaging.com.

Contact: