RSNA Margulis Award Honors AI Research in MSK Imaging
Algorithm helps researchers reduce missed fractures on X-ray by 29%
The 2022 RSNA Alexander R. Margulis Award for Scientific Excellence will be presented to Ali Guermazi, MD, PhD, MSc, for the Radiology article, “Improving Radiographic Fracture Recognition Performance and Efficiency Using Artificial Intelligence.”
“This year’s Margulis Award recognizes the increasing importance of artificial intelligence in our field. The authors studied fracture detection by 24 radiologists and clinicians with and without AI,” said Radiology Editor David A. Bluemke, MD, PhD. “Ten percent better fracture detection was present using AI, while reducing time for radiologists. This study validates the steady rise in the use of AI tools that are becoming a routine part of many clinical practices, particularly in musculoskeletal radiology.”
Study Compares Reader Performance With and Without AI Assistance
“AI can be a powerful tool to help radiologists and other physicians improve diagnostic performance and increase efficiency, while potentially improving patient experience at the time of hospital or clinic visit,” said Dr. Guermazi, director of the Quantitative Imaging Center, professor of radiology and medicine and assistant dean in the office of diversity at Boston University Chobanian & Avedisian School of Medicine and chief of radiology at VA Boston Healthcare System.
In the study, Dr. Guermazi and colleagues retrospectively analyzed 480 radiographic examinations, from various U.S. hospitals, that included adults over 21 years of age with indications of trauma and fracture prevalence of 50%.
The researchers included radiographs of limbs, pelvis, spine, and rib cage. There were 240 patients with a total of 350 fractures, and 240 patients with no fractures.
The studies were analyzed twice by 24 U.S. board-certified readers from six different specialties including radiology, orthopedic surgery, rheumatology, emergency medicine (including physicians and physician assistants), and family medicine.
According to Dr. Guermazi, readings were performed both with and without a commercially developed software utilizing an algorithm trained on accurately annotated X-ray images from multiple institutions, acquired on a large variety of systems. Readers had a one-month washout period between the two analyses.
Stand-alone artificial intelligence (AI) performance examples: false-positive and false-negative radiographs. (A) Radiograph shows a small corticated ossific fragment adjacent to inferior glenoid margin (arrow), likely sequela of prior trauma (chronic fracture) or calcified detached inferior labrum rather than acute fracture. AI noted this as an acute fracture using the DOUBT-FRACT threshold. Fifteen readers read this as acute fracture without AI. Four readers thought the fracture was chronic without using AI, but reversed their reading with AI. Only two radiologists, one rheumatologist, and two family medicine physicians recognized the chronicity of the fracture with and without AI. (B) Radiograph shows a subtle nondisplaced fracture of the fifth metacarpal base (arrow), which was not detected by AI. All readers missed this fracture with and without AI. Only ground truth readers noted the fracture. This fracture was only appreciable on the anteroposterior view shown here and was not clearly visible on (C) the oblique view or the lateral view (not shown) of the right hand. There were two predefined thresholds for fracture detection: high-sensitivity threshold named DOUBT-FRACT, equal to 50% after transformation, and high-specificity threshold named FRACT, equal to 90% after transformation.
https://doi.org/10.1148/radiol.210937 © RSNA 2022
Algorithm Shows Unexpected Gain in Specificity
“The results of the study showed an absolute gain in sensitivity in the detection of fractures of 10.4% with the help of the software, with the software showing a sensitivity of 75.2% against 64.8% without the assistance of the software,” Dr. Guermazi said. The results also revealed an absolute gain in specificity—from 90.6% to 95.6%—for fracture detection with software assistance.
While not surprised by the algorithm’s sensitivity, Dr. Guermazi did not expect the gain in specificity.
“Computer-Aided Detection systems can be easily sensitive but usually bring significant loss in specificity. Here, the algorithm also helped reduce false positive rates,” he said. “The time saving was a good surprise given that the algorithm brings additional information to look at on top of the native images. In reality, it was not obvious that the algorithm would speed up interpretation time.”
Dr. Guermazi noted that one of the most important challenges the team faced during the study was training 24 readers with various backgrounds on reading with AI. Despite this challenge, he said the readers thought the use of the AI algorithm was easy, friendly and extremely intuitive.
Clinical validation studies are ongoing with a focus on specific body parts where, according to Dr. Guermazi, the gold standard is established using CT/MRI to evaluate the algorithm’s ability to detect radiographically visible and occult lesions.
“Ultimately, I believe my colleague radiologists will join in viewing AI as a friend rather than a foe,” Dr. Guermazi said. “As it becomes clearer that it can beat the human eye at certain specific and repetitive or tedious tasks, AI will be viewed as a great add-on to heavy clinical workflow.”
Named for Alexander R. Margulis, MD, a distinguished investigator and inspiring visionary in the science of radiology, this annual award recognizes the best original scientific article published in Radiology. The Margulis Award will be presented during the RSNA 108th Scientific Assembly and Annual Meeting (RSNA 2022) in Chicago, Nov. 27-Dec. 1.
For More Information
Access the Radiology study, “Improving Radiographic Fracture Recognition Performance and Efficiency Using Artificial Intelligence.”
Read the related Radiology editorial, “Using AI to Improve Fracture Detection.”
Read about the 2021 Margulis Award recipient in RSNA News, Mammography Research Receives RSNA Margulis Award.