多阅片者多病例设计在人工智能辅助阅片影像诊断试验评价中的应用
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

上海市卫生健康委员会新兴交叉领域研究专项(2022JC011).


Application of multi-reader multi-case design in evaluating artificial intelligence-assisted imaging diagnostic trials
Author:
Affiliation:

Fund Project:

Supported by Emerging Interdisciplinary Research Project of Shanghai Municipal Health Commission (2022JC011).

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    目的 使用多阅片者多病例(MRMC)设计评价人工智能(AI)辅助阅片影像诊断试验的临床效能,以期为影像诊断试验的临床评价提供科学依据。方法 采用影像诊断试验中广泛应用的MRMC设计,详细阐述了MRMC设计中Obuchowski-Rockette(OR)法的模型构建及其检验方法。实例研究共收集了3家医院200例受试者的CT影像资料,其中133例为肋骨骨折患者,68例为非肋骨骨折患者,由3位阅片医师对所有CT影像进行判读。分析在2种阅片方式(医师+AI辅助阅片、医师独立阅片)下肋骨骨折检出的AUC值、灵敏度和特异度的差异。结果 AI辅助阅片组的AUC值为0.958,医师独立阅片组的AUC值为0.902,两组AUC值差异有统计学意义(P<0.001)。AI辅助阅片组总体的灵敏度为0.970,特异度为0.946;医师独立阅片组的灵敏度为0.838,特异度为0.966;两组灵敏度差值为0.131(95% CI 0.091~0.171),特异度差值为-0.020(95% CI -0.059~0.020),说明AI辅助阅片与医师独立阅片的灵敏度差异有统计学意义而特异度差异无统计学意义。两组的阳性似然比均大于10,阴性似然比均小于0.2,阳性预测值都接近1,说明AI辅助阅片影像诊断试验的诊断准确性高。结论 AI辅助阅片在提高诊断效能方面有显著优势,不仅可以提高肋骨骨折诊断的准确性和检出率,还能提高医师工作效率,优化医院服务。

    Abstract:

    Objective To evaluate the clinical efficacy of artificial intelligence (AI)-assisted imaging diagnostic trials using multi-reader multi-case (MRMC) design, so as to provide a scientific basis for clinical evaluation of imaging diagnostic trials. Methods The MRMC design, widely used in imaging diagnostic trials, was adopted in this study. The Obuchowski-Rockette (OR) method of MRMC design was detailed, including model construction and test methods. A case study was conducted, collecting imaging data of 200 subjects from 3 hospitals, with 133 cases of rib fractures and 68 cases of non-rib fractures. Three radiologists reviewed all CT images of the subjects. The area under curve (AUC) value, sensitivity and specificity in detecting rib fractures between 2 reading modalities (radiologists with AI assistance vs radiologists reading independently) were compared. Results The AI-assisted reading group had an AUC value of 0.958, while the radiologist-independent reading group had an AUC value of 0.902, showing a significant difference (P<0.001). The overall sensitivity and specificity of the AI-assisted reading group were 0.970 and 0.946, respectively; while the sensitivity and specificity of the radiologist-independent reading group were 0.838 and 0.966, respectively. The difference of sensitivity between groups was 0.131 (95% confidence interval [CL] 0.091-0.171), and the difference of specificity was -0.020 (95% CI -0.059-0.020), indicating a significant difference in sensitivity but not in specificity between AI-assisted and radiologist-independent reading groups. Both groups had positive likelihood ratios (+LR) greater than 10 and negative likelihood ratios (-LR) less than 0.2, with positive predictive values approaching 1, suggesting that the diagnostic accuracy of the AI-assisted imaging diagnostic trials was high. Conclusion The AI-assisted reading method demonstrates a significant advantage in enhancing diagnostic efficiency, not only improving the diagnostic accuracy and detection rate of rib fractures, but also improving the work efficiency of radiologists and optimizing hospital services.

    参考文献
    相似文献
    引证文献
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-11-14
  • 最后修改日期:2024-12-26
  • 录用日期:
  • 在线发布日期: 2025-04-16
  • 出版日期:
文章二维码
重要通知
友情提醒: 近日发现论文正式见刊或网络首发后,有人冒充我刊编辑部名义给作者发邮件,要求添加微信,此系诈骗行为!可致电编辑部核实:021-81870792。
            《海军军医大学学报》编辑部
关闭