Trustworthiness of Artificial Intelligence Models in Radiology and the Role of Explainability
Journal of the American College of Radiology - Volume 18, Number 8 - https://doi.org/10.1016/j.jacr.2021.02.008
Summary
Trust is the cornerstone of relationships in the medical profession. To provide the best course of action for each patient, medical professionals must rely on a web of trust that involves numerous players, from pharmaceutical companies to laboratory technicians to breakthrough results published in leading scientific journals, to name but a few.
Keywords:
Cornerstone | Trustworthiness | Action (physics) | Medical profession | Computer science | Medical education | Psychology | Knowledge management | Medicine | Internet privacy
References
1. Submit
2. Log in ACR Member Log in Non-Member Log in Get Institutional Access
3. ACR Member Log in
4. Non-Member Log in
5. Get Institutional Access
6. Register
7. open links dialog close links dialog Submit ACR Member Log in Non-Member Log in Get Institutional Access Register
8. Articles Download full issue Download started. Ok Latest Articles in Press Current issue Past Issues Popular Articles Original article: Health services research and policy ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee Tessler et al. Original Article: Leadership ACR Lung-RADS v2022: Assessment Categories and Management Recommendations Christensen et al. Original article: Clinical practice management Management of Incidental Adrenal Masses: A White Paper of the ACR Incidental Findings Committee Mayo-Smith et al. Latest Articles ACR Appropriateness Criteria® ACR Appropriateness Criteria® Myelopathy: 2026 Update Expert Panel on Neurological Imaging ACR Appropriateness Criteria ACR Appropriateness Criteria® Chronic Knee Pain: Update 2026 Expert Panel on Musculoskeletal Imaging Research Article Supply Chain Vulnerabilities in Breast Imaging: Site- and Network-Level Strategies for a Concentrated Consumable Market Kommula et al.
9. Download full issue Download started. Ok
10. Latest Articles in Press Current issue Past Issues
11. Articles in Press
12. Current issue
13. Past Issues
14. Popular Articles Original article: Health services research and policy ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee Tessler et al. Original Article: Leadership ACR Lung-RADS v2022: Assessment Categories and Management Recommendations Christensen et al. Original article: Clinical practice management Management of Incidental Adrenal Masses: A White Paper of the ACR Incidental Findings Committee Mayo-Smith et al.
15. Original article: Health services research and policy ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee Tessler et al.
16. Original Article: Leadership ACR Lung-RADS v2022: Assessment Categories and Management Recommendations Christensen et al.
17. Original article: Clinical practice management Management of Incidental Adrenal Masses: A White Paper of the ACR Incidental Findings Committee Mayo-Smith et al.
18. Latest Articles ACR Appropriateness Criteria® ACR Appropriateness Criteria® Myelopathy: 2026 Update Expert Panel on Neurological Imaging ACR Appropriateness Criteria ACR Appropriateness Criteria® Chronic Knee Pain: Update 2026 Expert Panel on Musculoskeletal Imaging Research Article Supply Chain Vulnerabilities in Breast Imaging: Site- and Network-Level Strategies for a Concentrated Consumable Market Kommula et al.
19. ACR Appropriateness Criteria® ACR Appropriateness Criteria® Myelopathy: 2026 Update Expert Panel on Neurological Imaging
20. ACR Appropriateness Criteria ACR Appropriateness Criteria® Chronic Knee Pain: Update 2026 Expert Panel on Musculoskeletal Imaging
21. Research Article Supply Chain Vulnerabilities in Breast Imaging: Site- and Network-Level Strategies for a Concentrated Consumable Market Kommula et al.
22. Publish For Authors Submit Article External Link Guide for Authors Aims & Scope Open Access Information Call for Papers Researcher Academy External Link Name Change Policy Why JACR? For Reviewers Reviewing for JACR
23. For Authors Submit Article External Link Guide for Authors Aims & Scope Open Access Information Call for Papers Researcher Academy External Link Name Change Policy Why JACR?
24. Submit Article External Link
25. Guide for Authors
26. Aims & Scope
27. Open Access Information
28. Call for Papers
29. Researcher Academy External Link
30. Name Change Policy
31. Why JACR?
32. For Reviewers Reviewing for JACR
33. Reviewing for JACR
34. Topics Appropriateness Criteria (AC) Current AC Issue AC and Patient Summaries AC Archive Virtual Collections JACR Leadership JACR Online CME
35. Appropriateness Criteria (AC) Current AC Issue AC and Patient Summaries AC Archive
36. Current AC Issue
37. AC and Patient Summaries
38. AC Archive
39. Virtual Collections JACR Leadership JACR Online CME
40. JACR Leadership
41. JACR Online CME
42. Multimedia JACR Video Library Visual Abstracts
43. JACR Video Library Visual Abstracts
44. JACR Video Library
45. Visual Abstracts
46. About Society About the ACR External Link Editorial Board Journal Information Aims & Scope Permissions Reprints External Link Access Subscribe Activate Online Access
47. Society About the ACR External Link Editorial Board
48. About the ACR External Link
49. Editorial Board
50. Journal Information Aims & Scope Permissions Reprints External Link
51. Aims & Scope
52. Permissions
53. Reprints External Link
54. Access Subscribe Activate Online Access
55. Subscribe
56. Activate Online Access
57. Contact Contact Contact Us Advertise with Us External Link Go to Product Catalog External Link Follow Us New Content Alerts X External Link Facebook External Link Instagram External Link YouTube External Link LinkedIn External Link
58. Contact Contact Us Advertise with Us External Link Go to Product Catalog External Link
59. Contact Us
60. Advertise with Us External Link
61. Go to Product Catalog External Link
62. Follow Us New Content Alerts X External Link Facebook External Link Instagram External Link YouTube External Link LinkedIn External Link
63. New Content Alerts
64. X External Link
65. Facebook External Link
66. Instagram External Link
67. YouTube External Link
68. LinkedIn External Link
69. Subscribe Direct Link
70. Email
71. X
72. Facebook
73. LinkedIn
74. Sina Weibo
75. Add to Mendeley
76. bluesky
77. Email
78. X
79. Facebook
80. LinkedIn
81. Sina Weibo
82. Add to Mendeley
83. Bluesky
84. Download Hi-res image
85. Download .PPT
86. Home
87. Access for Developing Countries
88. About the JACR
89. Subscription Options
90. Permissions
91. Reprints
92. Contact Information
93. Editorial Board
94. Call for Papers
95. Instructions for Authors
96. Name Change Policy
97. Submit a Manuscript
98. About Open Access
99. Researcher Academy
100. Why JACR?
101. For Reviewers
102. For Advertisers
103. About the ACR
104. Current Issue
105. Articles Online First
106. Visual Abstracts
107. List of Issues
108. Contrast Media Shortage
109. CME
110. Columns
111. Current AC Issue
112. AC and Patient Summaries
113. AC Archive
114. Sign up for JACR alerts
115. Submit a Manuscript
116. X
117. Instagram
118. Facebook
119. LinkedIn
120. YouTube
121. Existing Subscriber: Log in
122. New Subscriber: Claim access with activation code. New subscribers select Claim to enter your activation code.
123. Academic & Personal 24-hour online access. Suitable for academic or non-commercial use only. $27.95 Academic access
124. Corporate R&D Professionals 24-hour online access. Suitable for commercial use. $37.95 Corporate access
125. Full online access to your subscription and archive of back issues
126. Table of Contents alerts
127. Access to all multimedia content, e.g. podcasts, videos, slides
Other resources
The RSNA Pediatric Bone Age Machine Learning Challenge
Purpose The Radiological Society of North America (RSNA) Pediatric Bone Age Machine Learning Challenge was created to show an application of machine learning (ML) and artificial intelligence (AI) in medical imaging, promote collaboration to catalyze AI model creation, and identify innovators in medical imaging. Materials and Methods The goal of this challenge was to solicit individuals and teams to create an algorithm or model using ML techniques that would accurately determine skeletal age in a curated data set of pediatric hand radiographs. The primary evaluation measure was the mean absolute distance (MAD) in months, which was calculated as the mean of the absolute values of the difference between the model estimates and those of the reference standard, bone age. Results A data set consisting of 14 236 hand radiographs (12 611 training set, 1425 validation set, 200 test set) was made available to registered challenge participants. A total of 260 individuals or teams registered on the Challenge website. A total of 105 submissions were uploaded from 48 unique users during the training, validation, and test phases. Almost all methods used deep neural network techniques based on one or more convolutional neural networks (CNNs). The best five results based on MAD were 4.2, 4.4, 4.4, 4.5, and 4.5 months, respectively. Conclusion The RSNA Pediatric Bone Age Machine Learning Challenge showed how a coordinated approach to solving a medical imaging problem can be successfully conducted. Future ML challenges will catalyze collaboration and development of ML tools and methods that can potentially improve diagnostic accuracy and patient care. © RSNA, 2018 Online supplemental material is available for this article. See also the editorial by Siegel in this issue.
The RSNA International COVID-19 Open Radiology Database (RICORD)
The coronavirus disease 2019 (COVID-19) pandemic is a global health care emergency. Although reverse-transcription polymerase chain reaction testing is the reference standard method to identify patients with COVID-19 infection, chest radiography and CT play a vital role in the detection and management of these patients. Prediction models for COVID-19 imaging are rapidly being developed to support medical decision making. However, inadequate availability of a diverse annotated data set has limited the performance and generalizability of existing models. To address this unmet need, the RSNA and Society of Thoracic Radiology collaborated to develop the RSNA International COVID-19 Open Radiology Database (RICORD). This database is the first multi-institutional, multinational, expert-annotated COVID-19 imaging data set. It is made freely available to the machine learning community as a research and educational resource for COVID-19 chest imaging. Pixel-level volumetric segmentation with clinical annotations was performed by thoracic radiology subspecialists for all COVID-19–positive thoracic CT scans. The labeling schema was coordinated with other international consensus panels and COVID-19 data annotation efforts, the European Society of Medical Imaging Informatics, the American College of Radiology, and the American Association of Physicists in Medicine. Study-level COVID-19 classification labels for chest radiographs were annotated by three radiologists, with majority vote adjudication by board-certified radiologists. RICORD consists of 240 thoracic CT scans and 1000 chest radiographs contributed from four international sites. It is anticipated that RICORD will ideally lead to prediction models that can demonstrate sustained performance across populations and health care systems. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Bai and Thomasian in this issue.
