global

Variáveis

Utilitários

ESTILOS PERSONALIZADOS

Best Practices for the Safe Use of Large Language Models and Other Generative AI in Radiology

Authors:

Paul H. Yi; Hana L. Haver; Jean Jeudy; Woojin Kim; Felipe Kitamura; Eniola Oluyemi; Andrew D. Smith; Linda Moy; Vishwa S. Parekh

Publication

Radiology - Volume 316, Number 3 - https://doi.org/10.1148/radiol.241516

Download PDF

Summary

As large language models (LLMs) and other generative artificial intelligence (AI) models are rapidly integrated into radiology workflows, unique pitfalls threatening their safe use have emerged. Problems with AI are often identified only after public release, highlighting the need for preventive measures to mitigate negative impacts and ensure safe, effective deployment into clinical settings. This article summarizes best practices for the safe use of LLMs and other generative AI models in radiology, focusing on three key areas that can lead to pitfalls if overlooked: regulatory issues, data privacy, and bias. To address these areas and minimize risk to patients, radiologists must examine all potential failure modes and ensure vendor transparency. These best practices are based on the best available evidence and the experiences of leaders in the field. Ultimately, this article provides actionable guidelines for radiologists, radiology departments, and vendors using and integrating generative AI into radiology workflows, offering a framework to prevent these problems. © RSNA, 2025

Key points

Summary Best practices for the safe use of large language models and other generative artificial intelligence in radiology require vendor transparency and addressing potential pitfalls in three key areas: regulation, data privacy, and bias. Essentials
Best practices for the safe use of large language models and other generative artificial intelligence (AI) in radiology focus on addressing potential pitfalls in three key areas: regulatory issues, data privacy, and bias.
Evolving regulatory frameworks require examining potential failure modes of generative AI beyond statistical metrics: similarity between synthetic and real-world data, output reproducibility, robustness against hallucinations, and the effects of human-computer interaction.
Data privacy risks of generative AI tools in radiology can be mitigated through transparent discussions about the origin of the training data and how vendors will use the data resulting from clinical use.
Generative AI biases from homogeneous training data that exclude historically underserved populations may lead to performance variations among demographic groups, perpetuating harmful stereotypes and misinformation.
Vendor transparency is a key practical way for radiologists to identify potential data privacy issues and mitigate the use of biased generative AI tools in clinical practice.

Introduction

Since the release of OpenAI’s ChatGPT in November 2022 ¹, large language models (LLMs) and generative artificial intelligence (AI) have transformed the technology landscape. In medicine and radiology, LLMs have generated excitement for their potential to automate complex, labor-intensive tasks ²^–⁵ ( Fig 1 ). Early examples of LLMs in radiology include answering patient questions ⁶^–⁹, translating radiology reports into plain language ¹⁰^,¹¹, and extracting data from radiology reports ¹²^,¹³. Generative AI models have been explored for creating hyperrealistic medical images to replace traditional image processing techniques, such as cinematic rendering ¹⁴. One study evaluated a generative AI tool that produced radiology reports from chest radiograph images and found that the AI-generated reports were similar in accuracy and quality to radiologist reports while providing higher quality than teleradiology preliminary reports ¹⁵. A randomized trial also demonstrated that LLMs can improve physician diagnostic reasoning ¹⁶, highlighting their potential to improve clinical care.

Figure 1: Diagram shows potential use cases of large language models (LLMs) and other generative artificial intelligence (GenAI) models in radiology. Potential use cases include answering patient questions, patient-friendly radiology report generation, data extraction, image processing, radiology report generation (geared toward other physicians), and synthetic data generation. All visual elements used are under appropriate use agreements. “Patient-Friendly Radiology Reports,” “Data Extraction,” and “Radiology Report Generation” visuals are used under a Creative Commons license (CC0 1.0 DEED) from rawpixel.com , rawpixel.com , and openclipart.org , respectively; “Image Processing Techniques” visual is reprinted, with permission, from reference 14 ; “Synthetic Data Generation” visual is used under a Creative Commons license (CC BY-SA 3.0 DEED [ https://creativecommons.org/licenses/by/3.0/us/deed.en ] ) from Wikimedia.org .

Despite this excitement, LLMs have pitfalls that threaten their safe use in radiology, including performance drift ¹⁷ and bias ¹⁸^,¹⁹, which align with previous concerns regarding earlier AI applications in medicine ²⁰^–²⁴. Unique problems associated with LLMs have emerged, such as hallucinations, where LLMs generate convincing but factually incorrect text or replies ⁷^,²⁵. As generative AI models become integrated into clinical practice ²⁶^–²⁸, radiologists and vendors must remain aware of the potential drawbacks of these technologies to prevent unintended consequences in real-world clinical practice.

In response to the rapid evolution of generative AI models and the unique needs of radiologists, this article describes best practices and pitfalls in using generative AI models in radiology in three key areas: regulatory issues, data privacy, and bias. Although recent articles have provided broad overviews of LLMs and generative AI in radiology ²⁹^,³⁰, we focus on these potential pitfalls to guide radiologists and aligned stakeholders in safely evaluating and using generative AI models in radiology. By providing a list of best practices to avoid these pitfalls, we offer actionable guidelines for radiologists, radiology departments, and vendors engaged in using and integrating generative AI models into radiology workflows.

Regulatory Issues

Our review of regulatory issues surrounding LLMs and other generative AI tools in radiology will cover the following: (a) the need for generative AI–specific regulatory frameworks, (b) navigating a continually evolving regulatory landscape, and (c) the need for these regulatory frameworks to look beyond statistical metrics of accuracy. We present the efforts of the U.S. Food and Drug Administration (FDA) as an illustrative example; the FDA’s policies discussed herein are meant to be descriptive rather than prescriptive because specific regulatory guidelines will differ according to country or region. Generative AI regulatory frameworks are nascent but are being developed by the European Union and individual countries, including the United States, the United Kingdom, Australia, and Singapore ³¹. In addition to the FDA examples, we direct readers to the European Union’s AI Act ³², a comprehensive legal framework for the use of AI in high-risk settings across all sectors (not just health care); this framework will likely be adopted by the European Parliament ³³ and complements the medical-specific FDA examples, continuing to be updated as generative AI technologies evolve ³⁴.

Need for Generative AI–specific Regulatory Frameworks

Although generic frameworks for the regulation of AI have been proposed ³⁵, such as the White House Office of Science and Technology Policy’s “Blueprint for an AI Bill of Rights” ³⁶, the FDA advocates for a medicine-specific approach (precise regulation) ³⁵ by considering AI-based software as medical devices ³⁷. However, even these “precise” regulatory policies do not readily apply to generative AI tools. Nongenerative AI models in radiology that acquire, process, or analyze medical images are considered software as medical devices ³⁸ and have historically been classified by the FDA according to intended use ³⁹. In contrast, generative AI tools do not fit these historical categories ²⁵ because of their unique risks and technical and functional characteristics, which demand a new regulatory paradigm. For example, LLMs differ from previous deterministic deep learning tools in radiology because of their scale, complexity, stochastic behavior, versatile capabilities as generalist foundation models ²^–⁴, and reliance on massive datasets of often nontransparent origin ⁴⁰. Despite the attention that generative AI tools have received in medicine, generative AI models currently lack precise regulatory frameworks, an area in which policymakers, radiologists, researchers, and vendors should collaborate to develop.

Navigating a Continually Evolving Regulatory Landscape

The regulatory landscape of AI in medicine is evolving, with much of the global discussion led by the FDA ³⁸^,⁴⁰. The FDA has acknowledged that its “traditional paradigm of medical device regulation was not designed for adaptive artificial intelligence and machine learning technologies” ⁴¹. The FDA guidelines for computer-aided detection and diagnosis date back to 2012 ⁴², and the concept of software as a medical device, as defined by the International Medical Device Regulators Forum, dates back to at least 2013 ⁴³. However, the new wave of AI tools over the past decade has necessitated updated regulation. In response, the FDA has published iterative regulatory frameworks ³⁷^,⁴⁴^–⁴⁷ and collaborated internationally with agencies in Canada and the United Kingdom ⁴⁵. These recommendations have been summarized in detail elsewhere ³⁸^,⁴⁰. With the rapid development of generative AI tools, regulatory frameworks from the FDA and other agencies will continue to evolve. There is currently no specific FDA guidance for the use of generative AI tools in medicine.

Although definitive FDA guidelines for regulating generative AI tools have not been established, current general recommendations for AI-based software as a medical device provide practical insights. To meet the FDA classification criteria for a medical device, software must analyze medical images, signals, or patterns ⁴⁸. Many generative AI models may not qualify as medical devices under the latest FDA guidance for clinical decision support (CDS) software from September 2022 ⁴⁸. That guidance outlines four criteria for nondevice CDS software, defined as software that provides recommendations to health care providers rather than specific outputs or directives ( Fig 2 ) and (a) does not acquire, process, or analyze medical images, signals, or patterns; (b) displays, analyzes, or prints medical information normally communicated between health care professionals; (c) provides recommendations to a health care professional, rather than providing a specific output or directive; and (d) provides the basis of recommendations so that a health care professional does not rely primarily on these recommendations to make a decision. As noted by Goodman et al ²⁵, this guidance “provides an unintentional ‘roadmap’ for how LLMs could avoid FDA regulation;” if software does not meet one or more of the four criteria, it would not fall under FDA oversight. For example, an LLM-based tool that summarizes a patient’s medical chart or simplifies a radiology report ¹⁰ would be considered “nondevice CDS” because it summarizes patient data (ie, general information provision, which meets one of the four nondevice CDS criteria) and uses text data. However, if an LLM-based tool were used to generate a risk score for a specific disease, it could be classified as a medical device subject to FDA oversight. This ambiguity regarding whether generative AI tools in radiology qualify as medical devices remains unresolved. Radiologists, researchers, and vendors must be aware of the FDA’s regulatory criteria in their clinical deployment, development, and commercialization of generative AI tools. This ambiguity will likely grow as we enter the era of multimodal LLMs, such as OpenAI’s GPT-4o ⁴⁹, which can accept image and text inputs, and Google’s Med-Gemini ⁵⁰, which can handle multiple medical data modalities, including radiology images, pathology images, text, and genomics data. Questions also persist about how the FDA will regulate generative AI models integrated into previously cleared and approved medical devices.

Figure 2: U.S. Food and Drug Administration (FDA) guidance for determining whether a clinical decision support medical software is considered a device that falls under device regulation. This infographic has modified content released by the FDA at .

The unique technical aspects of generative AI models introduce regulatory challenges. For example, LLMs are often overshadowed by newer models: OpenAI’s GPT-3.5 was released in November 2022, followed by GPT-4 in March 2023 ⁵¹ and GPT-4o in May 2024 ⁴⁹; any regulation on one generation of an LLM may not apply to a new one. Even without formal upgrades, LLMs exhibit performance drift ¹⁷^,⁵², whereby their performance fluctuates over time, necessitating continuous monitoring. Open-source or locally deployed generative AI models may prevent these performance drifts by allowing users to control weights and biases. This is an advantage over proprietary (closed-source) models. As the FDA clears static versions of software, model updates and changes in LLM behavior pose ongoing regulatory challenges ³⁸ ( Fig 3 ). For example, if a medical device software builds a user interface on top of a pre-existing LLM (ie, a GPT wrapper ⁵³), has its underlying LLM undergone an unexpected update, and is it still technically FDA-cleared? Similarly, techniques for optimizing generative AI models, such as prompt engineering, hyperparameter adjustments (eg, adjusting temperature to control output randomness and creativity) ²⁹, and retrieval-augmented generation ⁵⁴ may complicate regulation. These could be considered “updates” to software because they affect performance and potential risks of model outputs. Radiologists should be aware of potential regulatory nuances related to the technical features of generative AI models to effectively interpret and engage with guidelines from the FDA and other regulatory organizations.

Figure 3: Technical features of large language models (LLMs) and other generative artificial intelligence (AI) models that present unique regulatory challenges to these technologies in radiology. Temperature refers to a parameter that controls the randomness or creativity of the outputs generated by an LLM or other generative AI model.

Amid the uncertain future of regulatory policies for generative AI models, a potential regulatory framework could distinguish between generalist foundation models and their specific end-user applications. For example, a generalist medical foundation model like Google’s Med-Gemini ⁵⁰ is unlikely to be feasibly regulated. In contrast, a specific end-user application, such as radiology report summarization built on Med-Gemini, can be feasibly regulated in a manner similar to more traditional medical devices regarding performance, repeatability, and bias. End-user applications are more easily regulated on their specific outputs compared with foundation models, representing a feasible path for regulation of LLMs and generative AI in real-world clinical use.

In the absence of clear guidance for regulating generative AI models, there is an opportunity for discussion among the FDA, other government agencies, radiologists, researchers, and vendors. The FDA has invited feedback on proposed regulatory frameworks for AI-based software as a medical device ³⁷. The National Institutes of Health’s Advanced Research Projects Agency for Health has recently launched the Chatbot Accuracy and Reliability Evaluation Exploration Topic ⁵⁵ to fund the development of approaches for evaluating and improving LLM-based chatbots for patient-facing applications to inform regulatory guidance. As generative AI evolves, opportunities to engage with the FDA and other regulatory bodies will likely continue. Radiologists involved in developing these technologies for clinical workflows should be prepared to participate in these discussions.

The Need for Regulatory Frameworks for Generative AI Tools in Radiology to Look beyond Accuracy

FDA clearance of AI-based software as medical devices has typically relied on standard diagnostic performance measures like area under the receiver operating characteristic curve, accuracy, sensitivity, and specificity ³⁸^,⁴². However, these statistical metrics can be misleading for AI tools, especially for generative AI models, which have unique failure modes that require evaluation. This includes measures of similarity between synthetic and real-world data, output reproducibility, robustness against hallucinations, and the effects of human-computer interaction ( Table 1 ). Even metrics of similarity between machine-generated and human-generated texts, such as BLEU 9 (Bilingual Evaluation Understudy) ⁵⁶ and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) ⁵⁷, have been shown to be misleading regarding the medical accuracy of LLM-generated radiology reports ⁵⁸. Because LLMs are probabilistic by design, they do not produce the same response to the same prompt, leading to inconsistent outputs ⁷^,¹⁷^,⁵²^,⁵⁹. This makes reproducibility an important diagnostic performance attribute for these tools in clinical practice ²⁵. Similarly, the robustness of LLMs to hallucinations is important to evaluate because of the insidious nature of hallucinated details in otherwise accurate text outputs. Specific forms of LLM hallucinations ²⁵ include sycophancy, where an LLM tailors text output to user expectations indicated in the prompt. For example, providing a leading differential diagnosis can cause an LLM to emphasize that particular diagnosis in its response. For “complete-the-narrative” errors, an LLM summarizes a clinical narrative, such as a radiology report, by adding a brief but clinically meaningful error, like describing a single radiologic finding that never existed. Further technical details on these pitfalls have been described elsewhere ⁵²^,⁶⁰.

Table 1: Performance Metrics beyond Standard Statistical Metrics * with Examples and Challenges for LLMs and Other Generative AI Models in Radiology

Performance Metric	Examples	Challenges
Quantitative measures of similarity between synthetic and real-world data, which can be evaluated using metrics such as the structural similarity index measure	BLEU (56) and ROUGE (57) for comparing similarity of LLM-generated text to human-generated text	These metrics may not capture medical accuracy and can be misleading regarding the quality of synthesized data
Reproducibility of outputs	Evaluating the consistency of outputs from an LLM to the same prompt over three separate sessions	Performance drift (17,52) makes it difficult to reliably know whether reproducibility of model outputs will hold in the future
Robustness to hallucinations	Sycophancy, in which an LLM tailors text output to user expectations indicated in the prompt (eg, providing a leading differential diagnosis can cause an LLM to emphasize that particular diagnosis in its response)	Evaluating hallucinations requires domain experts to evaluate the veracity of the generated text, image, or other data outputs
Human-computer interaction effects	Evaluating the effect of an LLM or other generative AI tool on a radiologist’s productivity and cognitive load	There is no one-size-fits-all set of metrics for these human-computer interaction effects; each metric will have to be tailored for a specific use case and clinical setting

Note.—AI = artificial intelligence, BLEU = Bilingual Evaluation Understudy, LLM = large language model, ROUGE = Recall-Oriented Understudy for Gisting Evaluation.

Standard statistical metrics refers to standard methods used to evaluate AI models statistically, such as area under the receiver operating characteristic curve, sensitivity, and specificity.

Another important consideration for evaluating and regulating generative AI models is interaction with human end-users, also known as human-computer interaction. How do these tools perform in real-world practice? It is crucial to evaluate their effect on human performance relevant to specific tasks. For example, if an LLM is used to predraft responses to electronic medical record inbox questions from patients or other health care providers, reductions in cognitive burden and time spent on these questions would be important factors to evaluate. A recent prospective study found that using LLMs to draft responses to patient questions in an electronic medical record inbox reduced measures of cognitive burden but did not change the time spent on the task ⁶¹. Evaluating physician diagnostic performance is also important, given the risk of automation bias (overreliance on AI) and potentially worsened performance. Recent studies on the effect of AI-based tools to augment physician interpretations of medical images have shown variable results ⁶²^–⁶⁵. Factors influencing AI’s effectiveness include medical specialty, model performance, whether a model is fair or biased ⁶²^,⁶⁴, and the need for personalized clinician-AI collaboration ⁶⁵.

Generative AI models in radiology may also be patient-facing to improve their health literacy and empower patients with accessible information for informed health care decisions. For such use cases, regulatory considerations must focus on the patient experience and depend on each clinical use case and deployment setting. Early examples of patient-centric metrics for LLMs include the readability of text (eg, radiology reports or answers to common patient questions) generated by an LLM ⁸^,¹⁰^,⁶⁶, with American Medical Association recommendations suggesting readability at the sixth-grade level or lower ⁶⁷. In addition, evaluating the empathy or bedside manner of LLM-generated responses is important ⁶⁸. Another issue related to health equity and inclusion is assessing LLM performance across different languages ⁶⁹^,⁷⁰ because many languages other than English are spoken in the United States ⁷¹.

Finally, as generative AI models expand from text generation to other modalities, such as images ¹⁴, new evaluation paradigms will be needed. Although quantitative metrics, such as the structural similarity index, can compare generated images with real-world images, they do not always reflect human perception or accurately portray such concepts as anatomy ¹⁴ ( Fig 4 ). As these models in radiology evolve into imaging and other modalities, new regulatory frameworks and supporting research will be essential.

Figure 4: Example of a hyperrealistic image produced by a generative artificial intelligence (AI) model (Midjourney, version 5.2; https://www.midjourney.com ) according to the following prompt: “A single human deltoid muscle, anatomic realistic, sectioned showing muscle tissue, black background, macro photography, ultra detailed, hyper realism, 8k, specular reflection, shiny–v 5.2–style raw.” The arrows show anatomically inaccurate tendon insertions. Reprinted, with permission, from reference 14 .

Data Privacy

This section discusses privacy concerns related to training LLMs and other generative AI models with patient data and the resulting privacy implications. Protecting patient data privacy is crucial in developing AI tools in radiology ⁴⁰^,⁷²^–⁷⁴, especially with generative AI models ²⁹^,⁴⁰. Generative AI models are trained on large datasets of often uncertain origin ⁷⁵, raising concerns about the potential inclusion of copyrighted material ⁷⁶ and its unintentional reproduction. Similar concerns exist for generative AI models in radiology, particularly regarding training datasets that may contain sensitive patient data, such as protected health information or records of patients who have opted out of data use for research. Data privacy concerns are amplified by the unique vulnerabilities of LLMs, which are prone to jailbreak attacks that remove models’ guardrails and safety measures through prompt engineering. These attacks enable LLMs to extract verbatim memorized data from their training ⁷⁷. Malicious jailbreak attacks could extract patient data from an LLM, posing a tremendous risk to patient privacy. Image generative AI models, such as Stable Diffusion, have shown the ability to memorize training images and output them during image generation ⁷⁸. Because a Stable Diffusion adaptation for chest radiograph synthesis has been developed ⁷⁹, such models could inadvertently generate images from the training dataset that violate patient privacy.

Beyond patient data privacy risks related to the models and their training data, there is a risk associated with using them on new, unexposed data. Although ChatGPT-3.5 and ChatGPT-4 have been applied to numerous radiology-related tasks ⁶^,¹⁰^,¹¹^,⁸⁰, these are proprietary models owned by OpenAI that require users to send their data directly to the company via its application programming interface. This precludes the use of patient data that has not been deidentified ⁷⁴ according to the Health Insurance Portability and Accountability Act (HIPAA) ⁸¹^,⁸². Some companies can meet HIPAA requirements through robust security measures and a signed business associate agreement. Deploying open-source models locally or on secure cloud servers can enable the entry of patient data into generative AI models while ensuring data security. The need to deidentify text and images is challenging because manual deidentification of medical data is labor-intensive, and automated methods are not fully effective ⁸³^–⁸⁶. Furthermore, depending on local or institutional policies, sending even deidentified data to an LLM or other generative AI tool may not be allowed; popular public medical datasets may be prohibited from being shared with third parties, including online services such as ChatGPT ⁸⁷. Prohibitions on sending data to external parties may arise from concerns that companies such as OpenAI store data to update and improve their models ⁸⁸. Users of patient data or anonymized research data must understand any applicable data restrictions or policies, such as HIPAA ²⁹. An alternative to using commercial third-party LLMs for radiology report disease label extraction is locally deploying open-source LLMs that protect patient data privacy ⁷⁴. Similar approaches with locally deployed generative AI models may allow for compliance with HIPAA and other data privacy regulations, even when working with patient data that have not been deidentified. Federated learning enables decentralized model training without sharing sensitive data ⁷², providing a practical solution for safely using LLMs with patient data.

Radiologists considering purchasing commercial products that use LLMs or other generative AI models should ask for transparency regarding data use ²⁹. A clear data use agreement should be provided, along with the vendors’ willingness to answer questions about it. Vendors should also provide a standard data privacy policy outlining protective measures for patient data and plans to address potential data breaches. Radiologists can inquire about measures taken by vendors to prevent security risks like jailbreaking attacks. The same principles for selecting any radiology AI software ⁸⁹^,⁹⁰ will apply to generative AI models, and radiologists should engage allied partners, such as the local IT or informatics team, when evaluating potential generative AI tools for clinical deployment. For more on ensuring data security when using LLMs clinically, which is beyond the scope of this article, interested readers can refer to the relevant literature ⁹¹^–⁹⁴.

Bias

AI models in radiology can produce biased predictions ²²^,²³^,⁹⁵, threatening their safe and equitable use. Bias often stems from homogeneous training datasets that exclude historically underserved and underrepresented populations ²³^,⁹⁶^–⁹⁸. In addition, LLMs and other generative AI models have demonstrated bias in generating text and other modality outputs in radiology-related tasks ¹⁸^,¹⁹^,⁹⁹^,¹⁰⁰, likely because of biased training datasets. LLMs produce different outputs for radiology report simplification ¹⁰⁰ or differential diagnosis generation from a clinical vignette ¹⁹ by simply changing the race or sex mentioned in the prompt, likely reflecting biases in the training data. Commercial LLMs from major technology companies such as Google and OpenAI have also demonstrated a tendency to propagate inaccurate and harmful race-based medical tropes ¹⁸, raising concerns about their potential to contribute to medical misinformation and perpetuate bias. Omiye and Lester et al ¹⁸ showed that four commercial LLMs generated debunked race-based content in response to questions about race-based medicine or racial misconceptions, such as asserting that race should be used when calculating kidney function and lung capacity or that Black and White people have differences in skin thickness ( Fig 5 ). In another study evaluating generative AI image generation, OpenAI’s DALL-E 2 model was prompted to generate images of a “radiologist,” amplifying real-world demographic disparities in the radiology workforce (eg, 80% of images depicted men compared with 71% male representation in the workforce), and showing radiologists from minority groups in less formal attire than White men ⁹⁹. These early examples of bias in generative AI models in radiology and medicine show risks of directly harming patients through differential bias in clinical recommendations and indirectly harming patients and physicians by perpetuating stereotypes. Radiologists using these tools must be aware of the potential harm from underlying biases in these models to avoid introducing or perpetuating health inequities.

Figure 5: Large language model (LLM) outputs to questions and scenarios that check for race-based medicine or misconceptions about race and medicine and health care. For each question and model evaluated, the rating represents the number of runs (out of five total runs) that had concerning race-based responses. Red correlates with increasing number of concerning race-based responses. The specific models evaluated include Bard, now called Gemini (Google; May 18 and August 3, 2023, versions); ChatGPT (OpenAI; May 12 and August 3, 2023, versions); Claude (Anthropic; May 15 and August 3, 2023, versions); and GPT-4 (OpenAI). eGFR = estimated glomerular filtration rate. The chart is reproduced and the caption adapted from reference 18 under a Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/deed.en ) .

Strategies to mitigate biases in AI models are well described for radiology ¹⁰¹^–¹⁰⁴ and apply to generative AI models. The key strategy is to collect data representing demographic groups reflecting the underlying populations. Radiology imaging datasets used for AI model training have often omitted demographic variables ²⁴^,⁹⁷, which should be considered standard metadata for any AI model—generative or otherwise—to enhance transparency regarding potential bias. Federated learning ⁷² can train models on large diverse datasets without central data aggregation, potentially reducing biases. Evaluating bias and algorithmic fairness should be routine in assessing generative AI models, although established frameworks and definitions for bias are still lacking. The unique nature of training generative AI models necessitates novel methods to reduce bias, such as targeted reinforcement learning with feedback from physician end-users. However, as Zack and Lehman et al ¹⁹ note, such methods may be limited to open-source LLMs because proprietary ones are typically not modifiable. Early work shows promise for measuring biases using standardized frameworks, such as EquityMedQA, a set of medical question–answering datasets designed to identify biases ¹⁰⁵, and “stress testing” LLMs with modified prompts, such as swapping races in clinical vignettes ¹⁸^,¹⁹.

As with identifying potential data privacy issues in commercial generative AI models, calling for vendor transparency is a key practical way for radiologists to mitigate the use of biased tools in clinical practice. This includes engaging with vendors to discuss dataset information used in product training and evaluation, as well as subanalyses of relevant demographic groups to identify potential biases. Radiologists should feel empowered to request these data, which are increasingly being solicited at a professional society level by groups such as the American College of Radiology’s Data Science Institute through its Transparent-AI program ¹⁰⁶. As with other AI tools ¹⁰⁷, radiologists must conduct postimplementation surveillance of generative AI models to evaluate biases, especially because of the unpredictable nature of performance drift ¹⁷^,⁵², which could worsen biases over time.

Clinical Implications of These Pitfalls

The aforementioned theoretical details are crucial for understanding the safe use of LLMs and other generative AI tools, but they must be contextualized to specific clinical use cases and settings. Previous work has described the range of radiology clinical use cases for applied technologies ³⁰, including tasks before imaging, such as protocol automation ¹⁰⁸; tasks after imaging, such as making imaging diagnoses ¹⁰⁹^,¹¹⁰; and patient-facing tasks, such as answering questions about imaging ⁷. Each task has unique safety implications requiring radiologists to use their domain knowledge to anticipate and prepare. For example, a generative AI tool that makes imaging diagnoses can have its outputs checked by a human radiologist as a safeguard against errors or hallucinations, whereas an LLM that answers patient questions does not naturally have an expert to verify its responses (referred to as expert-in-the-loop); thus, the latter may require more extensive robustness and safety testing before deployment. Each use case will be unique, but the goal should remain consistent: to do the right thing for patients.

Future Directions

Tables 2 – 4 summarize best practices and future directions for implementation of generative AI models in real-world settings to avoid pitfalls in three key areas: regulation, data privacy, and bias. We provide future directions in the form of open questions to guide further study and policy development.

Table 2: Best Practices and Future Directions for Avoiding Pitfalls Related to Regulatory Issues in LLMs and Other Generative AI Models in Radiology

Best Practice Recommendation	Future Directions (Open Questions)
LLMs and other generative AI models in radiology should undergo “precise regulation” to account for radiology’s unique workflows and the unique pitfalls of generative AI technology while being guided by “general regulation” principles	What specific regulatory frameworks will apply to generative AI models in radiology?
Regulatory guidance and recommendations from the FDA should be closely followed because they are frequently being updated	Do FDA definitions of SaMD need to be revised or expanded to accommodate for generative AI models?
Radiologists should be aware of the FDA’s definitions of clinical decision support software and SaMD to guide the evaluation, development, and clinical use of generative AI models according to FDA and other local and institutional regulatory guidelines	What standard metrics should be used to evaluate generative AI models in radiology beyond mere accuracy?
Radiologists should be aware of the potential technical limitations of generative AI models that may inform FDA regulatory frameworks	How should regulation of generative AI models account for unique technical features of these tools, such as stochasticity?
Radiologists are encouraged to engage in discussion and collaboration with the FDA and other regulatory bodies to develop regulatory frameworks for generative AI models
Generative AI models should be evaluated for measures beyond standard diagnostic performance, including repeatability, hallucinations, and human-centric measures (relevant to both physicians and patients)

Table 3: Best Practices and Future Directions for Avoiding Pitfalls Related to Data Privacy in LLMs and Other Generative AI Models in Radiology

Best Practice Recommendation	Future Directions (Open Questions)
When training models with radiology patient data, prevent the inadvertent inclusion of prohibited data (eg, protected health information or data from patients who have opted out of research)	What are technically robust methods to prevent the inadvertent inclusion of prohibited patient data during training and development?
Engineer safeguards to prevent jailbreaking that could result in patient data extraction	How can multiple privacy-preserving techniques, such as differential privacy, data encryption, and secure multiparty computation, be combined to ensure data security?
Ensure compliance with local and institutional data privacy policies; exercise similar caution with public datasets, which may prohibit sharing data with third parties	What are the potential pitfalls in applying techniques such as the ones listed above in medical data? How can we best translate these techniques in ways that will best work for the unique privacy issues in patient data?
As alternatives to third-party commercial models, consider using open-source models deployed locally or in secure cloud environments to maintain patient data privacy	How susceptible are generative AI models trained on radiology patient data to malicious jailbreaking attempts? Do these susceptibilities differ from those of domain-agnostic generative AI models?
When purchasing commercial products, ask vendors for transparency about data use policies and measures to prevent data jailbreaking	How well do open-source models perform compared with third-party commercial models for varying radiology tasks of clinical interest? What is the trade-off in performance versus security of these various model choices?
Involve interdisciplinary teams in discussions about potential product purchases; sample questions to ask vendors, including specific language and terms to look for in contracts, are outlined elsewhere (29)
Consider methods and tools to address data privacy concerns, such as federated learning (72), that allow for training on multicenter data without sharing outside a center’s firewall; these strategies have shown success in computer vision tasks in radiology (111–113) and are promising for generative AI models in the field

Table 4: Best Practices and Future Directions for Avoiding Pitfalls Related to Bias in LLMs and Other Generative AI Models in Radiology

Best Practice Recommendation	Future Directions (Open Questions)
Bias should be recognized in multiple forms, including erroneous patient care recommendations and indirect perpetuation of stereotypes	How diverse are the datasets used to train generative AI models in radiology?
Datasets used to train and evaluate models should be diverse and represent the underlying populations from which they are drawn	What are the best ways to identify and measure biases in generative AI models in radiology?
Radiologists should lead the development of frameworks to evaluate for bias and fairness of generative AI in radiology, which are not yet established	What technical methods can be used to mitigate biases in generative AI models in radiology?
Vendors should be asked to provide transparency regarding their models’ potential for biases, including training dataset demographic characteristics and any bias evaluations that have been performed
Consider implementation of techniques and tools during model evaluation that can help identify bias concerns, such as stress-testing LLMs with modified prompts by changing the race or sex (18,19); federated learning strategies (72) that allow for training on multicenter data without requiring sharing of data outside of a center’s firewall can help mitigate bias by increasing the diversity of training data; these strategies have proven effective in computer vision radiology tasks in radiology (111–113) and are promising for generative AI models in the field

Note.—AI = artificial intelligence, FDA = U.S. Food and Drug Administration, LLM = large language model, SaMD = software as a medical device.

Note.—AI = artificial intelligence, LLM = large language model.

Conclusion

As large language models (LLMs) and other generative artificial intelligence (AI) models are integrated into radiology workflows, unique pitfalls threatening their safe use have emerged. These pitfalls are often identified after public release. To mitigate the negative impacts of LLMs and other generative AI models, it is essential to prevent potential pitfalls in three key areas: regulation, data privacy, and bias. We hope these best practices will serve as actionable guidelines for radiologists, radiology departments, and vendors interacting with generative AI models to ensure their safe and effective use.

Keywords:

References

1. Introducing ChatGPT. https://openai.com/blog/chatgpt . Published November 30, 2022. Accessed June 9, 2023.

2. Christensen M, Vukadinovic M, Yuan N, Ouyang D. Vision-language foundation model for echocardiogram interpretation. Nat Med 2024;30(5):1481–1488.

3. Chen RJ, Ding T, Lu MY, et al. Towards a general-purpose foundation model for computational pathology. Nat Med 2024;30(3):850–862.

4. Lu MY, Chen B, Williamson DFK, et al. A visual-language foundation model for computational pathology. Nat Med 2024;30(3):863–874.

5. Jiang LY, Liu XC, Nejatian NP, et al. Health system-scale language models are all-purpose prediction engines. Nature 2023;619(7969):357–362.

6. Haver HL, Lin CT, Sirajuddin A, Yi PH, Jeudy J. Evaluating ChatGPT’s accuracy in lung cancer prevention and screening recommendations. Radiol Cardiothorac Imaging 2023;5(4):e230115.

7. Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology 2023;307(4):e230424.

8. Haver HL, Lin CT, Sirajuddin A, Yi PH, Jeudy J. Use of ChatGPT, GPT-4, and Bard to improve readability of ChatGPT’s answers to common questions about lung cancer and lung cancer screening. AJR Am J Roentgenol 2023;221(5):701–704.

9. Rahsepar AA, Tavakoli N, Kim GHJ, Hassani C, Abtin F, Bedayat A. How AI responds to common lung cancer questions: ChatGPT versus Google Bard. Radiology 2023;307(5):e230922.

10. Amin KS, Davis MA, Doshi R, Haims AH, Khosla P, Forman HP. Accuracy of ChatGPT, Google Bard, and Microsoft Bing for simplifying radiology reports. Radiology 2023;309(2):e232561.

11. Lyu Q, Tan J, Zapadka ME, et al. Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art 2023;6(1):9.

12. Lehnen NC, Dorn F, Wiest IC, et al. Data extraction from free-text reports on mechanical thrombectomy in acute ischemic stroke using ChatGPT: a retrospective analysis. Radiology 2024;311(1):e232741.

13. Fink MA, Bischoff A, Fink CA, et al. Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology 2023;308(3):e231362.

14. Javan R, Mostaghni N. AI-powered hyperrealism: next step in cinematic rendering? Radiology 2024;310(1):e231971.

15. Huang J, Neill L, Wittbrodt M, et al. Generative artificial intelligence for chest radiograph interpretation in the emergency department. JAMA Netw Open 2023;6(10):e2336100.

16. Goh E, Gallo RJ, Strong E, et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nat Med 2025;31(4):1233–1238. [Published correction appears in Nat Med 2025;31(4):1370.]

17. Li D, Gupta K, Bhaduri M, Sathiadoss P, Bhatnagar S, Chong J. Comparing GPT-3.5 and GPT-4 accuracy and drift in Radiology Diagnosis Please cases. Radiology 2024;310(1):e232411.

18. Omiye JA, Lester JC, Spichak S, Rotemberg V, Daneshjou R. Large language models propagate race-based medicine. NPJ Digit Med 2023;6(1):195.

19. Zack T, Lehman E, Suzgun M, et al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health 2024;6(1):e12–e22. [Published correction appears in Lancet Digit Health 2024;6(7):e445.]

20. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 2018;15(11):e1002683.

21. Gichoya JW, Banerjee I, Bhimireddy AR, et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health 2022;4(6):e406–e414.

22. Bachina P, Garin SP, Kulkarni P, et al. Coarse race/ethnicity labels mask underdiagnosis biases in deep learning models for chest radiograph diagnosis. Radiology 2023;309(2):e231693.

23. Beheshtian E, Putman K, Santomartino SM, Parekh VS, Yi PH. Generalizability and bias in a deep learning pediatric bone age prediction model using hand radiographs. Radiology 2023;306(2):e220505.

24. Garin SP, Parekh VS, Sulam J, Yi PH. Medical imaging data science competitions should report dataset demographics and evaluate for bias. Nat Med 2023;29(5):1038–1039.

25. Goodman KE, Yi PH, Morgan DJ. AI-generated clinical summaries require more than accuracy. JAMA 2024;331(8):637–638.

26. Cool stuff now: Epic and generative AI. Epic . https://www.epic.com/epic/post/cool-stuff-now-epic-and-generative-ai . Accessed May 6, 2024.

27. Generate clinical notes with AI: AWS HealthScribe. Amazon Web Services, Inc. https://aws.amazon.com/healthscribe/ . Accessed May 6, 2024.

28. Kim W. Seeing the unseen: advancing generative AI research in radiology. Radiology 2024;311(2):e240935.

29. Bhayana R, Biswas S, Cook TS, et al. From bench to bedside with large language models: AJR expert panel narrative review. AJR Am J Roentgenol 2024;223(3):e2430928.

30. Bhayana R. Chatbots and large language models in radiology: a practical primer for clinical and research applications. Radiology 2024;310(1):e232756.

31. Palaniappan K, Lin EYT, Vogel S. Global regulatory frameworks for the use of artificial intelligence (AI) in the healthcare services sector. Healthcare (Basel) 2024;12(5):562.

32. Proposal for a regulation laying down harmonised rules on artificial intelligence. Shaping Europe’s digital future . 2021. https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-laying-down-harmonised-rules-artificial-intelligence . Accessed May 18, 2024.

33. Harding T, Cunningham E, Jebran A, Havasy R; HIMSS. 2024. Section 5 – Navigating the regulatory landscape: an analysis of legal and ethical oversight for large language models (LLMs). https://www.himss.org/resources/section-5-navigating-regulatory-landscape-analysis-legal-and-ethical-oversight-large . Accessed May 17, 2024.

34. Coulter M. What is the EU AI Act and when will regulation come into effect? Reuters . https://www.reuters.com/technology/what-are-eus-landmark-ai-rules-2023-12-06/ . Accessed May 18, 2024.

35. Reddy S. Navigating the AI revolution: the case for precise regulation in health care. J Med Internet Res 2023;25:e49989.

36. Blueprint for an AI Bill of Rights. OSTP. White House . https://www.whitehouse.gov/ostp/ai-bill-of-rights/ . Accessed May 6, 2024.

37. Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) - discussion paper and request for feedback. Food and Drug Administration . https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf . Accessed May 6, 2024.

38. Zhang K, Khosravi B, Vahdati S, Erickson BJ. FDA review of radiologic AI algorithms: process and challenges. Radiology 2024;310(1):e230242.

39. Rothenberg SA, Savage CH, Abou Elkassem A, et al. Prospective evaluation of AI triage of pulmonary emboli on CT pulmonary angiograms. Radiology 2023;309(1):e230702.

40. Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med 2023;6(1):120.

41. Artificial intelligence and machine learning in software as a medical device. Food and Drug Administration . 2024; https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device . Accessed May 6, 2024.

42. Clinical Performance Assessment: considerations for computer-assisted detection devices applied to radiology images and radiology device data in premarket notification (510(k)) submissions. Food and Drug Administration . https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-performance-assessment-considerations-computer-assisted-detection-devices-applied-radiology . Accessed May 2024.

43. Software as a medical device (SaMD): key definitions. International Medical Device Regulators Forum . 2013. https://www.imdrf.org/documents/software-medical-device-samd-key-definitions . Accessed May 2024.

44. Artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) action plan. Food and Drug Administration . 2021. https://www.fda.gov/media/145022/download?attachment . Accessed May 2024.

45. Good machine learning practice for medical device development: guiding principles. Food and Drug Administration . 2023; https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles . Accessed May 6, 2024.

46. Artificial intelligence and medical products: how CBER, CDER, CDRH, and OCP are working together. Food and Drug Administration . https://www.fda.gov/media/177030/download?attachment . Accessed May 2024.

47. Predetermined change control plans for machine learning-enabled medical devices: guiding principles. Food and Drug Administration . 2023; https://www.fda.gov/medical-devices/software-medical-device-samd/predetermined-change-control-plans-machine-learning-enabled-medical-devices-guiding-principles . Accessed May 6, 2024.

48. Clinical decision support software - guidance for industry and Food and Drug Administration Staff. Food and Drug Administration . https://www.fda.gov/media/109618/download . Accessed May 2024.

49. Hello GPT-4o. OpenAI . https://openai.com/index/hello-gpt-4o/ . Published May 13,2024. Accessed May 24,2024.

50. Yang L, Xu S, Sellergren A, et al. Advancing multimodal medical capabilities of Gemini. arXiv 2024. Preprint posted online May 6, 2024; doi:.

51. GPT-4. https://openai.com/index/gpt-4-research . Published March 14, 2023. Accessed May 7, 2024.

52. Chen L, Zaharia M, Zou J. How is ChatGPT’s behavior changing over time? arXiv 2023. Preprint posted online July 18, 2023; doi:.

53. Kamps HJ. Startups learn the hard way that relying on OpenAI’s tech can burn them. TechCrunch . https://techcrunch.com/2023/11/06/get-the-pdf-outta-here/ . Published November 6, 2023. Accessed May 7, 2024.

54. Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv 2020. Preprint posted online May 22, 2020; doi:.

55. ARPA-H launches exploration topic to improve chatbots for patient-facing applications. ARPA-H . 2024. https://arpa-h.gov/news-and-events/arpa-h-launches-exploration-topic-improve-chatbots-patient-facing-applications . Accessed May 7, 2024.

56. Papineni K, Roukos S, Ward T, Zhu WJ. BLEU: a method for automatic evaluation of machine translation. Proc 40th Annu Meet Assoc Comput Linguist. Association for Computational Linguistics ; 2002.p.311-318. https://doi.org/10.3115/1073083.1073135 .

57. Lin CY. ROUGE: a package for automatic evaluation of summaries. Text Summ Branches Out . Barcelona, Spain: Association for Computational Linguistics; 2004;. p.74–81. https://aclanthology.org/W04-1013 . Accessed May 18, 2024.

58. Yu F, Endo M, Krishnan R, et al. Evaluating progress in automatic chest x-ray radiology report generation. Patterns (N Y) 2023;4(9):100802.

59. Krishna S, Bhambra N, Bleakney R, Bhayana R. Evaluation of reliability, repeatability, robustness, and confidence of GPT-3.5 and GPT-4 on a radiology board–style examination. Radiology 2024;311(2):e232715.

60. Farquhar S, Kossen J, Kuhn L, Gal Y. Detecting hallucinations in large language models using semantic entropy. Nature 2024;630(8017):625–630.

61. Garcia P, Ma SP, Shah S, et al. Artificial intelligence–generated draft replies to patient inbox messages. JAMA Netw Open 2024;7(3):e243201.

62. Groh M, Badri O, Daneshjou R, et al. Deep learning-aided decision support for diagnosis of skin disease across skin tones. Nat Med 2024;30(2):573–583.

63. Gaube S, Suresh H, Raue M, et al. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ Digit Med 2021;4(1):31–38.

64. Jabbour S, Fouhey D, Shepard S, et al. Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study. JAMA 2023;330(23):2275–2284.

65. Yu F, Moehring A, Banerjee O, Salz T, Agarwal N, Rajpurkar P. Heterogeneity and predictors of the effects of AI assistance on radiologists. Nat Med 2024;30(3):837–849.

66. Haver HL, Gupta AK, Ambinder EB, et al. Evaluating the use of ChatGPT to accurately simplify patient-centered information about breast cancer prevention and screening. Radiol Imaging Cancer 2024;6(2):e230086.

67. Weiss B. Health Literacy: A Manual for Clinicians . American Medical Association Foundation and American Medical Association, Chicago, IL. 2003.

68. Ayers JW, Poliak A, Dredze M, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med 2023;183(6):589–596.

69. Adams LC, Truhn D, Busch F, et al. Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: a multilingual feasibility study. Radiology 2023;307(4):e230725.

70. Cozzi A, Pinker K, Hidber A, et al. BI-RADS category assignments by GPT-3.5, GPT-4, and Google Bard: a multilanguage study. Radiology 2024;311(1):e232133.

71. Dietrich S, Hernandez E. Nearly 68 million people spoke a language other than English at home in 2019. United States Census Bureau . https://www.census.gov/library/stories/2022/12/languages-we-speak-in-united-states.html . Accessed May 7, 2024.

72. Gupta S, Kumar S, Chang K, Lu C, Singh P, Kalpathy-Cramer J. Collaborative privacy-preserving approaches for distributed deep learning using multi-institutional data. RadioGraphics 2023;43(4):e220107.

73. Tayebi Arasteh S, Lotfinia M, Nolte T, et al. Securing collaborative medical AI by using differential privacy: domain transfer for classification of chest radiographs. Radiol Artif Intell 2024;6(1):e230212.

74. Mukherjee P, Hou B, Lanfredi RB, Summers RM. Feasibility of using the privacy-preserving large language model vicuna for labeling radiology reports. Radiology 2023;309(1):e231147.

75. Roth E. OpenAI training Sora on YouTube videos would violate the platform’s rules. The Verge . https://www.theverge.com/2024/4/4/24121463/openai-training-sora-on-youtube-videos-would-violate-the-platforms-rules . Posted April 4, 2024. Accessed May 7, 2024.

76. New York Times sues OpenAI and Microsoft over use of copyrighted work. The New York Times . https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html . Accessed May 18, 2024.

77. Nasr M, Carlini N, Hayase J, et al. Scalable extraction of training data from (production) language models. arXiv. 2023. Preprint posted online November 28, 2023; doi:.

78. Somepalli G, Singla V, Goldblum M, Geiping J, Goldstein T. Understanding and mitigating copying in diffusion models. arXiv 2023. Preprint posted online May 31, 2023; doi:.

79. Chambon P, Bluethgen C, Delbrouck JB, et al. RoentGen: vision-language foundation model for chest x-ray generation. arXiv 2022. Preprint posted online November 23, 2022; doi:.

80. Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology 2023;307(5):e230582.

81. Edwards A, Hollin I, Barry J, Kachnowski S. Barriers to cross–institutional health information exchange: a literature review. J Healthc Inf Manag 2010;24(3):22–34.

82. Institute of Medicine (US). Committee on Health Research and the Privacy of Health Information. The HIPAA Privacy Rule. Beyond the HIPAA Privacy Rule: Enhancing Privacy, Improving Health Through Research. Nass SJ, Levit LA, Gostin LO, editors. Washington (DC): National Academies Press (US), 2009. http://www.ncbi.nlm.nih.gov/books/NBK9578/ . Accessed May 7, 2024.

83. Chambon PJ, Wu C, Steinkamp JM, Adleberg J, Cook TS, Langlotz CP. Automated deidentification of radiology reports combining transformer and “hide in plain sight” rule-based methods. J Am Med Inform Assoc 2023;30(2):318–328.

84. Horng H, Steinkamp J, Kahn CE, Cook TS. Ensemble approaches to recognize protected health information in radiology reports. J Digit Imaging 2022;35(6):1694–1698.

85. Steinkamp JM, Pomeranz T, Adleberg J, Kahn CE, Cook TS. Evaluation of automated public de-identification tools on a corpus of radiology reports. Radiol Artif Intell 2020;2(6):e190137.

86. Khosravi B, Mickley JP, Rouzrokh P, et al. Anonymizing radiographs using an object detection deep learning algorithm. Radiol Artif Intell 2023;5(6):e230085.

87. Responsible use of MIMIC data with online services like GPT. https://physionet.org/news/post/gpt-responsible-use . Published April 18,2023. Accessed May 7, 2024.

88. Ray S. Samsung bans ChatGPT among employees after sensitive code leak. Forbes . https://www.forbes.com/sites/siladityaray/2023/05/02/samsung-bans-chatgpt-and-other-chatbots-for-employees-after-sensitive-code-leak/ . Accessed May 7, 2024.

89. Tadavarthi Y, Vey B, Krupinski E, et al. The state of radiology AI: considerations for purchase decisions and current market offerings. Radiol Artif Intell 2020;2(6):e200004.

90. Omoumi P, Ducarouge A, Tournier A, et al. To buy or not to buy—evaluating commercial AI solutions in radiology (the ECLAIR guidelines). Eur Radiol 2021;31(6):3786–3796.

91. Ong JCL, Chang SYH, William W, et al. Ethical and regulatory challenges of large language models in medicine. Lancet Digit Health 2024;6(6):e428–e432.

92. Kanter GP, Packel EA. Health care privacy risks of AI chatbots. JAMA 2023;330(4):311–312.

93. Marks M, Haupt CE. AI chatbots, health privacy, and challenges to HIPAA compliance. JAMA 2023;330(4):309–310.

94. Li J. Security implications of AI chatbots in health care. J Med Internet Res 2023;25:e47551.

95. Larrazabal AJ, Nieto N, Peterson V, Milone DH, Ferrante E. Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proc Natl Acad Sci U S A 2020;117(23):12592–12594.

96. Kaushal A, Altman R, Langlotz C. Geographic distribution of US cohorts used to train deep learning algorithms. JAMA 2020;324(12):1212–1213.

97. Yi PH, Kim TK, Siegel E, Yahyavi-Firouz-Abadi N. Demographic reporting in publicly available chest radiograph data sets: opportunities for mitigating sex and racial disparities in deep learning models. J Am Coll Radiol 2022;19(1 Pt B):192–200.

98. Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med 2021;27(12):2176–2182.

99. Onuh I, Yi PH, Bui M, Kuckelman IJ, Anderson JA, Ross AB. Evaluating the diversity and representation of artificial intelligence-generated images of radiologists: an observational study. J Am Coll Radiol 2024;21(9):1497–1500.

100. Amin KS, Forman HP, Davis MA. Even with ChatGPT, race matters. Clin Imaging 2024;109:110113.

101. Rouzrokh P, Khosravi B, Faghani S, et al. Mitigating bias in radiology machine learning: 1. Data handling. Radiol Artif Intell 2022;4(5):e210290.

102. Tejani AS, Ng YS, Xi Y, Rayan JC. Understanding and mitigating bias in imaging artificial intelligence. RadioGraphics 2024;44(5):e230067.

103. Zhang K, Khosravi B, Vahdati S, et al. Mitigating bias in radiology machine learning: 2. Model development. Radiol Artif Intell 2022;4(5):e220010.

104. Tejani AS, Retson TA, Moy L, Cook TS. Detecting common sources of AI bias: questions to ask when procuring an AI solution. Radiology 2023;307(3):e230580.

105. Pfohl SR, Cole-Lewis H, Sayres R, et al. A toolbox for surfacing health equity harms and biases in large language models. Nat Med 2024;30(12):3590–3600.

106. Transparent-AI. https://www.acrdsi.org/DSI-Services/AI-Central/Transparent-AI . Accessed May 7, 2024.

107. Allen B, Dreyer K, Stibolt R, et al. Evaluation and real-world performance monitoring of artificial intelligence models in clinical practice: try it, buy it, check it. J Am Coll Radiol 2021;18(11):1489–1496.

108. Gertz RJ, Bunck AC, Lennartz S, et al. GPT-4 for automated determination of radiologic study and protocol based on radiology request forms: a feasibility study. Radiology 2023;307(5):e230877.

109. Suh PS, Shim WH, Suh CH, et al. Comparing diagnostic accuracy of radiologists versus GPT-4V and Gemini Pro Vision using image inputs from diagnosis please cases. Radiology 2024;312(1):e240273.

110. Zhou Y, Ong H, Kennedy P, et al. Evaluating GPT-V4 (GPT-4 with Vision) on detection of radiologic findings on chest radiographs. Radiology 2024;311(2):e233270. [Published correction appears in Radiology 2024;311(2):e249016.]

111. Pati S, Baid U, Edwards B, et al. Federated learning enables big data for rare cancer boundary detection. Nat Commun 2022;13(1):7346. [Published correction appears in Nat Commun 2023;14(1):436.]

112. Tang R, Liang H, Guo Y, et al. Pan-mediastinal neoplasm diagnosis via nationwide federated learning: a multicentre cohort study. Lancet Digit Health 2023;5(9):e560–e570.

113. Lee EH, Han M, Wright J, et al. An international study presenting a federated learning AI platform for pediatric brain tumors. Nat Commun 2024;15(1):7615.