From Bench to Bedside With Large Language Models: <i>AJR</i> Expert Panel Narrative Review

Large language models (LLMs) promise to revolutionize the health care industry. In radiology, early research illustrates the potential of LLMs to transform various aspects of clinical practice, education, and research ^1–3. For instance, LLMs can help with decision-making, differential diagnoses, report generation, report summarization, and report extraction tasks ^1,3–5. Existing reviews cover fundamental LLM concepts, limitations, and potential applications in radiology ^6–11. These articles are critical in providing radiologists and referring clinicians with a deeper understanding of LLMs and sparking imagination as to how clinical care might be transformed ⁹. Figure 1 provides examples of how LLMs can augment efficiency and quality in radiology practice.

Fig. 1: —Diagram shows examples of how large language models (LLMs) can augment efficiency and quality in radiology practice.

Considering rapid advancements in LLMs and the growing list of applications in radiology, this AJR Expert Panel Narrative Review takes a different approach. Whereas research and validation for applications in radiology remain in their infancy, commercial products incorporating LLMs are increasingly available with a promise to transform practice. For radiologists, navigating this evolving landscape while ensuring quality and safety in care can be overwhelming. Thus, we aim to provide a practical guide to LLMs for radiologists in multiple practice settings and environments, encompassing considerations from bench to bedside ( Fig. 2 ). As a primer for this practical guide to LLMs, we provide a summary of basic terminology and concepts in Table 1 .

Fig. 2: —Diagram shows multidimensional perspective from bench phase to bedside phase with large language models (LLMs). Bench phase introduces how models can be optimized and validated for particular uses in radiology. Translational phase involves bridging model development with useful applications in radiology, highlighting what radiologists should consider when evaluating applications that incorporate LLMs. Bedside phase addresses practical considerations for using LLMs in clinical practice, and patient phase covers how LLMs can impact patient care and education.

TABLE 1: Essential Definitions for Basic Terminology Related to LLMs and Associated Concepts

Note—The ordering of definitions is provided to build on previous concepts, moving from more general to more specific. LLM = large language model, AI = artificial intelligence, NLP = natural language processing, RAG = retrieval-augmented generation.

First, in the bench phase, we consider LLM development and research, introducing how models can be optimized and validated for specific uses in radiology. Next, we discuss the translational phase, which involves bridging model development with clinically relevant use cases and highlighting what radiologists should consider when evaluating applications that incorporate LLMs. The bedside phase addresses practical considerations for using LLMs in clinical practice. Last, in the patient phase, we discuss how LLMs can impact patient care and education, including considerations related to patient-facing tools. Altogether, this review will equip radiologists with a multidimensional perspective on LLMs, whether reviewing the literature, considering purchase of software that incorporates LLMs, using these tools in practice, or advising patients on how to use LLMs appropriately.

Task-Specific Development and Optimization for LLM Safety

The importance of ensuring the safe use of LLMs must first be highlighted. Public artificial intelligence (AI) chatbots, like ChatGPT (OpenAI), have dominated the public discourse on LLMs, as they broadly showcase the impressive versatility of LLMs. Although research focusing on popular public AI chatbots in radiology sparks imagination, these chatbots are not designed for use in radiology practice and should be used with caution for many reasons. These include issues related to privacy, transparency, scope, accuracy, variability, and tendency to perpetuate bias.

Protected health information should never be entered into public chatbots. This violates HIPAA and breaches patient confidentiality, as information can be retained on company servers ¹². Because retained data can be used to train future models, patients may be at risk of their data being revealed or misused. State-ofthe-art foundation models, like OpenAI's GPT-4, are closed source. Details related to model architecture and weights, training data, and fine-tuning steps were not made public, so models operate in a black box. Updates to a model can modify its behavior and performance (i.e., model drift), which could render prior validation efforts irrelevant and could be dangerous in health care settings ¹³. Public AI chatbots and foundation models are general purpose and have not been specifically created for medicine or radiology. Impressive accuracy on domain-specific standardized examination questions is intriguing but has been far from perfect in most clinical specialties, including radiology ^14,15. Examination performance does not necessarily translate to practical clinical uses of LLMs. LLMs are stochastic by design, so they can produce variable responses to the same prompt. This approach can make them less reliable for even simple language tasks with default settings in radiology, like extracting reported findings ¹⁶. General-purpose models can also perpetuate bias and potentially perpetuate medical misinformation and bias ^17,18.

For these reasons, tools that leverage LLMs must be optimized for radiology-specific tasks before use in practice ( Fig. 3 ). LLM-based applications used in radiology must be HIPAA compliant if using protected health information, validated for specific use cases, and tuned to be accurate and reliable. The underlying model architecture should be transparent and, ideally, open source, which would facilitate third-party evaluations by independent groups (e.g., academic researchers). Care must be taken, however, to balance the benefits of making software open source with the risks, such as the potential for misuse and/or prompt injection of malware into the code. More broadly, applications built with LLMs should take into account government regulations and institutional guidelines. Although AI regulation is evolving, many advocate for LLM-specific regulation to ensure privacy and safety in medical contexts considering the unique limitations of LLMs ¹⁹.

Fig. 3: —Diagram shows safety concerns of public artificial intelligence (AI) chatbots and outlines why task-specific development and optimization are required for use in clinical settings.

Bench Phase

LLMs have rapidly improved in performance as they have scaled in size (i.e., in number of parameters), facilitated by advancements in deep learning (transformer architecture), availability of data, and computational capabilities. As LLMs have increased in size, they have revolutionized natural language processing (NLP). Unlike traditional models, state-of-the-art LLMs show robust performance on NLP tasks with only a few examples (few-shot learning), making them versatile for tasks like summarizing and extracting text ²⁰. They have also demonstrated knowledge and higher-order reasoning in many professional domains, including radiology ^14,15,21. As such, several exciting applications in radiology have the potential to improve efficiency and quality of care.

Although default AI chatbots and LLMs show flashes of brilliance, LLM-based tools must be optimized and validated for specific uses for the reasons previously mentioned. They should be thought of as powerful but blunt tools that can be curated to specific tasks. In the next sections, we consider the development, optimization, and validation of LLM-based tools ( Table 2 ). This discussion provides the background required for radiologists to appraise literature, understand products, and spark research ideas.

TABLE 2: Strategies for LLM Development, Optimization, and Validation in Radiology

Note—LLM = large language model.

Defining Scope

Before optimization, the specific purpose and intended use of a particular application must be defined. Choosing a suitable application requires an in-depth understanding of radiology workflows and the strengths and limitations of LLMs. The intended use of the tool should be defined, including the intended level of supervision. Given the limitations inherent to LLMs, applications in clinical radiology will require supervision for the foreseeable future.

The criteria for acceptable performance must be clearly defined in consultation with clinical stakeholders. Acceptable performance varies widely by application, level of supervision, and potential for harm. For instance, errors in a tool that aids radiologist dictation efficiency are less likely to cause harm than errors in an autonomous chatbot providing direct medical advice to patients. A tool designed to identify incidental findings in radiology reports may prioritize recall (sensitivity) over precision; for example, a tool meant to screen radiology reports to identify all mentions of coronary artery calcium may prioritize recall/sensitivity similar to image-based screening examinations. Once the application's scope is defined, model optimization strategies should be considered to help produce a tool with acceptable performance.

Optimization of LLMs

As LLMs continue to improve, acceptable performance on some simple radiology-specific tasks (i.e., report extraction) may be achieved with little optimization ⁵. However, in most cases, including in more advanced applications, optimization techniques must be strategically used to achieve performance goals. Given that state-of-the-art models are computationally and resource intensive, smaller models optimized for narrower uses may achieve similar or better performance at a lower cost (in terms of computing resources and corresponding monetary charges and carbon emissions). Furthermore, locally developed and deployed tools may have lower costs compared with commercial products, which may have high licensing costs.

Optimization techniques— Popular techniques that can be used to optimize LLM performance for particular uses include: prompt engineering, retrieval-augmented generation (RAG), fine-tuning, and pretraining a model. Each approach provides distinct benefits and can be used in isolation or combined to maximize performance for a particular application.

Prompt engineering refers to providing an LLM with natural language instruction that indicates how the model should behave. Prompts include clear instructions for a particular task and may include additional context or a few examples of the task (few-shot learning). Prompting can markedly affect LLM performance by providing the model with specific knowledge, stipulating the desired output format, or showing how to approach a problem. For instance, in radiology, providing additional context to an LLM enables better report summaries ²². One prompting strategy—question-specific automatic prompt generation—improves the accuracy of examination questions ²³. Another—chain-of-thought prompting, which involves instructing the model to approach a problem in a stepwise fashion—can enable improved reasoning on medical examination questions ²⁴. In one preliminary study, chain-of-thought prompting also improved the accuracy of pancreatic cancer resectability categorization based on radiology reports ²⁵. Prompt engineering is usually the easiest, cheapest, and most fruitful way to improve performance at a low cost. It should be the first consideration before other optimization techniques.

RAG involves connecting an LLM to external data that can provide it with new knowledge that enables functionality ²⁶. Put simply, custom text data are converted into vectors (numeric representation of text that can be analyzed by an LLM), and the LLM can retrieve chunks of data most relevant to a specific prompt when generating a response. Foundation models are pretrained on general data, so grounding the model in high-quality domain-specific data can decrease the frequency of hallucinations for related tasks ²⁷. This ability can be particularly useful in radiology and provides the model with information that it can use to carry out specific specialty-related tasks. For instance, an LLM connected to the American College of Radiology (ACR) Appropriateness Criteria performed better at applying the criteria than radiologists ³.

Whereas training an LLM from scratch can be prohibitively expensive, fine-tuning an existing model that has been pretrained (i.e., developed on a larger dataset, frequently from a different domain) can substantially improve performance for a particular application. Most model layers remain frozen, and fine-tuning on task-specific examples fundamentally adjusts the weights of the model's top layers. This technique can adjust model behavior to align with its intended function. For instance, in radiology, fine-tuning an LLM improves performance in generating radiology impressions from reports ^3,28. RadBERT, a family of language models trained and fine-tuned using radiology reports, outperforms baseline models on radiology-specific NLP tasks ²⁹. Smaller models that have undergone fine-tuning can outperform bigger models on specific tasks, which is more computationally efficient and sustainable ³⁰. For tasks that may require substantial prompt instruction, fine-tuning can reduce the prompt length required, which is more cost-effective.

LLM hyperparameter adjustment— For LLMs, a number of adjustable hyperparameters determine model behavior, including model temperature (i.e., propensity to generate variable responses). LLMs are probabilistic by nature, with the ability to produce different responses to the same prompt. This ability can be desirable in conversational or creative settings in which randomness increases response diversity and creativity. The temperature parameter controls randomness, with higher temperatures contributing greater randomness ³¹. Default AI chatbots generally use relatively high temperatures (i.e., 0.7–1.0). A temperature of 0 is nearly deterministic (i.e., more consistently producing the highest probability response). For many applications in radiology, a lower temperature is more desirable. For instance, in one study, lower temperatures led to better agreement for a report extraction task ¹⁶.

Approach to optimization— The best approach to optimization depends on the particular application and stage of development. For many applications, robust performance can be achieved with state-of-the-art models through prompt engineering alone. So, prompt engineering with state-of-the-art models can facilitate exploration for various uses relatively quickly and at low cost.

To improve performance further, and for production-ready applications, iterating with RAG, fine-tuning, pretraining, and different models should be explored to overcome previously described limitations. Ultimately, the goal is to enable applications that improve care, efficiency, and cost-effectiveness in a safe and reliable way.

Performance Validation

Once an LLM-based tool has been optimized, it must be validated for performance, reliability, and safety. For many applications, this validation should involve evaluating performance on sizable datasets not used to create the model (i.e., the test set), including external data. Because LLMs are stochastic by design, reliability and repeatability should also be verified. If tools reach an acceptable and safe level of performance, they should ideally be evaluated in prospective and randomized practice settings to measure benefits and potential harms.

Radiologists must be able to critically appraise literature exploring LLM-based tools by evaluating method, study sample, performance metrics, and external validity. A basic understanding of performance metrics relevant to machine learning is helpful ( Table 3 ). For particular applications in the literature, radiologists should consider whether similar tools could be deployed locally, taking into account institutional guidelines, privacy considerations, and cost. Local deployment requires computational resources and specific expertise; therefore, commercially available solutions may be most viable. However, these tools should be critically assessed before procurement.

TABLE 3: Basic Performance Metrics Commonly Reported in Machine Learning

Note—Many of these metrics are generally familiar to radiologists and referring clinicians. Most commonly, precision, recall, and F1 score are reported. TP = true-positive, TN = true-negative, FP = false-positive, FN = false-negative.

Translational Phase

When considering implementing LLM-based applications in practice, radiologists must decide whether to build a custom solution or procure a commercially available product. In this section, we focus on the latter option. Many commercially available medical software packages, including EHRs and radiology reporting software, have integrated LLMs. Additionally, some applications are designed explicitly around LLMs, such as those that generate automated report impressions and patient-friendly radiology reports. Despite the nascent stage of research and validation in the academic literature, vendors are actively releasing products that claim to enhance efficiency, elevate the quality of care, and deepen patient engagement. To ensure the safety and quality of care, radiologists must communicate with vendors to better understand these products and critically appraise their claims. Table 4 provides a summary of the recommendations in the next section in the form of suggested questions to ask vendors during discussions regarding LLM-based products. Many of these considerations generally apply to other AI-based products in radiology.

TABLE 4: Sample Questions for Radiologists to Ask Vendors in a Request for Proposals for LLM Products

Note—LLM = large language model, AI = artificial intelligence.

Understanding the Product

Radiologists must understand a product's intended use, ease of use, and expected learning curve. Assessing how the tool integrates into existing workflows is crucial. An awareness of the strengths and limitations of these products will aid radiologists in setting realistic expectations and effectively leveraging these technologies in clinical settings.

The underlying architecture should be considered at a high level. For instance, the deployment (on-premises vs cloud-based), the type of LLM used (potentially not disclosed by the vendor), and optimization methods should be considered. LLMs have a propensity to perpetuate biases; therefore, inquiries into training methods, datasets used, and strategies for bias mitigation are essential. In proprietary closed models, it is advisable to inquire about the impact of model updates and architectural changes over time.

Given the sensitive nature of medical data, security and privacy are paramount. Understanding data requirements, usage, and protection measures, especially concerning training of LLMs, is critical. Vendors must be transparent about the necessity of each data element they request. Radiologists should be well-informed about how the technology aligns with health information privacy laws (like HIPAA), institutional guidelines, and regulatory standards, ensuring secure and ethical handling of patient data. Radiologists should also be aware of unique security risks associated with LLMs, such as susceptibility to prompt injections and jailbreaking, and should inquire about the vendor's guard-rails and preventive strategies.

Critically Assessing Vendors and Appraising Claims

The relative ease of launching startup companies on the backbone of LLMs, like generative pretrained transformer (GPT) models, necessitates carefully assessing vendors and their claims. First, the vendor's track record in developing and deploying radiology applications, particularly those based on LLMs, should be evaluated. Second, the extent of clinical domain expertise within the vendor's team, such as the involvement of radiologists, is a critical consideration. Third, when vendors claim enhancements in efficiency or quality, these assertions must be critically evaluated. Radiologists should inquire about the methods used in measuring these metrics and the validation processes for accuracy and safety. Given that LLM evaluation metrics have limitations, the importance of human evaluation in the assessment process cannot be overstated. For example, metrics commonly used to evaluate LLMs do not necessarily correlate with clinical correctness or usefulness ³². Accounting for individual radiologists' preferences can improve clinical utility ³³. Fourth, addressing how vendors mitigate the risk that results when LLMs produce erroneous outputs (i.e., hallucination or confabulation) is essential. Last, it is crucial to understand how vendors monitor and respond to the drift in LLMs over time, particularly with closed models.

Communicating Needs and Goals

Radiologists should clearly articulate their specific needs, challenges, and objectives to vendors. This communication includes details regarding types of procedures, existing infrastructure, local regulatory frameworks, and anticipated outcomes from implementing a specific tool. Such clarity enables vendors to accurately assess implementation and provide tailored recommendations or customizations.

Bedside Phase

To maximize benefits and minimize risk, an LLM-based tool procured for use in practice should take into account the previously described considerations. However, additional factors unique to LLMs must be considered before implementation. LLMs possess a unique set of limitations, and end users of a particular software must be informed of associated strengths and weaknesses to guide appropriate use. Radiologists must understand the potential consequences of errors in a specific application and the implications of liability. Practice leadership should set clear guidelines on how a particular tool should be used and to ensure performance is monitored over time.

Understanding Strengths, Limitations, and Appropriate Use

End users should be informed of a particular tool's strengths and limitations. This should include a basic understanding of how LLMs work and key limitations described previously. Radiologists should be equipped with application-specific performance metrics to understand the level of accuracy and reliability they should expect. The intended use must be made clear. As mentioned, LLM-based tools will require supervision in practice for the foreseeable future, so radiologists must be diligent in confirming the accuracy of model outputs. The specific practice settings in which a particular tool can be applied should be clear to ensure a tool is used appropriately. Further, how to optimize performance at an end-user level should also be communicated. For instance, an LLM-based dictation tool may work best when radiologists use standardized terminology in their descriptions.

Liability as a Consequences of Errors

LLMs are imperfect, and tools built upon them will make errors. Radiologists must be informed that supervision is required and should consider the potential consequences of errors for a particular tool. Liability implications should also be considered but are yet to be well-defined given the rapid evolution and development of LLMs and AI. Legislation for LLMs has yet to be definitively established ³⁴, given the technologies' relative nascency, and will likely develop rapidly as the tools mature.

The potential consequences of errors depend on the specific implementation. LLMs should be viewed as tools that can augment efficiency and quality and not as autonomous entities that replace decision-making, at least for the time being; in the future, LLMs may perform some tasks autonomously, which will require unique liability considerations. However, for the foreseeable future, radiologists remain responsible for the content of their reports, diagnoses, and recommendations.

Patient Phase

Using LLMs in practice can improve the efficiency, quality, and cost-effectiveness of care (for example, by automating tasks typically requiring human labor or improving the efficiency of laborious tasks such as extracting incidental findings from radiology reports). Simply put, the impact on patients should be the single most important factor. Medical imaging is central in modern medical decision-making, and patients stand to benefit immensely from improved efficiency and quality in radiology. However, unnoticed errors may negatively impact care. With an understanding of LLMs, tool-specific performance, limitations, and the need for supervision, the benefits of integrating LLMs in practice are likely to strongly outweigh mitigated risks. Patients and patient advocacy groups should be involved in the design and use of LLM products, especially those that are patient facing.

Health Literacy

LLMs also have immense potential to improve patient health literacy via patient-facing tools, which could improve outcomes and screening rates. These tools will likely have the greatest impact on marginalized patients with poor access to physicians. General AI chatbots, like ChatGPT and Med-PaLM (Google), have already demonstrated impressive accuracy when answering patient medical and disease-specific questions ^35–37. They have also demonstrated promise in summarizing complex medical text for patients, including radiology reports ³⁸. Models optimized for specific uses, such as chatbots connected to domain-specific knowledge, could be an immense resource for patients.

Safety

For patient-facing chatbot applications, radiologists and vendors must ensure products incorporate additional safety measures. For instance, guardrails must be placed on the chatbot to narrow its scope. Patient-facing chatbots in medical practice should refuse to answer questions outside of their expertise, which includes unrelated, inappropriate, and dangerous questions. Further, chatbots are susceptible to jailbreaking attacks, in which end users can prompt the model in specific ways to remove guardrails and safety measures. Vendors and radiologists should consider how these risks can be mitigated. Given these potential risks, supervised use of these tools by physicians may be appropriate for the time being. For instance, an LLM-based tool could produce draft responses to patient questions that require approval by a health care professional (an approach that would not practically work for real-time patient chatbot encounters). Additionally, patients should be informed of privacy concerns before they enter their data into LLMs, which would ideally automatically remove this data on completion of the tool's use.

Limitations

Last, patients must be informed with respect to LLMs and tool-specific limitations before use. In addition to providing disclaimers at the time of use, health care professionals should instruct patients on how application outputs should be interpreted, in what contexts the tools excel, and in what contexts the outputs are susceptible to failure. Given that LLMs are becoming ubiquitous, they are likely to transform patient-physician interactions. Once LLMs are more widely in use, by the time a patient presents to a physician, the patient may be equipped with preliminary information and preconceived ideas. These changes, added to the increased accessibility of information, will likely be similar to the transformation seen as Internet search engines emerged (i.e., Dr. Google) ³⁹. Ultimately, this transformation is expected to present unique benefits and challenges that will likely become the new normal over time.

Future Directions

Multimodal LLMs can accept and/or generate more than one form of data (i.e., text, images, video, audio). They can perform many vision-language tasks, including captioning images and answering questions related to images. In the context of medical imaging, captioning images can produce radiology reports given an image. In fact, multimodal LLMs have already demonstrated promise in producing chest radiograph reports that are preferred to some radiologist reports ^40,41. Multimodality use also enables consideration of multiple forms of data when producing an output. In medicine, this technique can involve interpreting both imaging and clinical data to generate more accurate diagnoses ⁴².

Traditional LLMs produce a single output given a human-generated prompt. In contrast, autonomous LLM agents (e.g., AutoGPT, AutoGen) have immense potential and can repeatedly prompt themselves to achieve more substantial goals. In the context of radiology research, this may involve performing research given a single prompt. The agent may decide to collect certain radiology reports, extract specific pieces of information, perform analyses, and then write a research paper. For now, LLM agents remain unreliable, though promising.

Notas

Provenance and review: Solicited; externally peer reviewed. Peer reviewers: Ramin Javan, George Washington University Medical Center; additional individual(s) who chose not to disclose their identity.