Large language models are moving into clinical workflows that involve documentation, decision support, patient communication, education and administrative activity. Their potential value sits alongside concerns about accuracy, reliability, data privacy, clinician trust and regulatory responsibility. A 2026 publication in BMJ Health & Care Informatics sets out a clinician-led, risk-stratified framework for responsible adoption in clinical practice. The framework groups applications into low, moderate, high and critical risk tiers, with oversight increasing as potential harm rises. It links safe use to local governance, clinician education, quality management and statutory regulatory scrutiny where needed. Self-regulation functions as an institutional safety layer, led by clinicians and supported by professional societies, while regulatory pathways and clinical evidence continue to mature.
Tiered Risk Guides Oversight
The framework categorises clinical LLM use according to potential harm rather than treating all applications as equivalent. It draws conceptually on established risk management for medical devices and software, including approaches linked to the US Food and Drug Administration and European Union Medical Device Regulation, while adapting those principles to clinical settings. The focus is clinician oversight, patient safety and institution-level governance rather than formal probabilistic scoring.
Must Read: Healthcare AI Uptake Accelerated in 2025
Low-risk applications include patient education and frequently asked questions, administrative tasks, public health content drafted for human review, workflow optimisation, medical content creation for professionals and training materials for non-clinical staff. Examples include explaining conditions such as asthma, supporting advice on chronic condition management, assisting appointment scheduling, handling billing or coding queries, drafting vaccine awareness campaigns for expert approval, translating health-related content and creating front-desk communication materials. Main risks include misinformation in public content, limited exposure of administrative data, minor inaccuracies, cultural misalignment in translations and errors unlikely to harm patients. Governance measures include local standard operating procedures, approved prompt templates and human sign-off before externally shared content.
Clinician Review Supports Safer Use
Moderate-risk applications inform clinical work but do not directly determine treatment plans. This tier includes medical record summarisation, symptom checkers, non-autonomous clinical decision support, drug information queries, medical documentation assistance, clinical trial matching and medical guideline summarisation. Examples include condensing lengthy records for clinician review, giving preliminary triage suggestions to patients, offering drug interaction warnings, suggesting guideline-based diagnostic steps, drafting follow-up notes or referrals and identifying patients who may be eligible for specific clinical trials.
Risks rise because omissions, outdated information or flawed outputs may affect clinical judgement. Record summaries may miss critical details. Symptom checkers may misclassify symptoms and delay care. Drug information tools may rely on outdated or incorrect database integration. Documentation support may introduce inaccurate records. Guideline summaries may be incomplete or inaccurate, contributing to suboptimal care. Moderate-risk use therefore requires deliberate guardrails, including mandatory clinician review and confirmation before outputs enter care. Training in AI literacy and automation bias supports critical appraisal, uncertainty awareness and trust calibration. Audit logging, periodic quality review and feedback loops strengthen ongoing oversight.
Validation and Consent Shape High-Risk Deployment
High-risk applications directly influence important clinical decisions and patient outcomes. This tier includes personalised treatment recommendations, predictive analytics in diagnostics, generative content in sensitive contexts, telehealth triage support, risk stratification models, real-time decision support in critical care and provider–patient communication enhancement. Examples include suggesting individualised therapy plans, predicting cancer risk from imaging or laboratory results, writing discharge summaries, consent forms or instructions, suggesting initial teleconsultation questions, identifying high-risk patients for chronic disease management and automating responses to patient portal messages about care plans. Errors may cause harm through misleading recommendations, false positives, false negatives, misstatements, misprioritisation, misclassification or poor communication.
Deployment requires formal clinical validation, clearly defined intended-use boundaries, multidisciplinary approval, continuous monitoring, change control for model updates and regulatory assessment where applicable. Critical-risk applications involve autonomous prescribing, autonomous diagnosis, mental health chatbots without clinician oversight, data synthesis across patient records, training on real patient data, population health management, predictive care interventions and research applications using patient datasets. Routine deployment is strongly discouraged without extensive regulatory approval, safety trials, explicit consent frameworks, real-time human override and independent safety monitoring.
LLMs may support documentation, communication, information access and workflow efficiency, but clinical use depends on the level of risk attached to each application. Low-risk functions need local controls and human sign-off. Moderate-risk tools need clinician confirmation, training and audit processes. High-risk applications require formal validation, clear boundaries, monitoring and change control. Critical-risk applications should not enter routine care without strong regulatory approval, safety evidence, consent structures and human override. A tiered governance model gives healthcare organisations a practical route for safer adoption while regulation and evidence continue to develop.
Source: BMJ Health & Care Informatics
Image Credit: iStock
References:
Mohammad M, Jimenez-Solem E, Hejmadi M & Pihl A (2026) Self-regulating the use of large language models in clinical practice: a risk-stratified approach. BMJ Health & Care Informatics;33:e101921.
