Healthcare AI Regulation Needs Lifecycle Oversight

Healthcare AI regulation in the UK is being shaped by evidence that support depends on clear safety, effectiveness, transparency and human oversight in practice. A report produced by the Medicines and Healthcare products Regulatory Agency for the National Commission into the Regulation of AI in Healthcare brings together research and engagement on future oversight of AI-enabled health technologies. The evidence covers patients and the public, healthcare professionals, industry and system partners, using deliberative work, stakeholder discussions, surveys, evidence reviews and a regulatory sandbox. Across these activities, confidence depends less on general enthusiasm for innovation and more on whether AI can improve care safely in real-world settings.

Trust Depends on Safeguards and Human Oversight

Support for AI in healthcare is consistently conditional. Lower-risk administrative uses are viewed more favourably, while tools that influence diagnosis, treatment, triage or access to care attract greater caution. Public and professional confidence depends on meaningful human involvement, clear communication, visible safeguards and evidence that systems improve outcomes in practice.

Public awareness of AI use in healthcare remains low and uneven. Survey findings show cautiously positive attitudes, with stronger support when AI supports healthcare professionals rather than replaces them. Acceptance declines sharply for autonomous systems, particularly in higher-risk clinical contexts. Safety, evidence and human checks are prioritised over speed or convenience.

Engagement with underserved groups reinforces these expectations. Participants wanted plain-language information, the ability to speak to a person and meaningful choice where AI is used. They raised concerns about commercial incentives, use of health data and the erosion of human aspects of care. People with learning disabilities, unpaid carers and young people emphasised that inclusion, accessible consent and alternative routes to care are central to trust, not optional additions.

Professional discussions reached similar conclusions. AI may support communication, shared decision-making and efficiency, but it also creates risks around misunderstanding, over-reliance and reduced human interaction. Trust depends on systems being understandable, challengeable and safely integrated into clinical workflows.

Regulation Must Match Adaptive Clinical Systems

Existing regulatory approaches are not well matched to AI systems that may change over time, perform differently across settings or depend heavily on local workflows, data quality and human behaviour. Current frameworks were largely designed for fixed products assessed at a single point, while AI systems can be adaptive, context-dependent and shaped by deployment conditions.

Across stakeholders, lifecycle oversight is a recurring priority. Pre-market assessment remains important, but it is not sufficient where performance may drift, outputs may vary and risks may emerge only after systems are used in practice. Ongoing monitoring, post-market surveillance and clear thresholds for action are needed to identify changing performance, unintended consequences and safety concerns.

The regulatory sandbox work shows that AI as a medical device exposes gaps in existing approaches. Challenges include synthetic data, large language models, hallucinations, non-deterministic outputs, model drift, explainability and real-time monitoring. Existing reporting mechanisms provide a foundation, but they are not designed specifically for systems that change over time or interact with users in complex environments.

Risk-based regulation remains widely supported, but risk is not defined only by the intended function of a tool. Autonomy, system architecture, deployment setting, patient population and human-AI interaction all influence safety. The same technology may present different risks in different care environments, making real-world evidence and context-sensitive oversight central to effective regulation.

Must Read: Ethical Governance for Routine LLM Use in Healthcare

Accountability, Equity and Monitoring Need Clarity

Responsibility and liability remain unclear across manufacturers, healthcare providers, professionals and other actors involved in AI development and deployment. Stakeholders consistently identify this uncertainty as a barrier to safe adoption, professional confidence and public trust. Legal and regulatory frameworks are seen as poorly aligned with adaptive, multi-actor systems where causation may be difficult to establish.

Healthcare professionals remain central to clinical decisions, but they may have limited influence over system design, updates or deployment conditions. Professional roundtables identified concerns that clinicians could carry practical accountability for AI-influenced decisions even when responsibility for harms is shared or unclear. Industry discussions also raised concern about healthcare professionals becoming default holders of liability when wider system design contributes to errors.

Patients may lack clarity on when AI is used, how it influences care and where to seek redress if harm occurs. Transparency therefore needs to include more than technical explainability. Clear communication, defined responsibilities, routes for challenge and accessible mechanisms for raising concerns are essential parts of trustworthy deployment.

Equity is also a safety issue. Bias, uneven performance, digital exclusion and differential access recur throughout the evidence. AI systems trained on unrepresentative data may produce unequal or unsafe outcomes for different groups. Uneven rollout, digital-first pathways and lack of accessible information may also disadvantage people with complex needs or limited digital access. Fair implementation requires attention across design, testing, evaluation, deployment and monitoring.

The evidence points towards a regulatory approach that is risk-based, proportionate, adaptive and focused on the full lifecycle of AI systems. Public confidence and professional trust depend on safety, demonstrated benefit, transparency, accountability, equity and meaningful human oversight. AI in healthcare is not unacceptable, but its use is expected to meet clear conditions that reflect real-world clinical practice. Regulation therefore needs to support timely access to beneficial technologies while maintaining safeguards that protect patients, professionals and the wider health system.

Source: GOV.UK

Image Credit: iStock