Crossing the Rubicon in the Age of Agentic AI: Clinical Risk, Exposure and Cost in Digital Health

As AI capabilities shift from advisory to agentic roles, health systems face a structural inflection: whether they can govern clinical risk, operational exposure and financial cost before these compound. Clinical-grade innovation, resilience and financial sustainability must take precedence over uncritical technological expansion if connected digital health systems are to deliver on their promise.

Key Points

• Digital health now depends on governing consequences, not only expanding capability.
• Agentic AI requires auditability, accountability and human oversight.
• Resilience depends on structural redesign and tested continuity under stress.
• Current financial models fail to capture the full value of digital health.
• Aligned innovation, resilience and financing determine whether care is reinforced.

Health systems are not short of innovation. They are approaching the limits of managing its consequences clinically, operationally and financially.

As digital technologies and artificial intelligence become embedded across care pathways, operational workflows and system infrastructure, the challenge is no longer access to capability (European Commission 2022a). It is the ability to ensure that increasingly complex, interdependent systems remain safe, governable and aligned with clinical reality.

This marks a structural shift. For over a decade, digital health has been defined by expansion: more data, more connectivity, more automation. Yet in practice, the constraint is no longer technological progress, but the capacity of systems to remain in control under conditions of increasing complexity. As capabilities scale, so do dependencies, risks and unintended consequences. What emerges is not a failure of innovation, but a transition from adding tech layers onto legacy systems to governing what those systems do.

This transition is unfolding under sustained pressure. Ageing populations, workforce constraints and rising expectations are accelerating digital adoption (OECD/European Commission 2024). Yet when deployment outpaces integration, technology does not resolve fragmentation, it compounds it.

This article takes a systems-level perspective of that tension. It examines how innovation, resilience and financial architecture interact in real-world health systems and why their alignment, not their individual strength, determines whether digital health delivers fit-for-purpose care.

Innovation and Clinical Risk. From the Limits of Safe Automation to Clinical-Grade Systems

Innovation in health has moved beyond experimentation. AI-supported decision tools, predictive analytics and automated workflows are now embedded in clinical and operational environments (Topol 2019). As systems begin to act, not just inform, the nature of innovation changes. A critical distinction is emerging between assistive AI and agentic AI. Assistive systems support clinicians by generating insights or recommendations. Agentic systems initiate actions such as triaging patients, triggering workflows, allocating resources or navigating care pathways with limited human intervention. This shift from advisory to action-oriented systems fundamentally alters how decisions are made, executed and governed (OECD 2026b).

The trajectory of this shift is not incremental. In The Coming Wave, Suleyman and Bhaskar argue that AI represents a compression of scientific and technological progress so rapid that the distance between capability and its consequences is collapsing; what once took decades now takes years, and what takes years will soon take months. In healthcare, this compression is already visible: diagnostic models trained on millions of cases, drug discovery platforms accelerating molecular screening, agentic systems beginning to navigate clinical pathways with limited human oversight. The potential is real. So are the consequences of deploying it without governance frameworks capable of matching its pace. But the same properties that make these systems powerful, i.e. speed, scale, autonomy, make their failure modes categorically different from anything clinical governance frameworks were designed to manage.

Suleyman's containment problem is particularly acute in health. The wave, he argues, cannot be stopped, it can only be shaped (Suleyman 2023). Yet shaping requires institutions capable of governing what they cannot fully understand, and deploying oversight mechanisms before, not after, capabilities embed themselves in critical systems. In digital health, the window for that governance is narrowing. Agentic systems that triage, allocate and recommend are already moving from pilot to deployment. The asymmetry between the pace of capability and the pace of governance is not a temporary lag but a structural risk. In the context of rising geopolitical instability and dual-use exposure, that asymmetry is no longer a patient safety question. It is a question of who controls the systems, and to what end. It is no longer a question whether technology performs, but how decisions are executed, by whom, and within what boundaries of accountability (WHO 2021).

Clinical safety frameworks were not designed for this shift. Established standards such as those enforced by regulators like the Care Quality Commission (CQC) and operationalised through clinical risk management frameworks, eg DCB0129 (NHS Digital 2018a)/DCB0160 (NHS Digital 2018b) in the UK, require that digital systems used in care delivery are demonstrably safe, validated, auditable and governed throughout their lifecycle. These standards assume traceability of decision-making, defined accountability and the ability to intervene when systems behave unexpectedly (Schmidt et al. 2024).

Agentic systems challenge each of these assumptions. Evidence from human factors and clinical informatics shows that increased automation introduces specific failure modes, including automation bias, reduced verification and inappropriate reliance on system outputs. In clinical practice, this shifts risk from isolated human error to distributed system error, where failures emerge from the interaction between human judgement, algorithmic output and operational context.

Five governance requirements for clinical-grade AI systems

1. Human checkpoints built into system architecture not as contingency measures, but as mandatory decision gates for high-stakes clinical actions, ensuring clinician authority is structurally preserved.
2. Full auditability of decision pathways. Systems must generate traceable records of how outputs were produced, enabling retrospective review and accountability assignment.
3. Explainability at the point of use. Outputs must be interpretable by the clinicians acting on them, not only by developers or auditors.
4. Defined and documented accountability. For every automated action or recommendation, a named role, team or governance body must carry accountable responsibility.
5. Intervention and override capability. Health organisations must retain the technical and operational means to suspend, override or roll back system actions when behaviour is unexpected or unsafe.

Safety, therefore, cannot be inferred from performance. It must be designed, validated and continuously governed. Taken together, these requirements define what clinical-grade governance of AI systems must look like in practice (Sendak et al. 2020).

In this context, innovation must evolve toward clinical-grade systems; systems that operate within clearly defined parameters of safety, oversight and regulatory compliance.

As digital capabilities extend into infrastructure, they expand not only what systems can do, but what they are exposed to. This transition brings resilience into focus, not as a secondary consideration, but as a defining condition of safe innovation.

Resilience and Operational Risk. From Digital Exposure to Strategic System Sovereignty

Resilience is no longer defined by the ability to respond to isolated shocks. It is defined by the ability to maintain continuity of care under sustained and compound pressure, including pressure generated by digital dependency itself.

Six years after COVID-19, and despite an unprecedented surge in digital health solutions, there is limited evidence that health systems are structurally better prepared for large-scale disruption (WHO Europe 2023). Many have digitised processes. Few have fundamentally strengthened their ability to operate under crisis conditions.

Evidence confirms the depth of this gap. A comprehensive OECD assessment of health systems following COVID-19 identified three structural vulnerabilities — underprepared populations, understaffed workforces, and chronic underinvestment in prevention and infrastructure — and concluded that even the most advanced health systems were not resilient to the scale of disruption they faced (OECD 2023). Crucially, the report found that health systems spent less than 3% of total health expenditure on prevention prior to the pandemic, despite sustained rhetoric about investment over cost. This reflects a deeper issue: transformation has been pursued through layered augmentation, not structural redesign.

The investment data reflects this pattern. While spending on ICT equipment and software in the health sector has grown faster than other areas of health capital investment over the past decade — at 5.6% and 7.2% per year respectively — it still accounts for a minor fraction of the overall capital increase required to strengthen system resilience (Morgan et al. 2026). The structural orientation of health investment has shifted incrementally toward digital transformation, but spending levels on digital infrastructure remain low relative to system need.

Fragmented systems have been overlaid with digital tools that do not integrate into coherent workflows. The result is increased complexity, higher cognitive load for clinicians, and growing friction for patients. In practice, this creates new risks: clinical, operational, and systemic.

Too often, the response has been behavioural rather than structural, training healthcare professionals to tolerate what is, in effect, a design failure. This approach does not build resilience, it normalises dysfunction. Resilience cannot be trained into individuals when it is absent from system design.

Across Europe, the development of the European Health Data Space (EHDS) (European Commission 2022b) and the implementation of the EU AI Act (European Parliament and Council of the European Union 2024) are accelerating the creation of data-sharing ecosystems and regulatory frameworks. In parallel, the United States is advancing legislation on AI governance and health data infrastructure. These initiatives signal a strategic shift: health data and AI are now central to economic competitiveness, national security and public health.

This transformation introduces significant dual-use risk — the vulnerability of health infrastructure to exploitation beyond clinical purposes, including cyberattacks, data weaponisation and strategic economic competition — and demands robust cybersecurity and governance infrastructure. Connected digital health systems must be stress-tested before health data and AI capabilities become targets — whether through cyberattacks, data exploitation or broader strategic threat vectors (ENISA 2023). Health systems are increasingly exposed as critical infrastructure. Their vulnerabilities now extend beyond the clinical domain to civilian safety and democratic stability.

At the same time, governance maturity remains uneven. Advanced technologies are being deployed without fully developed frameworks for accountability, oversight or coordinated control. Recent OECD analysis underscores the depth of this problem: even where additional financial resources have been mobilised for health, it remains unclear whether they are being directed to those areas — governance, digital infrastructure, workforce resilience — that most require strengthening (Morgan et al.2026). As the OECD concluded in its post-pandemic resilience assessment, investing in health system resilience is not purely a matter of spending more but of spending better (OECD 2023).

The implication is direct: capability without governance is not resilience. It is exposure. Resilience is no longer a function of capacity alone. It is a function of preparedness, foresight and the institutional discipline to direct resources where they matter most. Health systems must be able to:

• maintain continuity when systems fail or are compromised
• understand and actively manage technological dependencies
• retain oversight over critical digital infrastructure and decision-making

Structured adversarial AI testing offers a practical and as yet unapplied mechanism for building this capacity. Recognising that health policy makers lack tools to test how their systems would cope under extreme stress, the OECD and the European Observatory on Health Systems and Policies developed a structured resilience testing methodology inspired by stress tests applied in the banking and critical infrastructure sectors, and piloted it across three European countries in 2023 (OECD/European Observatory 2024). Notably, the handbook identifies digital failures alongside pandemics, geopolitical conflicts and financial crises as distinct shock scenarios requiring structured testing. Red team and blue team exercises — long established in cybersecurity practice, where simulated attackers test defences against active defenders — represent the logical extension of this methodology into digital health infrastructure, and have yet to find meaningful application in this domain. Applied here, such exercises would involve deliberately simulating system failures, dependency disruptions or targeted intrusions to test whether continuity and oversight mechanisms hold under pressure. This approach shifts resilience from a stated capability to a demonstrated one.

This is the basis of strategic system sovereignty: a health system's capacity to retain independent control over its data, infrastructure and decision-making under sustained pressure — not as a policy aspiration, but as an operational requirement. Resilience is achieved when embedded in preparedness and response strategies and continuously tested under real-world conditions, not when systems perform as expected, but when they do not.

Value and Financial Risk. From Cost Centres to the Health Dividend

If innovation defines what is possible, and resilience defines what holds under pressure, financing determines what becomes embedded in practice.

Health systems continue to operate within financial models designed for a different reality, focused on cost containment, short-term cycles and fragmented procurement. Recent data captures the scale of this misalignment. Following the post-pandemic fiscal contraction, average real per capita health spending across OECD countries fell by around 2% in 2022 and stagnated in 2023 even as demand grew, with public debt now projected to reach 113% of GDP by 2027 (Morgan et al. 2026). Refocusing health spending where it creates most value, rather than increasing volume, is rapidly becoming the only viable policy response.

The problem is not that health systems lack ambition for digital transformation. It is that the financial logic governing investment decisions was built for a world where value is transactional, immediate and contained within organisational boundaries and digital health is none of those things.

The conditions in which this misalignment now operates have become structurally hostile. Across health systems, the post-pandemic period has not delivered the structural investment that COVID-19 appeared to mandate. Instead, it has exposed a long-standing pattern of deferred capital expenditure, workforce attrition and chronic underinvestment in prevention, a pattern that predates the pandemic and has deepened since. Health systems entered the digital era already running on depleted foundations: ageing infrastructure, understaffed workforces, and prevention budgets that accounted for less than 3% of total health expenditure even before fiscal consolidation began (OECD, 2023). In this context, innovative financing models are not a policy preference; they are the only viable mechanism for closing the gap between what health systems need to function safely and what conventional public financing can sustain.

The returns from smarter digital investment are not speculative. OECD analysis estimates that operational spending on health data infrastructure — through interoperability, AI-driven analytics and robust governance — could yield a return of approximately 3:1 through efficiency gains, improved outcomes and avoided crisis costs (Morgan et al. 2026, citing OECD 2019). Yet health systems have tended to remain data-rich but information-poor, with fragmented systems that limit the ability to make timely, coordinated decisions, precisely because the financial architecture required to capture and attribute that value does not yet exist.

Digital systems create value cumulatively through improved outcomes, operational efficiency, coordination of care and reduction of low-value activity. Yet existing payment mechanisms have not kept pace: evidence indicates that fragmentation between care settings, combined with payment models misaligned with long-term benefit, continues to undermine the sustainable implementation of digital health services (OECD 2025). These benefits are diffuse, delayed and cross-organisational by nature, accruing across providers, payers and systems over time rather than within the budget cycles or organisational boundaries through which most health investment decisions are made. The evidence base for the economic value of digital health interventions remains limited, in part because available frameworks struggle to capture benefits that span multiple stakeholders and extend well beyond the point of deployment (Wilkinson et al. 2024). This structural mismatch — between where value is generated and where costs are accounted — is not a failure of digital health itself, but of the financial architecture built to evaluate it; a gap that value-based care models, where implemented, have begun to close.

Financial constraints remain a primary barrier to integrating digital innovation, even where clinical and operational benefits are demonstrable. This reflects a deeper structural issue: innovation is advancing faster than the financial logic required to sustain it. This is where the concept of the health dividend becomes central.

Investment in health, particularly in digital infrastructure and system transformation, generates returns beyond healthcare itself. These include increased productivity, reduced economic burden of disease, improved workforce participation and stronger societal resilience.

Health is not a cost centre. It is a strategic economic asset. This framing is now gaining institutional traction: the OECD Business Forum has called explicitly for recognising health expenditures as investments with long-term societal and economic returns rather than costs, and for transparent frameworks to measure ROI not only within the health sector but in terms of broader economic and social impact, including through cross-government coordination between health, finance, labour and innovation ministries (OECD/BIAC 2025). Realising this dividend requires a shift in financial architecture. Health systems must move toward innovative financing models that reflect evolving care delivery, including:

• the transition from inpatient to outpatient and community-based care
• the reduction of low-value interventions
• the integration of digital-first and hybrid care models that demonstrate ROI
• outcome-based and value-based payment and procurement structures

Financial decisions shape system behaviour. What is funded is implemented. What is implemented becomes standard. This makes financing a determinant of system evolution.

Crucially, financial sustainability is inseparable from clinical safety. Systems that fail to manage risk effectively generate downstream costs, through inefficiencies, adverse events and operational disruption, that erode long-term viability.

Sustainable health systems are not those that spend less, but those that invest in ways that reinforce safety, continuity and long-term performance.

Health systems are not held together by technology. They are held together by their ability to remain safe, accountable and operational as complexity increases. What emerges is not a question of digital maturity, but of system integrity. In the age of agentic AI, system integrity has three non-negotiable dimensions: safety, sovereignty and sustainability.

As innovation accelerates, dependencies deepen and financial pressure intensifies, the central challenge becomes maintaining coherence across clinical care, infrastructure and decision-making. Innovation alone does not strengthen systems. Resilience alone does not sustain them. Financing alone does not transform them. It is their interaction, under real-world conditions, that determines whether digital health reinforces or destabilises care delivery.

As health systems become critical digital infrastructure — holding population data, running agentic decision systems, connecting across jurisdictions — the question of who controls them becomes inseparable from the question of whether they are safe. Sovereignty is not a geopolitical abstraction. In connected health systems, it is a precondition for safety and a prerequisite for sustainability.

The cost of not acting is already being borne, though rarely attributed. AI hallucinations in clinical decision support introduce diagnostic error at a scale no individual clinician could produce — automation removes the verification instinct that human judgement provides (Lyell et al. 2017; Cabitza et al. 2017). Agentic systems that misfire do not produce isolated incidents: they produce distributed failures across every patient pathway they touch. Cyber vulnerabilities in connected health infrastructure do not interrupt a service. They can halt entire hospital networks, compromise patient records at population scale, and constitute direct threats to civilian safety (ENISA 2023). The proliferation of off-the-shelf digital solutions — acquired through procurement processes unequipped to evaluate them, deployed into workflows never redesigned to absorb them — generates a cost less visible but equally real: the cost of complexity without coherence, of technology that adds cognitive load without adding capability, of systems that were promised disruption and received fragmentation instead — and in doing so, compromised the very safety, sovereignty and sustainability that connected health systems depend on to function.

Set against these downstream costs, investment in genuine system redesign (ie, governance infrastructure, clinical-grade validation, workforce capability and financial architecture) is not the expensive option. It is the cheaper one. It is also the only path to financial sustainability that does not require perpetually increasing volume to compensate for preventable cost. The real fiscal risk in digital health is not over-investment in transformation. It is the accumulated cost of under-investing in the conditions that make transformation safe, sovereign and sustainable, and therefore real.

Systems that hold are not necessarily the most advanced, but the most coherent: those able to integrate innovation without compromising quality and safety, absorb exposure without losing continuity, and invest without disconnecting from care realities.

The question is no longer whether health systems can become more connected. It is whether they can remain in control of what that connection produces.

Conflict of interest

The author declares no conflict of interest.