Healthcare systems face compounded pressures from financial strain, workforce shortages and the rising burden of non-communicable diseases. Digital health and artificial intelligence technologies (DHAITs) are often positioned as part of the solution, with potential to improve access, sustainability, efficiency and quality. Yet adoption remains uneven because evaluation models have not kept pace with iterative, adaptive tools, particularly those powered by AI. Many current approaches were borrowed from pharmaceutical and medical technology assessments and do not reflect the realities of rapidly evolving software. What healthcare needs is an evaluation consensus that enables consistent, context-sensitive decision-making across jurisdictions while allowing national flexibility. Without such a foundation, innovation risks stalling, safety may be compromised, and value remains unproven at scale.
Distinct Needs of HCP-Facing Tools
Evaluation frequently treats digital health solutions for patients and for healthcare professionals (HCPs) as if they were interchangeable, but their aims diverge. Patient-facing tools are judged on direct clinical outcomes and adherence to standards of care. HCP-facing tools, by contrast, are designed to support those who treat rather than those being treated. Their primary goals include improving performance, streamlining workflows, reducing unwarranted variation and enhancing provider efficiency. Applying patient-centred criteria to professional tools obscures these objectives and constrains progress.
Must Read: Evaluating Large Language Models in 2025
This mismatch is reinforced by health technology assessment protocols that are predominantly patient-focused. As populations live longer with complex needs, HCP capacity becomes a bottleneck. Integrating non-human actors such as AI is therefore a practical necessity, but it cannot proceed responsibly without appropriate evaluation frameworks. Tools used by professionals require context-sensitive evidence that captures effects on decision quality, throughput, coordination and system performance over time. When such evidence is missing or misaligned, products proliferate without a clear demonstration of long-term value, remain short-lived and fail to scale within real-world services.
Generative AI and Evaluation Complexity
AI introduces profound uncertainty into traditional assurance models. Unlike deterministic software or devices, AI components need not behave consistently across cases. Tracing how outputs are produced from inputs can be difficult, and opacity is unwelcome in settings where certainty and transparency matter.
The contrast between discrete classification and generative outputs illustrates the challenge. A radiological tool that flags disease presence or absence invites a focused quality assurance regime. A large language model (LLM) supporting psychiatric care can generate a practically infinite range of responses. Attempting to verify the safety and quality of every possible recombination through conventional methods is infeasible. Evaluation must therefore shift from exhaustive output checking to principled control of development processes, operational safeguards and performance boundaries.
Progress depends on an evidence-based taxonomy that decomposes DHAITs into dimensions where risk and bias arise. Breaking technologies down clarifies how they are built and where targeted requirements should apply. A framework grounded in a common foundation can set consistent, nuanced expectations for functionality, performance and risk across diverse tools. Such clarity helps distinguish what should be tested, monitored or constrained at each stage of the lifecycle and reduces ambiguity for developers, evaluators and implementers.
Building a Common Evaluation Ground
Regulators often reference the International Medical Device Regulators Forum (IMDRF) when classifying digital medical products, including software as a medical device. Nevertheless, jurisdictions have developed their own schemes, such as tiered models in the United Kingdom and France or delineations of clinical decision support in the United States. The result is a piecemeal approach without a shared interpretation of AI-related risks. Because risk classification embodies normative choices about values, divergent foundations lead to fragmentation that hinders deployment, comparability and scale.
An international classification framework can provide a stable taxonomy and risk logic that countries implement consistently while retaining flexibility on benchmarks and thresholds to reflect local health system priorities. This balance is vital for public benefit and market feasibility. Predetermined Change Control Plans (PCCPs) demonstrate how a unified approach can align innovation with oversight by allowing pre-approved algorithmic modifications within defined limits, reducing repeated submissions for anticipated updates that do not alter performance. Jurisdictions that do not adopt such mechanisms risk longer re-clearance cycles and reduced competitiveness compared with peers that enable pre-authorised adjustments.
Evidence generation must also evolve. Randomised controlled trials (RCTs) remain important but are not always suited to rapidly iterating technologies. Economic evaluation that relies on quality-adjusted life-years (QALYs) may miss benefits of HCP-facing tools that are indirect or not directly tied to patient health outcomes. Aligning methods with the realities of digital iteration and professional use will better capture operational, safety and system-level effects needed for procurement, funding and reimbursement decisions.
Leadership is currently diffuse. No single international body has assumed a coordinating role for AI in the way IMDRF has for medical devices. Global organisations could convene regional efforts to establish classification coherence and accelerate an evaluation consensus. A co-ordinated pathway would help avoid a splintered regulatory landscape that leaves some countries behind and would build the trust necessary for responsible deployment within health systems.
Healthcare cannot afford evaluation approaches that lag behind the technologies they are meant to govern. HCP-facing tools require criteria that reflect performance, workflow and system impact, not solely patient-oriented endpoints. Generative AI multiplies complexity, making taxonomy-led, process-aware evaluation essential. A common international foundation for classification and risk, paired with mechanisms such as PCCPs and methods that accommodate iteration, will reduce fragmentation and support safe, scalable adoption. By advancing a flexible evaluation consensus, healthcare leaders can unlock the promised gains in access, sustainability, efficiency and quality while ensuring that innovation serves patients, professionals and health systems in practice.
Source: Healthcare Transformers
Image Credit: iStock
