AI-based prediction in clinical settings: can we trust it?

The challenge

Users in healthcare are facing the situation that it is becoming increasingly difficult for them to incorporate information available for their decisions in a limited amount of time.

Therefore it is necessary to provide relevant information for the respective decision situation – a context-sensitive presentation of a medical history. In a further step, then clinical decision support can be given.

Appropriate algorithms collect and link available information, they derive indications and recommendations for decision-makers (doctors, nursing staff, therapists, patients) and make it available at the point of care in real time.

You may also like: Digital transformation and the subconscious

But scepticism and mistrust of users have to be adressed: They must be enabled to gain confidence in such systems and trust their information by evident benefits (especially for the patients) and intuitive integration in their user interface.

An AI case already in clinical practice

In this case, a prediction for delirium is offered as decision support, based on statistical and machine learning models ("controlled" and "qualitatively evaluated"). These models are derived from extensive data from the hospital operating company KAGes with approximately 2.1 million longitudinal medical histories from the Austrian province of Styria with approximately 1.2 million inhabitants. In this area about 90% of the acute care beds are operated by KAGes.

Delirium is a highly relevant syndrome with elderly patients. Delirium means an acute, organically induced impairment of the brain. Affected patients respond inappropriately to environmental stimuli, seem "confused" and unable to orient themselves.

In up to 44% of cases, delirium is considered preventable. If left untreated, however, it will lead to a significantly higher risk of complications during hospitalisation or even death. According to studies, the hospital stay is considerably prolonged by a delirium suffered (Dowal 2015).

In a supervised learning process, models were generated with several algorithms. In order to avoid overfitting, a training (75%) and a test dataset (25%) were used. The models were optimised exclusively on the training data set. The performance was validated with the test data set. The different algorithms yielded very similar results. The Random Forest as the best algorithm in this use case achieved an AUROC of 0.90 (“area under the curve“ as statistical performance indicator).

Based on this model, the probability of suffering from delirium is calculated for each newly-admitted patient with data extracted from his medical history. In case a warning is displayed at the clinical work station, the explanatory component can then be activated by clicking on the warning symbol. This then shows the data of the patient which led to the calculated probability, eg the patient has a certain elevated laboratory value, a certain prediagnosis, an accumulation of disease events in the recent past, difficult social conditions, etc.

The attending physician or nurse can then take preventive measures to prevent the impending delirium if possible.

Benefits

The benefit for a hospital group such as KAGes can be seen in the case of widespread use as follows (estimated conservatively):

• Approx. 100,000 inpatients in the risk population (60+) per year
• With an assumed (low) incidence of 3%: 3,000 patients with delirium per year (assuming some potentially preventable delirium would have been detected early enough to prevent delirium)
• Average 7 days longer stay for preventable delirium, assumed €500 per day: additional costs of €10,500,000
• Predictive approach detects > 80% of delirium patients early and 25% (low estimate) are considered preventable.
• Cost savings potential per year: € 2,100,000

Further use cases are in development, eg prediction of avoidable readmissions, prediction of the intensive care necessity of surgical patients to improve the disposition of expensive intensive care beds, etc.

Trust in AI

The prediction of delirium was compared with the estimation of the delirium risk by staff members in a nine-month evaluation project proving the high predictive quality of the model in practice. The acceptance of the users was evaluated as well. This approach has allowed users to gain confidence in using this form of artificial intelligence. This very context-sensitive support in the explanatory component makes work easier for the users and increases patient safety. As a result, the senior physicians and nursing staff of the pilot departments have adopted the solution and have recommended it for wide use in KAGes.

Outlook

Further application modules are being developed and there is also interest of other hospital providers to use these models, which are based on an extensive body of data from longitudinal medical histories. There are numerous publications, which show good statistical metrics of their prediction models. But the trust of doctors and care staff proved by real clinical use is hardly found. The "last mile" to clinical use has obviously not been made yet. KAGes has already succeeded and is optimistic it will add more use cases.

User scepticism and mistrust have to be adressed and they must be enabled to gain confidence in such systems

In up to 44% of cases, delirium is considered preventable but, if untreated, will lead to complications

Key Points

• Healthcare users are experiencing information overload
• Algorithms provide point-of-care information on indications and reccomendations
• Machine learning is predicting and preventing delirium in an Austrian hospital
• The potential annual cost savings are €2,100,000
• Success has enabled user trust in AI

Next Article: Artificial Intelligence: a next way forward for healthcare