HealthManagement, Volume 23 - Issue 1, 2023
PRINT OPTIMISED
Key Points
- AI hold great promise as a screening tool in medicine
- Increasing evidence that AI is safe ethically and practically
- Further need for rigorous testing of AI software is paramount
Introduction
The Current DESP Screening Process
- No retinopathy (R0 M0)
- Background retinopathy (R1)
- Pre-proliferative retinopathy (R2)
- Proliferative retinopathy (R3)
- Maculopathy (M1)
- Inadequate / unassessable images (U)
Primary grading
- All patient’s images are initially assessed by a primary grader. 90% of all patients screened will have no DR (R0M0 grade) which is their final grading, and they will receive annual recalls.
Secondary grading
- All patients graded R1M0, R2M0, R1M1 and R2M1 pathology by primary grading go to secondary grading.
- First and second graders agree: For grade R1M0 (background retinopathy) this is complete and the patient receives a result and annual recall.
- The DESP software selects 10% of R0M0 patients, which are sent to secondary grader for quality assurance (QA) checks.
Referral Outcome grading (ROG) and arbitration
- First and second graders agree: For grades R1M1, R2M0 and R2M1 the referable pathology goes to ROG for a referral outcome decision.
- First and second graders disagree: This will go to arbitration for review. If arbitration grade is R1M0 the patient receives a result and annual recall. If the final grade has referable pathology identified (R1M1, R2M0 and R2M1) this goes to ROG for a referral outcome decision.
- All R3M0 and R3M1 pathology from any level goes directly to ROG, as this is urgent pathology and takes priority
Why Incorporate AI into the Screening Process?
The traditional screening process can be time
consuming due to several layers of grading, costly, and
requires the need for highly skilled graders who undergo
regular quality assurance and training. All human
graders must consistently demonstrate a sensitivity of
over 85%, and specificity of over 80% for identifying
referrable DR, and are routinely monitored (David Taylor,
2016).
AI may provide a cost-effective alternative to human grading to overcome these limitations and provide faster results. For instance, AI can be trained to identify patterns and anomalies in retinal images. By analysing these images, AI algorithms can detect potential medical issues and could act as a triage to separate those patients who have diabetic retinopathy, or other abnormalities, from those who have no retinopathy.
This could aid the providers in diagnosis and reduce their workload, which would allow them to focus their expertise more on higher risk patients.
Why Is InHealth Intelligence Working with AI, What Are We Hoping For?
InHealth Intelligence,
the leading provider of diabetic eye screening services in the UK, has collaborated with two AI based
companies, Thirona, based in the Netherlands, and Optos, based in Scotland, to
research the validity of using AI within the DESP service.
Results from the study with Thirona were published in 2023 (Meredith 2023).
InHealth Intelligence provided Thirona with 9,817 anonymised image sets which were processed by their deep learning artificial intelligence software. The sensitivity and specificity of the artificial intelligence system for detecting diabetic retinopathy was determined.
The results indicate that the artificial intelligence system was superior for no or mild diabetic retinopathy vs significant or referrable diabetic retinopathy where the sensitivity of the artificial intelligence grading system was 69.7% and specificity 92.2%.
The performance of the artificial intelligence system was superior for no or mild diabetic retinopathy vs significant or referrable diabetic retinopathy with a sensitivity of 95.4% and specificity of 92.0%. Significantly, no cases were identified in which the artificial intelligence grade had missed significant diabetic retinopathy.
The
collaboration between InHealth Intelligence and Optos is in the early stages;
100,000 images which have been completed by graders at InHealth Intelligence
have been shared with Optos. Optos have regraded the images using their AI. Any
grading outcome differences being re-graded by InHealth Intelligence to
identify the discrepancies and determine the sensitivity of the AI software.
What Are the Benefits?
AI could be used as a quality assurance tool in the primary
grading process. Approximately 90% of
patients screened in the DESP are negative for diabetic retinopathy, these
cases are graded by only one human grader in the. Adding in AI as quality
assurance would mean all images were graded by AI and at least one human grader.
Alternatively, AI systems could potentially take out a layer of grading. The results from the Thirona study are significant, notably no cases were identified in which the artificial intelligence grade had missed significant diabetic retinopathy. This is important for implementation into live grading; AI could be utilised as a first layer to filter patients with disease versus no disease patients. This would have impact in reducing the workload on grading; better utilising the specialist skills of dedicated human graders allowing them to focus on grading patients identified with disease. Additionally, this could benefit patients as AI can process vast amounts of data in seconds. This speed is critical in the early detection of serious medical conditions and could reduce waiting times to diagnosis for patients.
Reducing the workload on healthcare staff also has the bonus of reducing costs of the screening programme. The InHealth Intelligence and Optos clinical study is thereforeexploring if automated grading is clinically and cost effective for the NHS’ Diabetic Eye Screening Programme.
What Challenges Do We Face?
There are, however, some
challenges to the widespread adoption of AI-powered medical screening.
One of the biggest challenges is the need for high-quality medical data to train the AI algorithms. If the data used to train the algorithms is inaccurate or incomplete, the resulting diagnoses will also be inaccurate. AI can only identify what it has been trained to detect in images. To overcome this challenge, medical organisations need to ensure that they have access to high-quality, accurate data, from a diverse ethnic mix of individuals and populations, that can be used to train AI software to detect a wide range of diagnoses.
One can interpret from the results of the Thirona study that the AI system has a high sensitivity and tended to over grade the images. Although this has benefits in being overly cautious, it could result in increased referrals to the hospital eye service (HES) and added pressure on the health service.
Another challenge is the need for regulatory approval. AI-powered medical screening systems must undergo rigorous testing and be approved by regulatory bodies before they can be used in a clinical setting. This process can take several years and requires significant resources.
There are also ethical considerations to be addressed surrounding the use of AI in medical screening, and in healthcare overall.
There is concern about the potential for AI to be used to make medical decisions without human input. Should AI be incorporated into the screening process, patients must be fully informed of the grading process involving AI, and how their images are being used.
Finally, there are also concerns about the privacy of medical data and the security of AI-powered medical screening systems.
To address these concerns, it is important for medical organisations to implement appropriate security measures and to establish clear ethical guidelines for the use of AI in medical screening. Public Health England is working on guidance to help developers of artificial intelligence understand the process for incorporating their new technologies into screening programmes (Dunbar 2019).
Future Research
Further research is essential to provide healthcare systems with further confidence in using this technology, and to determine if incorporation of AI is a cost-effective solution.
The University of Liverpool recently announced a new spin-out company, AI Sight Ltd, that will commercialise a next generation AI system for diabetic eye screening (News 2023).
Their technology has been trained on over 1.6 million images. It is a highly sensitive and specific, web-based screening system that uniquely measures and displays the level of certainty of every automated image analysis. The system has the benefit of being easily integrated into different healthcare systems and is compatible with any retinal camera images.
Conclusion
AI holds great promise to advance medical screening and is attracting a lot of attention and investment. There is increasing evidence that AI systems are safe to use within diabetic eye screening. Whether AI is used to replace a level of grading or to assist with quality assurance, there is potential for AI to benefit patients and healthcare providers by providing fast, efficient diagnosis. Before an artificial intelligence system is to be incorporated within healthcare it must undergo rigorous independent evaluation.
Conflict of Interest
None.
Acknowledgement
InHealth Intelligence, Optos, Moorfields Eye Hospital and Queens University Belfast won the Artificial Intelligence in Health and Care Award runby the Accelerated Access Collaborative in partnership with NHSX and the National Institute for Health Research. The Award supports technologies across the spectrum of development from initial feasibility to evaluation within the NHS. The award is funding a clinical study to accelerate the implementation and validation of AI into NHS DESP and determine if automated grading is clinically and cost effective.
References:
David T S W(2016) The management of grading quality. Available fromAssets.publishing.service.gov.uk: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/512832/The_Management_of_Grading.pdf
Dunbar D K (2019)PHE Screening. Available from phescreening.blog.gov.uk:https://phescreening.blog.gov.uk/2019/03/14/new-guidance-for-ai-in-screening/
Meredith S G(2023) Performance of an artificial intelligence automated system for diabetic eye screening in a large English population. Diabetic Medicine.
News U (2023) New spin-out to deploy novel AI solution to reduce vision loss for people with diabetes. Available from https://news.liverpool.ac.uk/2023/02/02/new-spin-out-to-deploy-novel-artificial-intelligence-solution-to-reduce-vision-loss-for-people-with-diabetes/
