New research shows that using augmented reality in the operating theatre could help surgeons to improve the outcome of reconstructive surgery for patients.
In a series of procedures carried out by a team at Imperial College London at St Mary’s Hospital, researchers have shown for the first time how surgeons can use Microsoft HoloLens headsets while operating on patients undergoing reconstructive lower limb surgery.
The HoloLens is a self-contained computer headset that immerses the wearer in ‘mixed reality’, enabling them to interact with ‘holograms’ – computer-generated objects made visible through the visor. In the UK however, headsets are currently only available for developers.
The technology was used to overlay images of CT scans, including the actual position of bones and key blood vessels, onto each of the patient’s leg. This allows the surgeon to essentially ‘see through’ the limb during surgery.
Following the procedure, the team involved in trialling the technology agreed that the approach could in fact help surgeons locate and reconnect key blood vessels during reconstructive surgery, which could then improve outcomes for patients.
Dr. Philip Pratt, a Research Fellow in the Department of Surgery & Cancer and lead author of the study, published in European Radiology Experimental, talks of the success of the technology, “We are one of the first groups in the world to use the HoloLens successfully in the operating theatre.
“Through this initial series of patient cases we have shown that the technology is practical, and that it can provide a benefit to the surgical team. With the HoloLens, you look at the leg and essentially see inside of it. You see the bones, the course of the blood vessels, and can identify exactly where the targets are located.”
Repairing damage
It appears as though augmented reality may offer an alternative form of reconstructive surgery. Rather than using handheld scanners which uses ultrasound to identify blood vessels under the skin by detecting the movement of blood pulsing through them, Dr. Pratt believes that augmented reality may be the answer.
“Augmented reality offers a new way to find these blood vessels under the skin accurately and quickly by overlaying scan images onto the patient during the operation.”
Dr. Dimitri Amiras, a consultant radiologist at Imperial College Healthcare NHS Trust (ICHNT), explained the benefit of this in an email to HealthManagement.org:
“For this surgical technique where the vessel perforates the muscle, fascia is crucial to the surgical planning. This anatomy is very patient specific and using handheld ultrasound scanners can sometimes be time consuming to perform. Previously we used rough measurements on the CT images but this proved difficult to translate to the operating table. This indeed was the problem we wanted to fix with this technology.
“Being able to do this is in a resilient fashion we managed to reduce the anaesthetic time and operative time performing dissection to identify these blood vessels for the patient. The ability to understand the patient’s specific anatomy during the procedure makes the surgeon more confident in his surgical approach, plan for a less invasive reconstructive flap and reduce the potential morbidity associated with a more extensive resection.”
Behind the scenes
In the lead up to the procedures, five patients requiring reconstructive surgery on their legs underwent CT scans to map the structure of the limb, including the position of bones and the location and course of blood vessels.
Dr. Amiras used the images from the scans to divide the bone, muscle, fatty tissue and blood vessels into intermediary software to create 3D models of the leg.
These models were then fed into specially designed software that depicts the images for the HoloLens headset, which in turn overlays the model onto what the surgeon can see in the operating theatre.
Clinical staff are able to manipulate these AR images through hand gestures to make adjustments and ensure that the model is lined up correctly with the surgical landmarks on the patient’s limbs.
Dr Amiras said: “Using the HoloLens, we can identify where the blood vessels are in 3D space and use virtual 3D arrows to guide the surgeon.
"Currently, data preparation is a time consuming process, but in the future much of this could be automated, with the consultant radiologist checking the accuracy of the model against the original scan.
"I think this is a great example of what can be achieved in an Academic Health Science Centre.”
The surgeon's view
Mr. Jon Simmons a plastic and reconstructive surgeon at ICHNT, led the team who carried out the procedures using the HoloLens headset and augmented reality models.
“The application of AR technology in the operating theatre has some really exciting possibilities,” said Mr Simmons. "While the technology can’t replace the skill and experience of the clinical team, it could potentially help to reduce the time a patient spends under anaesthetic and reduce the margin for error. We hope that it will allow us to provide more tailored surgical solutions for individual patients.”
The group highlights a few limitations with the technology, which could include errors during the modelling stages as well as the potential for the overlaid model to be misaligned.
Dr. Amiras explained, “Although the spatial localisation on the HoloLens is great for most uses, it will need to be improved for other surgical procedures where millimetres count. It’s a bit like comparing the GPS your phone uses to one the military may use.”
Indeed, it seem as though researchers are confident that once refined, the approach could be applied to other areas of reconstructive surgery requiring tissue flaps, such as breast reconstruction following mastectomy. The next steps include trialling the technology in a larger set of patients, with procedures carried out by teams at multiple centres.
“We will push this project forwards focusing on the use cases we have already demonstrated and looking to apply the technology to other areas. I would hasten to add that this is still early days for augmented reality in medicine and there are several stumbling blocks before we see it being used regularly in clinical practice.
“My feeling is the expertise comes in understanding the technology, the clinical problem and how to successfully translate this into improved patient outcomes. One of the main reasons we have been successful is the nature of the cross discipline team with radiologists, reconstructive surgeons and computer scientists working together to solve a clear clinical problem.”
The general consensus is that the HoloLens technology appears to be more practical. “With the HoloLens the relevant imaging information is visible for the surgeon to see without turning his head, clicking a mouse or scrolling through a CT stack. It’s like having the radiologist in the room with you. Being a hands-free device; able to take voice commands and hand gestures, the HoloLens leaves the operator's hand free to do a procedure,” according to Dr. Amiras.
Dr. Amiras explained that the AR hologram is the key to the success of this technique.
“If you imagine a surgeon visualises the anatomy prior to starting a procedure, the hologram now acts as the support which helps project that patient specific’s anatomy directly onto the patient, potentially making the surgery safer and quicker to perform.”
What’s clear is just how important patient safety is to everyone involved. The patient appears to be at the centre of everything the doctors do and every member of the team involved has something to add.
According to Dr. Amiras, “Open lines of communication between the surgeon and radiologist is the key to understanding each teams’ perspective and how they can help improve the patients’ outcome.”
Source: Imperial College London
Image Credit: Imperial College London
