Summary: Computer scientists in Pisa are developing Virtual Reality software that shows how 3D technology from dentistry can be applied to the imaging space.
Virtual Reality (VR) is an increasingly applied technology in medicine and surgery. The possibility for the user to experience a totally immersive three-dimensional environment, together with real-time activity simulation, makes VR an interesting tool both for medical training and surgical planning. In the last few years applications have been proposed in several surgical fields ranging from orthopaedics to vascular surgery (Kusumoto et al. 2006; Hirsch et al. 2013; Jou et al. 2001).
The application of haptic-based simulation with VR increases the close-to-reality effect of the virtual environment, reproducing the surgical setting, thus supporting preoperative planning and improving the understanding of the procedures (Junlei Hu and Xiaojun 2018).
Most frequently, the VR system includes a haptic device to be worn by the operator, which replicates the movement on a screen (Figure 1). The higher the visual rendering bandwidth, the better the rendering, especially in cases of complex scenarios where several elements need to cooperate.
In oral and maxillofacial surgery, VR simulators have been developed for implantology, trauma, and orthognatic surgery (Junlei and Xiaojun 2018; Kusumoto et al. 2006; Hirsch et al. 2013; Csaszar and Niederdellmann 1999; Elby et al. 2017; Buchanan 2017). To our knowledge, however, VR has never been used for implant planning or for CAD design.
A new VR tool for addressing the latter application has been developed in which the operator represents 3D using the 3 Cartesian axes XYZ, where the z-axis indicates depth. This allows us to represent the 3 planes of space on a sheet or on a monitor which by their nature are 2D, that is, they do not have depth. In this way the operator uses a graphical representation that is difficult to manage and requires a steep learning curve so that any modification on one plane is adequately represented in the other two.
However, if 3D CAD data are converted into augmented or virtual reality by using a 3D viewer, a realistic representation is obtained that simulates reality: an object will be seen by the operator as if it were real and with depth (Chi-Hsun et al.2018; Bakr et al. 2017).
The advantage of the idea is that the STL data (files used in CAD-CAM) are converted into VR, then the implants are gripped by the operator’s hands which are equipped with special sensors. The implants can therefore be positioned inside the jaw with the same ease with which a fischer-type screw is inserted into a wax structure. The position of the implants is thus planned, ie their coordinates (Figure 2), are saved and then everything is automatically converted into CAD which can be sent to anyone who wants to further view or design in a traditional way.
All this is applicable to any type of CAD, be it dental or other (eg hip prosthesis). You can draw anything immersed in virtual reality where you operate in a ‘virtually analogue’ manner without the mouse, but with your hands.
What is done in virtual reality can therefore be printed onsite or sent electronically to remote locations.
As it is currently designed, it can be viewed in virtual reality even through a normal smartphone: one needs merely to purchase special cardboard boxes (eg Google Cardboard) in which to insert the mobile phone that will act as a viewer, and download a free app.
Technological improvements on-going development of better performing computers means that CAD nowadays extends to an ever wider range of sectors.
The diffusion of information technologies in multiple forms of implementation has introduced computerised design systems capable of supporting both prototyping processes and the simulation of dynamic actions more or less articulated in countless areas.
Multiple skills are contained and made available within the technology commonly recognised with the acronym CAD, which generally identifies a set of IT tools to support the design activities.
By means of the various functional units that make up the electronic processor, such as mice and keyboards, the execution of the appropriate assisted design algorithms is possible. At the same time, thanks to a series of human-machine interfaces it offers the possibility of interacting with the software in the various virtual model manufacturing processes.
This technology, which has traditionally played a decisive role in mechanical, electronic or architectural engineering, currently encompasses many branches of medicine and various applications ranging from diagnostics to surgical simulation up to the creation of custom-made prostheses or surgical devices (Csaszar and Niederdellmann 1999).
Specific software technologies are used for this purpose, proposing a workspace in which digital models are automatically created and/or imported from a real model (Aras MA and D’Souza 2012).
In the first case, the digital models are already designed in numerical format as they are created by special software capable of drawing one or more entities. In the second case, they are generated by direct acquisition of a model thanks to the use of special capture devices, such as a simple digital camera (Photogrammetry), a 3D scanner and even a common 3D medical device, such as Computerised Tomography (CT) or Magnetic Resonance Imaging (MRI) (Marchal et al. 1996; DeLorenzo et al. 2009).
A limit of this known technique derives from the need of the CAD or 3D imaging to represent objects which in real space are three-dimensional (such as for example the aforementioned display) but in a two-dimensional environment.
Improvement can be obtained by working on several planes, usually orthogonal to each other, by means of multiple views possibly displayed side by side of the objects being studied.
However, it is evident that this limit is only partially obviated, and that it becomes even more marked when in addition to simply presenting an object or a compound assembly one must also modify, assemble or break it down.
As a consequence of these constraints, the operator operates with greater difficulty, taking a long time and obtaining results of lower quality or that do not fulfil the processing requirements (Clancy et al. 2002).
The appearance of 3D design software, the ability to produce artefacts using milling and printing techniques (industrial and in-house), and the availability of digital data collection methods (intraoral scanning, CBCT) have all significantly improved to the point that a new era of dental implantology has begun (Marchack 2017; Kattadiyil et al. 2014).
Thanks to the ability to customise screens, the software can show axial, coronal, sagittal, cross sections as well as panorex and 3D reconstructions and allows users to obtain the superposition of the DICOM data with the STL files, obtained through intra or extra oral optical scans. The software allows the selection of the most suitable site, shape and length of each implant, and predicts the need for, and required quantity of, a possible bone graft (Ganz 2016; Jacobs et al. 2015).
Further progress in dentistry is represented by computer-assisted surgery. Guided implantology is the result of the combination of clinical data, three-dimensional imaging, and CAD/CAM design of surgical guides (Kattadiyil et al. 2014).
This technique involves pre-surgical planning that is currently supported by the use of 3D diagnostic techniques for the study of bone and surrounding soft tissues in multiple cross-sectional views. It provides the possibility to first draw and then use a customised surgical template in which there are cylinders (sleeves) that constrain the direction of the drills during osteotomy and the optimal positioning of the implants in compliance with ideal axes and prosthetic components with the information of the final prosthetic drawing provided by the diagnostic wax-up (Buser et al. 2008).
The possibility of obtaining surgical guides and 3D models from CT images has dramatically improved the clinical precision in implant positioning with the possibility of pre-operative assessment of the size of the implant, ideal depth, and angulation (Arce et al. 2014). Moreover, prosthetically directed implant placement using dedicated software can ensure precise placement and predictable prosthetic outcomes in order to achieve aesthetic and functional success of the restoration. CAD/CAM technologies have been used to produce surgical guides (Kero et al. 2014).
The surgical template is a completely limiting guide that directs the drilling process and the subsequent implant positioning offering a significant advantage to the surgeon by improving precision and minimising complications such as mandibular nerve damage, sinus perforation, fenestrations, and dehiscences (Kattadiyil et al. 2014; Koop et al. 2013).
When working with 3D images, like CBCT, MRI, CT or CAD, it is difficult to represent the 3 Cartesian planes on a monitor. The difficulty is to build a 3D model and work on it on a two-dimensional screen (Jasinevicius et al. 2004).
Dr. Luigi Rubino introduced a software called VRubino, which simplifies a great difficulty that all physicians or CAD designers of any discipline have: designing computer interventions working with the 3 Cartesian planes but using a two-dimensional monitor, which by its nature is 2D, that is, it does not have depth. Dr. Rubino delegated the development of the software to a team of computer scientists from the ITALIA 3D ACADEMY School in Pisa.
Planning or drawing any element in 3D is very complex and it requires a steep learning curve. Understanding what happens on the x or z planes when moving an object on the y plane (or vice versa) is not so intuitive. With normal computers, the operation is lengthy and complex and it often discourages newbies.
This is why CAD drawing or implant planning in guided surgery is the prerogative of only few expert operators. On the contrary, with virtual reality, planning a surgery now becomes very simple. When the operator wears a viewer on his/her eyes and sensors with haptic-based simulation on his/her fingers, he/she can immerse into a virtual world. For example, the jaw that is seen by the operator wearing the viewer appears as if it were suspended in front of him/her. He/she can look at it from below or from above simply by lowering or raising it, respectively.
Thanks to the sensors in his/her hands (Figure 3) he/she can rotate it, move it or even take an implant from a library and position it with his/her hands in the ideal bone site giving it the optimal inclination on the x, y, and z axes (Ferreira 2002). This is done in accordance with the prosthetic project, ie, in accordance with the diagnostic wax-up which can be superimposed onto the image of the jaw by a simple movement of the hand. The position thus chosen can be saved and reported on the starting CBCT, after which one can proceed to the possible drawing of the surgical template, or it can be printed or sent as a 3Dfile wherever it is needed.
In other words, this method allows to ‘virtually bring the operator into an analogic world’ where he/she places the implants in the virtual jaw with his/her hands with the same ease with which it is possible to insert screws into a plasticine structure and then save their position.
Once the planning in VR is completed, all the obtained data is converted back to a traditional CAD CAM system, and one can proceed with the design of the surgical template and any 3D printing of what is planned.
Presently, VR technology is ripe. Although it does not require a large financial investment or expensive computers, VR in medical imaging is still not widespread enough.
Radiologists, CAD designers, dentists or anyone who has to work with the 3 Cartesian planes can easily and quickly transform the 3D digital representations into forms that are compatible with the virtual reality environment in which the visualisation, the simulation or the diagnosis is simpler.
The VRubino software has added a further new functionality: the possibility to 3D design and/or plan a guided surgery directly in the virtual environment.
It is a new, easier and more natural method that does not require a steep learning curve, it can facilitate and speed up the work of the more experienced operators and could encourage beginners to approach 3D imaging.
- Virtual Reality is increasingly being used in the medical setting.
- Computer scientists are developing a new VR tool for implant planning and CAD design.
- 3D design software has opened doors to a new era of dental implantology.
- Virtual Reality can improve diagnosis in the imaging setting.
Aras MA, D’Souza (2012) KMTypes of implant surgical guides in dentistry: a review. The Journal of oral implantology. 38(5):643-52.
Arce RM, Chenin DL, Mora MA (2014) Software tools and surgical guides in dental-implant-guided surgery. Dental clinics of North America. 58(3):597-626.
Bakr MM, George R, Roy E (2017) The need for virtual reality simulators in dental education: a review. Saudi Dent 29:41-7
Buchanan JA (2001) Overview of three years’ experience with virtual reality-based technology in dental education. J Dent Educ. 65(1):58(Abstract 148)
Buser D, Bornstein MM, Halbritter S, Harnisch H, Weber HP (2008) A retrospective analysis of patients referred for implant placement to a specialty clinic: indications, surgical procedures, and early failures. The International journal of oral & maxillofacial implants. 23(6):1109-16.
Chi-Hsun Y, Chun-Cheng H, Jen-Chyan W, Ta-Ko H, Yu-Hsin H, (2018) Augmented reality (AR) and virtual reality (VR) applied in dentistry Kaohsiung Journal of Medical Sciences 34, 243-248
Clancy JM, Johnson LA, Lindquist TJ, Palik JF (2002) A comparison of student performance in a simulation clinic and a traditional laboratory environment: three-year results. J Dent Educ 66(12):1331-7
Csaszar G R, Niederdellmann H (1999) Reliability of bimaxillary surgical planning with the 3-D orthognathic surgery simulator. The International journal of adult orthodontics and orthognathic surgery.15: 51-58.
DeLorenzo C, Flossmann S, Neff M, Papademetris X, Spencer DD, Vives KP et al (2009) From medical image computing to computer aided intervention: development of a research interface for image guided navigation. Int J Med Robot 5:147-57
Elby R, Mahmoud MB, Roy G (2017) The need for virtual reality simulators in dental education: A review The Saudi Dental Journal 29, 41–47
Ferreira MA, Gonzalez LF, Henn JS, Lemole GM Jr, Preul MC, Schornak M, Spetzler RF (2002) Interactive stereoscopic virtual reality: a new tool for neurosurgical education.Technical note. J Neurosurg 2002;96(1):144-9
Ganz SD (2015) Three-dimensional imaging and guided surgery for dental implants. Dental clinics of North America. 59(2):265-90.