A Virtual Reality Surgical Guidance System
The human body is 3D, but the screens that visualize it are not. This becomes a problem during complex medical procedures when 3D information is condensed into 2D images. During this process, important information is hidden and sometimes lost, which makes performing complex procedures very challenging.
The above animation shows x-ray images of the heart during a complex medical procedure called the transseptal puncture. During the procedure, a sharp needle pokes a hole through the center of the heart. Like throwing a dart at a dartboard, the goal is to hit the center of a small target. However, unlike throwing darts, a miss can be fatal. A hole in the wrong part of the heart can cause life threatening issues, including cardiac tomponade — a condition that causes your heart to bleed (see image below). Avoiding such mistakes is paramount, but extremely challenging when looking at 2D images that do not show the heart’s 3D structure.
I wondered if VR could make procedures safer by preserving the three-dimentionality of medical information. So, during my dissertation work, I created CathEye — a VR surgical guidance system — like an immersive Google Maps for the human body.
CathEye takes the same information that’s displayed on flat screens and visualizes around the operating physician (see image above), giving them the ability to perform procedures as if they were inside of their patients. The experience was visually stunning and profoundly intuitive, but was it useful?
To understand CathEye’s benefits my colleagues and I asked doctors to perform a series of tasks in a 3D printed model using both CathEye and x-ray images (fluoroscopy). We found that doctors were able to understand anatomy better and perform procedures more accurately when using CathEye. In addition, novice doctors benefited more from CathEye than experienced doctors.
The results are important for a few reasons. First, they demonstrate the potential VR has to improve safety. Doctors were able to navigate their medical instruments more accurately and with greater confidence in VR as a result of the intuitive user interface. Several participants were so impressed they said they would use VR immediately if it were available during their real procedures. I think CathEye would help them avoid severe consequences, such as cardiac tomponade, by helping them avoid deadly punctures.
Second, these results shed light on a new opportunity for data collection. Data drives the development of new, safe technologies, such as CathEye. Collecting data is therefore important. Head-mounted displays, such as VR, track where users are looking, and accessories, such as controllers, gloves and body suits, add additional data. These new data streams can be used in future applications to unlock things that were not previously possible. I am particularly interested in understanding how these new types of data influence artificial intelligence and telerobotic procedures.
Finally, these results highlight a few areas to explore in future research, including user interface design, human-computer interaction, simulation/training for novice physicians, other procedures that can benefit from VR, and the economic impact of VR on insurance, hospitals, patients and families.
For more details please read the original abstract and the full journal article. The full journal article will be published by Feb, 2020.
[1]: Ryan James, Wayne Monsky, Stephen Seslar. Virtual reality versus fluoroscopically guided transseptal puncture in a cardiac phantom. Journal of Invasive Cardiology, accepted for publication.
[2]: Ryan James, Wayne Monsky, Stephen Seslar. Initial experience performing a simulated transseptal puncture with a virtual reality head mounted display catheter navigation system.
