Better Care for COVID Patients Through Virtual Reality

Since early March, New York City has suffered the most acute outbreak of the COVID-19 pandemic in the United States. As of the time of writing, the NYC Department of Health has recorded 171,723 cases of COVID-19 — well over ten percent of the 1,171,510 national cases reported by the CDC.

Kevin Ching, a physician at Weill Cornell Medicine, New York Presbyterian Hospital, realized that the knowledge hard-won by medical workers in New York City might save lives elsewhere, in areas of the country that haven’t yet been impacted by the pandemic. Ching wanted to create a VR video that demonstrated best practices, especially for medical personnel operating in regions and communities without a large teaching hospital.

Ching reached out to Kyle Cassidy, a staff member at Penn’s Annenberg School for Communication, to discuss a fast-tracked collaboration with Weill colleagues Neel Naik and Amos Shemesh. Earlier this year, Cassidy received a Penn Model of Excellence Award for his transdisciplinary work with virtual reality (VR), and one of Cassidy’s past projects was readily translatable to the COVID-19 training that Ching, Naik, and Shemesh were envisioning.

Cassidy had partnered with Penn Nursing in 2018 to create a VR Narcan tutorial for medical professionals. Narcan is a nasal spray which can reverse an opioid overdose, and its administration requires training. The VR Narcan tutorial, in fact, proved to be as effective as in-person instruction.

In March, when the possibility of a campus lockdown in response to COVID-19 became a certainty, Cassidy checked out VR cameras from the Penn Libraries’ Vitale Digital Media Lab with the sense that they might prove useful. 

Cassidy has a long working relationship with Vitale manager David Toccafondi, with whom he co-founded the Penn Working Group on Virtual Reality and with whom he collaborated on the Narcan project. Toccafondi recognized the potential of VR when it first became commercially available and procured two Oculus Rift headsets for use by anyone on campus. 

This early investment has resulted in an advanced degree of VR expertise among Vitale staff. “From the Media Lab’s inception,” says Toccafondi, “it’s been my goal to make available technology that people wouldn’t normally have access to.” Another priority has been the creation of interdisciplinary communities, both on Penn’s campus and beyond. “Technologies like VR cut across every discipline,” Toccafondi says. “The centralized nature of the Libraries brings together people in the Media Lab who might never meet otherwise.” 

Toccafondi cites Cassidy’s collaboration with Ching as a perfect example. Ching drafted a script with colleagues from Cornell. Meanwhile, Cassidy coordinated with Ann Marie Hoyt Brennan, Director of the Helene Fuld Pavilion for Innovative Learning and Simulation at Penn Nursing, and Lauren Weinberg Conlon at Penn Medicine to cast “actors” — real nurses and doctors — for the video. And Toccafondi arranged with Vitale staff member Christopher Vandegrift to prepare for editing the footage into its final form.

“So you have people from Weill Cornell Medicine, Annenberg, Penn Medicine, Penn Nursing, and Penn Libraries working on a project to support medical personnel and serve patients nationwide,” says Toccafondi. 

A little over a week from Ching’s first email, Cassidy and the medical staff convened in the Emergency Department at the Hospital of the University of Pennsylvania to film, with Conlon serving as the principal actor. The end result is an open access, 360° training video, available on YouTube. 

“The video is geared towards medical workers,” explains Cassidy, “but it can be viewed by anybody who's curious about what it's like to be in an actual room in the ER when a COVID-19 patient comes in and needs to be intubated and put on a ventilator.” 

Users can watch the video on their computers and survey all 360 degrees of the simulation by clicking and dragging, or they can view it with a virtual reality headset (such as Google Cardboard, a relatively inexpensive option) and simply turn to observe the team in action. Pop-up text describes best practices as they’re demonstrated by the team, such as “Keep all equipment out of patient rooms” and “Optimize first attempt success: most experienced provider should intubate.”

The project meets Ching’s original objective. “This video sets forth best practices for crisis teams developing resuscitation protocols and workflows in a clear, concise manner,” he says. “It presents evolving standards of care not yet found in medical literature but developed through real-time, clinical practice, based on the experiences of emergency providers working with the most critically ill COVID-19 patients in New York City and Philadelphia.” 

Though the video has been made publicly available, Cassidy’s work isn’t finished. Together with his colleagues at Annenberg and using tech from Vitale, Cassidy has already produced and debuted one of two planned COVID-19 VR trainings for Penn Medicine and the Children’s Hospital of Philadelphia. These simulations — which are compatibly interactive with the conferencing platform BlueJeans — are geared towards medical students in remote classes.

As for the original project, Cornell has invited Vandegrift to present on the original COVID-19 video at this fall’s TeleSimulation medical conference. 

“It’s recognition well deserved,” says Cassidy. “I’m grateful to the Libraries for giving me the opportunity to do this work. If it wasn’t for Dave, I wouldn’t have had the cameras in my living room — and if it wasn’t for Chris, we wouldn’t have been able to turn this around so quickly.”

Toccafondi is likewise grateful to Penn and the Libraries for supporting Vitale’s mission. “The Media Lab isn’t a ‘traditional’ library resource, but it’s a natural extension of what libraries do,” he says. “The Penn Libraries realized early on that research, knowledge creation, and the type of projects people work on in the twenty-first century require technologies that go beyond traditional information resources. We’re really fortunate to be at a place that supports this type of innovation because it allows projects like this to happen.”