How a Singapore hospital uses holograms to assist surgery

Interview with Associate Professor Ngiam Kee Yuan, Group Chief Technology Officer, National University Health System, Singapore.

Surgeons will be able to see the MRI scan of a patient’s brain as they operate on it. Nurses will identify veins by looking underneath the skin. Surgeons-in-training will experience an operation being carried out from the perspective of the senior doctor.

All these opportunities have been opened by the Mixed Reality technology developed by Singapore’s National University Health System (NUHS). Using Microsoft’s HoloLens 2 device, NUHS has created a new way of providing care and educating colleagues.

GovInsider caught up with Associate Professor Ngiam Kee Yuan, Group Chief Technology Officer at NUHS on the sidelines of the device’s recent launch. He shares the benefits that holographic tech offers and the challenges they face in wider adoption.

Precision in surgery

Traditionally the surgeon looks at the scan on the screen, then he looks at the patient's anatomy, says Ngiam. It would be left to the skills of the doctor to “approximate where a particular object is on the patient's anatomy” from what they see on a screen, he explains.

But this extra effort is no longer so essential. The Mixed Reality device allows for surgeons to see a holographic representation of objects as they conduct surgery, even placing the image over the patient’s anatomy.

The technology doesn’t require constant connection to a computer or laptop, meaning greater mobility, doctors shared at the announcement. The device is also controlled using hand gestures, meaning that it can be used hygienically in the operating theatre.

The device can use AI to identify objects in the room, for example different medical tools. For new medical staff who must attend various different kinds of surgeries, the device can point out specialty specific tools, helping them deal with the learning curve much quicker, says Ngiam.

The HoloLens also features an infrared emitter and camera. With processing algorithms, it can reveal certain objects under the skin, such as veins. This will assist with tasks such as blood-taking, NUHS doctors revealed.

Training junior doctors 

Attendees at the NUHS conference were able to see the user experience as doctors streamed video straight from the device’s camera. This has multiple applications for training and supporting junior doctors.

Medical students used to stand behind doctors, trying to peek over their heads during an operation and see what's actually happening. But now the device’s camera provides “a true front row seat” from the surgeon’s point of view, says Ngiam.

Sometimes a junior doctor may need support in the operating theatre. The device allows senior doctors to monitor the situation and advise accordingly. “This allows the junior doctor more confidence in what they're doing,” he says.

For patients, this device can help build “informed consent”, said the announcement. Patients will be able to better understand various procedures as they share a digital space with doctors, seeing scans and images in 3D.

NUHS plan to make the device commercially available. As it gets adopted internationally, doctors expect that it will help bridge the gap between healthcare in rural and urban areas. Local GPs will be able to collaborate with specialists in hospitals.

Laying the groundwork

The device relies “on the fact that you need to have secure methods of connecting and sending data from the cloud to the HoloLens”, says Ngiam. Creating a secure data system is one of the requirements needed for the programme, he says.

One key challenge is ensuring there is sufficient bandwidth to have the devices running smoothly. Insufficient bandwidth leads to lag, making the device less reliable. As a result of this, establishing 5G across the hospital environment has become a priority, the announcement said.

When asked about other exciting technologies coming to NUHS, Ngiam highlights the importance of the cloud. While it has always been able to develop and host new applications, the “game-changer” is expanding their use through the cloud, he says.

The device’s software and applications demonstrate an innovative step forward in Singapore’s medical technology. While its main use will be enabling more precise surgery, the holographic device offers a range of other services to help healthcare delivery in the 21st century.

Images from NUHS