The SCIEMENT Viewtify solution transforms a series of medical images into a 3D computer graphic shape that doctors can examine as a surface in real-time, providing unprecedented, intuitive understanding.
Video courtesy of SCIEMENT, Inc.
The challenge of 2D images
CT, MRI and echography are indispensable imaging tools that help guide medical practice. But they all generate 2D images of 3D subjects. According to Dr. Hirofumi Seo, president of SCIEMENT, “In pediatric cardiology, it’s difficult to imagine the accurate 3D structure from such images. And even when 3D computer graphics are synthesized, doctors are still looking at 2D displays, making it difficult to understand depth.”
SCIEMENT’s Viewtify solution is poised to transform medical 3D imaging. The system uses the Unreal Engine 4 computer graphics software to synthesize multiple 2D images into a 3D surface shape, which doctors can examine on Sony’s Spatial Reality Display. Says Dr. Seo, “Sony’s glasses-free, volumetric 3D display can show depth intuitively, so it’s easy to understand even complex structures.”
This naked-eye stereoscopic solution will revolutionize the medical field.
Dr. Hirofumi Seo
M.D., Ph.D., President and CEO, SCIEMENT, Inc.
The power of 3D display
Starting from contrasted cardiac CT images, Viewtify and Sony’s Spatial Reality Display can help doctors understand the detailed anatomy of the infant heart. “Usually, congenital heart defects are classified by looking at echocardiography, but it is often difficult,” says Dr. Seo. “With the Viewtify solution, it’s very easy to classify each defect. And we can understand that even when the classification is the same, the actual shape or structure is different in each case.”
Spatial Reality Display
The solution enables doctors to manipulate the 3D image in the computer, rotating, translating, zooming in or out, and make a cross section by any plane, all in real-time. In addition, doctors can “look around” the 3D image on Sony’s Spatial Reality Display as if examining a solid object. Dr. Seo says, “With the Spatial Reality Display’s high resolution and eye tracking, we can really understand intracardiac structures.”
Dr. Seo reports tremendous interest among pediatric cardiologists and pediatric cardiac surgeons. Other potential applications include confirming the intervertebral gap for thoracic epidural anesthesia, understanding the shape of bone fragments in compound fractures, viewing the complex shape of blood vessels in cerebral aneurysms, plus forensic medicine and medical education.
3D display versus
3D printing in medical practice and medical education, 3D display also offers practical advantages over 3D printing. “With 3D display, you can adjust the cross section, the look and the scaling as needed,” says Dr. Seo. “On a 3D printout, we can’t undo a cut and re-cut at another place. But with Sony’s Spatial Reality Display, we can (virtually) cut as many times as we want.” In addition, compared to 3D printing, 3D images are faster, consume no costly materials, need no physical storage, and lend themselves to making figures for papers and textbooks.
Engineering the solution
Dr. Seo considers 3D viewing to be a supplement, not a replacement for 2D medical monitors, such as the Sony surgical model display (LMD-X2705MD) that he uses side-by-side with the Spatial Reality Display. The Spatial Reality Display can only be viewed by one person at a time and is not designed for 2D medical images. “Medical doctors will always want to look at DICOM images at their very best,” Dr. Seo says.
In addition to his medical training, Dr. Seo has deep expertise in computer graphics. His visualized video “Multi-Scale Multi-Physics Heart Simulator” won the award for Best Visualization or Simulation at the international SIGGRAPH 2015 Computer Animation Festival. That was also the year he started working with Unreal Engine 4. The Viewtify solution converts DICOM-standard medical images into Unreal Engine 4 content. He says, “Getting 3D computer graphics onto Sony’s Spatial Reality Display was the easy part. You also need appropriate shading and lighting. And the real challenge was generating simultaneous versions for the 2D and 3D displays. The results were worth the effort. This naked-eye stereoscopic solution will revolutionize the medical field.”
Engineering the solution
Dr. Seo considers 3D viewing to be a supplement, not a replacement for 2D medical monitors, such as the Sony surgical model display (LMD-X2705MD) that he uses side-by-side with the Spatial Reality Display. The Spatial Reality Display can only be viewed by one person at a time and is not designed for 2D medical images. “Medical doctors will always want to look at DICOM images at their very best,” Dr. Seo says.
In addition to his medical training, Dr. Seo has deep expertise in computer graphics. His visualized video “Multi-Scale Multi-Physics Heart Simulator” won the award for Best Visualization or Simulation at the international SIGGRAPH 2015 Computer Animation Festival. That was also the year he started working with Unreal Engine 4. The Viewtify solution converts DICOM-standard medical images into Unreal Engine 4 content. He says, “Getting 3D computer graphics onto Sony’s Spatial Reality Display was the easy part. You also need appropriate shading and lighting. And the real challenge was generating simultaneous versions for the 2D and 3D displays. The results were worth the effort. This naked-eye stereoscopic solution will revolutionize the medical field.”
