Helping Augmented and Virtual Reality realise their potential

Though the concepts of Augmented Reality (AR) and Virtual Reality (VR) were first demonstrated back in the 1950s and 1960s, a major effort is needed to solve the slew of technical issues that are stopping them achieving their full potential.

So say the organisers of 'Visual Perception in AR/VR', a recent incubator meeting held by The Optical Society (OSA). The organisers include researchers in this Department.

Technical challenges with the hardware, poor image quality, a lack of content, and perceptual problems that cause users visual discomfort – and sometimes even nausea – are key challenges that need to be addressed, they argue.

If we do, we can solve the limitations holding back technologies that could have impacts in many fields, from entertainment to medical imaging.

Dr Özgür Yöntem (pictured right, wearing VR goggles), a Senior Research Associate here in Graphics, Vision and Imaging Science, co-organised the event with academics from Norway and US. Özgür was also one of the hosts (in his role as the OSA's Display Technology Group Chair).

Using perception science to inform the design of AR/VR displays
Meanwhile his colleague Dr Rafal Mantiuk, Reader in Graphics and Displays, was one of the speakers, while two of Rafal’s PhD students Akshay Jindal and Fangcheng Zhong were helpers during the event.

The meeting explored how perception science can inform the design and implementation of AR/VR displays. Leading academic and industry experts addressed issues including 'vergence-accommodation conflict', colour perception, depth perception, and limited field of view.  

'Vergence-accommodation conflict' occurs due to the discrepancy between the perceived distance of the 3D image and the actual focal distance of the images shown on the AR/VR screen. Because the human visual system is forced to adapt this, and most conventional AR/VR systems don’t account for the variations in human visual system, this can lead to users experiencing anything from visual discomfort to nausea.

Then there is the issue that human visual performance still outstrips current AR/VR displays – even the latest high-resolution AR/VR display technologies coming out of optics and photonics development. For example, AR displays do not yet adapt to changes in ambient light levels and contrast in the way that human vision does.

Says Özgür, "The state-of-the-art display technologies cannot overcome all of the current problems with a single solution due to stringent requirements both in resolution, refresh rate, colour production, etc, of the image generation units and in focusing power, focus control, size, etc, of the optics.

Major challenges ahead
"It is a significant challenge in AR/VR technology to reduce the disparity between the images delivered by the display optics of these devices and the perception of the content by the human visual system."

And there are other big challenges ahead in making AR/VR devices socially acceptable as mainstream technologies at costs low enough to encourage widespread adoption.

"This will require further research and development in flat optics, tiny light sources and light modulators, focus tuneable lenses, and high resolution displays," explains Özgür.

"Also, the rendering of digital content for AR/VR devices will be based on the display technology chosen to deliver the images. Therefore, new rendering techniques and algorithms need to be developed based on the capabilities and drawbacks of the implemented display technology by also taking the perceptual aspects of human visual system into account."

AR/VR is a highly interdisciplinary technology, the meeting agreed, and there is a practical challenge in bringing together scientists and engineers across different disciplines with academic and industrial resources to solve these limitations.

This meeting was a useful step in this direction and it is hoped that it will be followed by more meetings in future.

• Read more about the meeting in the article Visual Perception in AR/VR (by Rigmor C. Baraas, Francisco Imai, Özgür Yöntem and Jon Y. Hardeberg) in OSA's Optics & Photonics News, April issue. It is available to access from 5-11 April 2021.
• If you have further questions, please email Özgür Yöntem: aoy20@cam.ac.uk

Main image courtesy of Unsplash