Future Leaders Fellowship for 3D digital reality researcher

It’s your weekly meeting with your team and it’s taking place virtually. At the moment, that means looking at a row of miniaturised faces on the laptop screen in front of you. But how would it be if you could put on a pair of virtual reality glasses instead and – thanks to computer-generated 3D models – see all your team-mates apparently sitting in your office alongside you?

Or take another scenario: you have a bad fall and need surgery to repair an injured joint. Normally that means having to attend time-consuming follow-up appointments at a clinic to measure that the affected joint is regaining its full range of movement. So wouldn’t it be useful if instead you could create 3D scans of it with your mobile phone and transmit them to your doctors for them to evaluate?

Those are the kind of future experiences our colleague Dr Cengiz Oztireli is interested in. And for his research, which could have impacts in areas from remote working and virtual learning to healthcare, gaming and cinema-going, he has just been awarded a UKRI Future Leaders Fellowship.

As announced today, Cengiz is one of the beneficiaries of a fellowship scheme designed to establish the careers of world-class research and innovation leaders across the UK. The scheme is supported through an investment of £109 million by UK Research and Innovation (UKRI).

Cengiz, a Senior Lecturer here, has research interests in computer graphics, vision and machine learning/artificial intelligence with a focus on 3D digital reality. His goal is to have a form of digital reality that is easy to create, manipulate, and experience.

Some of the problems he has been working on to realise this goal are understanding structures learned by neural networks, 3D capture, rendering, stochastic sampling techniques, geometry representations, neural geometry processing, 3D shape and pose estimation, character animation, imaging, and image/video processing.

He is working with a number of industrial collaborators who are eagerly exploring the potential of this technology. Before he came to Cambridge, Cengiz worked as a research scientist at Disney Research for three years. He still has links with Disney as well as with other high-tech companies who have interests in 3D digital reality. 

Designing, creating and experiencing 3D content
"All the high-tech companies are investing in this field because they want people to be able to design, create and experience 3D content – for example for holographic displays, for films, and for augmented reality apps and games. There's a desire to be able to provide tools to do this not just to professionals, but to everyone," Cengiz says.  

As a lecturer himself who is currently delivering classes online to his students, he is interested in the potential of 3D digital reality to make it a much more engaging experience.

"With the technology we're using at the moment, I'm just seeing a row of faces or icons on the screen. It's not the best way to create interaction between me and my students," he says. "But using 3D digital reality, we could create 3D models of ourselves and see ourselves meeting in a virtual classroom. That would be a more immersive and satisfying experience."

There is a need for new tools to do this. "Designing animated characters and scenes in 3D is very difficult: the tools to do this are currently super-complex and very cumbersome," he says. "And designing products in 3D – for example, designing a copy of your favourite cup in 3D on your computer and sending it to a 3D printer – is also something we can't really do at the moment." 

Addressing such design issues is something he will be working on during his Fellowship. And he'll also be working further on the use of 3D digital reality in the field of healthcare. Colleagues of his here in the Department are working on devices that use auditory signals (body sounds) to monitor people's health. "I'm interested in doing the same with visual signals," he reveals. 

Diagnosing conditions remotely using 3D scans
"If patients could take 3D scans of their skin, or eyes, these could be used by doctors remotely to diagnose conditions or track their recovery after surgery. Longer term, the scans could also be shared with others (with patients' consent), for example to train junior doctors – that's something I’ve already been looking at.

"And if we could build up a database of such scans, it could also be used to train AI to diagnose issues automatically – for example, to recognise the difference between 'normal' and broken bones.

This prestigious UKRI Fellowship will fund Cengiz's research for a minimum of four years and offer the option of extending it for a further three. "I am delighted to be receiving this Fellowship as it will enable me to take the right steps to realize my vision for the 3D digital reality of the future," he says. And he adds that in the current environment, his line of research has gained an unexpected urgency.

Using technology to enhance remote meetings
"When I started preparing my Fellowship proposal, the Covid-19 pandemic was coincidental to my research. But now there's a really pressing business use case for having a 3D reality that we can use for teaching, or for working in distributed teams, because that's now happening so widely. 

"With more companies like Twitter, Facebook and Microsoft announcing that remote working will continue beyond the pandemic and be a permanent option in future, we urgently need technologies that will make our experience of remote meetings much more like the ones in the real world."

• UKRI's Future Leaders Fellowships initiative aims to support the creation of a new cohort of research and innovation leaders who will have links across different sectors and disciplines. Awardees will each receive between £400,000 and £1.5 million over an initial four years. The grant supports challenging and novel projects, and the development of the fellows' careers. The funding can also be used to support team members and their development and pay for equipment and other needs.