Data Acquisition, Processing and Combining for Realism in VR

Timeframe: 2019-2022

Immersive developers have made huge technological leaps in the last 5 years. However, the limitations of current technology and the use of traditional capture techniques have meant that these platforms are unable to reach their full potential of giving fully visually realistic, immersive, and interactive applications. Several promising avenues such as lightfields and impostors have offered a glimpse of a possible solution. However, such technologies carry several drawbacks, including lack of interactivity; expensive and bulky hardware; very large datasets; and the lack of a standardised software/engine pipeline. This project is aimed to create a new set of software and hardware tools to scan and capture realistic environments, e.g. the inside of a building, a movie set, or a factory floor, using novel computer graphic techniques including lightfields and photogrammetry. This will streamline the creation of hyper-realistic immersive experiences and virtual reality applications, which can be used both in consumer products such as games, and for business applications such as training, virtual storefronts and remote collaboration.

In this project we are to explore, research, design and prototype a low-cost, easy-to-use software and hardware pipeline solution for the capture and creation of high-quality light field matrices, impostors, as well as combining it with traditional photogrammetry techniques.  We are hoping to use the pipeline in the creation of the world’s first hyper-realistic video game, as well as for creating new immersive applications with external partners. The project goal is to contribute new consumer-facing products and business solutions for Pocket Money Games. Beyond this, having access to low-cost and high-quality capture and display technology would allow the UK industry to leap ahead and produce the next generation of immersive content for both emerging XR devices and more traditional software platform.