Virtual reality was primitive in the mid 1990s. A typical VR experience would be a few thousand polygonal objects with very crude shading and the whole system, including the head tracking, was slow to respond. The problem was computing speed, as it still is to some extent today.
The VR experience is directly linked to the quality and fidelity of the computer graphics, which demand incredible hardware resources. A VR piece needs to display images that are twice HD resolution at 90 times a second, and dropped frames or jittering are not acceptable in such an immersive medium. This means that graphics computations must happen in less than 10 milliseconds, and if newer headsets bring higher refresh rates this number could drop even more! Even with today’s amazing hardware there is still a lot that we cannot achieve in .01 seconds.
Although computer graphics can deliver immersion today, it can only produce real time photorealistic video in a few isolated cases. For example, realistic cars can be viewed in real time. But a VFX studio has spent an incredible amount of energy to fix the details of that car and to make the digital driver seem lifelike. Motion capture cannot catch all of the details of human movement, and if the lighting or the animation changes slightly, the piece will fall apart. Reflections and global illumination can be rendered in a real time game, but this is done by spreading the cost of expensive computations over multiple frames. For example, parts of the image like reflections can be rendered on every two or four frames instead of rendering it for every frame. These kinds of cheats are not possible in VR.
Much of the VR industry is focused on delivering better computer graphics in VR. Big studios like Disney’s ILM xLabs project, and some smaller studios (Within, Felix and Paul, Penrose, Baobab etc.) are raising the bar for great computer graphic imagery. Chaos Group has been creating 360° camera rigs in their popular V-Ray renderer, as well as blogging about how to do things in VR. Google has jumped on the bandwagon with Seurat, validating the importance of 6DoF.
Live action is also very important in VR, and there is going to be an increasing integration between live action and computer graphics. From an economic standpoint, no one will create a synthetic human if they can shoot a video of a real person instead, but they might capture a live action performance and then use computer graphics for the final render. We will probably see more and more sampled data in computer graphics. Even today there are few 100 percent live action or 100 percent purely synthetic images, and the fusion between the two will only accelerate.
The challenge for computer graphics in VR today is to bring photorealism and live action into VR, with fully realistic environments, deforming geometry, view-dependent lighting, and unlimited FX. And all of it must be rendered in only 10 milliseconds! When this challenge is solved, VR is going to greatly accelerate. Until that day, here is a list of some of our favorite CG VR experiences available by leading creators today.
- In Arden’s Wake, Penrose has created another VR masterpiece, this time in a lovingly crafted undersea world.
- The engaging Invasion! is a light-hearted romp with spirit.
- Life of Us is an adventure that lets you and your friends evolve from cell to human to robot.
- If you are in the mood for a trippy neon dance party, Chocolate will take you there!
- Werewolves Within brings party games to a new level of engagement and interaction.
- In Star Trek: Bridge Crew, you use virtual controllers to reach warp speed and fight Klingons with your friends in a Federation starship.
- Dear Angelica, a dreamlike journey through a woman’s memories of her mother, is magical. It was created by Oculus Story Studio using Quill.
- From Wevr, theBlu is a delightfully immersive underwater experience with CG creatures that swim around you.
- Unreal’s UE4 graphics in the Robo Recall game are stellar.