The Mill tackles monstrous VR challenges for Google ATAP

The pioneers of VR continue making progress, despite (or perhaps because of) the incredible technical and creative challenges it poses. 

One of the latest projects to get mainstream attention is “Help,” an action-packed short film directed by Justin Lin (of the Fast and the Furious franchise) with considerable visual effects and technical development help from The Mill.

A window on an alternate reality

“Help” isn’t a VR experience in the traditional sense of the term; you don’t need to strap an Oculus Rift to your head to watch it. But you do need a compatible mobile device running the Google Spotlight Stories app, currently only available on Android. (According to Variety, Spotlight Stories will be soon be coming to YouTube with a native iPhone app to follow.)

Spotlight Stories allow the user’s phone to act like a window on a 360 degree storyspace. As you move your smartphone, you can “look around” an immersive world and optionally trigger interactive story points. Spotlight Stories originally launched as an exclusive experience on Moto X devices in 2013 with “Windy Day,” a short animation directed by Jan Pinkava, co-director of Pixar’s Ratatouille.

Spotlight Stories were born out of Motorola’s Advanced Technology and Projects (ATAP) department, which Google kept alive and moved under the Google brand after acquiring Motorola for $12.5 billion USD in 2011. Google ATAP has carried the Spotlight Stories torch ever since, funding several other shorts, including the Glen Keane directed “Duet.”

ATAP is also creating a Story Development Kit (SDK) that will extend the Spotlight Stories platform to a larger audience of filmmakers and storytellers using familiar production tools like Maya.

Leaping into live action

“Help” is the first live action-based Spotlight Story, bringing with it considerable technical challenges. The Mill’s behind the scenes featurette explains some of the hairier issues they dealt with.

The camera rig is a custom-built array of four RED cameras with fish eye lenses shooting at 6k. While this enabled the capture of high-definition footage across a huge range of vision, it also introduced chromatic shifting, strong distortion and stitching challenges (not to mention monstrous file sizes). So The Mill wrote custom software to ameliorate those issues.

One piece of software they wrote, Mill Stitch, is a hardware-software tool that allowed for real-time stitching of footage on set. It essentially emulates the Spotlight Stories app, letting the crew control a “window” on the full range of footage they were capturing.

The Mill’s investment in “Help” puts them in the small but growing company of studios prepared for the bright (and hopefully profitable) horizon of VR filmmaking. Eventually, there will be some trickle down benefits that empower smaller shops and individuals to dip their toes into VR storytelling without a phalanx of technical directors standing behind them. In the meantime, though, high-end VR production work is expensive, difficult work that requires architecting new solutions at every turn.