The rise of affordable virtual production technology has encouraged a wider pool of artists to explore the creative opportunities it brings.
This increase in popularity also means that there are a lot of professionals out there who are using the technology for the first time.
Keep reading our virtual production essentials if you want to get a better understanding of the virtual production process and avoid making the usual rookie mistakes.
1) Question if the scene should be shot on a volume. Don’t use in-camera effects for the sake of it.
Is VP the best option for your scene?
Yes, if there are any reasons that make the scene difficult or expensive to capture on location or green screen. For example, productions with limited time for company moves or post-production often choose LED backdrops to centralize a production in one studio and capture everything during the shoot.
If a location is going to be used multiple times, or if the production requires hopping to many locations in one day, the investment will be more than justified. For projects with stunts, special effects, and A-list celebrities, these benefits can be even more pronounced.
Other factors, such as the feasibility of introducing physical foreground and/or background elements to ground the virtual content, or having enough room to capture the main drama within the volume, should be taken into consideration.
Vehicle shoots tend to be the trickiest for determining feasibility, as even the largest LED installations will be suitable only for interior shots, with exterior action (such as reflections on the hood of a vehicle) requiring specialized facilities for car work.
The fact is, as the industry matures, a variety of LED stages are becoming available, and the options for those on lower budgets are increasing. Gone are the days when only “Mando-sized” stages exist. At ARwall, we specialize in helping build small and efficient setups, 20-30ft in width, that attract a wider range of budgets and still deliver award-winning results used by major studios.
2) Adjust your workflow, plan, and previz, and fix it in pre-production.
Once you’ve ensured that VP is the right option for your scene, adjusting your workflow is the next natural step. To make the most of the stage time with the LED volume, assets such as models, characters, and 3D environments must be completely camera-ready before production starts; in some ways making this a return to traditional set-building workflows. Virtual environments required for VP shooting need to be generated, dressed, and lit during pre-production. As a result, major creative decisions and final quality imagery are things you should achieve during pre-production instead of deferring them to post-production. Instead of “fix it in post,” VFX departments have started to say “fix it in pre,” which dramatically reduces the uncertainty of the effects process.
In addition to performing test renders of the scene and lighting in a virtual environment, the filmmakers most well-versed with virtual production also generate previsualizations complete with camera movement and digital characters for blocking and action. This way, an entire scene can be mocked-up ahead of time into an animatic, perhaps even an entire project or show.
Imagine the sense of confidence and clarity that a production team has knowing there’s an animatic and previs of every single camera setup, perfectly replicated on the LED backdrop and ready to shoot. This is the promise of virtual production.
3) Shaping your color turns a scene into a story.
Color consistency in VP is famously a time-consuming process, but also extremely important due to its huge role in how audiences emotionally respond to what they see. Considering that in virtual production workflows, you are aligning the emissive output of an LED backdrop, rendered digital content, and what the camera sensor captures, you have a stronger need for improved color control, scopes, and visual acuity. Paying extra attention to make sure that what you are recording matches the existing footage is essential.
For the basics, this means having a quality video monitor with large or separate video scopes to track hue, luminance, and saturation. When you’re dialing in color on the LED backdrop, you’ll do this on the LED hardware first, then tweak using Unreal Engine’s post-process volume color controls.
ARwall’s ARFX Pro Plugin is the tool we developed to make this crucial process a simple task that takes seconds instead of minutes. Are you looking at your virtual environment and wondering how to make it look more real? You need color tools that allow you to dial in contrast, brightness, tint, and temp to match your live-action subjects.
When it comes to lighting, make sure your director of photography is aware of the new world of possibilities LED walls have opened up. With much of the lighting provided by the volume itself, off-camera LED screens or DMX-controlled practical lighting can be used to create additional reflections and immersive effects. The Digital Multiplex (DMX) network protocol enables you to imitate the lighting effects of the content that appears in-camera, achieving a greater level of realism that samples the virtual environment and spills that light into the physical space.
For our work on Muppets Haunted Mansion, for which we won a Children’s & Family Emmy for the Art, Sets, and Scenic Design, we worked with the DP Craig Kief, ASC, for months ahead of the shoot to ensure there was a consistent color motif and options to select from which adhered to the overall color palette in the story. Production designer Darcy Prevost was also essential in planning the color of all sets and coordinating that process for the physical production’s job.
4) Understanding LED pixel pitch and camera distance.
Pixel pitch, measured in millimeters, is the distance between individual LED diodes on the array panel used to build up these huge screens. The panels themselves are small, usually smaller than a meter, and the individual diodes are embedded into them in a grid similar to a consumer TV but much brighter and with much more powerful image processing.
The lower the pitch distance, the denser the diodes and resolution of the screen, resulting in a great resolution and fewer moiré artifacts. Generally, VP is done on panels that are 1.5mm-3mm, with lower pitch sizes allowing for smaller screens with the same resolution as the larger pitch sizes, meaning your camera can get closer to the screens with lower pitches.
The main factors determining what LED panel is optimal for your production are the size of the backdrop required, the width and height of the studio, and the available space between camera, subject, and backdrop.
For smaller spaces, you’ll want to choose a smaller pitch size and smaller dimension of screen. However, for productions where you want a 360 environment, larger dimensions with larger pitch sizes will likely be a better fit.
At ARwall, we’ve standardized for indie productions on a 30x12ft 1.5mm pitch size using the Absen AX Pro panels, which also have excellent view angles, color reproduction, cadence (7680Hz), brightness, and contrast.
This specific size is an excellent balance that allows for the camera to get as close as a few meters and still achieve crisp imagery without moiré. We also developed the ARFX Pro Server System specifically to run these types of LED walls.
5) VP is a revolution in on-set collaboration; embrace it!
Focusing on pre-production means that different departments should be brought together early on: effects, camera, lighting, production design, and more. To plan the LED content, you not only need to involve the creative team, director, and DP but also the technical team, art department, post-production, and VFX teams. Including them from the start will guarantee a seamless blend between practical and virtual worlds.
Fear not; even though it might seem challenging to coordinate all these teams from the get-go, this is the purpose of a Virtual Production Supervisor, who will ideally be someone with a broad background working with different departments. A cohesive process will ultimately give everyone more confidence in the success of the production and will foster more creativity by enabling the exchange of more ideas.
A great example of how we at ARwall have seen departments work well together was specifically on Muppets Haunted Mansion. This was a production achieved deep into the pandemic, and we were all newbies using Zoom to collaborate in pre-production.
The results speak for themselves, as daily meetings resulted in a vision that bridged virtual and physical.
6) Get expert help on fitting your show’s budget to VP.
One of the most crucial components to starting this process is working with a great Virtual Production Supervisor or Producer who can break down a script and work with you early on to figure out what can be virtual and what needs to be real. Even though budgets work differently in VP–consider the creation of digital environments, LED volumes, and computer power here– you will still need to build sets and foreground props and send make-up artists and crew to location.
Here is the basic thing to consider: if an actor or their shadow is touching it, it should be physical. This will include floors, railings, chairs, counters, doors, desks, tables, and other hand props. Even a production that is nearly fully virtual will almost always include a table or door flat to complete the illusion.
At ARwall, our expert VP supervisors and producers have helped dozens of writers and filmmakers process their scripts into virtual production budgets, including suggestions on which scenes or shots to produce traditionally or virtually. It’s a new and exciting process that takes 1-2 weeks and results in actionable line-item budgets, schedules, and more to consider.
7) Stay up to date on the tech, so it’s not scary.
I get asked a lot now whether real-time graphics and AI will completely change filmmaking or whether traditional 3D VFX is still going to be a thing in five years. If anything, AI tools will simplify labor-intensive processes and make them more accessible to a wider pool of filmmakers.
As creators ourselves, we can’t deny the impact technological advancements have on our craft and creative workflows: it’s been dramatic even in the past 30 days. It is for this reason that we should always stay up to date when it comes to new releases, such as game engine improvements, launches of helpful apps, advances in LED volumes, or the latest updates in computer graphics or cloud technology.
Virtual production can save you money and time, but you certainly can’t take that for granted. To be successful doing VP requires a focus on pre-production, bringing departments together early on, and using a proven VP team that can see issues coming before they arise.
It is only with careful planning that virtual production can offer significant savings and creative opportunities in certain areas of the production process. Our own team has delivered up to 70% in cost savings compared to traditional VFX. Make sure you are aware of the tools available, which are becoming easier to use and more affordable by the day. And remember, you can’t expect to get different results if you are doing the same thing. The field is still new and ripe for experimentation!
About the Author
Rene boasts 24 years of experience with 16 startups, starting young working for his parent’s startups in silicon valley. He’s also directed over 350 commercials, short films, and pilots, primarily as co-founder and creative director for eight years at digital agency Automaton Creative. Most recently, as co-founder at ARwall, Rene won the SXSW Accelerator 2018 AR/VR category pitch, has been acknowledged as a top AR tech evangelist in Hollywood, and designed the first ARwall Lab in Burbank.