Taking Unreal Engine's latest in-camera VFX toolset for a spin
November 8, 2021
Recently, Epic Games and filmmakers’ collective Bullitt assembled a team to test out the latest in-camera VFX toolset, part of the extensive suite of virtual production tools in Unreal Engine 4.27, while it was still in Preview. To put each of the new tools through their paces, they created a short test piece to mimic a production workflow.
Shot entirely on NantStudios’ LED stage in Los Angeles, the production test capitalized on new workflows that enable dynamic lighting and background changes on set with minimal downtime, together with multi-camera and traveling vehicle shoots. It also harnessed the power of twin NVIDIA Quadro A6000 GPUs. The team was able to produce final pixels in just four days—a demonstration of the efficiency and creative freedom the toolset offers.
Shot entirely on NantStudios’ LED stage in Los Angeles, the production test capitalized on new workflows that enable dynamic lighting and background changes on set with minimal downtime, together with multi-camera and traveling vehicle shoots. It also harnessed the power of twin NVIDIA Quadro A6000 GPUs. The team was able to produce final pixels in just four days—a demonstration of the efficiency and creative freedom the toolset offers.
The team also took advantage of new Level Snapshots that enable directors to easily match previous takes when returning to an earlier set or lighting configuration; sophisticated remote control UI building tools that offer creatives direct control of the Unreal Engine-powered elements of the shoot from the convenience of a tablet or laptop; and OpenColorIO color management that ensures artists creating Unreal Engine assets can see what the final captured output will look like.
In the first video in the series, the filmmakers talk about the huge benefits they witnessed when working with these latest updates to Unreal Engine’s in-camera VFX toolset. The entire team, including the director, producer, production designer, director of photography, and first assistant director, all agreed that what they experienced represented a quantum leap for on-set filmmaking.
In the first video in the series, the filmmakers talk about the huge benefits they witnessed when working with these latest updates to Unreal Engine’s in-camera VFX toolset. The entire team, including the director, producer, production designer, director of photography, and first assistant director, all agreed that what they experienced represented a quantum leap for on-set filmmaking.
The ability to find new shots on the stage is just incredible. In the same amount of time that it would take me to change a light in the physical world, I'm able to have new lighting on the wall. As a director, that's a game changer.
”Recently, some intrepid filmmakers have been reaping the benefits of using in-camera VFX within LED volumes in productions such as The Mandalorian. With existing tools, however, digital set changes could require hours or even days of time for light baking, losing much of the spontaneity in the process. The new toolset in Unreal Engine 4.27 includes GPU Lightmass, which takes advantage of light baking on multiple GPUs to radically accelerate set changes, making them as much as an order of magnitude quicker. Instead of breaking for the day, teams can now just take a coffee break.
Ryan McNeely, who acted as director on the production test, explains why this is such a huge step forward. “As a director, time is your biggest enemy, and the thing about this, the speed at which we're able to move the amount of setups, that's honestly the dream,” he says. “The ability to find new shots on the stage is just incredible. In the same amount of time that it would take me to change a light in the physical world, I'm able to have new lighting on the wall. As a director, that's a game changer.”
For renowned filmmaker Anthony Russo, it’s the ability of in-camera VFX to bring everything together and enable the team to see the final result right on set. “One of the things that excites us most, of course, about this is the fact that we can do in-camera choreography, where all the elements of the frame are actually in concert with one another and organically working on one another to create a more visceral experience,” he says.
Director of Photography Matthew Jensen agrees. “It's so much easier now to be able to make the changes in real time, see their effects, and then make adjustments from there,” he says. “It allows for a happy accident. It allows for more spontaneity in the creative process.”
Director of Photography Matthew Jensen agrees. “It's so much easier now to be able to make the changes in real time, see their effects, and then make adjustments from there,” he says. “It allows for a happy accident. It allows for more spontaneity in the creative process.”
What’s more, in-camera VFX has the ability to remove the silos that can cause creative intent to get lost in translation.
“It's not separated into components of pre-production, shoot, and post-production,” says Diane Castrup, Producer. “Usually you just have your physical set that you're standing on and the green screen is behind you and you just kind of give it over after you're done shooting and you hope that it all works out.”
Production Designer Quito Cooksey echoes these sentiments. “Once principal photography ends, post takes over,” he says. “And if you're not part of that, everything that you really envisioned and designed goes through a filter and doesn't always come out the other end in the way in which you envisioned. With this, you have the virtual art department with control of the concept designs all the way through to final pixel.”
“It's not separated into components of pre-production, shoot, and post-production,” says Diane Castrup, Producer. “Usually you just have your physical set that you're standing on and the green screen is behind you and you just kind of give it over after you're done shooting and you hope that it all works out.”
Production Designer Quito Cooksey echoes these sentiments. “Once principal photography ends, post takes over,” he says. “And if you're not part of that, everything that you really envisioned and designed goes through a filter and doesn't always come out the other end in the way in which you envisioned. With this, you have the virtual art department with control of the concept designs all the way through to final pixel.”
Sean Harner, First Assistant Director, admits to being skeptical when he first arrived on set. He’s now a convert. “I am totally blown away with how you can move things around, how we can change sun direction, how I can physically change sets,” he says. “We’re taskmasters, and it's all about time. If I lose an hour on a commercial or a movie, I can't get that time back and that could be $100,000 or that could be $500,000.”
It was photoreal and it was ready to reel. That means that you could go from shooting it to projection in the same day. That's technology at its finest.
”Another huge advantage of the new toolset is the ability to quickly and easily recall a set that you’d shot earlier—perfect for those additional pickups, as Harner discovered when working on the project. “And just in between the sets is 20, 30 minutes’ setup,” he says. “It was photoreal and it was ready to reel. That means that you could go from shooting it to projection in the same day. That's technology at its finest.”
The last word goes to Joe Russo. “It feels like we're moving from filmmaking 1.0, jumping to filmmaking 5.0—and I don't see any other path forward that is nearly as exciting, compelling, or freeing for artists as this path,” he says.
The last word goes to Joe Russo. “It feels like we're moving from filmmaking 1.0, jumping to filmmaking 5.0—and I don't see any other path forward that is nearly as exciting, compelling, or freeing for artists as this path,” he says.
At Epic, we have an extensive virtual production team who are dedicated to finding better ways for filmmakers to work. Here, we chat to some of them about the Unreal Engine 4.27 features used in this production test.
Unreal Engine 4.27 is available now; it's free to use for creating linear content like films and television shows. The exciting new workflows and tools it offers are influenced by input from leading filmmakers who are pushing the limits of virtual production today. As the Bullitt team discovered, the toolset represents an evolutionary leap forward in quality, efficiency, and ease of use—delivering whole new levels of creative freedom.
To explore the toolset for yourself, download the full sample project from this ICVFX production test and take a look at the full tutorial series that goes along with it.
To explore the toolset for yourself, download the full sample project from this ICVFX production test and take a look at the full tutorial series that goes along with it.
Ready to try Unreal Engine 4.27’s ICVFX toolset?
Download the full sample project from the production test and check out the full tutorial series that accompanies it today.