Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
Chart-topping sensation Jackson Wang embraces virtual production for superpowered music video
May 18, 2023
A post-apocalyptic cityscape, acrobatic fight scenes, and fire-breathing powers. No, we’re not describing the last half hour of a Hollywood blockbuster, we’re calling out some of our favorite moments from Jackson Wang’s Cruel, a recent music video that shows just how far virtual production can take a pop song looking for some visual weight.
We’re a long way from the golden age of MTV when record labels would happily rain cash on the most opulent video concepts. But forward-thinking artists like Wang are still finding ways to intrigue digital natives through a medium that has always elevated songs.
We’re a long way from the golden age of MTV when record labels would happily rain cash on the most opulent video concepts. But forward-thinking artists like Wang are still finding ways to intrigue digital natives through a medium that has always elevated songs.
You don’t have to be an expert on Wang’s output to see that he doesn’t do anything by halves. From immaculately produced pop bangers to bombastic live performances, there’s a creative eye to everything he does. So when it came time to create a video for his latest single, the Coachella 2023-bound Wang had a fittingly elaborate concept in mind, filled with fire-breathing powers and demon-infused dance sequences.
Wang’s creative brief landed on the desk of Rich Lee, an experienced director with a background in VFX. Known for his work with Billie Eilish, Eminem, and Rihanna, Lee is a filmmaker of considerable caliber. But even for a director this versed, the Cruel video would be no walk in the park.
“From a budget and timing standpoint, we couldn’t shoot this on location,” says Lee, who also serves as Chief Creative Officer at Synapse Virtual Production. “We also needed the ability to control the environment and its lighting, which traditional methods don’t always allow.”
A virtual production workflow built on Unreal Engine became the only solution. Fortunately, Lee and his team had just finished building a virtual production stage that would allow them to deliver Wang’s vision to the highest quality possible. That didn’t, however, mean unanimous excitement from Wang and his team.
A leap of faith
Cruel would be the launch point for the next era of Wang’s career, his Magic Man album cycle, and it needed to make an impact. With just a couple of weeks to pull everything off, there was justifiable hesitation to embrace a technology the artists involved had never worked with. What constraints would the LED backdrop impose on camera movements? How would it interact with the dance choreography? And how convincing could the end result really look with a limited time to create the bespoke virtual environments?All those concerns melted away when the artists got on set. Backed by a 55' diameter LED wall and dressed with dirt and rubble, the stage allowed the artists and production team to see something close to final shots in real time. What’s more, the digital environment displayed on the screen could be rotated 360 degrees without losing any of its quality. The stage proved so immersive that, if anything, it enhanced the artist’s performance.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
“The artists reacted very differently than they would have on a green screen,” says Louise Baker Lee, Executive Producer, Previs/VAD/VFX at Synapse Virtual Production. “Bailey Sok, who played the demon queen, spoke about the vast difference it made just to feel like she was actually part of that world. You just can't get that instant gratification from traditional software and pipelines. If I don't have to work with a green screen again, I'll be a very happy person.”
With the digital backdrop already on set, Lee and director of photography Christopher Probst, ASC didn’t need to shoot according to the allocated budget for compositing and VFX. Instead, they had an unlimited number of shots with which the team could improvise and discover happy accidents that simply wouldn’t have been possible otherwise. They could even rearrange elements of the digital backdrop at the touch of a button.
“You're able to improvise,” adds Probst. “The artists are inspired by the environment. They would look around and go, ‘wow, what if I jumped off of this? That would be crazy.’ Or ‘I'd love to grab this prop.’ They're actually responding to and being enveloped into the environment in a way that is wholly different to working in a green-screen limbo.”
With the digital backdrop already on set, Lee and director of photography Christopher Probst, ASC didn’t need to shoot according to the allocated budget for compositing and VFX. Instead, they had an unlimited number of shots with which the team could improvise and discover happy accidents that simply wouldn’t have been possible otherwise. They could even rearrange elements of the digital backdrop at the touch of a button.
“You're able to improvise,” adds Probst. “The artists are inspired by the environment. They would look around and go, ‘wow, what if I jumped off of this? That would be crazy.’ Or ‘I'd love to grab this prop.’ They're actually responding to and being enveloped into the environment in a way that is wholly different to working in a green-screen limbo.”
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
Setting the stage
Virtual production is, in many ways, a return to the classic essence of filmmaking. Probst sees their newly built studio as a logical progression from the painted backdrops of The Wizard of Oz, the rear projection of North By Northwest, and the matte paintings of Star Wars. Rather than bringing elements together in post-production, everything—down to the finest details—can be ironed out prior to shooting.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
Lee and the Synapse Visual Art Department (VAD) team spent a week building Cruel’s post-apocalyptic environment, anticipating and overcoming potential issues in the rough edit, before a second of footage was shot. Shooting began the following week and by the end of day one, the team had an edit that looked 90 percent as good as the finished article.
“Traditionally, this would probably have taken two months of post-production, just staring at composites, making notes, and going through that whole process before we got to a place where maybe it looked only 75 percent as good as it would after shooting on a virtual production stage.”
To create the ruined cityscape setting of Cruel, Lee and his team embraced the art of kitbashing, combining pieces from various 3D models into new designs. The production team were no strangers to this method; not only was it a key aspect of their VFX work, but even on traditional shoots they would build bespoke sets from elements of different locations or items from prop shops.
“Traditionally, this would probably have taken two months of post-production, just staring at composites, making notes, and going through that whole process before we got to a place where maybe it looked only 75 percent as good as it would after shooting on a virtual production stage.”
To create the ruined cityscape setting of Cruel, Lee and his team embraced the art of kitbashing, combining pieces from various 3D models into new designs. The production team were no strangers to this method; not only was it a key aspect of their VFX work, but even on traditional shoots they would build bespoke sets from elements of different locations or items from prop shops.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
This time, though, the team could utilize the Unreal Engine Marketplace, a huge library of assets for use in 3D work, including characters, plugins, Blueprints, animations, and more. “It's such a tremendous resource for props, environments, and things like fire and atmosphere,” says Lee. “We would go to the Marketplace to look for different elements, add them to our archive, and drop them seamlessly into Unreal Engine. Without too much work, we got them functioning beautifully in the environment. It was insanely fun, it's like decorating a room, moving things around, and getting that instant gratification.”
With an outdoor setting, fires, and more, lighting was at the forefront of Probst's mind. Illumination from the vast LED screen provided a base level of exposure on the charcoal-covered faces of the performers. Meanwhile, cuts of white silk lit from below and blown by a fan stood in for flames on the studio floor, a surprisingly convincing method that the team borrowed from Disney’s Pirates of the Caribbean ride.
With an outdoor setting, fires, and more, lighting was at the forefront of Probst's mind. Illumination from the vast LED screen provided a base level of exposure on the charcoal-covered faces of the performers. Meanwhile, cuts of white silk lit from below and blown by a fan stood in for flames on the studio floor, a surprisingly convincing method that the team borrowed from Disney’s Pirates of the Caribbean ride.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
Probst had only to respond to the physics of the stage’s illumination, asking their on-set Unreal Engine operator to insert lighting cards that allowed him to make adjustments to the direction, brightness, and color instantly. The last time Probst lent his talents to a post-apocalyptic music video, he needed a huge crane-mounted lighting rig to create a light source.
The engineering team at Synapse Virtual Production designed a volume stage with a seamless transition from the wall’s LED panels to the ceiling panels. It meant that Probst could tilt the camera up to show the true scale of the huge nuclear cooling towers seen in the video’s climactic moments.
LED, camera, action!
An all-singing, all-dancing music video like Cruel needs intricate choreography and fluid camera movement to complement it. With the action taking place in the crumbling remains of a city, the camera also needed to convey the scale of the performers' surroundings.The engineering team at Synapse Virtual Production designed a volume stage with a seamless transition from the wall’s LED panels to the ceiling panels. It meant that Probst could tilt the camera up to show the true scale of the huge nuclear cooling towers seen in the video’s climactic moments.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
The team didn’t shy away from making the most of the studio’s potential. The opening shot of the video sees Wang jump over the camera before Probst rapidly tilts down. For the dance sequences, Lee and Probst were able to use handheld camera shots to get up close to the action and show the intricacies of the performances.
Courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
In their climactic dance battle, the demon queen lifts Wang high into the air. Traditionally, this would require the performers to be lifted up on a wire, a solution that wasn’t possible on the 25-foot-tall virtual production stage.
Instead, Lee and the team built a rotating turntable for the performers to stand on. With a handheld camera, they would shoot the actors as the turntable spun, with the Unreal Engine operators moving the digital environment down, making it appear as if the talent was rising up in the air. To achieve this effect, they needed to first calculate exactly how many revolutions per minute the turntable would need to complete in order to look convincing. Then, armed with this information, they could match the Unreal Engine environment’s movements accordingly.
This unique solution not only resulted in a more convincing effect but saved the entire production time and money. The team saved the cost of hiring a stunt team to manage the wire harnesses and negated the need to paint bulky equipment out in post-production.
Image courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
The future is virtual
Despite his initial concern, Wang fully embraced the process of shooting on the virtual production stage and was thrilled with the results. “Not too long after Cruel, I got a text message from Jackson saying, ‘Hey, can we do another one? Let's do another video entirely in virtual production.’ He loved it so much.”Lee’s editor was another convert, and was actually unaware that Cruel would be shot using a virtual production workflow. Deciding not to mention it, Lee simply handed over the hard drives, knowing that his editor had little to no knowledge of the technology or its capabilities. ‘I got a phone call saying, how did you guys do this? What am I looking at?’ He just couldn't imagine that we had enough money to create these visuals, let alone have them in the can already.”
Cruel was such a success that Wang and the production team headed to the virtual production stage once again on the video for his single Come Alive, this time depicting a twisted cyberpunk carnival setting.
Courtesy of 88 Rising | Team Wang | Drive Studios | Synapse VP
“Virtual production is opening up so many possibilities that, from my perspective, it’s absolutely here to stay. It’s presenting new narrative opportunities and democratizing them to lower production paradigms because you can create an intricate spaceship cockpit with minimal resources, or you can do a scaled-down version just with a 72-inch monitor behind someone’s head. You can do anything—it’s incredible.”
Get Unreal Engine today!
Get the world’s most open and advanced creation tool.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.