Image courtesy of SPP and Imerza
Transforming real-estate visualization with an XR-based digital twin of Tampa
Ken Pimentel |
June 19, 2020Among the bustling restaurants, shops, and street food stalls of Sparkman Wharf, you’ll find a world first. An XR-based digital twin of Tampa, 17 feet in diameter, showing the Water Street district’s buildings in miniature.
At the touch of a screen, imagery projected onto the model highlights different aspects of the area, including public transit routes, the location of cultural centers, and development plans for the waterfront. Two huge video walls encircling the model display synchronized real-time 3D views looking out across the city from any floor of individual buildings, while the physical model shows a projected view cone from the same building.
This is the crown jewel of the high-tech marketing center in the company headquarters of real estate developer Strategic Property Partners (SPP). Combining 3D-printed replicas, projection mapping, and real-time technology, it’s the largest and most sophisticated model of its kind known to be developed to date. It even features integrated software capabilities.
At the touch of a screen, imagery projected onto the model highlights different aspects of the area, including public transit routes, the location of cultural centers, and development plans for the waterfront. Two huge video walls encircling the model display synchronized real-time 3D views looking out across the city from any floor of individual buildings, while the physical model shows a projected view cone from the same building.
This is the crown jewel of the high-tech marketing center in the company headquarters of real estate developer Strategic Property Partners (SPP). Combining 3D-printed replicas, projection mapping, and real-time technology, it’s the largest and most sophisticated model of its kind known to be developed to date. It even features integrated software capabilities.
A projection-mapped scale model of Tampa
SPP is the company behind the multibillion-dollar Water Street Tampa development. As the centerpiece of its marketing hub, SPP envisioned a projection-mapped scale model of Tampa that could show data in real time and offer the flexibility to change as the design of the 56-acre development evolves over the next five years.SPP teams can use the model in a multitude of ways—for conversations around real estate sales and leasing, meetings with government and city planners, internal design review sessions, billboard advertising sales, and much more.
Because of the complexity of the model, a significant degree of innovation was required to bring the vision to fruition. SPP turned to IMERZA, a company that focuses on experiential technology for real-estate, seeking to leverage its expertise in this area.
Partnering with projection mapping experts DCBolt, IMERZA created all of the custom software and content for the project. The project took around a year from concept to completion, with the 17-foot-diameter 3D-printed scale model designed modularly to allow for easy replacement of future-phase buildings as they are designed.
Image courtesy of SPP and Imerza
Real-time rendering the visualizations projected onto the model removes the need to re-render hundreds of animations when something changes, reducing the overall cost of the project to a fraction of what it would have been if the content was rendered traditionally.
All of the applications work in concert with each other, communicating via the IMERZA platform API. For example, the SPP team is able to highlight an individual building and show that building’s cone-of-view on the physical model, while simultaneously showing the views from every floor of that building on the surrounding video walls, then rotate the view around 360 degrees by simply turning a compass on the custom iPad controller. Both the projection model and video walls rotate together perfectly in sync.
Visualizing data from real-time sources
IMERZA wrote six custom applications over the course of the project, including interactive touchscreen kiosks, a video wall application, an iPad application, a custom data aggregator, a content management system, and the projection-mapped content application. Unreal Engine is at the core of four of them.All of the applications work in concert with each other, communicating via the IMERZA platform API. For example, the SPP team is able to highlight an individual building and show that building’s cone-of-view on the physical model, while simultaneously showing the views from every floor of that building on the surrounding video walls, then rotate the view around 360 degrees by simply turning a compass on the custom iPad controller. Both the projection model and video walls rotate together perfectly in sync.
Image courtesy of SPP and Imerza
Another innovative feature the team created is the ability to drop in a potential advertiser’s brand asset package into the CMS, and have the system automatically apply it to all the billboards and digital signage locations throughout the environment—both on the scale model and the video wall content.
“Features like this could only be possible with real-time rendering,” says Dorian Vee, Co-Founder and CTO of IMERZA. “Unreal Engine renders all of the content, and technologies such as nDisplay and NVIDIA Quadro RTX cards enable us to keep the multiple PCs frame-locked and rendering 12 separate cameras to the 12 projectors in real time.”
For the scale model, IMERZA used Lightact servers with Quadro RTX 6000 cards. These servers were frame-locked—synchronized on each display—using Unreal Engine’s nDisplay technology. The nDisplay system works with Unreal Engine to render 3D content simultaneously to multiple displays in real time. The team modified the nDisplay code to allow for more granular control of passing data, and to enable easier automation. Modifying the nDisplay code provided the ability to access nDisplay commands from the command-line interface, which dramatically improved the 24/7 operational requirement.
To align the digital model with the physical model, a three-point algorithm was used to get the approximate camera location in real-world space, from which IMERZA fine-tuned the position using further calculations authored in the Blueprint visual scripting system. Finally, the DCBolt team used blends and masks to fine-tune each projector for maximum visual quality.
For data that changes on a regular or real-time basis, IMERZA wrote a data aggregator that pulls from multiple sources, including commercial real-estate data sources and ESRI ArcGIS—a geographic information system for working with maps and geographic information developed by the Environmental Systems Research Institute. The aggregator normalizes the data into a new geocoded database from which the team translates the data into visuals.
“This allowed us to show all data types together in context, such as market data, rent growth, leasing rates over time, building data, city data, projected traffic data, real-time traffic from Google, and solar analysis,” explains Vee. “For data that changes less regularly, such as census data, we worked closely with the ESRI team to author a method to render ArcGIS map data to JPG textures, then ingest the textures at runtime.”
“Features like this could only be possible with real-time rendering,” says Dorian Vee, Co-Founder and CTO of IMERZA. “Unreal Engine renders all of the content, and technologies such as nDisplay and NVIDIA Quadro RTX cards enable us to keep the multiple PCs frame-locked and rendering 12 separate cameras to the 12 projectors in real time.”
For the scale model, IMERZA used Lightact servers with Quadro RTX 6000 cards. These servers were frame-locked—synchronized on each display—using Unreal Engine’s nDisplay technology. The nDisplay system works with Unreal Engine to render 3D content simultaneously to multiple displays in real time. The team modified the nDisplay code to allow for more granular control of passing data, and to enable easier automation. Modifying the nDisplay code provided the ability to access nDisplay commands from the command-line interface, which dramatically improved the 24/7 operational requirement.
To align the digital model with the physical model, a three-point algorithm was used to get the approximate camera location in real-world space, from which IMERZA fine-tuned the position using further calculations authored in the Blueprint visual scripting system. Finally, the DCBolt team used blends and masks to fine-tune each projector for maximum visual quality.
For data that changes on a regular or real-time basis, IMERZA wrote a data aggregator that pulls from multiple sources, including commercial real-estate data sources and ESRI ArcGIS—a geographic information system for working with maps and geographic information developed by the Environmental Systems Research Institute. The aggregator normalizes the data into a new geocoded database from which the team translates the data into visuals.
“This allowed us to show all data types together in context, such as market data, rent growth, leasing rates over time, building data, city data, projected traffic data, real-time traffic from Google, and solar analysis,” explains Vee. “For data that changes less regularly, such as census data, we worked closely with the ESRI team to author a method to render ArcGIS map data to JPG textures, then ingest the textures at runtime.”
Image courtesy of SPP and Imerza
Leveraging ray tracing to enhance realism
The team opted for Unreal Engine to power its applications because it needed a solution that could produce stunning visuals while also providing the ability to connect to multiple third-party tools and handle complex datasets. “The choice to use Unreal Engine was really a no-brainer,” recalls Vee. “Unreal Engine allows our artists to create beautiful visuals faster utilizing real-time ray tracing, and the open-source code and access to technologies such as nDisplay allowed us to customize the engine to suit our needs and innovate quickly.”One of the most impressive aspects of the project from a visual standpoint is the clarity of the projections the team was able to achieve. This was no walk in the park. “The biggest challenge was rendering 12 different cameras to 12 projectors and ensuring everything, down to each particle, was absolutely frame-locked,” explains Vee. “If it wasn't frame-locked, the projection would be a blurry mess.”
Image courtesy of SPP and Imerza
Unreal Engine, NVIDIA Quadro cards, and nDisplay technology were the key to frame-locking the 24 million pixels being projected and rendered at over 90 frames per second.
For the video walls and touchscreens, IMERZA also used Quadro RTX 6000 cards, leveraging real-time ray tracing to give extra realism to the scenes. In order to keep everything running efficiently at 4K, the team used Blueprint-driven logic to toggle between RTX-GI and screen-space GI based on view distance. RTX capability can be turned on and off within materials using dynamic RTX reflections based on a material node, driven by camera visibility and distance.
Vee is equally emphatic about the overall impact real-time technology is having on real estate and urban development. “It isn’t often something comes along that can revolutionize an industry. Something that revolutionizes several industries at once happens maybe once a century,” he says. “Unreal Engine is doing just that. AEC, film production, product design, vehicle design, training and manufacturing—even creating new industries. It’s exciting, and I feel lucky to be a part of it.”
For the video walls and touchscreens, IMERZA also used Quadro RTX 6000 cards, leveraging real-time ray tracing to give extra realism to the scenes. In order to keep everything running efficiently at 4K, the team used Blueprint-driven logic to toggle between RTX-GI and screen-space GI based on view distance. RTX capability can be turned on and off within materials using dynamic RTX reflections based on a material node, driven by camera visibility and distance.
Real-time technology revolutionizing industries
Interactive workflows like those behind the Water Street Tampa Marketing Center project are ironing out some of the friction that exists in the traditional architectural design process. “I could go on for days about this, but I’ll keep it simple,” says Vee. “The majority of people have an extremely difficult time visualizing design. Real-time technology allows everyone to see the same thing and make decisions faster. This translates to reduced costs, fewer change-orders, and overall better design.”Vee is equally emphatic about the overall impact real-time technology is having on real estate and urban development. “It isn’t often something comes along that can revolutionize an industry. Something that revolutionizes several industries at once happens maybe once a century,” he says. “Unreal Engine is doing just that. AEC, film production, product design, vehicle design, training and manufacturing—even creating new industries. It’s exciting, and I feel lucky to be a part of it.”
Get Unreal Engine today!
Get the world’s most open and advanced creation tool.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.