Image courtesy of GMC
USA’s largest car maker builds HMI systems in Unreal Engine
October 7, 2020
When you sit in the driver’s seat of a modern car today, the way you interact with the vehicle is different from a few years ago. Buttons and screens dominate the dashboard, steering wheel, and instrument pane, providing features that may be new to you. Electrification is at the root of many of these advances, with digital hardware such as cameras, sensors, and displays, combined with cutting-edge technology like Unreal Engine, forming a framework on which to build new experiences.
Human-machine interfaces (HMI) are one of the next great frontiers for the automotive industry. At Epic Games, we’ve long been interested in the way the requirements for HMI development overlap with those of game development. Much of the functionality required to create HMI systems has been available in Unreal Engine for some time.
Today, we’re proud to officially announce our human-machine interface initiative, combining new partnerships and workflows to enable the creation of automotive HMI, infotainment, and digital cockpit experiences. Our dedicated HMI team is committed to this effort and we are continually adding new features to benefit all manufacturers.
Additionally, we can announce that General Motors’ recently announced GMC HUMMER EV will be the first vehicle to use Unreal Engine in its digital cockpit.
Human-machine interfaces (HMI) are one of the next great frontiers for the automotive industry. At Epic Games, we’ve long been interested in the way the requirements for HMI development overlap with those of game development. Much of the functionality required to create HMI systems has been available in Unreal Engine for some time.
Today, we’re proud to officially announce our human-machine interface initiative, combining new partnerships and workflows to enable the creation of automotive HMI, infotainment, and digital cockpit experiences. Our dedicated HMI team is committed to this effort and we are continually adding new features to benefit all manufacturers.
Additionally, we can announce that General Motors’ recently announced GMC HUMMER EV will be the first vehicle to use Unreal Engine in its digital cockpit.
We’ll be able to reveal more upon the launch of the all-electric HUMMER truck on October 20, 2020, but in the meantime, let’s explore the existing HMI functionality that has convinced America’s largest automobile manufacturer to opt for Unreal Engine—and take a look at some of the new features and further investment for HMI we have planned for the future.
Design-driven development
Unreal Engine helps automotive companies create compelling vehicle experiences by better enabling design-driven development.In traditional HMI workflows, UI/ UX designers produce “hero screens” or reference images that outline how the UI is supposed to look. They rarely get to see their designs in action soon enough to iterate on them, because it’s up to engineers to implement the design and the functionality into the vehicle.
Unreal Engine provides a toolset that supports version control, visual scripting via the Blueprint system, and a C++ codebase that enables more and more functionality to be exposed. This enables designers to deliver not just hero screens, but to implement functionality in their designs as well.
Image courtesy of Vectorform
With one-click deployment of a project to devices like iOS and Android, designers can also see their design in near real time in a vehicle. Fast iteration like this gives them the immediate feedback they need to do critical testing of designs.
For example, a proposed HMI module may have trouble areas when the sun is low and behind the vehicle. Rather than the tester filing a ticket and sending an awkward photograph of the issue to the designer, the designer can sit in the vehicle and make the adjustments themselves.
This efficiency improves designs and takes less time, allowing for more polish on the overall experience. Efficiency for the designer helps engineers as well, who are now freed to work on critical technical features and monitor performance with the designers to ensure a smooth experience.
Assets created by product design teams working on HMI can then later be repurposed for use in marketing materials. Ultimately, this makes for more efficient use of company resources, and keeps the designs consistent across an entire brand.
The functionality of backend services like navigation, music, and car settings services can be exposed via a clean API to the visual scripting language in Blueprint. For example, navigation turn-by-turn directions could fire a C++ delegate, which can fire a Blueprint event. Designers can then hook up camera motions or widgets to the Blueprint event, triggering animations on screen. Note, the designer is the final enabler of this functionality in their design—not the engineer. This is at the core of enabling design-driven development.
In addition to the Blueprint visual scripting system and extensible C++ codebase, HMI developers have Unreal Engine’s comprehensive toolset at their fingertips. This includes the Niagara visual effects system used for creating and previewing particle effects in real time.
For example, a proposed HMI module may have trouble areas when the sun is low and behind the vehicle. Rather than the tester filing a ticket and sending an awkward photograph of the issue to the designer, the designer can sit in the vehicle and make the adjustments themselves.
This efficiency improves designs and takes less time, allowing for more polish on the overall experience. Efficiency for the designer helps engineers as well, who are now freed to work on critical technical features and monitor performance with the designers to ensure a smooth experience.
Assets created by product design teams working on HMI can then later be repurposed for use in marketing materials. Ultimately, this makes for more efficient use of company resources, and keeps the designs consistent across an entire brand.
New creative options and marketing opportunities
For those involved in building HMI systems, access to Unreal Engine’s extensible C++ API opens up new opportunities.The functionality of backend services like navigation, music, and car settings services can be exposed via a clean API to the visual scripting language in Blueprint. For example, navigation turn-by-turn directions could fire a C++ delegate, which can fire a Blueprint event. Designers can then hook up camera motions or widgets to the Blueprint event, triggering animations on screen. Note, the designer is the final enabler of this functionality in their design—not the engineer. This is at the core of enabling design-driven development.
In addition to the Blueprint visual scripting system and extensible C++ codebase, HMI developers have Unreal Engine’s comprehensive toolset at their fingertips. This includes the Niagara visual effects system used for creating and previewing particle effects in real time.
Niagara opens up a world of creative opportunities for HMI design: charging effects could show what an electric vehicle is doing to its battery state, different effects could show how different features in a car work, and for level four autonomous vehicles where the vehicle needs to earn the trust of the driver, what better way than to present them with an interpretation of what the car is seeing?
With over twelve million Unreal Engine licensees production testing the engine’s features everyday across all kinds of devices, the toolset is battle-hardened and can be relied upon to perform for commercial use.
And when it comes to marketing the new vehicle, Unreal Engine attracts an appealing young, entrepreneurial demographic that car companies can tap into simply by using the engine. Popular globally played titles like vehicular soccer video game Rocket League offer a unique co-marketing ecosystem for automotive companies to capitalize on.
In some production vehicles, HMIs take 10 seconds or more to load. Significant improvements have been made to HMI startup processes in Unreal Engine, enabling it to boot extremely quickly. Content that is not needed at startup can be loaded after the initial boot, reducing boot time even further.
Unreal Engine provides best-in-class visuals for production HMI, with car paint materials and reflections that bring the highest quality real-time graphics to the vehicle. Designers also have the opportunity to work with various automotive materials and shaders that extend themselves to mobile HMI.
With over twelve million Unreal Engine licensees production testing the engine’s features everyday across all kinds of devices, the toolset is battle-hardened and can be relied upon to perform for commercial use.
And when it comes to marketing the new vehicle, Unreal Engine attracts an appealing young, entrepreneurial demographic that car companies can tap into simply by using the engine. Popular globally played titles like vehicular soccer video game Rocket League offer a unique co-marketing ecosystem for automotive companies to capitalize on.
HMI-specific features in Unreal Engine
We are actively implementing HMI-specific features in Unreal Engine—and there are a number HMI workflows you can leverage right now.In some production vehicles, HMIs take 10 seconds or more to load. Significant improvements have been made to HMI startup processes in Unreal Engine, enabling it to boot extremely quickly. Content that is not needed at startup can be loaded after the initial boot, reducing boot time even further.
Unreal Engine provides best-in-class visuals for production HMI, with car paint materials and reflections that bring the highest quality real-time graphics to the vehicle. Designers also have the opportunity to work with various automotive materials and shaders that extend themselves to mobile HMI.
A comprehensive set of visual effects features that have been hardened by game development give automotive designers a robust set of tools to express creativity in their HMI designs. Automotive gauges—such as energy visualizations showing energy consumption and ADAS visualization describing how the car perceives its environment—have yet to be defined in an iconic way. With digital screens and the Unreal Engine toolkit at your disposal, the possibilities for these and the next iconic gauge (RPM, for example) are suddenly wide open.
Unreal Engine’s OpenGL ES3.2 and Vulkan support on automotive hardware facilitates state-of-the-art rendering that brings the best visuals to the screen efficiently, while Unreal Motion Graphics UI Designer (UMG) provides an extensible and powerful built-in UI framework that enables designers to use Blueprints backed by C++ to easily create the HMI widgets they need.
The engine’s suite of profiling tools, such as Unreal Insights, ensures designers can keep applications running smoothly, guaranteeing high-performance and fluid user interactions.
Downloadable content packs can be dynamically loaded from Over-the-Air (OTA) updates quickly, enabling you to rapidly change functionality and content in your HMI experience. And fast style switching gives users the ability to make their car their own by changing every aspect of the user interface with the press of a button.
As vehicles cruise towards level 5 autonomy, in-car entertainment systems will become a much bigger factor in purchasing decisions. With game development in its DNA, Unreal Engine is the perfect platform to develop gamification and entertainment in vehicles. What if an automotive brand had their own games that only their vehicle users can access? Or drivers could customize their HMI visuals and voice assistant with the personality of their favorite video game character? With Unreal Engine as the HMI platform, opportunities such as this become possible.
We’ll be rolling out more HMI-specific features in Unreal Engine in the coming months and throughout 2021—so stay tuned for more exciting developments like our partnership with General Motors.
Unreal Engine’s OpenGL ES3.2 and Vulkan support on automotive hardware facilitates state-of-the-art rendering that brings the best visuals to the screen efficiently, while Unreal Motion Graphics UI Designer (UMG) provides an extensible and powerful built-in UI framework that enables designers to use Blueprints backed by C++ to easily create the HMI widgets they need.
The engine’s suite of profiling tools, such as Unreal Insights, ensures designers can keep applications running smoothly, guaranteeing high-performance and fluid user interactions.
Downloadable content packs can be dynamically loaded from Over-the-Air (OTA) updates quickly, enabling you to rapidly change functionality and content in your HMI experience. And fast style switching gives users the ability to make their car their own by changing every aspect of the user interface with the press of a button.
Designing better HMI experiences
Our dedicated HMI team believes that designers who are better enabled to execute on their vision and design are best equipped to create compelling in-car experiences. Likewise, we believe that engineers who are freed up to work on critical high-value technical features can provide more value to the product itself. All of this adds up to a better user experience. We're making this easier to achieve than ever by forming new alliances with HMI technology leaders including BlackBerry QNX, Mapbox, Qualcomm, Siili Auto, and Vectorform.As vehicles cruise towards level 5 autonomy, in-car entertainment systems will become a much bigger factor in purchasing decisions. With game development in its DNA, Unreal Engine is the perfect platform to develop gamification and entertainment in vehicles. What if an automotive brand had their own games that only their vehicle users can access? Or drivers could customize their HMI visuals and voice assistant with the personality of their favorite video game character? With Unreal Engine as the HMI platform, opportunities such as this become possible.
We’ll be rolling out more HMI-specific features in Unreal Engine in the coming months and throughout 2021—so stay tuned for more exciting developments like our partnership with General Motors.
Dive into real-time automotive workflows
Download for free today to start exploring Unreal Engine’s HMI features. And don’t miss our upcoming automotive-focused episode of The Pulse to hear more about real-time workflows in the automotive industry!