In-Cabin AI First, Software-Defined Car Next—Mercedes-Benz Unveils EV Strategy

Article By : Junko Yoshida

CES 2021 has clarified an emerging theme among automakers: First comes in-cabin AI, then software-defined cars.

CES 2021 has clarified an emerging theme among automakers: First comes in-cabin AI, then software-defined cars. Typifying the trend, Mercedes-Benz is integrating in-cabin AI into its upcoming EV model with a user interface that qualifies as state of the art (for the time being).

The company showed off a single gigantic Gorilla Glass display at CES that unifies three screens: instrument cluster, infotainment, and passenger displays. Called the Mercedes-Benz User Experience (MBUX) Hyperscreen, the 141-cm (about 4.5-foot) screen extends pillar to pillar across the cabin.

However, as Mercedes-Benz chief Ola Källenius claimed during the company’s CES press conference, size isn’t everything. “This is a user interface that does not distract the driver,” he said.

With its MBUX Hyperscreen, the German carmaker is evidently maneuvering to one-up Tesla, seeking to lure EV customers toward Mercedes-Benz’s upcoming EQV.

MBUX Hyperscreen - Mercedes-Benz
MBUX Hyperscreen incorporates three screens — the instrument cluster, infotainment, and passenger displays — into one. (Source: Mercedes-Benz)
Instrument cluster - Mercedes-Benz
Instrument cluster for the driver (Source: Mercedes-Benz)

‘Zero Layer’

The EQV’s MBUX Hyperscreen will offer a “zero layer” interface, meaning that drivers will not have to scroll through sub-menus or enter voice commands to access a needed display, according to Mercedes-Benz. Common tasks are programmed to be immediately available. Navigation is always at the center of the screen.

MBUX Hyperscreen Navigation - Mercedes-Benz
Navigation is always at the center of the MBUX Hyperscreen. (Source: Mercedes-Benz)

The MBUX Hyperscreen’s AI software is trained to learn each driver’s preferences and habits, according to the company. Central to that capability is an underlying architecture supplied by Nvidia’s hardware and software platform.

AI software - Mercedes-Benz
The AI software learns each driver’s preferences and habits. (Source: Mercedes-Benz)

Partnership with Nvidia

Mercedes-Benz already uses Nvidia chips to enable the AI voice assistant, AV cockpit, interactive graphics, and an augmented-reality head-up display in its flagship S-Class vehicles.

Neither Mercedes-Benz nor Nvidia, however, has disclosed the specific chips designed into the EQV. The only hardware-component disclosures made thus far identify an eight-core CPU, 24-GB RAM, and 46.6-GB/s memory bandwidth. The in-vehicle instruments run on Linux.

That said, “in-cabin AI is one of the clear trends among OEMs,” said Danny Shapiro, Nvidia’s senior director of automotive. AI enables carmakers to offer an interface that shows information relevant to the driver and occupants, with alerts specific to the time of day, he said. The AI-driven UI customizes information by studying and remembering user behavior. Shapiro also mentioned that the smarts in the UI work not just for the driver and co-driver but also for passengers.

As Källenius noted during the press conference, “UI will find you.”

Software-defined Car

In addition to in-cabin AI, a new direction for many carmakers that was evident at CES is a desire to offer “software-defined vehicles”—à la Tesla—capable of over-the-air updates that can add and activate new features to vehicles already purchased and on the road. Mercedes-Benz is no exception.

Building on their long partnership, the automaker and Nvidia have embarked on “a much bigger project,” said Shapiro. In a deal announced in June, the companies will put Nvidia’s Orin software-defined platform into every automated Mercedes-Benz driving system starting in 2024. That’s also the time frame in which Mercedes-Benz plans to launch its own MBUX operating system.

The Orin SoC, which consists of 17 billion transistors, integrates Nvidia’s next-generation GPU architecture and ARM Hercules CPU cores, in addition to new deep-learning and computer-vision accelerators. The SoC, although not yet complete, will be the linchpin of Mercedes-Benz’s autonomous future.

Nvidia isn’t alone in its pursuit of enabling software-defined or “user defined” vehicles. NXP just announced BlueBox 3.0, a new development platform for car OEMs looking to add a wider portfolio of user-defined applications and AI.

Nvidia has been on that path since late 2019, when it unveiled Orin. According to the company, Orin is designed to handle myriad applications and deep neural networks while achieving systematic safety standards such as ISO 26262 ASIL-D.

Leading up to CES 2021, Nvidia disclosed that leading Chinese EV makers Nio, Li Auto, and Xpeng were developing their intelligent electric fleets on Nvidia Drive. Nio and Li Auto will use Orin in their future vehicles; Xpeng is producing new EVs based on Nvidia’s Xavier SoC.

At a recent event, Nio unwrapped its ET7 sedan, slated to ship in 2022. The car will feature Adam, a new Nvidia-powered supercomputer that uses Orin to deploy advanced automated driving technology.

“Every carmaker that has a desire to develop a Tesla-like software-defined vehicle is turning to us,” said Nvidia’s Shapiro.

Leave a comment