CES 2021 revealed in-cabin AI as the hottest trend in automotive. This has profound implications, with Qualcomm potentially set for its DMS dominance.
CES 2021 revealed in-cabin AI as the hottest trend in automotive. This has profound implications for the driver monitoring system (DMS) market, potentially setting Qualcomm up for its DMS dominance.
For a technology which most people know very little about and that many experts have already written-off as obsolete, the DMS market is experiencing a period of intense activity, although hardly any of it gets reported.
There are four clear strategies that I have identified for automotive-grade DMS: digital cockpit/in-vehicle Infotainment (IVI), highway assist, China and NCAP. This is the first of two articles in which I shall take each of those four strategies and offer my assessment of the developments, key issues and leading DMS vendors.
This article focuses on the two areas driving state-of-the-art performance for automotive-grade DMS, namely digital cockpit/IVI and highway assist. A second article shall follow at a later date looking at the distinctly separate, lower performance and decidedly more cost sensitive strategies covering China and NCAP.
The digital cockpit and in-vehicle infotainment (IVI) domain will form the heart of the in-cabin AI revolution about to descend on the automotive industry. Ahead lies a breathtaking battle between Qualcomm and Nvidia for the immersive user experience that will cover almost every new and established automaker by mid-decade. Think of everything you have ever loved about the content rich, high-definition, interactive experience of smartphones coming to your car. Finally.
Qualcomm revealed the breadth of its automotive ecosystem in November during a keynote by division head Nakul Duggal at the Reuters Automotive Summit. You can watch the presentation in full, but I took a screenshot of the key information.
This image shows essentially every Tier 1 that is active in the digital cockpit and IVI sector, including: Alps Alpine, Aptiv, Bosch, Continental, Denso, Garmin, Harman, LG, Mobis, Panasonic and Visteon.
On the right we see the Qualcomm partnerships with Amazon and Google, almost certainly for the integration of Alexa Auto and Android Automotive OS. Qualcomm is also partnered with Blackberry for integration of the QNX CAR Platform for Infotainment.
Seeing Machines is presented seemingly as the exclusive provider of DMS for the Qualcomm automotive ecosystem. In my opinion this is because Seeing Machines is the clear technology leader for automotive-grade DMS, having refined the reliability of its optical path and proven the robustness of its vision algorithms from starting out in the most hostile, least hospitable transportation application possible: Mining trucks.
Seeing Machines has also validated the stability of its head-, face- and eye-tracking software in full flight simulators, where sophisticated eye-gaze analysis has begun to be used to track the instrument scan patterns of trainee pilots. Where a pilot looks, how often and for how long are all fundamental to the skill and proficiency of flying an aircraft.
In-time this technology will be added to airliner cockpits, tracking glance patterns and analyzing distraction, drowsiness and workload levels both in the pilot flying and pilot monitoring. Twenty years of R&D and the expertise and experience from working in the mining and aviation sectors is all being embraced by Qualcomm to bring robust DMS to a car cockpit near you starting around 2024.
DMS for the digital cockpit and IVI sector is going to push the boundaries of what is possible with precision eye-gaze tracking, leading to an immersive user experience that you will barely believe can be offered in a mass-market car. Just as with premium smartphones, performance and functionality will be more important than lowest price, as the battle of the Silicon Valley tech giants moves on from cellphones to cars.
Automakers that want to compete with Tesla will work with Nvidia, but automakers that want to redefine the possibilities of the digital cockpit will work with Qualcomm. Yet again, Intel missed the key technology pivot and is nowhere to be seen with a viable DMS strategy in automotive.
Seeing Machines looks extremely well positioned to own the digital cockpit and IVI sector by mid-decade. The exception is those vehicles running Nvidia-powered Cerence Look using eye-tracking from Smart Eye, such as the upcoming Mercedes-Benz flagship, the EQS. In my opinion, there is only one contender for DMS supplier in any future iCar.
Responsible automakers look set to universally embrace vision-based DMS as a means to prevent the automation complacency that we have witnessed by Tesla owners in recent years. The ineffectiveness of Tesla’s steering wheel torque sensor to monitor driver attention when Autopilot is engaged is well documented and demonstrated perfectly in this video.
Introduced back in 2017, Cadillac Super Cruise was the first example of a highway assist system to effectively monitor driver attention with vision-based DMS, using software from Seeing Machines.
During 2021 BMW looks set to launch the latest generation of its automated driving system on the new iX, with Ford ready to introduce Active Drive Assist on both the F-150 and Mustang Mach-E, and Mercedes-Benz on the S-Class. My research suggests all three automakers have followed Cadillac’s lead and worked with Seeing Machines.
Most automakers have abandoned any short-to-medium term plans for autonomous driving in privately-owned passenger vehicles, predominantly on grounds of cost, liability and reliability. Automakers are instead opting for nevertheless sophisticated automated driving systems, but those where the human driver is responsible — meaning legally liable — for the driving task at all times.
This has resulted in the emergence of terminology such as “L2+” and “L2++” and “L3-” and even “L2.99,” as automakers and Tier 1 suppliers find themselves limited to systems above L2 but below L3 and seeking to somehow differentiate their product offering from that of the competition.
Some of the most interesting highway assist features in development today are from Veoneer, which took a different approach from almost every other supplier to blend the best aspects of human drivers with the best aspects of machine intelligence to create what it calls “Collaborative Driving.” I expect to be writing a lot more about collaborative, or augmented, driving in the months ahead, revisiting an idea I first explored in April 2019.
DMS for highway assist systems will be very high performance, but below the level required for driver monitoring in the digital cockpit and IVI sector. I anticipate this will mostly be a battle between Nvidia, Qualcomm and Xilinx. Support for functional safety will be critical, as I see a rapid convergence of the driver monitoring and ADAS functions to enhance safety on public roadways.
Mobileye will quickly come to understand the consequences of focusing its engineering resources onto autonomous driving, mapping (REM), driving policy (RSS), FMCW-based lidar and software-defined radar, all the while overlooking the most important trend in the market which is towards DMS. In comparison, Qualcomm identified and embraced that trend, perhaps from its partnership with Veoneer.
Intel is about to discover that even $15.3 billion can buy a company with a myopic product strategy and it will be pained to realize it could probably have hedged its bets in 2017 and for mere millions of dollars also bought Seeing Machines. The fate of Mobileye’s CEO Amnon Shashua now depends on delivering on his promise of “consumer AVs” in 2025 and I am seriously skeptical of that prediction.
By mid-decade Seeing Machines looks set to dominate the supply of DMS in the highway assist segment, owing to its significant technology lead over competitors such as Cipia (formerly Eyesight), Jungo, Mitsubishi, Smart Eye and Xperi in areas such as optical path expertise, human factors research and provision of reliable, robust and stable DMS signals.
Qualcomm’s virtual technology showcase, called “Automotive Redefined” is coming on January 26-27. It will be presented by Nakul Duggal, who has led Qualcomm’s automotive operation only since September, and President and incoming CEO Cristiano Amon, who headed Qualcomm’s diversification strategy away from cellular and into automotive.
I am expecting some jaw-dropping news to come out of this event. As the Snapdragon Summit in December showed us, Qualcomm likes to involve partners and customers when making major announcements, so I anticipate an appearance by Veoneer CEO Jan Carlson and the formal launch of the Qualcomm/Veoneer partnership.
We might also see GM’s CEO Mary Barra, emboldened by her excellent CES keynote which was heavy on EV developments and environmental issues, but surprisingly light on electronics and technology.
Although forecasts are only ever lucky or wrong, I predict that GM could be revealed as the first automaker to adopt the Qualcomm/Veoneer solution — possibly as the underpinnings of Ultra Cruise — and further that it will use the next generation Google Android Automotive OS and kickstart the trend to the digital cockpit.
I don’t know the specifics, but I shall be watching next week and am certain the event will generate a lot of exciting headlines. My best wishes go to Nakul and Cristiano and to the entire team at Qualcomm, who are set to demonstrate once more that the company making the least noise is almost always the one making the most progress.