Am I the only one sensing the DMS disaster coming to Mobileye and Nvidia? Right. It's not exactly the most popular thread you'd encounter either in technical pubs or in social media...
Am I the only one sensing the DMS disaster coming to Mobileye and Nvidia? Right. It’s not exactly the most popular thread you’d encounter either in technical pubs or in social media.
Speaking of social media, it is the least appealing aspect of my role as an analyst. I forgo Facebook and tolerate Twitter, but lean-in to LinkedIn as an altogether more professional and courteous platform. However, as an EE Times contributing writer no input gets my attention more than direct feedback from readers on www.eetimes.com.
Commenting in response to my article Qualcomm’s Automotive Redefined: Collaborate or Go, a reader wrote:
Just using DMS & seeing machines as example, author writes off mobileye & nvidia. seems like another promotion article for Qualcomm, almost 3rd one in last 2 weeks.
I offer these points in response:
As it relates to driver monitoring systems (DMS), the technical merits of the Qualcomm and Seeing Machines offering over Mobileye and Nvidia are evident to me, but clearly warrant further discussion. So, let’s get technical and go into the specifics.
Seriously Skeptical gets seriously technical
Let us begin with a direct comparison of the DMS offerings from Seeing Machines and Nvidia by revisiting two videos, one from each supplier. I ask you to remember that these videos are PR and thus will showcase the best technical accomplishments of the company.
First up, Nvidia:
And Seeing Machines:
What is evident to me from the Nvidia video is that it has approached driver monitoring entirely from a software perspective (AI for vision algorithms) not as a complete system. System-wide expertise for DMS begins with the optical path, as we can see in this slide from Seeing Machines.
Reliable, robust and stable DMS signals depend on the quality of the optical path. Design challenges include IEC 62471 for eye safety, quantum efficiency when working with 940nm IR light and proprietary illumination techniques to maximize contrast of the eye glints to obtain an accurate eye-gaze vector.
Nvidia is undoubtedly adept at developing vision algorithms using AI and deep learning, but needs to greatly expand its 940nm IR optical path expertise, which is the domain of imaging specialists such as ON Semiconductor and Omnivision. At the beginning of January Omnivision announced its OAX 8000 DMS ASIC.
For state-of-the-art DMS, the CMOS image sensor, image processor and imaging algorithms are increasingly being considered together as a signal chain, rather than individually. This imaging signal chain results in higher performance, higher signal availability and lower power consumption. Nvidia needs to expand its thinking beyond imaging algorithms and consider the entire optical path to compete with the Qualcomm and Seeing Machines offering.
Next, let us consider training data. In an era of AI and deep learning, the company with the largest data set has the greatest competitive advantage — Amazon, Facebook and Google teach us that. For DMS, the company with the most training data is Seeing Machines, gathered from more than 26,500 heavy trucks installed with its aftermarket “Guardian” system capturing on-road, in-cabin video.
Seeing Machines has captured almost 6 billion kilometers of on-road naturalistic driving data, more than 8.3 million distraction events and about 165,000 fatigue interventions over the last 12 months.
Guardian uses an automotive-grade IR optical path, meaning the training data is captured using the IR spectrum. Training using full-color (RGB) still images and video results in less robust DMS performance in real-world conditions. Nvidia needs to massively expand its training data set to improve the robustness of its DMS algorithms to compete with the Qualcomm and Seeing Machines offering.
Having detected the driver is distracted, drowsy, or disengaged from the driving task, what does the DMS do? My third point is understanding of human factors science and behavioral research to enable automakers to use signals from the DMS to safely modify the behavior of the driver, the vehicle, or both. Nvidia needs to develop human factors expertise to compete with the Qualcomm and Seeing Machines offering.
As the video shows, Nvidia’s DMS is rudimentary, covering head pose estimation, eye size and mouth size, but lacks eye-gaze vector. In terms of technology leadership for DMS, I approximately rank the suppliers as follows: Seeing Machines, Smart Eye, Cipia (formerly Eyesight), Jungo, Xperi and Mitsubishi.
So, I’m not writing Nvidia off at all, but based on the evidence, its in-house DMS offering ranks no better than seventh. Now name the automaker that aspires to use the seventh best safety technology?
Intel saving Mobileye’s life
Mobileye has developed incredible technology on the road to autonomous driving, including mapping (REM), driving policy (RSS), FMCW-based lidar and software-defined radar. We can see its long-term strategy to proceed through the levels of driving automation as specified in SAE International’s standard J3016 from the EyeQ roadmap.
Mobileye expected to offer L3 with EyeQ4 in 2017 and L4-5 with EyeQ5 in 2020. Fast-forward to today and many commentators are questioning the plausibility of L3 owing to the problems of the machine-to-human handover, while I question the plausibility of L4 passenger vehicles on the grounds of cost, legal liability and reliability. At CES 2021 in January, Mobileye’s CEO Amnon Shashua offered a new target of “L4 consumer AVs” in 2025.
Euro NCAP signaled its intent to include DMS testing as part of its 5-star safety rating back in 2017, while the European Commission introduced plans to include DMS in all on-road vehicles with four or more wheels in May 2018. The resulting revisions to the European General Safety Regulation (GSR) were voted into law in November 2019 and DMS is required for any new model going for type-approval in Europe from mid-2024. Similar legislation is likely to pass in the United States under the Biden Administration.
In buying Mobileye, Intel looks to have paid $15.3 billion for an ADAS processor company that’s been asleep at the wheel with regards to the regulatory environment for DMS. Qualcomm, meanwhile, agreed partnerships with Arriver for the vision and driving stack, Valeo for parking and Seeing Machines for DMS to create a solution that solves all of its automaker customers’ problems at once.
So, let’s follow Omnivision’s lead and sketch-out a DMS processor. Using the OAX 8000 DMS ASIC as a template, Intel’s fastest route to market is to license the Occula NPU DMS core from Seeing Machines, beef up the CPUs to quad-core Arm Cortex-A53 and add dual-core Arm Cortex-R5 as a safety island for ISO 26262 compliance.
I estimate Intel has about six months to get working silicon in the hands of automakers and Tier 1 suppliers. I think these timescales are impossible, but incoming CEO Pat Gelsinger is going to have to try something radical if Mobileye is to meet Euro NCAP and European GSR requirements with a system-wide solution to compete against Qualcomm from 2024 onwards. Rather than saving lives, Intel is now in a race to save Mobileye by developing the DMS technology it either discounted or dismissed.
By trying to win the race to L4 passenger vehicles, Mobileye and Nvidia ignored the growing importance of DMS and in so doing left the door wide open for Qualcomm to develop partnerships and an ecosystem to fulfill the automakers’ needs for digital cockpit/in-vehicle infotainment (IVI), highway assist at L2+ and solutions to meet the requirements for NCAP and GSR.
In years to come, entire MBA case studies will be devoted to analyzing this carelessness. But the main lesson is really very simple: Automotive is a heavily regulated, standards-based industry, so pay attention to the advisory and regulatory bodies first, before trying to out-smart your competitors.