Anyone who believes vision-based DMS is unnecessary or obsolete is thoroughly mistaken. This is technology which is finally emerging into the light.
Anyone who believes vision-based driver monitoring systems (DMS) are unnecessary or obsolete has not been paying attention to the recent market developments covered in EE Times. Both underreported and underappreciated — especially among U.S.-based carmakers and consumers — this is technology which is finally emerging into the light.
I presented the DMS keynote at the recent EE Times event “Roadmap to the Next-Gen EV&AV” and I used the following slide to show the evolution of the ADAS forward camera market as a reference to the growth opportunities that lie ahead for DMS and in-cabin monitoring systems (ICMS).
The ADAS forward camera market has thus far been dominated by Mobileye, with a market share that I estimate at about 60%. Mobileye was purchased in a very high-profile deal by Intel in 2017 for $15.3 billion, but what this slide demonstrates is the humble beginnings of the ADAS forward camera market in the ten years before the Intel acquisition.
Form F-1 filed by Mobileye in 2014 with the Securities and Exchange Commission in preparation for its listing on the New York Stock Exchange is now a historical document, but nonetheless is hugely informative for analysts like me who seek precise details of the past as useful guides to the future. Below is a screenshot of one page I found particularly insightful.
This shows not only how extensively Mobileye had already developed its direct relationships with the carmakers (OEMs) by 2014, but also with the tier one suppliers. Note in the tier one supplier list that Magna is named first, and as this August 2006 article highlights, Magna was the first tier one to truly grasp the significance of the opportunity presented by ADAS forward camera technology and to bring its engineering prowess to a partnership with Mobileye.
Why am I writing about events from fifteen years ago? Read on and I will explain the significance of a recent announcement which was mostly lost among the hoopla and razzmatazz of Nvidia’s recent 2021 GPU Technology Conference.
Magna enters the OMS mirror market
As shown in the following slide from Seeing Machines, the market for automotive interior vision systems is evolving from driver monitoring, using near-field-of-view (NFOV) image sensors, to occupant monitoring systems (OMS) which use wide-field-of-view (WFOV) image sensors to analyze all the occupants in the cabin.
Unlike ADAS forward camera, Magna wasn’t anywhere close to being the first tier one to offer DMS, having been beaten to series-production by Aptiv, Denso, Joyson, LG Electronics, Mitsubishi, Mobis and Veoneer. However, Magna looks set to steal a march on its competitors in the area of OMS and specifically where the optical components and associated electronics are integrated in a system-level solution within the interior mirror.
I have screenshotted one frame of the video and presented it below, which shows the disassembled mirror and reveals the integration of the CMOS image sensor and the PCB with the infrared LED emitters and image processor. This is an extraordinary feat of engineering made all the more impressive by the apparent absence of a fan to cool the image processor.
Magna isn’t the first tier one to demonstrate an interior mirror with integrated optical components, as the following images of Continental’s interior camera (ICAM) and Gentex’s in-cabin sensing mirror show. However, just as it accomplished for the ADAS forward camera market, I believe Magna now has first-mover advantage offering carmakers an all-in-one OMS mirror solution.
Although I have seen no technical specifications for Magna’s OMS mirror, my judgment is that it most likely combines a CMOS image sensor from OmniVision (probably the 2-megapixel OV2311 ), a Xilinx FPGA (either Zynq-7000 or UltraScale+ MPSoC) for high-performance power-efficient image processing, with occupant monitoring software provided by Seeing Machines.
Many of the technical issues related to occupant and driver monitoring were covered in detail in the recent EE Times panel discussion on this subject, the summary of which can be read here.
As my keynote discusses, shipments for DMS/OMS are set to grow dramatically throughout this decade as advisory bodies such as Euro NCAP and mandates such as the European General Safety Regulation direct carmakers to adopt technology which reliably monitors drivers for distraction, drowsiness and impairment. This applies even to low-cost, mass-market models and irrespective of whether the vehicle has a hands-free driving function, such as GM’s SuperCruise or Ford’s BlueCruise which both use robust vision-based DMS to accurately assess driver engagement.
The placement of the IR optical components inside the cabin is a difficult but crucial decision for carmakers and a complicated engineering challenge for tier one suppliers. Current implementations of driver monitoring have typically seen the image sensor sited on the steering column (Cadillac CT6), the instrument cluster (BMW X5), or on the center stack console (Subaru Forester).
The easiest, fastest and most cost-effective location for carmakers to integrate the necessary IR optical components and associated electronics is the interior mirror. With Euro NCAP set to add DMS/OMS testing to its 5-star safety rating in either 2023 or 2024, Magna’s recent announcement of a fully-integrated interior monitoring mirror therefore really matters.
Tesla reinvents the [OMS] wheel
Tesla has been making headlines recently as it seeks to repurpose the interior camera installed in the Model 3 and Model Y to provide driver and occupant monitoring functions. We can see more of its progress in this video, in particular how noisy the driver-state classifier is and how easily the system can be spoofed with a photograph.
We could politely describe Tesla’s OMS algorithms as being at the early prototype stage and while likely to improve as the vehicle fleet gathers data, the reliability of the system for safety purposes will ultimately be limited by the absence of IR illumination. No over-the-air update can overcome that design oversight, thus preventing the image sensor from working in darkness, as shown in this video.
Now that Consumer Reports has proven how easy it is to defeat the torque-based driver monitor in Tesla’s Autopilot, other carmakers look certain to partner with tier one suppliers such as Magna and software suppliers such as Seeing Machines to integrate reliable driver and occupant monitoring systems which bring about a meaningful improvement in road safety, as discussed in the video below.
Any safety system that can be defeated by an orange isn’t safe. Will Elon Musk and Tesla have the humility to admit they were wrong about the significance of DMS and OMS? Anything is possible with Tesla but in this instance, I’m not holding my breath. For everyone else, the adoption of robust driver and occupant monitoring is just beginning.
This article was originally published on EE Times.
Colin Barnden is principal analyst at Semicast Research and has over 25 years of experience as an industry analyst. He is considered a world expert on market trends for automotive vision-based driver monitoring systems (DMS). He holds a B.Eng. (Hons) in Electrical & Electronic Engineering from Aston University in England and has covered the automotive electronics market since 1999.