In-Cabin Monitoring Driving ST’s Global Shutter Image Sensor Innovation, Growth

Article By : Stephen Las Marias

STMicroelectronics' global shutter portfolio captures distortion-free images in in-cabin monitoring applications.

The driver monitoring system (DMS) market is projected to reach $8.77 billion by 2032, growing at a compound annual growth rate (CAGR) of 7.5% over the 2022–2032 forecast period, according to Future Market Insights (FMI).

Mainly driving this growth is the increasingly stringent government regulations when it comes to the safety of drivers, passengers, and pedestrians. In fact, according to the World Health Organization, road accidents claim around 1.3 million lives every year, and between 20 and 50 million more people suffer non-fatal injuries, with many incurring a disability because of their injury.

Human error is one of the critical factors contributing to this figure. Speeding, for instance. An increase in average speed, according to WHO’s report, is directly related both to the likelihood of a crash and the severity of the consequences of the crash. Another issue is driving under the influence of alcohol and any psychoactive substance or drug.

Distracted driving is also a factor and is a growing concern at that. According to WHO, drivers using mobile phones are approximately four times more likely to be involved in a crash than drivers not using a mobile phone. It also slows reaction times, such as braking reaction time as well as reaction to traffic signals.

According to FMI, DMS are also essential for Level 3 and 4 autonomous vehicles, which require drivers to re-engage and ready to control the vehicle at vital points during the trip. FMI notes that the continuing development of autonomous vehicle as well as regulations governing them are expected to further create lucrative opportunities for the DMS market.

In-cabin monitoring driving image sensor innovations

Apart from driver monitoring, another safety feature in cars is cabin or occupancy monitoring.

Vincent Lin

“With driver monitoring, of course, we’re focusing on the driver. The first point, which is the most important—we need to know the driver’s attention or distraction or drowsiness, because these three behaviors will affect driving,” says Vincent Lin, Technical Marketing Manager, Imaging Division, Asia Pacific, at STMicroelectronics. “With cabin or occupancy monitoring, we’re more focused on passenger monitoring, especially child detection.”

Such system, CMS/OMS (cabin/occupancy monitoring system), is now included in ratings of the European New Car Assessment Program (Euro NCAP), which is a car safety performance assessment program that helps consumers, families, and businesses compare vehicles more easily and identify the safest choice for their needs.

One key component here is the in-cabin camera. For the DMS, which only needs to monitor the face of the driver, the lens doesn’t need to be very large, according to Lin. A 50° or 60° field of view (FOV) is sufficient, with a resolution of 1 to 2.3Mp.

“One important point is that with driver monitoring, you must use NIR [near infrared] camera because you also need to monitor a driver during night driving,” Lin says. “When you’re driving at night, in very dark environment, you need to have an illumination source on your face. In this case, since you cannot use visible light, you need to use NIR light.”

OMS is different because the system needs to monitor the whole cabin. “Because we need to do object detection or recognition, we need color image. And as the space is wider, of course, our lens needs to be wider and the resolution needs to be higher because we want to know all the details in this field,” says Lin.

According to Lin, the trend is toward the development of an all-in-one camera—which will integrate a camera for CMS/OMS and a camera for DMS. “In the occupancy monitoring, we need an RGB color image sensor, while we need an NIR image sensor for driver monitoring,” says Lin. “In this case, if you want to use a single camera to implement an all-in-one system, this camera needs to be RGB-NIR—a single camera supporting both RGB color and NIR mono.”

STMicroelectronics, one of the leading semiconductor providers worldwide, has a portfolio of high-speed image sensors utilizing “global shutter” for capturing distortion-free images in applications such as DMS and CMS/OMS.

“For the first generation, we have four products: the 1.6Mp VD5661A and the 2.3Mp VD5761A sensors for DMS applications; and for OMS, the VD6763A color RGB sensor and the all-in-one VD1762A sensor, which is RGB-NIR,” says Lin.

Measuring 2.6-by-2.5mm and 3.6-by-4.3mm, respectively, ST’s VD55G0 and VD56G3 are the smallest on the market in relation to resolution and feature low pixel-to-pixel crosstalk at all wavelengths, specifically near-infrared, ensuring high contrast for superior image clarity.

ST’s first generation global shutters feature high MTF, which means very high contrast; very flexible data management; and high dynamic range (HDR). ST’s advanced pixel technology, including full Deep Trench Isolation (DTI), enables extremely small 2.61-by-2.61μm pixels that combine low Parasitic Light Sensitivity (PLS), high Quantum Efficiency (QE), and low crosstalk in a single die layer—attributes that other pixel technologies cannot achieve simultaneously.

The ST approach enables a small pixel on a backside illumination (BSI) die, which allows space-saving vertical stacking of the optical sensor and associated signal-processing circuitry on the bottom die to achieve an extremely small sensor size and to embed a greater number of key features, including a fully autonomous optical flow block.

The bottom die is fabricated in ST’s 40nm technology and integrates digital and analog circuitry. The high-density, low-power digital circuitry incorporates hardware features including an exposure algorithm with statistics gathered from up to 336 zones, automatic defect correction, and automatic dark calibration. The fully autonomous low-power optical-flow block can calculate 2,000 movement vectors at 60fps.

Apart from DMS and OMS applications, these sensors are suited to augmented and virtual reality (AR/VR), simultaneous localization and mapping (SLAM), and 3D scanning.

“Our second-generation product is called VB56G4A,” explains Lin. “This is a 1.5Mp Mono, which is suited to DMS. There are three key points for this product—first, is high sensitivity. When you have high sensitivity in the NIR domain, it means your NIR LED doesn’t need to be very powerful. The second is footprint. The sensor is very small, so you can build smaller camera modules that will be easier for mechanical design. Third is built-in processing: for example, we have smarter auto exposure algorithms in the sensor that can help the processor to manage exposure, so it doesn’t need to be managed by the host.”

ST’s global-shutter sensor VB56G4A leverages the company’s in-house investment in manufacturing advanced 3D-stacked backside illuminated (BSI-3D) image sensors. These are more sensitive, smaller, and more reliable than conventional front-side illuminated (FSI) sensors typically used in first-generation DMS. The sensor achieves high Quantum Efficiency (QE), reaching 24% at 940nm near-infrared wavelength, with linear dynamic range up to 60dB. This enables a simple low-power, non-visible LED emitter to provide adequate illumination for the sensor. Operating outside the visible spectrum also ensures consistent response in day or night driving and in bright or overcast conditions.

The sensor’s high QE, combined with a pixel size of just 2.6µm, helps optimize total power consumption and camera size. In addition, integrated automatic exposure control eases use and simplifies the application-software design by minimizing system interaction with the sensor.

What’s next

For its succeeding generations global shutters, Lin notes that ST will continue to enhance pixel performance and focus on better power management to ensure ultra-low power consumption.

“We will also increase the number of embedded systems or processors in the sensor,” he says. “We also aim to reduce pixel size to enable smaller camera dimensions with higher resolution. Those points are our focus for our next generation of products.”

While most in-cabin monitoring systems just have basic functions, future designs are seen to gear towards a smart cockpit concept. “Your cameras, DMS or OMS, will link to the safety features on the car. Take for example, the airbag. Normally, you have a lot of air bags in a car nowadays—maybe 10 or even more,” explains Lin. “If each time you have an accident, you will trigger all the airbags, the cost will be very high and, sometimes, it is not safe. When you have an OMS camera, you will know where the passenger is and therefore which airbag to trigger. This is kind of a new application.”

Another future application is gesture recognition, which can help control the entertainment system or the air-conditioning system. “We also see a new application very recently: payment or face ID payment,” says Lin. “In the future, some people may want to buy something or pay money in the entertainment system because it may have an application store. You may pay using your face ID through the DMS camera.”

 

Stephen Las Marias is the editor of EETimes Asia. He may be reached at stephen.lasmarias@aspencore.com.

 

Subscribe to Newsletter

Leave a comment