Hikvision’s thermal camera, developed in 2016, caught the eye of analysts at System Plus Consulting, for one key reason: AI...
China’s Hikvision is one of the top suppliers of surveillance cameras in a global market that has grown to $20 billion last year.
The demand for surveillance systems has kicked into high gear in the last several years, both in China and worldwide. Many municipalities and corporations find cameras indispensable in building so-called “smart cities” with surveillance everywhere.
Hikvision’s thermal camera, developed in 2016, caught the eye of analysts at System Plus Consulting, for one key reason: AI.
Hikvision created the first camera embedded with AI features — by combining both AI hardware and software. It prompted System Plus Consulting, a Yole Développement company, to look under the hood to “understand the technology choices behind it.”
Most notable about the Hikvision camera, for System Plus, is that it embodied the best of the East and the West — “a made-in-China microbolometer and camera processor” combined with “non-Chinese AI / analog / other processing components.”
Hikvision’s competitors in this specialty include: Dahua and Uniview, both based on China, Bosch (Germany) and Axis (Sweden).
Setting Hikvision apart, however, is that “the company is able to design and manufacture its own products,” according to System Plus. The Chinese company has its own MEMS production line, MEMS packaging/tests, numerous surface-mount technology (SMT) lines and final assembly capabilities.
Intel, Hikvision and Movidius (now an Intel company) supply three key components unique to Hikvision’s camera:
In the following pages, we got System Consulting’s help to look inside Hikvision’s thermal camera.
A thermal camera is able to detect the heat produced by the human body and translate it into an image through a sophisticated signal analysis process. The images are reproduced through the detection and analysis of the temperatures. In the last few years, thermal imaging has found its way into low-cost applications by relying on microbolometers.
Microbolometers are sensors that detect infrared. They consist of a grid of sensitive points (called “pixels”) composed of different layers and different absorbent materials such as vanadium oxide or amorphous silicon (α-Si).
In an interview with System Plus Consulting, we examined the technical and structural aspects of the Hikvision DS-2TD2166-15/V1 Thermal Network Camera. The technicians of System Plus Consulting described the electronic and physical hardware structure of the system, highlighting the different elements that make it up.
Hikvision DS-2TD2166-15/V1 Thermal Network Camera is equipped with an image sensor based on Vanadium Oxide Uncooled Focal Plane Arrays (Figure 1). It is featured with the support of intelligent analysis algorithms for several critical infrastructures such as airports, railways and so on. This thermal camera is based on several chipsets such as RTD6171MR Microbolometer 640×512 pixel (17µm Pitch); FPGA Cyclone V 550MHz 224I/O (FBGA484); SDRAM 2Gb (128Mx16) 800MHz 13.75ns ( TFBGA96); Temperature Controller for Peltier Module (TQFN48); SoC for Professional HD IP Camera; Vision Processor Unit 2x32Bit RISC Proc. 4Gb LPDDR3; and DDR4 DRAM 8Gb (512Mx16) 2400Mbps.
Its technical and structural characteristics make it ideal for preventing fires and promptly detecting overheating and temperature changes within company and industrial processes.
Its VCA (Video Content Analysis) function supports two entirely different processes: detecting events in time and space and analyzing video. It has 4 VCA rule types (Line Crossing, Intrusion, Region Entrance, and Region Exiting), and supports up to 8 VCA rules.
The camera acquires thermal images, which allow users to detect people, objects, and accidents in complete darkness and under difficult conditions. Because it is sensitive only to infrared radiation emitted by bodies, its ability to view and record images is not affected by the light in the scene being recorded.
The temperature measurement function enables to measure the actual temperature of the spot being monitored. The device sets off an alarm when the temperature exceeds the threshold value. Let’s see the structure of the hardware part of this room
The Thermal Camera consists of 6 boards inside, each one intended for a specific purpose. Let’s analyze some parts (Figures 2 and 3). The Cyclone V SoC FPGA is built on TSMC 28-nm low power (28LP) process; It consists of a dual-core ARM Cortex-A9 MPCore processor, a rich set of peripherals, and a shared multiport SDRAM controller. The use of this FPGA offers a reduction in power consumption and supports over 100 Gbps peak bandwidth with integrated data coherency between the processor and the FPGA.
The Signal Conditioning/Amplifying part consists of various ICs, in particular the AD8605ARTZ-REEL General Purpose Amplifier, LT6203IMS8 Dual Amplifier 100MHz, and LT1994IMS8 Differential Amplifier 70MHz. AD8605ARTZ features very low offset voltage, low input voltage and current noise, and wide signal bandwidth. It uses the Analog Devices, Inc. patented DigiTrim trimming technique, which adjusts circuit performance by programming digitally weighted current sources.
LT6202 features 1.9nV/√Hz noise voltage and draws only 2.5mA of supply current per amplifier. This amplifier combines low noise and supply current with a 100MHz gain-bandwidth product, a 25V/µs slew rate, and is optimized for low supply signal conditioning systems. Harmonic distortion is less than –80dBc at 1MHz, making these amplifiers suitable in low power data acquisition systems such as this thermal camera.
The LT1994 is ideal for level shifting ground-referenced signals for driving differential input, single-supply ADCs. The output common-mode voltage of the LT1994 is independent of the input common-mode voltage and is adjustable by applying a voltage on the VOCM pin as described in its datasheet.
ADS1112IDGSR ADC 16-Bit and LT3042IDD support the FPGA for the conditioning circuit. ADS1112 is designed for applications requiring high-resolution measurement, where space and power consumption are major features to be taken into account. The LT3042IDD is instead a low dropout linear regulator featuring for powering noise-sensitive RF applications. Onboard 3 and 1 there are other integrated circuits to support the power supply of the relevant integrated subsystems, such as linear regulators and step-down converters.
The main part that determines 80% of the cost is the microbolometer (Vanadium Oxide). It is supported by Peltier’s cell with its temperature control circuit.
The main module supporting the microbolometer consists of various lenses to optimize the IR beam on the sensors. From figures 4 and 5 we can see a Germanium (Ge) lens with a diameter of 19.6 mm, and two Arsenic triselenide (As2Se3) lenses with different diameters, one of 17.6 mm and the other of 27.6 mm.
In optics, the f/number (sometimes called focal ratio or relative aperture) is a parameter of an optical system, which expresses the area of light acceptance. That is, the focal length divided by the diameter of the aperture.
A lens with a large aperture diameter allows more light, or infrared radiation, to pass through it. And therefore, a greater amount of infrared radiation will improve the measurement according to the signal-to-noise ratio. A parameter that allows identifying the measurement quality is called “NETD” or “Noise Equivalent Temperature Difference”. It is typically expressed in milli-Kelvin (mK), and measures how well a thermal image detector can distinguish small differences in thermal radiation image. Typical values for uncooled thermal imaging cameras of the micro-bolometer detector are in the order of 45 mK.
A grid of resistors pixels forms the uncooled sensors. These types of sensors are called Microbolometers. Each incident radiation on the absorber element raises its temperature above the resistor temperature; the higher the absorbed power, the higher the temperature rise. The value of the resistor changes depending on the incident radiation, particularly the infrared radiation that heats the surface. Each pixel is represented by CMOS Input Cell (Read-Out Integrated Circuit -ROIC), and is processed through the conditioning circuit in order to generate an image on our computer or monitor by means of the FPGA (Figures 6 and 7). Typically the structure of microbolometers is optimized for higher sensitivity in the 8-14µm spectral band. The sensor used in the Hikvision DS-2TD2166-15/V1 is IRAY RTD6171MR with 640×512 pixels (17µm Pitch) 60Hz (Analog Output), SMD.
System Plus has highlighted the physical characteristics of the microbolometer, summarised below:
By analyzing it internally, we can see that a reflector is located underneath the absorbent material and in contact with the substrate that redirects spurious light through to optimize the signal. The absorbent material is “suspended” from the substrate to allow thermal insulation while the pixel grid composed is vacuum-encapsulated to improve durability and reliability. Most of the microbolometers used in thermal imaging cameras use vanadium oxide as the absorbent material because of the better thermal contrast that ensures more accurate and defined images.
Vanadium oxide detectors have an impedance of around 100Kohm for a typical resistor, unlike α-Si detectors which typically have an impedance of 30Mohm. Under these conditions, the Vanadium Oxide material has a lower Johnson noise voltage and therefore the measurement will be subject to less noise. The Johnson noise voltage depends on three conditions: resistor value, circuit bandwidth and temperature.
The camera is equipped with a temperature reference element and Peltier temperature stabilizing by means of AD5645RBRUZ Quad 14-Bit DAC with On-Chip Reference and MAX1978ETM+T Temperature Controller for Peltier Module.
Peltier cells are inexpensive thermoelectric devices used as a technology for power generators, cooling, and precise temperature control, as in the case of this camera, to keep the temperature of the object constant at a predefined level. Peltier cells are based on the principle of thermoelectric phenomena. These phenomena are based on the formation of a difference in voltage levels in the PN junctions of two different metal materials.
MAX1978 has on-chip power FETs and thermal control-loop circuitry to minimize external components while maintaining high efficiency. An ultralow-drift chopper amplifier maintains ±0.001°C temperature stability. The temperature sensor is on the lens module and based on NTC/PTC thermistor. An additional digital temperature sensor, TMP75AIDRG4, monitors the system (environment) temperature directly managed by the FPGA.
Unlike other types of infrared detection equipment, vanadium oxide microbolometers do not need to be cooled. Vanadium Oxide shows a different behavior depending on temperature. Coated glass blocks infrared radiation (but not visible light) at certain specific temperatures, allowing the camera electronics to process an image from the electromagnetic spectrum and reproduce it in pseudo-colors.
The thermal camera supports RS232 transmission (via SP3232EEN-L) for industrial interfaces and Ethernet transmission with the support of RTL8201FI-VC-CG. Board 6, as shown in figure 2, includes the AC/DC power supply system with Transient Voltage Suppression Diodes (TVS) to protect electronic circuits against transients and overvoltage threats such as EFT (electrically fast transients) and ESD (electro-static discharge).
The thermal camera has the PoE (Power Over Ethernet) interface that is supported by TPS2378DDDAR PoE High-Power PD Interface and TL2845BDR-8 Current-Mode PWM Controller from Texas Instruments. The latter provides all the features that are necessary to implement off-line or dc-to-dc fixed-frequency current-mode control schemes, with a minimum number of external components.
The low 0.5-Ω internal switch resistance of TPS2378DDAR, combined with the enhanced thermal dissipation of the PowerPAD package, enables the PoE system to continuously handle up to 0.85 A. Power over Ethernet (PoE) is a technology that transmits electrical power through a twisted pair Ethernet cable: the device that provides the power is called Power Sourcing Equipment (PSE) while the powered one is called Powered Device (PD). When a PD is connected to a PSE, the PoE standard specifies the inrush current to the PD to prevent high current spikes. In addition, the PoE standard provides an analog handshake (classification) between the PSE and PD to negotiate power.
The thermal camera has video support with HI3519 V111 SoC for Professional HD IP Camera. It uses the H.265 video compression encoder as well as advanced low power technology and architecture design. Hi3519 V101 supports 90° or 270° rotation and lens distortion correction by using hardware, algorithms to design various models of IP cameras and audio CODEC. This SoC is supported by two pairs of DDR4 memory 4Gb each and GD5F2GQ4UB9IGR Flash NAND 2Mb SPI.
The Intel Movidius MA2450 VPU 2x32Bit RISC Proc. 4Gb LPDDR3 at 933 MHz is situated on board 4 (Figure 2) and allows the system to quickly recognize objects and people, analyze public demographics, inspect manufactured products, and much more. Computer vision uses deep learning to form neural networks that guide systems in their image processing and analysis.
Various thermal camera models with cooled and uncooled detectors stand out in the market. Thermal cameras with a cooled sensor are more expensive. A modern cooled thermal imaging camera has an integrated image sensor with a cryocooler.
Thanks to the microbolometer, the thermal imaging camera can offer a good degree of accuracy at a low cost. The camera takes a measurement of the surface temperature of the heat emitted by an object and projects it as an image on a thermogram screen.
— Additional reporting by Junko Yoshida