Vehicle mounted image sensor chip: technology evolution and application panorama
Overview
With the rapid development of intelligent driving and connected car technologies, the automotive CMOS Image Sensor (CIS) is emerging as an increasingly critical component of the vehicle's "visual system". It not only undertakes basic functions such as environmental perception and safety monitoring, but also serves as a vital hardware foundation for realizing Advanced Driver Assistance Systems (ADAS), autonomous driving, and intelligent cockpits. This paper systematically sorts out the current development status and future trends of automotive CIS chips from the aspects of application scenarios, technical requirements, and localization progress, and further delves into their technical details, market dynamics, and industrial ecology, presenting a more comprehensive industry landscape for readers.
Based on differences in installation locations and functions, automotive CMOS image sensors can be divided into two major application areas: exterior-cabin and interior-cabin, covering comprehensive needs from active safety to occupant management. As the level of intelligence increases, the number of sensors and the complexity of their functions have significantly grown, forming a three-dimensional perception network.
I. Exterior-Cabin Applications: Building the Vehicle’s "External Visual Network"
1. Front-view Perception CIS
Installation Location: Behind the front windshield or on both sides of the roof; some models adopt a split binocular layout.
Functions: Implements core ADAS functions including Forward Collision Warning (FCW), Lane Departure Warning (LDW), Traffic Sign Recognition (TSR), and Pedestrian Detection (PD), providing basic data for L2+ autonomous driving.
Configuration Schemes: Evolving from monocular to binocular and trinocular systems, achieving both short-range and long-range coverage through the combination of different Fields of View (FOV). For example, the main-view camera (FOV ≈ 50°) covers a distance of 100 meters, while the narrow-view camera (FOV ≈ 25°) extends the range to 200 meters, improving reaction time in high-speed scenarios.
Key Technical Indicators:
Resolution: Gradually upgraded from 2MP to 8MP; some high-end models adopt 12MP solutions to support more precise target recognition.
High Dynamic Range (HDR): HDR ≥ 120dB has become mainstream; some high-end products achieve 140dB through multi-frame synthesis technology to cope with extreme scenarios such as tunnel entry/exit and backlighting.
LED Flicker Mitigation (LFM) Support: Eliminates image artifacts caused by the stroboscopic effect of traffic lights through the collaboration of hardware circuits and algorithms.
Functional Safety Compliance: Meets the ISO 26262 functional safety standard, with ASIL-B to ASIL-D ratings, ensuring safety redundancy in the event of system failure.
2. Surround-view Imaging CIS
Installation Location: Near the front and rear logos, below the left and right side-view mirrors; some models add fender cameras.
Quantity: Usually 4 cameras, forming a 360° Around View Monitor (AVM) system; some high-end models expand to 6 cameras to support the "transparent chassis" function.
Functions: Supports Automatic Parking Assist (APA), Blind Spot Detection (BSD), and narrow-passage driving assistance, generating a bird’s-eye view through image stitching to eliminate visual blind spots.
Technical Features:
Resolution: Upgraded from 1MP to 3MP, improving image clarity and enabling centimeter-level obstacle recognition.
HDR ≥ 120dB: Combined with ISP (Image Signal Processor) optimization, ensuring balanced details in both shadow and highlight areas.
Built-in Fisheye Correction Algorithm: Performs real-time distortion correction and outputs stitched images consistent with human visual perception.
Excellent Night Vision Capability: Achieves imaging in environments with illumination below 5 lux through near-infrared supplementary lighting or high-sensitivity technology.
Application Trend: With the increasing demand for automated parking, surround-view systems are gradually upgrading to 4D surround view, integrating millimeter-wave radar data to display the movement trajectories of obstacles in real time.
3. Rear-view Imaging CIS
Installation Location: Above the rear license plate frame or integrated into the rear spoiler.
Functions: Provides Rear View Camera (RVC) display for parking assistance, supports dynamic guide lines, marks obstacle distances and trajectories, and reduces parking risks.
Parameter Levels:
Resolution: Upgraded from VGA (640×480) to 2MP (1920×1080); some models adopt Wide Dynamic Range (WDR) technology.
HDR ≥ 120dB: Ensures clear visibility at night or in severe weather conditions.
ISP Real-time Noise Reduction and Contrast Enhancement Support: Reduces interference from rain and fog.
Innovative Application: Some automakers have launched the "rear-view streaming media" function, which replaces traditional rear-view mirrors with low-latency transmission (≤ 30ms) and wide-angle lenses, expanding the field of view and reducing wind resistance.
4. Side-view Perception CIS
Installation Location: Below the side-view mirrors, on door handles, or integrated into the B-pillars in some models.
Functions: Blind Spot Monitoring (BSM), Lane Change Assist (LCA), and Door Opening Warning (DOW), monitoring vehicles and pedestrians in the rear side areas in real time.
Performance Requirements:
Resolution: Up to 3MP, supporting a wide field of view (FOV ≥ 120°).
HDR ≥ 120dB: Adapts to transitions between direct sunlight and tunnel shadows.
LFM and Functional Safety Certification Support: Ensuring the reliability of traffic light recognition.
Data Fusion in Some Models: Integrates ultrasonic sensor data to improve short-range detection accuracy.
Technical Challenge: Side-view cameras need to balance wide-angle and low distortion, often adopting dual-lens stitching or free-form surface optical design.
5. Streaming Media/Electronic Mirror CIS
Installation Method: Cameras at the rear and sides of the vehicle capture images, which are compressed and transmitted to high-definition displays inside the car.
Advantages: Resists strong light, compensates for low light, penetrates rain and fog, and has no visual blind spots, significantly improving safety especially on rainy days or at night.
Technical Parameters:
Resolution: Up to 3MP, supporting a high frame rate of 60fps.
HDR ≥ 120dB: Combined with dynamic exposure adjustment, suppressing glare from high beams of rear vehicles.
Low-latency Transmission (≤ 50ms) and High-stability Display Support: Avoiding image freezes that may affect driving decisions.
AI Algorithm Integration in Some Products: Marks dangerous areas and issues early warnings.
Regulatory Dynamics: The European Union has approved the replacement of traditional optical rear-view mirrors with Camera Monitor Systems (CMS). Relevant standards in China are also being gradually improved, promoting the large-scale application of the technology.
6. Dash Cam CIS
Installation Location: Inside the front windshield, often sharing modules with front-view ADAS or streaming media rear-view mirrors.
Functions: Records driving processes for accident traceability, driving behavior analysis (e.g., statistics on sudden acceleration and braking), and insurance liability determination.
Common Integration Schemes:
Resolution: Mainly 1MP, gradually transitioning to 2MP.
ISP and YUV Format Output Support: Compatible with cloud storage.
Excellent Night Vision Performance: Achieves recording in lightless environments through starlight-level sensors and infrared supplementary lighting.
Expanded Application: Some high-end models add in-car recorders to monitor driver status and occupant behavior, providing safety monitoring for shared mobility services.
II. Interior-Cabin Applications: Building the "Intelligent Cockpit Perception Hub"
1. Driver Monitoring System (DMS)
2. Occupant Monitoring System (OMS)
Function Expansion: Rear passenger detection (child left-behind reminder), seat belt status recognition, occupant posture analysis (e.g., seat belt wearing detection, abnormal behavior warning), and emotional interaction (adjusting in-car atmosphere based on facial expression recognition).
Technical Features:
Resolution: Up to 5MP, supporting wide-angle coverage of the entire cabin.
RGB-IR Fusion Lens: Balances visible light and infrared imaging, adapting to night or window-tinted scenarios.
AI Algorithm Input Support: Recognizes occupant behaviors and intentions through deep learning.
Vital Signs Monitoring (VSM) Integration in Some Models: Detects vital data such as heartbeat and respiration through millimeter-wave radar to improve safety levels.
Ethical Considerations: Data privacy protection must comply with regulations such as GDPR. Users are required to authorize data usage, and the system must be equipped with local encryption and permission management capabilities.
III. Core Technical Trends of Automotive CIS Chips
1. High Dynamic Range (HDR) Becomes Standard Configuration
Vehicles operate in environments with drastic light changes (e.g., tunnel entry/exit, high beams at night). HDR ≥ 120dB has become a mainstream requirement, and some high-end products have reached 140dB. Technologies such as multi-frame synthesis and dual exposure further expand the dynamic range, eliminating overexposure or underexposure (dead black) phenomena.
Technology Evolution: Developing from traditional 2-frame HDR to 3-frame and multi-frame HDR, combined with AI noise reduction algorithms to achieve more natural transition effects.
2. LED Flicker Mitigation (LFM) Capability is Critical
It addresses high-frequency flickering light sources (50Hz~120Hz) such as traffic lights and electronic displays, avoiding image distortion and recognition errors. Special pixel designs are adopted in hardware, and precise suppression is achieved in software through frequency synchronization and phase compensation algorithms.
Testing Standard: Must pass the ISO 6722 road vehicle signal light recognition test to ensure stability in different stroboscopic scenarios.
3. Functional Safety and Reliability
All key applications must meet the ISO 26262 functional safety standard. Especially in ADAS-related scenarios, chips need to have fault detection and redundancy mechanisms. For example, dual-core architecture, built-in ECC verification, and temperature monitoring are adopted to ensure that single-point failures do not affect system operation.
Reliability Verification: Must pass the AEC-Q100 automotive-grade certification, including tests for high-temperature operation (85℃~125℃), vibration and shock (-40℃~85℃), and ESD electrostatic protection.
4. Continuous Optimization of Low-light and Night Vision Performance
Back-illuminated (BSI) and stacked (Stacked) processes are used to improve light-sensing efficiency. Combined with on-chip noise reduction algorithms (e.g., 3DNR, TDNR), starlight-level imaging (≤ 0.1 lux) is achieved. Some high-end chips integrate near-infrared enhancement functions, realizing lightless environment perception through 940nm infrared supplementary lighting.
Technology Comparison: The BSI process improves light sensitivity by 30%~50% compared with the traditional Front-illuminated (FSI) process, while the Stacked structure further separates the pixel and circuit layers to reduce noise interference.
5. Integration and Intelligence Trends
Chips are gradually integrating ISP and AI preprocessing units (e.g., convolution accelerators), reducing reliance on external computing power and lowering system latency and costs. For example, built-in target detection modules can output ROI (Region of Interest) in real time, improving the efficiency of downstream algorithms.
SoC Development Trend: Some manufacturers have launched CIS+ISP+AI SoC chips, which realize the full process from image acquisition to preprocessing with a single chip, promoting the improvement of system integration and energy efficiency ratio.
6. Anti-interference and Stability
The automotive environment has strong electromagnetic interference (EMI). Chips must pass strict EMC tests and adopt metal-shielded packaging and anti-noise circuit design. Meanwhile, they need to have high vibration resistance (e.g., complying with the ISO 16750 vibration standard) to ensure stability under extreme road conditions.
Thermal Management Design: Heat dissipation pads and copper sheet heat conduction structures are used to control the chip junction temperature ≤ 105℃, avoiding performance degradation caused by high temperatures.
IV. Localization Progress of Automotive CIS Chips
Application Scenarios: Widely used in automotive video, security monitoring, commercial vehicle DVR and other fields, with high cost performance and strong compatibility, promoting the process of localization replacement.
Ecological Advantages: Deeply integrated with BYD models, and gradually opening up cooperation with other automakers, forming a closed-loop ecosystem of "hardware + algorithms + data".
V. Future Development Trends of the Industry
1. Continuous Resolution Improvement and Expansion of Perception Capability Boundaries
2. Multi-modal Fusion Perception to Build a Redundant Safety Network
CIS will be deeply integrated with millimeter-wave radar, LiDAR, and ultrasonic sensors. Through spatio-temporal synchronization and data fusion algorithms, it makes up for the defects of single sensors (e.g., CIS is affected by weather, while LiDAR has high costs). For example, in rainy and snowy weather, radar data is used to supplement the blind spots of visual perception.
3. Deepening of AI on Sensor Trend and Rise of Edge Intelligence
Chips embed lightweight AI engines (e.g., convolutional neural network accelerators) to realize edge intelligence. For example, target detection, classification and tracking are completed on the chip side, and only key information is transmitted to the central computing platform, reducing communication latency and bandwidth requirements. Meanwhile, private data can be processed locally, complying with regulatory requirements.
4. Accelerated Localization Replacement and Industrial Chain Collaborative Upgrading
Driven by policies (e.g., *Intelligent Vehicle Innovation and Development Strategy*) and industrial chain collaboration, domestic CIS will continue to catch up in terms of performance, reliability and ecological adaptation. Upstream equipment and material enterprises (e.g., lithography machines, target materials) are accelerating technological breakthroughs; midstream manufacturing links are advancing the construction of automotive-grade production lines; downstream automakers are opening up cooperation, forming a sound ecosystem.
Challenges and Opportunities: It is necessary to conquer core technologies such as high dynamic range and functional safety design, establish a complete test and verification system and automotive-grade certification capabilities, and break through the patent barriers of international giants.
5. Integration of Intelligent Cockpits and Connected Cars to Expand Application Scenarios
Interior-cabin CIS will not only be limited to safety monitoring, but also become an intelligent interaction portal. For example, dynamically adjust air conditioning temperature, music style and seat posture through passenger expression and behavior analysis; realize remote identity authentication and personalized service push combined with connected car technologies.
6. Cost and Power Consumption Optimization to Promote Large-scale Application
Advanced manufacturing processes (e.g., 22nm/14nm) and SoC integration are used to reduce chip costs and power consumption, enabling high-end functions to gradually extend to mid-to-low-end models. For example, integrating ISP, AI units and CIS into a single chip reduces the demand for peripheral devices.
Automotive image sensor chips are not only the "eyes" of vehicle intelligence, but also a key bridge connecting the physical world and digital decision-making. From exterior-cabin environmental perception to interior-cabin intelligent interaction, from single-point applications to multi-modal fusion, CIS technology is undergoing unprecedented changes. With the advancement of technological iteration and localization process, we will see more high-performance, high-reliability and cost-effective domestic CIS chips stepping onto the global stage, providing a solid foundation for intelligent mobility. In this wave of transformation, innovation and collaboration will become the core driving forces for industrial development—technological innovation breaks through performance limits, industrial chain collaboration builds a safe ecosystem, and cross-border integration expands application boundaries. It is expected that China's automotive CIS industry will stand out in global competition and write a new chapter in the era of intelligent vehicles.