Vendors agree that "convergence" is the key to ensuring the safety of ADAS and autonomous vehicles, but they also face challenges when merging different types of sensor data. . .
As more and more automakers begin to integrate different sensors into advanced (ADAS)/automated vehicles, they often agree that “sensor fusion†is the key to highly automated driving safety.
But what is not known is that the data used by these vendors - raw data or processed data? -- Details, and the challenges they face when merging different types of sensor data; as market research firm Strategy AnalyTIcs Automotive Market Analyst Ian Riches puts it: "The current sensor fusion is not done on raw sensor data, usually per sensor Will do their own local processing."
EDA supplier Mentor Graphics will showcase how it will come from different sensing devices – radar, optical, visual or ultrasonic – at the annual SAE World Congress in Detroit, USA. . Etc. -- Raw data is fused in real time to achieve significant improvements in sensing accuracy and overall system efficiency.
Mentor Graphics is launching an autopilot platform called DRS360, which claims to transmit unfiltered information from all system sensors directly to a central processing unit, and then fuse those raw sensor data in real time at all levels; In an interview with EE TImes, Glenn Perry, general manager of the cum department, compared the difference between "sensor fusion" and "raw data fusion" and pointed out that there are "subtle but very important differences" between the two.
In general, the modular design is provided to the car manufacturer to pre-process the data; Riches of Strategy AnalyTIcs explains: "For example, the data transmitted from the camera to the fusion system is not the actual image data, but the image. Description of the important areas - like where there is a white line, where there is a car, where there is a traffic sign... and all the fusions are carried out at this time with the higher level data.
Mentor believes that the pre-processing of the sensor modules used in each terminal node and the selection of raw data enable ADAS/autopilot vehicle design engineers to significantly improve real-time performance and reduce overall system cost and complexity; The sensor data obtained establishes the highest resolution model for the environment in which the vehicle is located and the driving road conditions.
DRS360 is a platform for developing systems that comply with the ISO 26262 ASIL D standard; Mentor's Perry points out that the platform handles raw data fusion with Xilinx's Zynq series, with SoCs responsible for ADAS or autopilot functions (can Is an ARM or x86 architecture), and an MCU (such as -Infineon's secure microcontroller).
Phil Magney, founder of Vision Systems Intellige, an industry consultancy, said: "Sensor fusion is a complex task, and it can be harder to use raw data;" he explained that raw data fusion can make computing resources more efficient in centralized systems. Integration, but the integration is facing more difficult challenges. "
Why is raw data fusion difficult?
Where is the fusion of raw data? Mentor's Perry said that the first is that each sensor will stream data, and it is based on different screen update rates, different rates and intervals: "This is an asynchronous system;" Second, different types of data: "Needs Coordinated representation (cohesive repreaTIon)."
For example, the original image of the camera (2D), the point cloud of the light (3D), and the ADC output of the radar, all of which must be merged into a time/space-compliant 3D environment map; and third, as Perry said. Must be integrated: "It's really a huge data stream." For unspecified 3D datasets, Mentor has also developed a patented algorithm that enables spatial position data to be transmitted, and vehicle speed from radar. Data, and the fusion of object detection data from visual devices.
Perry said that in the past distributed sensor data modules, each sensor usually discarded a large amount of data; but the centralized solution uses unfiltered sensor data to ensure the accuracy and reliability of the enhancement; he stressed Mentor It took two years to develop the platform: "We are surprised by the improvement in system benefits -- including the reduction in latency and the increase in processing performance, all implemented by algorithms that fuse raw data."
Industry experts believe that the original data fusion is expected to create a more "open" smart vehicle platform in the future, replacing the current "child" approach. . .
Phil Magney, founder and chief consultant of Vision Systems Intelligence, an industry consultancy, called the fusion of raw sensor data "the direction of the industry." He believes: "Other technological advances can also be combined with raw data fusion; for example, neural networks will Can improve performance due to raw data from different sources."
Analysts also believe that the relatively large "openness" of Mentor Graphics' DRS360 platform will be one of its advantages; such as Strategy Analytics analyst Ian Riches: "The platform can provide first-line automotive component suppliers or The carmakers have the greatest flexibility to design their own solutions and bring them value."
Riches further pointed out that now the system like Mobileye is usually a "black box". First-line auto parts suppliers have to use their solutions to build their own products, they have to compete with cost, because they can not adjust the core algorithm of the system. Mobileye treats the algorithm as an IP.
However, DRS360 is flexible regardless of the chip used or the algorithm being executed. Magney said: "So far, most ADAS applications are done through object data, that is, the sensor module will do all Things, such as detecting and classifying related objects in the scene."
Maney agrees with Riches that Mobileye's solution is like a "black box"; he further points out that the ADAS application can use "edge processing", but for the vehicle, the processing is integrated. It's more practical together.
The scalability of the DRS360 is also very important. Instead of developing platforms for different ADAS or autonomous vehicles, manufacturers can expand and develop their own solutions with the DRS360 platform; many car manufacturers believe that this is the profit.
But one platform has FPGAs and SoCs and MCUs. Is it not too costly for manufacturers? Riches doesn't think: "A car will have a lot of standard ADAS functions and sensors. The centralized architecture like DRS360 can save money in theory;" he explained: "Yes, you need to add a device. However, it is not necessary to equip each sensor with multiple smaller processing units that match the vehicle."
Who can benefit from a centralized platform?
According to Strategy Analytics' Riches, the DRS360 platform offers significant advantages for vendors who want their final product to include their own unique IP, but this is a disadvantage for those who want a turnkey solution because it means To get the product to market, you have to do more.
Glenn Perry, vice president and general manager of Mentor Graphics, said: "Every vendor -- whether they are experts or not -- has specific needs and needs some customization elements." In this regard, he believes that DRS360 can satisfy those Demand, if the car factory really does not have experts, do not know how to find the desired function, the first-line auto parts suppliers can use the platform to make their own technical adjustments, and then provide them to the depot.
Luca De Ambroggi, market chief analyst at IHS Markit, another market research organization, believes that DRS360 is good for many vendors in the supply chain, from system vendors looking for development platforms to chip vendors, and those who don’t want or can’t People who develop software, system security, or AI.
Grab a business with a chip supplier?
But the biggest problem is that Mentor's launch of the DRS360 platform is ready to grab business with chipmakers such as Mobileye/Intel and Nvidia, which are actively rushing into the automotive market.
In this regard, Riches pointed out that the sensor module may be the most affected manufacturer, because many of these operators add value to the product with the processing performance and algorithm of the sensor, but the centralized architecture of the DRS360 platform makes the sensor manufacturer "downgraded". Only need to provide raw data.
"Mentor is not a chip supplier, I guess they will keep the platform's non-off-chip, but maybe Mobileye/Intel and Nvidia will have different ideas;" IHS Markit's De Ambroggi said that Mentor's view is from architecture and design support. The departure of the software platform is not to promote a specific chip.
Mentor's Perry admits that central processing of raw data fusion, chip vendors may have their own methods, such as the use of TFLOP level processing performance: "usually people with hardware and software, there will be incentives to do this business;" Said that Mentor believes that DRS360 differs from other solutions in that it can achieve raw data fusion with minimal processing performance.
While there are concerns about revealing the processing performance details of the DRS360 platform with FPGAs, SoCs and MCUs, Perry claims that the platform can be built as a fully automated vehicle with only 100 watts of power; specifically, he points out that the FPGA is about 24 watts. The two SoCs each have 15 watts and the safety microcontroller consumes 5 watts.
Perry also pointed out that the implementation of the original data fusion algorithm using FPGA is because of its high performance and high capacity, it can directly process the original data, the data packet does not need to go in and out of the memory; in addition: "You can directly connect the FPGA with different types. Sensor data stream links, which are flexible and adapt to the different types of sensor entity interfaces specified by the vendor.
According to Strategy Analytics' Riches, "This flexibility can be practical for ADAS/automated vehicles, and because there are no standard combinations of sensors, the standard way to move those sensor data has not yet appeared."
Dongguan Deli Plastic Co.,Ltd is a manufacturer specialized in the research, development ,plastic injection mould and making mass production with well-equipped facilities and strong technical force.
Our products are extensively used in household industry/electronic industry/automobile industry/building industry and other industries.
We have rich experience on one-stop solution, provide various services from new product design,prototype,mold making,mass production,assembly and logistics. The most important advantage is we have our own R&D team to help clients to turn ideas into actual parts. All of these engineers and designers have over 15 years experience in these plastic products fields.
We have a strict quality control system, an excellent management team and also a dedicated sales force, enable us to fulfill our commitment in high quality products and outstanding services.
If you are looking for a trustworthy supplier of customized items, please do not hesitate to contact us. We are always striving to establish a win-win partnership with customers from all over the world and help our partners to stay one step in front of your competitors.
usb Tower Fan,small Tower Fan,double tower fan
Dongguan Yuhua Electronic Plastic Technology Co.,Ltd , https://www.yuhuaportablefan.com