The miniature H.264 core combines the Xilinx Zynq SoC in a small, fast video streaming system.
The ASSP architecture is not flexible, and the FPGA-based and microprocessor-based systems are large but flexible, and designers have been creating IP-based streaming video systems with small PCB footprints, in addition to repeating between the two. There is no other choice in weighing. Integrating a soft-core microprocessor into an FPGA eliminates the need for a separate processor and DRAM, but ultimately the performance of the system may not be built with external ARM® processors and may also include USB, Ethernet, and other useful peripherals. The performance provided by the solution is comparable. With the advent of the Xilinx Zynq®-7000 All Programmable SoC and the small H.264 core, it is now possible to build an ARM dual core with multiple high-speed AXI4 buses on a very small PCB board using only one DRAM. High-performance systems implemented with high-speed peripherals (see Figure 1).
Although the H.264 core for FPGAs has been around for a long time, there is still no H.264 core that is fast enough to achieve enough conversion to 1080p30 frames, and still suitable for small, low-cost devices. . Using A2e Technologies' latest micro H.264 core in conjunction with the Zynq SoC, a low latency system can be built that can encode multiple video streams between 720p and 4K at different frame rates of 15-60 fps. /decoding. Integrating the A2e Technologies H.264 core into Zynq SoC devices dramatically reduces board space and significantly reduces component count, while integrating ARM dual cores in the Zynq SoC avoids the use of separate microprocessors and the banks they must be connected to.
This creates a complete streaming video system with a Linux driver and Real Time Streaming Protocol (RTSP) server that compresses 1080p30 frames from two cameras with an end-to-end delay of approximately 10 millisecond. The A2e Technologies H.264 core is available in both purely encoded and encoded/decoded versions. In addition, there is a low latency version that reduces the encoding delay to less than 5 milliseconds. The 1080p30 pure encoding version requires 10,000 lookup tables (LUTs), or about 25% of the resources of the Zynq Z7020 FPGA architecture, while the encoded/decoded version requires 11,000 lookup tables.
SOC-FPGA based system
Zynq SoC-based products are more flexible than products built with Application Specific Standard Products (ASSP). For example, by building four H.264 cores in the FPGA fabric and connecting each camera input to the FPGA, it is easy to build a system that supports 1080p30 input. Many ASSP products have only two inputs, which makes designers have to figure out how to multiplex several video streams to one input. Each A2e Technologies H.264 core can handle six VGA resolution cameras or one 1080p30 resolution camera. Therefore, it is possible to build a dual-core system to compress video from 12 VGA camera inputs.
ASSP products typically only provide on-screen visual control (OSD) functionality for decoded video, forcing designers to send OSD information as metadata or use the ARM core to determine video frame time and programmatically write OSD data to the video buffer. In an FPGA, adding an OSD before video compression is as simple as accessing an IP module. It is also relatively easy to add other processing modules such as fisheye lens correction before the compression engine. In addition, the FPGA also supports on-site and future upgrades of features such as the addition of H.265 compression. Figure 2 is a block diagram of an H.264 compression engine with two 1080p30 camera inputs, where the OSD is for uncompressed images.
How to deal with delays
Some applications, such as remote-controlled aircraft (RPV) control, are based on streaming image feedback sent back by the remote control. To control the remote control, the delay between sending the video from the sensor to the compression engine to the decoded image display (referred to as "glass to glass") is typically less than 100 milliseconds. Some designers set the delay to 50 milliseconds. The total delay is the sum of the following:
• Video processing time (ISP, fisheye lens correction, etc.)
• Fill the frame buffer delay
• Compression time
• Software delay caused by sending packets
• Network delay
• Software delay caused by receiving packets
• Video decoding time
Many systems use hardware to encode video, but end up with a standard video player.
Decode, such as VLC running on a PC. Even though the media player's buffering delay can be significantly reduced from the usual 500-1000 milliseconds, the latency is still well over 50 milliseconds. To truly control latency and stay at an absolute minimum, hardware decoding of the compressed video stream is required and buffering is minimal.
The H.264 encoder delay is usually expressed in frames, such as the time it takes to buffer a full frame before compression begins. Assuming that the encoder speed can be increased, the encoding delay can be reduced only by doubling the frame rate, that is, the frame delay is 33 milliseconds at a frame rate of 30 fps, and the frame delay is 16.5 at a frame rate of 60 fps. millisecond.
A typical streaming video system uses an RTSP server to create a streaming video connection between the camera and the client (decoding/recording) device. The RTSP server delivers the compressed video to the client for display or storage.
Many times, this solution cannot be implemented due to the performance limitations of cameras and encoders. Therefore, the solution is specifically designed for low-latency encoders. The latest A2e Technologies low-latency encoders only have 16 video lines that need to be buffered before compression begins. For 1080p30 video streams, the delay is less than 500 microseconds (μs). For 480p30 video streams, the latency is less than 1 millisecond. Designers using this low-latency encoder can build systems with predictable latency and low latency.
In order to minimize the total delay, the delay caused by the buffering, network protocol stack, RTSP server/client, etc. on the encoding side and the decoding side must be minimized at the same time, because the software path will generate a long delay, and In this case, it makes no sense to use a low-latency encoder. RTSP servers are typically used to create streaming video connections between a server (camera) and a client (decoding/recording) device. Once the connection is established, the RTSP server will deliver the compressed video to the client for display or storage.
Delay minimum minimum delay
Often, the software components of the server and client only require bandwidth matching to facilitate the delivery of compressed video, rather than to minimize latency. For non-real-time operating systems such as Linux, it is difficult to guarantee latency. A typical solution is to create a low latency custom protocol for the server and client. But the downside of this approach is that it does not meet industry standards. Another approach is to use an RTSP-like standard that minimizes latency by modifying the lower layers of the software while ensuring compliance with standards.
However, measures can also be taken to minimize copy operations between the kernel and user space, thereby reducing the associated latency. As far as the entire software path is concerned, to reduce the delay, the RTSP server and the compressed information forwarding task need to be separated, so that the Linux driver is used instead of the RTSP server to perform the sending task.
To reduce latency, we made two changes to the A2e Technologies low latency RTSP server. First, remove the RTSP server on the forwarding path. The RTSP server still maintains statistics using the Real-Time Control Protocol (RTCP) and periodically (or asynchronously) updates the kernel drivers as the network destination address (ie, IP or MAC destination address) changes. Second, the kernel driver attaches the necessary headers (based on the information provided by the RTSP server) and immediately forwards the packet by directly entering a network driver (such as udp_send), eliminating the need for memory copying between the kernel and user space.
Figure 3 shows a complete encoding/decoding system based on H.264 IP with a total delay of less than 50 ms. The system was built on the Zynq SoC, A2e Technologies low latency H.264 encoder/decoder and A2e Technologies low latency RTSP server/client. It should be noted that from a hardware perspective, the only real difference between the encoding and decoding systems is that the encoding side must be connected to the camera/sensor, while the decoding side must be able to provide the drive for the flat panel display. You can easily design a board with all the necessary hardware features you need for encoding and decoding.
To minimize the latency of video compression/decompression in real-time control applications, designers need special encoders and optimized software. Utilizing Xilinx's Zynq SoC and A2e Technologies' H.264 low-latency encoders and optimized RTSP server/client, you can create a very low latency, highly configurable system on a small PCB. .
ZGAR AZ CC Disposable
ZGAR electronic cigarette uses high-tech R&D, food grade disposable pod device and high-quality raw material. All package designs are Original IP. Our designer team is from Hong Kong. We have very high requirements for product quality, flavors taste and packaging design. The E-liquid is imported, materials are food grade, and assembly plant is medical-grade dust-free workshops.
Our products include disposable e-cigarettes, rechargeable e-cigarettes, rechargreable disposable vape pen, and various of flavors of cigarette cartridges. From 600puffs to 5000puffs, ZGAR bar Disposable offer high-tech R&D, E-cigarette improves battery capacity, We offer various of flavors and support customization. And printing designs can be customized. We have our own professional team and competitive quotations for any OEM or ODM works.
We supply OEM rechargeable disposable vape pen,OEM disposable electronic cigarette,ODM disposable vape pen,ODM disposable electronic cigarette,OEM/ODM vape pen e-cigarette,OEM/ODM atomizer device.
ZGAR AZ CC Vape,ZGAR AZ CC Vape disposable electronic cigarette,ZGAR AZ CC Vape vape pen atomizer , AZ CC E-cig,AZ CC disposable electronic cigarette
ZGAR INTERNATIONAL(HK)CO., LIMITED , https://www.sze-cigarette.com