The quad sensor MIPI CSI-2 receiver capture pipeline is shown in the following figure.
The Avnet Multicamera FMC module bundles fours ON Semi image sensors (AR0231) with GMSL (Gigabit Multimedia Serial Link) serializers (MAX96705) and deserialzer (MAX9286).
The MIPI CSI-2 subsystem, see the MIPI CSI-2 Receiver Subsystem Product Guide (PG232), captures images from the deserializer in RAW12 format on four lanes and outputs AXI4- Stream video data.
The AXI subset converter converts the raw 12-bit (RAW12) AXI4-Stream input data to raw 10- bit (RAW10) AXI4-Stream output data by truncating the four least significant bits (LSB) of each data word. The AXIS switch splits the incoming data into four streams using the destination id.
The ISP IP receives the RAW AXI4-Stream input data and interpolates the missing color components for every pixel to generate a 24-bit, 8 bits per pixel (8 bpc) RGB output image transported via AXI4-Stream. For more information refer to Vitis Vision Libraries Image Sensor Processing pipeline.
The VPSS takes AXI4-Stream input data in 24-bit RGB format and converts it to a 16-bit, 8 bpc YUV 4:2:2 output format.
The video frame buffer takes YUV 4:2:2 sub-sampled AXI4-Stream input data and converts it to AXI4-MM format which is written to memory as 16-bit packed YUYV.
All of the IPs in this pipeline are configured to transport 2 ppc @ 150 MHz, enabling up to 1920x1080 resolution at 120 fps, or 30 fps per stream.
Time to transfer one frame: (1920 + 280) x (1080 + 45) / (150 MHz * 2 ppc) = 0.00825 ms
Number of frames transferred per second = 1/0.00825 = 120 frames
Note: The AR0231 sensor is limited to 1080p30 applications.