The Xilinx low-latency mode has the following limitations:
- The synchronization module between the capture source (v4l2src) and the encoder (omxh264enc/omxh265enc) can only handle synchronization for upto three frames. If the encoder lags behind the producer by more than three frames, the pipeline may crash.
- The VCU encoder and decoder use mono-threaded micro-controller based scheduler for sending commands to underlying hardware IP blocks. For the multi-stream environment it is possible that command requests for one stream can get blocked if the scheduler is busy serving commands for another stream in parallel. This may cause momentary spike in latency. Also the spike in latency can occur in an already running pipeline whenever a new pipeline is launched or closed as the command requests for running stream can get blocked while a new channel is being created for encoding a new stream, or an existing channel for an already running stream gets destroyed during closure. Latencies may go higher as the number of streams increase.