Normal Latency
This is standard latency and can be used with any of the rate control mode.
- Gstreamer Encoder Pipeline
-
gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc ! video/x-h264, alignment=au! Fakesink
- Gstreamer Decoder Pipeline
-
gst-launch-1.0 filesrc location="input-file.mp4" ! qtdemux name=demux demux.video_0 ! h264parse ! video/x-h264, alignment=au ! omxh264dec low-latency=0 ! queue max-size-bytes=0 ! fakevideosink
- Ctrl-SW Command
- The v4l2 capture control software encoder application
demonstrates AMD's Low-Latency feature using the VCU
ctrlsw
APIs. It is an enhanced version of normal VCUctrlsw
app (ctrlsw_encoder
).Refer to this Wiki page section for commands and additional details.
No Reordering (Reduced Latency)
For encoder, set rate control mode to low-latency with I only, IPPP, or low-delay-P mode. For decoder, set low-latency parameter with alignment to AU through caps for sink pad.
- Gstreamer Encoder Pipeline
-
gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc gop-mode= (basic or low-delay-p), b-frames=0 control-rate=low-latency ! fakesink
- Gstreamer Decoder Pipeline
-
gst-launch-1.0 filesrc location="input-file.mp4" ! qtdemux name=demux demux.video_0 ! h264parse ! video/x-h264, alignment=au ! omxh264dec low-latency=1 ! queue max-size-bytes=0 ! fakevideosink
- Ctrl-SW command
- At the control software level, it can be specified using command line
arguments.
ctrlsw_decoder -avc -in input-avc-file.h264 -out ouput.yuv --no-reordering
Low Latency
This mode supports sub-frame latency. For encoder, recommended to set rate control mode to low-latency rate for best performance and pass alignment = NAL through caps at encoder source pad. For decoder, set low-latency parameter with alignment set to NAL through caps for sink pad. Low latency can be enabled at the encoder and decoder sides as follows:
omxh264enc ! video/x-h264, alignment=nal ! ... ! omxh264dec low-latency=1 ! ...
- Gstreamer Encoder Pipeline
-
gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc gop-mode= (default or low-delay-p), b-frames=0 control-rate=low-latency ! video/x-h264, alignment=nal ! fakesink
- Gstreamer Decoder Pipeline
-
gst-launch-1.0 filesrc location="input-file.mp4" ! qtdemux name=demux demux.video_0 ! h264parse! video/x-h264, alignment=nal ! omxh264dec low-latency=1 ! queue max-size-bytes=0 ! fakevideosink
- Ctrl-SW command
- At the control software level, it can be specified through
command line
arguments.
ctrlsw_encoder –cfg encode_simple.cfg –-slicelat ctrlsw_decoder -avc -in input-avc-file.h264 -out ouput.yuv -slicelat
Xilinx Low Latency
Xilinx low latency can be enabled by adding an extra caps video/x-raw(memory:XLNXLL) at v4l2src and decoder side as follows
v4l2src ! video/x-raw(memory:XLNXLL), width=3840, height=2160, ... ! omxh264enc ! video/x-h264, alignment=nal ! ... ! omxh264dec low-latency=1 ! video/x-raw(memory:XLNXLL) ...
- 4x 1080p60 AVC NV12 Xilinx Low-Latency Pipeline
-
gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw/(memory:XLNXLL/), width=1920, height=1080, format=NV12, framerate=60/1 ! queue ! omxh264enc control-rate=low-latency target-bitrate=6000 prefetch-buffer=TRUE num-slices=8 periodicity-idr=240 cpb-size=500 initial-delay=250 gdr-mode=horizontal gop-mode=low-delay-p ! video/x-h264, alignment=nal ! queue max-size-buffers=0 ! omxh264dec low-latency=1 ! video/x-raw/(memory:XLNXLL/) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a00c0000.v_mix plane-id=30 hold-extra-sample=1 show-preroll=false sync=true' sync=true -v
gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video1 ! video/x-raw/(memory:XLNXLL/), width=1920, height=1080, format=NV12, framerate=60/1 ! queue ! omxh264enc control-rate=low-latency target-bitrate=6000 prefetch-buffer=TRUE num-slices=8 periodicity-idr=240 cpb-size=500 initial-delay=250 gdr-mode=horizontal gop-mode=low-delay-p ! video/x-h264, alignment=nal ! queue max-size-buffers=0 ! omxh264dec low-latency=1 ! video/x-raw/(memory:XLNXLL/) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a00c0000.v_mix plane-id=31 hold-extra-sample=1 show-preroll=false sync=true' sync=true -v
gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video2 ! video/x-raw/(memory:XLNXLL/), width=1920, height=1080, format=NV12, framerate=60/1 ! queue ! omxh264enc control-rate=low-latency target-bitrate=6000 prefetch-buffer=TRUE num-slices=8 periodicity-idr=240 cpb-size=500 initial-delay=250 gdr-mode=horizontal gop-mode=low-delay-p ! video/x-h264, alignment=nal ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=fakesink sync=true' sync=true -v
gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video3 ! video/x-raw/(memory:XLNXLL/), width=1920, height=1080, format=NV12, framerate=60/1 ! queue ! omxh264enc control-rate=low-latency target-bitrate=6000 prefetch-buffer=TRUE num-slices=8 periodicity-idr=240 cpb-size=500 initial-delay=250 gdr-mode=horizontal gop-mode=low-delay-p ! video/x-h264, alignment=nal ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=fakesink sync=true' sync=true -v
- 4kp60 AVC NV12 Xilinx Low-Latency Serial Pipeline
-
gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV12, framerate=60/1 ! omxh264enc num-slices=8 control-rate=low-latency gop-mode=low-delay-p target-bitrate=25000 cpb-size=500 gdr-mode=horizontal initial-delay=250 periodicity-idr=240 filler-data=0 prefetch-buffer=true ! video/x-h264, alignment=nal ! queue max-size-buffers=0 ! omxh264dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false sync=true' sync=true -v
- 2x 4kp30 HEVC NV12 Xilinx Low-Latency Serial Pipeline
-
ENC_EXTRA_OP_BUFFERS=10 gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV12, framerate=30/1 ! omxh265enc control-rate=low-latency target-bitrate=12500 filler-data=0 prefetch-buffer=true num-slices=8 periodicity-idr=240 cpb-size=500 gdr-mode=horizontal initial-delay=250 gop-mode=low-delay-p ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false render-rectangle=<0,0,1920,2160> max-lateness=10000000 sync=true' sync=true -v > /run/pipeline0.txt 2>&1 &
ENC_EXTRA_OP_BUFFERS=10 gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video1 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV12, framerate=30/1 ! omxh265enc control-rate=low-latency target-bitrate=12500 filler-data=0 prefetch-buffer=true num-slices=8 periodicity-idr=240 cpb-size=500 gdr-mode=horizontal initial-delay=250 gop-mode=low-delay-p ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=34 hold-extra-sample=1 show-preroll=false render-rectangle=<1920,0,3840,2160> max-lateness=10000000 sync=true' sync=true -v > /run/pipeline1_$filename.txt 2>&1 &
- 4kp60 AVC NV12 Xilinx Low-Latency Streaming Pipeline
-
- Client
-
gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H264" ! rtpjitterbuffer latency=7 ! rtph264depay ! h264parse ! video/x-h264, alignment=nal ! omxh264dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a0070000.v_mix plane-id=33" sync=true -v > /run/client0_pipeline.log 2>&1 &
- Server
-
gst-launch-1.0 -v v4l2src device=/dev/video0 io-mode=4 ! video/x-raw\(memory:XLNXLL\),format=NV12,width=3840,height=2160,framerate=60/1 ! omxh264enc num-slices=8 periodicity-idr=240 cpb-size=500 gdr-mode=horizontal initial-delay=250 control-rate=low-latency prefetch-buffer=true target-bitrate=25000 gop-mode=low-delay-p ! video/x-h264, alignment=nal ! rtph264pay ! udpsink buffer-size=60000000 host=192.168.0.2 port=5004 max-lateness=-1 qos-dscp=60 async=false max-bitrate=120000000 -v > /run/server0_pipeline.log 2>&1 &
- Ctrl-SW command
- The Xilinx Low Latency mode is not supported in the example
control software encoder application
ctrlsw_encoder
, that comes with PetaLinux as this latency mode is used mainly for live use-cases that thectrlsw_encoder
does not support. However, there is a separate v4l2 capture control software encoder application implemented which also supports AMD’s Low-Latency feature using the ctrlsw APIs. It is an extension to the normal VCUctrlsw
app (ctrlsw_encoder
) to support live record and streamout use-cases. Refer to this Wiki page for commands and additional details.