Insertion of SEI Data at Encoder - 2023.1 English

AL_Encoder_AddSei() adds SEI NAL to the stream. You are responsible for the SEI payload and have to write it as specified in Annex D.3 of ITU-T. The encoder does the rest including anti-emulation of the payload. The SEI cannot exceed 2 KB. This should be largely sufficient for all the SEI NALs.

The stream buffer needs to have enough space to add the SEI section. If not, AL_Encoder_AddSei() reports a failure. Prefix and suffix SEI are supported. The encoder puts the SEI section at the correct place in the stream buffer to create a conformant stream. The control software application show cases adding a SEI message using AL_Encoder_AddSei(). To insert multiple SEI messages to stream, AL_Encoder_AddSei() can be called multiple times or multiple SEI payloads can be added to AL_Encoder_AddSei() API. 2 KB limit is per stream buffer, i.e. all SEI messages per AU should be with-in the 2 KB limit. SEI messages are already synchronized with corresponding video frames; there should not be a need for timestamping mechanism for synchronization. VCU-SW supports adding SEI meta-data through omx/gstreamer application. The following OMX_API /Indexes and Struct are used to insert SEI meta-data.

• OMX_ALG_IndexConfigVideoInsertPrefixSEI
• OMX_ALG_IndexConfigVideoInsertSuffixSEI
• Struct: OMX_ALG_VIDEO_CONFIG_DATA

You can insert your own data using gstreamer application at any frame. The reference implementation function is as follows:

static gboolean send_downstream_event (GstPad * pad, GstStructure * s)
{
GstEvent *event;
GstPad *peer;
event = gst_event_new_custom (GST_EVENT_CUSTOM_DOWNSTREAM, s);
peer = gst_pad_get_peer (pad);
if (!gst_pad_send_event (peer, event))
    _g_printerr ("Failed to send custom event\n");_
    gst_object_unref (peer);
    return TRUE;
}
static GstPadProbeReturn buffer_probe (GstPad * pad, GstPadProbeInfo * info, 
gpointer user_data)
{
const gchar *data = "some data";
GstBuffer *sei;
GstStructure *s;
sei = gst_buffer_new_wrapped (g_strdup (data), strlen (data));
s = gst_structure_new ("omx-alg/insert-prefix-sei", "payload-type", G_TYPE_UINT, 77, 
"payload", GST_TYPE_BUFFER, sei, NULL);
send_downstream_event (pad, s);
gst_buffer_unref (sei); 
}

Refer to VCU Control Software API for more details.