SEI Data - SEI Data - 3.0 English - PG447

Data insertion API function AL_Encoder_AddSei adds SEI NAL to the stream. The caller is responsible for the SEI payload and must write it as specified in Annex D.3 of ITU-T264/265. The encoder does the rest including anti-emulation of the payload. The SEI cannot exceed 2 KB. This should be largely sufficient for all the SEI NALs.

The stream buffer needs to have enough space to add the SEI section. If not, AL_Encoder_AddSei reports a failure. Prefix and suffix SEI are supported. The encoder puts the SEI section at the correct place in the stream buffer to create a con-formant stream. The control software application showcases adding an SEI message using AL_Encoder_AddSei. To insert multiple SEI messages to a stream, AL_Encoder_AddSei can be called multiple times or multiple SEI payloads can be added when calling AL_Encoder_AddSei. A 2 KB limit applies per stream buffer; all SEI messages per AU should be within the 2 KB limit. SEI messages are already synchronized with corresponding video frames; there is no need for a time stamping mechanism for synchronization. The VCU2 software supports adding SEI meta-data through the OMX/GStreamer application. The following OMX_API Indices and Struct are used to insert SEI meta-data.

OMX_ALG_IndexConfigVideoInsertPrefixSEI

OMX_ALG_IndexConfigVideoInsertSuffixSEI

Struct: OMX_ALG_VIDEO_CONFIG_DATA

static gboolean send_downstream_event (GstPad * pad, GstStructure * s)
{
  GstEvent *event;
  GstPad *peer;
  event = gst_event_new_custom (GST_EVENT_CUSTOM_DOWNSTREAM, s);
  peer = gst_pad_get_peer (pad);
  
  if (!gst_pad_send_event (peer, event)) {
    _g_printerr ("Failed to send custom event\n");_
    gst_object_unref (peer);
    return TRUE;
  }
}
static GstPadProbeReturn buffer_probe(GstPad * pad, GstPadProbeInfo * info, 
                                      gpointer user_data) 
{
  const gchar *data = "some data";
  GstBuffer *sei;
  GstStructure *s;
  sei = gst_buffer_new_wrapped (g_strdup (data), strlen (data));
  s = gst_structure_new ("omx-alg/insert-prefix-sei", "payload-type", G_TYPE_UINT,
                         77, "payload", GST_TYPE_BUFFER, sei, NULL);
  send_downstream_event (pad, s);
  gst_buffer_unref (sei); 
}

For SEI retrieval when decoding, an AL_CB_ParsedSEI callback is invoked each time the decoder parses an SEI. The sei_payload data is provided as described in Annex D.3 of ITU-T264/265. Emulation prevention bytes are removed by the decoder. The control software application shows an example of API usage which can be triggered using the -sei-file option on the command line. The VCU2 software supports parsing of SEI meta-data using the OMX/GStreamer also.

When SEI is parsed, the OMX component should call EventHandle with eEvent based on the SEI type: OMX_ALG_EventSEIPrefixParsed and OMX_ALG_EventSEISuffixParsed.

The reference implementation to handle SEI event using GStreamer.

static gboolean bus_call (GstBus * bus, GstMessage * msg, gpointer data)
{
  GMainLoop *loop = (GMainLoop *) data;
  switch (GST_MESSAGE_TYPE (msg)) {
    case GST_MESSAGE_APPLICATION: {
       const GstStructure *s;
       guint sei_type;
       GstBuffer *buf;
       gboolean is_prefix;
       s = gst_message_get_structure (msg); 
       if (gst_structure_has_name (s, "omx-alg/sei-parsed")) {
         gst_structure_get (s, "payload-type", G_TYPE_UINT, &sei_type, "payload", 
           GST_TYPE_BUFFER, &buf, "prefix", G_TYPE_BOOLEAN, &is_prefix, NULL);
         gst_util_dump_buffer (buf);
         gst_buffer_unref (buf);
       }
       break;
    }
  }
  return TRUE;
}

The SEI callbacks are non-blocking and are in decode order; as soon as SEI NAL unit is parsed in the decoder, the callback is triggered. The caller will have to buffer if there is any re-ordering need to apply due to the difference between decode and display order.