GStreamer API for Media Pipeline

Published: May 20, 2020
Edited: May 21, 2020

Video Contract

Applications play video using GStreamer. Video pipelines are created using playbin and rely on auto-plugging. Applications must not create specific elements. Applications may specify appsrc as the pipeline source element by setting the uri property of playbin to appsrc:// and use the playbin source-setup signal to obtain and configure the appsrc element.

Video Sink

The video sink element is selected by playbin via auto-plugging. The application may obtain the video sink element selected by playbin for the pipeline using the playbin video-sink property:

GstElement *videoSink;
g_object_get( playbin, “video-sink”, &videoSink, NULL);

Alpha Hole Punching

Video is presented beneath graphics. The application is responsible for performing alpha hole punching so the video is visible. This involves drawing graphics pixels with zero alpha (for full video visibility) or pixels with alpha values less than opaque (for translucent graphics blended with video). See the rectangle property for video positioning.

Video Sink Contract

QoS

The video sink will supply QoS messages that the application can monitor to obtain information such as total frames and dropped frames. The application can obtain this information within a GStreamer bus watch function using:

gst_message_parse_qos_stats( GstMessage msg, GstFormat format, quint64 *processed, quint64 *dropped );

HDR

For HDR content where HDR meta-data is at the container level rather than the elementary stream level, for example with VP9, the video sink will require and accept HDR meta-data via GstCaps. The following pseudo code illustrates how the sink would access this data:

GGstStructure *structure;
const char *colorimetry;
const char *masteringDisplayMetadata;
const char *contentLightLevel;

structure= gst_caps_get_structure(caps, 0);

if ( gst_structure_has_field(structure, "colorimetry") )
    {

    colorimetry= gst_structure_get_string(structure,"colorimetry")

    }
if ( gst_structure_has_field(structure, "mastering-display-metadata") )
    {

    masteringDisplayMetadata= gst_structure_get_string(structure,"mastering-display-metadata");

    }

if ( gst_structure_has_field(structure, "content-light-level") )

    {

   contentLightLevel= gst_structure_get_string(structure,"content-light-level");

    }

Video Sink Interface

The application may interact with the video sink using properties and signals.

Properties
PropertiesRead/WriteDescription
rectangleWrite OnlySets the video size and position in application display coordinates where (0,0) is the top left of the application display and (width, height) is the bottom right of the application display.  The application must perform alpha hole punching to reveal video beneath graphics.

g_object_set( videoSink, “rectangle”, “100,100,640,360”, NULL);
force-aspect-ratioRead/WriteWhen enabled, scaling will respect original aspect ratio. Valid values are true of false.
asyncRead/WriteGo asynchronously to PAUSED. Valid values are true of false.
Signals
SignalsDescription
first-video-frame-callbackSignals the display of the first video frame

Audio Contract

Applications play audio using GStreamer. Audio pipelines are created using playbin and rely on auto-plugging. Applications must not create specific elements. Applications may specify appsrc as the pipeline source element or set the uri property of playbin.

Audio Sink

The audio sink element is selected by playbin via auto-plugging.  The application may obtain the video sink element selected by playbin for the pipeline using playbin’s audio-sink property:

GstElement *audioSink;
g_object_get( playbin, “audio-sink”, &audioSink, NULL);

Audio Sink Contract

Audio Sink Interface

The application may interact with the audio sink using properties and signals.

Properties
PropertiesRead/WriteDescription
muteRead/WriteSets whether audio is muted or unmuted. Valid values are true or false.
volumeRead/WriteSets the volume level. Valid range is 0.0 to 1.0.
asyncRead/WriteGo asynchronously to PAUSED. Valid values are true or false.
Signals
SignalsDescription
first-audio-frame-callbackSignals the rendering of the first audio frame

PHP Code Snippets Powered By : XYZScripts.com