Gstreamer playbin3框架

1. Introduction

  playbin3 provides a stand-alone everything-in-one abstraction for an audio and/or video player. It differs from the previous playbin by supporting publication and selection of available streams via the stream collection message and select streams event API.

  It reduces CPU, memory and I/O resource usage. And it provides faster stream switching and full control over the stream selection process. Moreever, new adaptive streaming is supported.

                   

2. Componets

2.1 parsebin

• perform the autoplugging of demuxer/depayloaders/parser
• expose pad when detect decoder element
• no queueing

2.2 uirsourcebin

• detect stream type(the source element and typefind)
• perform the autoplugging of parsebin or adaptive demuxers

2.3 decodebin3

• perform the autoplugging of decoders
• reuse the decoder if caps matched when switching streams

2.4 uridecodebin3

• combination of urisourcebin and decodebin3

2.5 playsink

• handles display of audio/video/text
• allows for setting audio/video sinks or does automatic sink selection.
• allows for configuration of visualisation element.
• allows for enable/disable of visualisation, audio and video

2.6 playbin3

• combination of 1or moe uridecodebin3 elements to read uri/subtitle uri
• support for queuing new media to support gapless playback
• handles stream selection
• uses playsink to display
• selcetion of sinks and configuration of uridecodebin3 with raw output formats

3. Pipeline

4. Multiqueue interleave mechanism in Playbin3

4.1 Update interleave

• Traverse all single queues to obtain the low (minimum) and high (maximum) values of cached_sinktime
• Calculate the interval value and update it in the current single queue
• Obtain the maximum interval value from all single queues
• Update the latest interleaved value to the max_size.time of each single queue

4.2 Check the current single queue if is full

• Return true if it is EOS or segment done
• Return true if the byte size of queue exceed the specified value
• Return true if the difference of sink_time and src_time exceed the max_size.time(ignore subtitle stream).

5. module Analysis

5.1 parsebin

5.1.1 Description

• Autoplug and parse media
• Input: single pad with ANY caps
• Output: Dynamic pads

5.1.2 Contents

• Typefind
• ParseChain
• A single chain of parsing
• Ended with two srcpad(demuxer) , top level chain.
• Or ended with the element has a single srcpad and output raw data
• ParseGroup
• Combines a number of chains that are created by a demuxer
• When autopluggin, all streams coming out of a demuxer will be grouped in a ParseGruop
• Ended if all child chains are complete or no suitable plugins could be found

5.1.3 Basic process flow

• Create typefind, which will create typefind element and set ''have-type" signal handler
• Parsebin catch “have-type” signal and create the proper element to link to typefind and activate it
• Parsebin catch "pad-added" and create the elements recursively until find decoder element
• Parsebin catch "no more pad“ and check if need expose pad
• Exit autoplugging if find the decoder element

 

5.2 urisourcebin

5.2.1 Description

 

• Autoplug and parse media
• Output: Dynamic pads     

5.2.2 Contents

• Source element such as filesrc or souphttpsrc
• Typefind
• Parsebin or adaptive demuxers such as dashdemux2, hlsdemux2
• Queuing element, such as multiqueue
• Slot: a data structure that records a pair of pads (sink/src) for multiqueue

5.2.3 Basic process flow

• Create source element and typefind, and start autoplugging in the "have type" callback function of typefind
• If a parse stream is required, create a parsebin and create the corresponding slot
• Create ghost src pad and expose it

5.3 decodebin3

5.3.1 Description

• Autoplug and decode media
• Input: single pad with ANY caps
• Output: Dynamic pads

5.3.2 Contents

• Input
• Input is the element before multiqueue, such as identity or parsebin
• Input stream refers to the input of each track
• Or ended with the element has a single srcpad and output raw data
• Multiqueue
• A slot corresponds to an input stream, and a slot is actually a data structure that records a pair of pads (sink/src) for multi queues and manages them
• Audio or video decoder

5.3.3 Basic process flow

• ​​​Create multiqueue element
• Create identity when receiving stream collection event
• Probe buffer or gap event and create one slot of multiqueue
• Probe caps event in src pad of the slot and try to create decoder element
• Add selected src pad to decodebin3

5.4 uridecodebin3

5.4.1 Basic Flow

• Create decodebin3 and set the related signal callback functions
• Create sourcebin when setting URI and set the relevant signal callback function
• In the “pad added” callback function of urisourcebin, connect urisourcebin with decodebin3
• In the “pad added” callback function of decodebin3, create a ghost pad and add it to uirdecodebin3

 

 

5.5 playbin3

5.5.1 Basic process flow

• Create uridecodebin3,playlink and set relevant signal callback functions
• Process the stream collection message from bus, update the related data structures and request sink pad of playsink
• When uridecodebin3 emit pad-added, connect uridecodebin3 with playsink
• If there is a customer combiner, it will be added between the two elements

5.5.2 Bus handling mechanism

• Element: set internal bus
• Post message: gst_bus_post function will call sync callback function
• Bin: sets the sync callback function for the bus
• When adding an element to bin, the bus of that element will be set to bin's bus
• Playbin3, uridecodebin3, decodebin3, parsebin has the same bus
• Playbin3: rewrite handle bus message function to obtain stream collection messages for all internal elements
• Get the stream collection message from decodebin3 and send stream select event back