gstreamer基础教程12-Streaming

索引：https://blog.youkuaiyun.com/knowledgebao/article/details/84621238

Goal

Playing media straight直接 from the Internet without storing it locally is known as Streaming. We have been doing it throughout the tutorials whenever we used a URI starting with http://. This tutorial shows a couple of additional points to keep in mind when streaming. In particular:

• How to enable buffering (to alleviate network problems)
• How to recover from interruptions中断 (lost clock)

Introduction

流播放由于缓存原因，可能数据块延迟，导致播放暂停。一般的解决方案是，设置缓存。播放稍微延迟一点，增加一点缓存，保证数据你能够正常播放。可以对消息GST_MESSAGE_BUFFERING进行处理，进而控制视频的播放和暂停。

为了保持各个流之间的通过，会参考系统时钟，但是系统时钟有可能会发生变化，通过监听GST_MESSAGE_CLOCK_LOST消息，结合暂停开始，达到重置系统基准的问题。

When streaming, media chunks are decoded and queued for presentation as soon as they arrive form the network. This means that if a chunk is delayed (which is not an uncommon situation at all on the Internet) the presentation queue might run dry and media playback could stall.

The universal一般通用 solution is to build a “buffer”, this is, allow a certain number of media chunks to be queued before starting playback. In this way, playback start is delayed a bit, but, if some chunks are late, reproduction is not impacted影响 as因为 there are more chunks in the queue, waiting.

As it turns out事实证明, this solution is already implemented实施 in GStreamer, but the previous tutorials have not been benefiting受益 from it. Some elements, like the queue2 and multiqueue found inside playbin, are capable of building this buffer and post bus messages regarding the buffer level (the state of the queue). An application wanting to have more network resilience弹性, then, should listen to these messages and pause playback if the buffer level is not high enough (usually, whenever每当 it is below 100%).

To achieve synchronization同步 among multiple sinks (for example and audio and a video sink) a global clock is used. This clock is selected by GStreamer among all elements which can provide one. Under some circumstances情况, for example, an RTP source switching streams or changing the output device, this clock can be lost and a new one needs to be selected. This happens mostly when dealing with streaming, so the process is explained解释 in this tutorial.

When the clock is lost, the application receives a message on the bus; to select a new one, the application just needs to set the pipeline to PAUSED and then to PLAYING again.

A network-resilient example

Copy this code into a text file named basic-tutorial-12.c.

basic-tutorial-12.c

#include <gst/gst.h>
#include <string.h>

typedef struct _CustomData {
  gboolean is_live;
  GstElement *pipeline;
  GMainLoop *loop;
} CustomData;

static void cb_message (GstBus *bus, GstMessage *msg, CustomData *data) {

  switch (GST_MESSAGE_TYPE (msg)) {
    case GST_MESSAGE_ERROR: {
      GError *err;
      gchar *debug;

      gst_message_parse_error (msg, &err, &debug);
      g_print ("Error: %s\n", err->message);
      g_error_free (err);
      g_free (debug);

      gst_element_set_state (data->pipeline, GST_STATE_READY);
      g_main_loop_quit (data->loop);
      break;
    }
    case GST_MESSAGE_EOS:
      /* end-of-stream */
      gst_element_set_state (data->pipeline, GST_STATE_READY);
      g_main_loop_quit (data->loop);
      break;
    case GST_MESSAGE_BUFFERING: {
      gint percent = 0;

      /* If the stream is live, we do not care about buffering. */
      if (data->is_live) break;

      gst_message_parse_buffering (msg, &percent);
      g_print ("Buffering (%3d%%)\r", percent);
      /* Wait until buffering is complete before start/resume playing */
      if (percent < 100)
        gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
      else
        gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
      break;
    }
    case GST_MESSAGE_CLOCK_LOST:
      /* Get a new clock */
      gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
      gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
      break;
    default:
      /* Unhandled message */
      break;
    }
}

int main(int argc, char *argv[]) {
  GstElement *pipeline;
  GstBus *bus;
  GstStateChangeReturn ret;
  GMainLoop *main_loop;
  CustomData data;

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

  /* Initialize our data structure */
  memset (&data, 0, sizeof (data));

  /* Build the pipeline */
  pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
  bus = gst_element_get_bus (pipeline);

  /* Start playing */
  ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (pipeline);
    return -1;
  } else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
    data.is_live = TRUE;
  }

  main_loop = g_main_loop_new (NULL, FALSE);
  data.loop = main_loop;
  data.pipeline = pipeline;

  gst_bus_add_signal_watch (bus);
  g_signal_connect (bus, "message", G_CALLBACK (cb_message), &data);

  g_main_loop_run (main_loop);

  /* Free resources */
  g_main_loop_unref (main_loop);
  gst_object_unref (bus);
  gst_element_set_state (pipeline, GST_STATE_NULL);
  gst_object_unref (pipeline);
  return 0;
}

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. In the console window, you should see a buffering message, and playback should only start when the buffering reaches 100%. This percentage might not change at all if your connection is fast enough and buffering is not required.

Walkthrough

需要关注的是直播流，直播流没有暂停状态，所以设置播放后返回值不是GST_STATE_CHANGE_SUCCESS 而是GST_STATE_CHANGE_NO_PREROLL。

The only special thing this tutorial does is react to certain messages; therefore, the initialization code is very simple and should be self-explanative by now. The only new bit is the detection of live streams:

/* Start playing */
ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
  g_printerr ("Unable to set the pipeline to the playing state.\n");
  gst_object_unref (pipeline);
  return -1;
} else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
  data.is_live = TRUE;
}

Live streams cannot be paused, so they behave in PAUSED state as if they were in the PLAYING state. Setting live streams to PAUSED succeeds, but returns GST_STATE_CHANGE_NO_PREROLL, instead of GST_STATE_CHANGE_SUCCESS to indicate that this is a live stream. We are receiving the NO_PREROLL return code even though we are trying to set the pipeline to PLAYING, because state changes happen progressively逐渐 (from NULL to READY, to PAUSED and then to PLAYING).

We care about live streams because we want to disable buffering for them, so we take note of the result of gst_element_set_state() in the is_live variable.

Let’s now review the interesting parts of the message parsing callback:

case GST_MESSAGE_BUFFERING: {
  gint percent = 0;

  /* If the stream is live, we do not care about buffering. */
  if (data->is_live) break;

  gst_message_parse_buffering (msg, &percent);
  g_print ("Buffering (%3d%%)\r", percent);
  /* Wait until buffering is complete before start/resume playing */
  if (percent < 100)
    gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
  else
    gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
  break;
}

处理GST_MESSAGE_BUFFERING消息，使用gst_message_parse_buffering()处理当前消息，如果小于100设置暂停，如果大于100，设置播放。

First, if this is a live source, ignore buffering messages.

We parse the buffering message with gst_message_parse_buffering() to retrieve the buffering level.

Then, we print the buffering level on the console and set the pipeline to PAUSED if it is below 100%. Otherwise, we set the pipeline to PLAYING.

At startup, we will see the buffering level rise up to 100% before playback starts, which is what we wanted to achieve. If, later on, the network becomes slow or unresponsive and our buffer depletes耗尽, we will receive new buffering messages with levels below 100% so we will pause the pipeline again until enough buffer has been built up.

case GST_MESSAGE_CLOCK_LOST:
  /* Get a new clock */
  gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
  gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
  break;

如果系统时钟丢失，可以通过设置PAUSED和PLAYING来重置时钟。

For the second network issue, the loss of clock, we simply set the pipeline to PAUSED and back to PLAYING, so a new clock is selected, waiting for new media chunks to be received if necessary.

Conclusion

This tutorial has described how to add network resilience弹性 to your application with two very simple precautions防范:

• Taking care of buffering messages sent by the pipeline
• Taking care of clock loss

Handling these messages improves the application’s response to network problems, increasing the overall统筹兼顾 playback smoothness平滑.

It has been a pleasure having you here, and see you soon!