gstreamer基础教程13-Playback speed

索引：https://blog.youkuaiyun.com/knowledgebao/article/details/84621238

Goal

本章的目的是如何是播放实现一些特技，比如：快进，快退，慢放等。如何一帧一帧的播放视频。

Fast-forward快进, reverse-playback回退 and slow-motion慢放 are all techniques技术 collectively通用的 known as trick特技 modes and they all have in common that modify the normal playback rate. This tutorial shows how to achieve these effects and adds frame-stepping into the deal. In particular, it shows:

• How to change the playback rate, faster and slower than normal, forward and backwards.
• How to advance前进播放 a video frame-by-frame

Introduction

快进：以大于常速播放。慢放：以小于常速播放。快退：反方向播放。最后体现一个常速播放速率，=1表示常速，>1表示快速，>0且<1慢速，<0表示反向播放(当然反向也有快慢)。GStream提供两种机制来改变播放速率：Step事件和Seek事件。Step事件可以跳过指定数据(比如逐帧播放)和设置正向速率。Seek事件可以跳转到任意位置，并且可以设置正向和反向速率。

Fast-forward is the technique that plays a media at a speed higher than its normal (intended打算的) speed; whereas slow-motion uses a speed lower than the intended one. Reverse playback does the same thing but backwards, from the end of the stream to the beginning.

All these techniques do is change the playback rate, which is a variable equal to 1.0 for normal playback, greater than 1.0 (in absolute value) for fast modes, lower than 1.0 (in absolute value) for slow modes, positive for forward playback and negative for reverse playback.

GStreamer provides two mechanisms机制 to change the playback rate: Step Events and Seek Events. Step Events allow skipping a given amount of media besides除...之外 changing the subsequent playback rate (only to positive values). Seek Events, additionally额外的, allow jumping to any position in the stream and set positive and negative playback rates.

第四节已经使用了seek事件，这一章详细解释如何使用这些事件。因为Step事件需要的参数更少，所以使用更加方便，但是Step有一些缺点，所以这里我们使用seek事件。Step事件只能作用于sink，而Seekp事件作用于整个pipeline。Step的优点是比较快，但是无法改变速率反向（不能快退）。

In Basic tutorial 4: Time management seek events have already been shown, using a helper function to hide their complexity. This tutorial explains a bit more how to use these events.

Step Events are a more convenient更方便 way of changing the playback rate, due to the reduced number of parameters needed to create them; however, they have some downsides缺点, so Seek Events are used in this tutorial instead代替. Step events only affect the sink (at the end of the pipeline), so they will only work if the rest of剩下 the pipeline can support going at a different speed, Seek events go all the way through the pipeline so every element can react to them. The upside of Step events is that they are much faster to act. Step events are also unable to change the playback direction.

为了使用这些事件，我们创建之后传递给pipline，pipline向上传递知道找到可以处理他们的element。如果这个事件传递给先playbin这样的复合element中，playbin只是简单的把事件通知所有的sink去执行。一般经常使用的方法是检索video或audio其中的一个sink，然后把是事件直接作用到指定的sink。逐帧播放就是让视频一帧一帧的播放，逐帧播放是通过暂停pipeline，然后发送step事件每次跳过一帧。

To use these events, they are created and then passed onto the pipeline, where they propagate传播 upstream until they reach an element that can handle them. If an event is passed onto a bin element like playbin, it will simply feed the event to把事件供应给 all its sinks, which will result in multiple seeks being performed执行. The common approach方法 is to retrieve检索 one of playbin’s sinks through the video-sink or audio-sink properties and feed the event directly into the sink.

Frame stepping is a technique that allows playing a video frame by frame. It is implemented by pausing the pipeline, and then sending Step Events to skip one frame each time.

A trick mode player

Copy this code into a text file named basic-tutorial-13.c.

basic-tutorial-13.c

#include <string.h>
#include <stdio.h>
#include <gst/gst.h>

typedef struct _CustomData {
  GstElement *pipeline;
  GstElement *video_sink;
  GMainLoop *loop;

  gboolean playing;  /* Playing or Paused */
  gdouble rate;      /* Current playback rate (can be negative) */
} CustomData;

/* Send seek event to change rate */
static void send_seek_event (CustomData *data) {
  gint64 position;
  GstFormat format = GST_FORMAT_TIME;
  GstEvent *seek_event;

  /* Obtain the current position, needed for the seek event */
  if (!gst_element_query_position (data->pipeline, &format, &position)) {
    g_printerr ("Unable to retrieve current position.\n");
    return;
  }

  /* Create the seek event */
  if (data->rate > 0) {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_NONE, 0);
  } else {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position);
  }

  if (data->video_sink == NULL) {
    /* If we have not done so, obtain the sink through which we will send the seek events */
    g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
  }

  /* Send the event */
  gst_element_send_event (data->video_sink, seek_event);

  g_print ("Current rate: %g\n", data->rate);
}

/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
  gchar *str = NULL;

  if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
    return TRUE;
  }

  switch (g_ascii_tolower (str[0])) {
  case 'p':
    data->playing = !data->playing;
    gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED);
    g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE");
    break;
  case 's':
    if (g_ascii_isupper (str[0])) {
      data->rate *= 2.0;
    } else {
      data->rate /= 2.0;
    }
    send_seek_event (data);
    break;
  case 'd':
    data->rate *= -1.0;
    send_seek_event (data);
    break;
  case 'n':
    if (data->video_sink == NULL) {
      /* If we have not done so, obtain the sink through which we will send the step events */
      g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
    }

    gst_element_send_event (data->video_sink,
        gst_event_new_step (GST_FORMAT_BUFFERS, 1, ABS (data->rate), TRUE, FALSE));
    g_print ("Stepping one frame\n");
    break;
  case 'q':
    g_main_loop_quit (data->loop);
    break;
  default:
    break;
  }

  g_free (str);

  return TRUE;
}

int main(int argc, char *argv[]) {
  CustomData data;
  GstStateChangeReturn ret;
  GIOChannel *io_stdin;

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

  /* Initialize our data structure */
  memset (&data, 0, sizeof (data));

  /* Print usage map */
  g_print (
    "USAGE: Choose one of the following options, then press enter:\n"
    " 'P' to toggle between PAUSE and PLAY\n"
    " 'S' to increase playback speed, 's' to decrease playback speed\n"
    " 'D' to toggle playback direction\n"
    " 'N' to move to next frame (in the current direction, better in PAUSE)\n"
    " 'Q' to quit\n");

  /* Build the pipeline */
  data.pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);

  /* Add a keyboard watch so we get notified of keystrokes */
#ifdef G_OS_WIN32
  io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
  io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
  g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);

  /* Start playing */
  ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (data.pipeline);
    return -1;
  }
  data.playing = TRUE;
  data.rate = 1.0;

  /* Create a GLib Main Loop and set it to run */
  data.loop = g_main_loop_new (NULL, FALSE);
  g_main_loop_run (data.loop);

  /* Free resources */
  g_main_loop_unref (data.loop);
  g_io_channel_unref (io_stdin);
  gst_element_set_state (data.pipeline, GST_STATE_NULL);
  if (data.video_sink != NULL)
    gst_object_unref (data.video_sink);
  gst_object_unref (data.pipeline);
  return 0;
}

这个例子播放一个互联网上的音视频文件(受网络影响可能会有卡顿)。控制台会显示可用的命令，通过输入大写或小写字母+回车，开控制视频播放。

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The console shows the available commands, composed of a single upper-case or lower-case letter字母, which you should input followed by the Enter key.

Walkthrough

这里没有其他新的东西，唯一增加的就是键盘的监控，输入键盘内容，跳转到键盘监控函数，如下：p来控制暂停/开始(切换还是有gst_element_set_state()控制),S速率*2，s速率/2，d控制方向的变化，n控制一帧一帧播放。

There is nothing new in the initialization code in the main function: a playbin pipeline is instantiated, an I/O watch is installed to track keystrokes敲击键盘 and a GLib main loop is executed.

Then, in the keyboard handler function:

/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
  gchar *str = NULL;

  if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
    return TRUE;
  }

  switch (g_ascii_tolower (str[0])) {
  case 'p':
    data->playing = !data->playing;
    gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED);
    g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE");
    break;

Pause / Playing toggle切换 is handled with gst_element_set_state() as in previous tutorials.

case 's':
  if (g_ascii_isupper (str[0])) {
    data->rate *= 2.0;
  } else {
    data->rate /= 2.0;
  }
  send_seek_event (data);
  break;
case 'd':
  data->rate *= -1.0;
  send_seek_event (data);
  break;

Use ‘S’ and ‘s’ to double or halve the current playback rate, and ‘d’ to reverse the current playback direction. In both cases, the rate variable is updated and send_seek_event is called. Let’s review this function.

/* Send seek event to change rate */
static void send_seek_event (CustomData *data) {
  gint64 position;
  GstEvent *seek_event;

  /* Obtain the current position, needed for the seek event */
  if (!gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position)) {
    g_printerr ("Unable to retrieve current position.\n");
    return;
  }

这个函数就是发送一个Seek事件，来控制速率变化。position是一个返回值，返回当前的位置。这里不使用step事件的原因前边已经说明。（注：原文档内容与实际Demo中的内容不一致，以Demo内容为主）。seek事件通过通过函数gst_event_new_seek来创建，主要参数包括：速率，开始位置，结束位置。开始位置必须小于结束位置。(-1表示文件末尾).正如前边解释，为了防止多次Seek，这个Seek事件仅仅发送给一个sink。这里发送给video-sink.

This function creates a new Seek Event and sends it to the pipeline to update the rate. First, the current position is recovered with gst_element_query_position(). This is needed because the Seek Event jumps to another position in the stream, and, since we do not actually want to move, we jump to the current position. Using a Step Event would be simpler, but this event is not currently fully functional, as explained解释 in the Introduction.

  /* Create the seek event */
  if (data->rate > 0) {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_SET, -1);
  } else {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position);
  }

The Seek Event is created with gst_event_new_seek(). Its parameters are, basically主要, the new rate新的速率, the new start position新的开始位置 and the new stop position新的停止位置. Regardless of the playback direction, the start position must be smaller小于 than the stop position, so the two playback directions are treated differently.

if (data->video_sink == NULL) {
  /* If we have not done so, obtain the sink through which we will send the seek events */
  g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
}

As explained in the Introduction, to avoid performing multiple Seeks, the Event is sent to only one sink, in this case, the video sink. It is obtained from playbin through the video-sink property. It is read at this time instead at initialization time because the actual sink may change depending on the media contents, and this won’t be known until the pipeline is PLAYING and some media has been read.

/* Send the event */
gst_element_send_event (data->video_sink, seek_event);

The new Event is finally sent to the selected sink with gst_element_send_event().

通过n可以控制逐帧播放，通过step事件来控制逐帧播放。通过gst_event_new_step创建step，主要参数有帧数、速率等。

Back to the keyboard handler, we still miss the frame stepping code, which is really simple:

case 'n':
  if (data->video_sink == NULL) {
    /* If we have not done so, obtain the sink through which we will send the step events */
    g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
  }

  gst_element_send_event (data->video_sink,
      gst_event_new_step (GST_FORMAT_BUFFERS, 1, ABS (data->rate), TRUE, FALSE));
  g_print ("Stepping one frame\n");
  break;

A new Step Event is created with gst_event_new_step(), whose parameters basically specify the amount数量 to skip (1 frame in the example) and the new rate (which we do not change).

The video sink is grabbed获取 from playbin in case we didn’t have it yet, just like before.

And with this we are done. When testing this tutorial, keep in mind that backward playback is not optimal最佳的 in many elements.

改变速率可能仅仅对本地文件有效，如果上述URL无效，可以尝试把URL改为本地文件，以file:///开头。(这里我没有尝试成功，我把Demo中的文件sintel_trailer-480p.webm下载到本地，然后URL修改为本地地址，直接播放失败)

Changing the playback rate might only work with local files. If you cannot modify it, try changing the URI passed to playbin in line 114 to a local URI, starting with file:///

Conclusion

This tutorial has shown:

• How to change the playback rate using a Seek Event, created with gst_event_new_seek() and fed to the pipeline with gst_element_send_event().
• How to advance a video frame-by-frame by using Step Events, created with gst_event_new_step().

It has been a pleasure having you here, and see you soon!