前言
我们已经分析了demo中的第一步 avformat_open_input 这是一段不短的旅程,总的来说,我们已经完成了AVFormatContext的创建和初始化,找到了文件对应的AVInputFormat,也已经创建了AVStream,当然也初始化了AVIOContext,找到了操作文件的URLProtocol。
下一个调用的方法是 avformat_find_stream_info 我们会读取一些媒体文件的数据,然后分析文件的真正信息。
该方法也是一个比较复杂的方法了,这里我做了一个调用关系图
avformat_find_stream_info
先来看一下函数的定义,参数只有两个,一个是AVFormatContext,还有一个是很少用到的额外参数字典。函数的介绍很长,但是总结来说就是获取media的信息。
/**
* Read packets of a media file to get stream information. This
* is useful for file formats with no headers such as MPEG. This
* function also computes the real framerate in case of MPEG-2 repeat
* frame mode.
* The logical file position is not changed by this function;
* examined packets may be buffered for later processing.
*
* @param ic media file handle
* @param options If non-NULL, an ic.nb_streams long array of pointers to
* dictionaries, where i-th member contains options for
* codec corresponding to i-th stream.
* On return each dictionary will be filled with options that were not found.
* @return >=0 if OK, AVERROR_xxx on error
*
* @note this function isn't guaranteed to open all the codecs, so
* options being non-empty at return is a perfectly normal behavior.
*
* @todo Let the user decide somehow what information is needed so that
* we do not waste time getting stuff the user does not need.
*/
int avformat_find_stream_info(AVFormatContext *ic, AVDictionary **options);
函数的实现非常长,我们一点点来看
int avformat_find_stream_info(AVFormatContext *ic, AVDictionary **options)
{
.........
av_opt_set(ic, "skip_clear", "1", AV_OPT_SEARCH_CHILDREN);
//初始化最大读取时长
max_stream_analyze_duration = max_analyze_duration;
max_subtitle_analyze_duration = max_analyze_duration;
if (!max_analyze_duration) {
max_stream_analyze_duration =
max_analyze_duration = 5*AV_TIME_BASE;
max_subtitle_analyze_duration = 30*AV_TIME_BASE;
if (!strcmp(ic->iformat->name, "flv"))
max_stream_analyze_duration = 90*AV_TIME_BASE;
if (!strcmp(ic->iformat->name, "mpeg") || !strcmp(ic->iformat->name, "mpegts"))
max_stream_analyze_duration = 7*AV_TIME_BASE;
}
if (ic->pb)
av_log(ic, AV_LOG_DEBUG, "Before avformat_find_stream_info() pos: %"PRId64" bytes read:%"PRId64" seeks:%d nb_streams:%d\n",
avio_tell(ic->pb), ic->pb->bytes_read, ic->pb->seek_count, ic->nb_streams);
//遍历所有通道
for (i = 0; i < ic->nb_streams; i++) {
.........
// only for the split stuff
//如果parser解析器还没有被设置,那么需要去初始化解析器
//在demo中pareser解析器已经被设置了,所以不需要运行。
//实际上 av_parser_init 方法就是遍历解析器列表,和codec_id进行对比,选择一个。
//然后调用 AVCodecParser 的 parser_init方法
//解析器仅用于需要将数据进行切片的地方,比如h264需要将数据切成一个一个packat
if (!st->parser && !(ic->flags & AVFMT_FLAG_NOPARSE) && st->request_probe <= 0) {
st->parser = av_parser_init(st->codecpar->codec_id);
if (st->parser) {
if (st->need_parsing == AVSTREAM_PARSE_HEADERS) {
st->parser->flags |= PARSER_FLAG_COMPLETE_FRAMES;
} else if (st->need_parsing == AVSTREAM_PARSE_FULL_RAW) {
st->parser->flags |= PARSER_FLAG_USE_CODEC_TS;
}
} else if (st->need_parsing) {
av_log(ic, AV_LOG_VERBOSE, "parser not found for codec "
"%s, packets or times may be invalid.\n",
avcodec_get_name(st->codecpar->codec_id));
}
}
解析器的是整个ffmpeg工作流程中比较重要的一部分,我们可能会在以后慢慢研究,当前我们关注的就是如何读取信息的,
.........
ret = avcodec_parameters_to_context(avctx, st->codecpar);
if (ret < 0)
goto find_stream_info_err;
if (st->request_probe <= 0)
st->internal->avctx_inited = 1;
//搜索AVStream匹配的decoder
codec = find_probe_decoder(ic, st, st->codecpar->codec_id);
find_probe_decoder我们在后面详细分析
..............
//初始化AVCodecContext,打开decoder
/* Ensure that subtitle_header is properly set. */
if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE
&& codec && !avctx->codec) {
if (avcodec_open2(avctx, codec, options ? &options[i] : &thread_opt) < 0)
av_log(ic, AV_LOG_WARNING,
"Failed to open codec in %s\n",__FUNCTION__);
}
// Try to just open decoders, in case this is enough to get parameters.
if (!has_codec_parameters(st, NULL) && st->request_probe <= 0) {
if (codec && !avctx->codec)
if (avcodec_open2(avctx, codec, options ? &options[i] : &thread_opt) < 0)
av_log(ic, AV_LOG_WARNING,
"Failed to open codec in %s\n",__FUNCTION__);
}
............
.............
read_size = 0;
//循环读取数据获取媒体信息
for (;;) {
.......
//遍历所有AVStream,查看是否还有通道需要读取数据,如果某个通道还需要获取数据,直接退出这个循环
/* check if one codec still needs to be handled */
for (i = 0; i < ic->nb_streams; i++) {
int fps_analyze_framecount = 20;
int count;
st = ic->streams[i];
//has_codec_parameters 这个方法用来判断AVStream是否已经读取到了需要的信息(参数是否被赋值),
//实现上挺简单的
if (!has_codec_parameters(st, NULL))
break;
......
}
//如果所有AVStream的信息都已经被读出,那么退出循环。
..........
//读取一帧压缩码流数据
/* NOTE: A new stream can be added there if no header in file
* (AVFMTCTX_NOHEADER). */
ret = read_frame_internal(ic, &pkt1);
//根据这一帧获取信息
......
/* If still no information, we try to open the codec and to
* decompress the frame. We try to avoid that in most cases as
* it takes longer and uses more memory. For MPEG-4, we need to
* decompress for QuickTime.
*
* If AV_CODEC_CAP_CHANNEL_CONF is set this will force decoding of at
* least one frame of codec data, this makes sure the codec initializes
* the channel configuration and does not only trust the values from
* the container. */
//如果还没有足够的信息,那么就会尝试着打开decoder,解码数据。
try_decode_frame(ic, st, pkt,
(options && i < orig_nb_streams) ? &options[i] : NULL);
if (ic->flags & AVFMT_FLAG_NOBUFFER)
av_packet_unref(pkt);
st->codec_info_nb_frames++;
count++;
}
.........
DTS、PTS 的概念如下所述:
- DTS(Decoding Time Stamp):即解码时间戳,这个时间戳的意义在于告诉播放器该在什么时候解码这一帧的数据。
- PTS(Presentation Time Stamp):即显示时间戳,这个时间戳用来告诉播放器该在什么时候显示这一帧的数据。
需要注意的是:虽然 DTS、PTS 是用于指导播放端的行为,但它们是在编码的时候由编码器生成的。
/**
* Real base framerate of the stream.
* This is the lowest framerate with which all timestamps can be
* represented accurately (it is the least common multiple of all
* framerates in the stream). Note, this value is just a guess!
* For example, if the time base is 1/90000 and all frames have either
* approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.
*/
//计算AVStream 的 r_frame_rate
ff_rfps_calculate(ic);
//计算 AVStream avg_frame_rate
..........
// 用于估算AVFormatContext以及AVStream的时长duration
if (probesize)
estimate_timings(ic, old_offset);
..........
find_probe_decoder
这是一个比较重要的方法,用于初始化AVStream的AVCodec。
static const AVCodec *find_probe_decoder(AVFormatContext *s, const AVStream *st, enum AVCodecID codec_id)
{
const AVCodec *codec;
// 如果显示指定了使用h264,那就大胆的去搜索h264的decodec,规则就是去匹配AVCodec的name字段。
#if CONFIG_H264_DECODER
/* Other parts of the code assume this decoder to be used for h264,
* so force it if possible. */
if (codec_id == AV_CODEC_ID_H264)
return avcodec_find_decoder_by_name("h264");
#endif
// 寻找codec
codec = find_decoder(s, st, codec_id);
if (!codec)
return NULL;
if (codec->capabilities & AV_CODEC_CAP_AVOID_PROBING) {
const AVCodec *probe_codec = NULL;
while (probe_codec = av_codec_next(probe_codec)) {
if (probe_codec->id == codec_id &&
av_codec_is_decoder(probe_codec) &&
!(probe_codec->capabilities & (AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_EXPERIMENTAL))) {
return probe_codec;
}
}
}
return codec;
}
static const AVCodec *find_decoder(AVFormatContext *s, const AVStream *st, enum AVCodecID codec_id)
{
//如果AVCodecContext的codec不是空,那么直接返回。
#if FF_API_LAVF_AVCTX
FF_DISABLE_DEPRECATION_WARNINGS
if (st->codec->codec)
return st->codec->codec;
FF_ENABLE_DEPRECATION_WARNINGS
#endif
//如果AVStream指定了默认的Codec,那么返回。
switch (st->codecpar->codec_type) {
case AVMEDIA_TYPE_VIDEO:
if (s->video_codec) return s->video_codec;
break;
case AVMEDIA_TYPE_AUDIO:
if (s->audio_codec) return s->audio_codec;
break;
case AVMEDIA_TYPE_SUBTITLE:
if (s->subtitle_codec) return s->subtitle_codec;
break;
}
//根据codec_id寻找对应的AVCodec
return avcodec_find_decoder(codec_id);
}
try_decode_frame
该函数用于尝试解码。定义如下
/* returns 1 or 0 if or if not decoded data was returned, or a negative error */
static int try_decode_frame(AVFormatContext *s, AVStream *st, AVPacket *avpkt,
AVDictionary **options)
入参都很正,完全能看懂。
/* returns 1 or 0 if or if not decoded data was returned, or a negative error */
static int try_decode_frame(AVFormatContext *s, AVStream *st, AVPacket *avpkt,
AVDictionary **options)
{
AVCodecContext *avctx = st->internal->avctx;
const AVCodec *codec;
int got_picture = 1, ret = 0;
AVFrame *frame = av_frame_alloc();
AVSubtitle subtitle;
AVPacket pkt = *avpkt;
int do_skip_frame = 0;
enum AVDiscard skip_frame;
if (!frame)
return AVERROR(ENOMEM);
//如果decoder没有打开,或者没有decoder
if (!avcodec_is_open(avctx) &&
st->info->found_decoder <= 0 &&
(st->codecpar->codec_id != -st->info->found_decoder || !st->codecpar->codec_id)) {
AVDictionary *thread_opt = NULL;
//寻找decoder
codec = find_probe_decoder(s, st, st->codecpar->codec_id);
if (!codec) {
st->info->found_decoder = -st->codecpar->codec_id;
ret = -1;
goto fail;
}
/* Force thread count to 1 since the H.264 decoder will not extract
* SPS and PPS to extradata during multi-threaded decoding. */
av_dict_set(options ? options : &thread_opt, "threads", "1", 0);
if (s->codec_whitelist)
av_dict_set(options ? options : &thread_opt, "codec_whitelist", s->codec_whitelist, 0);
//打开decoder
ret = avcodec_open2(avctx, codec, options ? options : &thread_opt);
if (!options)
av_dict_free(&thread_opt);
if (ret < 0) {
st->info->found_decoder = -avctx->codec_id;
goto fail;
}
st->info->found_decoder = 1;
} else if (!st->info->found_decoder)
st->info->found_decoder = 1;
if (st->info->found_decoder < 0) {
ret = -1;
goto fail;
}
if (avpriv_codec_get_cap_skip_frame_fill_param(avctx->codec)) {
do_skip_frame = 1;
skip_frame = avctx->skip_frame;
avctx->skip_frame = AVDISCARD_ALL;
}
//如果pkt有内容,并且AVStream还没有获得足够数据,那么进行解码。
while ((pkt.size > 0 || (!pkt.data && got_picture)) &&
ret >= 0 &&
(!has_codec_parameters(st, NULL) || !has_decode_delay_been_guessed(st) ||
(!st->codec_info_nb_frames &&
(avctx->codec->capabilities & AV_CODEC_CAP_CHANNEL_CONF)))) {
got_picture = 0;
//如果是音频流或者视频流
if (avctx->codec_type == AVMEDIA_TYPE_VIDEO ||
avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
ret = avcodec_send_packet(avctx, &pkt);
if (ret < 0 && ret != AVERROR(EAGAIN) && ret != AVERROR_EOF)
break;
if (ret >= 0)
pkt.size = 0;
ret = avcodec_receive_frame(avctx, frame);
if (ret >= 0)
got_picture = 1;
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
ret = 0;
//如果是字幕流
} else if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
ret = avcodec_decode_subtitle2(avctx, &subtitle,
&got_picture, &pkt);
if (ret >= 0)
pkt.size = 0;
}
if (ret >= 0) {
if (got_picture)
st->nb_decoded_frames++;
ret = got_picture;
}
}
if (!pkt.data && !got_picture)
ret = -1;
fail:
if (do_skip_frame) {
avctx->skip_frame = skip_frame;
}
av_frame_free(&frame);
return ret;
}
字幕流和音视频流的处理方案不同,不过每种流的解码处理我们后面再介绍。
estimate_timings
estimate_timings()用于估算AVFormatContext以及AVStream的时长duration。
从estimate_timings()的代码中可以看出,有3种估算方法:
(1)通过pts(显示时间戳)。该方法调用estimate_timings_from_pts()。它的基本思想就是读取视音频流中的结束位置AVPacket的PTS和起始位置AVPacket的PTS,两者相减得到时长信息。
(2)通过已知流的时长。该方法调用fill_all_stream_timings()。它的代码没有细看,但从函数的注释的意思来说,应该是当有些视音频流有时长信息的时候,直接赋值给其他视音频流。
(3)通过bitrate(码率)。该方法调用estimate_timings_from_bit_rate()。它的基本思想就是获得整个文件大小,以及整个文件的bitrate,两者相除之后得到时长信息。
结语
参考:https://blog.youkuaiyun.com/leixiaohua1020/article/details/44084321