Intel MSDK 硬解码

Intel MSDK 是 Intel 公司提供的基于硬件加速功能的多媒体开发框架，通过 Intel 显卡的硬件加速功能（Intel® Quick Sync Video），可实现快速视频转码和图像处理。

• 硬编码格式：HEVC（h.265），AVC（h.264），MPEG-2，JPEG
• 硬解码格式：HEVC，AVC，VP8，MPEG-2，VC1，JPEG，MJPEG
• Filter：颜色空间转换，去隔行，降噪，缩放，旋转

Key Specifications （支持的硬件平台，操作系统，编译器等） 请参考 Intel 官网。

Intel MSDK samples

下载链接：https://software.intel.com/en-us/media-sdk/documentation/code-samples

Sample Name	Description
Transcoding	sample_multi_transcode Transforms an elementary video stream from one compressed format to another.
Encoding	sample_encode Converts raw video frames into an elementary compressed stream.
Decoding	sample_decode Transforms a compressed video stream into raw frames using HEVC decode and VP8 decode. The plug-in and the included sample_decvpp demonstrate decode functions with color conversion of raw video sequences.
Video Processing	sample_vpp Demonstrates how to use algorithms that process raw frames using denoising, deinterlacing, inverse telecine, and color conversion techniques.
OpenCL™ Video Motion Estimation (VME)	ocl_motion_estimation Provides step-by-step guidelines on using Intel’s motion estimation extension for the OpenCL standard. The motion estimation extension includes a set of host-callable functions for frame-based VME.
OpenCL™ Interoperability	ocl_media_sdk_interop Shows how to use Intel Media SDK and Intel? SDK for Open CL? applications together for efficient video decoding and fast post-processing.
HEVC GPU Assist APIs	sample_h265_gaa Supplies examples of the typical data and control flow for using HEVC GPU Assist APIs effectively.

注：本文的代码基于 Intel_Media_SDK_2017_R1。

Intel MSDK decoding sample

以下是我改编过的 decoding sample 用法，利用 FFmpeg 进行 demux：

>intel_msdk_dec.exe
Intel(R) Media SDK Decoding Sample Version 3.5.915.45327

Usage: intel_msdk_dec.exe -i input_file [-o output_file]
Options:
   [-hw]            - use platform specific SDK implementation, if not specified software implementation is used
   [-d3d]           - work with d3d9 surfaces
   [-d3d11]         - work with d3d11 surfaces

Intel MSDK decoding 代码

以下是整个解码及播放过程的概要代码，略去各个函数的具体实现和资源释放：

    CDecodingPipeline dec_pipeline; // pipeline for decoding, includes decoder and output file writer
    SDL_video_helper sdl_helper;
    ffmpeg_reader g_file_reader;
    
    sts = ParseInputString(argv, (mfxU8)argc, &params);
    sts = g_file_reader.Init( params.strSrcFile );
    
    AVCodecID codec_id = g_file_reader.get_codec_id();
    switch (codec_id) {
   
    case AV_CODEC_ID_H264:
        params.videoType = MFX_CODEC_AVC;
        break;
    case AV_CODEC_ID_MPEG2VIDEO:
    case AV_CODEC_ID_MPEG2TS:
        params.videoType = MFX_CODEC_MPEG2;
        break;
    }
    
    int width = 0, height = 0;
    hr = g_file_reader.get_frame_size(width, height, g_pitch);
    
    double frame_rate = 0;
    hr = g_file_reader.get_frame_rate(frame_rate);
    
    hr = sdl_helper.init( SDL_PIXELFORMAT_IYUV, width, height, (int)frame_rate, sdl_get_frame_cb );
    sts = dec_pipeline.Init( &params, &g_file_reader );
    
    g_consumed_evt = CreateEvent( NULL, FALSE, FALSE, NULL );
    g_decoded_evt = CreateEvent( NULL, FALSE, FALSE, NULL );
    HANDLE decoding_thread_handle = CreateThread( NULL, 0, decoding_thread, &dec_pipeline, 0, NULL );
    
    hr = sdl_helper.run();

ffmpeg_reader::Init 函数

H.264 有两种封装，一种是 annexb 模式，即传统模式，有 startcode，SPS 和 PPS 是在 ES 中；一种是 mp4 模式，没有 startcode，SPS 和 PPS 以及其它信息被封装在 container 中，每一个 frame 前面是这个 frame 的长度。很多解码器只支持 annexb 这种模式，因此需要将 mp4 做转换。

H.264 的 SPS 和 PPS 串包含了初始化 H.264 解码器所需要的信息参数，包括编码所用的 profile，level，图像的宽和高，deblock 滤波器等。

mfxStatus ffmpeg_reader::Init(const TCHAR *file_path, mfxU32 video_type = 0)
{
   
    AVCodecContext* dec_ctx = NULL;
    m_video_stream_idx = open_input_file(file_path, AVMEDIA_TYPE_VIDEO, &m_fmt_ctx, NULL);
    RETURN_IF_FALSE_EX(m_video_stream_idx >= 0, MFX_ERR_UNKNOWN)