[RFC] New controls for codec devices

Hi,

I would like to propose a new set of controls for codec devices. By codec
devices I mean hardware that can encode raw video into compressed stream
and can decode compressed streams.

This topic has been discussed during the v4l2 brainstorming session and this
email contains the propositions that have been made.

The conclusion was that we should use the existing V4L2_CID_MPEG_CLASS for
controls related do encoding and decoding data. I had to distinguish which
controls should be common for all codecs and which should be specific to
MFC 5.1 (I am working on the driver for this hardware). To do this I
have analysed the options of ffmpeg and x264 software decoders. If the
options was used only in MFC then it should be hardware specific. If it
was also used in ffmpeg or x264 then it would qualify as a common control.
I would welcome comments from other hw codec driver developers.

I have another question - regarding v4l2_buffer flags. What do you think
about adding a new flag - V4L2_BUF_FLAG_SKIPPED? It would be used in encoding
to indicate that the frame to be encoded should be skipped and write information
that the decoder should display the previous frame again.

Please find the table with controls attached below:
- the names have the "V4L2_CID_MPEG_" prefix truncated
- letter in the second column means D for decoder, E for encoder and A for
 both
- last column is MFC if the control should be hw specific, common otherwise

*** Controls for decoding ***
 1) DECODER_MPEG4_LOOP_FILTER           D   MPEG4           MFC
Deblocking filter on the decoder output for MPEG4.
 2) DECODER_SLICE_INTERFACE             D   all             common
If set the decoder can accept separate picture slices on input, otherwise
it requires a whole frame.
 3) DECODER_H264_DISPLAY_DELAY          D   H264            MFC
Programmable display delay for H264. MFC will return a decoded frame after
the set number of frames (this may cause that frames are not returned in
display order).
 4) DECODER_H264_DISPLAY_DELAY_

ENABLE   D   H264            MFC
Enable display delay for H264.

*** Controls for both decoding and encoding ***
 5) H264_AR_VUI_IDC                     A   H264            common
VUI aspect ratio IDC for H.264 encoding.  The value is defined in VUI Table
E-1 in the standard.
 6) H264_EXT_SAR_WIDTH                  A   H264            common
Extended sample aspect ratio width for H.264 VUI encoding.
 7) H264_EXT_SAR_HEIGHT                 A   H264            common
Extended sample aspect ratio height for H.264 VUI encoding
 8) MIN_REQ_BUFS_OUT                    A   all             common
This is the minimum number of buffers required for the output queue. This
option may be useful if the encoding settings require a minimum number of
buffers required as reference and the application would like to have N more
buffers. For example - the encoding options require 3 buffers and the
application wants to have 2 more. One can read this value (3) and supply
(3+2) to the reqbufs call.
 9) MIN_REQ_BUFS_CAP                    A   all             common
This is the minimum number of buffers required for the capture queue. This
option may be useful if the decoder requires a minimum number of buffers
required as reference and the application would like to have N more buffers.
For example - the stream requires 3 buffers and the application wants
to have 2 more. One can read this value (3) and supply (3+2) to the
reqbufs call.

*** Controls for encoding ***
10) GOP_SIZE                            E   all             common
The size of group of pictures. For H264 this is the IDR period.
11) H264_LEVEL                          E   H264            common
The level information for H264.
12) MPEG4_LEVEL                         E   MPEG4           common
The level information for MPEG4.
13) H264_PROFILE                        E   H264            common
The profile information for H264.
14) MPEG4_PROFILE                       E   MPEG4           common
The profile information for MPEG4.
15) B_FRAMES                            E   H264, MPEG4     common
The number of B-frames to use between P frames.
16) MAX_REF_PIC                         E   H264            common
The maximum number of reference pictures used for encoding.
17) NUM_REF_PIC_FOR_P                   E   H264            MFC
The number of reference pictures used for encoding a P picture.
18) MULTI_SLICE_MODE                    E   H264, MPEG4     common
Determines how multiple slices are handled.
19) MULTI_SLICE_MB                      E   H264, MPEG4     common
The upper limit of macroblocks for a slice.
20) MULTI_SLICE_BITS                    E   H264, MPEG4     common
The upper limit of size for a slice.
21) H264_LOOP_FILTER_MODE               E   H264            common
Loop filter mode for H264.
22) H264_LOOP_FILTER_ALPHA              E   H264            common
Loop filter alpha coefficient, defined in the standard.
23) H264_LOOP_FILTER_BETA               E   H264            common
Loop filter alpha coefficient, defined in the standard.
24) H264_SYMBOL_MODE                    E   H264            common
Symbol mode for H264 - CABAC/CAVALC.
25) INTERLACE                           E   H264, MPEG4     common
Enable interlace mode.
26) H264_8X8_TRANSFORM                  E   H264            common
Enable 8X8 transform for H264.
27) INTRA_REFRESH_MB                    E   all             common
Period of random intra macroblock refresh.
28) PADDING_ENABLE                      E   all             MFC
Padding enable - use a color instead of repeating border pixels.
29) PADDING_LUMA                        E   all             MFC
30) PADDING_CB                          E   all             MFC
31) PADDING_CR                          E   all             MFC
32) FRAME_RC_ENABLE                     E   all             common
Frame level rate control enable.
33) MB_RC_ENABLE                        E   H264            common
Macroblock level rate control enable.
34) FRAME_RATE                          E   all             common
Frames per second in 1000x scale (e.g., 7500 stands for 7.5 frames/sec).
35) RC_BITRATE                          E   all             common
Bitrate for rate control.
36) RC_REACTION_COEFF                   E   all             MFC
Reaction coefficient for MFC rate control.
37) H264_ADAPTIVE_RC_DARK_DISABLE       E   H264            MFC
Disable adaptive rate control for dark region.
38) H264_ADAPTIVE_RC_SMOOTH_DISABLE     E   H264            MFC
Disable adaptive rate control for smooth region.
39) H264_ADAPTIVE_RC_STATIC_DISABLE     E   H264            MFC
Disable adaptive rate control for static region.
40) H264_ADAPTIVE_RC_ACTIVITY_DISABLE   E   H264            MFC
Disable adaptive rate control for region with activity.
41) MPEG4_QPEL_DISABLE                  E   MPEG4           common
Disable quarter pixel motion estimation for MPEG4.
42) I_FRAME_QP                          E   all             common
Quantization parameter for an I frame.
43) MIN_QP                              E   all             common
Minimum quantization parameter.
44) MAX_QP                              E   all             common
Maximum quantization parameter.
45) P_FRAME_QP                          E   all             common
Quantization parameter for an P frame.
46) B_FRAME_QP                          E   H264, MPEG4     common
Quantization parameter for an B frame.
47) VBV_BUF_SIZE                        E   H264, MPEG*     common
VBV buffer size. I think it is valid for all MPEG versions.
48) FRAME_SKIP_MODE                     E   all             MFC
Mode of skipping frames for VBV compliance.
49) RC_FIXED_TARGET_BIT                 E   all             MFC
50) MPEG4_VOP_TIME_RES                  E   MPEG4           MFC
Used to compute vop_time_increment and modulo_time_base in MPEG4.
51) MPEG4_VOP_FRAME_DELTA               E   MPEG4           MFC
Used to compute vop_time_increment and modulo_time_base in MPEG4.
52) H264_OPEN_GOP                       E   H264            common
Enable open GOP in H264.
53) H264_I_PERIOD                       E   H264            common
Period between I frames in open GOP for H264.
54) H264_AR_VUI_ENABLE                  E   H264            MFC
Enable writing aspect ratio in VUI.
55) HEADER_MODE                         E   all             common
Determines whether the header is returned as the first buffer or is
it returned together with the first frame.
56) FORCE_FRAME_TYPE                    E   all             MFC
Force frame type on the encoder - either I-frame or skipped.
I hope that buffer flags could be used instead of this.
57) FRAME_TAG                           E   all             MFC
Frame tag is assigned to an input buffer passed to hardware, and
the same frame tag is then assigned to the buffer that contains the
result of processing that frame.

Best regards,
--
Kamil Debski
Linux Platform Group
Samsung Poland R&D Center