视频自适应

Video Adaptation

视频自适应

Video adaptation is a mechanism which reduces the bandwidth or CPU consumption by reducing encoded video quality.

视频自适应是一种通过降低编码视频质量来减少带宽或CPU消耗的机制。

Overview

概述

Adaptation occurs when a Resource signals that it is currently underused or overused. When overused, the video quality is decreased and when underused, the video quality is increased. There are currently two dimensions in which the quality can be adapted: frame-rate and resolution. The dimension that is adapted is based on the degradation preference for the video track.

当资源发出当前未充分使用或过度使用的信号时，就会发生自适应。当过度使用时，视频质量会降低，而当未充分使用时，则视频质量会提高。目前有两个维度可以调整质量：帧速率和分辨率。所调整的维度是基于视频轨道的降级偏好。

Resources

资源

Resources monitor metrics from the system or the video stream. For example, a resource could monitor system temperature or the bandwidth usage of the video stream. A resource implements the Resource interface. When a resource detects that it is overused, it calls SetUsageState(kOveruse). When the resource is no longer overused, it can signal this using SetUsageState(kUnderuse).

​资源监视来自系统或视频流的度量。例如，资源可以监视系统温度或视频流的带宽使用情况。资源实现resource接口。当资源检测到它被过度使用时，它会调用SetUsageState（kOveruse）。当资源不再被过度使用时，它可以使用SetUsageState（kUnderuse）发出信号。

There are two resources that are used by default on all video tracks: Quality scaler resource and encode overuse resource.

默认情况下，在所有视频轨道上使用两种资源：质量缩放器资源和编码过度使用资源。

QP Scaler Resource

QP缩放器资源

The quality scaler resource monitors the quantization parameter (QP) of the encoded video frames for video send stream and ensures that the quality of the stream is acceptable for the current resolution. After each frame is encoded the QualityScaler is given the QP of the encoded frame. Overuse or underuse is signalled when the average QP is outside of the QP thresholds. If the average QP is above the high threshold, the QP scaler signals overuse, and when below the low threshold the QP scaler signals underuse.

​质量缩放器资源监视视频发送流的编码视频帧的量化参数（QP），并确保流的质量对于当前分辨率是可接受的。在对每个帧进行编码之后，QualityScale被赋予编码帧的QP。当平均QP在QP阈值之外时，用信号通知过度使用或未充分使用。如果平均QP高于高阈值，则QP缩放器发出过度使用的信号，而当低于低阈值时，QP缩放器则发出未充分使用的信号。

The thresholds are set by the video encoder in the scaling_settings property of the EncoderInfo.

​阈值由视频编码器在EncoderInfo的scaling_settings属性中设置。

Note: that the QP scaler is only enabled when the degradation preference is MAINTAIN_FRAMERATE or BALANCED.

注意：QP缩放器仅在降级首选项为MAINTAIN_FRAMERATE或BALANCED时启用。

Encode Usage Resource

编码使用情况资源

The encoder usage resource monitors how long it takes to encode a video frame. This works as a good proxy measurement for CPU usage as contention increases when CPU usage is high, increasing the encode times of the video frames.

​编码器使用资源监视对视频帧进行编码所需的时间。当CPU使用率高时，竞争增加，增加了视频帧的编码时间，这可以作为CPU使用率的良好代理测量。

The time is tracked from when frame encoding starts to when it is completed. If the average encoder usage exceeds the thresholds set, overuse is triggered.

跟踪从帧编码开始到完成的时间。如果编码器的平均使用量超过设置的阈值，则会触发过度使用。

Injecting other Resources

注入其他资源

A custom resource can be injected into the call using the Call::AddAdaptationResource method.

​可以使用Call::AddAdaptationResource方法将自定义资源注入到调用中。

Adaptation

自适应

When a a resource signals the it is over or underused, this signal reaches the ResourceAdaptationProcessor who requests an Adaptation proposal from the VideoStreamAdapter. This proposal is based on the degradation preference of the video stream. ResourceAdaptationProcessor will determine if the Adaptation should be applied based on the current adaptation status and the Adaptation proposal.

​当一个资源发出信号表示它已被过度使用或未被充分使用时，该信号到达ResourceAdaptationProcessor，后者向VideoStreamAdapter请求自适应建议。此建议基于视频流的降级偏好。ResourceAdaptationProcessor将根据当前自适应状态和自适应建议来确定是否应应用自适应。

Degradation Preference

降级首选项

There are 3 degradation preferences, described in the RtpParameters header. These are

​RtpParameters头中描述了3个降级首选项。这些是

• MAINTIAIN_FRAMERATE: Adapt video resolution
• MAINTIAIN_FRAMERATE：调整视频分辨率
• MAINTIAIN_RESOLUTION: Adapt video frame-rate.
• MAINTIAIN_RESOLUTION:调整视频帧速率。
• BALANCED: Adapt video frame-rate or resolution.
• BALANCED：调整视频帧速率或分辨率。

The degradation preference is set for a video track using the degradation_preference property in the RtpParameters.

​使用RtpParameters中的degradation_preference属性为视频音轨设置降级首选项。

VideoSinkWants and video stream adaptation

VideoSinkWants和视频流自适应

Once an adaptation is applied it notifies the video stream. The video stream converts this adaptation to a VideoSinkWants. These sink wants indicate to the video stream that some restrictions should be applied to the stream before it is sent to encoding. It has a few properties, but for adaptation the properties that might be set are:

​一旦应用了自适应，它就通知视频流。视频流将此改编转换为VideoSinkWants。这些接收器想要向视频流指示在流被发送到编码之前应该对其应用一些限制。它有一些属性，但为了适应，可能设置的属性为：

• target_pixel_count: The desired number of pixels for each video frame. The actual pixel count should be close to this but does not have to be exact so that aspect ratio can be maintained.
• target_pixel_count：每个视频帧所需的像素数。实际像素计数应该接近此值，但不一定要精确，以便可以保持纵横比。
• max_pixel_count: The maximum number of pixels in each video frame. This value can not be exceeded if set.
• max_pixel_count：每个视频帧中的最大像素数。如果设置了该值，则不能超过该值。
• max_framerate_fps: The maximum frame-rate for the video source. The source is expected to drop frames that cause this threshold to be exceeded.
• max_framerate_fps：视频源的最大帧速率。源将丢弃导致超过此阈值的帧。

The VideoSinkWants can be applied by any video source, or one may use the AdaptedVideoTraceSource which is a base class for sources that need video adaptation.

​VideoSinkWants可以由任何视频源应用，也可以使用AdaptedVideoTraceSource，它是需要视频自适应的源的基类。