FEC and RTX

FEC 的 Pattern定义为

[m,k],n=m+k,m,kFEC,n

n个包为一个Group, 在这个Group中, 任意丢k个包, 都可以通过收到的m个包恢复. 这里的k个包可以是数据包或者FEC包.
所以在给定丢包率 l, 给定源数据包的个数 m, 给定源数据丢失的概率, 可以推算出n. 不过在有重传的网络中, 如何来推算 n呢?
为了简单起见, 假定FEC只存在于数据包的发送, 重传包不参与FEC过程.

在给定丢包率l的网络, 单个数据包被收到的概率

ϱ=1l

n个包,都收到的概率为:
ϱ=(1l)n

n个包, 收到m个包的概率为:
ϱ=Cmn(1l)ml(nm)

根据FEC Pattern, 只要收到m个或者更多的包, 源数据可以被恢复, 则恢复的概率为:
ϱ=i=mnCin(1l)il(ni)

当传输层存在重传机制时, 数据包不能恢复的时候, 还可以通过重传来恢复, 假定传输层的RTT为r, 通信的最大时延要求为d, 则可以大致推测出可以容忍的最大重传次数x, 重传只针对数据包, FEC包不做重传. 于是, 数据包丢失的概率变为l(x+1), FEC包丢失的概率依然是l
数据源可恢复的概率为:
ϱ=i+j>=m(Cim(1l(x+1))il(x+1)(mi))×(Cjk(1l)jl(kj)))

给定数据恢复的最低概率, 就可以计算出最小n, 从而推算出k, 也就是FEC的最低冗余度.
<!-- go/cmark --> <!--* freshness: {owner: 'sprang' reviewed: '2021-04-12'} *--> # Paced Sending The paced sender, often referred to as just the "pacer", is a part of the WebRTC RTP stack used primarily to smooth the flow of packets sent onto the network. ## Background Consider a video stream at 5Mbps and 30fps. This would in an ideal world result in each frame being ~21kB large and packetized into 18 RTP packets. While the average bitrate over say a one second sliding window would be a correct 5Mbps, on a shorter time scale it can be seen as a burst of 167Mbps every 33ms, each followed by a 32ms silent period. Further, it is quite common that video encoders overshoot the target frame size in case of sudden movement especially dealing with screensharing. Frames being 10x or even 100x larger than the ideal size is an all too real scenario. These packet bursts can cause several issues, such as congesting networks and causing buffer bloat or even packet loss. Most sessions have more than one media stream, e.g. a video and an audio track. If you put a frame on the wire in one go, and those packets take 100ms to reach the other side - that means you have now blocked any audio packets from reaching the remote end in time as well. The paced sender solves this by having a buffer in which media is queued, and then using a _leaky bucket_ algorithm to pace them onto the network. The buffer contains separate fifo streams for all media tracks so that e.g. audio can be prioritized over video - and equal prio streams can be sent in a round-robin fashion to avoid any one stream blocking others. Since the pacer is in control of the bitrate sent on the wire, it is also used to generate padding in cases where a minimum send rate is required - and to generate packet trains if bitrate probing is used. ## Life of a Packet The typical path for media packets when using the paced sender looks something like this: 1. `RTPSenderVideo` or `RTPSenderAudio` packetizes media into RTP packets. 2. The packets are sent to the [RTPSender] class for transmission. 3. The pacer is called via [RtpPacketSender] interface to enqueue the packet batch. 4. The packets are put into a queue within the pacer awaiting opportune moments to send them. 5. At a calculated time, the pacer calls the `PacingController::PacketSender()` callback method, normally implemented by the [PacketRouter] class. 6. The router forwards the packet to the correct RTP module based on the packet's SSRC, and in which the `RTPSenderEgress` class makes final time stamping, potentially records it for retransmissions etc. 7. The packet is sent to the low-level `Transport` interface, after which it is now out of scope. Asynchronously to this, the estimated available send bandwidth is determined - and the target send rate is set on the `RtpPacketPacer` via the `void SetPacingRates(DataRate pacing_rate, DataRate padding_rate)` method. ## Packet Prioritization The pacer prioritized packets based on two criteria: * Packet type, with most to least prioritized: 1. Audio 2. Retransmissions 3. Video and FEC 4. Padding * Enqueue order The enqueue order is enforced on a per stream (SSRC) basis. Given equal priority, the [RoundRobinPacketQueue] alternates between media streams to ensure no stream needlessly blocks others. ## Implementations The main class to use is called [TaskQueuePacedSender]. It uses a task queue to manage thread safety and schedule delayed tasks, but delegates most of the actual work to the `PacingController` class. This way, it's possible to develop a custom pacer with different scheduling mechanism - but ratain the same pacing logic. ## The Packet Router An adjacent component called [PacketRouter] is used to route packets coming out of the pacer and into the correct RTP module. It has the following functions: * The `SendPacket` method looks up an RTP module with an SSRC corresponding to the packet for further routing to the network. * If send-side bandwidth estimation is used, it populates the transport-wide sequence number extension. * Generate padding. Modules supporting payload-based padding are prioritized, with the last module to have sent media always being the first choice. * Returns any generated FEC after having sent media. * Forwards REMB and/or TransportFeedback messages to suitable RTP modules. At present the FEC is generated on a per SSRC basis, so is always returned from an RTP module after sending media. Hopefully one day we will support covering multiple streams with a single FlexFEC stream - and the packet router is the likely place for that FEC generator to live. It may even be used for FEC padding as an alternative to RTX. ## The API The section outlines the classes and methods relevant to a few different use cases of the pacer. ### Packet sending For sending packets, use `RtpPacketSender::EnqueuePackets(std::vector<std::unique_ptr<RtpPacketToSend>> packets)` The pacer takes a `PacingController::PacketSender` as constructor argument, this callback is used when it's time to actually send packets. ### Send rates To control the send rate, use `void SetPacingRates(DataRate pacing_rate, DataRate padding_rate)` If the packet queue becomes empty and the send rate drops below `padding_rate`, the pacer will request padding packets from the `PacketRouter`. In order to completely suspend/resume sending data (e.g. due to network availability), use the `Pause()` and `Resume()` methods. The specified pacing rate may be overriden in some cases, e.g. due to extreme encoder overshoot. Use `void SetQueueTimeLimit(TimeDelta limit)` to specify the longest time you want packets to spend waiting in the pacer queue (pausing excluded). The actual send rate may then be increased past the pacing_rate to try to make the _average_ queue time less than that requested limit. The rationale for this is that if the send queue is say longer than three seconds, it's better to risk packet loss and then try to recover using a key-frame rather than cause severe delays. ### Bandwidth estimation If the bandwidth estimator supports bandwidth probing, it may request a cluster of packets to be sent at a specified rate in order to gauge if this causes increased delay/loss on the network. Use the `void CreateProbeCluster(...)` method - packets sent via this `PacketRouter` will be marked with the corresponding cluster_id in the attached `PacedPacketInfo` struct. If congestion window pushback is used, the state can be updated using `SetCongestionWindow()` and `UpdateOutstandingData()`. A few more methods control how we pace: * `SetAccountForAudioPackets()` determines if audio packets count into bandwidth consumed. * `SetIncludeOverhead()` determines if the entire RTP packet size counts into bandwidth used (otherwise just media payload). * `SetTransportOverhead()` sets an additional data size consumed per packet, representing e.g. UDP/IP headers. ### Stats Several methods are used to gather statistics in pacer state: * `OldestPacketWaitTime()` time since the oldest packet in the queue was added. * `QueueSizeData()` total bytes currently in the queue. * `FirstSentPacketTime()` absolute time the first packet was sent. * `ExpectedQueueTime()` total bytes in the queue divided by the send rate. [RTPSender]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/rtp_rtcp/source/rtp_sender.h;drc=77ee8542dd35d5143b5788ddf47fb7cdb96eb08e [RtpPacketSender]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/rtp_packet_sender.h;drc=ea55b0872f14faab23a4e5dbcb6956369c8ed5dc [RtpPacketPacer]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/pacing/rtp_packet_pacer.h;drc=e7bc3a347760023dd4840cf6ebdd1e6c8592f4d7 [PacketRouter]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/pacing/packet_router.h;drc=3d2210876e31d0bb5c7de88b27fd02ceb1f4e03e [TaskQueuePacedSender]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/pacing/task_queue_paced_sender.h;drc=5051693ada61bc7b78855c6fb3fa87a0394fa813 [RoundRobinPacketQueue]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/pacing/round_robin_packet_queue.h;drc=b571ff48f8fe07678da5a854cd6c3f5dde02855f 翻译
最新发布
12-03
RTP_PAYLOAD_PCMU = 0, // ITU-T G.711 PCM µ-Law audio 64 kbit/s (rfc3551) RTP_PAYLOAD_G723 = 4, // ITU-T G.723.1 8000/1, 30ms (rfc3551) RTP_PAYLOAD_PCMA = 8, // ITU-T G.711 PCM A-Law audio 64 kbit/s (rfc3551) RTP_PAYLOAD_G722 = 9, // ITU-T G.722 audio 64 kbit/s (rfc3551) RTP_PAYLOAD_CN = 13, // Real-time Transport Protocol (RTP) Payload for Comfort Noise (CN) (rfc3389) RTP_PAYLOAD_MP3 = 14, // MPEG-1/MPEG-2 audio (rfc2250) RTP_PAYLOAD_G729 = 18, // ITU-T G.729 and G.729a audio 8 kbit/s (rfc3551) RTP_PAYLOAD_SVACA = 20, // GB28181-2016 RTP_PAYLOAD_JPEG = 26, // JPEG video (rfc2435) RTP_PAYLOAD_MPV = 32, // MPEG-1 and MPEG-2 video (rfc2250) RTP_PAYLOAD_MP2T = 33, // MPEG-2 transport stream (rfc2250) RTP_PAYLOAD_H263 = 34, // H.263 video, first version (1996) (rfc2190) RTP_PAYLOAD_AV1X = 35, // https://bugs.chromium.org/p/webrtc/issues/detail?id=11042 RTP_PAYLOAD_MP2P = 100, // MPEG-2 Program Streams video (rfc2250) RTP_PAYLOAD_MP4V = 97, // MP4V-ES MPEG-4 Visual (rfc6416) RTP_PAYLOAD_H264 = 96, // H.264 video (MPEG-4 Part 10) (rfc6184) RTP_PAYLOAD_SVAC = 99, // GB28181-2016 RTP_PAYLOAD_H265 = 98, // H.265 video (MPEG-H Part 2) (rfc7798) RTP_PAYLOAD_MP4A = 101, // MPEG4-generic audio/video MPEG-4 Elementary Streams (rfc3640) RTP_PAYLOAD_LATM = 102, // MP4A-LATM MPEG-4 Audio (rfc6416) RTP_PAYLOAD_OPUS = 103, // RTP Payload Format for the Opus Speech and Audio Codec (rfc7587) RTP_PAYLOAD_MP4ES = 104, // MPEG4-generic audio/video MPEG-4 Elementary Streams (rfc3640) RTP_PAYLOAD_VP8 = 105, // RTP Payload Format for VP8 Video (rfc7741) RTP_PAYLOAD_VP9 = 106, // RTP Payload Format for VP9 Video draft-ietf-payload-vp9-03 RTP_PAYLOAD_AV1 = 107, // https://aomediacodec.github.io/av1-rtp-spec/ RTP_PAYLOAD_H266 = 108, // https://www.ietf.org/archive/id/draft-ietf-avtcore-rtp-vvc-18.html RTP_PAYLOAD_RTX = 110, // RTP Retransmission Payload Format (rfc4588) RTP_PAYLOAD_RED = 111, // RTP Payload for Redundant Audio Data (rfc2198) RTP_PAYLOAD_FEC_ULP = 112, // RTP Payload Format for Generic Forward Error Correction (rfc5109) RTP_PAYLOAD_FEC_FLEX = 113, // RTP Payload Format for Flexible Forward Error Correction (rfc8267) RTP_PAYLOAD_FEC_RS = 114, // RTP Payload Format for Reed-Solomon(non-standard/private)如果要打包xml成rtp应该选哪个
10-15
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值