{"id":21105,"date":"2024-03-20T12:40:15","date_gmt":"2024-03-20T19:40:15","guid":{"rendered":"https:\/\/engineering.fb.com\/?p=21105"},"modified":"2024-03-26T09:55:22","modified_gmt":"2024-03-26T16:55:22","slug":"mobile-rtc-video-av1-hd","status":"publish","type":"post","link":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/","title":{"rendered":"Better video for mobile RTC with AV1 and HD"},"content":{"rendered":"<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">We\u2019ve seen significant benefits by adopting the <\/span><a href=\"https:\/\/atscaleconference.com\/av1-improving-rtc-video-quality-at-scale\/\"><span style=\"font-weight: 400;\">AV1 codec for RTC<\/span><\/a><span style=\"font-weight: 400;\">.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Here\u2019s how we are improving the RTC video quality for our apps with tools like the AV1 codec, the challenges we face, and how we mitigate those challenges.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">The last few decades have seen tremendous improvements in mobile phone camera quality as well as video quality for streaming video services. But if we look at real-time communication (RTC) applications, while the video quality also has improved over time, it has always lagged behind that of camera quality.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">When we looked at ways to improve video quality for RTC across our family of apps, AV1 stood out as the best option. Meta has increasingly<\/span><a href=\"https:\/\/engineering.fb.com\/2018\/04\/24\/video-engineering\/facebook-video-adds-av1-support\/\"> <span style=\"font-weight: 400;\">adopted the AV1 codec<\/span><\/a><span style=\"font-weight: 400;\"> over the years because it offers high video quality at bitrates much lower than older codecs. But, <\/span><a href=\"https:\/\/atscaleconference.com\/av1-improving-rtc-video-quality-at-scale\/\"><span style=\"font-weight: 400;\">as we\u2019ve implemented AV1 for mobile RTC<\/span><\/a><span style=\"font-weight: 400;\">, we\u2019ve also had to address a number of challenges including scaling,\u00a0 improving video quality for low-bandwidth users as well as high-end networks, CPU and battery usage, and maintaining quality stability.<\/span><\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/zWvteeEkjJg?si=GjUfwYM0dcXgAZHz\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h2><span style=\"font-weight: 400;\">Improving video quality for low-bandwidth networks<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">This post is going to focus on peer-to-peer (P2P, or 1:1) calls, which involve two participants.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">People who use our products and services experience a range of network conditions \u2013 some have really great networks, while others are using throttled or low-bandwidth networks.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This chart illustrates what the distribution of bandwidth looks like for some of these calls on Messenger:<\/span><\/p>\n<figure id=\"attachment_21106\" aria-describedby=\"caption-attachment-21106\" style=\"width: 1024px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21106\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?w=1024\" alt=\"\" width=\"1024\" height=\"611\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png 1600w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=916,546 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=768,458 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=1024,611 1024w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=1536,916 1536w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=96,57 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-1.png?resize=192,114 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21106\" class=\"wp-caption-text\">Figure 1: Bandwidth distribution of P2P calls on Messenger.<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">As seen in Figure 1, some calls operate in very low-bandwidth conditions.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We consider anything less than 300 Kbps to be a low-end network, but we also see a lot of video calls operating at just 50 Kbps, or even under 25 Kbps.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Note that this bandwidth is the share for the video encoder. Total bandwidth is shared with audio, RTP overhead, signaling overhead, RTX (re-transmissions of packets to handle lost packets)\/FEC (forward error correction)\/duplication (packet duplication), and so on. The big assumption here is that the bandwidth estimator is working correctly and estimating true bitrates.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">There are no universal definitions for low, mid, and high networks, but for the purpose of this blog post, less than 300 Kbps will be considered as low, 300-800 Kbps as mid, and above 800 Kbps as a high, HD-capable, or high-end network.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">When we looked into improving the video quality for low-bandwidth users, there were few key options. Migrating to a newer codec such as AV1 presented the greatest opportunity. Other options such as better video scalers and region-of-interest encoding offered incremental improvements.\u00a0<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Video scalers<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We use WebRTC in most of our apps, but the video scalers shipped with WebRTC don\u2019t have the best quality video scaling. We have been able to improve the video scaling quality significantly by leveraging in-house scalers.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">At low bitrates, we often end up downscaling the video to encode at \u00bc resolution (assuming the camera capture is 640&#215;480 or 1280&#215;720). With our custom scaler implementations, we have seen significant improvements in video quality. From public tests we saw gains in peak signal-to-noise ratio (PSNR) by 0.75 db on average.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Here is a snapshot showing results with the default<\/span><a href=\"https:\/\/chromium.googlesource.com\/libyuv\/libyuv\/\"> <span style=\"font-weight: 400;\">libyuv<\/span><\/a><span style=\"font-weight: 400;\"> scaler (a box filter):<\/span><\/p>\n<figure id=\"attachment_21107\" aria-describedby=\"caption-attachment-21107\" style=\"width: 866px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21107\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2a.png?w=866\" alt=\"\" width=\"866\" height=\"290\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2a.png 866w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2a.png?resize=768,257 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2a.png?resize=96,32 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2a.png?resize=192,64 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21107\" class=\"wp-caption-text\">Figure 2.a: Video image results using WebRTC\/libyuv video scaler.<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">And the results after downscaling with our video scaler:<\/span><\/p>\n<figure id=\"attachment_21108\" aria-describedby=\"caption-attachment-21108\" style=\"width: 866px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21108\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2b.png?w=866\" alt=\"\" width=\"866\" height=\"290\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2b.png 866w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2b.png?resize=768,257 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2b.png?resize=96,32 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-2b.png?resize=192,64 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21108\" class=\"wp-caption-text\">Figure 2.b: Video image results using Meta\u2019s video scaler.<\/figcaption><\/figure>\n<h3><span style=\"font-weight: 400;\">Region-of-interest encoding<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Identifying the region of interest (ROI) allowed us to optimize by spending more encoder bitrate in the area that\u2019s most important to a viewer (the speaker\u2019s face in a talking head video, for example). Most mobile devices have APIs to locate the face region without utilizing any CPU overhead. Once we have found the face region we can configure the encoder to spend more bits on this important region and less on the rest. The easiest way to do this was to have some APIs on encoders to configure the quantization parameters (QP) for ROI versus the rest of the image. These changes provided incremental improvements in the video quality metrics like PSNR.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Adopting the AV1 video codec<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">The video encoder is a key element when it comes to video quality for RTC. H.264 has been the most popular codec over the last decade, with hardware support and most applications supporting it. But it is a 20-year-old codec. Back in 2018, the Alliance for Open Media (AOMedia) standardized the AV1 video codec. Since then, several companies including Meta, YouTube, and Netflix have <\/span><a href=\"https:\/\/atscaleconference.com\/av1-improving-rtc-video-quality-at-scale\/\"><span style=\"font-weight: 400;\">deployed it at a large scale for video streaming<\/span><\/a><span style=\"font-weight: 400;\">.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">At Meta, moving from H.264 to AV1 led us to our greatest improvements in video quality at low bitrates.<\/span><\/p>\n<figure id=\"attachment_21109\" aria-describedby=\"caption-attachment-21109\" style=\"width: 1024px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21109\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?w=1024\" alt=\"\" width=\"1024\" height=\"427\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png 1600w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=916,382 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=768,320 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=1024,427 1024w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=1536,640 1536w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=96,40 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-3.png?resize=192,80 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21109\" class=\"wp-caption-text\">Figure 3: Improvements over time, moving from H.262 to AV1 and H.266<\/figcaption><\/figure>\n<h3><span style=\"font-weight: 400;\">Why AV1?<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We chose to use AV1 in part because it\u2019s royalty-free. Codec licensing (and concurrent fees) was an important aspect in our decision-making process. Typically, if an application uses a device\u2019s hardware codec, no additional codec licensing costs will be incurred. But if an application is shipping a software version of the codec,\u00a0 there will most likely be licensing costs to cover.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">But why do we need to use software codecs even though most phones have hardware-supported codecs?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Most mobile devices have dedicated hardware for video encoding and decoding. And these days most mobile devices support H.264 and even H.265s. But those encoders are designed for common use cases such as camera capture, which uses much higher resolutions, frame rates, and bitrates. Most mobile device hardware is currently capable of encoding 4K 60 FPS in real time with very low battery usage, but the results of encoding a 7 FPS, 320&#215;180, 200 Kbps video are often worse than those of software encoders running on the same mobile device.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The reason for that is prioritization of the RTC use case. Most independent hardware vendors (IHVs) are not aware of the network conditions where RTC calls operate; hence, these hardware codecs are not optimized for RTC scenarios, especially for low bitrates, resolutions, and frame rates. So, we leverage software encoders when operating in these low bitrates to provide high-quality video.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">And since we can\u2019t ship software codecs without a license, AV1 is a very good option for RTC.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">AV1 for RTC<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">The biggest reason to move to a more advanced video codec is simple: The same quality experience can be delivered with a much lower bitrate, and we can deliver a much higher-quality real-time calling experience for our users who are on bandwidth-constrained networks.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Measuring video quality is a complex topic, but a relatively simple way to look at it is to use the <\/span><a href=\"https:\/\/arxiv.org\/pdf\/2401.04039.pdf\"><span style=\"font-weight: 400;\">Bjontegaard Delta-Bit Rate<\/span><\/a><span style=\"font-weight: 400;\"> (BD-BR) metric. BD-BR compares how much bitrate various codecs need to produce a certain quality level. By generating multiple samples at different bitrates, measuring the quality of the produced video provides a rate-distortion (RD) curve, and from the RD curve you can derive the BD-BR (as shown below).<\/span><\/p>\n<p><span style=\"font-weight: 400;\">As can be seen in Figure 4, AV1 provided higher quality for all bitrate ranges in our local tests.<\/span><\/p>\n<figure id=\"attachment_21110\" aria-describedby=\"caption-attachment-21110\" style=\"width: 1024px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21110\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?w=1024\" alt=\"\" width=\"1024\" height=\"650\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png 1282w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?resize=916,582 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?resize=768,488 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?resize=1024,650 1024w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?resize=96,61 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-4.png?resize=192,122 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21110\" class=\"wp-caption-text\">Figure 4: Bitrate distortion comparison chart.<\/figcaption><\/figure>\n<h3><span style=\"font-weight: 400;\">Screen-encoding tools<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">AV1 also has a few key tools that are useful for RTC. Screen content quality is becoming an increasingly important factor for Meta, with relevant use cases, including screen sharing, game streaming, and VR remote desktop, requiring high-quality encoding. In these areas, AV1 truly shines.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Traditionally, video encoders aren\u2019t well suited to complex content such as text with a lot of high-frequency content, and humans are sensitive to reading blurry text. AV1 has a set of coding tools\u2014palette mode and intra-block copy\u2014that drastically improve performance for screen content. Palette mode is designed according to the observation that the pixel values in a screen-content frame usually concentrate on the limited number of color values. Palette mode can represent the screen content efficiently by signaling the color clusters instead of the quantized transform-domain coefficients. In addition, for typical screen content, repetitive patterns can usually be found within the same picture. Intra-block copy facilitates block prediction within the same frame, so that the compression efficiency can be improved significantly. That AV1 provides these two tools at the baseline profile is a huge plus.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Reference picture resampling<\/span><span style=\"font-weight: 400;\">: Fewer key frames<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Another useful feature is reference picture resampling (RPR), which allows resolution changes without generating a key frame. In video compression, a key frame is one that\u2019s encoded independently, like a still image. It\u2019s the only type of frame that can be decoded without having another frame as reference.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For RTC applications, since the bandwidth keeps on changing often, there are frequent resolution changes needed to adapt to these network changes. With older codecs like H.264, each of these resolution changes requires a key frame that is much larger in size and thus inefficient for RTC apps. Such large key frames increase the amount of data needing to be sent over the network and result in higher end-to-end latencies and congestion.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">By using RPR, we can avoid generating any key frames.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-21111\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-5.png?w=788\" alt=\"\" width=\"788\" height=\"242\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-5.png 788w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-5.png?resize=768,236 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-5.png?resize=96,29 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-5.png?resize=192,59 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><\/p>\n<h2><span style=\"font-weight: 400;\">Challenges around improving video quality for low-bandwidth users<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">CPU\/battery usage<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">AV1 is great for coding efficiency, but codecs achieve this at the cost of higher CPU and battery usage. A lot of modern codecs pose these challenges when running real-time applications on mobile platforms.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Based on local lab testing, we anticipated a roughly 4 percent increase in battery usage, and we saw similar results in public tests. We used a power meter to do this local battery measurement.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Even though the AV1 encoder itself increased CPU usage three-fold when compared to H.264 implementation, the overall contribution of CPU usage from the encoder was a small part of the battery usage. The phone display screen, networking\/radio, and other processes using the CPU contribute significantly to battery usage, hence the increase in battery usage was 5-6 percent (a significant increase in battery usage).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">A lot of calls run out of device battery, or people hang up once their operating system indicates a low battery, so increasing battery usage isn\u2019t worthwhile for users unless it provides increased value such as video quality improvement. Even then it\u2019s a trade-off between video quality versus battery use.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We use WebRTC and Session Description Protocol (SDP) for codec negotiation, which allows us to negotiate multiple codecs (e.g., AV1 and H.264) up front and then switch the codecs without any need for signaling or a handshake during the call. This means the codec switch is seamless, without users noticing any glitches or pauses in video.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We created a custom encoder that encapsulates both H.264 and the AV1 encoders. We call it a hybrid encoder. This allowed us to switch the codec during the call based on triggers such as CPU usage, battery level, or encoding time \u2014 and to switch to the more battery-efficient H.264 encoder when needed.\u00a0<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Increased crashes and out of memory errors<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Even without new leaks added, AV1 used more memory than H.264. Any time additional memory is used, apps are more likely to hit out of memory (OOM) crashes or hit OOM sooner because of other leaks or memory demands on the system from other apps. To mitigate this, we had to disable AV1 on devices with low memory. This is one area for improvement and for further optimizing the encoder\u2019s memory usage.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">In-product quality measurement<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">To compare the quality between H.264 and AV1 using public tests, we needed a low-complexity metric. Metrics such as encoded bitrates and frame rates won\u2019t show any gains as the total bandwidth available to send video is still the same, as these are limited by the network capacity, which means the bitrates and frame rates for video will not change much with the change in the codec. We had been using composite metrics that combine the quantization parameter (QP is often used as a proxy for video quality, as this introduces pixel data loss during the encoding process), resolutions, and frame rate, and freezes it to generate video composite metrics, but QP is not comparable between AV1 and H.264 codecs, and hence can\u2019t be used.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">PSNR is a standard metric, but it\u2019s reference-based and hence does not work for RTC. Non-reference, video-quality metrics are quite CPU-intensive (e.g., BRISQUE: Blind\/Referenceless Image Spatial Quality Evaluator), though we are exploring those as well.<\/span><\/p>\n<figure id=\"attachment_21112\" aria-describedby=\"caption-attachment-21112\" style=\"width: 1024px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21112\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?w=1024\" alt=\"\" width=\"1024\" height=\"635\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png 1220w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?resize=916,568 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?resize=768,476 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?resize=1024,635 1024w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?resize=96,59 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-6.png?resize=192,119 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21112\" class=\"wp-caption-text\">Figure 6: High-level architecture for PSNR computation in RTC.<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">We have come up with a framework for PSNR computation. We first modified the encoder to report distortions caused by compression (most software encoders already have support for this metric). Then we designed a lightweight, scaling-distortion algorithm that estimates the distortion introduced by video scaling. This algorithm can combine these scaling distortions with the encoder distortions to produce output PSNR. We developed and verified this algorithm locally and will be sharing the findings in publications and at academic conferences over the next year. With this lightweight PSNR metric, we saw 2 db improvements with AV1 compared to H.264.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Challenges around improving video quality for high-end networks<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">As a quick review: For our purposes, high bandwidth covers users for whom bandwidth is greater than 800 kbps.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Over the years, there have been huge improvements in camera capture quality. As a result, people\u2019s expectations have gone up, and they want to see RTC video quality on par with local camera capture quality.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Based on local testing, we settled on settings resulting in video quality that looks similar to that of camera recordings. We call this HD mode. We found that with a video codec like H.264 encoding at 3.5 Mbps and 30 frames per second, 720p resolution looked very similar to local camera recordings. We also compared 720p to 1080p in subjective quality tests and found that the difference is not noticeable on most devices except for those with a larger screen when we conducted subjective quality tests.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Bandwidth estimator improvements<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Improving the video quality for users who have high-end phones with good CPUs, good batteries, hardware codecs, and good network speeds seems trivial. It may seem like all you have to do is increase the maximum bitrate, capture resolution, and capture frame rates, and users will send high-quality video. But, in reality, it\u2019s not that simple.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">If you increase the bitrate, you expose your bandwidth estimation and congestion detection algorithm to hit congestion more often, and your algorithm will be tested many more times than if you were not using these higher bitrates.\u00a0<\/span><\/p>\n<figure id=\"attachment_21113\" aria-describedby=\"caption-attachment-21113\" style=\"width: 1024px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21113\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?w=1024\" alt=\"\" width=\"1024\" height=\"395\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png 1650w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=916,354 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=768,296 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=1024,395 1024w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=1536,593 1536w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=96,37 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-7.png?resize=192,74 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21113\" class=\"wp-caption-text\">Figure 7: Example showing how using higher bandwidth increases the instances for congestion.<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">If you look at the network pipeline in Figure 7, the higher the bitrates you are using, the more your algorithm\/code will be tested for robustness over the time of the RTC call. Figure 7 shows how using 1 Mbps hits more congestion than using 500 Kbps and using 3 Mbps hits more congestion than 1 Mbps, and so on. If you are using bandwidths lower than the minimum throughput of the network, however, you won\u2019t hit congestion at all. For example, see the 500-Kbps call in Figure 7.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To mitigate these issues, we improved congestion detection. For example, we added custom ISP throttling detection, something that was not being caught by the traditional delay-based estimator of WebRTC.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Bandwidth estimator and network resilience comprise a complex area on their own, and this is where RTC products stand out. They have their own custom algorithms that work best for their products and customers.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Stable quality<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">People don\u2019t like oscillations in video quality. These can happen when we send high-quality video for a few seconds and then drop back to low-quality because of congestion. Learning from past history, we added support in\u00a0 bandwidth estimation to prevent these oscillations.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Audio is more important than video for RTC<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">When network congestion occurs, all media packets could be lost. This causes video freezes and broken audio, (aka, robotic audio). For RTC, both are bad, but audio quality is more important than video.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Broken audio often completely prevents conversations from happening, often causing people to hang up or redial the call. Broken video, on the other hand, often results in less delightful conversations, but, depending on the scenario, it could also be a block for some users.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">At high bitrates like 2.5 Mbps and higher, you can afford to have three to five times more audio packets or duplication without any noticeable degradation to video. When operating in these higher bitrates with cell phone connections, we saw more of these congestion, packet loss, and ISP throttling issues, so we had to make changes to our network resiliency algorithms. And since people are highly sensitive to data usage on their cell phones, we disabled high bitrates on cellular connections.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">When to enable HD?<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We used ML-based targeting to guess which call should be HD-capable. We relied on the network stats from the users\u2019 previous calls to predict if HD should be enabled or not.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Battery regressions<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">We have lots of metrics, including performance, networking, and media quality, to track the quality of RTC calls. When we ran tests for HD, we noticed regressions in battery metrics. What we found was that most battery regressions do not come from higher bitrates or resolution but from the capture frame rates.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To mitigate the regressions, we built a mechanism for detecting both caller and callee device capabilities, including device model, battery levels, Wi-Fi or mobile usage, and so on. To enable high-quality modes, we check both sides of the call to ensure that they satisfy the requirements and only then do we enable these high-quality, resource-intensive configurations.<\/span><\/p>\n<figure id=\"attachment_21114\" aria-describedby=\"caption-attachment-21114\" style=\"width: 972px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-21114\" src=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png?w=972\" alt=\"\" width=\"972\" height=\"608\" srcset=\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png 972w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png?resize=916,573 916w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png?resize=768,480 768w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png?resize=96,60 96w, https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video_Figure-8.png?resize=192,120 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" \/><figcaption id=\"caption-attachment-21114\" class=\"wp-caption-text\">Figure 8: Signaling server setup for turning HD on or off.<\/figcaption><\/figure>\n<h2><span style=\"font-weight: 400;\">What the future holds for RTC<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Hardware manufacturers are acknowledging the significant benefits of using AV1 for RTC. The new Apple iPhone 15 Pro supports AV1\u2019s hardware decoder, and the Google Pixel 8 supports AV1 encoding and decoding. Hardware codecs are an absolute necessity for high-end network and HD resolutions. Video calling is becoming as ubiquitous as traditional audio calling and we hope that as hardware manufacturers recognize this shift, there will be more opportunities for collaboration between RTC app creators and hardware manufacturers to optimize encoders for these scenarios.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">On the software side, we will continue to work on optimizing AV1 software encoders and developing new encoder implementations. We try to provide the best experience for our users, but at the same time we want to let people have full control over their RTC experience. We will provide controls to the users so that they can choose whether they want higher quality at the cost of battery and data usage, or vice versa.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We also plan to work with IHVs to collaborate on hardware codec development to make these codecs usable for RTC scenarios including low-bandwidth use cases.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We also will investigate forward-looking features such as video processing to increase the resolution and frame rates on the receiver\u2019s rendering stack and leveraging AI\/ML to improve bandwidth estimation (BWE) and network resiliency.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Further, we\u2019re investigating<\/span><a href=\"https:\/\/research.facebook.com\/publications\/pixel-codec-avatars\/\"> <span style=\"font-weight: 400;\">Pixel Codec Avatar<\/span><\/a><span style=\"font-weight: 400;\"> technologies that will allow us to transmit the model\/share once and then send the geometry\/vectors for receiver side rendering. This enables video rendering with much smaller bandwidth usage than traditional video codecs for RTC scenarios. <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp. We\u2019ve seen significant benefits by adopting the AV1 codec for RTC. Here\u2019s how we are improving the RTC video quality for our apps with tools like the AV1 codec, the challenges we face, and how we [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":51,"featured_media":21140,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[74],"tags":[],"class_list":["post-21105","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-video-engineering","fb_content_type-article"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v19.3 (Yoast SEO v19.12) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Better video for mobile RTC with AV1 and HD - Engineering at Meta<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Shyam Sadhwani\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\"},\"author\":{\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#author\",\"name\":\"\"},\"headline\":\"Better video for mobile RTC with AV1 and HD\",\"datePublished\":\"2024-03-20T19:40:15+00:00\",\"dateModified\":\"2024-03-26T16:55:22+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\"},\"wordCount\":3268,\"publisher\":{\"@id\":\"https:\/\/engineering.fb.com\/#organization\"},\"articleSection\":[\"Video Engineering\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\",\"url\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\",\"name\":\"Better video for mobile RTC with AV1 and HD - Engineering at Meta\",\"isPartOf\":{\"@id\":\"https:\/\/engineering.fb.com\/#website\"},\"datePublished\":\"2024-03-20T19:40:15+00:00\",\"dateModified\":\"2024-03-26T16:55:22+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/engineering.fb.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Better video for mobile RTC with AV1 and HD\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/engineering.fb.com\/#website\",\"url\":\"https:\/\/engineering.fb.com\/\",\"name\":\"Engineering at Meta\",\"description\":\"Engineering at Meta Blog\",\"publisher\":{\"@id\":\"https:\/\/engineering.fb.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/engineering.fb.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/engineering.fb.com\/#organization\",\"name\":\"Meta\",\"url\":\"https:\/\/engineering.fb.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg\",\"contentUrl\":\"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg\",\"width\":29011,\"height\":12501,\"caption\":\"Meta\"},\"image\":{\"@id\":\"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Engineering\/\",\"https:\/\/twitter.com\/fb_engineering\"]},[]]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Better video for mobile RTC with AV1 and HD - Engineering at Meta","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/","twitter_misc":{"Written by":"Shyam Sadhwani","Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#article","isPartOf":{"@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/"},"author":{"@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#author","name":""},"headline":"Better video for mobile RTC with AV1 and HD","datePublished":"2024-03-20T19:40:15+00:00","dateModified":"2024-03-26T16:55:22+00:00","mainEntityOfPage":{"@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/"},"wordCount":3268,"publisher":{"@id":"https:\/\/engineering.fb.com\/#organization"},"articleSection":["Video Engineering"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/","url":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/","name":"Better video for mobile RTC with AV1 and HD - Engineering at Meta","isPartOf":{"@id":"https:\/\/engineering.fb.com\/#website"},"datePublished":"2024-03-20T19:40:15+00:00","dateModified":"2024-03-26T16:55:22+00:00","breadcrumb":{"@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.fb.com\/2024\/03\/20\/video-engineering\/mobile-rtc-video-av1-hd\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.fb.com\/"},{"@type":"ListItem","position":2,"name":"Better video for mobile RTC with AV1 and HD"}]},{"@type":"WebSite","@id":"https:\/\/engineering.fb.com\/#website","url":"https:\/\/engineering.fb.com\/","name":"Engineering at Meta","description":"Engineering at Meta Blog","publisher":{"@id":"https:\/\/engineering.fb.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.fb.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/engineering.fb.com\/#organization","name":"Meta","url":"https:\/\/engineering.fb.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/","url":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg","contentUrl":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg","width":29011,"height":12501,"caption":"Meta"},"image":{"@id":"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Engineering\/","https:\/\/twitter.com\/fb_engineering"]},[]]}},"jetpack_featured_media_url":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2024\/03\/RTC-Video-hero-B.png","jetpack_shortlink":"https:\/\/wp.me\/pa0Lhq-5up","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/21105","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/users\/51"}],"replies":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/comments?post=21105"}],"version-history":[{"count":3,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/21105\/revisions"}],"predecessor-version":[{"id":21172,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/21105\/revisions\/21172"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/media\/21140"}],"wp:attachment":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/media?parent=21105"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/categories?post=21105"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/tags?post=21105"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}