Apple’s Worldwide Developer Conference is an annual event used to showcase new software and technologies in the Apple ecosystem. It was created with developers in mind, but sometimes new hardware and devices are announced and its keynote presentations have become must-see events for a much wider audience. There is also usually news about changes and additions to the HTTP Live Streaming (HLS) spec and associated video playback APIs. These HLS updates are often necessary to support new features and capabilities of the announced OS and hardware updates. This post will expand on Apple’s “What’s new in HTTP Live Streaming” document, with additional context for the latest developments that content creators, developers, and streaming services should be aware of.

The lastest HLS updates for 2024

The first draft of the HLS spec (draft-pantos-http-live-streaming) was posted in 2009, then superseded by RFC 8216 in 2017. There are usually draft updates published once or twice per year with significant updates and enhancements. A draft proposal was shared on June 7, that details proposed changes to the spec to be added later this year. Let’s look at some of the highlights below. 

Updated Interstitial attributes

In May 2021, Apple introduced HLS Interstitials to make it easier to create and deliver interstitial content like branding bumpers and mid-roll ads. Now, new attributes have been introduced for Interstitial EXT-X-DATERANGE tags, aimed at enhancing viewer experience and operational flexibility. 

1. X-CONTENT-MAY-VARY: This attribute provides a hint regarding coordinated playback across multiple players. It can be set to “YES” or “NO”, indicating whether all players receive the same interstitial content or not. If X-CONTENT-MAY-VARY is missing, it will be considered to have a value of “YES”.

2. X-TIMELINE-OCCUPIES: Determines if the interstitial should appear as a single point “POINT” or a range “RANGE” on the playback timeline. If X-TIMELINE-OCCUPIES is missing, it will be considered to have a value of “POINT”. “RANGE” is expected to be used for ads in live content.

3. X-TIMELINE-STYLE: Specifies the presentation style of the interstitial—either as a “HIGHLIGHT” separate from the content or as “PRIMARY”, integrated with the main media. If X-TIMELINE-STYLE is missing, it is considered to have a value of “HIGHLIGHT”. The “PRIMARY” value is expected to be used for content like ratings bumpers and post-roll dub cards. 

More detail is available in the WWDC Session “Enhance ad experiences with HLS interstitials“.

Signal enhancements for High Dynamic Range (HDR) and timed metadata

HDR10+

Previously, the specification had not defined how to signal HDR10+ content in a multi-variant HLS playlist. Now you can use the SUPPLEMENTAL-CODECS attribute with the appropriate format, followed by a slash and then the brand (‘cdm4’ for HDR10+). The example Apple provided shows the expected syntax: SUPPLEMENTAL-CODECS=”hvc1.2.20000000.L123.B0/cdm4″. For a long time, HDR10+ was only supported on Samsung and some Panasonic TVs, but in recent years it has been added by other TV brands and dedicated streaming devices like Apple TV 4K and a few Roku models.

Dolby Vision with AV1

Dolby Vision has been the more popular and widespread dynamic HDR format (compared to HDR10+) and now with Apple adding AV1 decoders in their latest generation of processors, they’ve defined how to signal that content within HLS playlists. They are using Dolby Vision Profile 10, which is Dolby’s 10-bit AV1 aware profile. HLS will now support 3 different Dolby Vision profiles: 10, 10.1 and 10.4. Profile 10 is “true” Dolby Vision, 10.1 is their backward compatible version of HDR10 and 10.4 their backward compatible version of Hybrid Log Gamma (HLG). For profiles 10.1 and 10.4, you need to use a SUPPLEMENTAL-CODECS brand attribute and the correct VIDEO-RANGE. For these, 10.1 should use ‘db1p’ and PQ, and 10.4 should use ‘db4h’ and HLG. The full example codec string they provided is: CODECS=”av01.0.13M.10.0.112″,SUPPLEMENTAL-CODECS=”dav1.10.09/db4h”,VIDEO-RANGE=HLG.

If you’re interested in Apple’s overall AV1 Support, you can find more details in this blog post.

Enhanced timed metadata support

HLS now supports multiple concurrent metadata tracks within Fragmented MP4 files, enabling richer media experiences with timed metadata (‘mebx’) tracks. This will enable new opportunities for integrating interactive elements and dynamic content within HLS streams. .

Metrics and logging advancements

The introduction of the AVMetrics API to AVFoundation will allow developers to monitor performance and playback events. This opt-in interface lets you select which subsets of events to monitor and provides detailed insights into media playback, allowing you to optimize streaming experiences further.

More details are available in the AVFoundation documentation and the WWDC 2024 session “Discover media performance metrics in AVFoundation”.

Common Media Client Data (CMCD) standard integration

HLS now supports the CMCD standard, enhancing Quality of Service (QoS) monitoring and delivery optimization through player and CDN interactions. AVPlayer only implemented the preferred mode of transmitting data via HTTP request headers. They have not included support for all of the defined keys and for now is only supported in iOS and tvOS v18 and above. There was no mention of support in Safari. 

Bitmovin and Akamai debuted our joint CMCD solution at NAB 2023. You can learn more in our blog post or check out our demo.

FairPlay content decryption key management

As part of ongoing improvements, HLS is deprecating AVAssetResourceLoader for key loading in favor of AVContentKeySession. AVContentKeySession was first introduced at WWDC 2018 and until now, Apple had been supporting both methods of key loading for content protection in parallel. Using AVContentKeySession promises more flexibility and reliability in content key management, aligning with evolving security and operational requirements. This move means any existing use of AVAssetResourceLoader must be transitioned to AVContentKeySession. 

Conclusion

The recent HLS updates show Apple’s commitment to enhancing media streaming capabilities across diverse platforms and scenarios. For developers and content providers, staying updated with these advancements not only ensures compliance with the latest standards but also unlocks new opportunities to deliver compelling streaming experiences to audiences worldwide. 

If you’re interested in being notified about all of the latest HLS updates or you want to request features or provide feedback, you can subscribe to the IETF hls-interest group.

The post WWDC 2024 HLS Updates for Video Developers appeared first on Bitmovin.

Apple made waves across the video encoding and streaming communities when they announced the iPhone 15 Pro and 15 Pro Max would have a dedicated AV1 hardware decoder, making them the first Apple devices with official AV1 codec support. We’ve compiled all the details from their announcement, the HLS interest group, and product release notes to bring you everything you need to know about Apple AV1 codec support. If you’re looking for more information about AV1 playback on Android, Smart TVs and set-top boxes, you can find more information at https://bitmovin.com/av1-playback-support/. Otherwise, keep reading to learn more!

• Hints that Apple AV1 support was coming
• iPhone 15 Pro announcement
• More details about HDR, DRM, HLS and Safari support for AV1
• Apple M3 processor announcement
• Apple M4 processor iPad announcement
• AV1 Software Decoding Support?
• Ready to take advantage of AV1 Encoding?
• Related links

Hints that Apple AV1 support was coming

Prior to the iPhone 15 announcement in September 2023, there were several indications that Apple would eventually support AV1. Back in 2018, Apple joined the Alliance for Open Media, the organization responsible for creating and promoting AV1 encoding and many took it as a sign that Apple would eventually support AV1. More recently, updates to Apple’s AVFoundation core media framework showed the addition of a new global variable “kCMVideoCodecType_AV1“, and earlier in 2023, the Safari 16.4 Beta release notes actually showed AV1 support was coming, but it was removed without comment shortly after and never added to Safari 16. AV1 WebCodecs support did eventually become available as an experimental  option in the Safari Technology Preview, but enabling it didn’t seem to have any effect.

Still with all of these hints being dropped, the announcements of Apple’s M series of processors and the most recent update to the HLS draft specification in May 2023 all came and went with no mention of AV1. Everyone who was paying close attention and anticipating Apple AV1 support was left disappointed, especially knowing how much weight their decision carried for the rest of the streaming ecosystem. Overall AV1 adoption has been slower than many had hoped and expected, and Apple’s lack of support was often cited as a reason to wait and avoid updating video encoding stacks. 

iPhone 15 Pro announcement

This all changed on September 12, 2023, when Apple announced their new A17 Pro mobile processor would include support for AV1 hardware decoding. You can watch the full replay here, with the section about the 15 Pro’s new processor beginning at 1:01:20. VP of the Apple Silicon Engineering Group, Sribalan Santhanam presented the new A-series processor and shared details about the industry’s first 3 nm chip, including a 6-core CPU and a new Pro-class, 6-core GPU. It also has a 16-core neural engine that can process up to 35 trillion operations per second and run machine learning models on the device, without sending personal data to the cloud. It also includes a dedicated engine for Apple’s own ProRes codec in addition to the big one for video streaming services, the AV1 hardware decoder. 

“We also included a dedicated AV1 decoder, enabling more efficient and high-quality video experiences for streaming services.”

Sribalan Santhanam – VP, Apple Silicon Engineering Group

More details about HDR, DRM, HLS and Safari support for AV1

After the presentation, co-author of the HLS specification Roger Pantos shared more details via the hls-interest mailing list. He confirmed that indeed, that both the iPhone 15 Pro and 15 Pro Max would be the first Apple devices with hardware decoding support for AV1 video content. The dedicated hardware meant that in addition to Standard Dynamic Range (SDR) content, it would also support High Dynamic Range (HDR10) as well as content that was protected by FairPlay Streaming DRM, things that software decoders typically cannot handle well or securely. Playback would be supported in Apple’s native AVPlayer or AVSampleBufferDisplayLayer, including using Media Source Extensions (MSE), or Managed Media Source (MMS) as Apple calls their new version, under an experimental setting on iOS Safari.

HLS playback of AV1 will work without any new signaling requirements, just the regular CODEC and VIDEO-RANGE attributes. The SCORE attribute can also be used to force the playback client to prefer AV1 over other encodings, but renditions encoded with AVC and/or HEVC should still be included for older devices and AirPlay support. The WebKit blog provided more information about Safari 17.0, confirming support for the AV1 video codec was added on devices with hardware decoding support. They also shared this html code snippet for presenting single-file progressive video that has been encoded with AV1, HEVC and VP9, which allows the browser to choose the best option for playback. It should be noted that outside of very short clips, adaptive streaming with HLS is preferred over progressive streaming in order to provide the best quality of experience and bandwidth efficiency.

The ‘type’ attribute signals the type of container being used and ‘codecs’ parameter string lets the browser know which codec was used and other characteristics like profile, level, color space, bit depth and dynamic range. This informs the browser and lets it decide whether it supports those attributes or needs to fall back on an older codec. It’s also possible to use a simpler codecs=”av01”, but it’s best to provide as much detail as possible if you can. More information on the AV1 codecs parameter string from the Alliance for Open Media can be found here, and details about codec and profile parameters are available in this IETF doc. 

While not directly related to the Apple AV1 news, Safari 17.0 also added a new media player stats overlay similar to YouTube’s “stats for nerds”. This is a nice addition for video developers doing any troubleshooting and will be very helpful as people begin experimenting with adding AV1 encoding. It’s available to anyone who checks the “Show features for web developers” box in the advanced settings of Safari.  

Apple M3 processor announcement

In late October 2023, Apple announced their newest generation of desktop processors would include AV1 hardware decoders. This includes the M3, M3 Pro and M3 Max chips, meaning all new models of Macbooks, iMacs and desktop computers with an M3 processor will support AV1 video playback. Some were disappointed that the M3 did not also include support for AV1 encoding, but for video playback, the decoding is all that really matters, so this will be another nice wave of new devices that streaming services can target with AV1 encoded video. 

Apple M4 processor iPad announcement

Announced in May 2024, the new iPad Pro is powered by Apple’s latest system on a chip, the M4. The media engine of the M4 supports multiple codecs, including H.264, HEVC, ProRes and now AV1, making it the most advanced media processor ever in an iPad. With this, Apple continues their march toward full AV1 support. Will the Vision Pro 2 be next?

Apple AV1 Dolby Vision Support

Usually around the time of Apple’s World Wide Developer Conference there are some new updates or features around HLS and AVPlayer. During WWDC24, Apple shared a “What’s new in HTTP Live Streaming 2024” doc with several interesting new additions. For AV1 specifically, they called out support for using Dolby Vision Profile 10, which is Dolby’s 10-bit AV1 aware profile. Apple now supports 3 different Dolby Vision profiles: 10, 10.1 and 10.4. Profile 10 is “true” Dolby Vision, 10.1 is their backward compatible version of HDR10 and 10.4 their backward compatible version of Hybrid Log Gamma (HLG). For profiles 10.1 and 10.4, you need to use a SUPPLEMENTAL-CODECS attribute and the correct VIDEO-RANGE. For these, 10.1 should use ‘db1p’ and PQ, and 10.4 should use ‘db4h’ and HLG. The full example codec string they provided is: CODECS=”av01.0.13M.10.0.112″,SUPPLEMENTAL-CODECS=”dav1.10.09/db4h”,VIDEO-RANGE=HLG.

AV1 Software Decoding Support?

When Apple released the iPhone 6s with the A9 chip, it became the first iOS device to support HEVC(H.265) hardware decoding, which included support for FairPlay Streaming with HEVC. When this happened, they also included an HEVC software decoder as part of the next iOS and macOS updates for older devices without hardware support. While the software decoding didn’t support FairPlay Streaming, it was still a big boost for HEVC support and was one of the first things we wondered about after seeing the AV1 decoder announcement.

Unfortunately when asked, Roger Pantos shared that Apple would not be shipping an AV1 video software decoder at this time. He did confirm that iOS 17 does include some AV1 codec support, but only for still images using the Alliance for Open Media’s AVIF format. For now, we can only hope that AV1 video software decoding (like Meta is already using in their iOS apps) will be coming soon.

Ready to take advantage of AV1 Encoding?

Bitmovin has been ready for AV1 adoption to spread for some time now, dating back to 2017 when we partnered with Mozilla to enable AV1 playback in the Firefox browser using the Bitmovin Player. We’ve added AV1 codec support to our Per-Title and 3-pass encoding optimizations and just recently made AV1 encoding available in our dashboard UI, so now you can perform your first AV1 encode without any code, API calls, or configuration necessary! Bitmovin’s AV1 encoding has supported DASH streaming together with Widevine content protection for a long time, but we’ve now also added support for fMP4 in HLS playlists together with FairPlay content protection to take advantage of Apple AV1 support for premium content. It’s also available in our free trial, so there’s never been a better time to check it out and begin taking advantage of the bandwidth savings and quality improvements that AV1 can provide. 

Click here to start your free trial today!

Related links

• Read the latest info about our AV1 playback support and device testing here.
• Learn how using Bitmovin’s Per-Title Encoding together with AV1 can let you stream 4K video at bitrates that had been limited to Standard Definition with older codecs. 
• Check out our AV1 hub and download our datasheet to learn all about the codec’s development, performance and how it can lower your CDN costs.

The post Everything you need to know about Apple AV1 Support appeared first on Bitmovin.

Table of Contents

This post covers some recent updates, focusing on the new Firefox AV1 support in Encrypted Media Extensions. Bitmovin has been supporting and advocating for use of the AV1 codec for several years, even though there have been gaps in playback support preventing adoption for some workflows. Slowly but surely, those gaps are being filled and the reasons not to use AV1 are going away. Keep reading to learn more.

Firefox 125 adds support for encrypted AV1

A couple of years ago, Bitmovin began testing several different combinations of AV1 encoding, muxing and DRM support across browsers and playback devices. We were somewhat surprised to learn that even though Firefox was the first major browser to support AV1 playback, they had not implemented support for encrypted AV1 as they had for other codecs. We found there was actually an open bug/request filed 5 years ago. 

Shortly after we began watching closely, there was an update…

Ouch. Once the ticket got reassigned, Bitmovin got involved and gave our feedback that for premium/studio content, this support would be needed soon. We also provided a Widevine-protected sample for them to use in testing. Fast-forward to this spring, we saw some action on the ticket and support for AV1 with Encrypted Media Extensions was officially added to Firefox 125!

This means premium content workflows can now use AV1 on all of the major desktop browsers. Apple added support to Safari last fall, including with FairPlay Streaming, but for now it’s limited to devices with AV1 hardware decoders (iPhone 15 Pro, iPad Pro, new Macs with M3 processors).

Previous Bitmovin and Firefox AV1 collaboration

Way back in 2017, before the AV1 spec was finalized, Bitmovin and Firefox collaborated on the first HTML5 AV1 playback. Because the bitstream was still under development and subject to change, Bitmovin and Mozilla agreed on a common codec string to ensure compatibility between the version in the Bitmovin encoder and the decoder in Mozilla Firefox. It was made available in Mozilla’s experimental development version, Firefox Nightly, for users to manually enable. 

Even earlier in 2017, Bitmovin demonstrated the first broadcast quality AV1 live stream at NAB, winning a Best of Show award from Streaming Media Magazine. 

Other recent AV1 playback updates

Android adds dav1d decoder

In March 2024, VideoLAN’s “dav1d” became available to all Android devices running Android 12 or higher. Apps need to opt-in to using AV1 for now, but according to Google, most devices can at least keep up with software decoding of 720p 30fps video. YouTube initially opted to begin using dav1d on devices without a hardware decoder, but may have reverted that decision, likely due to battery concerns on phones. For plug-in Android devices, dav1d is still a great option and a welcome addition to the ecosystem.

iPad Pro gets AV1 playback support with M4 processor

In early May 2024, Apple continued their march toward full AV1 support with the announcement of their new M4 chip, which will power the new iPad Pro. The Media Engine of M4 is the most advanced to come to iPad, supporting several popular video codecs, like H.264, HEVC, and ProRes, in addition to AV1.

Ready to get started with AV1?

Bitmovin has added AV1 codec support to our Per-Title and 3-pass encoding optimizations and made AV1 encoding available in our dashboard UI, so now you can perform your first AV1 encode without any code, API calls, or configuration necessary! Bitmovin’s AV1 encoding has supported DASH streaming together with Widevine content protection for a long time, but we’ve now also added support for fMP4 in HLS playlists together with FairPlay content protection to take advantage of Apple AV1 support for premium content. It’s also available in our free trial, so there’s never been a better time to check it out and begin taking advantage of the bandwidth savings and quality improvements that AV1 can provide.

Related Links

Website: Bitmovin’s AV1 hub   

Blog: State of AV1 Playback Support

Blog: Everything you need to know about Apple’s AV1 Support

Blog: 4K video at SD bitrates with AV1

The post New Firefox AV1 support for Encrypted Media Extensions appeared first on Bitmovin.

In this post, I’ll be taking a look at the current state of AV1 playback support, covering which browsers, mobile devices, smart TVs, consoles and streaming sticks are compatible with the AV1 codec right now.  I’ll also touch on some of the incredible bandwidth savings companies like Netflix are seeing with AV1 and detail the latest announcements, rumors and speculation around future AV1 playback support.

AV1: The Story So Far (2017-2023)

Back in 2017, Bitmovin debuted the world’s first AV1 live encoding at the NAB Show in Las Vegas, earning a Best of NAB award. While it was an exciting proof of concept at the time, AV1 playback support was extremely limited and large-scale production usage wouldn’t come until years later. In 2020, YouTube and Netflix began delivering AV1 to the first compatible Android devices, and last year Netflix shared details about their expanded use of AV1 for 4K streams.

Netflix also published a report that showed over the course of one month in early 2022, 21% of their streamed content benefited from the most recent improvements in codec efficiency, like Per-Title optimized AV1 and HEVC. They estimated that without those improvements, total Netflix traffic globally would have been around 24% higher, proving that you can see massive bandwidth and overall cost savings by encoding just a portion of your most popular content with AV1.

Apple adds AV1 hardware decoding support to iPhone 15 Pro and new Macbooks

Many of us who have been tracking the adoption and progress of AV1 were disappointed when the announcements for Apple’s M-series processors over the past couple years did not include AV1 hardware decoding support. But on September 12, 2023, the big moment we’ve been waiting for finally arrived when Apple announced that the A17 Pro chip in their new iPhone 15 Pro would include a dedicated AV1 decoder. This is a big line in the sand for Apple and for the wider industry and will hopefully prove to be the day that revitalized interest and momentum for AV1 adoption across the industry.

“We also included a dedicated AV1 decoder, enabling more efficient and high-quality video experiences for streaming services.”

Sribalan Santhanam – VP, Apple Silicon Engineering Group

After the presentation, co-author of the HLS spec Roger Pantos shared more details via the hls-interest mailing list: 

The iPhone 15 Pro (both screen sizes) will be the first Apple product to support hardware decode of AV1 content. This includes SDR, HDR10, and content protected by FairPlay Streaming, played back through either AVPlayer or AVSampleBufferDisplayLayer (including MSE on Safari).

There is no new signaling necessary for HLS, just the regular content-specific values for the CODECS and VIDEO-RANGE attributes in the MVP. If you wish, you can use the SCORE attribute to make the client prefer AV1 over other encodings (but please continue to provide renditions encoded with AVC and/or HEVC for compatibility with earlier devices and AirPlay).

A month later in October 2023, Apple announced their newest generation of desktop processors would include AV1 hardware decoders. This includes the M3, M3 Pro and M3 Max chips, meaning all new models of Macbooks, iMacs and desktop computers with an M3 processor will also support AV1 video playback.

Earlier in 2023, while everyone was waiting for Apple to officially support AV1, Meta took matters into their own hands, sharing how they brought AV1 to their Reels videos for Facebook and Instagram, including on iOS devices. This became possible through ongoing open source software decoding efficiency improvements, in particular with the dav1d decoder, developed by VideoLAN. Meta also said they believe for their video products, AV1 is the most viable codec for the coming years. The image below shows how they significantly improved visual quality with AV1 over VP9 and H.264, while keeping the bitrate constant.

At Bitmovin we also believe in the potential of AV1 and have explored the possibilities of software decoding on mobile devices. At a recent internal hackathon, one of our senior software engineers, Roland Kákonyi, built a custom iOS player using the dav1d decoder that was able to decode and smoothly play 1080p AV1 content. We’ll continue exploring this further as a way to fill gaps in playback coverage for devices lacking hardware support.

AV1 Playback Support News in 2024

Following 2023’s big announcements from Apple, 2024 got off to a strong start with Android, Firefox and (again) Apple adding new AV1 playback support. The barriers and arguments against adopting AV1 continue falling, slowly, but surely.

Android adds dav1d decoder

In March 2024, VideoLAN’s “dav1d” became available to all Android devices running Android 12 or higher. Apps need to opt-in to using AV1 for now, but according to Google, most devices can at least keep up with software decoding of 720p 30fps video. YouTube initially opted to begin using dav1d on devices without a hardware decoder, but may have reverted that decision, likely due to battery concerns on phones. For plug-in Android devices, dav1d is still a great option and a welcome addition to the ecosystem.

Firefox adds AV1 support in Encrypted Media Extensions

While Firefox was the first major browser to support AV1 playback, a long-standing bug (or lack of implementation) prevented DRM-protected AV1 from playing. When Apple added support to Safari for HLS + FairPlay streaming, it meant Firefox was the only major browser that still did not support premium, secure content. That changed in April 2024, when Firefox 125 added AV1 support in encrypted media extensions, meaning Widewine-protected AV1 is now supported.

iPad Pro gets AV1 playback support with M4 processor

In early May 2024, Apple continued their march toward full AV1 support with the announcement of their new M4 chip, which will power the new iPad Pro. The Media Engine of M4 is the most advanced to come to iPad, supporting several popular video codecs, like H.264, HEVC, and ProRes, in addition to AV1.

Current State of AV1 Playback support

To answer the question of current playback support as thoroughly as possible, we created several sample streams with different combinations of containers, muxings and DRM. While there will be some exceptions and omissions, especially when you go back to the 2021 and 2020 models, I’ll use the emojis below to show the general level of support you can expect from these platforms and brands right now and give the full results of our direct testing in the table at the end. 

• Fully Supported – Successful AV1 playback with all test streams, including DRM

• Partial or Documented Support – Successfully played at least one, but not all of our test streams OR the product documentation claims AV1 playback support, but has not yet been verified by Bitmovin

• Not Supported – AV1 playback not supported here currently

Browsers and Operating Systems

*Safari 17 or later, when a hardware decoder is present

**AV1 is also supported in Chrome and Firefox on macOS

Generally speaking, the Chrome browser and Android ecosystem handle AV1 well across phones, tablets, smart TVs and set-top boxes/streaming sticks. Unfortunately, the same cannot be said for Safari and iOS where support had been lacking until the iPhone 15 Pro announcement.

Firefox was the first major browser to support AV1, and recently Firefox 125 added support for AV1 in Encrypted Media Extensions, meaning Widevine-protected content is now playable.

The Edge browser on Windows 10 and later supports AV1, but you may need to install the free AV1 Video Extension from the Microsoft Store. 

For more details about the specific versions and less common browsers that support AV1, check out the table from CanIUse.com here. 

Smart TVs

As mentioned, Android handles AV1 quite nicely, which also applies to the Smart TVs running Android TV and Google TV operating systems. These include Sony Google TV models from 2021 on and many Amazon Fire TV models as far back as 2020. (FireOS is based on Android)

Samsung TVs (and phones) from late 2020 onward have AV1 hardware decoders and were mentioned by Netflix as some of the first outlets for their 4K AV1 content. 

LG has developer documentation stating AV1 is supported for their UHD TVs and projectors running WebOS 5.0 and above, although our testing on some 2020 models was unsuccessful.

Consoles and Streaming Sticks

Amazon Fire TV Stick 4K Max

Playstation 4 Pro

Xbox One

Roku Streaming Stick 4K

Playstation 4 Pro was also called out by Netflix as one of the targets for their 4K AV1 streams and it takes advantage of GPU-accelerated decoding. Netflix didn’t publicly mention delivering AV1 to Xbox One, but the same decode libraries that the PS4 Pro uses were first made available for Xbox One, so it should be possible.

The Amazon Fire TV Stick 4K Max has AV1 + DRM support, making it one of the cheapest and best options for giving older 4K TVs an AV1 upgrade. 

Roku is a little bit of a gray area at the moment. Officially, they still do not support AV1 as an adaptive streaming video codec, but newer models like the Roku Ultra that have a USB port do support AV1 playback via USB media. There does appear to be some level of support for AV1 adaptive streaming, as the YouTube “stats for nerds” overlay reveals a combination of AV1 video and opus audio playing on many of the popular recommended videos. Hopefully wider support is coming, but in the meantime, did confirm successful playback of our single file “progressive” AV1 MP4 files on the Streaming Stick 4K.

Looking Ahead: Future AV1 Playback Support

Even with gaps in support on some platforms, there is plenty of opportunity to see tangible bandwidth savings and quality improvements from AV1 right now and thankfully, the future looks even brighter. Intel, AMD, Samsung and Qualcomm have all announced additional AV1 support coming at the chip level.

Will Apple add AV1 software decoding support for older devices? 

There have been several indications that Apple would eventually support AV1. Apple joined the Alliance for Open Media, the organization responsible for creating and promoting AV1 encoding, back in 2018, which many took as a sign that Apple would eventually support it. We’re hopeful that with the addition of AV1 hardware decoding support to the iPhone Pro 15, iPad Pro and Macbooks, Apple will also add official HLS support and fallback software decoding for older devices that are capable.

Conclusion

While AV1 support and adoption has been on the rise and we’ve seen some encouraging announcements, universal support like we have with H.264 is just not there yet. That means AV1 will need to be part of a multi-codec approach for the foreseeable future, but that’s ok! Not that long ago, it took millions of views to offset the higher encoding costs of AV1, but with recent improvements, we’ve seen the break-even point drop to as low as 4,000 views! So for a whole lot of content, encoding with AV1 can already save you money right now and those savings will only increase as more supporting devices become available. 

Ready to get started with AV1 encoding? You can try it for free with a Bitmovin Trial, sign up here!

: Playback supported

: Playback not supported

: Not yet tested

	fMP4 (DASH)	fMP4 with Widevine and Playready (DASH)	Single file “progressive” MP4 (.mp4)	Single file “progressive” MP4 + Widevine (DASH)	WebM (DASH)	WebM + Widevine (DASH)	Single file “progressive” WebM (DASH)	Single file “progressive” WebM + Widevine (DASH)		fMP4 (HLS)	fMP4 + Fairplay (HLS)
Chrome
Edge
Firefox
Safari
Android Native
Android Web
iOS
Fire TV Max
Fire TV Max Web (Silk Browser)
Roku Streaming Stick 4K
Samsung Tizen (2020 and 2021)

The post The State of AV1 Playback Support: 2024 appeared first on Bitmovin.

We concluded the event with our first annual Bitmovin Innovators Network partner awards to recognize and celebrate the amazing work of our partners who embrace the fact that the industry is “Better Together”, by creating solutions with partners that are designed to simplify customers’ video workload needs and advance the viewing experience for audiences.

I am incredibly proud to share the winners of the Bitmovin Innovators Network partner awards below, and the contributions they’ve made: 

Accenture – Global Systems Integrator of the Year:

Accenture and Bitmovin exemplify the “better together” approach through their close strategic partnership, including an ongoing collaboration with the world’s largest motorsports content owners that led to joint engagements with several of the largest sports and media brands in the world.

Broadpeak – Global ISV Partner of the Year:

Broadpeak embodies the “Better Together” spirit through its unwavering strategic collaboration with Bitmovin. This powerful partnership has yielded several key benefits. Together, they have developed solutions that integrated with Bitmovin’s encoder, player, and analytics, resulting in improved workflows for customers; created a consistent two-way communication between sales teams which has resulted in successful deals with European media brands, and joint marketing and PR initiatives at local events to strengthen their joint brand presence.

MediaKind –  Global Service Provider Partner of the Year:

MediaKind and Bitmovin have developed and maintained a robust strategic partnership that has launched sports applications for world-renowned sports leagues. These applications, including launching an app with a sports league on Apple Vision Pro that garnered rave reviews at the Apple launch event, have significantly boosted market visibility for both brands.

Microsoft Azure Marketplace – Cloud Marketplace of the Year:

Bitmovin has had unprecedented success with the Microsoft Azure Marketplace, including more than 200 new customer wins since June 2023. Azure Marketplace has quickly become Bitmovin’s largest and most successful sales channel.

Nomad Media – Americas Regional Channel Partner of the Year:

Nomad Media has deployed over 30 customers on the Bitmovin Play platform in 2023 alone as part of its Nomad Media platform. Nomad Media has also innovated on the player capabilities with dynamic multi-view capabilities. These advancements were showcased to major US clients, propelling both companies forward. This collaboration not only built a strong pipeline but also significantly boosted brand recognition in the US market.

G&L Geißendörfer & Leschinsky – EMEA Regional Channel Partner of the Year:

G&L is a proactive and committed industry partner, who has worked with Bitmovin on both successful sales and marketing initiatives. The collaboration between the two companies resulted in joint revenue, a new logo, and G&L also exhibited on the Bitmovin stand at IBC 2023 where it highlighted how the two companies’ solutions work together. Bitmovin and G&L also hosted a joint CMCD webinar together, which attracted attendees from key German broadcasters and various telecoms and content providers, and it recently published an e-commerce case study with Home Shopping Europe.

Viet Communications – APAC Regional Channel Partner of the Year

Vietcoms was the first licensee for the Bitmovin Player in the Asia Pacific region. Vietcoms was selected for its hard work and efforts in securing our impressive player business in Vietnam and developing agile operational models to meet the specific customer and TelCo business needs and technical requirements.

Once again, I’d like to give huge congratulations to all the winners. A huge thank you to everyone who attended the Bitmovin Innovators Network Partner Executive Networking Event, and to every single one of our partners who continue to embrace the spirit of “Better Together.” IBC is just around the corner, and we will have some exciting initiatives and announcements coming soon to share with you ahead of the show.

The post The Bitmovin Innovators Network “Better Together” Award Winners! appeared first on Bitmovin.

What is RDK?

RDK stands for Reference Design Kit. It’s an open source software bundle for video, broadband and IoT devices. It’s commonly used in consumer devices which include video streaming apps, for example, set top boxes and is a popular development platform for service providers in the cable, satellite and OTT industries.

What is the RDK Tech Summit?

The RDK Tech Summit is a live gathering for RDK members worldwide to discuss technical advancements, new releases and innovations. It consists of presentation sessions grouped into themes, with regular networking breaks for discussions. This year’s summit was in Amsterdam, hosted by Liberty Global and Infosys.

Highlights from RDK Tech Summit 2024

Sessions were themed on the topics of:

• Connectivity: Next-Gen Networking
• Entertainment: Cloud Strategies and Firebolt
• Device Operations: Operational Monitoring & Test Automation
• Sustainability

Highlights included:

• The keynotes focused on collaboration, finding common challenges and working to solve them together. Liberty Global gave insight into how they build for, and support, a vast range of devices, across many years of manufacture. They highlighted the need for collaboration where there’s commonality and thinking about what’s common compared to differentiating your service.
• There was a deep dive on Automatics which is a test automation platform for RDK. This can handle the automation of running tests across many different test cases. During networking discussions, this was a hot topic and we learnt that Automatics is in use by a major telco.
• The Lightning team highlighted Lightning 3.0 with enhancements focused on performance, quality and a better developer experience. They also noted that Lightning was designed for lower power devices from the beginning, so it can achieve better performance than other recently popular development frameworks.
• Comcast introduced a layered build system for RDK, allowing component layers to be built independently, including application, middleware and vendor layers. This can speed up development & build times compared to a single monolithic build.
• Details about Firebolt 2.0 were shared together with Firebolt certifications and Firebolt Connect which is an automated test harness for apps running on RDK. 

Separately in the demonstration area Irdeto were showing their App Watch solution running on RDK. This aggregates apps and provides app launching functionality on RDK. 

Bitmovin’s RDK support

Bitmovin’s Playback supports the widest range of devices on the market today. This includes set top boxes. Bitmovin provides a common Player SDK which can be used across these platforms, enabling adaptive bitrate playback. 

A key challenge highlighted by many presenters & attendees was supporting the wide range of devices which consumers use today, some of which date back 5-7 years or more. It’s also not economically viable to scale engineering teams linearly with the growing number of devices that streaming services need to support. Leveraging a common video player across each device can increase the number of platforms which engineering teams can support, allowing them to focus on differentiating features ahead of common, core functionality. A vision which Bitmovin supports among the RDK community.

The data from Bitmovin’s annual Video Developer Report also illustrates the challenge of supporting multiple generations and a constant stream of new playback devices. When asked about the challenges video developers are facing, “Playback on all devices” is a mainstay near the top of the list, coming in at #3 in our most recent survey. 17% of respondents said they are currently supporting playback on RDK-powered set top boxes and another 9% said they planned to begin supporting them in the coming year. 

Bitmovin are a member of the RDK Open-Source Community and regularly attend RDK events including Tech Summits & Global Summits.

If you’d like to learn more about RDK and how Bitmovin can help, you can let us know in the comments, ask questions in the Bitmovin Community or contact us here. 

Related Links

RDK website

RDK Tech Summit 2024 website

Bitmovin Player SDK – RDK support

Bitmovin Player SDK – set top box support

The post Highlights from RDK Tech Summit 2024 appeared first on Bitmovin.

For the past few years, AI has been one of the top buzzwords at the NAB Show. While other hot topics like “web3” seem to have peaked and faded, interest in video AI has continued to grow and this year there were more practical solutions being showcased than ever before. A personal highlight for Bitmovin was winning a TV Technology Best of Show award for our AI-powered Analytics session interpreter. Keep reading to learn more about other interesting and useful applications of AI that we saw at NAB 2024.

NAB Video AI Highlights: 2024

While there was some variation in implementation and features, the majority of the AI solutions I encountered at NAB fell into one of these categories:

• Generative AI (genAI) for video creation, post-production, or summaries and descriptions
• Automatic subtitling and captioning with multi-language translations
• Object or event detection and indexing
• Video quality enhancement

This summary is definitely not exhaustive, but highlights some of the things that stood out to me on the show floor and in the conference sessions. Please let us know in the comments if you saw anything else noteworthy.

Booths and Exhibits

Adobe

Adobe has been showing AI-powered editing and post-production tools as part of their creative suite for a couple years now and they seem to be continuously improving. They teased a new Firefly video model that will be coming to Premiere Pro later this year that will enable a few new Photoshop-like tools for video. Generative Extend will allow you to extend clips with AI generated frames for perfectly timed edits and the new Firefly model will also enable object removal, addition, and replacement. They’ve also implemented content credentials into the platform that will signal when generative AI was used in the creation process and which models were used, as they prepare for supporting 3rd party genAI models like OpenAI’s Sora.  

Amazon Web Services (AWS)

AWS had one of the busiest booths in the West hall and were showcasing several AI-powered solutions, including using genAI for creating personalized ads and Intel’s Video Super Resolution upscaling. But they also had the most eye-catching and fun application of AI in the South Hall, a genAI golf simulator where you could design and play your own course.

axle.ai

Axle.ai was sharing their face, object, and logo recognition technology that can index recognized objects and search for matching objects in other videos or clips. Their software also has automatic voice transcription and translation capabilities. It can run either on-premises or in the cloud and integrates with Adobe Premiere, Final Cut Pro and other editing suites. While other companies offer similar capabilities, they stood out as being particularly focused on these use cases.

BLUEDOT

BLUEDOT was showcasing a few different solutions for improving QoE in the encoding and processing stage. Their DeepField-SR video super resolution product uses a proprietary deep neural network to upscale video up to 4K resolution, leveraging FPGAs. They were also showing AI-driven perceptual quality optimized video encoding.

Twelve Labs

Twelve Labs was featuring their multimodal AI for Media & Entertainment workflows, aiming to bring human-like understanding to video content. They use both video and audio information to inform object and event detection and indexing.  This enables you to easily find moments in a video, like when a certain player scores or when a product is mentioned. They also power generative text descriptions of videos and clips. Their solution seemed more flexible than others I saw and can be integrated into media asset management systems, editing software or OTT streaming workflows.

Conference Sessions and Presentations

Beyond the Hype: A Critical look at AI in Video Streaming

In this session, as the title suggests, Jan Ozer took a close look at the current state of AI applications for video streaming workflows. He conducted several interviews with executives and product leaders ahead of NAB and shared his notes and links to the full interviews. He also called out a few times that many of the companies featured, including Bitmovin, have been researching and working on AI-powered video solutions for several years now, even before the current wave of hype. He shared Bitmovin’s new Analytics session interpreter and our Super Resolution capabilities, which you can hear more about in his interview with our VP of Product, Reinhard Grandl.

Some other things that stood out for me included Interra Systems’ BATON Captions, which uses natural language processing to break text in a more natural, human readable way. This is a small, subtle feature that can really make a big difference in improving accessibility and the viewer experience, that I haven’t heard anyone else focus on. DeepRender also caught my attention with their claims of an AI-based video codec that will have 45% better compression than VVC by the end of 2024. That’s a really bold claim and I’ll be watching to see if they live up to the hype. Video of the session is available here, thanks to Dan Rayburn and the Streaming Summit.

Running OpenAI’s Whisper Automatic Speech Recognition on a Live Video Transcoding Server

This was a joint presentation led by NETINT’s COO Alex Liu and Ampere’s Chief Evangelist Sean Varley. They presented a practical demo of real-time live transcoding and subtitling using NETINT’s T1U Video Processing Unit (VPU) together with Ampere’s Altra Max CPU running OpenAI Whisper. The NETINT VPU is capable of creating dozens of simultaneous adaptive bitrate outputs with H.264, H.265 and AV1 codecs. The Ampere processor was being positioned as a more environmentally-friendly option for AI inference workflows, consuming less power than similarly capable GPUs. While there were some hiccups with the in-room A/V system, the live captioning demo was impressive and worked very well. Video of the session is available here, again thanks to Dan Rayburn and the Streaming Summit.

Leveraging Azure AI for Media Production and Content Monetization Workflows

Microsoft’s Andy Beach and MediaKind’s Amit Tank led this discussion and showcase of using genAI in media and entertainment workflows. They discussed how AI can help with each part of the production and delivery workflow to boost monetization. This included things like brand detection, contextual ad placements, metadata automation, translations, captioning and personalization. One area they discussed that I hadn’t heard anyone else talk about was using AI for content localization, not just for language translation via captions and dubbing, but for compliance with local and regional norms and in some cases regulations. For example, some areas and countries may prefer or even require removal or censorship of things like alcohol and drug use or guns and excessive violence, so AI can help automate content preparation in different ways for a global audience. They also shared their own personal “most-used” AI applications, which included Microsoft’s Copilot and related AI add-ons to Teams and other Microsoft products.

Did you see an interesting or innovative use of AI at NAB that wasn’t mentioned here? Please let us know in the comments!

The post NAB Video AI Highlights appeared first on Bitmovin.

When evaluating the industries, the term “buyer or builder” comes to mind. This is a common phrase that describes platforms that either use pre-existing third-party solutions to get their workflow up and deployed quickly so they can get to market faster (buyer) or build it themselves to maintain control of their tools and feature set (builder). Since financial aspects play a significant role in the choices every platform makes, it’s essential to understand that going the builder route is often connected to a higher risk and can turn quite cost-intensive, whereas being a buyer could lead to a faster platform launch with a predictable budget, allowing to implement monetization strategies such as advertising video on demand (AVOD), subscriber video on demand (SVOD) or a hybrid model between the two.

In this blog, we will highlight what leaders and anyone focused on maintaining and optimizing team budgets can do to reduce their financial obligation and TCO from a technology standpoint. We will discuss the different payment methods that streaming platforms use, the technology that can help them innovate, reduce costs and development time, and how Bitmovin fits into that conversation.

Understanding Total Cost of Ownership (TCO) in Video Streaming

Total Cost of Ownership (TCO) in streaming platforms extends beyond just one number. It encompasses various operational expenses, including (but not limited to) infrastructure provision and maintenance, content delivery network (CDN) fees, licensing costs, and the workforce required for ongoing platform and workflow management. Moreover, TCO is significantly influenced by the platform’s scalability, adaptability to new technologies, and ability to stream content efficiently. It’s not just about minimizing expenses but optimizing spending to enhance value and maximize return on investment. 

Maintaining viewer engagement is increasingly expensive for content providers in a fiercely competitive market. They face the challenge of managing costs while still delivering top-notch streaming experiences. This requires striking a careful balance where efficiency doesn’t undermine quality, ensuring the platform’s sustainability and user satisfaction in the long run. Sacrificing quality to meet budget constraints is not an option. Instead, the focus should be on innovative strategies to maximize the streaming workflow’s effectiveness. To illustrate, let’s delve into the financial risks, challenges, and opportunities present in the market.

Financial Risks, challenges, and opportunities within the space:

Financial and product leaders navigate multiple risks, challenges, and opportunities when evaluating potential concerns or objectives. Let’s take a closer look at what they look like:

Financial Risks

• Development costs – Initial and ongoing development expenses for open-source or custom-built solutions can be substantial.
• Infrastructure overspending – Risk of unnecessary expenditure on storage and bandwidth if not aligned with usage.
• Scaling costs – As the audience grows, the fees for scaling up (more bandwidth or storage) can rise sharply.
• Technology dependency – Relying on specific technologies can lead to financial risk if they become outdated or unsupported.
  

Challenges

• Deployment delays – Technical challenges can arise during the deployment of streaming workflows, increasing costs and delaying any monetary return.
• Infrastructure management – Monitoring and maintaining multiple parts of the streaming infrastructure can be time-consuming and costly for teams.
• Technological evolution – There are always rapid advancements in streaming technology, and depending on the workflow, it could limit innovation and opportunities for further efficiencies in the future.
  

Opportunities to implement

• Fast Channels – Launching FAST channels or linear streaming can attract a broader audience.
• SVOD – Subscription models provide consistent revenue and help build a loyal viewer base.
• AVOD – Ad-based revenue models appeal to a broader audience by offering free or more affordable content.
• Hybrid Monetization – Combining multiple revenue models into a single service can open up revenue streams in additional markets.

By understanding these financial risks, adapting to any operational challenges, and implementing the right monetization model, stakeholders within these streaming platforms can position themselves for sustainable success and continued innovation in this rapidly evolving market.

Choosing the right pricing model for your needs

When implementing the service, pricing models matter, as the costs can quickly grow if not monitored. There are two main pricing models currently available in the industry:

• Pay-as-you-go (Consumption-based):
  • You only pay for the resources you use, such as bandwidth, storage, encoding minutes, and impressions. This means there is no commitment.
  • It scales easily to your needs, allowing you to increase or decrease based on actual demand. This is ideal for services starting up without reliable usage data or services with fluctuating viewership.
  • This model ensures you’re not locked into paying for unused resources, aligning costs directly with usage levels. However, it usually comes with a higher unit price compared to set consumption contracts.
    
• Set Consumption Contracts (Commitment Model):
  • You agree to purchase a predetermined amount of resources, often at a reduced rate compared to pay-as-you-go pricing.
  • This model can offer cost savings for platforms with predictable consumption patterns. Accurate forecasting would help avoid overpaying for unused resources or receiving unexpected bills for overage.
  • It suits services with stable, predictable viewership levels, providing budget certainty and potential savings.

When choosing between these pricing models, you must evaluate your viewership patterns, budgetary constraints, and strategic goals to select the one that aligns with your needs and objectives. This ensures a balance between flexibility, cost-efficiency, and scalability in your service delivery.

There are also benefits when buying through a cloud provider where you already have a commitment versus directly from a vendor. Marketplaces such as AWS, Azure, and Google Cloud can help you if you’re already using their services (or want to use a big cloud provider for your global backbone) by accessing solutions that can be beneficial for your workflows and utilize your committed consumption without adding additional costs for the team and minimizing financial burdens.

Boosting workflow efficiency with advanced encoding solutions

Continuing to look at TCO, the efficiency and effectiveness of a platform’s encoding workflow is crucial for maintaining content quality and optimizing costs (check out our blog on this topic). The right encoding partner plays a vital role in this context, assisting platforms in refining their content preparation and ensuring videos are encoded in the correct formats, bitrates, and codecs while supporting the latest features. While open-source VOD encoding options are viable and can be effective for a narrow range of use cases, they may offer limited adaptability and advanced features required for a wide range of scenarios. Bitmovin’s VOD Encoder distinguishes itself by enabling platforms to reduce storage and bandwidth costs dramatically. This is achieved through per-title encoding and the utilization of advanced codecs like AV1 while enhancing the bitrates and video quality through smart chunking and 3-pass encoding capabilities. Bitmovin’s VOD Encoder not only simplifies the transition from production to playback but also ensures that video content, even up to 8K, is encoded in superior quality with optimal compression, making it a robust choice for platforms seeking to elevate their encoding workflow while managing TCO effectively.

With Bitmovin’s VOD Encoder, you’re able to save 20% to 50% of your CDN, egress, and storage costs (depending on the scope of usage). To show how this looks for a specific business case across each piece, you can see how much is saved in the image below:

An additional area where savings are also generated is through the usage of spot instances, which can help reduce encoding costs by over 80%. Bitmovin’s pre-built orchestration algorithm saves you and your team from needing to build this out and helps you automate how your workflow utilizes available encoding resources. To get a full understanding of how these savings will benefit your budget, other aspects to consider are your yearly salary costs and other factors that will be able to show the full view of what you can save when making the decision to deploy the Bitmovin VOD Encoder.  To see more use case examples, check out our pricing breakdown page.

Supported features that help optimize TCO:

• Advanced codecs, which Bitmovin supports, include the latest in the industry, such as AV1, VP9, H.265, and others.
• Per-Title Encoding is the automatic selection of the optimal bitrate ladder for each piece of content, which helps to minimize storage and bandwidth costs.
• Smart Chunking enhances the quality of processed content by intelligently allocating bitrates throughout each asset and respecting scene boundaries, ensuring the highest quality is continuously streamed.
• Split and Stitch – separates each file-based asset into smaller segments and makes content processing possible in speeds up to 100X real-time.
• Multi-Pass Encoding (2-Pass and 3-Pass): Up-front analysis to achieve the perfect balance of quality and file size tailored to specific use cases.

Bitmovin’s approach to encoding, which integrates horizontal and vertical scaling within its cloud-agnostic infrastructure, exemplifies its commitment to innovation and efficiency. With their horizontal scaling, streaming platforms can scale the distribution of encoding tasks across multiple nodes and spin up new instances as needed, enhancing speed and handling numerous jobs simultaneously, while vertical scaling optimizes the processing capabilities of each node for complex tasks. This is bolstered by Bitmovin’s Cloud Connect solution, ensuring flexibility and scalability through seamless cloud integration, enabling streaming platforms to utilize the Bitmovin VOD Encoder within Bitmovin’s Cloud environment or on their cloud infrastructure within one or more of the major cloud providers. 
To showcase how Bitmovin’s VOD Encoder helps reduce TCO for streaming platforms further, our case study highlighted by Forrester underscores our impact. The report shows companies saving over 355% by leveraging Bitmovin’s solutions. This result alone highlights Bitmovin’s role as a leading encoding solution provider and a vital partner in the future-proofing and economic optimization of video processing for streaming platforms.

Enhancing the efficiency of live streams

Live encoding is essential for delivering captivating real-time content and reaching audiences worldwide. While open-source and other solutions offer a good starting point with flexibility and cost advantages that benefit TCO initially, they fall short in the long run. Depending on the use case, these solutions can present challenges in scalability, reliability, and the need for significant customization, which can lead to unforeseen expenses and resource allocation, particularly when streaming multiple events concurrently. The complexity of managing these systems increases as viewership grows, as does the necessity for continuous maintenance and technical support, which can significantly inflate costs and detract from core business objectives. 

Diagram of Bitmovin’s Live Encoder Workflow

Compared to the above, Bitmovin’s Live Encoder offers a more efficient, cost-effective solution that enhances financial possibilities with its SCTE-35 support for live ad insertion and leverages deployments with cloud partners that have optimized delivery networks for media distribution, such as Akamai. Its integration across every major cloud platform enables seamless scalability to meet live encoding needs, allowing global stream availability and delivery with exceptional quality and minimal latency, the key to maintaining audience engagement. This is possible on the platform’s private, secure cloud infrastructure with our cloud connect feature or can be done on the Bitmovin cloud environment, simplifying cloud infrastructure management and minimizing the need for extensive in-house technical expertise. 

Additionally, platforms can dynamically deploy live streams on the fly, from ad-hoc event-based to continuous 24/7 streams, using a pay-as-you-go consumption-based pricing model. This ensures you can adapt quickly to changing demands without incurring unnecessary and excessive costs, giving you the tools to lower your financial risks and enhance TCO. 

A player solution that maximizes reach and viewer experience 

The video player is an essential part of the video workflow, serving as the final touchpoint in delivering content to viewers. While open-source and native players offer viable and often license-free options, their support and integration can dramatically increase development demands, resulting in higher TCO and hidden financial burdens. Development teams frequently face significant hurdles related to player maintenance, cross-platform compatibility, and feature development, as well as being limited by the architecture of the base player. Additionally, open-source players rely on community-based support, which is great for side projects, but with no SLA for response times, any issues can take much longer to resolve.

In contrast, Bitmovin’s Player mitigates these support and maintenance challenges while accelerating time-to-market and expanding viewer reach. Bitmovin’s Player enables platforms to launch quickly across the broadest range of devices, significantly cutting deployment time and empowering streaming platforms to generate revenue faster. Thanks to regular updates from Bitmovin’s engineering team, our Player’s broad device support and reduced maintenance needs free up internal resources to focus on core business objectives and create more customer value. Moreover, with Bitmovin’s Player, streaming platforms can avoid the extensive development and hours of troubleshooting typically required with open-source and native solutions by leveraging pre-built features and expert support to streamline deployment and enhance viewer experiences across diverse streaming models. This is especially true regarding the native platforms, as Bitmovin’s Player helps save over 600 hours annually in baseline maintenance only.

Some of the supported platforms for the Bitmovin Player

• Web browsers
• Apple mobile devices and Apple TV
• Android mobile devices and Android TV
• Apple Vision Pro and other VR headsets
• Samsung, LG, Hisense, Vizio, Panasonic, and other smart TVs
• Roku, Amazon Fire Stick, Chromecast, and other connected devices
• X1, Sky Q, Foxtel, Android, and other set-top boxes
• PlayStation and Xbox gaming consoles
• React Native
• Flutter

Bitmovin also allows streaming platforms to test streams on physical devices with its Stream Lab feature to expedite pre-deployment testing. With the Bitmovin Player, platforms can test their streams on physical target devices in natural streaming environments with complete test reports and quality of experience metrics. Lastly, with a robust API set and out-of-the-box features such as ad modules, adaptive bitrate streaming, customizable UI, and offline playback, coupled with pre-integrations for Bitmovin’s Analytics and more, platforms are equipped with the tools necessary to implement their chosen monetization model effectively, enhancing viewer engagement and revenue generation opportunities.

Gain streaming insights and optimize workflows with the right video Analytics

Streaming platforms need to collect and analyze vital playback data to get the complete picture of how their content is streaming and to understand their workflow efficiency. This analysis is essential for pinpointing service issues, measuring audience engagement, and enhancing overall service performance to help them retain their user base and avoid churn. With these insights, they can identify and remedy specific streaming challenges by pinpointing those with the highest impact, such as reducing buffering, player load times, video start delays, ad playback issues, poor playback quality, errors, and more. These data points are critical to ensuring the efficient use of internal resources and guaranteeing an enjoyable viewing experience while avoiding any issues with revenue generation from specific streaming models like fast channels or AVOD. 

This is precisely what Bitmovin’s real-time Analytics empowers platforms to do, as they can track events across every Player deployment (with the Bitmovin Player and open-source), enabling them to analyze actionable information and prioritize their resources on the areas that have the most significant impact on audiences. They can also proactively address playback issues before they affect audiences and fine-tune their streaming service for an optimal viewer experience in real-time. Some of the metrics tracked with Bitmovin Analytics include:

• Quality of Experience
• Quality of Service
• Error Tracking
• Stream Performance
• Ad Engagement
• Audience Engagement
• Real-time Monitoring 
• Data Portability
• Granular Session Data

Bitmovin’s Analytics fits seamlessly into major data architectures as it allows multi-faceted access to data via UI, API, and structured exports, enabling integration and correlation with other data analysis tools, both homegrown and commercial. Moreover, this data informs strategic decisions, guiding content selection, efficient streaming service improvements, and advertising efforts to meet viewer preferences and trends. Additionally, the new AI feature in Bitmovin’s Analytics (Debuting at NAB) makes it so anyone (technical to non-technical) can view the data and understand how to optimize their streams for a better experience, further showcasing how it improves overall TCO.

Conclusion 

As the streaming industry continues to evolve and expand, understanding and managing total cost of ownership (TCO) becomes increasingly crucial for business leaders seeking to innovate while maintaining financial viability. The choice between building in-house solutions or adopting third-party technologies like Bitmovin’s suite of tools can significantly impact a platform’s agility, quality of service, and bottom line. Bitmovin’s solutions, from its Live Encoder to its Player and Analytics, offer a way to address the immediate challenges of delivering high-quality, engaging content and strategically position yourself for sustainable growth and success in a competitive marketplace.

Bitmovin’s advanced encoding solutions, cloud-agnostic live encoding capabilities, and comprehensive analytics provide you with the tools you need to optimize your workflows, enhance viewer experiences, and make informed decisions that drive revenue and growth. By leveraging these technologies, you can minimize your TCO while maximizing your reach and impact, ensuring you can quickly adapt to market changes and viewer demands. In a landscape where viewer expectations are constantly rising, having the right technological partners is more than just a tactical choice, it’s a strategic necessity that can define a platform’s trajectory and success in the streaming ecosystem.

The post Financial Considerations for Video Streaming Workflows appeared first on Bitmovin.

NVIDIA GTC Video Streaming Workflow Highlights

NVIDIA GTC (GPU-Technology Conference) is an annual conference with training and exhibition for all aspects of GPU(Graphics Processing Unit) accelerated computing. GTC 2024 was held in March with the tagline “The Conference for the Era of AI” and as expected, generative AI was a huge focus this year. There were also several other emerging applications of AI including advanced robotics, autonomous vehicles, climate modeling and new drug discovery. 

When GPUs were first introduced, they were mainly used for rendering graphics in video game systems. In the mid-late ‘90s, NVIDIA’s programmable GPUs opened up new possibilities for accelerated video decoding and transcoding workflows. Even though GPUs may now be more associated with powering AI solutions, they still play an important role for many video applications and there were several sessions and announcements covering the latest video-related updates at GTC24. Keep reading to learn more about the highlights. 

Video technology updates

In a session titled NVIDIA GPU Video Technologies: New Features, Improvements, and Cloud APIs, Abhijit Patait, Sr. Director of Multimedia and AI at NVIDIA, shared the latest updates and new features available for processing video with their GPUs. Some highlights that are now available in NVIDIA’s Video Codec SDK 12.2:

• 15% quality improvement for HEVC encoding, thanks to several enhancements:
  • UHQ (Ultra-high quality) tuning info for latency-tolerant use cases
  • Increased lookahead analysis
  • Temporal filtering for noise reduction
  • Unidirectional B-frames for latency-sensitive use cases
• Encode 8-bit content as 10-bit for higher quality (HEVC and AV1)

There were also several “Connect with Experts” sessions held where attendees could meet and ask questions of various NVIDIA subject matter experts. In the Building Efficient Video Transcoding Pipelines Enabling 8K session, they shared how multiple NVENC instances can be used in parallel for split-frame encoding to speed up 8K transcoding workflows. This topic is also covered in detail in their developer blog here. 

VMAF-CUDA: Faster video quality analysis

Snap and NVIDIA gave a joint presentation around a collaborative project they worked on (including participation from Netflix) to optimize and implement VMAF (Video Multi-Method Assessment Fusion) quality calculations on NVIDIA CUDA cores. CUDA (Compute Unified Device Architecture) cores are general-purpose processing units available on NVIDIA GPUs that allow for parallel processing and applications that are complementary to the dedicated GPU circuits.

During the talk, they explained how implementing VMAF-CUDA enabled Snap to run their video quality assessments in parallel to the transcoding being done on NVIDIA GPUs. The new method runs several times faster and more efficiently than running VMAF on CPU instances. It was so successful that Snap is now planning to transition all VMAF calculations to GPUs, even for transcoding workflows that are CPU-based. They also published the technical details in this blog post for those interested in learning more. 

Netflix Vision AI workflows

In a joint presentation by Netflix and NVIDIA, Streamed Video Processing for Cloud-Scale Vision AI Services, they shared how Netflix is using computer vision and AI at scale throughout their stack. Netflix is a bit unique not only in their massive scale, but also that they are vertically integrated and have people working on every part of the chain from content creation through distribution. This opens a lot of opportunities for using AI along with the challenge of deploying solutions at scale. 

They shared examples from: 

• Pre-production: Storyboarding, Pre-visualization
• Post-production: QC, Compositing and visual fx, Video search
• Promotional media: Generating multi-format artwork, posters, trailers; Synopsis 
• Globalization/localization of content: Multi-language subtitling and dubbing

They also discussed the pros and cons of using an off-the-shelf framework like NVIDIA’s DeepStream SDK for computer vision workflows (ease of use, efficiency of set up) vs building your own modular workflow (customization, efficiency of use)  with components like CV-CUDA Operators for pre- and post-processing of images and TensorRT for deep-learning inference. 

They also went into some detail on one application of computer vision in the post-production process, where they used object detection to identify when the clapperboard appeared in footage and sync the audio with the moment it closed, with sub-frame precision. This is something that has been a tedious, manual process for editors for decades in the motion picture industry and now they are able to automate it with consistent, precise results. While this is really more on the content creation side, it’s not hard to imagine how this same method could be used for automating some QA/QC processes for those on the content processing and distribution side. 

Ready to try GPU encoding in the cloud?

Bitmovin VOD Encoding now supports the use of NVIDIA GPUs for accelerated video transcoding. Specifically, we use NVIDIA T4 GPUs on AWS EC2 G4dn instances, which are now available to our customers simply by using our VOD_HARDWARE_SHORTFORM preset. This enables incredibly fast turnaround times using both H.264 and H.265 codecs. For time-critical short form content like sports highlights and news clips, it can make a huge difference. You can get started today with a Bitmovin trial and see the results for yourself. 

Related Links

Blog: GPU Acceleration for cloud video encoding

Guide: How to create an encoding using hardware acceleration 

Data Sheet: Raise the bar for short form content 

The post NVIDIA GTC24: Highlights for Video Streaming Workflows appeared first on Bitmovin.

What is SCTE?

The acronym SCTE is short for The Society of Cable and Telecommunications Engineers. SCTE is a non-profit professional organization that creates technical standards and educational resources for the advancement of cable telecommunications engineering and the wider video industry. When talking about it, you may hear people abbreviate SCTE with the shorthand slang “Scutty”. 

SCTE was founded in 1969 as The Society of Cable Television Engineers, but changed its name in 1995 to reflect a broader scope as fiber optics and high-speed data applications began playing a bigger role in the cable tv industry and became the responsibility of its engineers. Currently there are over 19,000 individual SCTE members and nearly 300 technical standards in their catalog, including SCTE-35, which will be the focus of this post.  

What is SCTE-35?

SCTE-35 was first published in 2001 and is the core signaling standard for advertising and program control for content providers and content distributors. It was initially titled “Digital Program Insertion Cueing Message for Cable” but recent revisions have dropped “for Cable” as it has proven useful and versatile enough to be extended to OTT workflows and streaming applications. There have been several revisions and updates published to incorporate member feedback and adapt to advancements in the industry, most recently on Nov, 30 2023.

SCTE-35 signals are used to identify national and local ad breaks as well as program content like intro/outro credits, chapters, blackouts, and extensions when a live program like a sporting event runs long. Initially, these messages were embedded as cue tones that dedicated cable tv hardware or equipment could pick up and enable downstream systems to act on. For modern streaming applications, they are usually included within an MPEG-2 transport stream PID and then converted into metadata that is embedded in HLS and MPEG-DASH manifests. 

SCTE-35 markers and their applications for streaming video

While SCTE-35 markers are primarily used for ad insertion in OTT workflows, they can also signal many other events that allow an automation system to tailor the program output for compliance with local restrictions or to improve the viewing experience. Let’s take a look at some common use cases and benefits of using SCTE-35 markers.

Use cases and benefits of SCTE-35

Ad Insertion – As mentioned above, inserting advertisements into a video stream is the main use case for SCTE-35 markers. They provide seamless splice points for national, local and individually targeted dynamic ad replacement. This allows for increased monetization opportunities for broadcasters and content providers by enabling segmenting of viewers into specific demographics and geographic locations. When ad content can be tailored for a particular audience, advertisers are willing to pay more, leading to higher revenue for content providers and distributors. 

Program boundary markers – Another common use case is to signal a variety of program boundaries. This includes the start and end of programs, chapters, ad breaks and unexpected interruptions or extensions. Many of these become particularly useful in Live-to-VOD scenarios. Ad break start/end markers can be used as edit points in a post-production workflow to automate the removal of ads for viewers with ad-free subscriptions. A program end marker can be used to trigger the next episode in a series for binge viewing sessions if so desired. All of these markers open new possibilities for improving the user experience and keeping your audience happy and engaged.  

Blackouts and alternate content – Another less common, but important use case is to signal blackouts, when a piece of content should be replaced or omitted from a broadcast. This often applies to regional blackouts for sporting events. Respecting blackout restrictions is crucial for avoiding fines and loss of access to future events. Using SCTE-35 allows your automation system to take control and ensure you are compliant.

Types of SCTE-35 markers

SCTE-35 markers are delivered in-band, meaning they are embedded or interleaved with the audio and video signals. There are five different command types defined in the specification. The first 3 are legacy commands: splice_null(), splice_schedule() and splice_insert(), but splice_insert() is still used quite often. The bandwidth_reservation() command may be needed in some satellite transmissions, but the most commonly used command with modern workflows is time_signal(). Let’s take a closer look at the 2 most important command types, splice_insert and time_signal. 

splice_insert commands

Splice_insert commands are used to mark splice events, when some new piece of content like an ad should be inserted in place of the program or a switch from an ad break back into the main program. Presentation time stamps are used to note the exact timing of the splice, enabling seamless, frame accurate switching.  

time_signal commands

Time_signal commands can also be used to insert new content at a splice point, but together with segmentation descriptors, they can handle other use cases like the program boundary markers mentioned above. This enables the segmenting and labeling of content sections for use by downstream systems.

Using SCTE-35 markers in streaming workflows

MPEG-2 transport streams

In MPEG-2 transport streams, SCTE markers are carried in-band on their own PID within the transport stream mux. These streams are usually used as contribution or backhaul feeds and in most cases are not directly played by the consumer. They may be delivered over dedicated satellite or fiber paths or via the public internet through the use of streaming protocols like SRT or proprietary solutions like Zixi.

HLS 

The Bitmovin Live Encoder supports a range of different HLS tags that are written when SCTE-35 triggers are parsed from the MPEG-TS input stream. Multiple marker types can be enabled for each HLS manifest. Which marker types to use depends on the consumer of the HLS manifest. An example consumer would be a Server Side Ad Insertion (SSAI) service. They usually state in their documentation which HLS tags they support for signaling SCTE-35 triggers.

• EXT_X_CUE_OUT_IN: Ad markers will be inserted using #EXT-X-CUE-OUT and #EXT-X-CUE-IN tags.

• EXT_OATCLS_SCTE35: Ad markers will be inserted using #EXT-OATCLS-SCTE35 tags. They contain the base64 encoded raw bytes of the original SCTE-35 trigger.
• EXT_X_SPLICEPOINT_SCTE35: Ad markers will be inserted using #EXT-X-SPLICEPOINT-SCTE35 tags. They contain the base64 encoded raw bytes of the original SCTE-35 trigger.
• EXT_X_SCTE35: Ad markers will be inserted using #EXT-X-SCTE35 tags. They contain the base64 encoded raw bytes of the original SCTE-35 trigger.
• EXT_X_DATERANGE: Ad markers will be inserted using #EXT-X-DATERANGE tags as specified in the HLS specification. They contain the ID, start timestamp, and hex-encoded raw bytes of the original SCTE-35 trigger.

Example HLS manifest with Cue Out, duration and Cue In tags:

#EXTINF:4.0,
2021-07/video/hls/360/seg_18188.ts
#EXT-X-CUE-OUT:120.000
…
#EXTINF:4.0,
2021-07/video/hls/360/seg18218.ts
#EXT-X-CUE-IN

Example HLS manifest using EXT-OATCLS-SCTE35 tag with base64 encoded marker:

#EXTINF:4.0,
2021-07/video/hls/360/seg_18190.ts
#EXT-OATCLS-
SCTE35:/DBcAAAAAAAAAP/wBQb//ciI8QBGAh1DVUVJXQk9EX+fAQ5FUDAxODAzODQwMDY2NiEEZAIZQ1VFSV0JPRF/3wABLit7AQVDMTQ2NDABAQEKQ1VFSQCAMTUwKnPhdcU=

Note: You can copy the base64 encoded marker above, (beginning with the first / after SCTE35: ) and paste it into this payload parser to see the full message structure.

MPEG-DASH 

In MPEG-DASH streams, SCTE-35 defined breaks and segments are added as new periods to the .mpd file.

<MPD>
<Period start="PT0S" id="1">
   <!-- Content Period -->
</Period>

<Period start="PT32S" id="2">
    <!-- Ad Break Period -->
   <EventStream timescale="90000"
    schemeIdUri="urn:scte:scte35:2014:xml+bin">
     <Event duration="2520000" id="1">
       <Signal xmlns="urn:scte:scte35:2013:xml">
         <Binary>      /DAlAAAAAAAAAP/wFAUAAAAEf+/+kybGyP4BSvaQAAEBAQAArky/3g==
         <Binary>
       </Signal>
      </Event>
    </EventStream> 
</Period>

<Period start="PT60S" id="3"> 
   <!-- Content Period -->

With SCTE messages embedded in the stream, various forms of automation can be triggered, whether it’s server or client-side ad insertion, content switching, interactive elements in the application or post-production processing.

Bitmovin Live Encoding SCTE Support

SCTE message pass-through and processing 

Bitmovin supports the parsing of SCTE-35 triggers from MPEG-TS input streams for Live Encodings, the triggers are shown below as splice decisions. The triggers are then mapped to HLS manifest tags.

Splice Decisions

Certain SCTE-35 triggers signal that an advertisement or break (to from the original content starts or ends. The following table describes how the Bitmovin Live Encoder treats SCTE-35 trigger types and SCTE-35 Segmentation Descriptor types as splice decision points, and the compatibility of those types with the different command types, Spice Insert and Time Signal.

✓= Supported

= Not currently supported

Segmentation UPID Type (Start/End)	Descriptor Type Name	SPLICE_INSERT	TIME_SIGNAL
–		✓
0x10, 0x11	PROGRAM	✓
0x20, 0x21	CHAPTER	✓
0x22, 0x23	BREAK	✓	✓
0x30, 0x31	PROVIDER_ADVERTISEMENT	✓	✓
0x32, 0x33	DISTRIBUTOR_ADVERTISEMENT	✓	✓
0x34, 0x35	PROVIDER_PLACEMENT_OPPORTUNITY	✓	✓
0x36, 0x37	DISTRIBUTOR_PLACEMENT_OPPORTUNITY	✓	✓
0x40, 0x41	UNSCHEDULED_EVENT	✓
0x42, 0x43	ALTERNATE_CONTENT_OPPORTUNITY	✓
0x44, 0x45	PROVIDER_AD_BLOCK	✓
0x46, 0x47	DISTRIBUTOR_AD_BLOCK	✓
0x50, 0x51	NETWORK	✓

Live cue point insertion API

In addition to the SCTE-35 pass-through mode, Bitmovin customers can insert new ad break cue points in real-time, using live controls in the user dashboard or via API. These can be inserted independently of existing SCTE-35 markers in the input stream and may be useful for live events when the time between ads is variable depending on breaks in the action. This allows streamers that don’t have SCTE-35 markers embedded in their source to take advantage of the same downstream ad insertion systems for increased monetization.

API Call:

POST /encoding/encodings/{encoding_id}/live/scte-35-cue
    {
      "cueDuration": 60, // duration in seconds between cue tags (ad break length)
    }

The #EXT-X-CUE-OUT tag will be inserted into the HLS playlist, signaling the start and duration of a placement opportunity to the DAI provider. Based on the cueDuration and the segment length, the #EXT-X-CUE-IN tag will be inserted after the configured duration and the ad opportunity will end, continuing the live stream.

HLS manifest with Cue Out, duration and Cue In tags inserted via the API call above:

#EXTINF:4.0,
    2021-07-09-13-18-34/video/hls/360_500/segment_18188.ts
    #EXT-X-CUE-OUT:60.000
    #EXTINF:4.0,
    2021-07-09-13-18-34/video/hls/360_500/segment_18189.ts
    ...
    #EXTINF:4.0,
    2021-07-09-13-18-34/video/hls/360_500/segment_18203.ts
    #EXT-X-CUE-IN
    #EXTINF:4.0,
    2021-07-09-13-18-34/video/hls/360_500/segment_18204.ts

Want to get started using SCTE-35 in your streaming workflow? Get in touch to let us know how we can help.

Resources

Tutorial: Bitmovin Live Encoding with SCTE-35, HLS and SSAI

Guide: Bitmovin Live Encoding and AWS MediaTailor for SSAI

Guide: Bitmovin Live Encoding with Broadpeak.io for SSAI

SCTE website 

SCTE-35 specification

SCTE-35 payload parser

Bitmovin Live Encoding data sheet

The post The Essential Guide to SCTE-35  appeared first on Bitmovin.