Content playback issues using Firefox, Audio gets fully missing (v1.5.20)

[rsmvdl]

What do you want to do with Hls.js?

I have an ABR stream that uses multiple codecs for audio and video, primarily AV1, AVC1, AAC, HE-AAC-V2, E-AC3 etc ... As with any other ABR stream, I also hold the video in multiple retentions like 360p, 720p, 1080p, 2160p etc ... Now when I play the content using HLS.js the actual playback is working great, but sometimes when the ABR stream switches to another retention/resolution of the video, the audio simply drops out completely. This happens especially often when seeking through the timeline of the video. The logs don't point out any error or warning using debug mode, the audio is simply getting lost when the resolution switches. And this appears to only happen with Firefox, while Chrome based browsers don't show this behavior at all ... Chrome, Edge etc. all working fine ...

What have you tried so far?

I already tested my streams using https://hlsjs.video-dev.org/, but the outcome was the same with Firefox, when I switch through the video retentions using the buttons of the demo player the audio also gets lost in almost any case when I seek. So this appears to be a content issue, not a player issue as it appears. But why is it working with Chrome and without any noticeable issues?

My first guess was that it might be a PTS issue due to a distributed encoding pipeline I run, but even when I encode the whole stream using a single ffmpeg command instead of many where a PTS issues might occur, I get the same outcome, the audio keeps dropping on seek ... So a PTS issue also does not appear to be the issue.

I also tried different input videos, same outcome.

Can someone maybe show me his FFmpeg command(s) to create an ABR stream. Would be awesome to have something at hand I can compare with.

Thanks in advance :)

[robwalch]

Can you share the HLS asset used to reproduce this issue?

[rsmvdl]

@robwalch Sure, please take a look at the following link:

https://tinyurl.com/272vdhtf

Just click through the quality levels at the "Quality-levels" tab ...

Thanks in advance

[robwalch]

https://tinyurl.com/272vdhtf

The asset URL is not valid or not accessible from my location. Can you provide a URL without spaces and special characters in the path?

[rsmvdl]

@robwalch Ahh sorry yes, that stuff is geo-blocked, I just enabled USA at my list, please try again.

[robwalch]

The Multivariant Playlist is invalid because it contains a Media Playlist Tag: #EXT-X-PLAYLIST-TYPE.

4.4.3. Media Playlist Tags

Media Playlist tags describe global parameters of the Media Playlist.
There MUST NOT be more than one Media Playlist tag of each type in
any Media Playlist.

A Media Playlist tag MUST NOT appear in a Multivariant Playlist.

For best compatibility, avoid starting VOD playlists with #EXT-X-DISCONTINUITY. That tag is only needed between media segments. Sometime it is used at the start of live playlists in the place of incrementing DISCONTINUITY-SEQUENCE, but that has been known to cause issues as well.

Which renditions are not audible in Firefox? Is it specific to the handling of "mp4a.40.29" or switching between that and "mp4a.40.2"? If that is the case you can either remove all levels with one of the other, or call hls.recoverMediaError() after switching variants as a workaround. We'll need to see if appending changeType for the HE-AAC v2 is needed for switching between these renditions and whether hls.js is extracting the segment sample format correctly.

[rsmvdl]

@robwalch: Good hint about the #EXT-X-DISCONTINUITY, I just dropped that and reencoded the clip, but still the same issue.

According to your question, which retentions aren't audible in Firefox, well it happens totally randomly that the audio fully drops, doesn't matter the codec, I was also kinda curious if it's about the audio codec switching or so, but it isn't as it appears. I have this Audio getting fully lost issue when switching the video tracks, like 1080p to 360p for example, while sticking to the audio codec like "mp4a.40.2". It does not happen every time, but it happens like every 3-4 time when switching the resolutions of the video (It's Flaky) ... In practice when you set quality to "auto" and you throttle your browsers' bandwidth by using the dev tools, the ABR controller will at some point choose a lower bitrate of the stream, here I also lose audio pretty often. But as I already pointed out, this issue only happens with Firefox, not with Chrome based browsers ... I'm on Linux (6.8.0-52) using Firefox 136.0 (64-bit) ... I also tried other versions of Firefox, but they have the same issue. Do you actually have the same issues with OS X and FireFox ? I assume you are on OS X, if not, please excuse me.

I also tried different video codecs like h.264, h.265 instead of AV1, but the outcome is exactly the same... The audio gets lost at some point when resolution/bitrates changes. Another funny discovery was using the "basic-usage" example of hls.js I found here -> https://hlsjs.video-dev.org/demo/basic-usage ... Of course, I don't have any audio selection or ABR switching stuff with this basic player setup, but the audio isn't getting lost or is getting async or anything like that, it just normally plays as expected ...

Another discovery was to use the "Recover Media Errors" button on the https://hlsjs.video-dev.org/demo page, this in most cases got the audio back. But that doesn't appear to be a good solution in production ^^.

Often when the Audio drops, I also get a BufferStaleError ... Not every time, but sometimes (Flaky)

Is it maybe possible that the ABR controller behaves differently with Firefox or so? It kinda feels like that ...
If you take a look at the attached image, you can also see that there is some kinda hole appearing in the timeline, the little black bar within the gray bar on the left side ...

No idea why that happens, but it also made the audio drop again ... :(