Audio delay with RTMP/HTTP stream but not with RSTP

[katschub88]

Describe the problem you are having

I set up a mini pc (AMD 5700U, 32GB ram, 4TB nvme) as frigate server at a friends house (used the same hardware and setup several times before and they all work fine) and added a Reolink Doorbell V2 (Firmware: v3.0.0.4110_2410111119).

If I use the recommended settings for Reolink Doorbells (http+flv), there is an audio dalay in the recordings.
At first it seems to be synchronous, but after 6 hours there is a delay of about 0.5 seconds.
After 24 hours there is a delay of about 1 second.
After 3 days there is a delay of about 2 seconds.
So far it stayed under 3 seconds after running for over a week.

I have tried other options (see config), including HTTPS+FLV, RTMP and RTSP.
The only stream without any audio delay is RTSP, this seems to be the most reliable variant so far.

Unfortunately the RTSP stream causes occasional errors, which result in image corruption and these errors seem to get worse over time, up to the point where you have to restart the doorbell and frigate.

I've done a lot of research, but found no solution yet. Any idea what is causing the audio delay in the recommended FLV stream and how I can fix this?
Or if not, any idea what is causing the RTSP errors and how to fix these?
Anything would be fine, if the recordings keep running stable without major image corruption.

It doesn't seem to be a server problem, as the same setup works fine in other cases.
There are plenty of ressources left and it happens with the cleaned up "record-only" config (no detection, no zones, etc.) too.

Version

0.13, 0.14 and 0.15 (all the same behavior)

Frigate config file

...

ffmpeg:
  hwaccel_args: preset-vaapi  #<<<(Also tried without)

go2rtc:
  log:
    format: text
    level: "debug"
    exec: trace
  streams:
    doorbell_cam:
      - rtsp://user:password@ip/Preview_01_main
      #- "ffmpeg:rtmp://ip/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=password#video=copy#audio=copy#audio=opus"
      #- rtmp://ip/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=password
      #- "ffmpeg:http://ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=user&password=password#video=copy#audio=copy#audio=opus"
      #- "ffmpeg:https://ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=user&password=password#video=copy#audio=copy#audio=opus"


cameras:
  doorbell:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/doorbell
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
      output_args:
        record: preset-record-generic-audio-copy

    record:
      enabled: True
      retain:
        days: x
        mode: all

Relevant Frigate log output

No log entries with the RTMP/HTTP/HTTPS streams.

Different errors with RTSP (occasionally, seem to build up over time):
- error while decoding MB x xx, bytestream -xx
- left block unavailable for requested intra mode

I can see the "error while decoding MB" errors in the recordings. They show as blurry areas at the bottom of the screen and only last a fraction of a second. Some are smaller, others larger.
The "left block" errors are more severe and last longer.

Relevant go2rtc log output

No errors

Frigate stats

No response

Operating system

Other Linux

Install method

Proxmox via Docker

docker-compose file or Docker CLI command

Not relevant

Object Detector

CPU (no coral)

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[hawkeye217]

What firmware version are you running on the doorbell?

[katschub88]

As I've said in the original post, it's currently at the latest firmware (Firmware: v3.0.0.4110_2410111119).

But I've also tried different versions. In particular:

• 4110_2410111119
• 3308_2407315182
• 3215_2401262240

Always reset the configuration to start clean.

These are the latest errors from the current RTSP stream:

2025-02-15 18:36:18.149233801 [h264 @ 0x72992811c940] left block unavailable for requested intra4x4 mode -1
2025-02-15 18:36:18.149238610 [h264 @ 0x72992811c940] error while decoding MB 0 110, bytestream 29285

(go2rtc logs show no errors at all)

The video image is damaged about 10 seconds before the logging time. It affects about 40% of the total height (from the bottom) for about 2 seconds. The audio seems completely fine, no stuttering, no missing parts.

[katschub88]

I've read many of these threads and tried so many things mentioned there, this has been a problem for weeks now.
There are so many little details that might change the outcome, like 24/7 recording vs alerts only, doorbell variant and version, poe or wifi, firmware, ffmpeg version, go2rtc version, server hardware, networking, etc.

So far, the RTSP stream has the best quality, but has decoding errors.
I've noticed that the cpu usage of the Reolink Doorbell was about between 80+% with just streaming to frigate using the above config.
(Possible cause of these problems)

Anyway, I wanted to use the recommended stream, as mentioned in the frigate docs.
However, this stream introduced the audio delay. So I am looking for tweaks to fix this issue.
Or more input from people who have been using that doorbell without any issues for an extended period.

I've changed the config for testing like this yesterday:

go2rtc:
  streams:
    doorbell:
      - rtsp://user:password@ip:554/Preview_01_main

cameras:
  doorbell:
    ffmpeg:
      inputs:
        - path: https://ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=user&password=password
          input_args: preset-http-reolink
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/doorbell
          input_args: preset-rtsp-restream
          roles:
            - detect
      output_args:
        record: preset-record-generic-audio-copy

I'll have to wait until later today to get the results.
Will also try out the config you've posted and the preset-record-generic-audio-aac, although I might have already tried that before (I'm not sure).

[hawkeye217]

I've noticed that the cpu usage of the Reolink Doorbell was about between 80+% with just streaming to frigate

Yep, this is why the HTTP stream is recommended.

Let us know if you figure it out.

[katschub88]

Okay, so he said, there were some detection/stream errors yesterday after I've left, which crashed the detection process.
(Not sure if it automatically recovered or not. He copied the error text, but it was gone from the clipboard afterwards.)
He then applied some minor changes I suggested before and restarted frigate. So far, there have been no errors or audio delays in the recording and detections are generated as well.

The question is, if the detection stream is stable and working 100% of the time. There are detections, so the process seems to work in general. But is it possible that the detection stream (config above, using go2rtc) experiences decoding errors without having errors logged? Can we rely on the detection stream / process or is it possible that the recording is fine but the detection isn't working sometimes while the stream is broken without having any errors in the log file?

Also, what does this go2rtc log entry mean? Who is consuming the stream and when does this event happen?
It doesn't seem to be triggered by any view the user can open and there are about 40 entries now. Is this normal?

DBG [webrtc] new consumer src=doorbell

[hawkeye217]

Perhaps there are small errors that go2rtc is able to recover from and are not logged, but generally anything that cannot be recovered from or is severe enough to cause a disruption in the stream will be logged.

Also, what does this go2rtc log entry mean?

This just means that a new client has connected to the webrtc stream provided by go2rtc. It's a debug statement, so it's not of any concern.

[katschub88]

The latest config using the HTTP(S) stream for recording and RTSP for detection and live views was not working properly.
The network traffic doubled due to using 2 streams now, instead of one. The doorbell clearly had to do more work now.
Still, the logs showed no errors whatsoever, so at first glance it looked good.

However, we found video corruption in the recordings. We then pushed the server and network to its limits in order to force packet loss and image corruption, just for testing purposes. The corruption was massive (the whole image was broken) and it lasted for as long as we ran the tests. Still no errors in the logs. So it seems like these HTTP/RTMP streams create a bit of a false feeling of stability.

We went back to the original RTSP config and changed a few things. It's been running for 2 days now without any errors, but it's far too early to draw conclusions.