[Discussion] Huge bandwidth consumption

[Jerroder]

Describe the bug

I am self-hosting Invidious at home, with a 1Gbps fiber and with OPNsense as a firewall/router. The screenshot below is a traffic report of 450Mbps coming to the WAN interface and then Invidious' IP (2001:(...)9c) answers back at 361Mbps. It already happened before when I only had IPv4. This happens every few seconds to every few minutes.
Now this isn't someone spamming or DDoSing because I use a reverse proxy, so that traffic would be forwarded to Nginx' IP, this is really directly to invidious, as if it made the request first and this is a response? However the responses are going to the SLAAC address (::ff:fe::) and not a privacy address so that's weird.
I looked up the IPs it's coming from and it seems to come from my ISP, and I've had this problem with 2 ISPs so I don't believe it's something on their end.

This is obviously extremely hard to troubleshoot, I'm just curious if anyone has noticed the same behavior?

Steps to Reproduce

I have no idea

Logs

Screenshots

Additional context

Invidious is running on Debian 12, behind OPNsense 23.7.9.

[unixfox]

How many subscriptions do you have? Is this a single user instance? Do you have pubsub enabled?

[Jerroder]

I got 53 subscriptions and I'm the only user. Pubsub is disabled.

[unixfox]

Sounds like invidious just refreshing the channels every 30 minutes. You can change the frequency if you prefer to consume less traffic: https://github.com/iv-org/invidious/blob/master/config/config.example.yml#L324C2-L324C2

Your issue looks a lot like this one: #3234

Also make sure that you have decrypt_polling: false, otherwise you will run into #3234 (comment)

[Jerroder]

Sounds like invidious just refreshing the channels every 30 minutes. You can change the frequency if you prefer to consume less traffic: https://github.com/iv-org/invidious/blob/master/config/config.example.yml#L324C2-L324C2

But is it supposed to consume so much bandwidth? 400Mbps over 15sec is 750MB, is it reloading all the thumbnails of all the channels?

Your issue looks a lot like this one: #3234

Ah, it didn't show up when I searched, didn't think about searching for "traffic", sorry!

Also make sure that you have decrypt_polling: false, otherwise you will run into #3234 (comment)

Not sure I understand this, is the decryption function's settings that big?

[absidue]

@rderensy No it's not loading thumbnails. Disable the full refresh option if you have it enabled, as that makes Invidious fetch all pages of videos for each channel you are subscribed to. YouTube paginates at 30 items and each response will be 200kb+ of JSON, if it's not compressed (I haven't checked if Invidious supports compression in it's requests to YouTube), because sending small responses is something YouTube doesn't know how to do. So it adds up quite quickly.

[Jerroder]

full_refresh is already false for me

[unixfox]

Do you take into account my last comment? Did it help?

[Jerroder]

I see the traffic peaks every few minutes, not every 30 minutes so I don't think it's subscriptions refreshing.

Also make sure that you have decrypt_polling: false, otherwise you will run into #3234 (comment)

That would prevent me from seeing music videos right? What would it change for the bandwidth?

[absidue]

@rderensy Pretty sure it doesn't work anymore anyway, so you might as well just disable it.

[unixfox]

Closing as a lot as changed since then, decrypt_polling is gone. Please comment here if you can still reproduce the issue.