Fun behavior in cache_starred_images

Moral quandary in which I feel this edge case should be shared, at least so Fox can laugh at me.

I enabled cache_starred_images before a trip, so that anything I starred for later would be cached on my drive. I was also informed once I was home that “the internet was acting up,” and it sure was, intermittently giving pings of 400-1500+ ms for extended periods.

Long story shortened, my TTRSS machine was saturating my downstream by trying to cache a 261 MB video from Reddit every time a new update task was spawned. I’m not super knowledgeable with PHP, but as near as I can guess a 261 MB video doesn’t fit in a memory_limit of 128M when strlen() is called to compare against MIN_CACHE_FILESIZE, causing the cache action to fail, leading to no file being created, which is why it just kept trying. I’m also not sure where it dumps errors, if it even does, making detection more interesting.

So that was my pre-sleep entertainment.

Bear Tax.

lol

this needs a better implementation I guess
same probably goes for other caching plugins

Curl supports the CURLOPT_FILE option that saves a file directly to disk so memory limits aren’t an issue. I always thought it was a good approach but never came up with a simple way to implement it for a pull request.

i would prefer to limit via accept range tbh so we wouldn’t even try to download huge files

this needs some thinking over

i’m checking this and it seems there’s no fetch size restrictions in af_zz_imgproxy either, i’m gonna see if there’s a relatively painless way to limit this with a sane default and then apply to all caching functions

it’s easy to do this with curl via for example progress function but for normal file_get_contents() idk

e: now that i think about it it may be a good idea to automatically limit all tt-rss downloads with a (smaller) sane value so that we would never get into a situation where a multi-gigabyte feed or whatever else is downloaded which could be arguably called a (very stupid) DOS attack

this would cause repeated transfer attempts i guess but at least those would be clipped to something that wouldn’t take forever and possibly OOM tt-rss

https://git.tt-rss.org/fox/tt-rss/commit/b14f6d58b472e20b3e6cffd54916ea4fea526acd

unfortunately this currently only works for CURL. if someone has ideas on how to cleanly implement this with file_get_contents() i’m all ears