HTTP Redirects and caching

Tiny Tiny RSS Development
1

Hi,

I’m not hosting tiny-rss but I see tiny-rss in my logs.
I have identified two issues with the clients hitting my feed.

When tiny tiny RSS is requesting a feed, it might end up on a feed which change location.
In my case the feed is doing an HTTP 301, aka permanent redirect. Tiny Tiny RSS is following the instructions and goes fetch the feed. So far so good.

174.142.230.147 - - [02/Jan/2019:07:43:58 +0000] "GET /feed.rdf HTTP/1.1" 301 242 "-" "Tiny Tiny RSS/17.4 (http://tt-rss.org/)"
174.142.230.147 - - [02/Jan/2019:07:43:58 +0000] "GET /feed.atom HTTP/1.1" 200 89288 "-" "Tiny Tiny RSS/17.4 (http://tt-rss.org/)"

The issue is that it doesn’t remember it was a permanent redirect.
For example, this IP address does:

grep 174.142.230.147 access.log.1545868800 | grep rdf | grep '01/Jan' | wc -l

48 access in 24h, one every 30 minutes to the /feed.rdf before following the redirection.
It should probably remember it had received the instructions that the feed had changed location.

Second issue, the caching, the server is setting an ETAG and an expiration date. If the ETAG has not changed, Tiny Tiny RSS should not try reloading the resource.

HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: Keep-Alive
Content-Length: 89288
Content-Type: application/atom+xml;charset=utf-8
Date: Wed, 02 Jan 2019 07:51:46 GMT
ETag: "15cc8-57e5f6761aa40"
Expires: Wed, 09 Jan 2019 07:51:46 GMT
Keep-Alive: timeout=5, max=100
Last-Modified: Tue, 01 Jan 2019 06:11:45 GMT
Server: Apache/2.4.37 (FreeBSD)

from what I gather in my logs it seemts the caching issue has been fixed in the version 18.8+. That’s cool!

17.4 seems to not follow/understand the cache information.

It would be cool to fix the redirect issue too.

Thanks.

2

web browsers don’t auto update your bookmarks because of a redirect, and neither would tt-rss

i suggest taking a better hobby than reading nginx logs. a book would be more fulfilling, i think.

3

Just ban it. It’s ruining your website.

4

48 requests per day, serious shit right there

5

I just checked on of my servers, that’s about 4 seconds worth of traffic.

6

There may be an argument for alerting the user that a feed is 301’ing though, as there’s no guarantee the redirect will stay in place, and if it goes away the user has lost the feed.

Now I’m wondering if it’s already listed on the feed errors page, which I next to never check…

7

what if redirect was temporary or misconfigured or something else, just because this http code says permanent doesn’t mean people use it as such. it sounds to me like adding complexity for no good reason, potentially confusing users.

that said, this feature could be easily implemented via a plugin. if you want it, open your favorite text editor and start coding. :man_shrugging:

8

Users define the feed URL in TT-RSS. Users should be the ones to change them. This has been discussed here in the past but I’m on mobile now so I’m not going to look up the thread.

We have HTTP codes for temporary and permanent redirects, TT-RSS can follow these. When a site admin is satisfied with how long they have allowed the redirects they can switch it to 410/Gone and the TT-RSS user can follow-up with the new URL.

This isn’t a real problem that needs to be handled. Keep in mind many sites use crap like Feedburner and it makes sense to keep the site’s proper domain in TT-RSS instead of a third-party domain. We don’t want those sites reporting problems for their entire existence.