Tiny Tiny RSS: Community

Change on Tumblr: RSS feeds not working


#1

Hello,

this thread is not because there is a problem in Tiny Tiny RSS - Its working fine and I am using it since long years. Thank you for your great work!

This thread is about the fact that Tumblr changed something with its RSS feeds, causing problems. And maybe you can give me a hint to find a workaround.

The facts:
Tumblr-RSS feeds worked fine before. But now they made a change: When accessing any Tumblr-RSS-feed in any browser (for example: http://funnygamememes.tumblr.com/rss), there is a redirect, showing a page with privacy-related informations. After hitting the OK-button, the RSS-page is shown.

Because of this redirect Tiny Tiny RSS is not able anymore to read the feeds. I am searching for a way to override this OK-button, so the feeds are working again. Maybe anyone has an idea?

By the way: When checking the feed with https://fakecake.org/myfeedsucks/, it works. I dont really understand why…

Again: For sure its not the fault of Tiny Tiny RSS. It seems to be the changed behaviour of Tumblr.

My environment:
Tiny Tiny RSS v17.12 (4fa64e8)
MariaDB 10.1.23
Lighttpd 1.4.45
PHP 5.6.30
Debian Stretch


#2

this is probably gdpr
are you from eu by any chance?


#3

It’s working for me; I’m not in the EU.

If you have some technical skills, see if there is anything unique about the redirect (added query strings, cookies, etc.) and then modify the feed in TT-RSS accordingly.


#4

I have a handful of Tumblr feeds as well; both normal and login gated.

As far as testing with browser, I wouldn’t recommend it. Use feed debugger (f shift+d in TTRSS) and see what happens there. Especially, as JustAMacUser pointed out, what’s going on with the redirect URLs.

In a browser, Tumblr will still force a login for an RSS feed on a gated feed. TTRSS (or CURL, etc) will still fetch the feed without any questions.


#5

Same issue as the original poster. Works fine with a US IP address, doesn’t work with a EU one. Feed debugger:

[19:05:06/7669] start
[19:05:06/7669] local cache will not be used for this feed
[19:05:06/7669] last unconditional update request: 2018-05-19 19:04:19
[19:05:06/7669] stored last modified for conditional request: 
[19:05:06/7669] fetching [http://redacted.tumblr.com/rss] (force_refetch: 1)...
[19:05:07/7669] fetch done.
[19:05:07/7669] source last modified: 
[19:05:07/7669] fetch error: LibXML error 68 at line 21 (column 29): xmlParseEntityRef: no name

[19:05:07/7669] + LibXML error 68 at line 21 (column 29): xmlParseEntityRef: no name

[19:05:07/7669] + LibXML error 68 at line 21 (column 30): xmlParseEntityRef: no name

[19:05:07/7669] + LibXML error 68 at line 21 (column 41): xmlParseEntityRef: no name

[19:05:07/7669] + LibXML error 68 at line 21 (column 42): xmlParseEntityRef: no name

[19:05:07/7669] + LibXML error 76 at line 30 (column 313): Opening and ending tag mismatch: link line 10 and head

[19:05:07/7669] + LibXML error 68 at line 38 (column 1098): StartTag: invalid element name

[19:05:07/7669] + LibXML error 68 at line 38 (column 1482): StartTag: invalid element name

[19:05:07/7669] + LibXML error 68 at line 39 (column 81): StartTag: invalid element name

[19:05:07/7669] + LibXML error 38 at line 41 (column 111): Unescaped '<' not allowed in attributes values

[19:05:07/7669] + LibXML error 65 at line 41 (column 111): attributes construct error
…

#6

I forgot to mention. Yes, I am from EU and this problem is GDPR-related, right.

The OK-Button is not just a simple link, there is some script involved. And Im a noob in that matter… @mamil was faster and already provided the debugger log.

Ok, so this is just a problem for european users. I will try to understand how this redirect works (not very hopeful), maybe I will be able to give more informations.

Anyway, thank you for your answers already!

Edit:
When trying to access this page:
http://funnygamememes.tumblr.com/rss
Then its redirected to this page:
https://www.tumblr.com/privacy/consent?redirect=http%3A%2F%2Ffunnygamememes.tumblr.com%2Frss

Not sure if you are able to see the page outside the EU.


#7

The redirect disappears once Tumblr sets a cookie. The question is how to receive a cookie on a remote VPS without a browser. Annoying as hell.


#8

Yes, I was also thinking if its possible to catch the cookie and feed it to Tiny Tiny RSS :slight_smile:

Is Tiny Tiny RSS using curl? And is there a chance to set a cookie for curl? Maybe thats just an weird idea, but…


#9

The GDPR page promises to vanish once you log in. I’ve signed up to Tumblr and tried to login with curl hoping that it would set the cookie, but no luck so far. Here’s what I tried.


#10

Feedburner works with tumblr feeds, even in the EU, and even with NSFW feeds (it forces a safe-mode page on you otherwise if you’re not logged in), though I don’t know if it still gives Tiny Tiny RSS trouble.

You can the Firefox addon cliget to get the curl commandline (including cookies and everything) that works.

As the cookies don’t seem to change (at least if you’re logged in), you could use this to make a small proxy script on your webserver where you pass the url as a parameter, for example.


#11

I confirm. Thanks ever so much for the tip. I’ll stick with FeedBurner for now, hoping that Tumblr will eventually get its act together and remove the GDPR screen.


#12

maybe someone should contact tumblr and ask them to unfuck their rss feeds

e: we can of course also screw around with cookies and stuff


#13

maybe someone should contact tumblr and ask them to unfuck their rss feeds

I already did that before opening this thread, of course :wink: But I have little hope that they understand the problem right and I doubt that they are really caring about their feeds. Lets see.


#14

i find it deeply ironic that getting a vps in russia could also be solution to this issue


#15

Is there a way to use feedburner in a “transparent” way, meaning without creating new feeds in my own account for somenody elses content?


#16

@reibuehl I dont think so. I spent some hours now, creating all the feeds (a lot!) in feedburner and telling Tiny Tiny RSS about the new (feedburner-)URLs.

Its a lot of work, but thats the way to workaround the bulls**** that Tumblr has done.

@lllusion3418 thank you for the hint!


#17

wouldn’t it be easier to get a lowest tier VDS outside of EU? probably even something like openshift would work
put a simple noncaching http proxying script on it (that’s like 10 lines of code in php or w/e) and you’re done (well, limit it to tumblr rss feeds)
screwing around manually with feedburner looks like too much effort


#18

I managed to find a temporary workaround (temporary since there is an exp value with a unix timestamp in the cookie).

  1. Get a user-agent switcher addon for your browser. Set it to use your TTRSS user-agent (should be Tiny Tiny RSS/17.12 (a7265d9) (http://tt-rss.org/) with a7265d9 being a part of the git hashsum of the latest commit of your TTRSS).
  2. Go to a tumblr RSS URL. Accept or say “later” on the consent page
  3. Open your browser dev tools, get the content of the pfg cookie
  4. Apply this patch to your TTRSS (the line numbers may differ from yours as my TTRSS is patched for other purposes) and change the value of the cookie for yours:
diff --git a/include/functions.php b/include/functions.php
index 7b4747e9..fa737f65 100755
--- a/include/functions.php
+++ b/include/functions.php
@@ -383,9 +383,15 @@

                        $ch = curl_init($url);

+                       $c_headers = array();
                        if ($last_modified && !$post_query) {
-                               curl_setopt($ch, CURLOPT_HTTPHEADER,
-                                       array("If-Modified-Since: $last_modified"));
+                               array_push($c_headers, "If-Modified-Since: $last_modified");
+                       }
+                       if (preg_match('/tumblr\.com/', $url)) {
+                               array_push($c_headers, "Cookie: pfg=f1a5834399eeee1c85506760a5acbdb488884ff4cc755abc1cf5a923c182c525%23%7B%22eu_resident%22%3A1%2C%22exp%22%3A1558355685%7D%235789485527");
+                       }
+                       if (count($c_headers) > 0) {
+                               curl_setopt($ch, CURLOPT_HTTPHEADER, $c_headers);
                        }

                        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout ? $timeout : FILE_FETCH_CONNECT_TIMEOUT);

#19

Think I’ll pass, ta.


#20

+ array_push($c_headers, "Cookie: pfg=f1a5834399eeee1c85506760a5acbdb488884ff4cc755abc1cf5a923c182c525%23%7B%22eu_resident%22%3A1%2C%22exp%22%3A1558355685%7D%235789485527");

:man_facepalming:

also the rest of the post, jesus wept