Change on Tumblr: RSS feeds not working

[ontheair]

maybe someone should contact tumblr and ask them to unfuck their rss feeds

I already did that before opening this thread, of course But I have little hope that they understand the problem right and I doubt that they are really caring about their feeds. Lets see.

[fox]

i find it deeply ironic that getting a vps in russia could also be solution to this issue

[reibuehl]

@lllusion3418

Feedburner works with tumblr feeds, even in the EU, and even with NSFW feeds (it forces a safe-mode page on you otherwise if you’re not logged in), though I don’t know if it still gives Tiny Tiny RSS trouble.

Is there a way to use feedburner in a “transparent” way, meaning without creating new feeds in my own account for somenody elses content?

[ontheair]

@reibuehl I dont think so. I spent some hours now, creating all the feeds (a lot!) in feedburner and telling Tiny Tiny RSS about the new (feedburner-)URLs.

Its a lot of work, but thats the way to workaround the bulls**** that Tumblr has done.

@lllusion3418 thank you for the hint!

[fox]

wouldn’t it be easier to get a lowest tier VDS outside of EU? probably even something like openshift would work
put a simple noncaching http proxying script on it (that’s like 10 lines of code in php or w/e) and you’re done (well, limit it to tumblr rss feeds)
screwing around manually with feedburner looks like too much effort

[framasky]

I managed to find a temporary workaround (temporary since there is an exp value with a unix timestamp in the cookie).

1. Get a user-agent switcher addon for your browser. Set it to use your TTRSS user-agent (should be Tiny Tiny RSS/17.12 (a7265d9) (http://tt-rss.org/) with a7265d9 being a part of the git hashsum of the latest commit of your TTRSS).
2. Go to a tumblr RSS URL. Accept or say “later” on the consent page
3. Open your browser dev tools, get the content of the pfg cookie
4. Apply this patch to your TTRSS (the line numbers may differ from yours as my TTRSS is patched for other purposes) and change the value of the cookie for yours:

diff --git a/include/functions.php b/include/functions.php
index 7b4747e9..fa737f65 100755
--- a/include/functions.php
+++ b/include/functions.php
@@ -383,9 +383,15 @@

                        $ch = curl_init($url);

+                       $c_headers = array();
                        if ($last_modified && !$post_query) {
-                               curl_setopt($ch, CURLOPT_HTTPHEADER,
-                                       array("If-Modified-Since: $last_modified"));
+                               array_push($c_headers, "If-Modified-Since: $last_modified");
+                       }
+                       if (preg_match('/tumblr\.com/', $url)) {
+                               array_push($c_headers, "Cookie: pfg=f1a5834399eeee1c85506760a5acbdb488884ff4cc755abc1cf5a923c182c525%23%7B%22eu_resident%22%3A1%2C%22exp%22%3A1558355685%7D%235789485527");
+                       }
+                       if (count($c_headers) > 0) {
+                               curl_setopt($ch, CURLOPT_HTTPHEADER, $c_headers);
                        }

                        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout ? $timeout : FILE_FETCH_CONNECT_TIMEOUT);

[shabble]

@framasky

Apply this patch

Think I’ll pass, ta.

[fox]

+ array_push($c_headers, "Cookie: pfg=f1a5834399eeee1c85506760a5acbdb488884ff4cc755abc1cf5a923c182c525%23%7B%22eu_resident%22%3A1%2C%22exp%22%3A1558355685%7D%235789485527");

also the rest of the post, jesus wept

[aeritir]

I hacked together a small shell script which accepts the stupid GDPR page, stores the cookie and fetches the feed afterwards. Works for me™

#!/usr/bin/env bash

DIR="$(dirname $0)"

# Base URL
FEED_URL="${1:-http://sarahcandersen.com/rss}"

# URL and payload for GDPR consent
CONSENT_URL="https://www.tumblr.com/svc/privacy/consent"
CONSENT_PAYLOAD="{ \"eu_resident\": true, \"gdpr_consent_core\": true, \"redirect_to\": \"${FEED_URL}\" }"

# Mhhhh, cookies
COOKIE_JAR="${DIR}/cookies.txt"

# Output file for the fetched feed
OUTPUT_FILE="${2:-${DIR}/feed.rss}"

curl --silent --cookie ${COOKIE_JAR} --cookie-jar ${COOKIE_JAR} \
    --request POST --header "Content-Type: application/json" \
    --data "${CONSENT_PAYLOAD}" "${CONSENT_URL}" --output /dev/null

echo "Getting feed"
curl --silent --location "${FEED_URL}" --cookie ${COOKIE_JAR} --cookie-jar ${COOKIE_JAR} --output ${OUTPUT_FILE}

[JustAMacUser]

Folks,

Rather than hacking the core, do one of two things:

1. What fox suggested and just pay a few dollars a year for a cheap VPS somewhere outside Europe to proxy requests.
2. Write a plugin to hook FETCH_FEED and handle the cookie thing for you. You could even add a section the to Edit Feed dialogue that would allow you to update the cookie string as needed from the web UI.

Either are way better and more long-term solutions than messaging with core functions.

[fox]

3. contact tumblr and explain that automatic systems that access RSS feeds are not EU citizens, are not covered by GDPR protections, and shouldn’t accept consent screens not designed for them

i mean at least fucking filter shit out by user agent, christ, aaargh, you terminal cases of CYA syndrome

[framasky]

@JustAMacUser

Write a plugin to hook FETCH_FEED and handle the cookie thing for you

Sure, but I was in a hurry: I managed two TTRSS with 250 users each and I didn’t want to wait too much, otherwise I sure would have had messages from users. I think it’s better to have a working quick-and-dirty patch than nothing.

Note that I posted it here only to help people before we could found a proper way to make it work or Tumblr stops making shit (I know my patch is dirty, that’s why I didn’t even suggested a merge request).

[avocette]

It would be really nice if someone managed to make a plugin out your code.

[Arvedui]

I barfed up this. It seems to work for me, but I’m not a php guy so the code might be hideous.

[fox]

looks good to me, nicely done

[conrad784]

I wanted to use your plugin, but I can’t subscribe to feeds.
Maybe this only works with old feeds and subscribing to new ones needs another hook?

[GregThib]

I done a plugin too.
I hope Tumblr will correct this crap.

Notes (improvement later, if Tumblr still s**t on european citienz): this plugin:

• don’t use vanilla cache
• don’t store cookies
• don’t permit modification via prefs_tab
• use Proxy & some TT-RSS settings

Here

[Kierun]

@Arvedui

I barfed up this.

Seems to work for me! Thank you.

@GregThib

I done a plugin too.

Any reason your and Arvedui’s plugin cannot merge into one? Just trying to avoid a plethora of plugins with similar names…

[GregThib]

No reasons for that. It seems we done that work at the same time.
The idea is the same, but I just tried to respect tt-rss way of doing fetch.
Plus, I hooked SUBSCRIBE & BASICINFO callbacks.

But I hope it’s a matter of a week or two, time to Tumblr to learn how works machines.

[GregThib]

GDPR Plugin for Tumblr

Changed : added a pref tabs to add manually Tumblr with URL not of the *.tumblr.com form