Tiny Tiny RSS: Community

PDO is coming, here's what you need to know


#82

I’ve updated to the latest master and it works now :slight_smile: Thanks!


#84

newsplus plugin author here. Since I’m not running anymore on git and I haven’t been following discourse, I am caught off-guard by the PDO switch :slight_smile:

@fox as I can’t (currently) switch to git, could you recommend me a way to change the plugin so that it works both pre-PDO and with PDO? FYI the plugin was basically a copy paste of the “queryFeedHeadlines” function with a tiny wrapper around. I’ll probably look at the new code for such function, so I’m more interested in the “how to make it work in both enviromments”.

(“Go screw yourself” in 3… 2… 1…)


#85

well for example you can check if DB::pdo is a method and choose accordingly

unless you need to change something inside it you’d be better off just calling the stock one


#86

Are there any news about the api_newsplus plugin?


#87

You could find more news about the plugin issue on the GitHub issue I opened.
TLDR: A commit was picked from a fork to solve the issue. Not sure it’s the best way to fix it but it “should work”.


#88

Got error:

PHP Fatal error:  Uncaught PDOException: There is no active transaction in /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php:1159
Stack trace:
#0 /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php(1159): PDO->commit()
#1 /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php(185): RSSUtils::update_rss_feed('701', true, false)
#2 /home/rick/nginx/files/ibragimov.by/root/reader/update.php(168): RSSUtils::update_daemon_common()
#3 {main}
  thrown in /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php on line 1159

Details:

PDOStatement::execute(): SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xF0\x9F\x91\x8D! ...' for column 'title' at row 1

Reproducible with following feeds:

https://www.youtube.com/feeds/videos.xml?channel_id=UC6C-UDj_OxkC1VmvCi56t8w
https://www.youtube.com/feeds/videos.xml?channel_id=UCIGRrUgnJBRk38tIgInEWjA
https://www.youtube.com/feeds/videos.xml?channel_id=UC-NGUu-btrlaa5BJLIRTPww

#89

this could be an encoding issue with your database.

e: i can’t reproduce your errors on either mysql or postgres here.


#90
/usr/bin/php /home/rick/nginx/files/ibragimov.by/root/reader/update.php --debug-feed 1239
[19:21:51/2717] Lock: update.lock
[19:21:51/2717] start
[19:21:51/2717] local cache will not be used for this feed
[19:21:51/2717] last unconditional update request: 
[19:21:51/2717] maximum allowed interval for conditional requests exceeded, forcing refetch
[19:21:51/2717] fetching [https://www.youtube.com/feeds/videos.xml?channel_id=UC6C-UDj_OxkC1VmvCi56t8w] (force_refetch: 1)...
[19:21:52/2717] fetch done.
[19:21:52/2717] source last modified: 
[19:21:52/2717] saving cache/feeds/e860b81d8790486765726bf787fd6283e6dd696c.xml
[19:21:52/2717] language: english
[19:21:52/2717] processing feed data...
[19:21:52/2717] site_url: https://www.youtube.com/channel/UC6C-UDj_OxkC1VmvCi56t8w
[19:21:52/2717] feed_title: TeamFlightBrothers
[19:21:52/2717] checking favicon...
[19:21:54/2717] loading filters & labels...
**... skipped this**
[19:21:54/2717] 1 filters loaded.
[19:21:54/2717] processing articles...
[19:21:54/2717] guid 2,yt:video:CPnsW__xYSs / SHA1:b47335e077ab937871085f5e143c13ce6e8e542a
[19:21:54/2717] orig date: 1517219415
[19:21:54/2717] date 1517219415 [2018/01/29 09:50:15]
[19:21:54/2717] title 6'3" Jared Roth DUNKS are INSANE! 2018's Star Dunker!
[19:21:54/2717] link https://www.youtube.com/watch?v=CPnsW__xYSs
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 1132703d9ccd9854430671d70fabc0773efe39e8 [stored=1132703d9ccd9854430671d70fabc0773efe39e8]
[19:21:54/2717] stored article seems up to date [IID: 371157], updating timestamp only
[19:21:54/2717] guid 2,yt:video:j-838wFCKXM / SHA1:72c34e09311aae1536ad015730594bdea3e706e0
[19:21:54/2717] orig date: 1517249653
[19:21:54/2717] date 1517249653 [2018/01/29 18:14:13]
[19:21:54/2717] title DJ Stephens EASTBAY DUNK off VERT.... IN ALL STAR GAME!!!!
[19:21:54/2717] link https://www.youtube.com/watch?v=j-838wFCKXM
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: f361d39122493b4d9a5173630f8fc34c8fb6e3c8 [stored=f361d39122493b4d9a5173630f8fc34c8fb6e3c8]
[19:21:54/2717] stored article seems up to date [IID: 371158], updating timestamp only
[19:21:54/2717] guid 2,yt:video:I_BbISuvaR8 / SHA1:1acc86c9d74ba98fad9fec425839cd5c0df03840
[19:21:54/2717] orig date: 1517221589
[19:21:54/2717] date 1517221589 [2018/01/29 10:26:29]
[19:21:54/2717] title DJ Stephens DUNKS over 7'2" Fredrick Weis in Dunk Contest!
[19:21:54/2717] link https://www.youtube.com/watch?v=I_BbISuvaR8
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 28539b3fedc28370d09a162fd4e3b538d431fe55 [stored=28539b3fedc28370d09a162fd4e3b538d431fe55]
[19:21:54/2717] stored article seems up to date [IID: 371159], updating timestamp only
[19:21:54/2717] guid 2,yt:video:4stlyaywOTE / SHA1:4d3d90b0a3024dddad8d6953ff87a389e81976bf
[19:21:54/2717] orig date: 1516452724
[19:21:54/2717] date 1516452724 [2018/01/20 12:52:04]
[19:21:54/2717] title Will Bunton's 50" Vertical Leap & Dunks put to the test on 10'2" Hoops!
[19:21:54/2717] link https://www.youtube.com/watch?v=4stlyaywOTE
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 435d7ad5cf64f1fb0cce7ddb95a2395715a5cd9f [stored=435d7ad5cf64f1fb0cce7ddb95a2395715a5cd9f]
[19:21:54/2717] stored article seems up to date [IID: 371160], updating timestamp only
[19:21:54/2717] guid 2,yt:video:pIK7pKZOo0c / SHA1:9ad9cf88cf7223c0cdf733a8c14dce42f43569d0
[19:21:54/2717] orig date: 1516462966
[19:21:54/2717] date 1516462966 [2018/01/20 15:42:46]
[19:21:54/2717] title OMG! 5'9" Kid DUNKS ALL OVER DEFENDER!
[19:21:54/2717] link https://www.youtube.com/watch?v=pIK7pKZOo0c
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 6bc7b19309980c8d045dcaab1816cdaeaafd6329 [stored=6bc7b19309980c8d045dcaab1816cdaeaafd6329]
[19:21:54/2717] stored article seems up to date [IID: 371161], updating timestamp only
[19:21:54/2717] guid 2,yt:video:ltXL-pjt02E / SHA1:bf6d4510a5d5778f892021ed8c171ee5d240520c
[19:21:54/2717] orig date: 1516947668
[19:21:54/2717] date 1516947668 [2018/01/26 06:21:08]
[19:21:54/2717] title Brooklyn JAM DUNK CONTEST featuring Guy Dupuy, Max Pierce and John Carter!
[19:21:54/2717] link https://www.youtube.com/watch?v=ltXL-pjt02E
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: d259cc420de8bfdd662a97ae2d48372d86a11132 [stored=d259cc420de8bfdd662a97ae2d48372d86a11132]
[19:21:54/2717] stored article seems up to date [IID: 371162], updating timestamp only
[19:21:54/2717] guid 2,yt:video:9etGuEoBBnE / SHA1:f46a0aafaca6e6db067d1655b1e6875e4a445965
[19:21:54/2717] orig date: 1516247010
[19:21:54/2717] date 1516247010 [2018/01/18 03:43:30]
[19:21:54/2717] title Haashim Wallace's NASTY DUNK DESTROYS Defender!
[19:21:54/2717] link https://www.youtube.com/watch?v=9etGuEoBBnE
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 56838aa42ae45e5008e4d0dfb601cb15e0366214 [stored=56838aa42ae45e5008e4d0dfb601cb15e0366214]
[19:21:54/2717] stored article seems up to date [IID: 371163], updating timestamp only
[19:21:54/2717] guid 2,yt:video:ft5xqyHRlLw / SHA1:e53c18519e4caff109b62c61acacd3711ede2cec
[19:21:54/2717] orig date: 1517251124
[19:21:54/2717] date 1517251124 [2018/01/29 18:38:44]
[19:21:54/2717] title Chris Staples "SALT BAE" Dunk & 360 Eastbay with 2 HANDS!  DL2 LINK IN BIO!
[19:21:54/2717] link https://www.youtube.com/watch?v=ft5xqyHRlLw
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 2cfa68401c6a5bd082d9875fd1e417eaac519a52 [stored=2cfa68401c6a5bd082d9875fd1e417eaac519a52]
[19:21:54/2717] stored article seems up to date [IID: 371164], updating timestamp only
[19:21:54/2717] guid 2,yt:video:KnPIR53h-7g / SHA1:3d0e54790828e1fe6a55731b5237e9be91787054
[19:21:54/2717] orig date: 1516392318
[19:21:54/2717] date 1516392318 [2018/01/19 20:05:18]
[19:21:54/2717] title DUNK LEAGUE 2 | Best DUNK CONTEST EVER!
[19:21:54/2717] link https://www.youtube.com/watch?v=KnPIR53h-7g
[19:21:54/2717] author TeamFlightBrothers
[19:21:54/2717] num_comments: 0
[19:21:54/2717] looking for tags...
[19:21:54/2717] tags found: 
[19:21:54/2717] done collecting data.
[19:21:54/2717] article hash: 2cc905b46363039b79e9c1903108b0323bb1e760 [stored=]
[19:21:54/2717] hash differs, applying plugin filters:
[19:21:54/2717] ... Af_Unburn
[19:21:54/2717] === 0.0000 (sec)
[19:21:54/2717] plugin data: af_unburn,
[19:21:54/2717] matched filter rules: 
[19:21:54/2717] filter actions: 
[19:21:54/2717] article labels:
[19:21:54/2717] force catchup: 
[19:21:54/2717] base guid [2,yt:video:KnPIR53h-7g or SHA1:3d0e54790828e1fe6a55731b5237e9be91787054] not found, creating...
[19:21:54/2717] base guid found, checking for user record
[19:21:54/2717] initial score: 0 [including plugin modifier: 0]
[19:21:54/2717] user record not found, creating...
[19:21:54/2717] resulting RID: 371171, IID: 362470
[19:21:54/2717] assigning labels [other]...
[19:21:54/2717] assigning labels [filters]...
[19:21:54/2717] looking for enclosures...
[19:21:54/2717] article enclosures:
Array
(
    [0] => Array
        (
            [0] => https://www.youtube.com/v/KnPIR53h-7g?version=3
            [1] => application/x-shockwave-flash
            [2] => 
            [3] => Who will be Crowned the Best Dunker in the World?  

10 Dunkers 🚀
16 Challenges 💪
$50,000 on the line💰 

Find out 12/5 - Free on Go90
            [4] => 640
            [5] => 390
        )

)
[19:21:54/2717] filtered article tags:
Array
(
)
[19:21:54/2717] article processed
PHP Fatal error:  Uncaught PDOException: There is no active transaction in /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php:1159
Stack trace:
#0 /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php(1159): PDO->commit()
#1 /home/rick/nginx/files/ibragimov.by/root/reader/update.php(417): RSSUtils::update_rss_feed('1239')
#2 {main}
  thrown in /home/rick/nginx/files/ibragimov.by/root/reader/classes/rssutils.php on line 1159

So there are obvious problem with saving enclosure with emoji. So since my mysql tables has utf8_general_ci encoding, I changed they to utf8mb4_unicode_ci and aslo changed collation for ttrss_enclosures columns.

So my table looks like:

But anyway I get same error, with saving of enclosure.


#91

This was discussed at great length on the old forum. Most 4-byte characters are filtered when saving to MySQL; however, not everything.

I won’t repeat what was already discussed but you’re welcome to read the old threads and reply back if you have more questions, etc.


#92

yep, looks like the enclosures are not filtered for utf8mb4 characters on mysql, should be an easy fix.

btw this isn’t really PDO related :shrug:


#93

Okay, just not sure until now is it actually problem with encoding or smth else. I think it’s time to migrate to postgres. Thanks


#94

it’s not really a problem per se, more of an unexpected mysql-specific thing which required a workaround.

https://git.tt-rss.org/fox/tt-rss/commit/102a01354b4f872a12feba3c9e875d9b0cd9dc36

this should probably fix it. too lazy to check on mysql right now.

e: i probably need to unfuck indenting everywhere since phpstorm insists on using spaces :confused:


#95

Thanks, this actually fixes my problem.

To enforce phpstorm using tabs/spaces you can use editorconfig http://editorconfig.org/

For example add .editorconfig in project root:

[*]
end_of_line = lf
insert_final_newline = true

[*.php]
indent_style = tab

#96

i was hoping i wouldn’t need to add this but it looks like the least painful option