MSI RSS not showing all articles in browser

Describe the problem you’re having:

Only a few articles are appearing on this RSS:
The RSS seems ok. fakecake is saying, that it is valid. Even the debug of tt-rss is showing all articles being fetched. But they just don’t appear…
(Since I can’t upload text files and can’t post links, I can’t send you the debug output. And the RSS link :roll_eyes:)

If possible include steps to reproduce the problem:

Add this RSS and let tt-rss fetch it:

tt-rss version (including git commit id):

v19.8 (6060963)

Platform (i.e. Linux distro, PHP, PostgreSQL, etc) versions:

vserver, gentoo, PHP 7.3.10, mysql

Please provide any additional information below:

I have some filters, but they aren’t active for this RSS. I even disabled every filter and force-refetched with the same exact result.

Jeez - what a hassle to get a RSS URL. The one concerned here appears to be

https://www.msi.com/rss/feed/product/mb/X470-GAMING-PLUS/Download

Which 307’s to

https://www.msi.com/rss/feed/product/mb/X470-GAMING-PLUS/Download?utm_source=msi-news-feed&utm_medium=rss&utm_campaign=msi-news&utm_content=product

Only a few articles are appearing on this RSS:

Only those few articles are being served up by that stream.

$ curl "https://www.msi.com/rss/feed/product/mb/X470-GAMING-PLUS/Download?utm_source=msi-news-feed&utm_medium=rss&utm_campaign=msi-news&utm_content=product"  | grep pubDate
            <pubDate>Thu, 07 Nov 19 00:00:00 +0800</pubDate>
            <pubDate>Wed, 09 Oct 19 00:00:00 +0800</pubDate>
            <pubDate>Mon, 23 Sep 19 00:00:00 +0800</pubDate>
            <pubDate>Thu, 15 Aug 19 00:00:00 +0800</pubDate>
            <pubDate>Wed, 31 Jul 19 00:00:00 +0800</pubDate>
            <pubDate>Mon, 22 Jul 19 00:00:00 +0800</pubDate>
            <pubDate>Tue, 25 Jun 19 00:00:00 +0800</pubDate>
            <pubDate>Thu, 18 Apr 19 00:00:00 +0800</pubDate>
            <pubDate>Mon, 18 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Mon, 11 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Mon, 11 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 08 Mar 19 00:00:00 +0800</pubDate>
            <pubDate>Thu, 14 Feb 19 00:00:00 +0800</pubDate>
            <pubDate>Thu, 10 Jan 19 00:00:00 +0800</pubDate>
            <pubDate>Fri, 23 Nov 18 00:00:00 +0800</pubDate>
            <pubDate>Mon, 05 Nov 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 05 Sep 18 00:00:00 +0800</pubDate>
            <pubDate>Fri, 10 Aug 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 25 Jul 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 25 Jul 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 17 May 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 16 May 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 16 May 18 00:00:00 +0800</pubDate>
            <pubDate>Mon, 23 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 12 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 12 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Tue, 10 Apr 18 00:00:00 +0800</pubDate>
            <pubDate>Fri, 30 Mar 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 29 Mar 18 00:00:00 +0800</pubDate>
            <pubDate>Wed, 28 Mar 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 15 Mar 18 00:00:00 +0800</pubDate>
            <pubDate>Thu, 28 Dec 17 00:00:00 +0800</pubDate>
            <pubDate>Wed, 14 Jun 17 00:00:00 +0800</pubDate>
            <pubDate>Wed, 25 Jan 17 00:00:00 +0800</pubDate>
            <pubDate>Fri, 06 Jan 17 00:00:00 +0800</pubDate>
            <pubDate>Fri, 29 Jul 16 00:00:00 +0800</pubDate>
            <pubDate>Fri, 06 May 16 00:00:00 +0800</pubDate>
            <pubDate>Fri, 16 Oct 15 00:00:00 +0800</pubDate>

Ah - I see what you mean - Nothing after 4th Nov 18 is being shown - or at least they’re not showing in TTRSS for me either…

Screenshot

Debug log, forced refresh and hash:

Oldest article not displayed (Nov 23rd '18)
[12:58:03/32658] guid 2,Product%3A+X470+GAMING+PLUS+-+Download%3A+ / SHA1:7b6e07d38d46e98e103c570c6c15d3959807c04d
[12:58:03/32658] orig date: 1542902400
[12:58:03/32658] title Product: X470 GAMING PLUS - Download:
[12:58:03/32658] link https://www.msi.com/support/mb/X470-GAMING-PLUS#support_download?utm_source=msi-news-feed&utm_medium=rss&utm_campaign=msi-news&utm_content=Product%3A%20X470%20GAMING%20PLUS%20-%20Download%3A%20
[12:58:03/32658] language 
[12:58:03/32658] author 
[12:58:03/32658] looking for tags...
[12:58:03/32658] tags found: download
[12:58:03/32658] done collecting data.
[12:58:03/32658] article hash: 7b62e0519b7693e547f8c1b6b5c53b0d34c3fdd4 [stored=ae710ddcb1047e404bb562a3ecd6a73700e7bba2]
[12:58:03/32658] hash differs, applying plugin filters:
[12:58:03/32658] ... Af_Comics
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Psql_Trgm
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Readability
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_RedditImgur
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Unburn
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] plugin data: af_comics,af_psql_trgm,af_readability,af_redditimgur,af_unburn,
[12:58:03/32658] date 1542902400 [2018/11/22 16:00:00]
[12:58:03/32658] num_comments: 0
[12:58:03/32658] force catchup: 
[12:58:03/32658] base guid found, checking for user record
[12:58:03/32658] initial score: 0 [including plugin modifier: 0]
[12:58:03/32658] user record FOUND: RID: 106128, IID: 106128
[12:58:03/32658] resulting RID: 106128, IID: 106128
[12:58:03/32658] article updated, but we're forbidden to mark it unread.
[12:58:03/32658] assigning labels [other]...
[12:58:03/32658] assigning labels [filters]...
[12:58:03/32658] looking for enclosures...
[12:58:03/32658] article processed

Followed immediately by:

Most recent article displayed (Nov 5th '18)

[12:58:03/32658] guid 2,Product%3A+X470+GAMING+PLUS+-+Download%3A+Quick+Installation+Guide / SHA1:101f45d6bbc6b9dbefb2e507ad01e2e8414b0bae
[12:58:03/32658] orig date: 1541347200
[12:58:03/32658] title Product: X470 GAMING PLUS - Download: Quick Installation Guide
[12:58:03/32658] link https://www.msi.com/support/mb/X470-GAMING-PLUS#support_download?utm_source=msi-news-feed&utm_medium=rss&utm_campaign=msi-news&utm_content=Product%3A%20X470%20GAMING%20PLUS%20-%20Download%3A%20Quick%20Installation%20Guide
[12:58:03/32658] language 
[12:58:03/32658] author 
[12:58:03/32658] looking for tags...
[12:58:03/32658] tags found: download
[12:58:03/32658] done collecting data.
[12:58:03/32658] article hash: 6a81be3bfd7cef7c426964bb183685553052ec03 [stored=6a81be3bfd7cef7c426964bb183685553052ec03]
[12:58:03/32658] hash differs, applying plugin filters:
[12:58:03/32658] ... Af_Comics
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Psql_Trgm
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Readability
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_RedditImgur
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] ... Af_Unburn
[12:58:03/32658] === 0.0000 (sec)
[12:58:03/32658] plugin data: af_comics,af_psql_trgm,af_readability,af_redditimgur,af_unburn,
[12:58:03/32658] date 1541347200 [2018/11/04 16:00:00]
[12:58:03/32658] num_comments: 0
[12:58:03/32658] force catchup: 
[12:58:03/32658] base guid found, checking for user record
[12:58:03/32658] initial score: 0 [including plugin modifier: 0]
[12:58:03/32658] user record FOUND: RID: 106139, IID: 106139
[12:58:03/32658] resulting RID: 106139, IID: 106139
[12:58:03/32658] article updated, but we're forbidden to mark it unread.
[12:58:03/32658] assigning labels [other]...
[12:58:03/32658] assigning labels [filters]...
[12:58:03/32658] looking for enclosures...
[12:58:03/32658] article processed

Can’t see anything amiss there, not that I’m that good at spotting such things - may help someone else spot something though…

Full log

Thank you, shabble, for providing the debug output, links and confirming my problem.

Too bad nobody else responded. Would this thread fit in the Supported category, since it’s an installation from git and a vserver?

well, what’s the problem exactly? are the articles in feed XML but missing in tt-rss? something else?

That exactly.

TT-RSS is showing always only the articles which are in my screenshot. But the feed has alot more (and recent) articles, which also show up in the debug output like shabble showed.

Do you have any filters which could be marking the new articles as read or deleting them?

they are reusing article IDs. those are supposed to be globally unique.

the feed is broken, it’s not going to work.

Yes, but I disabled them and fetched again.

Ya, no surprise, these feeds are broken all over the place…
Thank you!