Fetching feed posts fails when the feed's site uses let's enctypt certs

Describe the problem you’re having:
Fetching feeds from sites using let’s encrypt certs doesn’t work. Debuggin such a feed produces the following output:

[19:05:42/13871] start
[19:05:42/13871] running HOOK_FETCH_FEED handlers...
[19:05:42/13871] ... Af_Comics
[19:05:42/13871] === 0.0002 (sec)
[19:05:42/13871] feed data has not been modified by a plugin.
[19:05:42/13871] local cache will not be used for this feed
[19:05:42/13871] last unconditional update request: 2020-04-08 08:30:55
[19:05:42/13871] maximum allowed interval for conditional requests exceeded, forcing refetch
[19:05:42/13871] fetching [https://www.laufzeit.de/feed/] (force_refetch: 1)...
[19:05:42/13871] fetch done.
[19:05:42/13871] source last modified: 
[19:05:42/13871] unable to fetch: ; 60 SSL certificate problem: unable to get local issuer certificate [0]

If possible include steps to reproduce the problem:
Subscribe to a rss feed where the site uses let’s encrypt certs for https connections.

tt-rss version (including git commit id):
latest origin master

You should be providing a bit more information when asking for support.

Certificate validation occurs within the curl module, which uses the operating system’s list of known and trusted certificate authorities. You should try manually making the request via SSH using curl, you’ll probably get the same error. From there you can update your operating system or manually added the necessary root CA certificates from Let’s Encrypt.

Not sure either of those will work.

Having had a play on two different systems, wget and curl both have problems with their feed URL.

$ curl -LI https://www.laufzeit.de/feed
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.


$ wget https://www.laufzeit.de/feed -O /dev/null
--2020-04-11 22:48:28--  https://www.laufzeit.de/feed
Resolving www.laufzeit.de (www.laufzeit.de)... 45.146.172.25
Connecting to www.laufzeit.de (www.laufzeit.de)|45.146.172.25|:443... connected.
ERROR: cannot verify www.laufzeit.de's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
  Unable to locally verify the issuer's authority.
To connect to www.laufzeit.de insecurely, use `--no-check-certificate'.

Firefox is quite happy with it however.

Had a quick look at the certificates, and I don’t think it’s a root CA problem - one of my own sites uses the same chain as they are* so I can only conclude there’s something wrong with the site certificate.


* root[DST Root CA X3] + intermediary[Let’s Encrypt Authority X3]

Yeah, thanks for pointing that out. That site seems broken:

$ openssl s_client -showcerts -servername www.laufzeit.de -connect www.laufzeit.de:443 </dev/null
CONNECTED(00000003)
depth=0 CN = laufzeit.de
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = laufzeit.de
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:CN = laufzeit.de
   i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
-----BEGIN CERTIFICATE-----
MIIHiTCCBnGgAwIBAgISAzPsw7IlksqR5Oho1lFvsjUNMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDAzMDMxODEyMjhaFw0y
MDA2MDExODEyMjhaMBYxFDASBgNVBAMTC2xhdWZ6ZWl0LmRlMIICIjANBgkqhkiG
9w0BAQEFAAOCAg8AMIICCgKCAgEAphWGpaFshGuHaSvGbaTSTZjpMaeAtYQKYh2a
UY32UgeL+we+PI/zsvQgj1IBghsR+LiX78KruIBu0yX6hhChE5ZlGggtJQvTF99A
3guIFesmUEZ1GhPLl/L0da55aEhf3nhErlDiJMBbG3MuiAgn7NhGbvTGpwv+wBIK
lKBx4v4OrVkYDVKw8eZb0QhOiQq7WB/hO9cmDJLvmU2NTqOBFSaWYNy1bHGwoaLA
CioPX6CuSgsxUBGgxAb1sbLiB5GbetZJdPhUXwOWYvh7HlwHym8fXkZ5W/i6J2hy
MwGUUdpkIr9lrghnDpmBBbEqb83n4ZSdH5hHLtuXq9jyYUYU2YZ0DmcKM6Gat6V5
tNSG+4IuDhqJlbMBGqG4tGorf48fL8zTlaWMCXI2sp8CVTeL7KJ6Ahi7CyPUViF4
L1ySP6cz9cb8ndLs1EgHRzkNHejNZmzuqznqqZ4OF5J1csgUUmxF5iB3ZhLh0844
RIlRlgpSiZ0O+vlfOg20u5suYIz3b8Q+hcLfNUKuH7UWDXnZQ1/dGOEuEJVul+8r
q+e6FiJKV36DTiL0YGsr1wODThWc9I+Kr3Djs373xwvl+jfH+KKjQkH9Ku+kIhtq
TxH70EVTcc7j2fwJ4pOlSgnPASnxwC0VmYEoCT71RZJEHmrnsbQFiwWqAQ3sxg2B
X9e2G60CAwEAAaOCA5swggOXMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUVgmdiEHW
EE815T/1x18QOJaet+4wHwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEw
bwYIKwYBBQUHAQEEYzBhMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMu
bGV0c2VuY3J5cHQub3JnMC8GCCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMu
bGV0c2VuY3J5cHQub3JnLzCCAU8GA1UdEQSCAUYwggFCghVjb25kaXRpb24tbGF1
ZnplaXQuZGWCFWNvbmRpdGlvbi1tYWdhemluLmNvbYIUY29uZGl0aW9uLW1hZ2F6
aW4uZGWCFGNvbmRpdGlvbm1hZ2F6aW4uY29tghNjb25kaXRpb25tYWdhemluLmRl
ghVsYXVmemVpdC1jb25kaXRpb24uZGWCC2xhdWZ6ZWl0LmRlghl3d3cuY29uZGl0
aW9uLWxhdWZ6ZWl0LmRlghl3d3cuY29uZGl0aW9uLW1hZ2F6aW4uY29tghh3d3cu
Y29uZGl0aW9uLW1hZ2F6aW4uZGWCGHd3dy5jb25kaXRpb25tYWdhemluLmNvbYIX
d3d3LmNvbmRpdGlvbm1hZ2F6aW4uZGWCGXd3dy5sYXVmemVpdC1jb25kaXRpb24u
ZGWCD3d3dy5sYXVmemVpdC5kZTBMBgNVHSAERTBDMAgGBmeBDAECATA3BgsrBgEE
AYLfEwEBATAoMCYGCCsGAQUFBwIBFhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9y
ZzCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB1AF6nc/nfVsDntTZIfdBJ4DJ6kZoM
hKESEoQYdZaBcUVYAAABcKHQTzcAAAQDAEYwRAIgRF+sCDZM+qRjI3w/TzFkAta4
3HfyouwiEjHNB3LRMhACIHEpUByzfSDbiuafqL2oyDIM6PNC0/f87E1Jx+rWi0IS
AHcAsh4FzIuizYogTodm+Su5iiUgZ2va+nDnsklTLe+LkF4AAAFwodBPIQAABAMA
SDBGAiEAty3sHs3HotH5YJJeh75DJaL7axuD8T6nWwyi1tAL7pYCIQC7QMXC5bRK
fajTM6litfVmjx7rSUnW7q3d2UjLZT/ynDANBgkqhkiG9w0BAQsFAAOCAQEABZrp
Si6Mq7ZJXoeWkLwXDAXZQSeaENPZPd4cwWvteuwiBpFh1BQnNwYcmucv1dNbx4pZ
HZBqzi9Mvf5xoBGHoLCuV8xJDpJ3rSV22/NJAad0i/OOxskVlPXaTo1I3IvmhojH
jWpoE7yQ9JfQZFvC9he1Wh6vdY7301xPtxbh3NzKPZkATeDIgoQ+GVIIWgVhH/Em
Q0Ne28aNBrmLC9aar7yLv5JDtsPljhl9X9pudzg71Q9XDakZF0he/s/YYrAuLaT6
h+KbK0tjn0OhlooEjkYNu5kbtQqgxPhDlaeoQDCYqPEkLJ7w0mK+Tx7j1dtkUBQR
kGmy7/NKMBSSFiSpcA==
-----END CERTIFICATE-----
---
Server certificate
subject=CN = laufzeit.de

issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2866 bytes and written 400 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 312C70BFD52AD9391C28B74E0CE52BC7950B209FCC84DF77A3ACFDF726148846
    Session-ID-ctx: 
    Master-Key: A69CE0C8ED9F8A8ACDDBE5C1DFA9E0289290E4ED0A20E146691C7A91B6FE471DA277B56E64EE88E3C849BFE1834FD434
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 9a 0f 6c bb 28 42 27 79-16 c3 92 00 ae c2 04 84   ..l.(B'y........
    0010 - 23 6d 16 5e 27 02 de 8f-eb 19 b0 f1 ba b9 b4 64   #m.^'..........d
    0020 - 50 f0 cd 19 26 8b 60 3e-b0 7b 10 1f 54 0c ad 21   P...&.`>.{..T..!
    0030 - 3f 6a 2a 75 4c 7b 54 ef-3e b7 44 21 f7 0d b3 84   ?j*uL{T.>.D!....
    0040 - c1 50 16 7d a4 db f1 d4-06 19 d0 9b 75 ca 5f f9   .P.}........u._.
    0050 - 4a cf 53 b7 77 85 65 25-c6 dc 27 63 4f 58 f5 d4   J.S.w.e%..'cOX..
    0060 - 1a 3f b3 88 a5 07 f3 6c-ad fd 58 8c f1 f8 5f b1   .?.....l..X..._.
    0070 - 04 d2 f0 b1 0f ac 01 ea-12 97 a9 8e 09 57 04 09   .............W..
    0080 - 25 d0 2e d4 40 6c c4 71-71 da 2d 1d 8c 8c 47 ec   %[email protected].
    0090 - 6a b3 b9 5b e6 f1 a0 52-35 ca 50 4e 57 dc 18 42   j..[...R5.PNW..B
    00a0 - 25 4a 85 a2 f6 b4 a3 38-fa 05 9d 65 36 32 cb 0a   %J.....8...e62..
    00b0 - 0d fe ce 35 25 d1 5a f1-51 c8 f3 9b e7 a4 2a e3   ...5%.Z.Q.....*.
    00c0 - 66 f5 48 9a ae b3 36 34-14 cb e1 b3 fc 33 f8 45   f.H...64.....3.E

    Start Time: 1586644868
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: yes
---
DONE

tl;dr the server needs to provide the intermediate CA cert in addition to the server cert.

the problem is that the server only provides its cert and doesn’t bundle the Let's Encrypt Authority X3 intermediate CA cert. this can be discerned from the openssl output: 1. only the server cert is displayed, ie -----BEGIN CERTIFICATE-----, and 2. openssl complains that the server cert’s issuer cert is missing (because it was neither provided by the server nor found locally, ie unable to get local issuer certificate). compare this to openssl s_client -showcerts -connect www.google.com:443 </dev/null where two different certs are printed: 1. the server’s cert and 2. the/an intermediate CA cert.

best practice is for the server to bundle the server and intermediate CA certs together and require the client to have the root CA, eg /etc/ssl/certs/DST_Root_CA_X3.pem on Debian/Ubuntu, which the client uses to verify the certificate chain.

cert verification works for firefox because firefox includes the Let's Encrypt Authority X3 intermediate cert in its own certificate store which it uses to verify the server’s lone cert: Preferences → Privacy & Security → View Certificates → Authorities → Digital Signature Trust Co. → Let’s Encrypt Authority X3.

https://www.ssllabs.com/ssltest/analyze.html?d=www.laufzeit.de&hideResults=on&latest

indeed.

firefox does no such thing, it’s in cache, not in permanent store.

True. But Firefox 75 (released a few days ago) goes one step further and downloads a list of known intermediate certificates before they’re needed (see Security/CryptoEngineering/Intermediate Preloading - MozillaWiki ).

it sure does

Sigh. It’s the eternal dilemma of a Firefox user. Do I continue to support[1] Mozilla even though they keep doing that kind of thing? Or do I switch to a Chromium-based browser and doom the web to be under Google’s will forever[2]?

[1] Yes, I donate yearly.
[2] Chromium may be open source, but even a company the size of Microsoft can’t practically fork it.

Yeah… I read that and was pretty taken aback, especially for an organization that touts privacy as one of their marquee features.

I proxied Microsoft’s new Chromium Edge browser and with the appropriate settings, it basically reports nothing that Windows itself doesn’t already report. (Not to mention it’s faster than Firefox.)

sorry for derailing the thread :slight_smile:

mozilla is actively trying to appear as a privacy-first non-profit while literally not being either. fuck them. at least google is open about what it is, i.e. a cyberpunk-tier evil megacorp.

i’m afraid it already is.

it has a rather questionable implementation of safe browsing:

at the end of the day all browsers - and the content you access via them - are spying platforms. might as well choose one that works best for you. :man_shrugging:

That it does. In fairness, if you disable SmartScreen it actually does honor the setting.

Being on the Internet means being spied upon. I don’t like it but I think it’s where we’re at, sadly.