Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1284655 - TLS certificate validation fails with alternative chains [NEEDINFO]
Summary: TLS certificate validation fails with alternative chains
Keywords:
Status: CLOSED DUPLICATE of bug 1286034
Alias: None
Product: Fedora
Classification: Fedora
Component: glib-networking
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Matthias Clasen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-11-23 19:34 UTC by Kai Engert (:kaie) (inactive account)
Modified: 2015-11-27 10:02 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-27 10:02:45 UTC
kengert: needinfo? (mclasen)


Attachments (Terms of Use)
test.c (deleted)
2015-11-24 08:55 UTC, Milan Crha
no flags Details
patch for glib-networking package in fedora f22 dist/git (deleted)
2015-11-24 12:45 UTC, Kai Engert (:kaie) (inactive account)
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
GNOME Bugzilla 750457 None None None Never

Description Kai Engert (:kaie) (inactive account) 2015-11-23 19:34:18 UTC
The evolution mail clients fails to verify a server certificate, if it's necessary to use a different chain of CA certificates to find a trusted root CA certificates.

(For a general explanation of the background, you could read my explanation at
https://rt.openssl.org/Ticket/Display.html?id=3621#txn-49999 )


evolution displays the error message:

"A mail account '...@gmail.com' cannot connect, because an SSL certificate 
 for 'imap.googlemail.com' is not trusted.
 Reason: The signing authority is not known."


This can be reproduced using the following environment:
- Fedora 22
- ca-certificates version 2015.2.6
  (from https://bodhi.fedoraproject.org/updates/FEDORA-2015-6fb2c59536 )
- CA legacy configuration set to disabled
  (execute as root: ca-legacy disable
   then the following should report DISABLED: ca-legacy check
  )

The connection works fine, if the ca-legacy configuration is set to default.


The cause must be the trust status of the following root CA:
# Issuer: OU=Equifax Secure Certificate Authority,O=Equifax,C=US
# Serial Number: 903804111 (0x35def4cf)
# Subject: OU=Equifax Secure Certificate Authority,O=Equifax,C=US
# Not Valid Before: Sat Aug 22 16:41:51 1998
# Not Valid After : Wed Aug 22 16:41:51 2018
# Fingerprint (MD5): 67:CB:9D:C0:13:24:8A:82:9B:B2:17:1E:D1:1B:EC:D4
# Fingerprint (SHA1): D2:32:09:AD:23:D3:14:23:21:74:E4:0D:7F:9D:62:13:97:86:63:3A


Upstream Mozilla has removed the "trusted for TLS servers" from this CA in version 2.6


The IMAP server is configured to send multiple CA certificates:

Certificate:
    Data:
        Serial Number:26:b1:28:a9:52:c6:c6:49
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=Google Internet Authority G2,O=Google Inc,C=US"
        Validity:
            Not Before: Thu Nov 12 18:49:26 2015
            Not After : Wed Feb 10 00:00:00 2016
        Subject: "CN=imap.googlemail.com,O=Google Inc,L=Mountain View,ST=California,C=US"
    Fingerprint (SHA-256): F3:B9:8F:AB:42:D4:95:F1:DA:21:5B:47:6F:4B:34:8E:9F:40:2D:89:FA:3A:27:30:C5:7E:4E:D1:CA:B7:9C:02
    Fingerprint (SHA1): 38:17:7E:0A:EF:EE:0E:14:A8:AF:9D:11:5D:92:8D:3A:40:BC:30:A5
Certificate:
    Data:
        Serial Number: 146051 (0x23a83)
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=GeoTrust Global CA,O=GeoTrust Inc.,C=US"
        Validity:
            Not Before: Fri Apr 05 15:15:56 2013
            Not After : Sat Dec 31 23:59:59 2016
        Subject: "CN=Google Internet Authority G2,O=Google Inc,C=US"
    Fingerprint (SHA-256): A4:12:4F:DA:F9:CA:C7:BA:EE:1C:AB:32:E3:22:5D:74:65:00:C0:9F:3C:F3:EB:B2:53:EF:3F:BB:08:8A:FD:34
    Fingerprint (SHA1): 17:8F:7E:93:A7:4E:D7:3D:88:C2:90:42:22:0B:9A:E6:E4:B3:71:CD
Certificate:
    Data:
        Serial Number: 1227750 (0x12bbe6)
        Signature Algorithm: PKCS #1 SHA-1 With RSA Encryption
        Issuer: "OU=Equifax Secure Certificate Authority,O=Equifax,C=US"
        Validity:
            Not Before: Tue May 21 04:00:00 2002
            Not After : Tue Aug 21 04:00:00 2018
        Subject: "CN=GeoTrust Global CA,O=GeoTrust Inc.,C=US"
    Fingerprint (SHA-256): 3C:35:CC:96:3E:B0:04:45:13:23:D3:27:5D:05:B3:53:23:50:53:49:0D:9C:D8:37:29:A2:FA:F5:E7:CA:1C:C0
    Fingerprint (SHA1): 73:59:75:5C:6D:F9:A0:AB:C3:06:0B:CE:36:95:64:C8:EC:45:42:A3


This is a *VALID* server configuration, and a configuration that is used by many Internet hosts.
We must be able to correctly support this configuration.


As can be seen using the command line diagnostic utilities of openssl, gnutls and nss, the libraries are able to correctly find a trusted root CA, and allow the connection to the server:

- openssl s_client -tls1_2 -connect imap.googlemail.com:993
  -> Verify return code: 0 (ok)

- tstclnt -V tls1.2: -C -Db  -h imap.googlemail.com -p 993
  -> locally found issuer certificate:
     Issuer: "CN=GeoTrust Global CA,O=GeoTrust Inc.,C=US"

- gnutls-cli -p 993 imap.googlemail.com
  -> Status: The certificate is trusted. 


I'm surprised that evolution doesn't trust the certificate.


Does evolution attempt to use its own implementation, instead of using library code to verify a certificate?

Does evolution need to set a new library flag to enable the discovery of alternative chains of trust?

Comment 1 Kai Engert (:kaie) (inactive account) 2015-11-23 19:36:28 UTC
When testing, please ensure you NEVER click the "accept permanently" buttons when evolution reports the error, to ensure your verification environment remains at the clean default.

Comment 2 Milan Crha 2015-11-24 08:55:02 UTC
Created attachment 1098069 [details]
test.c

Thanks for a bug report. Evolution's IMAPx (and Camel in general) uses glib's gio streams and relies on its results. If I recall correctly, the TLS gio stream uses gnutls in the background.

I extracted the relevant code from the evolution-data-server to a single file (attached). The first line contains a comment with a command line to compile and run it.

I can reproduce the issue with the test in the environment setup as you suggested above. Either the evolution-data-server is using the GIO callbacks incorrectly (unlikely, from my point of view, because before the update to the ca-certificates the same code works like a charm), or there should be done a fix on the glib/gio side, or somewhere deeper, just as you suggested above.

Comment 3 Nikos Mavrogiannopoulos 2015-11-24 09:47:05 UTC
Could that be related with
https://bugzilla.redhat.com/show_bug.cgi?id=1246492 ? It seems gio reimplements certificate verification rather than using gnutls' functions.

Comment 4 Tomas Mraz 2015-11-24 10:18:14 UTC
Yes, that looks like a duplicate.

Comment 5 Kai Engert (:kaie) (inactive account) 2015-11-24 11:47:27 UTC
Milan, thank for the test attachment.

I tried a few experiments, and the results are confusing.

First, I compiled the test as you said. I can reproduce the failure using it.
I tried to debug/trace with gdb, but I failed, because the optimized binaries make it difficult to follow the execution path.

Because of that, I decided to build glib2 myself.

I cloned branch glib-2-44 of glib and installed it to /opt/evolution.
With that, the test reported TLS not available.

In addition I built glib-2-44 of glib-networking.

With that combination, to my surprise, your test no longer reports a failure.
It prints Connection 0x... error: none


Does that mean that something has been fixed in latest git already?


In addition, I've also built evolution-data-server and evolution locally (prefix /opt/evolution), following the instructions in the evolution README.

When running my local evolution build, the connection to imap.googlemail.com works fine.


However, there are other errors remaining.

I didn't mention it earlier, but the mentioned error wasn't the only one.
In addition, I have google calendars configured.
With the standard f22 evolution, I also get error messages for each of the google calendar connections, which seem to be about the same cause, because they mention a certificate for www.google.com which has been issued by the same Google intermediate CA.

Can you suggest which additional component might require a rebuild to fix the calendar TLS verification?


Should we try to suggest to the maintainer of glib* to produce rebuilt RPM packages for f22 testing?

Comment 6 Kai Engert (:kaie) (inactive account) 2015-11-24 11:48:58 UTC
FYI, I built branch gnome-3-16 of evolution and data-server.
I think for all locally builds I used the same branches that are used by f22.

Comment 7 Kai Engert (:kaie) (inactive account) 2015-11-24 11:55:26 UTC
I see that my environment still executes data-server processes from /usr/libexec ... I need to figure out how to make it use the builds from /opt/evolution/libexec

Comment 8 Kai Engert (:kaie) (inactive account) 2015-11-24 12:04:00 UTC
Milan has explained to me how I run the local libexec binaries.
I need to start them first manually, source-registry first, then calendar-factory -w and addressbook-factory-w.

With that, no more certificates errors, both imap and calendar connections work.

Comment 9 Kai Engert (:kaie) (inactive account) 2015-11-24 12:09:50 UTC
Changing component to glib-networking.

Milan pointed me to 
https://git.gnome.org/browse/glib-networking/log/?h=glib-2-44

This commit from september seems the likely cause for the fix with the local build:
https://git.gnome.org/browse/glib-networking/commit/?h=glib-2-44&id=44d02b8c57e24f0a796d75b6270ecee265f35f6f

The commit points to
https://bugzilla.gnome.org/show_bug.cgi?id=750457

Comment 10 Kai Engert (:kaie) (inactive account) 2015-11-24 12:10:50 UTC
Could you please rebuild glib-networking for Fedora 22 and later, to ensure this fix gets picked up?

Comment 11 Kai Engert (:kaie) (inactive account) 2015-11-24 12:43:38 UTC
This is a scratch build that applies the patch.
http://koji.fedoraproject.org/koji/taskinfo?taskID=11967051

With this build installed, the bug is fixed for me.

Comment 12 Kai Engert (:kaie) (inactive account) 2015-11-24 12:45:27 UTC
Created attachment 1098181 [details]
patch for glib-networking package in fedora f22 dist/git

Comment 13 Kai Engert (:kaie) (inactive account) 2015-11-24 14:24:10 UTC
(In reply to Nikos Mavrogiannopoulos from comment #3)
> Could that be related with
> https://bugzilla.redhat.com/show_bug.cgi?id=1246492 ? It seems gio
> reimplements certificate verification rather than using gnutls' functions.

(In reply to Tomas Mraz from comment #4)
> Yes, that looks like a duplicate.

In bug 1246492, David said: not a duplicate

Comment 14 David Woodhouse 2015-11-24 14:29:24 UTC
This should be testable with the 'socket-client' test from gio, rather than having to use the Evolution stack?

Comment 15 David Woodhouse 2015-11-24 14:38:44 UTC
What I said in bug 1246492 was slightly more nuanced than 'not a duplicate'.

If you retitle this bug to "glib-networking reimplements stuff it shouldn't", and fix it *properly* to use the GnuTLS functionality, then it's a duplicate.

If you're going to paper over the cracks without fixing the real underlying problem, then it isn't — bug 1246492 covers a *different* symptom of the same problem.

So actually, I think we really *should* be treating them as the same bug.

Comment 16 Kai Engert (:kaie) (inactive account) 2015-11-25 12:13:25 UTC
(In reply to Kai Engert (:kaie) from comment #10)
> Could you please rebuild glib-networking for Fedora 22 and later, to ensure
> this fix gets picked up?

FYI, I just upgraded to Fedora 23, and it already uses a version of glib-networking that has picked up the "papering" fix, so, this specific bug doesn't occur in F23.

Comment 17 Nikos Mavrogiannopoulos 2015-11-27 09:56:07 UTC
(In reply to David Woodhouse from comment #15)
> What I said in bug 1246492 was slightly more nuanced than 'not a duplicate'.
> 
> If you retitle this bug to "glib-networking reimplements stuff it
> shouldn't", and fix it *properly* to use the GnuTLS functionality, then it's
> a duplicate.

Let's do that then. I believe we are going to have many more issues like that if we don't. I'll open a new bug and mark these as duplicates.

Comment 18 Nikos Mavrogiannopoulos 2015-11-27 10:02:45 UTC

*** This bug has been marked as a duplicate of bug 1286034 ***


Note You need to log in before you can comment on or make changes to this bug.