|Summary:||[3.5] extended route validation does not catch newlines in certificate|
|Product:||OpenShift Container Platform||Reporter:||Steven Walter <stwalter>|
|Status:||CLOSED ERRATA||QA Contact:||zhaozhanqi <zzhao>|
|Version:||3.5.1||CC:||aos-bugs, bbennett, bmeng, dmoessne, dsafford, fweimer, haowang, jtanenba, nbhatt, rhowe, sgaikwad, weliang|
|Fixed In Version:||Doc Type:||Bug Fix|
Cause: Invalid PEM data is left in the route config file during extended validation. Consequence: The router crashes Fix: Sanitize PEM data from route configs Result: Properly catch malformed certificates in extended validation.
|Last Closed:||2017-12-14 21:02:32 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
Description Steven Walter 2017-11-10 00:30:51 UTC
Description of problem: In the route definition we saw empty lines: ... certificate: | -----BEGIN CERTIFICATE----- ... key: | -----BEGIN PRIVATE KEY----- ... It is hardly noticeable in YAML format. On the OpenShift master console GUI it can be seen better. Normally it should be |+ instead of just a | character. Then the haproxy concatenates the certificates into a PEM file where the empty lines appear, and we think it causes those error messages. Version-Release number of selected component (if applicable): image: ose-haproxy-router:v22.214.171.124 OpenShift version: v126.96.36.199.24 How reproducible: Unconfirmed Steps to Reproduce: 1. Created route in web console Actual results: 2017-11-09T09:20:32.957490000Z [ALERT] 312/092032 (127789) : parsing [/var/lib/haproxy/conf/haproxy.config:142] : 'bind 127.0.0.1:10444' : 'crt-list' : error processing line 150 in file '/var/lib/haproxy/conf/cert_config.map' : unable to load SSL certificate from PEM file '/var/lib/haproxy/router/certs/example.pem'. 2017-11-09T09:20:32.957682000Z [ALERT] 312/092032 (127789) : Error(s) found in configuration file : /var/lib/haproxy/conf/haproxy.config 2017-11-09T09:20:32.957873000Z [ALERT] 312/092032 (127789) : Fatal errors found in configuration. Expected results: Non-fatal warning, not accepting the route Additional info: Verified oc set env dc/router EXTENDED_VALIDATION=true Deleting route causes error to go away
Comment 3 Steven Walter 2017-11-10 00:43:09 UTC
From Clayton: Extended validation in 3.5 had gaps. Most were fixed in 3.6. You can run the 3.6 diagnostic command against a 3.5 cluster to check whether any routes would fail EV - if they can recreate in a test cluster please have them rerun. Most of the things fixed were invalid PEM blocks and incomplete content.
Comment 7 Ryan Howe 2017-11-10 18:43:06 UTC
To reproduce this I removed one "-" from the -----END CERTIFICATE---- Was only able to reproduce in 3.5 removing a dash from -----END CERTIFICATE---- and only END CERTIFICATE section hit this issue. |+ |- and | all work fine with 3.5 routes. Fatal errors only happened after editing -----END CERTIFICATE---- In 3.6 the router corrected the pem file adding the - back to the route when the pem was added to /var/lib/haproxy/router/certs/, the route object itself still showed the missing -. This did not cause 3.6 router to fail.
Comment 13 Weibin Liang 2017-11-16 20:44:16 UTC
PR/926 fixed the problem. Testing passed
Comment 15 zhaozhanqi 2017-11-29 07:54:31 UTC
QE did the testing on 188.8.131.52.47 1. As comment 7 said: |+ |- and | all work fine 2. if I removed one "-" from the -----END CERTIFICATE----, the route will be marked as 'ExtendedValidationFailed'
Comment 16 Dana Safford 2017-11-30 16:24:41 UTC
This situation is now very critical. I just set the Customer Escalation flag and the am requesting a hotfix (so I flipped that flag too). Hello Jacob/Rashid I wrote yesterday in ref Bug 1511732 which is still in Status (Modified) and the following question below from the Action Plan: 1. Are we able to meet customers expectations on being able to deliver an errata fixing this by the 15th of December? If not 2. When is the cut off date on knowing if this expectation cant be achieved? 3. How long would the hotifix take? and when or who can deliver this. For more info, contact Aaron Ship (Critsit Manager) aship in irc
Comment 21 errata-xmlrpc 2017-12-14 21:02:32 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3438