Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1218251

Summary: The installer should check that the cert rpms installed on the system are corresponding to those present in ~/ssl-build (or in the capsule certs tar.gz)
Product: Red Hat Satellite 6 Reporter: Ivan Necas <inecas>
Component: InstallerAssignee: Ivan Necas <inecas>
Status: CLOSED ERRATA QA Contact: Martin Bacovsky <mbacovsk>
Severity: high Docs Contact:
Priority: high    
Version: 6.1.0CC: anerurka, bbuckingham, bkearney, dmoessne, inecas, jason.hayes, mbacovsk, sthirugn, xdmoon
Target Milestone: UnspecifiedKeywords: Triaged, UserExperience
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: katello-installer-base-3.0.0.51-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:24:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1171841, 1356955    

Description Ivan Necas 2015-05-04 12:16:41 UTC
Description of problem:

The katello-installer and capsule-certs-generate are using rpms to distribute the generated certificates. Newly-regenerated rpms with new certificates have increased version number, so that they should updated the previous certificates in the system.

However, in some cases (especially when experimenting with different katello-installer certs options and trying to re-install the katello), the rpms with the newly generated certificates installed on the system don't update already installed rpms on the system from previous attempts.

How reproducible:
always

Steps to Reproduce:
1. katello-installer
2. remove ~/ssl-build directory on the server
3. katello-installer --reset
4. capsule-certs-generate capsule-certs-generate --capsule-fqdn capsule.example.com --certs-tar ~/capsule.example.com.tar.gz 
5. on the capsule: capsule-installer (using the options suggested in the capsule-certs-generate output)

Actual results:

The capsule-installer fails on

ProxyAPI::ProxyException: ERF12-2749 [ProxyAPI::ProxyException]: Unable to get environments from Puppet ([OpenSSL::SSL::SSLError]: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verif...) for proxy https://capsule.example.com:9090/puppet

Expected results:

The katello-installer, capsule-certs-generate and capsule-installer check that the cert rpms installed on the system correspond with the rpms that are intended to be used.

Additional info:

The workaround for the issue is to remote the cert rpms manually before the installer call:

   for i in $(ls /etc/pki/katello-certs-tool/certs/*); 
   do
     rpm -e $(rpm -qf $i)
   done

The run of the installer should make the installer work again.

There is a kcs article about this workaround https://access.redhat.com/solutions/1311844 with a small suggested update here https://bugzilla.redhat.com/show_bug.cgi?id=1171841#c18

Comment 5 Ivan Necas 2016-06-28 16:16:23 UTC
Created redmine issue http://projects.theforeman.org/issues/15538 from this bug

Comment 6 Ivan Necas 2016-06-28 16:37:22 UTC
Proposed fix at https://github.com/Katello/puppet-certs/pull/91

Comment 7 Ivan Necas 2016-06-28 16:38:49 UTC
Steps I've tested the change against:

1 install katello
2 check the certificiate of web UI
3 cp ~/ssl-build{,.1}
4 foreman-installer --certs-update-all
5 check the certificiate of web UI
6 cp ~/ssl-build{,.2}
7 rm -rf ~/ssl-build
8 cp ~/ssl-build{.1,}
9 foreman-installer
10 the certificate of the web UI should change back to the one from step 2
11 foreman-installer --certs-update-all
12 the certificate of the web UI should be different than the one from step 2 or 5

Comment 8 Bryan Kearney 2016-07-06 12:34:38 UTC
Upstream is merged, moving this to POST.

Comment 11 Ivan Necas 2016-07-15 11:20:03 UTC
While testing this by removing the /root/ssl-build, I've hit another related issue that I track here https://bugzilla.redhat.com/show_bug.cgi?id=1356955.
Since it's just one of the cases that this BZ addresses, and in most cases, only the server-ca related certs are changed, not the default-ca itself, I suggest verifying this BZ based on the scenario described in https://bugzilla.redhat.com/show_bug.cgi?id=1218251#c7 and the second issue in the separate bug.

Comment 12 Ivan Necas 2016-07-15 12:08:24 UTC
*** Bug 1291065 has been marked as a duplicate of this bug. ***

Comment 13 Martin Bacovsky 2016-07-15 15:03:17 UTC
I tested the scenario from c#7 with ssl-build rollback and it worked fine. The original reproducer for this bug was blocked by two other bugs and needed workarounds from [1] and [2] to finish successfully.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1356955
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1357046


---- ssl-build rollback scenario
[root@sat-snap-rhel7 ~]# satellite-installer --reset
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  * Satellite is running at https://sat-snap-rhel7.example.com
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/satellite.log

[root@sat-snap-rhel7 ~]# cp -r ~/ssl-build{,.100}

[root@sat-snap-rhel7 ~]# satellite-installer --certs-update-all
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-router-server for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-router-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/pulp-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-puppet-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-apache for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/java-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-proxy-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-proxy for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-broker for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-client-cert for update
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  * Satellite is running at https://sat-snap-rhel7.example.com
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/satellite.log

[root@sat-snap-rhel7 ~]# mv ~/ssl-build{,.101}
[root@sat-snap-rhel7 ~]# cp -r ~/ssl-build{.100,}

[root@sat-snap-rhel7 ~]# satellite-installer
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  * Satellite is running at https://sat-snap-rhel7.example.com
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/satellite.log

[root@sat-snap-rhel7 ~]# satellite-installer --certs-update-all
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-router-server for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-router-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/pulp-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-puppet-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-apache for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/java-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-proxy-client for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-foreman-proxy for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-broker for update
Marking certificate /root/ssl-build/sat-snap-rhel7.example.com/sat-snap-rhel7.example.com-qpid-client-cert for update
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  * Satellite is running at https://sat-snap-rhel7.example.com
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/satellite.log
------



----- original reproducer test log
[root@sat-snap-rhel7 ~]# mv ssl-build{,.1}
[root@sat-snap-rhel7 ~]# rm -rf /etc/pki/katello/nssdb
[root@sat-snap-rhel7 ~]# mv /etc/candlepin/certs/amqp /etc/candlepin/certs/amqp.bak
[root@sat-snap-rhel7 ~]# satellite-installer --reset
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  foreman-tasks.service




Redirecting to /bin/systemctl stop  tomcat.service

could not change directory to "/root"


Redirecting to /bin/systemctl stop  httpd.service

Redirecting to /bin/systemctl stop  mongod.service


Redirecting to /bin/systemctl start  mongod.service


Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  * Satellite is running at https://sat-snap-rhel7.example.com
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/satellite.log
[root@sat-snap-rhel7 ~]# capsule-certs-generate --capsule-fqdn capsule-snap-rhel7.example.com --certs-tar ~/capsule-snap-rhel7.example.com.tar.gz
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!

  To finish the installation, follow these steps:

  If you do not have the capsule registered to the Satellite instance, then please do the following:

  1. yum -y localinstall http://sat-snap-rhel7.example.com/pub/katello-ca-consumer-latest.noarch.rpm
  2. subscription-manager register --org "Default_Organization"

  Once this is completed run the steps below to start the capsule installation:

  1. Ensure that the satellite-capsule package is installed on the system.
  2. Copy /root/capsule-snap-rhel7.example.com.tar.gz to the system capsule-snap-rhel7.example.com
  3. Run the following commands on the capsule (possibly with the customized
     parameters, see satellite-installer --scenario capsule --help and
     documentation for more info on setting up additional services):

  satellite-installer --scenario capsule\
                    --capsule-parent-fqdn                         "sat-snap-rhel7.example.com"\
                    --foreman-proxy-register-in-foreman           "true"\
                    --foreman-proxy-foreman-base-url              "https://sat-snap-rhel7.example.com"\
                    --foreman-proxy-trusted-hosts                 "sat-snap-rhel7.example.com"\
                    --foreman-proxy-trusted-hosts                 "capsule-snap-rhel7.example.com"\
                    --foreman-proxy-oauth-consumer-key            "BRbNWyWK4V7hfss67AiPCCbnQ3KdEM3M"\
                    --foreman-proxy-oauth-consumer-secret         "jVwNJrEEDwyWnA2ci6P87wDQmoFZbHQH"\
                    --capsule-pulp-oauth-secret                   "5mzD8KbyNRMLD8ieo3iWcF6FUwbh4KC5"\
                    --capsule-certs-tar                           "/root/capsule-snap-rhel7.example.com.tar.gz"
  The full log is at /var/log/capsule-certs-generate.log
[root@sat-snap-rhel7 ~]# scp capsule-snap-rhel7.example.com.tar.gz vagrant@capsule-snap-rhel7.example.com:
capsule-snap-rhel7.example.com.tar.gz                                                                                                                                            100%   60KB  60.3KB/s   00:00    
[root@sat-snap-rhel7 ~]# logout
[vagrant@sat-snap-rhel7 ~]$ logout
Connection to 192.168.121.228 closed.
[forklift]$ vagrant ssh capsule-snap-rhel7
Last login: Fri Jul 15 14:18:50 2016 from 192.168.121.1
[vagrant@capsule-snap-rhel7 ~]$ sudo su -
[root@capsule-snap-rhel7 ~]# cp /home/vagrant/capsule-snap-rhel7.example.com.tar.gz .
cp: overwrite ‘./capsule-snap-rhel7.example.com.tar.gz’? y
[root@capsule-snap-rhel7 ~]# satellite-installer --scenario capsule\
>                     --capsule-parent-fqdn                         "sat-snap-rhel7.example.com"\
>                     --foreman-proxy-register-in-foreman           "true"\
>                     --foreman-proxy-foreman-base-url              "https://sat-snap-rhel7.example.com"\
>                     --foreman-proxy-trusted-hosts                 "sat-snap-rhel7.example.com"\
>                     --foreman-proxy-trusted-hosts                 "capsule-snap-rhel7.example.com"\
>                     --foreman-proxy-oauth-consumer-key            "BRbNWyWK4V7hfss67AiPCCbnQ3KdEM3M"\
>                     --foreman-proxy-oauth-consumer-secret         "jVwNJrEEDwyWnA2ci6P87wDQmoFZbHQH"\
>                     --capsule-pulp-oauth-secret                   "5mzD8KbyNRMLD8ieo3iWcF6FUwbh4KC5"\
>                     --capsule-certs-tar                           "/root/capsule-snap-rhel7.example.com.tar.gz"
Installing             Done                                               [100%] [...............................................................................................................................]
  Success!
  The full log is at /var/log/foreman-installer/capsule.log
--------

Comment 14 Martin Bacovsky 2016-07-15 15:07:26 UTC
VERIFIED
sat6.2 snap20.1

Comment 15 Bryan Kearney 2016-07-27 11:24:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501