Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1517903 - after reboot TLS everywhere cluster swift_proxy_tls_proxy and glance_api_tls_proxy are Restarting
Summary: after reboot TLS everywhere cluster swift_proxy_tls_proxy and glance_api_tls_...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-containers
Version: 12.0 (Pike)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ga
: 12.0 (Pike)
Assignee: Martin André
QA Contact: Artem Hrechanychenko
Andrew Burden
URL:
Whiteboard:
Depends On:
Blocks: 1506020
TreeView+ depends on / blocked
 
Reported: 2017-11-27 16:41 UTC by Artem Hrechanychenko
Modified: 2018-02-05 19:18 UTC (History)
11 users (show)

Fixed In Version: openstack-tripleo-common-7.6.3-7.el7ost openstack-ec2-api-docker-12.0-20171201.1 openstack-glance-api-docker-12.0-20171201.1 openstack-neutron-server-docker-12.0-20171201.1 openstack-swift-proxy-server-docker-12.0-20171201.1 openstack-mistral-api-docker-1
Doc Type: Bug Fix
Doc Text:
Previously, if containers were shut down unexpectedly, Apache still left runtime files in the containers, which causes the containers to stay in a Restarting state after the host comes back up. If you use TLS everywhere, this means that the Glance and Swift services were unreachable after the host rebooted. This fix adds runtime cleanup in the container images startup scripts. Glance and Swift services are now functioning normally after the host reboots when deployed with TLS everywhere.
Clone Of:
Environment:
Last Closed: 2017-12-13 22:23:28 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1734879 None None None 2017-11-28 12:54:33 UTC
Red Hat Product Errata RHEA-2017:3462 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC
OpenStack gerrit 523404 None master: NEW tripleo-common: Prevent apache from listening on default ports (Ic2fd3f65b06755ae0869af7b39bcdbcb8ec03774) 2017-11-28 21:54:29 UTC

Description Artem Hrechanychenko 2017-11-27 16:41:38 UTC
Description of problem:
after reboot TLS everywhere cluster swift_proxy_tls_proxy and glance_api_tls_proxy are Restarting and we cannot launch instance


16:21:41 + swift_upload
16:21:41 + echo 'Checking if the swift container already exists.'
16:21:41 + swift list
16:21:41 + grep -q c1

16:22:14 ('Connection aborted.', BadStatusLine("''",))
16:22:14 There was an error running swift list. Exiting....

e602b7f2798c        192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171122.1                "kolla_start"            39 minutes ago      Restarting (0) 3 minutes ago                       glance_api_tls_proxy
c96e6587d52b        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server:12.0-20171122.1        "kolla_start"            40 minutes ago      Restarting (0) 3 minutes ago                       swift_proxy_tls_proxy

http://pastebin.test.redhat.com/535414

12.0-20171122.1  

How reproducible:
Always

puppet-openstacklib-11.3.0-2.el7ost.noarch
openstack-swift-account-2.15.1-3.el7ost.noarch
openstack-swift-proxy-2.15.1-3.el7ost.noarch
openstack-ironic-api-9.1.2-3.el7ost.noarch
openstack-tripleo-common-7.6.3-4.el7ost.noarch
openstack-mistral-common-5.2.0-1.el7ost.noarch
openstack-tripleo-validations-7.4.2-1.el7ost.noarch
openstack-nova-api-16.0.2-2.el7ost.noarch
openstack-nova-conductor-16.0.2-2.el7ost.noarch
openstack-neutron-openvswitch-11.0.1-5.el7ost.noarch
python-openstackclient-lang-3.12.0-1.el7ost.noarch
openstack-heat-engine-9.0.1-3.el7ost.noarch
openstack-mistral-executor-5.2.0-1.el7ost.noarch
openstack-nova-common-16.0.2-2.el7ost.noarch
openstack-heat-common-9.0.1-3.el7ost.noarch
openstack-keystone-12.0.0-3.el7ost.noarch
puppet-openstack_extras-11.3.0-2.el7ost.noarch
openstack-tripleo-common-containers-7.6.3-4.el7ost.noarch
openstack-tripleo-heat-templates-7.0.3-13.el7ost.noarch
openstack-mistral-engine-5.2.0-1.el7ost.noarch
openstack-zaqar-5.0.0-3.el7ost.noarch
openstack-swift-container-2.15.1-3.el7ost.noarch
openstack-heat-api-9.0.1-3.el7ost.noarch
openstack-ironic-common-9.1.2-3.el7ost.noarch
openstack-mistral-api-5.2.0-1.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.1-1.el7ost.noarch
openstack-nova-scheduler-16.0.2-2.el7ost.noarch
openstack-glance-15.0.0-3.el7ost.noarch
openstack-puppet-modules-11.0.0-1.el7ost.noarch
openstack-neutron-ml2-11.0.1-5.el7ost.noarch
openstack-heat-api-cfn-9.0.1-3.el7ost.noarch
openstack-tripleo-ui-7.4.3-4.el7ost.noarch
openstack-nova-compute-16.0.2-2.el7ost.noarch
openstack-neutron-11.0.1-5.el7ost.noarch
openstack-ironic-conductor-9.1.2-3.el7ost.noarch
python-openstackclient-3.12.0-1.el7ost.noarch
openstack-tempest-17.1.0-1.el7ost.noarch
openstack-selinux-0.8.11-1.el7ost.noarch
openstack-nova-placement-api-16.0.2-2.el7ost.noarch
openstack-swift-object-2.15.1-3.el7ost.noarch
openstack-neutron-common-11.0.1-5.el7ost.noarch
python-openstacksdk-0.9.17-1.el7ost.noarch
openstack-ironic-inspector-6.0.0-3.el7ost.noarch
openstack-tripleo-image-elements-7.0.1-1.el7ost.noarch


Steps to Reproduce:
Deploy TLS everywhere overcloud - http://tripleo.org/install/advanced_deployment/ssl.html#tls-everywhere-for-the-overcloud
Restart OC
check container status

Actual results:
swift_proxy_tls_proxy and glance_api_tls_proxy are Restarting

Comment 4 Martin André 2017-11-29 13:21:16 UTC
It is possible to patch the container images locally using the following commands (here for glance-api image):

(undercloud) [stack@undercloud-0 ~]$ cat glance_api_tls_workaround/Dockerfile 
FROM 192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171122.1
USER root
RUN sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf \
    && sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf \
    && echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_glance_extend_start
USER glance

(undercloud) [stack@undercloud-0 ~]$ docker build --rm -t 192.168.24.1:8787/rhosp12/openstack-glance-api:workaround ~/glance_api_tls_workaround/

(undercloud) [stack@undercloud-0 ~]$ docker push 192.168.24.1:8787/rhosp12/openstack-glance-api:workaround

Comment 5 Artem Hrechanychenko 2017-11-29 13:48:07 UTC
(In reply to Martin André from comment #4)
> It is possible to patch the container images locally using the following
> commands (here for glance-api image):
> 
> (undercloud) [stack@undercloud-0 ~]$ cat
> glance_api_tls_workaround/Dockerfile 
> FROM 192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171122.1
> USER root
> RUN sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf \
>     && sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf \
>     && echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/*
> /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_glance_extend_start
> USER glance
> 
> (undercloud) [stack@undercloud-0 ~]$ docker build --rm -t
> 192.168.24.1:8787/rhosp12/openstack-glance-api:workaround
> ~/glance_api_tls_workaround/
> 
> (undercloud) [stack@undercloud-0 ~]$ docker push
> 192.168.24.1:8787/rhosp12/openstack-glance-api:workaround

What about swift_tlx_proxy?

Comment 6 Martin André 2017-11-29 14:04:40 UTC
(In reply to Artem Hrechanychenko from comment #5)

> What about swift_tlx_proxy?

This would be needed for all the container images where we install apache via our overrides file. See the upstream patch at https://review.openstack.org/#/c/523404/ for all the containers that need to be rebuilt.

I've only posted here the procedure I used locally to patch the glance-api image as an example.

Comment 9 Artem Hrechanychenko 2017-11-30 19:33:39 UTC
FailedQA

openstack-tripleo-common-7.6.3-7.el7ost.noarch

b4423d0cfa5f        192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171129.1                "kolla_start"            37 minutes ago      Restarting (0) 2 minutes ago                       glance_api_tls_proxy
06c15e46b39e        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server:12.0-20171129.1        "kolla_start"            37 minutes ago      Restarting (0) 2 minutes ago                       swift_proxy_tls_proxy


INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Deleting /etc/httpd/conf.d/25-swift-proxy-api-proxy.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.d/25-swift-proxy-api-proxy.conf to /etc/httpd/conf.d/25-swift-proxy-api-proxy.conf
INFO:__main__:Deleting /etc/httpd/conf.d/ssl.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.d/ssl.conf to /etc/httpd/conf.d/ssl.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/access_compat.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/access_compat.load to /etc/httpd/conf.modules.d/access_compat.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/actions.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/actions.load to /etc/httpd/conf.modules.d/actions.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/alias.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/alias.conf to /etc/httpd/conf.modules.d/alias.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/alias.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/alias.load to /etc/httpd/conf.modules.d/alias.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/auth_basic.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/auth_basic.load to /etc/httpd/conf.modules.d/auth_basic.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/auth_digest.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/auth_digest.load to /etc/httpd/conf.modules.d/auth_digest.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authn_anon.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authn_anon.load to /etc/httpd/conf.modules.d/authn_anon.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authn_core.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authn_core.load to /etc/httpd/conf.modules.d/authn_core.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authn_dbm.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authn_dbm.load to /etc/httpd/conf.modules.d/authn_dbm.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authn_file.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authn_file.load to /etc/httpd/conf.modules.d/authn_file.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_core.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_core.load to /etc/httpd/conf.modules.d/authz_core.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_dbm.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_dbm.load to /etc/httpd/conf.modules.d/authz_dbm.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_groupfile.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_groupfile.load to /etc/httpd/conf.modules.d/authz_groupfile.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_host.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_host.load to /etc/httpd/conf.modules.d/authz_host.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_owner.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_owner.load to /etc/httpd/conf.modules.d/authz_owner.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/authz_user.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/authz_user.load to /etc/httpd/conf.modules.d/authz_user.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/autoindex.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/autoindex.conf to /etc/httpd/conf.modules.d/autoindex.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/autoindex.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/autoindex.load to /etc/httpd/conf.modules.d/autoindex.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/cache.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/cache.load to /etc/httpd/conf.modules.d/cache.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/cgi.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/cgi.load to /etc/httpd/conf.modules.d/cgi.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/dav.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/dav.load to /etc/httpd/conf.modules.d/dav.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/dav_fs.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/dav_fs.conf to /etc/httpd/conf.modules.d/dav_fs.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/dav_fs.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/dav_fs.load to /etc/httpd/conf.modules.d/dav_fs.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/deflate.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/deflate.conf to /etc/httpd/conf.modules.d/deflate.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/deflate.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/deflate.load to /etc/httpd/conf.modules.d/deflate.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/dir.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/dir.conf to /etc/httpd/conf.modules.d/dir.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/dir.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/dir.load to /etc/httpd/conf.modules.d/dir.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/env.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/env.load to /etc/httpd/conf.modules.d/env.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/expires.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/expires.load to /etc/httpd/conf.modules.d/expires.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/ext_filter.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/ext_filter.load to /etc/httpd/conf.modules.d/ext_filter.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/filter.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/filter.load to /etc/httpd/conf.modules.d/filter.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/headers.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/headers.load to /etc/httpd/conf.modules.d/headers.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/include.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/include.load to /etc/httpd/conf.modules.d/include.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/log_config.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/log_config.load to /etc/httpd/conf.modules.d/log_config.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/logio.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/logio.load to /etc/httpd/conf.modules.d/logio.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/mime.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/mime.conf to /etc/httpd/conf.modules.d/mime.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/mime.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/mime.load to /etc/httpd/conf.modules.d/mime.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/mime_magic.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/mime_magic.conf to /etc/httpd/conf.modules.d/mime_magic.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/mime_magic.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/mime_magic.load to /etc/httpd/conf.modules.d/mime_magic.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/negotiation.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/negotiation.conf to /etc/httpd/conf.modules.d/negotiation.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/negotiation.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/negotiation.load to /etc/httpd/conf.modules.d/negotiation.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/prefork.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/prefork.conf to /etc/httpd/conf.modules.d/prefork.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/prefork.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/prefork.load to /etc/httpd/conf.modules.d/prefork.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/proxy.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/proxy.conf to /etc/httpd/conf.modules.d/proxy.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/proxy.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/proxy.load to /etc/httpd/conf.modules.d/proxy.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/proxy_http.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/proxy_http.load to /etc/httpd/conf.modules.d/proxy_http.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/rewrite.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/rewrite.load to /etc/httpd/conf.modules.d/rewrite.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/setenvif.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/setenvif.conf to /etc/httpd/conf.modules.d/setenvif.conf
INFO:__main__:Deleting /etc/httpd/conf.modules.d/setenvif.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/setenvif.load to /etc/httpd/conf.modules.d/setenvif.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/socache_shmcb.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/socache_shmcb.load to /etc/httpd/conf.modules.d/socache_shmcb.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/speling.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/speling.load to /etc/httpd/conf.modules.d/speling.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/ssl.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/ssl.load to /etc/httpd/conf.modules.d/ssl.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/substitute.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/substitute.load to /etc/httpd/conf.modules.d/substitute.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/suexec.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/suexec.load to /etc/httpd/conf.modules.d/suexec.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/systemd.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/systemd.load to /etc/httpd/conf.modules.d/systemd.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/unixd.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/unixd.load to /etc/httpd/conf.modules.d/unixd.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/usertrack.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/usertrack.load to /etc/httpd/conf.modules.d/usertrack.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/version.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/version.load to /etc/httpd/conf.modules.d/version.load
INFO:__main__:Deleting /etc/httpd/conf.modules.d/vhost_alias.load
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf.modules.d/vhost_alias.load to /etc/httpd/conf.modules.d/vhost_alias.load
INFO:__main__:Deleting /etc/httpd/conf/httpd.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf/httpd.conf to /etc/httpd/conf/httpd.conf
INFO:__main__:Deleting /etc/httpd/conf/ports.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/httpd/conf/ports.conf to /etc/httpd/conf/ports.conf
INFO:__main__:Deleting /etc/swift/account-server.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/account-server.conf to /etc/swift/account-server.conf
INFO:__main__:Deleting /etc/swift/container-server.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/container-server.conf to /etc/swift/container-server.conf
INFO:__main__:Deleting /etc/swift/object-expirer.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/object-expirer.conf to /etc/swift/object-expirer.conf
INFO:__main__:Deleting /etc/swift/object-server.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/object-server.conf to /etc/swift/object-server.conf
INFO:__main__:Deleting /etc/swift/proxy-server.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/proxy-server.conf to /etc/swift/proxy-server.conf
INFO:__main__:Deleting /etc/swift/swift.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/swift.conf to /etc/swift/swift.conf
INFO:__main__:Deleting /etc/swift/account.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/account.ring.gz to /etc/swift/account.ring.gz
INFO:__main__:Deleting /etc/swift/container.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/container.ring.gz to /etc/swift/container.ring.gz
INFO:__main__:Deleting /etc/swift/object.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/object.ring.gz to /etc/swift/object.ring.gz
INFO:__main__:Deleting /etc/swift/account.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/account.builder to /etc/swift/account.builder
INFO:__main__:Deleting /etc/swift/container.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/container.builder to /etc/swift/container.builder
INFO:__main__:Deleting /etc/swift/object.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/object.builder to /etc/swift/object.builder
INFO:__main__:Deleting /etc/swift/backups/1512066633.account.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066633.account.builder to /etc/swift/backups/1512066633.account.builder
INFO:__main__:Deleting /etc/swift/backups/1512066633.container.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066633.container.builder to /etc/swift/backups/1512066633.container.builder
INFO:__main__:Deleting /etc/swift/backups/1512066633.object.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066633.object.builder to /etc/swift/backups/1512066633.object.builder
INFO:__main__:Deleting /etc/swift/backups/1512066638.account.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066638.account.builder to /etc/swift/backups/1512066638.account.builder
INFO:__main__:Deleting /etc/swift/backups/1512066638.account.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066638.account.ring.gz to /etc/swift/backups/1512066638.account.ring.gz
INFO:__main__:Deleting /etc/swift/backups/1512066638.object.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066638.object.builder to /etc/swift/backups/1512066638.object.builder
INFO:__main__:Deleting /etc/swift/backups/1512066638.object.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066638.object.ring.gz to /etc/swift/backups/1512066638.object.ring.gz
INFO:__main__:Deleting /etc/swift/backups/1512066639.container.builder
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066639.container.builder to /etc/swift/backups/1512066639.container.builder
INFO:__main__:Deleting /etc/swift/backups/1512066639.container.ring.gz
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/swift/backups/1512066639.container.ring.gz to /etc/swift/backups/1512066639.container.ring.gz
INFO:__main__:Deleting /etc/rsyncd.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rsyncd.conf to /etc/rsyncd.conf
INFO:__main__:Writing out command to execute
Running command: '/usr/sbin/httpd -DFOREGROUND'
[Thu Nov 30 19:27:58.198264 2017] [so:warn] [pid 1] AH01574: module access_compat_module is already loaded, skipping
[Thu Nov 30 19:27:58.198758 2017] [so:warn] [pid 1] AH01574: module actions_module is already loaded, skipping
[Thu Nov 30 19:27:58.198767 2017] [so:warn] [pid 1] AH01574: module alias_module is already loaded, skipping
[Thu Nov 30 19:27:58.198874 2017] [so:warn] [pid 1] AH01574: module auth_basic_module is already loaded, skipping
[Thu Nov 30 19:27:58.198882 2017] [so:warn] [pid 1] AH01574: module auth_digest_module is already loaded, skipping
[Thu Nov 30 19:27:58.198888 2017] [so:warn] [pid 1] AH01574: module authn_anon_module is already loaded, skipping
[Thu Nov 30 19:27:58.198894 2017] [so:warn] [pid 1] AH01574: module authn_core_module is already loaded, skipping
[Thu Nov 30 19:27:58.198986 2017] [so:warn] [pid 1] AH01574: module authn_dbm_module is already loaded, skipping
[Thu Nov 30 19:27:58.198998 2017] [so:warn] [pid 1] AH01574: module authn_file_module is already loaded, skipping
[Thu Nov 30 19:27:58.199108 2017] [so:warn] [pid 1] AH01574: module authz_core_module is already loaded, skipping
[Thu Nov 30 19:27:58.199229 2017] [so:warn] [pid 1] AH01574: module authz_dbm_module is already loaded, skipping
[Thu Nov 30 19:27:58.199243 2017] [so:warn] [pid 1] AH01574: module authz_groupfile_module is already loaded, skipping
[Thu Nov 30 19:27:58.199254 2017] [so:warn] [pid 1] AH01574: module authz_host_module is already loaded, skipping
[Thu Nov 30 19:27:58.199260 2017] [so:warn] [pid 1] AH01574: module authz_owner_module is already loaded, skipping
[Thu Nov 30 19:27:58.199267 2017] [so:warn] [pid 1] AH01574: module authz_user_module is already loaded, skipping
[Thu Nov 30 19:27:58.199273 2017] [so:warn] [pid 1] AH01574: module autoindex_module is already loaded, skipping
[Thu Nov 30 19:27:58.199281 2017] [so:warn] [pid 1] AH01574: module cache_module is already loaded, skipping
[Thu Nov 30 19:27:58.199612 2017] [so:warn] [pid 1] AH01574: module deflate_module is already loaded, skipping
[Thu Nov 30 19:27:58.199629 2017] [so:warn] [pid 1] AH01574: module dir_module is already loaded, skipping
[Thu Nov 30 19:27:58.199825 2017] [so:warn] [pid 1] AH01574: module env_module is already loaded, skipping
[Thu Nov 30 19:27:58.199838 2017] [so:warn] [pid 1] AH01574: module expires_module is already loaded, skipping
[Thu Nov 30 19:27:58.199847 2017] [so:warn] [pid 1] AH01574: module ext_filter_module is already loaded, skipping
[Thu Nov 30 19:27:58.199854 2017] [so:warn] [pid 1] AH01574: module filter_module is already loaded, skipping
[Thu Nov 30 19:27:58.199860 2017] [so:warn] [pid 1] AH01574: module headers_module is already loaded, skipping
[Thu Nov 30 19:27:58.199869 2017] [so:warn] [pid 1] AH01574: module include_module is already loaded, skipping
[Thu Nov 30 19:27:58.199989 2017] [so:warn] [pid 1] AH01574: module log_config_module is already loaded, skipping
[Thu Nov 30 19:27:58.200000 2017] [so:warn] [pid 1] AH01574: module logio_module is already loaded, skipping
[Thu Nov 30 19:27:58.200009 2017] [so:warn] [pid 1] AH01574: module mime_magic_module is already loaded, skipping
[Thu Nov 30 19:27:58.200018 2017] [so:warn] [pid 1] AH01574: module mime_module is already loaded, skipping
[Thu Nov 30 19:27:58.200032 2017] [so:warn] [pid 1] AH01574: module negotiation_module is already loaded, skipping
[Thu Nov 30 19:27:58.200239 2017] [so:warn] [pid 1] AH01574: module rewrite_module is already loaded, skipping
[Thu Nov 30 19:27:58.200252 2017] [so:warn] [pid 1] AH01574: module setenvif_module is already loaded, skipping
[Thu Nov 30 19:27:58.200645 2017] [so:warn] [pid 1] AH01574: module socache_shmcb_module is already loaded, skipping
[Thu Nov 30 19:27:58.200763 2017] [so:warn] [pid 1] AH01574: module substitute_module is already loaded, skipping
[Thu Nov 30 19:27:58.200779 2017] [so:warn] [pid 1] AH01574: module suexec_module is already loaded, skipping
[Thu Nov 30 19:27:58.200881 2017] [so:warn] [pid 1] AH01574: module unixd_module is already loaded, skipping
[Thu Nov 30 19:27:58.200980 2017] [so:warn] [pid 1] AH01574: module version_module is already loaded, skipping
[Thu Nov 30 19:27:58.200993 2017] [so:warn] [pid 1] AH01574: module vhost_alias_module is already loaded, skipping
[Thu Nov 30 19:27:58.201031 2017] [so:warn] [pid 1] AH01574: module dav_module is already loaded, skipping
[Thu Nov 30 19:27:58.201041 2017] [so:warn] [pid 1] AH01574: module dav_fs_module is already loaded, skipping
[Thu Nov 30 19:27:58.201936 2017] [so:warn] [pid 1] AH01574: module mpm_prefork_module is already loaded, skipping
[Thu Nov 30 19:27:58.201968 2017] [so:warn] [pid 1] AH01574: module proxy_module is already loaded, skipping
[Thu Nov 30 19:27:58.203300 2017] [so:warn] [pid 1] AH01574: module proxy_http_module is already loaded, skipping
[Thu Nov 30 19:27:58.203612 2017] [so:warn] [pid 1] AH01574: module ssl_module is already loaded, skipping
[Thu Nov 30 19:27:58.203646 2017] [so:warn] [pid 1] AH01574: module systemd_module is already loaded, skipping
[Thu Nov 30 19:27:58.203716 2017] [so:warn] [pid 1] AH01574: module cgi_module is already loaded, skipping
[Thu Nov 30 19:27:58.207916 2017] [alias:warn] [pid 1] AH00671: The Alias directive in /etc/httpd/conf.d/autoindex.conf at line 21 will probably never match because it overlaps an earlier Alias.
httpd (pid 1) already running

Comment 10 Martin André 2017-11-30 19:54:38 UTC
The fix is missing from the images tagged with 12.0-20171129.1:

(undercloud) [stack@undercloud-0 ~]$ docker run -ti --rm docker-registry.engineering.redhat.com/rhosp12/openstack-swift-proxy-server:12.0-20171129.1 cat /usr/local/bin/kolla_extend_start
(undercloud) [stack@undercloud-0 ~]$ docker run -ti --rm docker-registry.engineering.redhat.com/rhosp12/openstack-glance-api:12.0-20171129.1 cat /usr/local/bin/kolla_glance_extend_start
#!/bin/bash

# Bootstrap and exit if KOLLA_BOOTSTRAP variable is set. This catches all cases
# of the KOLLA_BOOTSTRAP variable being set, including empty.
if [[ "${!KOLLA_BOOTSTRAP[@]}" ]]; then
    glance-manage db_sync
    glance-manage db_load_metadefs
    exit 0
fi
(undercloud) [stack@undercloud-0 ~]$

Both images are missing the following line in their startup script:

if [[ "$(whoami)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi

Comment 11 Jon Schlueter 2017-12-01 12:36:09 UTC
openstack-ec2-api-docker
openstack-glance-api-docker
openstack-neutron-server-docker
openstack-swift-proxy-server-docker


4 Dockerfiles were updated

openstack-ec2-api

+RUN sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf \
+    && sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf \
+    && echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_extend_start
 

openstack-glance-api-docker

 RUN mkdir -p /openstack && \
     ln -s /usr/share/openstack-tripleo-common/healthcheck/glance-api /openstack/healthcheck && \
-    chmod -R a+rx /openstack
+    chmod -R a+rx /openstack && \
+    sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf && \
+    sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf && \
+    echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_glance_extend_start
 HEALTHCHECK CMD /openstack/healthcheck

openstack-neutron-server-docker

 
+RUN sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf \
+    && sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf \
+    && echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_neutron_extend_start
+

openstack-swift-proxy-docker

 RUN mkdir -p /openstack && \
     ln -s /usr/share/openstack-tripleo-common/healthcheck/swift-proxy /openstack/healthcheck && \
-    chmod -R a+rx /openstack
+    chmod -R a+rx /openstack && \
+    sed -i -r 's,^(Listen 80),#\1,' /etc/httpd/conf/httpd.conf && \
+    sed -i -r 's,^(Listen 443),#\1,' /etc/httpd/conf.d/ssl.conf && \
+    echo "if [[ "\$\(whoami\)" == 'root' ]]; then rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd*; fi" >> /usr/local/bin/kolla_extend_start
 HEALTHCHECK CMD /openstack/healthcheck

Comment 14 Jon Schlueter 2017-12-01 18:58:48 UTC
Also openstack-mistral-api-docker had similar set of changes

Comment 16 Artem Hrechanychenko 2017-12-04 21:01:29 UTC
VERIFIED

openstack-tripleo-common-7.6.3-8.el7ost.noarch
docker images from 12.0-20171201.1 
https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OT2_container_netiso_osp12_HA_TLS_everywhere/116/console


[heat-admin@overcloud-controller-0 ~]$ sudo docker ps |grep tls
d68ac20c0957        192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171201.1                "kolla_start"            31 minutes ago      Up 9 minutes (healthy)                       glance_api_tls_proxy
e3e4731bcd86        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server:12.0-20171201.1        "kolla_start"            31 minutes ago      Up 9 minutes (healthy)                       swift_proxy_tls_proxy
6281e0681a71        192.168.24.1:8787/rhosp12/openstack-redis:pcmklatest                          "kolla_start"            39 minutes ago      Up 9 minutes                                 redis_tls_proxy


(undercloud) [stack@undercloud-0 ~]$ ssh heat-admin@192.168.24.16 "sudo docker ps |grep tls "
8ca2e29e11c2        192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171201.1                "kolla_start"            34 minutes ago      Up 11 minutes (healthy)                       glance_api_tls_proxy
173c59b36b67        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server:12.0-20171201.1        "kolla_start"            34 minutes ago      Up 11 minutes (healthy)                       swift_proxy_tls_proxy
4ddc6da9d081        192.168.24.1:8787/rhosp12/openstack-redis:pcmklatest                          "kolla_start"            46 minutes ago      Up 11 minutes                                 redis_tls_proxy


3d247f4a6c8b        192.168.24.1:8787/rhosp12/openstack-glance-api:12.0-20171201.1                "kolla_start"            34 minutes ago      Up 12 minutes (healthy)                       glance_api_tls_proxy
ef2b218d2f16        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server:12.0-20171201.1        "kolla_start"            35 minutes ago      Up 12 minutes (healthy)                       swift_proxy_tls_proxy
cf3173da40e1        192.168.24.1:8787/rhosp12/openstack-redis:pcmklatest                          "kolla_start"            46 minutes ago      Up 12 minutes                                 redis_tls_proxy

Comment 20 errata-xmlrpc 2017-12-13 22:23:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.