Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1357504 - haproxy blocks httpd because of Gnocchi configuration
Summary: haproxy blocks httpd because of Gnocchi configuration
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Jiri Stransky
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-18 11:47 UTC by Gonéri Le Bouder
Modified: 2016-07-18 16:00 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-18 16:00:55 UTC


Attachments (Terms of Use)

Description Gonéri Le Bouder 2016-07-18 11:47:35 UTC
Description of problem:

I'm deploying OSP 2016-07-14.1 OSP-d 2016-07-14.2. My controllers deployment get blocked by the following error:


[2016-07-18 11:33:54,987] (heat-config) [INFO] Warning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.
Warning: Scope(Class[Swift]): swift_hash_suffix has been deprecated and should be replaced with swift_hash_path_suffix, this will be removed as part of the N-cycle
Warning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead.
Warning: Scope(Class[Glance::Api]): The known_stores parameter is deprecated, use stores instead
Warning: Scope(Class[Glance::Api]): default_store not provided, it will be automatically set to glance.store.http.Store
Warning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.
Warning: Scope(Class[Nova::Api]): ec2_listen_port, ec2_workers and keystone_ec2_url are deprecated and have no effect. Deploy openstack/ec2-api instead.
Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated
Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated
Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated
Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated
Warning: Scope(Class[Neutron]): The neutron::network_device_mtu parameter is deprecated, use neutron::global_physnet_mtu instead.
Warning: Scope(Class[Neutron::Server]): identity_uri, auth_tenant, auth_user, auth_password, auth_region configuration options are deprecated in favor of auth_plugin and related options
Warning: Scope(Class[Neutron::Agents::Dhcp]): The dhcp_delete_namespaces parameter was removed in Mitaka, it does not take any affect
Warning: Scope(Class[Neutron::Agents::L3]): parameter external_network_bridge is deprecated
Warning: Scope(Class[Neutron::Agents::L3]): parameter router_delete_namespaces was removed in Mitaka, it does not take any affect
Warning: Scope(Class[Neutron::Agents::Metadata]): The auth_password parameter is deprecated and was removed in Mitaka release.
Warning: Scope(Class[Neutron::Agents::Metadata]): The auth_tenant parameter is deprecated and was removed in Mitaka release.
Warning: Scope(Class[Neutron::Agents::Metadata]): The auth_url parameter is deprecated and was removed in Mitaka release.
Warning: Scope(Class[Ceilometer::Api]): The keystone_auth_uri parameter is deprecated. Please use auth_uri instead.
Warning: Scope(Class[Ceilometer::Api]): The keystone_identity_uri parameter is deprecated. Please use identity_uri instead.
Warning: Scope(Class[Heat]): "admin_user", "admin_password", "admin_tenant_name" configuration options are deprecated in favor of auth_plugin and related options
Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 123 in file /etc/puppet/modules/gnocchi/manifests/api.pp
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Warning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Error: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Wrapped exception:
Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Error: /Stage[main]/Apache::Service/Service[httpd]/ensure: change from stopped to running failed: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Warning: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Skipping because of failed dependencies

[2016-07-18 11:33:54,987] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/c8386989-3a7c-497b-aaf6-8b84f9446d74.pp. [6]

{"deploy_stdout": "\u001b[mNotice: Compiled catalog for overcloud-controller-1.cwdg720xd01.org in environment production in 13.47 seconds\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[galera-ready]/returns: executed successfully\u001b[0m\nkUvVWaYxBXHjDPYCDXGKJRrUy\npassword\nregionOne\n-1\nTrue\nrabbit\nA3whzYvTQq3W29d3ucXQBXr3a\n192.168.140.23,192.168.140.22,192.168.140.26\nredis://:MgRxUkvkFarTakgKcenPyKx6c@192.168.140.20:6379/\n600\nnotifications\n0.0.0.0\nDefault\nDefault\nTrue\ndatabase\nFalse\nhttp://192.168.140.21:5000/v2.0\ndatabase\n4952\nhttp://192.168.140.21:5000\nhttp://192.168.120.106:35357\n\u001b[mNotice: /Stage[main]/Gnocchi::Storage::Ceph/Package[python-cradox]/ensure: created\u001b[0m\n/var/log/ceilometer\n192.168.140.22\nservice\nceilometer\n/\n60\nservice\nguest\n2\nA3whzYvTQq3W29d3ucXQBXr3a\nceilometer\n-1\nmongodb://192.168.140.23:27017,192.168.140.22:27017,192.168.140.26:27017/ceilometer?replicaSet=tripleo\nFalse\n8777\nservice\nhttp://:8041\ngnocchi_resources.yaml\nlow\n7a8t22yXNvpB3tNv7PADU7fMQ\ninternalURL\n\u001b[mNotice: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Dependency Service[httpd] has failures: true\u001b[0m\n\u001b[mNotice: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Triggered 'refresh' from 1 events\u001b[0m\n\u001b[mNotice: Finished catalog run in 28.25 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Swift]): swift_hash_suffix has been deprecated and should be replaced with swift_hash_path_suffix, this will be removed as part of the N-cycle\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Api]): The known_stores parameter is deprecated, use stores instead\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Api]): default_store not provided, it will be automatically set to glance.store.http.Store\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Api]): ec2_listen_port, ec2_workers and keystone_ec2_url are deprecated and have no effect. Deploy openstack/ec2-api instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron]): The neutron::network_device_mtu parameter is deprecated, use neutron::global_physnet_mtu instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Server]): identity_uri, auth_tenant, auth_user, auth_password, auth_region configuration options are deprecated in favor of auth_plugin and related options\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::Dhcp]): The dhcp_delete_namespaces parameter was removed in Mitaka, it does not take any affect\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::L3]): parameter external_network_bridge is deprecated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::L3]): parameter router_delete_namespaces was removed in Mitaka, it does not take any affect\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::Metadata]): The auth_password parameter is deprecated and was removed in Mitaka release.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::Metadata]): The auth_tenant parameter is deprecated and was removed in Mitaka release.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Neutron::Agents::Metadata]): The auth_url parameter is deprecated and was removed in Mitaka release.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Ceilometer::Api]): The keystone_auth_uri parameter is deprecated. Please use auth_uri instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Ceilometer::Api]): The keystone_identity_uri parameter is deprecated. Please use identity_uri instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Heat]): \"admin_user\", \"admin_password\", \"admin_tenant_name\" configuration options are deprecated in favor of auth_plugin and related options\u001b[0m\n\u001b[1;31mWarning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 123 in file /etc/puppet/modules/gnocchi/manifests/api.pp\u001b[0m\n\u001b[1;31mWarning: Not collecting exported resources without storeconfigs\u001b[0m\n\u001b[1;31mWarning: Not collecting exported resources without storeconfigs\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mWarning: Not collecting exported resources without storeconfigs\u001b[0m\n\u001b[1;31mWarning: Not collecting exported resources without storeconfigs\u001b[0m\n\u001b[1;31mWarning: Not collecting exported resources without storeconfigs\u001b[0m\n\u001b[1;31mError: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See \"systemctl status httpd.service\" and \"journalctl -xe\" for details.\nWrapped exception:\nExecution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See \"systemctl status httpd.service\" and \"journalctl -xe\" for details.\u001b[0m\n\u001b[1;31mError: /Stage[main]/Apache::Service/Service[httpd]/ensure: change from stopped to running failed: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See \"systemctl status httpd.service\" and \"journalctl -xe\" for details.\u001b[0m\n\u001b[1;31mWarning: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Skipping because of failed dependencies\u001b[0m\n", "deploy_status_code": 6}[root@overcloud-controller-1 heat-admin]#


httpd fails to start:
[root@overcloud-controller-1 heat-admin]# /usr/sbin/httpd -DFOREGROUND
(98)Address already in use: AH00073: make_sock: unable to listen for connections on address [::]:8041
(98)Address already in use: AH00073: make_sock: unable to listen for connections on address 0.0.0.0:8041
no listening sockets available, shutting down

Comment 2 Gonéri Le Bouder 2016-07-18 12:01:44 UTC
I can start httpd if I manually change httpd configuration to bind 8041 (gnocci) and 8042 (aodh) to 192.168.140.22 (my API IP) instead of *.

Comment 3 Gonéri Le Bouder 2016-07-18 12:25:10 UTC
I used this work around to be able to continue my deploy:

--- /var/lib/heat-config/heat-config-puppet/c8386989-3a7c-497b-aaf6-8b84f9446d74.orig.pp        2016-07-18 12:07:23.170545875 +0000
+++ /var/lib/heat-config/heat-config-puppet/c8386989-3a7c-497b-aaf6-8b84f9446d74.pp     2016-07-18 12:21:03.644542597 +0000
@@ -1208,7 +1208,9 @@
   include ::aodh::config
   include ::aodh::auth
   include ::aodh::client
-  include ::aodh::wsgi::apache
+  class { '::aodh::wsgi::apache':
+    bind_host => hiera('apache::ip'),
+  }
   class { '::aodh::api':
     manage_service => false,
     enabled        => false,
@@ -1251,6 +1253,7 @@
   }
   class { '::gnocchi::wsgi::apache' :
     ssl => false,
+    bind_host => hiera('apache::ip'),
   }
   class { '::gnocchi::metricd' :
     manage_service => false,

Comment 4 Marius Cornea 2016-07-18 13:44:20 UTC
Hi,

Could you make sure that the ServiceNetMap you're using for your deployment contains the new services introduced in OSP9 such as the GnocchiApiNetwork? You can get the full ServiceNetMap from /usr/share/openstack-tripleo-heat-templates/overcloud.yaml

Thanks

Comment 5 Gonéri Le Bouder 2016-07-18 14:07:56 UTC
It seems to be my mistake. I just merged the last ones from the list. I did realized some new entries as also at the top of the list.

Comment 6 Gonéri Le Bouder 2016-07-18 16:00:55 UTC
Thanks Marius. Sounds good now.


Note You need to log in before you can comment on or make changes to this bug.