Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1690177

Summary: Undercloud deploy - fail if ctlplane-network not defined in undercloud.conf
Product: Red Hat OpenStack Reporter: Harald Jensås <hjensas>
Component: openstack-tripleo-heat-templatesAssignee: Harald Jensås <hjensas>
Status: POST --- QA Contact: Sasha Smolyak <ssmolyak>
Severity: urgent Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: agurenko, bfournie, hbrock, hjensas, jslagle, mburns, osp-e2e-qe, ramishra, rhos-maint, yobshans
Target Milestone: z2Keywords: Triaged, ZStream
Target Release: 14.0 (Rocky)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1689384 Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1689384    
Bug Blocks:    

Description Harald Jensås 2019-03-19 00:13:49 UTC
+++ This bug was initially created as a clone of Bug #1689384 +++

Description of problem:
Undercloud install failed when there are no `ctlplane-subnet` subnet defined in undercloud.conf.
python-tripleoclient - does _not_ put `ctlplane-subnet` into the parameters.

It happened when install undercloud with custom undercloud.conf for Spine-Leaf network topology.
$ cat undercloud.conf
[DEFAULT]
local_interface = eth0
local_ip = 192.168.24.1/24
undercloud_ntp_servers=clock.redhat.com
hieradata_override = /home/stack/hiera_override.yaml
container_images_file=/home/stack/containers-prepare-parameter.yaml
container_insecure_registries=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem
enable_routed_networks = true
subnets = leaf0,leaf1,leaf2
local_subnet = leaf0

[leaf0]
cidr = 192.168.24.0/24
dhcp_start = 192.168.24.10
dhcp_end = 192.168.24.90
inspection_iprange = 192.168.24.100,192.168.24.190
gateway = 192.168.24.254
masquerade = False

[leaf1]
cidr = 192.168.34.0/24
dhcp_start = 192.168.34.10
dhcp_end = 192.168.34.90
inspection_iprange = 192.168.34.100,192.168.34.190
gateway = 192.168.34.254
masquerade = False

[leaf2]
cidr = 192.168.44.0/24
dhcp_start = 192.168.44.10
dhcp_end = 192.168.44.90
inspection_iprange = 192.168.44.100,192.168.44.190
gateway = 192.168.44.254
masquerade = False

$ cat tripleo-config-generated-env-files/undercloud_parameters.yaml
parameter_defaults:
 { .. SNIP .. }
  ControlPlaneStaticRoutes:
  - ip_netmask: 192.168.34.0/24
    next_hop: 192.168.24.254
  - ip_netmask: 192.168.44.0/24
    next_hop: 192.168.24.254
 { .. SNIP .. }
  IronicInspectorInterface: br-ctlplane
 { .. SNIP .. }
  IronicInspectorSubnets:
  - gateway: 192.168.24.254
    ip_range: 192.168.24.100,192.168.24.190
    netmask: 255.255.255.0
    tag: leaf0
  - gateway: 192.168.34.254
    ip_range: 192.168.34.100,192.168.34.190
    netmask: 255.255.255.0
    tag: leaf1
  - gateway: 192.168.44.254
    ip_range: 192.168.44.100,192.168.44.190
    netmask: 255.255.255.0
    tag: leaf2
 { .. SNIP .. }
  MasqueradeNetworks: {}
 { .. SNIP .. }
  UndercloudCtlplaneLocalSubnet: leaf0
  UndercloudCtlplaneSubnets:
    leaf0:
      AllocationPools:
      - end: 192.168.24.90
        start: 192.168.24.10
      HostRoutes: []
      NetworkCidr: 192.168.24.0/24
      NetworkGateway: 192.168.24.254
    leaf1:
      AllocationPools:
      - end: 192.168.34.90
        start: 192.168.34.10
      HostRoutes: []
      NetworkCidr: 192.168.34.0/24
      NetworkGateway: 192.168.34.254
    leaf2:
      AllocationPools:
      - end: 192.168.44.90
        start: 192.168.44.10
      HostRoutes: []
      NetworkCidr: 192.168.44.0/24
      NetworkGateway: 192.168.44.254
  UndercloudEnableRoutedNetworks: true

Yet, when https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/post_deploy/undercloud_post.yaml#L248 passes `UndercloudCtlplaneSubnets` to the script setting up the networks there is a `ctlplane-subnet`.

config={\"cloud_name\": \"undercloud\", \"enable_routed_networks\": true, \"home_dir\": \"/home/stack\", \"local_ip\": \"192.168.24.3\", \"local_subnet\": \"leaf0\", \"mtu\": 1500, \"nameservers\": [], \"physical_network\": \"ctlplane\", \"subnets\": {\"ctlplane-subnet\": {\"DhcpRangeEnd\": \"192.168.24.24\", \"DhcpRangeStart\": \"192.168.24.5\", \"NetworkCidr\": \"192.168.24.0/24\", \"NetworkGateway\": \"192.168.24.1\"}, \"leaf0\": {\"AllocationPools\": [{\"end\": \"192.168.24.90\", \"start\": \"192.168.24.10\"}], \"HostRoutes\": [], \"NetworkCidr\": \"192.168.24.0/24\", \"NetworkGateway\": \"192.168.24.254\"}, \"leaf1\": {\"AllocationPools\": [{\"end\": \"192.168.34.90\", \"start\": \"192.168.34.10\"}], \"HostRoutes\": [], \"NetworkCidr\": \"192.168.34.0/24\", \"NetworkGateway\": \"192.168.34.254\"}, \"leaf2\": {\"AllocationPools\": [{\"end\": \"192.168.44.90\", \"start\": \"192.168.44.10\"}], \"HostRoutes\": [], \"NetworkCidr\": \"192.168.44.0/24\", \"NetworkGateway\": \"192.168.44.254\"}}}
Version-Release number of selected component (if applicable):


2019-03-15 15:42:00.832 14286 WARNING tripleoclient.v1.undercloud.InstallUndercloud [ ] Running: sudo --preserve-env \
 openstack tripleo deploy \
   --standalone \
   --standalone-role Undercloud \
   --stack undercloud \
   --local-domain=localdomain \
   --local-ip=192.168.24.1/24 \
   --templates=/usr/share/openstack-tripleo-heat-templates/ \
   --networks-file=network_data_undercloud.yaml \
   ## The following file includes the values:
   ## https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L154-L159
   --heat-native -e /usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \
   -e /home/stack/containers-prepare-parameter.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml \
   --public-virtual-ip 192.168.24.2 \
   --control-virtual-ip 192.168.24.3 \
   -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml \
   --deployment-user stack \
   --output-dir=/home/stack \
   --cleanup \
   ## The following file should override what's in environments/undercloud.yaml#L154-L159
   #
   # Yet it looks like it's somehow merged?
   #
   -e /home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml \
   --hieradata-override=/home/stack/hiera_override.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml \
   --log-file=install-undercloud.log \
   -e /usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml

How reproducible:
100%

Steps to Reproduce:
1. Edit default undercloud.conf with Spine-Leaf network topoly
as https://gitlab.cee.redhat.com/yobshans/rhos-qe-edge-stuff/raw/master/osp15/undercloud/undercloud.conf
2. Start undercloud install

Actual results:
Deployment Failed!
More details
http://pastebin.test.redhat.com/738312

Expected results:
No failure.

Additional info:

--- Additional comment from Harald Jensås on 2019-03-19 00:11:53 UTC ---

The patch actually implements the workaround.

tbh, I'm surprised, why does these dicts are merge? The default merge strategy should be "overwrite" according to heat docs[1].
I don't see anywhere where we override any environment's merge strategy.



[1] https://docs.openstack.org/heat/latest/template_guide/environment.html#environment-merging