Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1599609 - [UPGRADES][13] Ceph upgrade failed: Update to properties network of ExternalPort (OS::Neutron::Port) is not supported
Summary: [UPGRADES][13] Ceph upgrade failed: Update to properties network of ExternalP...
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-heat
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Zane Bitter
QA Contact: Ronnie Rasouli
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-10 07:50 UTC by Yurii Prokulevych
Modified: 2018-10-18 08:30 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-18 08:30:07 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Yurii Prokulevych 2018-07-10 07:50:41 UTC
Description of problem:
-----------------------
Major upgrade from RHOS-12 to RHOS-13 failed during ceph upgrade:

openstack overcloud ceph-upgrade run \
    --templates /usr/share/openstack-tripleo-heat-templates \
    --stack overcloud \
         -e /home/stack/composable_roles/roles/nodes.yaml \
        -e /home/stack/composable_roles/internal.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
        -e /home/stack/composable_roles/network/network-environment.yaml \
        -e /home/stack/composable_roles/enable-tls.yaml \
        -e /home/stack/composable_roles/inject-trust-anchor.yaml \
        -e /home/stack/composable_roles/public_vip.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
        -e /home/stack/composable_roles/hostnames.yaml \
        -e /home/stack/composable_roles/debug.yaml \
        -e /home/stack/composable_roles/config_heat.yaml \
        -e /home/stack/composable_roles/docker-images.yaml \
        -e /home/stack/composable_roles/docker-images.yaml \
        --roles-file /home/stack/composable_roles/roles/roles_data.yaml 2>&1
...
2018-07-09 18:09:34Z [overcloud-ControllerOpenstack-ema6kl3l64mj-1-tvqh66k4bl3v.StoragePort]: UPDATE_COMPLETE  state changed
2018-07-09 18:09:34Z [overcloud]: UPDATE_FAILED  Resource UPDATE failed: NotSupported: resources.ControllerOpenstack.resources[1].resources.ExternalPort.resources.ExternalPort: Update to properties network of ExternalPort (OS: : Neutron: : Port) is not supported.
2018-07-09 18:09:34Z [overcloud-Compute-iuaimtnpldny-0-g4pbwflptq4l.TenantPort]: UPDATE_COMPLETE  state changed
2018-07-09 18:09:35Z [overcloud-Compute-iuaimtnpldny-1-cx7qlk7bkins.ExternalPort]: UPDATE_COMPLETE  state changed
2018-07-09 18:09:35Z [overcloud-Networker-ha7plxntbkes-0-tulb7ybpxikc.NetworkConfig]: UPDATE_COMPLETE  state changed
2018-07-09 18:09:36Z [overcloud-Networker-ha7plxntbkes-0-tulb7ybpxikc.NetworkDeployment]: CREATE_IN_PROGRESS  state changed
2018-07-09 18:09:36Z [overcloud-Networker-ha7plxntbkes-1-tvuwiexdgjur.NetIpMap]: UPDATE_IN_PROGRESS  state changed
2018-07-09 18:09:36Z [overcloud-Networker-ha7plxntbkes-1-tvuwiexdgjur.NetworkConfig]: UPDATE_IN_PROGRESS  state changed
2018-07-09 18:09:37Z [overcloud-Database-gxncbo36rngs-2-cmvrk55zzto4.NetIpMap]: UPDATE_IN_PROGRESS  state changed
2018-07-09 18:09:37Z [overcloud-Database-gxncbo36rngs-2-cmvrk55zzto4.NetworkConfig]: UPDATE_IN_PROGRESS  state changed
2018-07-09 18:09:37Z [overcloud-Database-gxncbo36rngs-1-k3scqs7y6lov.InternalApiPort]: UPDATE_COMPLETE  state changed

 Stack overcloud UPDATE_FAILED 

overcloud.ControllerOpenstack.1.ExternalPort.ExternalPort:
  resource_type: OS::Neutron::Port
  physical_resource_id: fc01bce4-87f2-45c5-a242-dc37ccb34c60
  status: UPDATE_FAILED
  status_reason: |
    NotSupported: resources.ExternalPort: Update to properties network of ExternalPort (OS::Neutron::Port) is not supported.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-heat-common-10.0.1-0.20180411125640.el7ost.noarch
heat-cfntools-1.3.0-2.el7ost.noarch
openstack-heat-api-cfn-10.0.1-0.20180411125640.el7ost.noarch
openstack-heat-engine-10.0.1-0.20180411125640.el7ost.noarch
python-heat-agent-1.5.4-0.20180308153305.ecf43c7.el7ost.noarch
python2-heatclient-1.14.0-1.el7ost.noarch
puppet-heat-12.4.1-0.20180416203421.90e3fb0.el7ost.noarch
openstack-heat-api-10.0.1-0.20180411125640.el7ost.noarch
openstack-tripleo-heat-templates-8.0.2-43.el7ost.noarch

ceph-ansible-3.1.0-0.1.rc9.el7cp.noarch
puppet-ceph-2.5.0-1.el7ost.noarch


How reproducible:
-----------------
Not 100%

Steps to Reproduce:
-------------------
1. Follow upgrade procedure for RHOS-12 upgrade


Actual results:
---------------
Ceph upgrade failed

Expected results:
-----------------
Ceph upgrade succeeds


Additional info:
----------------
Re-running same command succeeded
Virtual env: 3controllers + 2computes + 3messaging + 3database + 3ceph + 2networker

Comment 2 Thomas Hervé 2018-07-10 09:03:24 UTC
I had a look at the environment. From heat's POV, properties didn't change, the port was still linked to the network named external. So what ought to have changed is the uuid of that network, which is not possible. We cache this value, so it's possible we had a race where we reset the cache while trying to get the UUID mapping to the name... Nothing actionable for now though.

Comment 3 Thomas Hervé 2018-10-18 08:30:07 UTC
Please reopen if that happens again, thanks.


Note You need to log in before you can comment on or make changes to this bug.