Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1601919 - [Upgrade][12][composable roles][ceph][tls] ERROR: 508828 -- ERROR configuring nova_placement
Summary: [Upgrade][12][composable roles][ceph][tls] ERROR: 508828 -- ERROR configuring...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Emilien Macchi
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-17 13:08 UTC by Irina Petrova
Modified: 2018-11-16 15:44 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-18 10:37:52 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Irina Petrova 2018-07-17 13:08:21 UTC
Description of problem:

Multiple bugs/failures in RHOSP-11 w/ Composable Roles and Ceph.

The latest one of them shows the following backtrace:

[...]              
2018-07-17 08:09:38Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.GpuComputeArtifactsDeploy]: UPDATE_IN_PROGRESS  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.GpuComputeArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ObjectStorageArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ComputeArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.CephStorageArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.BlockStorageArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.....ComputeArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:09:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerArtifactsDeploy]: UPDATE_COMPLETE  state changed
2018-07-17 08:10:27Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerHostPrepDeployment]: UPDATE_COMPLETE  state changed
2018-07-17 08:10:27Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerPreConfig]: UPDATE_IN_PROGRESS  state changed
2018-07-17 08:11:05Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerPreConfig]: UPDATE_COMPLETE  state changed
2018-07-17 08:11:06Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.BlockStorageDeployment_Step1]: UPDATE_IN_PROGRESS  state changed
2018-07-17 08:11:06Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerDeployment_Step1]: UPDATE_IN_PROGRESS  state changed
2018-07-17 08:11:07Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.CephStorageDeployment_Step1]: UPDATE_IN_PROGRESS  state changed
2018-07-17 08:11:09Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.BlockStorageDeployment_Step1]: UPDATE_COMPLETE  state changed
2018-07-17 08:11:43Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.CephStorageDeployment_Step1]: UPDATE_COMPLETE  state changed
2018-07-17 08:12:40Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf.ControllerDeployment_Step1]: UPDATE_FAILED  resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2018-07-17 08:12:41Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev-AllNodesPostUpgradeSteps-o4vhq5v26ayf]: UPDATE_FAILED  resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2018-07-17 08:12:41Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev.AllNodesPostUpgradeSteps]: UPDATE_FAILED  resources.AllNodesPostUpgradeSteps: resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2018-07-17 08:12:42Z [overcloud-AllNodesDeploySteps-cfl6qxic7iev]: UPDATE_FAILED  resources.AllNodesPostUpgradeSteps: resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2018-07-17 08:12:43Z [AllNodesDeploySteps]: UPDATE_FAILED  resources.AllNodesDeploySteps: resources.AllNodesPostUpgradeSteps: resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2018-07-17 08:12:44Z [overcloud]: UPDATE_FAILED  resources.AllNodesDeploySteps: resources.AllNodesPostUpgradeSteps: resources.ControllerDeployment_Step1: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
                   
 Stack overcloud UPDATE_FAILED                 
                   
overcloud.AllNodesDeploySteps.AllNodesPostUpgradeSteps.ControllerDeployment_Step1.1:
  resource_type: OS::Heat::StructuredDeployment            
  physical_resource_id: 9314a6aa-8752-4c86-8c70-ef7135c8584f
  status: CREATE_FAILED                        
  status_reason: |    
    Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: | 
    ...            
            "2018-07-17 10:12:37,564 INFO: 508830 -- Finished processing puppet configs", 
            "2018-07-17 10:12:37,565 ERROR: 508828 -- ERROR configuring nova_placement", 
            "2018-07-17 10:12:37,565 ERROR: 508828 -- ERROR configuring nova"
        ]          
    }              
        to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/ad42cb93-3b79-47dc-964e-9fa8c04c4632_playbook.retry
                   
    PLAY RECAP *********************************************************************
    localhost                  : ok=6    changed=1    unreachable=0    failed=1   
                   
    (truncated, view all with --long)          
  deploy_stderr: | 
                   
overcloud.AllNodesDeploySteps.AllNodesPostUpgradeSteps.ControllerDeployment_Step1.0:
  resource_type: OS::Heat::StructuredDeployment            
  physical_resource_id: 4ddc9fc2-ef76-4541-8258-a7164c9f9bbf
  status: CREATE_FAILED                        
  status_reason: | 
    CREATE aborted 
  deploy_stdout: | 
None               
  deploy_stderr: | 
None               
overcloud.AllNodesDeploySteps.AllNodesPostUpgradeSteps.ControllerDeployment_Step1.2:
  resource_type: OS::Heat::StructuredDeployment            
  physical_resource_id: 63172e26-0b11-4d2d-ba03-83413f979395
  status: CREATE_FAILED                        
  status_reason: | 
    CREATE aborted 
  deploy_stdout: | 
    ...            
            "2018-07-17 10:12:37,381 INFO: 650672 -- Finished processing puppet configs", 
            "2018-07-17 10:12:37,381 ERROR: 650671 -- ERROR configuring nova_placement", 
            "2018-07-17 10:12:37,382 ERROR: 650671 -- ERROR configuring nova"
        ]          
    }              
        to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/2b93d853-dcf4-4829-89b7-d49ea53aa664_playbook.retry
                   
    PLAY RECAP *********************************************************************
    localhost                  : ok=6    changed=1    unreachable=0    failed=1   
                   
    (truncated, view all with --long)          
  deploy_stderr: | 
                   
Heat Stack update failed.                      
Heat Stack update failed.                      


Version-Release number of selected component (if applicable):
Latest RHOSP-11 rpms.

How reproducible:
Every time.

Steps to Reproduce:
1. run minor upgrade
2. prep templates for the upgrade change from rhosp11 to rhosp12
3. run openstack overcloud deploy...

Actual results:
Upgrade fails.

Expected results:
Upgrade successful.

Additional info:

Comment 6 Irina Petrova 2018-07-18 10:37:52 UTC
Problem fixed after removing (in environments/*environment.yaml):

  ComputeExtraConfig:
    # Fix NOVA bug 1455793, VA 20/10/2017 
    nova::config::nova_config:
            notifications/notification_format:
                value: 'unversioned'

Reason: 'nova::config::nova_config' is read-only now.
If needed, use 'nova::notification_format' instead.

Example: nova::notification_format: 'unversioned'
(which is what we have by default currently)

Comment 7 Irina Petrova 2018-11-16 15:44:45 UTC
The following knowledge article might be helpful in case someone would like to change the default value of an already defined variable (such as 'notification_format'):

https://access.redhat.com/solutions/3699241


Note You need to log in before you can comment on or make changes to this bug.