Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1511426

Summary: Cannot disable nova-compute on the node that was added to the stack (scaling up)
Product: Red Hat OpenStack Reporter: Gurenko Alex <agurenko>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED CURRENTRELEASE QA Contact: Joe H. Rahme <jhakimra>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 12.0 (Pike)CC: agurenko, berrange, dansmith, eglynn, kchamart, mbooth, owalsh, rbryant, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-04 08:50:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Gurenko Alex 2017-11-09 10:26:25 UTC
Description of problem: After adding new compute node to the stack (scaling up) and later attempt to remove it from the stack (removal/replacement) nova-compute cannot be disabled on this node. I've used blacklisting feature during scaling up, I'm not sure whether it's related.


Version-Release number of selected component (if applicable): 2017-11-03.4


How reproducible: tried twice, with compute nodes and controller + compute nodes blacklisted, both times new nodes behaves as described.


Steps to Reproduce:
1. Provision 3 Controllers and 3 Compute nodes with InfraRed
2. Deploy 3 Controllers and 2 Computes
3. Add 1 controller and 1 compute to blacklist.yaml
4. Perform scaling up with blacklist.yaml used
5. After scale up complete successful try executing "(overcloud) [stack@undercloud-0 ~]$ openstack compute service set compute-2.localdomain nova-compute --disable"

Actual results:

Failed to set service status to disabled
Compute service nova-compute of host compute-2.localdomain failed to set.

Expected results:

Service disables successfully

Additional info:

No errors in heat, nova logs. Service disables fine on the initially deployed nodes (compute-0, compute-1).

Comment 1 Gurenko Alex 2017-11-09 10:36:40 UTC
Seems like another issue with Unified CLI, nova service-disable <ID> from nova service-list works fine

Comment 2 Matthew Booth 2017-11-16 17:18:05 UTC
Hi, Gurenko,

Could you please provide the exact output of the command in your step 5? We'll probably also want sosreports from the controller nodes after the failure.

Thanks,

Matt

Comment 3 Gurenko Alex 2017-11-22 15:39:43 UTC
(In reply to Matthew Booth from comment #2)
> Hi, Gurenko,
> 
> Could you please provide the exact output of the command in your step 5?
> We'll probably also want sosreports from the controller nodes after the
> failure.
> 
> Thanks,
> 
> Matt

The exact error in CLI is:

(overcloud) [stack@undercloud-0 ~]$ openstack compute service set compute-2.redhat.com nova-compute --disable
Failed to set service status to disabled
Compute service nova-compute of host compute-2.redhat.com failed to set.

SOS report is not yet available for the overcloud nodes on RHOS12, so maybe some specific log I can get?

Comment 4 Matthew Booth 2017-11-23 16:27:50 UTC
Could you please supply all nova api and nova conductor logs from all controller nodes?

Comment 5 Ollie Walsh 2017-11-23 18:58:41 UTC
I expect this is just a side-effect of https://bugzilla.redhat.com/show_bug.cgi?id=1516429

Comment 6 Matthew Booth 2017-11-24 11:38:45 UTC
(In reply to Ollie Walsh from comment #5)
> I expect this is just a side-effect of
> https://bugzilla.redhat.com/show_bug.cgi?id=1516429

In that case, we'd expect to see an error about cell mapping in the api logs? I'd still like to see the logs.

Comment 7 Gurenko Alex 2017-11-28 14:53:52 UTC
I've just tried build 2017-11-27.5 and this command worked for me on a split stack environment, I'm going to re-test it on regular environment as well ASAP

Comment 8 Matthew Booth 2017-11-30 16:40:03 UTC
Thanks for the update. Given that Ollie thinks its resolved can you just close the bug if your retest also works?

Comment 9 Gurenko Alex 2017-12-04 08:50:21 UTC
Yes, I can confirm on a latest build 2017-11-29.2 as well, it works now.