Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1360882 - Removing a cluster node leaves the node in the CIB if run on the node which is being removed
Summary: Removing a cluster node leaves the node in the CIB if run on the node which i...
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pcs
Version: 7.2
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Tomas Jelinek
QA Contact: cluster-qe@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-27 17:19 UTC by Miroslav Lisik
Modified: 2018-07-01 22:27 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1595829 (view as bug list)
Environment:
Last Closed:


Attachments (Terms of Use)

Description Miroslav Lisik 2016-07-27 17:19:22 UTC
Description of problem:
Running the 'pcs cluster node remove <node>' on the node which is to be removed itself, causes bad cib on the remaining cluster nodes.
Node is removed from corosync.conf but still present in cib.

Version-Release number of selected component (if applicable):
pcs-0.9.152-4.el7.x86_64

How reproducible:
always

Steps to Reproduce:

1. Setup cluster

[root@virt-020 ~]# pcs status nodes
Pacemaker Nodes:
 Online: virt-020.cluster-qe.lab.eng.brq.redhat.com virt-021.cluster-qe.lab.eng.brq.redhat.com virt-022.cluster-qe.lab.eng.brq.redhat.com
 Standby:
 Maintenance:
 Offline:
Pacemaker Remote Nodes:
 Online:
 Standby:
 Maintenance:
 Offline:

2. Run command on the node which is to be removed

[root@virt-020 ~]# pcs cluster node remove virt-020.cluster-qe.lab.eng.brq.redhat.com
virt-020.cluster-qe.lab.eng.brq.redhat.com: Stopping Cluster (pacemaker)...
virt-020.cluster-qe.lab.eng.brq.redhat.com: Successfully destroyed cluster
virt-021.cluster-qe.lab.eng.brq.redhat.com: Corosync updated
virt-022.cluster-qe.lab.eng.brq.redhat.com: Corosync updated


3. Check configuration on the remaining cluster nodes

[root@virt-021 ~]# pcs status nodes
Pacemaker Nodes:
 Online: virt-021.cluster-qe.lab.eng.brq.redhat.com virt-022.cluster-qe.lab.eng.brq.redhat.com
 Standby:
 Maintenance:
 Offline: virt-020.cluster-qe.lab.eng.brq.redhat.com
Pacemaker Remote Nodes:
 Online:
 Standby:
 Maintenance:
 Offline:
[root@virt-021 ~]# cat /etc/corosync/corosync.conf
totem {
    version: 2
    secauth: off
    cluster_name: STSRHTS9089
    transport: udpu
}

nodelist {
    node {
        ring0_addr: virt-021.cluster-qe.lab.eng.brq.redhat.com
        nodeid: 2
    }

    node {
        ring0_addr: virt-022.cluster-qe.lab.eng.brq.redhat.com
        nodeid: 3
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_logfile: yes
    logfile: /var/log/cluster/corosync.log
    to_syslog: yes
}
[root@virt-021 ~]# pcs cluster cib | sed -n "/<nodes>/,/<\/nodes>/"p
    <nodes>
      <node id="1" uname="virt-020.cluster-qe.lab.eng.brq.redhat.com"/>
      <node id="2" uname="virt-021.cluster-qe.lab.eng.brq.redhat.com"/>
      <node id="3" uname="virt-022.cluster-qe.lab.eng.brq.redhat.com"/>
    </nodes>


> The node virt-020.cluster-qe.lab.eng.brq.redhat.com is stil present in cib

[root@virt-022 ~]# pcs status nodes
Pacemaker Nodes:
 Online: virt-021.cluster-qe.lab.eng.brq.redhat.com virt-022.cluster-qe.lab.eng.brq.redhat.com
 Standby:
 Maintenance:
 Offline: virt-020.cluster-qe.lab.eng.brq.redhat.com
Pacemaker Remote Nodes:
 Online:
 Standby:
 Maintenance:
 Offline:
[root@virt-022 ~]# cat /etc/corosync/corosync.conf
totem {
    version: 2
    secauth: off
    cluster_name: STSRHTS9089
    transport: udpu
}

nodelist {
    node {
        ring0_addr: virt-021.cluster-qe.lab.eng.brq.redhat.com
        nodeid: 2
    }

    node {
        ring0_addr: virt-022.cluster-qe.lab.eng.brq.redhat.com
        nodeid: 3
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_logfile: yes
    logfile: /var/log/cluster/corosync.log
    to_syslog: yes
}
[root@virt-022 ~]# pcs cluster cib | sed -n "/<nodes>/,/<\/nodes>/"p
    <nodes>
      <node id="1" uname="virt-020.cluster-qe.lab.eng.brq.redhat.com"/>
      <node id="2" uname="virt-021.cluster-qe.lab.eng.brq.redhat.com"/>
      <node id="3" uname="virt-022.cluster-qe.lab.eng.brq.redhat.com"/>
    </nodes>

> The node virt-020.cluster-qe.lab.eng.brq.redhat.com is stil present in cib


Actual results:
Removed node is seen as offline and still present in cib on the cluster nodes.


Expected results:
Removed node is not seen in status or in cib on the cluster nodes.

Comment 2 Tomas Jelinek 2016-08-08 14:49:45 UTC
workaround: run "crm_node --force -R <removed node name>" on any of remaining cluster nodes

Comment 3 Tomas Jelinek 2017-07-21 08:58:09 UTC
(In reply to Tomas Jelinek from comment #2)
> workaround: run "crm_node --force -R <removed node name>" on any of
> remaining cluster nodes

This is now implemented in pcs:
pcs cluster node clear <removed node name>


Note You need to log in before you can comment on or make changes to this bug.