Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1063463 - failed to obtain rebalance status on nodes that have been upgraded from 2.1u1 to 2.1u2
Summary: failed to obtain rebalance status on nodes that have been upgraded from 2.1u1...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: rhsc
Version: 2.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Dusmant
QA Contact: Dustin Tsang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-10 19:14 UTC by Dustin Tsang
Modified: 2016-02-18 00:15 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-02-11 17:03:21 UTC


Attachments (Terms of Use)

Description Dustin Tsang 2014-02-10 19:14:22 UTC
Description of problem:

Unable to obtain rebalance status on nodes upgraded from rhs 2.1u1 to 2.1u2.

logs from rhsc-log-collector: http://rhsqe-repo.lab.eng.blr.redhat.com/dustin/sosreport-LogCollector-20140210135000.tar.xz

Below is an excerpt from the engine.log.

2014-02-10 13:39:16,103 ERROR [org.ovirt.engine.core.bll.gluster.GetGlusterVolumeRebalanceStatusQuery] (ajp-/127.0.0.1:8702-8) Query GetGlusterVolumeRebalanceStatusQuery failed. Exception message is VdcBLLException: XML error
error: <cliOutput>
  <opRet>0</opRet>
  <opErrno>115</opErrno>
  <opErrstr />
  <volRebalance>
    <task-id>abbed207-cbf8-4e4b-b86e-6a5fb20ef7f2</task-id>
    <op>3</op>
    <nodeCount>4</nodeCount>
    <node>
      <nodeName>localhost</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>1000</lookups>
      <failures>0</failures>
      <status>3</status>
    </node>
    <node>
      <nodeName>10.14.16.140</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>1000</lookups>
      <failures>0</failures>
      <status>3</status>
    </node>
    <node>
      <nodeName>10.14.16.189</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>1000</lookups>
      <failures>0</failures>
      <status>3</status>
    </node>
    <node>
      <nodeName>10.14.16.181</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>1000</lookups>
      <failures>0</failures>
      <status>3</status>
    </node>
    <aggregate>
      <files>0</files>
      <size>0</size>
      <lookups>4000</lookups>
      <failures>0</failures>
    </aggregate>
  </volRebalance>
</cliOutput> (Failed with error GlusterXmlErrorException and code 4106)




Version-Release number of selected component (if applicable):
rhsc-cb17

How reproducible:
every time


Steps to Reproduce:
1. create a 3.2 cluster
2. add 4 rhs 2.1u1 gluster servers to the cluster
3. create a dist-replicate volume with a brick on each server
4. put a node in maintenance
5. using the same node change entitlements and upgrade the node to 2.1u2
6. activate the node
7. repeat steps 4-6 for each of the remaining nodes
8. change cluster version from 3.2 to 3.3
9. on the volume main tab, right click and select rebalance from context menu
10. when rebalance status in main tab shows sarted, click on the status menu and chose to show rebalance status


Actual results:
Rebalance Status modal appears with message 'could not obtain rebalance status of vol'

Expected results:
Rebalance Status should show progress of rebalance


Additional info:

Comment 2 Shubhendu Tripathi 2014-02-11 09:42:23 UTC
Verified the gluster version in machine 10.14.16.140 and shows the version as "

[root@rhs-21u1-20131122-d ~]# gluster --version
glusterfs 3.4.0.44rhs built on Nov 13 2013 08:03:02
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@rhs-21u1-20131122-d ~]#


This version fo gluster seems to be pretty old one.

Also the output format of "gluster volume rebalance <volname> status --xml" is very different if the command is run on a latest u2 node as below -

<cliOutput>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
  <volRebalance>
    <task-id>f3004e77-2fbd-480e-b688-85ce92ed753b</task-id>
    <op>3</op>
    <nodeCount>2</nodeCount>
    <node>
      <nodeName>localhost</nodeName>
      <id>a8a7afad-b450-4af5-916e-2a22ebf4fe38</id>
      <files>0</files>
      <size>0</size>
      <lookups>2038</lookups>
      <failures>0</failures>
      <skipped>0</skipped>
      <status>3</status>
      <statusStr>completed</statusStr>
      <runtime>6.00</runtime>
    </node>
    <node>
      <nodeName>10.70.43.7</nodeName>
      <id>f6c50756-5e14-4da0-bd20-c7d132111696</id>
      <files>2</files>
      <size>0</size>
      <lookups>2040</lookups>
      <failures>0</failures>
      <skipped>673</skipped>
      <status>3</status>
      <statusStr>completed</statusStr>
      <runtime>10.00</runtime>
    </node>
    <aggregate>
      <files>2</files>
      <size>0</size>
      <lookups>4078</lookups>
      <failures>0</failures>
      <skipped>673</skipped>
      <status>3</status>
      <statusStr>completed</statusStr>
      <runtime>10.00</runtime>
    </aggregate>
  </volRebalance>
</cliOutput>

Request you to upgrade to the latest version of gluster and verify the same.

Comment 3 Dustin Tsang 2014-02-11 17:03:21 UTC
not a bug, gluster failed to update while upgrading.


Note You need to log in before you can comment on or make changes to this bug.