Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1063211 - rebalance : Number of files reported under "skipped+ failed" in rebalance status doesn't match with number of link files created by rebalance process
Summary: rebalance : Number of files reported under "skipped+ failed" in rebalance st...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute
Version: 2.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Nithya Balachandran
QA Contact: storage-qa-internal@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 1286130
TreeView+ depends on / blocked
 
Reported: 2014-02-10 09:52 UTC by spandura
Modified: 2015-11-27 11:41 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1286130 (view as bug list)
Environment:
Last Closed: 2015-11-27 11:41:08 UTC


Attachments (Terms of Use)

Description spandura 2014-02-10 09:52:28 UTC
Description of problem:
==========================
On a 1 x 3 replicate volume , added 3 more bricks to change the volume type to 2 x 3 distribute-replicate volume. Started rebalance on the files. 

Since the rep-0 bricks had higher disk space than the rep-1 bricks only link files were created on the rep-1 bricks and migration of data didn't happen . All the hashed files migration were skipped. But the rebalance status "skipped" count shows much lesser number than the number of link-to files. 

Also the other 2 bricks i.e client-1 and client-2 of rep-0 subvolume reports "scanned" count as "195252" , skipped" count as "0" , "failed" count as "0", "rebalanced-files" count as "0" and status to be "completed". What happened to all the scanned files by the rebalance process on client-1 and client-2 of rep-0 subvolume ? 

Version-Release number of selected component (if applicable):
===============================================================
glusterfs 3.4.0.59rhs built on Feb  4 2014 08:44:13

How reproducible:
====================

Steps to Reproduce:
=====================
1. Create 1 x 3 replicate volume. Start the volume. Create files/dirs from mount point. 

NOTE : Each the brick had 1.7TB of space

2. Add 3 more bricks to the volume making it 2 x 3 distribute-replicate volume.

NOTE : Each brick had 50GB of space 

3. Start rebalance. Rebalance of all the files are skipped. 

Actual results:
=========================
Number of files reported under "skipped" in rebalance status doesn't match with number of link files created  by rebalance process

root@rhs-client11 [Feb-10-2014- 3:34:55] >gluster v rebalance exporter status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes        195252             0         97827            completed            4773.00
                            rhs-client12                0        0Bytes        195252             0             0            completed            2196.00
                            rhs-client13                0        0Bytes        195252             0             0            completed            2196.00
                                    king                0        0Bytes          1302             0             0            completed              24.00
                                   hicks                0        0Bytes          1302             0             0            completed              24.00
                                  darrel                0        0Bytes          1302             0             0            completed              24.00
volume rebalance: exporter: success: 
root@rhs-client11 [Feb-10-2014- 3:35:00] >

On newly added brick files/link file counts
==========================================
root@king [Feb-10-2014-15:14:54] >ls /rhs/bricks/exporter/
create_dirs_files_multi_thread.py  user1   user2  user4  user6  user8
create_dirs_files.pl               user10  user3  user5  user7  user9

root@king [Feb-10-2014-15:14:59] >find /rhs/bricks/exporter/user* -type f | wc
 151569  151569 10236360

root@king [Feb-10-2014-15:15:14] >find /rhs/bricks/exporter/user* -type f | xargs ls -l | wc
 151569 1364121 16905396

root@king [Feb-10-2014-15:15:29] >find /rhs/bricks/exporter/user* -type f | xargs ls -l | grep -e "---------T" | wc
 151569 1364121 16905396
root@king [Feb-10-2014-15:15:44] >

Expected results:
====================

Additional info:
==================

root@rhs-client11 [Feb-10-2014- 2:18:46] >gluster v info
 
Volume Name: exporter
Type: Distributed-Replicate
Volume ID: 26eef3b2-e712-4f2c-ade6-9a26e3c85cc7
Status: Started
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: rhs-client11:/rhs/bricks/exporter
Brick2: rhs-client12:/rhs/bricks/exporter
Brick3: rhs-client13:/rhs/bricks/exporter
Brick4: king:/rhs/bricks/exporter
Brick5: darrel:/rhs/bricks/exporter
Brick6: hicks:/rhs/bricks/exporter
Options Reconfigured:
cluster.entry-self-heal: off
features.quota: on

Comment 3 Susant Kumar Palai 2015-11-27 11:41:08 UTC
Cloning this to 3.1. To be fixed in future.


Note You need to log in before you can comment on or make changes to this bug.