Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1355609 - [granular entry sh] - Clean up (stale) directory indices in the event of an `rm -rf` and also in the normal flow while a brick is down
Summary: [granular entry sh] - Clean up (stale) directory indices in the event of an `...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: replicate
Version: 3.8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Krutika Dhananjay
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-12 05:06 UTC by Krutika Dhananjay
Modified: 2016-08-12 09:47 UTC (History)
1 user (show)

Fixed In Version: glusterfs-3.8.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-12 09:47:17 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Krutika Dhananjay 2016-07-12 05:06:02 UTC
Description of problem:
This is to prevent on-disk inodes being leaked.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Vijay Bellur 2016-07-12 05:08:13 UTC
REVIEW: http://review.gluster.org/14896 (features/index: Delete parent dir indices when heal on it is complete) posted (#1) for review on release-3.8 by Krutika Dhananjay (kdhananj@redhat.com)

Comment 2 Krutika Dhananjay 2016-07-12 06:35:58 UTC
Two more patches to be ported.

Comment 3 Vijay Bellur 2016-07-14 07:02:38 UTC
REVIEW: http://review.gluster.org/14896 (features/index: Delete parent dir indices when heal on it is complete) posted (#3) for review on release-3.8 by Krutika Dhananjay (kdhananj@redhat.com)

Comment 4 Vijay Bellur 2016-07-14 07:02:41 UTC
REVIEW: http://review.gluster.org/14920 (afr, index: Clean up stale directory and file indices in granular entry sh) posted (#1) for review on release-3.8 by Krutika Dhananjay (kdhananj@redhat.com)

Comment 5 Vijay Bellur 2016-07-14 07:02:44 UTC
REVIEW: http://review.gluster.org/14921 (features/index: Ignore regular files when examining state of the inode wrt ENTRY_CHANGES) posted (#1) for review on release-3.8 by Krutika Dhananjay (kdhananj@redhat.com)

Comment 6 Vijay Bellur 2016-07-15 07:03:16 UTC
COMMIT: http://review.gluster.org/14896 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 05dacf07b4d427aa144aacb730e2296d9f96fe6a
Author: Krutika Dhananjay <kdhananj@redhat.com>
Date:   Wed Jun 22 14:52:58 2016 +0530

    features/index: Delete parent dir indices when heal on it is complete
    
            Backport of: http://review.gluster.org/#/c/14781
    
    In this patch, the state information about whether a directory
    gfid index is present or not is stored in the inode ctx with
    values IN and NOTIN. This saves index xl the need to perform
    stat() everytime an index_entry_create() is called.
    
    When a brick is restarted these in-memory inode ctx records will
    be gone. So when granular entry heal happens after a brick is restarted,
    and a post-op is done on the parent, if the state gotten from inode ctx
    is UNKNOWN, then index xl does a stat to initialize the state as IN or
    NOTIN. Note that this is a one-time operation for the lifetime of the
    brick. Such a change also helps avoid calling index_del() in
    xattrop_index_action() periodically even when granular self-heal is
    disabled or when the volume type is disperse.
    
    Change-Id: I037d0a8936381fbe3105e2e78489bfa571e5bdb0
    BUG: 1355609
    Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
    Reviewed-on: http://review.gluster.org/14896
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
    Smoke: Gluster Build System <jenkins@build.gluster.org>

Comment 7 Vijay Bellur 2016-07-15 13:43:16 UTC
COMMIT: http://review.gluster.org/14920 committed in release-3.8 by Jeff Darcy (jdarcy@redhat.com) 
------
commit 6c4908fe3f1a0b7900089b307b311940a2b7e1a1
Author: Krutika Dhananjay <kdhananj@redhat.com>
Date:   Mon Jun 27 15:21:26 2016 +0530

    afr, index: Clean up stale directory and file indices in granular entry sh
    
    	Backport of: http://review.gluster.org/14832
    
    Specifically when a directory tree is removed (rm -rf)
    while a brick is down, both the directory index and the
    name indices of the files and subdirs under it will remain.
    Self-heal will need to pick up these and remove them.
    
    Towards this, afr sh will now also crawl indices/entry-changes
    and call an rmdir on the dir if the directory index is stale.
    
    On the brick side, rmdir fop has been implemented for index xl,
    which would delete the directory index and its contents if present
    in a synctask.
    
    Change-Id: I08f45201adca56737ec2be1aab5433aebaefefd0
    BUG: 1355609
    Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
    Reviewed-on: http://review.gluster.org/14920
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

Comment 8 Vijay Bellur 2016-07-16 02:12:39 UTC
COMMIT: http://review.gluster.org/14921 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 8dccf7e222fa01c5e15a0ec1f0bc422abcde3779
Author: Krutika Dhananjay <kdhananj@redhat.com>
Date:   Thu Jul 7 14:33:20 2016 +0530

    features/index: Ignore regular files when examining state of the inode wrt ENTRY_CHANGES
    
    	Backport of: http://review.gluster.org/14870
    
    This prevents unnecessary sys_lstat() syscall when xattrop
    is operating on regular files.
    
    Also, to avoid (even) the one-time execution of sys_lstat()
    syscalls on disperse volumes or replicate volumes when granular
    entry self-heal is disabled, we pass a key in afr_selfheal_undo_pending()
    indicating to index that this xattrop is on a granular directory index.
    Index will accordingly decide whether or not to attempt the index state
    initialization.
    
    Change-Id: I86b69e83b6216c3371d997a4983d75d4be6a28b2
    BUG: 1355609
    Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
    Reviewed-on: http://review.gluster.org/14921
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

Comment 9 Niels de Vos 2016-08-12 09:47:17 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.2, please open a new bug report.

glusterfs-3.8.2 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/announce/2016-August/000058.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.