Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1362198 - [tiering]: Files of size greater than that of high watermark level should not be promoted
Summary: [tiering]: Files of size greater than that of high watermark level should not...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: tiering
Version: 3.8.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Milind Changire
QA Contact: bugs@gluster.org
URL:
Whiteboard: tier-migration
Depends On: 1349284
Blocks: 1299841
TreeView+ depends on / blocked
 
Reported: 2016-08-01 13:49 UTC by Milind Changire
Modified: 2016-08-18 05:49 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.8.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1349284
Environment:
Last Closed: 2016-08-12 09:48:35 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Milind Changire 2016-08-01 13:49:53 UTC
+++ This bug was initially created as a clone of Bug #1349284 +++

+++ This bug was initially created as a clone of Bug #1299841 +++

Description of problem:
Today, When a file is heated which is of size greater than the size of hot tier's high water mark, the file is promoted. Gluster should intelligently determine that the size of the file is more than high watermark and should not get promoted. 

with the following scenario,

hot tier capacity: 100Gb
high water mark - 70% i.e., 70Gb
File is created of size 80Gb

Having such file in hot tier has multiple consequences.

1) When this file is in hot tier, we always exceed high watermark and get into complete demotion mode. As a result, we make all other on hot tier a possible candidate for demotion.
2) No other files will be promoted until this file moves to cold tier
3) Possibility of moving such a huge file back and forth from cold to hot tier which might affect performance

Version-Release number of selected component (if applicable):
glusterfs-3.7.5-15.el7rhgs.x86_64

How reproducible:
Always

Steps to Reproduce:
1. create a gluster volume
2. create a 80Gb file, say file1
3. Attach hot tier of size 100Gb and set high watermark at 70%
4. heat the file - file1 created in step 2

Actual results:
File gets promoted

Expected results:
File should not get promoted

Additional info:

--- Additional comment from Red Hat Bugzilla Rules Engine on 2016-01-19 06:38:14 EST ---

This bug is automatically being proposed for the current z-stream release of Red Hat Gluster Storage 3 by setting the release flag 'rhgs‑3.1.z' to '?'. 

If this bug should be proposed for a different release, please manually change the proposed release flag.

--- Additional comment from Vijay Bellur on 2016-06-23 03:18:47 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote estimated block consumption > hi watermark) posted (#1) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-06-23 03:59:25 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote estimated block consumption > hi watermark) posted (#2) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-06-29 06:31:50 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#3) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-05 06:27:23 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#4) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-05 12:40:26 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#5) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-12 02:44:12 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#6) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-12 09:06:15 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#7) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-15 04:57:49 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#8) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Mohammed Rafi KC on 2016-07-15 06:33:30 EDT ---

upstream master patch : http://review.gluster.org/#/c/14780/

--- Additional comment from Vijay Bellur on 2016-07-19 06:54:31 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#9) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-20 02:52:21 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#10) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-20 09:04:04 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#11) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-21 03:30:03 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#12) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-27 09:54:35 EDT ---

REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#13) for review on master by Milind Changire (mchangir@redhat.com)

--- Additional comment from Vijay Bellur on 2016-07-29 10:14:40 EDT ---

COMMIT: http://review.gluster.org/14780 committed in master by Dan Lambright (dlambrig@redhat.com) 
------
commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4
Author: Milind Changire <mchangir@redhat.com>
Date:   Thu Jun 23 12:37:51 2016 +0530

    cluster/tier: dont promote if estimated block consumption > hi watermark
    
    Add test to fail promotion if estimated block consumption grows
    beyond hi watermark.
    
    Skip file migrations until next cycle if tier_get_fs_stat() fails
    in tier_migrate_using_query_file()
    
    Change-Id: Ice04572fa739c09109c4433e65965197482a7beb
    BUG: 1349284
    Signed-off-by: Milind Changire <mchangir@redhat.com>
    Reviewed-on: http://review.gluster.org/14780
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: mohammed rafi  kc <rkavunga@redhat.com>
    Reviewed-by: Dan Lambright <dlambrig@redhat.com>

Comment 1 Vijay Bellur 2016-08-01 13:52:57 UTC
REVIEW: http://review.gluster.org/15065 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#1) for review on release-3.8 by Milind Changire (mchangir@redhat.com)

Comment 2 Vijay Bellur 2016-08-01 15:25:39 UTC
REVIEW: http://review.gluster.org/15065 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#2) for review on release-3.8 by Milind Changire (mchangir@redhat.com)

Comment 3 Vijay Bellur 2016-08-05 08:12:15 UTC
COMMIT: http://review.gluster.org/15065 committed in release-3.8 by Niels de Vos (ndevos@redhat.com) 
------
commit 14b3a145a31e9f6760fb41bb104fcc2cf3bcc463
Author: Milind Changire <mchangir@redhat.com>
Date:   Thu Jun 23 12:37:51 2016 +0530

    cluster/tier: dont promote if estimated block consumption > hi watermark
    
    Add test to fail promotion if estimated block consumption grows
    beyond hi watermark.
    
    Skip file migrations until next cycle if tier_get_fs_stat() fails
    in tier_migrate_using_query_file()
    
    > Reviewed-on: http://review.gluster.org/14780
    > Smoke: Gluster Build System <jenkins@build.gluster.org>
    > CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    > Reviewed-by: mohammed rafi  kc <rkavunga@redhat.com>
    > Reviewed-by: Dan Lambright <dlambrig@redhat.com>
    
    (cherry picked from commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4)
    
    Change-Id: Ice04572fa739c09109c4433e65965197482a7beb
    BUG: 1362198
    Signed-off-by: Milind Changire <mchangir@redhat.com>
    Reviewed-on: http://review.gluster.org/15065
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Niels de Vos <ndevos@redhat.com>

Comment 4 Niels de Vos 2016-08-12 09:48:35 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.2, please open a new bug report.

glusterfs-3.8.2 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/announce/2016-August/000058.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.