Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1511973 - Difference between df output and libgfapi (96KB from 159GB)
Summary: Difference between df output and libgfapi (96KB from 159GB)
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: web-admin-tendrl-monitoring-integration
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Nishanth Thomas
QA Contact: sds-qe-bugs
URL:
Whiteboard:
Depends On:
Blocks: 1507930
TreeView+ depends on / blocked
 
Reported: 2017-11-10 14:29 UTC by Martin Kudlej
Modified: 2017-12-07 12:36 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1523216 (view as bug list)
Environment:
Last Closed: 2017-11-15 09:54:57 UTC


Attachments (Terms of Use)
screenshot from WA (deleted)
2017-11-10 14:38 UTC, Martin Kudlej
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1523216 None None None Never

Internal Links: 1523216

Description Martin Kudlej 2017-11-10 14:29:40 UTC
Description of problem:
I test Web Admin and it uses libgfapi to get volume utilization. I've checked that WA shows same values like ligfapi(I've used python bindings mentioned in Version-Release number of selected component section). But these values are different than output from command *df*:

$ df /mnt/volume_gama_disperse_4_plus_2x2
Filesystem                                 1K-blocks      Used Available Use%
_hostname_:volume_gama_disperse_4_plus_2x2 166772736 166772608       128 100%

$ python
>>> import gluster.gfapi
>>> vol = gluster.gfapi.Volume("_hostname_", "volume_gama_disperse_4_plus_2x2")
>>> vol.mount()
>>> stat = vol.statvfs("/")
>>> total = stat.f_blocks * stat.f_frsize
>>> free = stat.f_bfree * stat.f_frsize
>>> used = total - free
>>> print(total, used, free)
(170775281664L, 170775052288L, 229376L)
>>> # and in KB
>>> print((total + 0.0)/2**10, (used + 0.0)/2**10, (free + 0.0)/2**10)
(166772736.0, 166772512.0, 224.0)

As you can see total avail space is the same. There is difference between used space between *df* and *libgfapi* 96KB. Which is not much if we compare it with total available space. But I think there should not be any difference.

Version-Release number of selected component (if applicable):
glusterfs-3.8.4-52.el7_4.x86_64
glusterfs-3.8.4-52.el7rhgs.x86_64
glusterfs-api-3.8.4-52.el7rhgs.x86_64
glusterfs-cli-3.8.4-52.el7rhgs.x86_64
glusterfs-client-xlators-3.8.4-52.el7_4.x86_64
glusterfs-client-xlators-3.8.4-52.el7rhgs.x86_64
glusterfs-fuse-3.8.4-52.el7_4.x86_64
glusterfs-fuse-3.8.4-52.el7rhgs.x86_64
glusterfs-geo-replication-3.8.4-52.el7rhgs.x86_64
glusterfs-libs-3.8.4-52.el7_4.x86_64
glusterfs-libs-3.8.4-52.el7rhgs.x86_64
glusterfs-rdma-3.8.4-52.el7rhgs.x86_64
glusterfs-server-3.8.4-52.el7rhgs.x86_64
gluster-nagios-addons-0.2.10-2.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
libvirt-daemon-driver-storage-gluster-3.2.0-14.el7_4.3.x86_64
python-gluster-3.8.4-52.el7rhgs.noarch
vdsm-gluster-4.17.33-1.2.el7rhgs.noarch

+

master from python bindings for libgfapi https://github.com/gluster/libgfapi-python


How reproducible:
100%

Steps to Reproduce:
1. install Gluster cluster and setup there volume(in my case it is 159GB disperse 4+2x2)
2. try to copy data there as much as can
3. check libgfapi statvfs for volume utilization and output of *df* command on mounted volume

Actual results:
There is difference between available(used) space in statvfs from libgfapi and *df* command. It's 96KB difference from 159GB. 

Expected results:
libgfapi will provide same values like df command

Comment 2 Martin Kudlej 2017-11-10 14:38:37 UTC
Created attachment 1350543 [details]
screenshot from WA

Comment 3 Nishanth Thomas 2017-11-13 09:18:38 UTC
Atin, Looks like an issue to be addressed at gluster level. Could you please take a look?

Comment 4 Atin Mukherjee 2017-11-13 09:41:12 UTC
Poornima - This is the issue what I was discussing with you today morning. Request you to take a look at it.

Comment 5 Raghavendra Talur 2017-11-14 15:52:06 UTC
Please provide output of the following two commands

df --block-size=1K  /mnt/volume_gama_disperse_4_plus_2x2
df --block-size=1KB /mnt/volume_gama_disperse_4_plus_2x2

Comment 6 Raghavendra Talur 2017-11-14 23:38:17 UTC
Ok, this difference is because FUSE uses a block size and fragment size of 128K instead of using the backend filesystem's block size.

When statvfs call is made, this is the statvfs buffer content

      blocks  bfree   bavail
gfapi 259584, 248433, 251028
brick 259584, 251028, 251028
fuse    8112,   7763,   7844


As you can see, gfapi and brick match in total blocks and bavail.
There is a difference in bfree because of posix xlator. Posix xlator deducts 1% of the total blocks from free blocks.

The numbers in fuse are the numbers in gfapi row divided by 32(because the numbers obtained from brick are for 4K blocks and fuse wants to communicate in terms of 128K blocks). Here, we are converting into a larger unit and value is rounded off.

df on a mount point would get data from fuse and hence the discrepancy.


@atin, IMO the component to fix is fuse.

Comment 7 Martin Kudlej 2017-11-15 09:50:20 UTC
@Raghavendra feel free to file a bug (or move this one) to different component. I've just reported the difference between fuse (used from command line) and libgfapi (used in Web Admin).


Note You need to log in before you can comment on or make changes to this bug.