Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1103845 - glusterfs backend does not support discard (libvirt)
Summary: glusterfs backend does not support discard (libvirt)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1055487 1136534
Blocks: GlusterThinProvisioning
TreeView+ depends on / blocked
 
Reported: 2014-06-02 17:51 UTC by Hai Huang
Modified: 2016-11-03 18:09 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 1037503
Environment:
Last Closed: 2016-11-03 18:09:24 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2577 normal SHIPPED_LIVE Moderate: libvirt security, bug fix, and enhancement update 2016-11-03 12:07:06 UTC

Comment 1 Peter Krempa 2014-06-03 06:52:15 UTC
This works with libvirt with the following configuration:

    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' discard='unmap'/>  <-- important
      <source protocol='gluster' name='gv0/img7'>
        <host name='gluster-node-1'/>
      </source>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>


In guest:
# cat /sys/bus/scsi/devices/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0/provisioning_mode
unmap

Setting test-only keyword.

Comment 2 yangyang 2014-11-07 07:52:53 UTC
Verified on 
libvirt-1.2.8-5.el7 
qemu-kvm-rhev-2.1.2-4.el7.x86_64
qemu-kvm-1.5.3-76.el7

test matrix:
qemu: qemu-kvm-rhev, qemu-kvm
host fs type: ext4, xfs
disk format: raw, qcow2
disk interface: scsi, ide, sata, virtio
disk fs type: ext4, xfs


Steps:
mount the gluster server to your host

mount -t glusterfs $ip:gluster-vol1 -o direct-io-mode=1 /tmp/gluster

1. Boot guest with the following xml

<disk type='network' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source protocol='gluster' name='gluster-vol1/rhel7-qcow2.img'>
<host name='$ip'/>
</source>
<backingStore/>
<target dev='vda' bus='virtio'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='qcow2' discard='unmap'/>
<source protocol='gluster' name='gluster-vol1/t-qcow2.img'>
<host name='$ip'/>
</source>
<backingStore/>
<target dev='sda' bus='scsi'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='qcow2' discard='ignore'/>
<source protocol='gluster' name='gluster-vol1/y-qcow2.img'>
<host name='$ip'/>
</source>
<backingStore/>
<target dev='sdb' bus='scsi'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='qcow2'/>
<source protocol='gluster' name='gluster-vol1/z-qcow2.img'>
<host name='$ip'/>
</source>
<backingStore/>
<target dev='sdc' bus='scsi'/>
</disk>

2. check qemu line
# ps -ef|grep qemu | grep discard

-drive file=gluster://$ip/gluster-vol1/rhel7-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=gluster://$ip/gluster-vol1/t-qcow2.img,if=none,id=drive-scsi0-0-0-0,format=qcow2,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://$ip/gluster-vol1/y-qcow2.img,if=none,id=drive-scsi0-0-0-1,format=qcow2,discard=ignore -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -drive file=gluster://$ip/gluster-vol1/z-qcow2.img,if=none,id=drive-scsi0-0-0-2,format=qcow2 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2 -netdev tap,fd=179,id=hostnet0,vhost=on,vhostfd=180 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8f:13:eb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0

3. guest ]# lsscsi
[2:0:0:0] disk QEMU QEMU HARDDISK 2.1. /dev/sda
[2:0:0:1] disk QEMU QEMU HARDDISK 2.1. /dev/sdb

[2:0:0:2] disk QEMU QEMU HARDDISK 2.1. /dev/sdc

guest ]# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode
unmap

4. In guest, make ext4 filesystem on sda, sdb
#mkfs.ext4 /dev/sda
#mkfs.ext4 /dev/sdb

#mkfs.ext4 /dev/sdc

5. In host, check the disk block

# stat t-qcow2.img
  File: ‘t-qcow2.img’
  Size: 35061760      Blocks: 68360      IO Block: 131072 regular file

# stat y-qcow2.img
  File: ‘y-qcow2.img’
  Size: 35061760      Blocks: 68368      IO Block: 131072 regular file

# stat z-qcow2.img
  File: ‘z-qcow2.img’
  Size: 35061760      Blocks: 68368      IO Block: 131072 regular file

6. In guest, mount sda and sdb
#mount /dev/sda /sda
#mount /dev/sdb /sdb

#mount /dev/sdc /sdc

7. In guest, dd a big file
dd if=/dev/urandom of=/sda/file bs=1M count=300 && sync
dd if=/dev/urandom of=/sdb/file bs=1M count=300 && sync

dd if=/dev/urandom of=/sdc/file bs=1M count=300 && sync

8. In host, check the disk block

# stat t-qcow2.img
  File: ‘t-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

# stat y-qcow2.img
  File: ‘y-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

# stat z-qcow2.img
  File: ‘z-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

9. In guest, remove the file
#rm /sda/file -f && sync
#rm /sdb/file -f && sync

#rm /sdc/file -f && sync

#fstrim /sda

#fstrim /sdb

#fstrim /sdc

10. In host, check the disk block

# stat t-qcow2.img
  File: ‘t-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

# stat y-qcow2.img
  File: ‘y-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

# stat z-qcow2.img
  File: ‘z-qcow2.img’
  Size: 350093312     Blocks: 683664     IO Block: 131072 regular file

Comment 3 yangyang 2014-12-04 05:29:25 UTC
Per comment #5 in qemu bug 1136534, mark it as failedQA. Will verify it after qemu bug 1136534 fixed.

Comment 5 yisun 2015-08-28 05:52:50 UTC
bz 1055487 and bz 1136534 have been moved to rhel7.3.0, could you pls move this one to 7.3.0? thx

Comment 7 yisun 2016-08-10 11:54:18 UTC
Verified with pkgs:
libvirt-2.0.0-4.el7.x86_64
qemu-kvm-rhev-2.6.0-17.el7.x86_64
glusterfs-server-3.7.9-11.el7rhgs.x86_64




HOST# virsh dumpxml virtlab_test 
...
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' discard='unmap'/>
      <source protocol='gluster' name='gluster-vol1/qcow2.img'>
        <host name='$IP'/>
      </source>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none' discard='unmap'/>
      <source protocol='gluster' name='gluster-vol1/raw.img'>
        <host name='$IP'/>
      </source>
      <target dev='sdb' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
...

HOST# virsh start virtlab_test
Domain virtlab_test started

HOST# ps -ef | grep qemu | grep unmap
qemu      7090     1  4 19:26 ?        00:00:08 /usr/libexec/qemu-kvm -name guest=virtlab_test, .... -drive file=gluster://10.66.5.82/gluster-vol1/qcow2.img,format=qcow2,if=none,id=drive-scsi0-0-0-0,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://10.66.5.82/gluster-vol1/raw.img,format=raw,if=none,id=drive-scsi0-0-0-1,cache=none,discard=unmap ....


GLUSTER# stat raw.img 
  File: ‘raw.img’
  Size: 1073741824	Blocks: 8          IO Block: 4096   regular file

GLUSTER# stat qcow2.img 
  File: ‘qcow2.img’
  Size: 197120    	Blocks: 400        IO Block: 4096   regular file


GUEST# lsscsi
[2:0:0:0]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sda 
[2:0:0:1]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdb 

GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode 
unmap

GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:1/scsi_disk/2\:0\:0\:1/provisioning_mode 
unmap

GUEST# mkfs.ext4 /dev/sda
GUEST# mkfs.ext4 /dev/sdb



GUEST# mkdir /sda
GUEST# mkdir /sdb
GUEST# mount /dev/sda /sda
GUEST# mount /dev/sdb /sdb

GLUSTER# stat raw.img 
  File: ‘raw.img’
  Size: 1073741824	Blocks: 66352      IO Block: 4096   regular file

GLUSTER# stat qcow2.img 
  File: ‘qcow2.img’
  Size: 559546368 	Blocks: 67736      IO Block: 4096   regular file





GUEST# dd if=/dev/zero of=/sda/file bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 4.76586 s, 110 MB/s

GUEST# dd if=/dev/zero of=/sdb/file bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 6.36393 s, 82.4 MB/s


GLUSTER# stat raw.img 
  File: ‘raw.img’
  Size: 1073741824	Blocks: 1123136    IO Block: 4096   regular file

GLUSTER# stat qcow2.img 
  File: ‘qcow2.img’
  Size: 559546368 	Blocks: 1164176    IO Block: 4096   regular file



GUEST# rm /sda/file -f
GUEST# fstrim /sda
GUEST# rm /sdb/file -f
GUEST# fstrim /sdb

GLUSTER# stat raw.img 
  File: ‘raw.img’
  Size: 1073741824	Blocks: 344784     IO Block: 4096   regular file


GLUSTER# stat qcow2.img 
  File: ‘qcow2.img’
  Size: 559546368 	Blocks: 68624      IO Block: 4096   regular file

Comment 9 errata-xmlrpc 2016-11-03 18:09:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html


Note You need to log in before you can comment on or make changes to this bug.