Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1406527 - GetVolumeInfoVDSCommand fails on the hosted-engine image which has 'lease = EXCLUSIVE'
Summary: GetVolumeInfoVDSCommand fails on the hosted-engine image which has 'lease = E...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent vote
Target Milestone: ovirt-4.1.0-beta
: ---
Assignee: Maor
QA Contact: Nikolai Sednev
URL:
Whiteboard:
Depends On:
Blocks: 1361511 1398443
TreeView+ depends on / blocked
 
Reported: 2016-12-20 19:46 UTC by Simone Tiraboschi
Modified: 2017-02-15 15:00 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-15 15:00:58 UTC
oVirt Team: Storage
rule-engine: ovirt-4.1+
rule-engine: blocker+


Attachments (Terms of Use)

Description Simone Tiraboschi 2016-12-20 19:46:36 UTC
Description of problem:
the auto-import of the hosted-engine VM from the engine fails with:

2016-12-20 20:23:04,905+02 INFO  [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[d74ea91a-8dc0-433e-a655-ffb6f018ce96=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_IMPORTED$VmName HostedEngine>, HostedEngine=<VM_NAME, ACTION_TYPE_FAILED_NAME_ALREADY_USED>]', sharedLocks='[d74ea91a-8dc0-433e-a655-ffb6f018ce96=<REMOTE_VM, ACTION_TYPE_FAILED_VM_IS_BEING_IMPORTED$VmName HostedEngine>]'}'
2016-12-20 20:23:04,910+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000311', ignoreFailoverLimit='false', storageDomainId='73de876d-1138-4d56-a930-85f5195357b9', imageGroupId='ffa67066-8c85-4392-9e03-57ee80567c58', imageId='9d48f64e-8638-40bc-ad98-e26386b70c7d'}), log id: 7617c4d9
2016-12-20 20:23:04,911+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] START, GetVolumeInfoVDSCommand(HostName = puma18.scl.lab.tlv.redhat.com, GetVolumeInfoVDSCommandParameters:{runAsync='true', hostId='24a83841-9862-447c-a5a5-f301fc86b8cb', storagePoolId='00000001-0001-0001-0001-000000000311', storageDomainId='73de876d-1138-4d56-a930-85f5195357b9', imageGroupId='ffa67066-8c85-4392-9e03-57ee80567c58', imageId='9d48f64e-8638-40bc-ad98-e26386b70c7d'}), log id: 12502694
2016-12-20 20:23:06,964+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Failed building DiskImage: java.lang.String cannot be cast to java.util.Map
2016-12-20 20:23:06,964+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand' return value '
VolumeInfoReturn:{status='Status [code=0, message=Done]'}
status = OK
domain = 73de876d-1138-4d56-a930-85f5195357b9
voltype = LEAF
description = Hosted Engine Image
parent = 00000000-0000-0000-0000-000000000000
format = RAW
generation = 0
image = ffa67066-8c85-4392-9e03-57ee80567c58
ctime = 1482255474
disktype = 2
legality = LEGAL
mtime = 0
apparentsize = 53687091200
children:
[]
pool = 
capacity = 53687091200
uuid = 9d48f64e-8638-40bc-ad98-e26386b70c7d
truesize = 3719168000
type = SPARSE
lease = EXCLUSIVE
'

I think that the issue has been introduced here:
https://gerrit.ovirt.org/#/c/68732

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. deploy hosted-engine
2. add the first regular storage domain
3. check the engine logs

Actual results:
Failed importing the Hosted Engine VM

Expected results:
sucesfully imported the Hosted Engine VM

Additional info:

Comment 2 Allon Mureinik 2016-12-21 07:29:33 UTC
Maor, according to Simone's analysis above, it seems as though one of your recent patches introduced this issue. Can you take a look please?

Comment 3 Maor 2016-12-21 08:43:16 UTC
(In reply to Allon Mureinik from comment #2)
> Maor, according to Simone's analysis above, it seems as though one of your
> recent patches introduced this issue. Can you take a look please?

Doesn't seem to be related, the patch only adds a null check.
Looking into the exception I might guess it is related to an old VDSM version.
commit 406a32bd2f8ff1a7fca9211e5c005ddef4cd2c79 has changed the return value of LEASE_FREE and sc.LEASE_EXCLUSIVE to a map.

Simone, is the VDSM you are using includes this commit?

Comment 4 Simone Tiraboschi 2016-12-21 08:48:30 UTC
Nikolai hit this, look at https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c12

He said he was using vdsm-4.18.999-1020.git1ff41b1.el7.centos.x86_64

Comment 5 Maor 2016-12-21 09:50:56 UTC
(In reply to Simone Tiraboschi from comment #4)
> Nikolai hit this, look at
> https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c12

I assume you meant https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c14

> 
> He said he was using vdsm-4.18.999-1020.git1ff41b1.el7.centos.x86_64

Based on the build's git commit 1ff41b1, this looks like a build that was based on https://gerrit.ovirt.org/#/c/67619/ which was merged in Dec 1st although the patch which changed the lease type (https://gerrit.ovirt.org/#/c/67610/) was merged later in Dec 14th.
can he test this with a newer VDSM build?

Comment 14 Red Hat Bugzilla Rules Engine 2016-12-27 07:04:25 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 15 Simone Tiraboschi 2017-01-09 12:54:49 UTC
Latest appliance build seams fine; moving to ON_QA, TestOnly

Comment 16 Nikolai Sednev 2017-01-24 16:07:41 UTC
Initial logs not being reproduced.
Works for me on these components on engine:
Red Hat Virtualization Manager Version: 4.1.0.1-0.1.el7 
rhev-guest-tools-iso-4.1-3.el7ev.noarch
rhevm-doc-4.1.0-1.el7ev.noarch
rhevm-dependencies-4.1.0-1.el7ev.noarch
rhevm-setup-plugins-4.1.0-1.el7ev.noarch
rhevm-4.1.0.1-0.1.el7.noarch
rhevm-guest-agent-common-1.0.12-3.el7ev.noarch
rhevm-branding-rhev-4.1.0-0.el7ev.noarch
Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild@x86-030.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

On host:
rhvm-appliance-4.1.20170119.1-1.el7ev.noarch
ovirt-imageio-common-0.5.0-0.el7ev.noarch
ovirt-host-deploy-1.6.0-1.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
rhev-release-4.0.6-1-001.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch
mom-0.5.8-1.el7ev.noarch
vdsm-4.19.2-2.el7ev.x86_64
libvirt-client-2.0.0-10.el7_3.4.x86_64
sanlock-3.4.0-1.el7.x86_64
ovirt-hosted-engine-setup-2.1.0-2.el7ev.noarch
ovirt-imageio-daemon-0.5.0-0.el7ev.noarch
ovirt-hosted-engine-ha-2.1.0-1.el7ev.noarch
ovirt-setup-lib-1.1.0-1.el7ev.noarch
Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild@x86-030.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

1-Deployed HE over iSCSI.
2-Added single NFS data storage domain.
3-Got hosted-storage auto-imported.


Moving to verified.


Note You need to log in before you can comment on or make changes to this bug.