Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1101541

Summary: [engine-backend] [external-provider] engine failure while createVolume task is running in vdsm (as part of importing an image from glance), leaves image in LOCKED state
Product: Red Hat Enterprise Virtualization Manager Reporter: Elad <ebenahar>
Component: ovirt-engineAssignee: Daniel Erez <derez>
Status: CLOSED CURRENTRELEASE QA Contact: Elad <ebenahar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.4.0CC: amureini, fsimonce, gklein, iheim, lpeer, rbalakri, Rhev-m-bugs, scohen, tnisan, yeylon
Target Milestone: ---Keywords: ZStream
Target Release: 3.5.0   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: vt1.3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1102785 (view as bug list) Environment:
Last Closed: 2015-02-16 19:09:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1102785, 1142923, 1156165    
Attachments:
Description Flags
logs from engine and vdsm none

Description Elad 2014-05-27 13:27:41 UTC
Created attachment 899500 [details]
logs from engine and vdsm

Description of problem:
In a situation which engine crashes during createVolume, as part of import an image from glance, disk gets stuck in LOCKED stete.

Version-Release number of selected component (if applicable):
av9.2
rhevm-3.4.0-0.21.el6ev.noarch
vdsm-4.14.7-3.el6ev.x86_64


How reproducible:
Always

Steps to Reproduce:
On a shared DC with storage domains attached and integrated glance repository with images:
1. Import an image from glance repository.
2. Restart ovirt-engine service during createVolume task in vdsm

Actual results:
In case engine crashed before it got the response of the image guuid from vdsm, it assume that the image guuid is  00000000-0000-0000-0000-000000000000 and sends vdsm to perform the second phase of image import - downloadImage with image guuid of 00000000-0000-0000-0000-000000000000.

Thread-113::DEBUG::2014-05-27 11:30:26,000::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 100, 'message': u"Invalid parameter: 'volUUID=00000000-0000-0000-0000-000000000000'", 'taskState
': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'ae50a07b-1458-4de2-ad34-2cea5cce5fbd'}
Thread-113::DEBUG::2014-05-27 11:30:26,000::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'ae50a07b-1458-4de2-ad34-2cea5cce5fbd': {'code': 100, 'message': u"Invalid parameter: 'volUUID=00000000-000
0-0000-0000-000000000000'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'ae50a07b-1458-4de2-ad34-2cea5cce5fbd'}}
Thread-113::INFO::2014-05-27 11:30:26,000::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'ae50a07b-1458-4de2-ad34-2cea5cce5fbd': {'code': 100, 'mess
age': u"Invalid parameter: 'volUUID=00000000-0000-0000-0000-000000000000'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'ae50a07b-1458-4de2-ad34-2cea5cce5fbd'}}}
Thread-113::DEBUG::2014-05-27 11:30:26,000::task::1185::TaskManager.Task::(prepare) Task=`a5899d91-a84c-476e-bfb2-db7bbb316a0e`::finished: {'allTasksStatus': {'ae50a07b-1458-4de2-ad34-2cea5cce5fbd': {'code': 100,
'message': u"Invalid parameter: 'volUUID=00000000-0000-0000-0000-000000000000'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'ae50a07b-1458-4de2-ad34-2cea5cce5fbd'}}}


Disk gets stuck in LOCKED.

Expected results:
Not sure what should happen if engine is unable to get the response of the image guuid from vdsm. 

Additional info: logs from engine and vdsm

Comment 2 Daniel Erez 2014-06-30 13:23:33 UTC
As commented in bug 1102785 comment #3. There's another unresolved issue here. Pushed a patch for addressing it.

Comment 3 Elad 2014-08-26 11:06:30 UTC
If engine is stopped during createVolume task which is being executed by vdsm, when it comes up, it's being updated about the state of the task and in the DownloadImageVDSCommand it sends the right image guuid and not all zeros uuid:

2014-08-26 13:48:09,281 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DownloadImageVDSCommand] (org.ovirt.thread.pool-8-thread-25) -- downloadImage parameters:
                dstSpUUID=00000002-0002-0002-0002-000000000267
                dstSdUUID=68f09ca6-786f-42f3-ace2-2e9e21e5dd33
                dstImageGUID=fa679e23-bd87-474c-8d5d-e24500dece75
                dstVolUUID=78b1ad35-4f19-41c3-8ab3-a293eb3c54c6
                importLocation=HttpLocationInfo [url=http://glance.ovirt.org:9292/v1/images/865bfac3-f9cd-40fc-8d66-293639a9918f, headers={}]


Verified using upstream ovirt-3.5 RC1.1

Comment 4 Allon Mureinik 2015-02-16 19:09:24 UTC
RHEV-M 3.5.0 has been released, closing this bug.