Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 825992 - Guest stuck at "Wait For Launch"
Summary: Guest stuck at "Wait For Launch"
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Dan Kenigsberg
QA Contact: Haim
URL:
Whiteboard: virt
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-05-29 09:37 UTC by Chao Yang
Modified: 2014-01-13 00:52 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-06-07 09:48:39 UTC
Target Upstream Version:


Attachments (Terms of Use)
libvirt and vdsm log (deleted)
2012-05-29 09:37 UTC, Chao Yang
no flags Details

Description Chao Yang 2012-05-29 09:37:34 UTC
Created attachment 587369 [details]
libvirt and vdsm log

Description of problem:
I created two VMs from existing templates, then click on start, but they get stuck at "Wait for Launch". Threre is io error as well as libvirt error in vdsm.log:
IOError: [Errno 2] No such file or directory: '/dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06' but I checked, this file exists:
# ls -l /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10


Version-Release number of selected component (if applicable):
qemu-kvm-rhev-0.12.1.2-2.294.el6.x86_64
libvirt-0.9.10-20.el6.x86_64
vdsm-4.9.6-13.el6.x86_64


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Haim 2012-05-29 11:31:11 UTC
it appears that vdsm fails to communicate with libvirt; what is the status of the service ? does virhs -r capabilities works ? does libvirt service restart works for you ?

Comment 3 Chao Yang 2012-05-29 11:51:31 UTC
(In reply to comment #2)
> it appears that vdsm fails to communicate with libvirt; what is the status
> of the service ? does virhs -r capabilities works ? does libvirt service
> restart works for you ?

vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt service works, the guests in "Wait For Launch" status previously are now running.

Comment 4 Haim 2012-05-29 12:58:10 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > it appears that vdsm fails to communicate with libvirt; what is the status
> > of the service ? does virhs -r capabilities works ? does libvirt service
> > restart works for you ?
> 
> vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt
> service works, the guests in "Wait For Launch" status previously are now
> running.

we need to understand what caused vdsm to loose connection to libvirt service, do you have a reproducer ?

Comment 5 Dan Kenigsberg 2012-05-29 21:32:25 UTC
What's the version of libvirt? I recall that there was a recenet rpc bug in libvirt that blocked SI4 build.

Comment 6 Chao Yang 2012-05-30 05:21:49 UTC
(In reply to comment #5)
> What's the version of libvirt? I recall that there was a recenet rpc bug in
> libvirt that blocked SI4 build.

libvirt-0.9.10-20.el6.x86_64

Comment 7 Chao Yang 2012-05-30 05:38:35 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > it appears that vdsm fails to communicate with libvirt; what is the status
> > > of the service ? does virhs -r capabilities works ? does libvirt service
> > > restart works for you ?
> > 
> > vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt
> > service works, the guests in "Wait For Launch" status previously are now
> > running.
> 
> we need to understand what caused vdsm to loose connection to libvirt
> service, do you have a reproducer ?

Not sure if it is a reliable reproducer, I just started VM from a template, then it happened.

Comment 8 Itamar Heim 2012-05-30 11:51:06 UTC
> lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682
> /2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10"

permissions here are root/root. iirc they should be vdsm.
how did you get to this?
though it probably should have failed runnning the guest, rather than hang it.

Comment 10 Chao Yang 2012-06-01 09:07:12 UTC
(In reply to comment #8)
> > lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682
> > /2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10"
> 
> permissions here are root/root. iirc they should be vdsm.
> how did you get to this?
> though it probably should have failed runnning the guest, rather than hang
> it.

I don't know, I just created an iscsi storage from WPFClient, then New VMs on this storage.  ls tells this file is owned by root but getfact tells vdsm. 

# ls -l /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10

# getfacl /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
getfacl: Removing leading '/' from absolute path names
# file: dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
# owner: vdsm
# group: qemu
user::rw-
group::rw-
other::---

Comment 11 Dan Kenigsberg 2012-06-03 15:06:20 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > What's the version of libvirt? I recall that there was a recenet rpc bug in
> > libvirt that blocked SI4 build.
> 
> libvirt-0.9.10-20.el6.x86_64

please try -21, I suspect a dup of bug 821468

Comment 12 Chao Yang 2012-06-04 01:55:40 UTC
(In reply to comment #11)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > What's the version of libvirt? I recall that there was a recenet rpc bug in
> > > libvirt that blocked SI4 build.
> > 
> > libvirt-0.9.10-20.el6.x86_64
> 
> please try -21, I suspect a dup of bug 821468

I have already upgraded to -21. If still reproducible, will update here.

Comment 13 Itamar Heim 2012-06-07 09:48:39 UTC
chayang - please re-open if reproduced with -21 libvirt


Note You need to log in before you can comment on or make changes to this bug.