Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1512572 - vdsm Failed to connect to guest agent channel
Summary: vdsm Failed to connect to guest agent channel
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: 4.20.4
Hardware: x86_64
OS: Linux
unspecified
urgent vote
Target Milestone: ovirt-4.2.0
: ---
Assignee: Milan Zamazal
QA Contact: Michael Burman
URL:
Whiteboard:
: 1507939 1513001 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-13 14:52 UTC by Michael Burman
Modified: 2018-04-04 10:46 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-20 10:43:37 UTC
oVirt Team: Virt
rule-engine: ovirt-4.2+
rule-engine: blocker+


Attachments (Terms of Use)
vdsm logs (deleted)
2017-11-14 06:02 UTC, Michael Burman
no flags Details


Links
System ID Priority Status Summary Last Updated
oVirt gerrit 84002 master MERGED virt: Set correct agent channel name when not present in conf 2017-11-15 09:38:12 UTC

Description Michael Burman 2017-11-13 14:52:21 UTC
Description of problem:
vdsm Failed to connect to guest agent channel

2017-11-13 16:19:04,634+0200 INFO  (vm/960acdbb) [virt.vm] (vmId='960acdbb-69f1-4071-b10b-410b47cd6a84') Starting connection (guestagent:244)
2017-11-13 16:19:04,635+0200 ERROR (vm/960acdbb) [virt.vm] (vmId='960acdbb-69f1-4071-b10b-410b47cd6a84') Failed to connect to guest agent channel (vm:2364)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2362, in _vmDependentInit
    self.guestAgent.start()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 245, in start
    self._prepare_socket()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 287, in _prepare_socket
    supervdsm.getProxy().prepareVmChannel(self._socketName)
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda>
    **kwargs)
  File "<string>", line 2, in prepareVmChannel
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
OSError: [Errno 2] No such file or directory: '/var/lib/libvirt/qemu/channels/960acdbb-69f1-4071-b10b-410b47cd6a84.com.redhat.rhevm.vdsm'


MainProcess|vm/960acdbb::DEBUG::2017-11-13 16:19:04,635::supervdsm_server::90::SuperVdsm.ServerCallback::(wrapper) call prepareVmChannel with () {}
MainProcess|vm/960acdbb::ERROR::2017-11-13 16:19:04,635::supervdsm_server::94::SuperVdsm.ServerCallback::(wrapper) Error in prepareVmChannel
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 92, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/virt.py", line 35, in prepareVmChannel
    fsinfo = os.stat(socketFile)
OSError: [Errno 2] No such file or directory: '/var/lib/libvirt/qemu/channels/960acdbb-69f1-4071-b10b-410b47cd6a84.com.redhat.rhevm.vdsm'


Version-Release number of selected component (if applicable):
vdsm-4.20.6-62.gitd3023e4.el7.centos.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Run VM on latest vdsm 4.2, the guest should have a guest-agent installed and running

Actual results:
vdsm failed to connect to guest agent channel

Expected results:

Should work as expected

Comment 1 Michal Skrivanek 2017-11-14 05:48:48 UTC
logs?

Comment 2 Michael Burman 2017-11-14 06:01:49 UTC
(In reply to Michal Skrivanek from comment #1)
> logs?

I already spoke with Milan and he is aware of this report, also think that he already has a possible fix for it.
Attaching vdsm logs, let me know if more are needed.

Comment 3 Michael Burman 2017-11-14 06:02:56 UTC
Created attachment 1351789 [details]
vdsm logs

Comment 4 Michal Skrivanek 2017-11-15 06:38:29 UTC
*** Bug 1513001 has been marked as a duplicate of this bug. ***

Comment 5 Michael Burman 2017-11-27 08:12:15 UTC
Verified on - vdsm-4.20.8-29.git6ceb389.el7.centos.x86_64

Comment 6 Sandro Bonazzola 2017-12-20 10:43:37 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 7 Daniel Gur 2017-12-25 09:52:31 UTC
*** Bug 1507939 has been marked as a duplicate of this bug. ***

Comment 8 Yaniv Kaul 2018-04-03 07:59:15 UTC
I'm still seeing this all the time. Specifically, with hosted-engine.
See http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/276/ - log @ http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/276/artifact/exported-artifacts/test_logs/hc-basic-suite-master/post-002_bootstrap.py/lago-hc-basic-suite-master-host-0/_var_log/vdsm/vdsm.log :

2018-04-02 09:58:05,485-0400 INFO  (vm/96a15386) [virt.vm] (vmId='96a15386-2397-45fd-b739-e7b96c8f31a5') Starting connection (guestagent:250)
2018-04-02 09:58:05,486-0400 ERROR (vm/96a15386) [virt.vm] (vmId='96a15386-2397-45fd-b739-e7b96c8f31a5') Failed to connect to guest agent channel (vm:2462)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2460, in _vmDependentInit
    self.guestAgent.start()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 251, in start
    self._prepare_socket()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 293, in _prepare_socket
    supervdsm.getProxy().prepareVmChannel(self._socketName)
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 55, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 53, in <lambda>
    **kwargs)
  File "<string>", line 2, in prepareVmChannel
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
OSError: [Errno 2] No such file or directory: '/var/lib/libvirt/qemu/channels/96a15386-2397-45fd-b739-e7b96c8f31a5.com.redhat.rhevm.vdsm'
2018-04-02 09:58:05,488-0400 INFO  (vm/96a15386) [virt.vm] (vmId='96a15386-2397-45fd-b739-e7b96c8f31a5') CPU running: domain initialization (vm:6065)

Comment 9 Milan Zamazal 2018-04-04 10:46:33 UTC
There is no ovirt-guest-agent channel in HostedEngineLocal domain XML. So when the VM is recovered on Vdsm start, the fallback backwards compatible channel path is used as the socket name. The socket doesn't exist (there is no ovirt-guest-agent around) and the error above is raised on attempt to open it.


Note You need to log in before you can comment on or make changes to this bug.