Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1124762 - Unexpected error while setting up hosted-engine on EL7
Summary: Unexpected error while setting up hosted-engine on EL7
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 3.5.0
Assignee: Allon Mureinik
QA Contact: Artyom
URL:
Whiteboard: storage
Depends On:
Blocks: rhev35betablocker rhev3.5beta3 rhev35rcblocker rhev35gablocker
TreeView+ depends on / blocked
 
Reported: 2014-07-30 09:18 UTC by Sandro Bonazzola
Modified: 2016-02-10 17:15 UTC (History)
18 users (show)

Fixed In Version: vt4.1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-16 13:39:40 UTC
oVirt Team: Storage
Target Upstream Version:


Attachments (Terms of Use)
logs (deleted)
2014-07-30 09:18 UTC, Sandro Bonazzola
no flags Details
vdsm.log (deleted)
2014-10-05 14:37 UTC, Artyom
no flags Details

Description Sandro Bonazzola 2014-07-30 09:18:29 UTC
Created attachment 922471 [details]
logs

Trying to test hosted-engine on EL7, I got the following Traceback in VDSM:


 Thread-52::ERROR::2014-07-30 11:09:28,688::task::866::Storage.TaskManager.Task::(_setError) Task=`a041a63b-ef51-4352-98de-c5169bf8dcf1`::Unexpected error
 Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 3158, in getVolumePath
    volUUID=volUUID).getVolumePath()
  File "/usr/share/vdsm/storage/sd.py", line 400, in produceVolume
    volUUID)
  File "/usr/share/vdsm/storage/fileVolume.py", line 58, in __init__
    volume.Volume.__init__(self, repoPath, sdUUID, imgUUID, volUUID)
  File "/usr/share/vdsm/storage/volume.py", line 144, in __init__
    self.validate()
  File "/usr/share/vdsm/storage/volume.py", line 156, in validate
    self.validateImagePath()
  File "/usr/share/vdsm/storage/fileVolume.py", line 487, in validateImagePath
    if not self.oop.os.path.isdir(imageDir):
  File "/usr/share/vdsm/storage/outOfProcess.py", line 251, in isdir
    res = self._iop.stat(path)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 367, in stat
    resdict = self._sendCommand("stat", {"path": path}, self.timeout)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 344, in _sendCommand
    raise OSError(errcode, errstr)
 OSError: [Errno 2] No such file or directory

Storage is NFSv3 on a different host.

Comment 3 Allon Mureinik 2014-09-02 07:29:48 UTC
Saggie, do we have an ioprocess build available on EL7?

Comment 5 Barak 2014-09-21 15:20:33 UTC
There is an available package for ioprocess for EL7,
This should be handled by storage group , moving back to storage.

BTW the initial error is "No such file or directory" ...

Comment 6 Yoav Kleinberger 2014-09-28 11:32:53 UTC
Is this an engine running on a RHEL7 VM, or an engine running on any VM operating system, but the *host* is running RHEL7?

I'm pretty sure you meant the host, but I want to be sure.

Comment 7 Artyom 2014-09-28 11:45:07 UTC
Reporter is Sandro so the best way it ask him, but how I understand host is EL7.
Also hard to understand when it happen, on what stage, on deployment of hosted engine or when hosted-engine deployment already finished, Sandro can you give a little more details about the flow, when exception appear, what actions preceded.

Comment 8 Sandro Bonazzola 2014-09-29 05:59:09 UTC
This is ovirt-hosted-engine-setup running on EL7 for creating a VM where it would have been installed EL 6.5 + RHEV-M 3.5 (but it failed before creating the VM)

It failed at Volume creation stage, after having created the storage domain:

2014-07-30 11:09:28 DEBUG otopi.context context._executeMethod:152 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 142, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/sanlock/lockspace.py", line 163, in _misc
    lockspace + '.metadata': md_size,
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 336, in create
    service_size=size)
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 268, in create_volume
    raise RuntimeError(response["status"]["message"])
RuntimeError: [Errno 2] No such file or directory

There are full logs of the setup execution and corresponding vdsm logs in the attached archive.

Comment 9 Sandro Bonazzola 2014-10-02 12:32:21 UTC
I'm not able to reproduce the issue using upstream rpms,maybe we should move this to QE as test-only so it can be checked on next downstream build?

Comment 10 Allon Mureinik 2014-10-02 12:59:26 UTC
(In reply to Sandro Bonazzola from comment #9)
> I'm not able to reproduce the issue using upstream rpms,maybe we should move
> this to QE as test-only so it can be checked on next downstream build?
Agreed.

Comment 11 Artyom 2014-10-05 14:35:51 UTC
I finish deployment for vt4 and not encounter this kind of exception, but all vdsm logs full of:
Thread-20::ERROR::2014-10-05 17:23:10,961::sampling::475::vm.Vm::(collect) vmId=`f4675766-a472-4c49-bde7-ad68db30e77c`::Stats function failed: <AdvancedStatsFunction _sampleCpu at 0x2adf1b0>
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/sampling.py", line 471, in collect
    statsFunction()
  File "/usr/share/vdsm/virt/sampling.py", line 346, in __call__
    retValue = self._function(*args, **kwargs)
  File "/usr/share/vdsm/virt/vm.py", line 313, in _sampleCpu
    cpuStats = self._vm._dom.getCPUStats(True, 0)
  File "/usr/share/vdsm/virt/vm.py", line 670, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 111, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2016, in getCPUStats
    if ret is None: raise libvirtError ('virDomainGetCPUStats() failed', dom=self)
libvirtError: Requested operation is not valid: cgroup CPUACCT controller is not mounted
Thread-20::DEBUG::2014-10-05 17:23:10,964::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 55 edom: 10 level: 2 message: Requested operation is not valid: cgroup CPU controller is not mounted

So I can reassign this bug or just open one new, what you prefer?

Comment 12 Artyom 2014-10-05 14:37:20 UTC
Created attachment 944045 [details]
vdsm.log

Comment 13 Allon Mureinik 2014-10-05 15:29:37 UTC
(In reply to Artyom from comment #11)
> So I can reassign this bug or just open one new, what you prefer?
The two issues seem unrelated - please open a new BZ.

Comment 14 Artyom 2014-10-06 07:54:21 UTC
Verified on vdsm-4.16.5-2.el7.x86_64 and ovirt-hosted-engine-setup-1.2.0-1.el7.noarch


Note You need to log in before you can comment on or make changes to this bug.