Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1120049 - RHEV-H: vdsm not starting due to connectivity.log root ownership
Summary: RHEV-H: vdsm not starting due to connectivity.log root ownership
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.4.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.5.0
Assignee: Douglas Schilling Landgraf
QA Contact: Martin Pavlik
URL:
Whiteboard: network
Depends On:
Blocks: rhev3.5beta 1156165
TreeView+ depends on / blocked
 
Reported: 2014-07-16 07:32 UTC by cshao
Modified: 2016-02-10 19:53 UTC (History)
23 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-16 13:39:59 UTC
oVirt Team: Network
Target Upstream Version:


Attachments (Terms of Use)
vdsm-reg.log (deleted)
2014-07-16 07:32 UTC, cshao
no flags Details
engine.log (deleted)
2014-07-16 07:33 UTC, cshao
no flags Details
supervdsm.log (deleted)
2014-07-17 12:58 UTC, Ying Cui
no flags Details


Links
System ID Priority Status Summary Last Updated
oVirt gerrit 30507 master MERGED connectivity log: add to spec with proper ownership Never
oVirt gerrit 30558 ovirt-3.5 MERGED connectivity log: add to spec with proper ownership Never

Description cshao 2014-07-16 07:32:14 UTC
Created attachment 918336 [details]
vdsm-reg.log

Description of problem:
Register RHEV-H to RHEV-M av10.1 & is37 failed.
Network error during communication with the host.

======================================================
libvir: Network Driver error : Network not found: no network with matching name 'vdsm-rhevm'
libvir: Network Driver error : Network not found: no network with matching name 'vdsm-rhevm'

Version-Release number of selected component (if applicable):
rhev-hypervisor6-6.5-20140715.0
ovirt-node-3.0.1-18.el6_5.13.noarch
vdsm-4.14.7-7.el6ev.x86_64
vdsm-reg-4.14.7-7.el6ev.noarch
RHEVM av10.1
RHEVM is37

How reproducible:
100%

Steps to Reproduce:
1. Install rhev-hypervisor6-6.5-20140715.0
2. Register to RHEVM.
3.

Actual results:
Register RHEV-H to RHEV-M av10.1& is37 failed. Network error during communication with the host.

Expected results:
Register to RHEVM can successful.

Additional info:
Add RHEV-H from RHEV-M side can successful.

Comment 1 cshao 2014-07-16 07:33:23 UTC
Created attachment 918337 [details]
engine.log

Comment 2 cshao 2014-07-16 07:35:57 UTC
1. No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build, so add keyword regression.
2. This bug block our testing, so add keyword testblodker.

Comment 6 Lior Vernia 2014-07-17 12:28:57 UTC
Could we get relevant VDSM log as well?

Comment 7 Ying Cui 2014-07-17 12:29:51 UTC
(In reply to shaochen from comment #2)
> 1. No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build, so add
> keyword regression.
> 2. This bug block our testing, so add keyword testblodker.

No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build with vdsm-4.14.7-3.el6ev.x86_64

Comment 8 Ying Cui 2014-07-17 12:57:07 UTC
register failed, vdsmd service is not running, there is no vdsm.log, provide supervdsm.log in this bug.

Comment 9 Ying Cui 2014-07-17 12:58:23 UTC
Created attachment 918698 [details]
supervdsm.log

Comment 10 Dan Kenigsberg 2014-07-17 14:22:04 UTC
Vdsm refuses to start reminds me of recent cpopen and python-pthreading bugs. Which versions of them do you have installed?

Comment 11 Ying Cui 2014-07-18 02:58:11 UTC
(In reply to Dan Kenigsberg from comment #10)
> Vdsm refuses to start reminds me of recent cpopen and python-pthreading
> bugs. Which versions of them do you have installed?

python-cpopen-1.3-2.el6_5.x86_64
python-pthreading-0.1.3-3.el6ev.noarch

Comment 12 Dan Kenigsberg 2014-07-20 20:29:52 UTC
Ying Cui, does vdsm start up when you try to start it explicitly? If not, could you log into the node as root, and run

# su - vdsm -s /bin/bash
$ /usr/share/vdsm/vdsm

this may provide more hints regarding the failure.

Comment 13 Ying Cui 2014-07-21 02:57:28 UTC
Try to start vdsmd, but it is still not running.
# /etc/init.d/vdsmd start 
vdsm: already running                                      [  OK  ]
vdsm start                                                 [  OK  ]
# /etc/init.d/vdsmd status
VDS daemon is not running, and its watchdog is running

# su - vdsm -s /bin/bash
$ /usr/share/vdsm/vdsm
Traceback (most recent call last):
  File "/usr/share/vdsm/vdsm", line 218, in <module>
    run(**argDict)
  File "/usr/share/vdsm/vdsm", line 92, in run
    lconfig.fileConfig(loggerConfFile, disable_existing_loggers=False)
  File "/usr/lib64/python2.6/logging/config.py", line 84, in fileConfig
  File "/usr/lib64/python2.6/logging/config.py", line 162, in _install_handlers
  File "/usr/lib64/python2.6/logging/handlers.py", line 377, in __init__
  File "/usr/lib64/python2.6/logging/__init__.py", line 827, in __init__
  File "/usr/lib64/python2.6/logging/__init__.py", line 846, in _open
IOError: [Errno 13] Permission denied: '/var/log/vdsm/connectivity.log'

$ ll /var/log/vdsm/connectivity.log 
-rw-r--r--. 1 root root 0 2014-07-16 06:31 /var/log/vdsm/connectivity.log

Comment 14 Ying Cui 2014-07-21 08:27:37 UTC
Virt QE helps to qa_ack+ this bug.

Comment 15 Eyal Edri 2014-07-22 05:41:02 UTC
danken - can we move this to MODIFIED since we reverted the change?

2 commits were reverted:
7a743d1 - connectivty log: more readable changes
5f0f1e6 - introduce connectivity 

or open another bug to verify the new fix?

Comment 16 Dan Kenigsberg 2014-07-22 07:38:03 UTC
We should fix this properly on 3.5 (with the linked patch), so I'm changing the released version instead.

Comment 17 Douglas Schilling Landgraf 2014-07-23 14:06:50 UTC
Moving to MODIFIED since next rebase should include this fix.

Comment 19 cshao 2014-07-24 05:55:44 UTC
Test version:
rhev-hypervisor6-6.5-20140723.0.el6ev
vdsm-4.14.11-5.el6ev.x86_64
ovirt-node-3.0.1-18.el6.14.noarch
RHEVM av10.5

Test version:
1. Install rhev-hypervisor6-6.5-20140715.0
2. Register to RHEVM.
3. Add RHEV-H from RHEV-M side.
4. ll /var/log/vdsm/connectivity.log 

Test result:
1. Both "register" and "add host from rhevm side" can successful.
2. But there is no connectivity.log under /var/log/vdsm path.
# ll /var/log/vdsm
total 7284
drwxr-xr-x. 2 vdsm kvm     4096 2014-07-24 02:32 backup
-rw-r--r--. 1 vdsm kvm        0 2014-07-24 02:56 metadata.log
-rw-r--r--. 1 vdsm kvm     1395 2014-07-24 05:50 mom.log
-rw-r--r--. 1 vdsm kvm     2184 2014-07-24 02:58 spm-lock.log
-rw-r--r--. 1 root root   18248 2014-07-24 05:50 supervdsm.log
-rw-r--r--. 1 vdsm kvm  7420427 2014-07-24 05:52 vdsm.log

Comment 20 Ying Cui 2014-07-24 07:09:20 UTC
This bug need to clone to rhev 3.4.1, then we will verify this bug on 3.4.1 build.

Comment 21 Martin Pavlik 2014-08-07 13:58:31 UTC
works with 
oVirt Node Hypervisor release 3.1.0_master (20140805.0) (Edited, Unsigned)

[root@dell-r210ii-06 ~]# ll /var/log/vdsm/connectivity.log 
-rw-r--r--. 1 vdsm kvm 472 2014-08-07 13:55 /var/log/vdsm/connectivity.log


Note You need to log in before you can comment on or make changes to this bug.