Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1364126 - disableNetworkManager=bool:True on ovirt-node, too.
Summary: disableNetworkManager=bool:True on ovirt-node, too.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: redhat-release-rhev-hypervisor
Version: 4.0.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ovirt-4.0.2
: ---
Assignee: Fabian Deutsch
QA Contact: Huijuan Zhao
URL:
Whiteboard:
: 1370225 (view as bug list)
Depends On:
Blocks: 1304509 1361017
TreeView+ depends on / blocked
 
Reported: 2016-08-04 14:06 UTC by Dan Kenigsberg
Modified: 2016-08-30 07:56 UTC (History)
14 users (show)

Fixed In Version: redhat-release-virtualization-host-4.0-1.el7
Doc Type: Known Issue
Doc Text:
If a user configures Red Hat Virtualization Host (RHVH) networking using Cockpit and leaves the Cockpit browser tab open then adds RHVH to the Red Hat Virtualization Manager the DNS configuration of the host is reset. The user must ensure that they log out and close the Cockpit browser tab before adding RHVH to the Red Hat Virtualization Manager. This means that the DNS configuration will be configured as it was previously through RHVH.
Clone Of:
Environment:
Last Closed: 2016-08-23 21:13:16 UTC
oVirt Team: Node


Attachments (Terms of Use)
sosreport from alma03 host that was added to my environment (deleted)
2016-08-09 15:52 UTC, Nikolai Sednev
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1688 normal SHIPPED_LIVE redhat-release-virtualization-host bug fix and enhancement update for RHV 4.0 2016-08-24 00:37:04 UTC

Description Dan Kenigsberg 2016-08-04 14:06:20 UTC
Description of problem:
Unlike in node-less installation, ovit-node-ng sets disableNetworkManager=bool:False

Keeping NM running when a host is added to ovirt-engine proved to be hard: see bug 1356635, bug 1160423, bug 1361017

I suggest to turn it off because I'm not sure that we'd solve bug 1351095 on time on vdsm side, and I am worried about finding another glitch after 4.0-ga release.

The downside is that disableNetworkManager=bool:True was not tested recently on NGN. I would like to ask QE to test all add host flows (hosted engine included) after setting VDSM/disableNetworkManager=bool:True in /etc/ovirt-host-deploy.conf.d/90-ngn-keep-networkmanager.conf

Comment 1 Ying Cui 2016-08-08 08:55:43 UTC
Huijuan and Chen, could you test this bug for adding host flows(hosted engine included) on RC RHVH after setting VDSM/disableNetworkManager=bool:True in /etc/ovirt-host-deploy.conf.d/90-ngn-keep-networkmanager.conf?

Comment 2 Huijuan Zhao 2016-08-08 11:26:28 UTC
Tested this scenario on redhat-virtualization-host-4.0-20160803.3.x86_64.liveimg.squashfs, can add rhvh to rhvm successful.

Test version:
redhat-virtualization-host-4.0-20160803.3.x86_64.liveimg.squashfs
imgbased-0.7.4-0.1.el7ev.noarch
vdsm-4.18.10-1.el7ev.x86_64
NetworkManager-1.0.6-30.el7_2.x86_64
Red Hat Virtualization Manager Version: 4.0.2.4-0.1.el7ev

Test steps:
1. Install redhat-virtualization-host-4.0-20160803.3
2. Reboot and login RHVH, setting VDSM/disableNetworkManager=bool:True in /etc/ovirt-host-deploy.conf.d/90-ngn-keep-networkmanager.conf
3. Reboot RHVH
4. Add RHVH to RHVM

Test results:
After step4, add RHVH to RHVM successful

Comment 5 cshao 2016-08-09 06:21:48 UTC
(In reply to Ying Cui from comment #1)
> Huijuan and Chen, could you test this bug for adding host flows(hosted
> engine included) on RC RHVH after setting
> VDSM/disableNetworkManager=bool:True in
> /etc/ovirt-host-deploy.conf.d/90-ngn-keep-networkmanager.conf?

Test version:
redhat-virtualization-host-4.0-20160803.3.
imgbased-0.7.4-0.1.el7ev.noarch
vdsm-4.18.10-1.el7ev.x86_64
NetworkManager-1.0.6-30.el7_2.x86_64
cockpit-ovirt-dashboard-0.10.6-1.3.4.el7ev.noarch
cockpit-0.114-2.el7.x86_64
ovirt-hosted-engine-ha-2.0.1-1.el7ev.noarch
ovirt-hosted-engine-setup-2.0.1.3-1.el7ev.noarch
20160731.0-1.el7ev.4.0.ova


Test steps:
1. Install redhat-virtualization-host-4.0-20160803.3
2. Reboot and login RHVH, setting VDSM/disableNetworkManager=bool:True in /etc/ovirt-host-deploy.conf.d/90-ngn-keep-networkmanager.conf
3. Reboot RHVH
4. Login RHVH via cockpit UI.
5. Deploy Hosted Engine via cockpit with correct steps.
6. After vm shut down, wait a few minutes, check HE status.

Test results:
Met bug 1364034.
Hosted Engine always show "Not running" status after deploy it.

Comment 6 Dan Kenigsberg 2016-08-09 09:31:07 UTC
cshao, can you circumvent bug 1364034 by pre-installing vdsm on all cluster hosts, and retry?

Comment 7 Michael Burman 2016-08-09 09:54:22 UTC
Add host tested over dhcp and static ip with rhvh-4.0-0.20160803.0+1 

vdsm-4.18.10-1.el7ev.x86_64
rhevm-4.0.2.4-0.1.el7ev.noarch

1) Install rhvh-4.0-0.20160803.0+1(via foreman)
2) Configure static ip and dns via cockpit
3) Set VDSM/disableNetworkManager=bool:True 
4) Deploy/add host to latest rhv-m with success

- name server still present in resolv.conf..)

Comment 8 Michael Burman 2016-08-09 09:57:10 UTC
(In reply to Michael Burman from comment #7)
> Add host tested over dhcp and static ip with rhvh-4.0-0.20160803.0+1 
> 
> vdsm-4.18.10-1.el7ev.x86_64
> rhevm-4.0.2.4-0.1.el7ev.noarch
> 
> 1) Install rhvh-4.0-0.20160803.0+1(via foreman)
> 2) Configure static ip and dns via cockpit
> 3) Set VDSM/disableNetworkManager=bool:True 
> 4) Deploy/add host to latest rhv-m with success
> 
> - name server still present in resolv.conf..)

Small correction, name server removed from resolv.conf..

Comment 9 Michael Burman 2016-08-09 11:47:20 UTC
resolv.conf was trashed because NM was restarted immediately by something. Fabian, can you tell what that something may be?

2016-08-09 14:14:24 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/bin/systemctl', 'stop', 'NetworkManager.service') stderr:


Aug  9 14:14:24 camel-vdsa systemd: Stopped Network Manager.
Aug  9 14:14:24 camel-vdsa dbus-daemon: dbus[905]: [system] Activating via systemd: service name='org.freedesktop.NetworkManager' unit='dbus-org.freedesktop.NetworkManager.service'
Aug  9 14:14:24 camel-vdsa dbus[905]: [system] Activating via systemd: service name='org.freedesktop.NetworkManager' unit='dbus-org.freedesktop.NetworkManager.service'
Aug  9 14:14:24 camel-vdsa systemd: Starting Network Manager...
Aug  9 14:14:24 camel-vdsa systemd: Reloading.

Comment 10 Fabian Deutsch 2016-08-09 11:52:53 UTC
The only candidate that comes to my mind right now is cockpit.

If cockpit was open during testing, then it could be that the logic which is updating the charts is calling NM for metrics.

So, was cockpit running? Yes (from irc).

You might want to try to close cockpit before adding the host

Comment 11 Michael Burman 2016-08-09 12:02:41 UTC
(In reply to Michael Burman from comment #9)
> resolv.conf was trashed because NM was restarted immediately by something.
> Fabian, can you tell what that something may be?
> 
> 2016-08-09 14:14:24 DEBUG otopi.plugins.otopi.services.systemd
> plugin.execute:926 execute-output: ('/bin/systemctl', 'stop',
> 'NetworkManager.service') stderr:
> 
> 
> Aug  9 14:14:24 camel-vdsa systemd: Stopped Network Manager.
> Aug  9 14:14:24 camel-vdsa dbus-daemon: dbus[905]: [system] Activating via
> systemd: service name='org.freedesktop.NetworkManager'
> unit='dbus-org.freedesktop.NetworkManager.service'
> Aug  9 14:14:24 camel-vdsa dbus[905]: [system] Activating via systemd:
> service name='org.freedesktop.NetworkManager'
> unit='dbus-org.freedesktop.NetworkManager.service'
> Aug  9 14:14:24 camel-vdsa systemd: Starting Network Manager...
> Aug  9 14:14:24 camel-vdsa systemd: Reloading.

To work around it, we need to log out from cockpit session before adding the host(as it seems like he is the one who starting the NM again).

Comment 12 cshao 2016-08-09 13:40:42 UTC
(In reply to Dan Kenigsberg from comment #6)
> cshao, can you circumvent bug 1364034 by pre-installing vdsm on all cluster
> hosts, and retry?

Hi Danken,

I am little confused. do you mean pre-installing vdsm via ks? As I know, RHVH imgbase is read only during pre-installing, seems I can't pre-installing vdsm via ks. 

Could you point me how to circumvent bug 1364034?

Thanks.

Comment 16 Nikolai Sednev 2016-08-09 15:52:16 UTC
Created attachment 1189339 [details]
sosreport from alma03 host that was added to my environment

Comment 17 Dan Kenigsberg 2016-08-09 16:02:24 UTC
(In reply to shaochen from comment #12)

> Could you point me how to circumvent bug 1364034?

I now understand that you cannot; let us retry with a new image.

Comment 19 Fabian Deutsch 2016-08-09 22:08:44 UTC
Please retry with redhat-virtualization-host-image-update-4.0-20160810.0

Comment 21 Fabian Deutsch 2016-08-11 05:55:49 UTC
It should be available once Anton upates the repo.

But: We need another fix which is not in that build yet.

Comment 22 Fabian Deutsch 2016-08-11 16:15:41 UTC
Fix in redhat-release-virtualization-host-4.0-1.el7.

Implementation:
Systemd is watching /etc/pki/vdsm/certs - if a cert appears, a unit will be started which will stop and mask NetworkManager.

unmasking needs to be don manually

Comment 23 cshao 2016-08-12 06:43:37 UTC
Hosted Engine test result:

Test version:
redhat-virtualization-host-4.0-20160811.0
imgbased-0.8.4-1.el7ev.noarch
redhat-release-virtualization-host-4.0-1.el7.x86_64
vdsm-4.18.11-1.el7ev.x86_64
ovirt-hosted-engine-ha-2.0.2-1.el7ev.noarch
ovirt-hosted-engine-setup-2.0.1.4-1.el7ev.noarch
rhevm-appliance-20160731.0-1.el7ev.ova


network info:
# cat /etc/ovirt-host-deploy.conf.d/90-ngn-do-not-keep-networkmanager.conf 
[environment:init]
VDSM/disableNetworkManager=bool:True
[root@cshao210 ~]# 
[root@cshao210 ~]# systemctl status NetworkManager
● NetworkManager.service
   Loaded: masked (/dev/null)
   Active: inactive (dead)


Test steps:
1. Install redhat-virtualization-host-4.0-20160811.0
2. Login RHVH via cockpit UI.
3. Deploy Hosted Engine via cockpit with correct steps.
4. After vm shut down, wait a few minutes, check HE status.
5. Reboot Host, check HE status.

Test results:
HE can work well after step 4 & 5.

Comment 24 Huijuan Zhao 2016-08-12 09:10:55 UTC
Add RHVH to RHVM test result:

Test version:
redhat-virtualization-host-4.0-20160811.0
imgbased-0.8.4-1.el7ev.noarch
redhat-release-virtualization-host-4.0-1.el7.x86_64
vdsm-4.18.11-1.el7ev.x86_64
ovirt-hosted-engine-ha-2.0.2-1.el7ev.noarch
ovirt-hosted-engine-setup-2.0.1.4-1.el7ev.noarch
rhevm-appliance-20160731.0-1.el7ev.ova



Test steps:
1. Install redhat-virtualization-host-4.0-20160811
2. Reboot and login RHVH
3. check network info:
# cat /etc/ovirt-host-deploy.conf.d/90-ngn-do-not-keep-networkmanager.conf 
[environment:init]
VDSM/disableNetworkManager=bool:True

# systemctl status NetworkManager
● NetworkManager.service
   Loaded: masked (/dev/null)
   Active: inactive (dead)
4. Add RHVH to RHVM
5. Add storage and setup VM

Test results:
After step4, add RHVH to RHVM successful
After step5, setup VM successful

Comment 25 Huijuan Zhao 2016-08-12 09:20:34 UTC
According to Comment 23 and Comment 24, this bug is fixed in redhat-virtualization-host-4.0-20160811.0, change the status to VERIFIED.

Comment 26 Huijuan Zhao 2016-08-12 10:15:41 UTC
(In reply to Huijuan Zhao from comment #24)
> Add RHVH to RHVM test result:
> 
> Test version:
> redhat-virtualization-host-4.0-20160811.0
> imgbased-0.8.4-1.el7ev.noarch
> redhat-release-virtualization-host-4.0-1.el7.x86_64
> vdsm-4.18.11-1.el7ev.x86_64
> ovirt-hosted-engine-ha-2.0.2-1.el7ev.noarch
> ovirt-hosted-engine-setup-2.0.1.4-1.el7ev.noarch
> rhevm-appliance-20160731.0-1.el7ev.ova
> 
> 
> 
> Test steps:
> 1. Install redhat-virtualization-host-4.0-20160811
> 2. Reboot and login RHVH
> 3. check network info:
> # cat /etc/ovirt-host-deploy.conf.d/90-ngn-do-not-keep-networkmanager.conf 
> [environment:init]
> VDSM/disableNetworkManager=bool:True
> 
> # systemctl status NetworkManager
> ● NetworkManager.service
>    Loaded: masked (/dev/null)
>    Active: inactive (dead)
> 4. Add RHVH to RHVM
> 5. Add storage and setup VM
> 
> Test results:
> After step4, add RHVH to RHVM successful
> After step5, setup VM successful

Update step3 and step4:
3. check network info, NetworkManager is active:
# cat /etc/ovirt-host-deploy.conf.d/90-ngn-do-not-keep-networkmanager.conf 
[environment:init]
VDSM/disableNetworkManager=bool:True
4. Add RHVH to RHVM, NetworkManager is inactive.

Comment 28 errata-xmlrpc 2016-08-23 21:13:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1688.html

Comment 29 Dan Kenigsberg 2016-08-30 07:56:56 UTC
*** Bug 1370225 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.