Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1686250 - Ovirt-host deploy completes with error "Failed to configure management network on the host"and host status set to NonOperational.
Summary: Ovirt-host deploy completes with error "Failed to configure management networ...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Host-Deploy
Version: 4.3.1
Hardware: x86_64
OS: Linux
unspecified
medium vote
Target Milestone: ovirt-4.3.4
: ---
Assignee: Nir Levy
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-07 04:40 UTC by Shir
Modified: 2019-03-24 10:24 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-24 10:24:30 UTC
oVirt Team: Node
pm-rhel: ovirt-4.3+


Attachments (Terms of Use)

Description Shir 2019-03-07 04:40:56 UTC
Description of problem: Ovirt-host deploy completes with error "Failed to configure management network on the host"and host status set to  NonOperational.

error : Host <host FQDN> installation failed. Failed to configure management network on the host. 


Version-Release number of selected component (if applicable):Ovirt-4.3.1.1-1.el7


How reproducible: 


Steps to Reproduce:
1.deploy ovirt Engine : 4.3.1.1-1.el7 on Centos 7
2.deploy host from ovirt virtualisation manager
3.

Actual results:Host deploy completes with following errors,
VDSM <host FQDN> command HostSetupNetworksVDS failed: connectivity check failed
Failed to configure management network on host <host FQDN> due to setup networks failure.
Host <host FQDN> installation failed. Failed to configure management network on the host.


Expected results: Engine should deploy "ovirtmgmt" network on host, so that host status is operational.

Additional info: Workaround to solve this issue: Host -->click on <host name> -->Network Interface-->setup Host Network--> drag "ovirtmgmt" network  NIC on which you want to configure etwork

Comment 1 Sandro Bonazzola 2019-03-07 07:43:52 UTC
Can you please attach ovirt-log-collector report including the host which failed to activate?

Comment 2 Shir 2019-03-11 06:18:24 UTC
Hello Sandro,
log size is almost 40MB which i couldnt upload to bigzilla, would you be able to share me online storage whether i can upload the logs.

Comment 3 Shir 2019-03-12 03:21:03 UTC
Hello Team, Can you please share me online storage location where i can upload logs?

Comment 4 Sandro Bonazzola 2019-03-12 09:18:16 UTC
Shir, can you use something like dropbox or google drive and share the link?

Comment 5 Shir 2019-03-13 07:07:53 UTC
Hello Sandro, find the logs here : https://drive.google.com/open?id=1GcbdO4wIj_CCJ0mIZnx8ShSfqAi0gn9q

Comment 6 Nir Levy 2019-03-14 10:54:19 UTC
Hello Shir,

Inspecting the Engine logs indeed indicates a connection error between Engine and VDSM (Agent on host).
I will be helpfull getting 
/var/log/vdsm/ directory from the host for further investigation.

Regards.

Comment 7 Nir Levy 2019-03-17 13:51:58 UTC
 4591 2019-03-05 01:37:07,956-05 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [4d3a14d8] Connecting to /10.43.6.66
....
 4592 2019-03-05 01:37:08,097-05 INFO  [org.ovirt.engine.core.bll.host.HostConnectivityChecker] (EE-ManagedThreadFactory-engine-Thread-15) [15090f65-411a-4803-96c8-02400213d393] Engine managed to communicate wit      h VDSM agent on host 'inbauplhvl0001' with address '10.43.6.66' ('d195f2e3-aa22-45b2-a3d6-cbe85a7ad1be')
....
 4598 2019-03-05 01:37:11,863-05 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-15) [f043245] FINISH, HostSetupNetworksVDSCommand,
....
 4599 2019-03-05 01:37:47,093-05 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [4d3a14d8] No interaction with host '10.43.6.66' for 20000 ms

It is noticeable that there was a successfull communication between the engine and the host.
and 36 seconds later the engine reports, no interaction for 20 seconds.

/var/log/vdsm/ directory from the host is required for further investigation for understanding why the host does not interacts with the engine.

Comment 8 Shir 2019-03-18 04:40:44 UTC
Hello Nir Levy,

vdsn logs uploaded at https://drive.google.com/open?id=1GcbdO4wIj_CCJ0mIZnx8ShSfqAi0gn9q  
Please check and revert if any further info required.

Comment 9 Nir Levy 2019-03-18 12:28:12 UTC
Most logs from the 5/3 are unfortunately missing but there are several attempts to configure different NIC's with the same IP

Hi Shir.
While communication with the engine is over the ens3f3 interface

 889     MainProcess|jsonrpc/7::INFO::2019-03-05 13:14:41,237::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': False, u'nameservers': [u'10.43.129.82', u'10.42.2.5'], u'nic'     : u'ens3f3', u'ipaddr': u'10.43.6.66', u'netmask': u'255.255.255.192', u'mtu': 1500, u'switch': u'legacy', u'dhcpv6': False, u'stp': False, u'bridged': True, u'gateway': u'10.43.6.65', u'defaultRoute': True     , u'bootproto': u'none'})


As seen from this capabilities message, noth ens3fs and eno1 are configured with the same IP, it can be notice from the speed parameter that there is no link
on ens3fs and therefore no connectivity with the engine.

and the configuration of both ens3f3 
'ens3f3': {'ipv6autoconf': False, 'addr': '10.43.6.66', 'speed': 0, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '255.255.255.192', 'ipv     4defaultroute': True, 'ipv4addrs': ['10.43.6.66/26'], 'hwaddr': '3c:a8:2a:e8:7c:e3', 'ipv6gateway': '::', 'gateway': '10.43.6.65'}, 

'eno1': {'ipv6autoconf': True, 'addr': '10.43.6.66', 'speed': 1000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '255.255.255.192', 'ipv4defaultroute': True, 'ipv4addrs': ['10.4     3.6.66/26'], 'hwaddr': '94:18:82:7a:d5:84', 'ipv6gateway': '::', 'gateway': '10.43.6.65'},


Try to configure only one ip for NIC and send only 
/var/log/vdsm/supervdsm.log     and
var/log/ovirt-engine/engine.log

in case the problem persists.
Regards.
Nir.

Comment 10 Sandro Bonazzola 2019-03-19 09:19:32 UTC
Targeting to 4.3.4 for completing investigation.

Comment 11 Shir 2019-03-22 05:10:42 UTC
Hello Nir,

Thanks for the analysis, i was able to successfully add host to ovirt engine with single NIC and proper IP configuration.

Comment 12 Nir Levy 2019-03-24 10:24:30 UTC
Hi Shir,
Thanks for the update, glad this has helped you,
Closing issue, NOTABUG.


Note You need to log in before you can comment on or make changes to this bug.