Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1097674

Summary: [setupNetworks]IP address for BOND is not updated on the host
Product: Red Hat Enterprise Virtualization Manager Reporter: Meni Yakove <myakove>
Component: vdsmAssignee: Barak <bazulay>
Status: CLOSED ERRATA QA Contact: Meni Yakove <myakove>
Severity: high Docs Contact:
Priority: high    
Version: 3.4.0CC: bazulay, danken, gklein, iheim, lpeer, nyechiel, yeylon
Target Milestone: ---Keywords: Regression, ZStream
Target Release: 3.5.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: network
Fixed In Version: vt1.3, 4.16.0-1.el6_5 Doc Type: Bug Fix
Doc Text:
Previously, a change was introduced that prevented a bond's IP from being cleared when a new one was being set, causing the new IP address to be configured as secondary, alongside the previous one, instead of as the sole primary IP address. Now, the previous IP bonding configuration is removed when reconfiguring the IP, so that there are no leftover IPs from previous configurations.
Story Points: ---
Clone Of:
: 1105645 (view as bug list) Environment:
Last Closed: 2015-02-11 21:10:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1105645, 1142923, 1156165    
Attachments:
Description Flags
vdsm logs
none
Proper vdsm logs none

Description Meni Yakove 2014-05-14 09:48:58 UTC
Created attachment 895424 [details]
vdsm logs

Description of problem:
Only for network attached to BOND
When updating network with new IP address (for the network that already has IP) the new IP is not updated on the host and if we update the IP address again the previos IP is updated on the host.
For example:

Network "net1" with ip 1.1.1.1 > in setupNetworks edit the network and update the IP to 1.1.1.2
        Though on GUI the IP address is 1.1.1.2, on the host the IP for "net1" is still 1.1.1.1 and getVdsCaps report 1.1.1.1
In setupNetworks edit the network and update the IP to 1.1.1.3
        Though on GUI the IP address is 1.1.1.3, on the host the IP is 1.1.1.2 and getVdsCaps report 1.1.1.2

It seems that ifup called before the ifcfg file is updated with the new IP.



Version-Release number of selected component (if applicable):
vdsm-4.14.7-2.el6ev.x86_64
rhevm-3.4.0-0.20.el6ev.noarch

How reproducible:
100%

Steps to Reproduce:
1.Create network "net1", 
2.In seupNetworks create BOND and attach the network to the BOND and add IP 1.1.1.1
3.In setupNetworks edit "net1" IP to 1.1.1.2 > check IP on the host (ifconfig bond(X)) 
4.In setupNetworks edit "net1" IP to 1.1.1.3 > check IP on the host (ifconfig bond(X)) 

Actual results:
IP is not updated on the host

Expected results:
IP should be updated on the host

Additional info:

Comment 1 Meni Yakove 2014-05-14 10:37:26 UTC
Created attachment 895438 [details]
Proper vdsm logs

Comment 2 Meni Yakove 2014-05-14 10:38:05 UTC
Only happens for non-vm network over BOND.

Comment 3 Antoni Segura Puimedon 2014-05-19 13:50:42 UTC
From the logs I can see:

    Thread-16::DEBUG::2014-05-14 13:26:39,826::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.63]::call setupNetworks with ({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.1', 'netmask': '255.255.255.0', 'bridged': 'false'}, 'mode_4': {'remove': 'true'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {} flowID [1ff1e6a1]
    .
    .
    .
    Thread-16::DEBUG::2014-05-14 13:26:48,245::BindingXMLRPC::1074::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}}

And the next getCapabilities includes:

    'bond0': {'netmask': '255.255.255.0', 'addr': '1.1.1.1', 'slaves': ['eth2', 'eth3'], 'hwaddr': '00:10:18:24:4a:fc', 'cfg': {'DEFROUTE': 'no', 'IPADDR': '1.1.1.1', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'BONDING_OPTS': 'mode=4 miimon=100', 'DEVICE': 'bond0', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::210:18ff:fe24:4afc/64'], 'mtu': '1500'}

After that:

    Thread-16::DEBUG::2014-05-14 13:26:58,858::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.63]::call setupNetworks with ({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.2', 'netmask': '255.255.255.0', 'bridged': 'false'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {} flowID [7ad27bdf]
    .
    .
    .
    Thread-16::DEBUG::2014-05-14 13:27:09,008::BindingXMLRPC::1074::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}}
    
And the next getCapabilities includes:

Thread-16::DEBUG::2014-05-14 13:27:09,008::BindingXMLRPC::1074::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}}

   'bond0': {'netmask': '255.255.255.0', 'addr': '1.1.1.2', 'slaves': ['eth2', 'eth3'], 'hwaddr': '00:10:18:24:4a:fc', 'cfg': {'DEFROUTE': 'no', 'IPADDR': '1.1.1.2', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'BONDING_OPTS': 'mode=4 miimon=100', 'DEVICE': 'bond0', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::210:18ff:fe24:4afc/64'], 'mtu': '1500'} 

After that:

    Thread-16::DEBUG::2014-05-14 13:27:21,689::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.63]::call setupNetworks with ({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.3', 'netmask': '255.255.255.0', 'bridged': 'false'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {} flowID [6ad1c1da]
    .
    .
    .
    Thread-16::DEBUG::2014-05-14 13:27:32,053::BindingXMLRPC::1074::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}}

And the next getCapabilities includes:

    'bond0': {'netmask': '255.255.255.0', 'addr': '1.1.1.3', 'slaves': ['eth2', 'eth3'], 'hwaddr': '00:10:18:24:4a:fc', 'cfg': {'DEFROUTE': 'no', 'IPADDR': '1.1.1.3', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'BONDING_OPTS': 'mode=4 miimon=100', 'DEVICE': 'bond0', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::210:18ff:fe24:4afc/64'], 'mtu': '1500'}
    

I fail to see the issue with the provided logs.

Comment 4 Antoni Segura Puimedon 2014-05-19 14:14:00 UTC
Went to manually test on a host.

    In [19]: c.setupNetworks({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.1', 'netmask': '255.255.255.0', 'bridged': 'false'}}, {'bond0': {'nics': ['eth2', 'eth3']}}, {'connectivityCheck': False})
    Out[19]: {'status': {'code': 0, 'message': 'Done'}}

    In [24]: list(addr for addr in netinfo.iter_addrs() if addr['label'] == 'bond0')
    Out[24]: 
    [{'address': '1.1.1.1/24',
      'family': 'inet',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 24,
      'scope': 'universe'},
     {'address': 'fe80::201:a4ff:feac:8701/64',
      'family': 'inet6',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 64,
      'scope': 'link'}]

    In [25]: c.setupNetworks({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.2', 'netmask': '255.255.255.0', 'bridged': 'false'}}, {},{'connectivityCheck': False})
    Out[25]: {'status': {'code': 0, 'message': 'Done'}}

    In [26]: list(addr for addr in netinfo.iter_addrs() if addr['label'] == 'bond0')
    Out[26]: 
    [{'address': '1.1.1.1/24',
      'family': 'inet',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 24,
      'scope': 'universe'},
     {'address': '1.1.1.2/24',
      'family': 'inet',
      'flags': 129,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 24,
      'scope': 'universe'},
     {'address': 'fe80::201:a4ff:feac:8701/64',
      'family': 'inet6',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 64,
      'scope': 'link'}]

    In [27]: c.setupNetworks({'mode0_non_vm': {'bonding': 'bond0', 'ipaddr': '1.1.1.3', 'netmask': '255.255.255.0', 'bridged': 'false'}}, {},{'connectivityCheck': False})
    Out[27]: {'status': {'code': 0, 'message': 'Done'}}

    In [28]: list(addr for addr in netinfo.iter_addrs() if addr['label'] == 'bond0')
    Out[28]: 
    [{'address': '1.1.1.2/24',
      'family': 'inet',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 24,
      'scope': 'universe'},
     {'address': '1.1.1.3/24',
      'family': 'inet',
      'flags': 129,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 24,
      'scope': 'universe'},
     {'address': 'fe80::201:a4ff:feac:8701/64',
      'family': 'inet6',
      'flags': 128,
      'index': 10,
      'label': 'bond0',
      'prefixlen': 64,
      'scope': 'link'}]

We can see that the abnormal behavior is that after changing the address to
1.1.1.2 instead of changing it it has merely been added as a secondary ipv4
address and that after changing it to 1.1.1.3, 1.1.1.2 became the primary and
the new address became secondary.

The ipaddresses are still being configured and functional, and as shown in the
logs in the previous comment, the ifcfg files ('cfg' field) of the bond0 is
correct at each interval. I've got to check if we are doing some wrong flow
or if the bug lies with initscripts.

Comment 5 Antoni Segura Puimedon 2014-05-19 14:46:55 UTC
The bug is caused by the bond removal (before the addition of the modified network) not clearing the ifcfg file IP configuration fields.

Comment 6 Antoni Segura Puimedon 2014-05-19 14:59:34 UTC
Caused by http://gerrit.ovirt.org/18911

Comment 7 Dan Kenigsberg 2014-05-20 15:12:07 UTC
... and with us since 3.3, so not a recent regression.

Comment 12 Meni Yakove 2014-07-17 07:36:25 UTC
ovirt-engine-3.5.0-0.0.master.20140715172116.git4687dc1.el6.noarch
vdsm-4.16.0-27.git00146ed.el6.x86_64

Comment 16 errata-xmlrpc 2015-02-11 21:10:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0159.html