Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1065322 - host deploy fails because iptables cannot be stopped while trying to unload kernel modules
Summary: host deploy fails because iptables cannot be stopped while trying to unload k...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: ovirt-host-deploy
Classification: oVirt
Component: Plugins.VDSM
Version: 1.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent vote
Target Milestone: ---
: 1.2.0
Assignee: Alon Bar-Lev
QA Contact: yeylon@redhat.com
URL:
Whiteboard: infra
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-14 11:05 UTC by Jiri Belka
Modified: 2016-04-18 06:55 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-03-10 10:40:45 UTC
oVirt Team: Infra
jbelka: devel_ack?


Attachments (Terms of Use)
setup logs (deleted)
2014-02-14 11:05 UTC, Jiri Belka
no flags Details

Description Jiri Belka 2014-02-14 11:05:19 UTC
Created attachment 863216 [details]
setup logs

Description of problem:

RHEL host is installed and has _only_ RHN channel registered. Relevant channels to RHEV:

rhel-x86_64-rhev-mgmt-agent-6
rhel-x86_64-server-6

The RHEL is updated to latest rpms, then it is being added from Admin Portal into RHEV env. But the installation fails:

...
2014-02-14 11:15:20 DEBUG otopi.plugins.otopi.services.rhel plugin.executeRaw:364 execute-result: ('/sbin/service', 'iptables', 'stop'), rc=1
2014-02-14 11:15:20 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:412 execute-output: ('/sbin/service', 'iptables', 'stop') stdout:
iptables: Setting chains to policy ACCEPT: nat mangle filter [  OK  ]
iptables: Flushing firewall rules: [  OK  ]
iptables: Unloading modules:  ip_tables[FAILED]

2014-02-14 11:15:20 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:417 execute-output: ('/sbin/service', 'iptables', 'stop') stderr:


2014-02-14 11:15:20 DEBUG otopi.context context._executeMethod:130 method exception
Traceback (most recent call last):
  File "/tmp/ovirt-jc2cALxday/pythonlib/otopi/context.py", line 120, in _executeMethod
    method['method']()
  File "/tmp/ovirt-jc2cALxday/otopi-plugins/otopi/network/iptables.py", line 111, in _closeup
    self.services.state('iptables', False)
  File "/tmp/ovirt-jc2cALxday/otopi-plugins/otopi/services/rhel.py", line 184, in state
    'start' if state else 'stop'
  File "/tmp/ovirt-jc2cALxday/otopi-plugins/otopi/services/rhel.py", line 96, in _executeServiceCommand
    raiseOnError=raiseOnError
  File "/tmp/ovirt-jc2cALxday/pythonlib/otopi/plugin.py", line 422, in execute
    command=args[0],
RuntimeError: Command '/sbin/service' failed to execute
2014-02-14 11:15:20 ERROR otopi.context context._executeMethod:139 Failed to execute stage 'Closing up': Command '/sbin/service' failed to execute
...

Interesting that after couple of removing (Remove button) of the failed host from setup and re-adding it again (New button), no success but then I tried to 'service iptables restart' and re-added it again it it passed, reboot of the host appeared and the host was set up after some time.

Version-Release number of selected component (if applicable):
vdsm-python-4.13.2-0.9.el6ev.x86_64
vdsm-python-cpopen-4.13.2-0.9.el6ev.x86_64
vdsm-cli-4.13.2-0.9.el6ev.noarch
vdsm-4.13.2-0.9.el6ev.x86_64
vdsm-xmlrpc-4.13.2-0.9.el6ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. install RHEL from RHN with the channels written above, no other repos on the system!
2. add into 3.2 setup rhevm-3.2.5-0.49.el6ev.noarch
3.

Actual results:
fail, fail, fail, (passed??)

Expected results:
pass

Additional info:

Comment 1 Alon Bar-Lev 2014-02-14 11:59:47 UTC
this is probably a bug in iptables, not sure why you open it for rhev product...

service stop iptables should succeed.

Comment 2 Jiri Belka 2014-02-14 12:35:10 UTC
It seems to me that problem can be found here:

  /sbin/initctl', 'status', 'iptables'), rc=1

# /sbin/initctl status iptables ; echo $?
initctl: Unknown job: iptables
1

Not sure why "you" start iptables with '/sbin/service' but "you" query status with "/sbin/initctl", when '/sbin/service' has status as well.

Comment 3 Alon Bar-Lev 2014-02-14 17:24:45 UTC
(In reply to Jiri Belka from comment #2)
> It seems to me that problem can be found here:
> 
>   /sbin/initctl', 'status', 'iptables'), rc=1
> 
> # /sbin/initctl status iptables ; echo $?
> initctl: Unknown job: iptables
> 1

this is perfectly ok, then it falls back to sysv.

> 
> Not sure why "you" start iptables with '/sbin/service' but "you" query
> status with "/sbin/initctl", when '/sbin/service' has status as well.

The problem per what you wrote in commen#0:

2014-02-14 11:15:20 DEBUG otopi.plugins.otopi.services.rhel plugin.executeRaw:364 execute-result: ('/sbin/service', 'iptables', 'stop'), rc=1
2014-02-14 11:15:20 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:412 execute-output: ('/sbin/service', 'iptables', 'stop') stdout:
iptables: Setting chains to policy ACCEPT: nat mangle filter [  OK  ]
iptables: Flushing firewall rules: [  OK  ]
iptables: Unloading modules:  ip_tables[FAILED]

Please open a regression bug for iptables.

Comment 4 Alon Bar-Lev 2014-02-14 18:45:17 UTC
"""
The Problem you have is: Some open connection depending on the iptables modles. Mean,(this i what i think, not so sure) there is some open connection that is route via iptables. Disabling iptables would mean to interrupt this connection. 
"""[1]

Related: bug#442335, bug@313051, bug#212839.

This behavior was not changed from our side since rhev-3.2.

[1] http://www.linux.org/threads/iptables-problem-help-wanted.3211/

Comment 5 Alon Bar-Lev 2014-02-14 18:47:23 UTC
Not sure it is a valid workaround.

"""
Setting IPTABLES_MODULES_UNLOAD="no" in /etc/sysconfig/iptables-config works for me.
"""[1]

[1] https://www.centos.org/forums/viewtopic.php?t=9045

Comment 6 Barak 2014-03-03 19:26:01 UTC
(In reply to Alon Bar-Lev from comment #5)
> Not sure it is a valid workaround.
> 
> """
> Setting IPTABLES_MODULES_UNLOAD="no" in /etc/sysconfig/iptables-config works
> for me.
> """[1]
> 
> [1] https://www.centos.org/forums/viewtopic.php?t=9045

Not sure we should do that, 
I don't see any bug on iptables.

Comment 7 Alon Bar-Lev 2014-03-03 19:32:17 UTC
(In reply to Barak from comment #6)
> (In reply to Alon Bar-Lev from comment #5)
> > Not sure it is a valid workaround.
> > 
> > """
> > Setting IPTABLES_MODULES_UNLOAD="no" in /etc/sysconfig/iptables-config works
> > for me.
> > """[1]
> > 
> > [1] https://www.centos.org/forums/viewtopic.php?t=9045
> 
> Not sure we should do that, 
> I don't see any bug on iptables.

There were few in the past (unrelated to us), and apart from this single report we have not gotten any other report, if we see more we should open a bug against iptables.

Comment 10 Jiri Belka 2014-03-10 10:40:45 UTC
I can't reproduce:

# rpm -qa vdsm\* iptables\*
iptables-ipv6-1.4.7-11.el6.x86_64
vdsm-cli-4.13.2-0.11.el6ev.noarch
vdsm-xmlrpc-4.13.2-0.11.el6ev.noarch
vdsm-4.13.2-0.11.el6ev.x86_64
iptables-1.4.7-11.el6.x86_64
vdsm-python-4.13.2-0.11.el6ev.x86_64

...
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       2014-03-10 11:34:48 DEBUG otopi.plugins.otopi.services.rhel plugin.executeRaw:347 execute
: ('/sbin/service', 'iptables', 'stop'), executable='None', cwd='None', env=None
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       2014-03-10 11:34:48 DEBUG otopi.plugins.otopi.services.rhel plugin.executeRaw:364 execute
-result: ('/sbin/service', 'iptables', 'stop'), rc=0
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       2014-03-10 11:34:48 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:412 execute-ou
tput: ('/sbin/service', 'iptables', 'stop') stdout:
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       iptables: Setting chains to policy ACCEPT: nat mangle filter [  OK  ]
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       iptables: Flushing firewall rules: [  OK  ]
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       iptables: Unloading modules: [  OK  ]
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       2014-03-10 11:34:48 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:417 execute-ou
tput: ('/sbin/service', 'iptables', 'stop') stderr:
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       
2014-03-10 11:34:49 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       
...


Note You need to log in before you can comment on or make changes to this bug.