Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1684477 - Openvswich process is not restarted after a kill command, in 20% of the times
Summary: Openvswich process is not restarted after a kill command, in 20% of the times
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux Fast Datapath
Classification: Red Hat
Component: openvswitch2.11
Version: FDP 19.03
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Timothy Redaelli
QA Contact: Rick Alongi
URL:
Whiteboard:
Depends On:
Blocks: 1684483
TreeView+ depends on / blocked
 
Reported: 2019-03-01 11:05 UTC by Timothy Redaelli
Modified: 2019-04-02 15:55 UTC (History)
4 users (show)

Fixed In Version: openvswitch2.11-2.11.0-2.el7fdn
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1653717
: 1684483 (view as bug list)
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Description Timothy Redaelli 2019-03-01 11:05:53 UTC
+++ This bug was initially created as a clone of Bug #1653717 +++

Description of problem:

 Open vswich process is not restarted after a kill 20% of the times. 


Version-Release number of selected component (if applicable):

 OSP 14

 3 controllers + 3 computes + dvr

How reproducible:

 1. Install a baremetal with RHEL(theforeman)
 2. Intall with jenkins: OSP 14 -- 3 controllers + 3 computes + dvr
 3. Create sever vms with a Fip
 4. Kill openvswitch process in a compuete node



Actual results:

 the process is not restarted 20% of the times. 

Expected results:

 the process is restarted.

Additional info:

Logs when restart openvswich is failing:

[root@compute-2 heat-admin]# tail -f  /var/log/containers/neutron/metadata-agent.log
2018-11-26 08:51:10.883 8224 INFO eventlet.wsgi.server [-] 10.2.0.18,<local> "GET /2009-04-04/meta-data/placement/availability-zone HTTP/1.1" status: 200  len: 139 time: 0.1934390
2018-11-26 08:51:10.968 8223 INFO eventlet.wsgi.server [-] 10.3.0.25,<local> "GET /2009-04-04/meta-data/placement/availability-zone HTTP/1.1" status: 200  len: 139 time: 0.1648369
2018-11-26 09:11:20.883 8224 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: IOError: Socket closed
2018-11-26 09:11:34.307 7137 ERROR oslo.messaging._drivers.impl_rabbit [-] [357cec88-105c-4f2b-b26c-70ae3864ec0f] AMQP server controller-2.internalapi.localdomain:5672 closed the connection. Check login credentials: Socket closed: IOError: Socket closed
2018-11-26 09:11:35.323 7137 INFO oslo.messaging._drivers.impl_rabbit [-] [357cec88-105c-4f2b-b26c-70ae3864ec0f] Reconnected to AMQP server on controller-2.internalapi.localdomain:5672 via [amqp] client with port 50096.
2018-11-26 09:11:43.938 8223 ERROR oslo.messaging._drivers.impl_rabbit [-] [24405812-662e-4c40-b834-37a07d80366f] AMQP server controller-1.internalapi.localdomain:5672 closed the connection. Check login credentials: Socket closed: IOError: Socket closed
2018-11-26 09:11:44.955 8223 INFO oslo.messaging._drivers.impl_rabbit [-] [24405812-662e-4c40-b834-37a07d80366f] Reconnected to AMQP server on controller-1.internalapi.localdomain:5672 via [amqp] client with port 37994.
2018-11-26 09:11:45.218 8224 ERROR oslo.messaging._drivers.impl_rabbit [-] [7bf56d8a-afcf-4b87-b2dd-c865c2faf08f] AMQP server controller-2.internalapi.localdomain:5672 closed the connection. Check login credentials: Socket closed: IOError: Socket closed
2018-11-26 09:11:46.235 8224 INFO oslo.messaging._drivers.impl_rabbit [-] [7bf56d8a-afcf-4b87-b2dd-c865c2faf08f] Reconnected to AMQP server on controller-2.internalapi.localdomain:5672 via [amqp] client with port 50116.
2018-11-26 09:11:59.117 8223 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: IOError: Socket closed





vi /var/log/containers/neutron/openvswitch-agent.log


2018-11-26 09:12:04.417 7137 ERROR oslo.messaging._drivers.impl_rabbit [-] [8f9c38d5-3ad8-4cab-8446-a2f07e6d370f] AMQP server controller-1.internalapi.localdomain:5672 closed the connection. Check login credentials: Socket closed: IOError: Socket closed
2018-11-26 09:12:05.444 7137 INFO oslo.messaging._drivers.impl_rabbit [-] [8f9c38d5-3ad8-4cab-8446-a2f07e6d370f] Reconnected to AMQP server on controller-1.internalapi.localdomain:5672 via [amqp] client with port 38042.
2018-11-26 09:12:15.386 8224 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer

2018-11-26 14:23:44.338 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     raise RuntimeError(m)
2018-11-26 14:23:44.338 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int RuntimeError: Switch connection timeout
2018-11-26 14:23:44.338 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-11-26 14:23:44.339 28480 WARNING neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] OVS is dead. OVSNeutronAgent will keep running and checking OVS status periodically.
2018-11-26 14:23:44.340 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Agent rpc_loop - iteration:623 completed. Processed ports statistics: {'regular': {'updated': 0, 'added': 0, 'removed': 0}}. Elapsed:30.020 loop_count_and_wait /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1875
2018-11-26 14:23:44.340 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Loop iteration exceeded interval (2 vs. 30.0197079182)! loop_count_and_wait /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1882
2018-11-26 14:23:44.340 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Agent rpc_loop - iteration:624 started rpc_loop /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2086
2018-11-26 14:24:14.353 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Switch connection timeout
2018-11-26 14:24:14.354 28480 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbGetCommand(column=datapath_id, table=Bridge, record=br-int) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2018-11-26 14:24:14.355 28480 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:121
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Failed to communicate with the switch: RuntimeError: Switch connection timeout
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback (most recent call last):
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_int.py", line 52, in check_canary_table
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     flows = self.dump_flows(constants.CANARY_TABLE)
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 156, in dump_flows
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     (dp, ofp, ofpp) = self._get_dp()
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_bridge.py", line 69, in _get_dp
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     self._cached_dpid = new_dpid
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     self.force_reraise()
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     six.reraise(self.type_, self.value, self.tb)
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_bridge.py", line 52, in _get_dp
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     dp = self._get_dp_by_dpid(self._cached_dpid)
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 79, in _get_dp_by_dpid
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     raise RuntimeError(m)
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int RuntimeError: Switch connection timeout
2018-11-26 14:24:14.355 28480 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-11-26 14:24:14.356 28480 WARNING neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] OVS is dead. OVSNeutronAgent will keep running and checking OVS status periodically.
2018-11-26 14:24:14.356 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Agent rpc_loop - iteration:624 completed. Processed ports statistics: {'regular': {'updated': 0, 'added': 0, 'removed': 0}}. Elapsed:30.016 loop_count_and_wait /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1875
2018-11-26 14:24:14.357 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Loop iteration exceeded interval (2 vs. 30.0161988735)! loop_count_and_wait /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1882
2018-11-26 14:24:14.357 28480 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-cb6bf194-1ceb-415b-aecc-3ba950053b37 - - - - -] Agent rpc_loop - iteration:625 started rpc_loop /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2086

overcloud) [stack@undercloud-0 ~]$ openstack versions show
+-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+
| Region Name | Service Type | Version | Status | Endpoint | Min Microversion | Max Microversion |
+-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+
| regionOne | block-storage | 2.0 | DEPRECATED | http://10.0.0.101:8776/v2/ | None | None |
| regionOne | block-storage | 3.0 | CURRENT | http://10.0.0.101:8776/v3/ | 3.0 | 3.55 |
| regionOne | placement | None | CURRENT | http://10.0.0.101:8778/placement/ | None | None |
| regionOne | network | 2.0 | CURRENT | http://10.0.0.101:9696/v2.0/ | None | None |
| regionOne | alarm | 2.0 | CURRENT | http://10.0.0.101:8042/v2 | None | None |
| regionOne | cloudformation | 1.0 | CURRENT | http://10.0.0.101:8000/v1/ | None | None |
| regionOne | event | 2.0 | CURRENT | http://10.0.0.101:8977/v2 | None | None |
| regionOne | orchestration | 1.0 | CURRENT | http://10.0.0.101:8004/v1/ | None | None |
| regionOne | object-store | 1.0 | CURRENT | http://10.0.0.101:8080/v1/ | None | None |
| regionOne | compute | 2.0 | SUPPORTED | http://10.0.0.101:8774/v2/ | None | None |
| regionOne | compute | 2.1 | CURRENT | http://10.0.0.101:8774/v2.1/ | 2.1 | 2.65 |
| regionOne | image | 2.0 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.1 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.2 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.3 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.4 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.5 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.6 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | image | 2.7 | CURRENT | http://10.0.0.101:9292/v2/ | None | None |
| regionOne | metric | 1.0 | CURRENT | http://10.0.0.101:8041/v1/ | None | None |
| regionOne | identity | 3.10 | CURRENT | http://10.0.0.101:5000/v3/ | None | None |
+-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+
(overcloud) [stack@undercloud-0 ~]$ cat /etc/re
redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf
(overcloud) [stack@undercloud-0 ~]$ cat /etc/re
redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf
(overcloud) [stack@undercloud-0 ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)

--- Additional comment from Candido Campos on 2019-02-28 14:32:57 CET ---

the change that avoids the issue is:

/usr/lib/systemd/system/ovsdb-server.service

...
[Service]
Type=forking
+PIDFile=/var/run/openvswitch/ovsdb-server.pid
Restart=on-failure

...

/usr/lib/systemd/system/ovs-vswitchd.service

....
[Service]
Type=forking
+PIDFile=/var/run/openvswitch/ovs-vswitchd.pid
Restart=on-failure
....


diff --git a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
index 525deae0b..82925133d 100644
--- a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
+++ b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
@@ -9,6 +9,7 @@ PartOf=openvswitch.service
 
 [Service]
 Type=forking
+PIDFile=/var/run/openvswitch/ovs-vswitchd.pid
 Restart=on-failure
 Environment=XDG_RUNTIME_DIR=/var/run/openvswitch
 EnvironmentFile=/etc/openvswitch/default.conf
diff --git a/rhel/usr_lib_systemd_system_ovsdb-server.service b/rhel/usr_lib_systemd_system_ovsdb-server.service
index 70da1ec95..a7a1e03cb 100644
--- a/rhel/usr_lib_systemd_system_ovsdb-server.service
+++ b/rhel/usr_lib_systemd_system_ovsdb-server.service
@@ -8,6 +8,7 @@ PartOf=openvswitch.service
 [Service]
 Type=forking
 Restart=on-failure
+PIDFile=/var/run/openvswitch/ovsdb-server.pid
 EnvironmentFile=/etc/openvswitch/default.conf
 EnvironmentFile=-/etc/sysconfig/openvswitch
 ExecStartPre=/usr/bin/chown ${OVS_USER_ID} /var/run/openvswitch /var/log/openvswitch
[ccamposr@localhost ovs]$

Comment 3 Rick Alongi 2019-04-02 15:55:23 UTC
Created the script below to test the issue:

[root@netqe20 ~]# cat bz1684477.sh 
#!/bin/bash

iterations=${iterations:-"50"}
result_file="/home/result_file.txt"
rm -f $result_file

check_final_result()
{
	if [[ $(grep FAIL $result_file) ]]; then
		echo "$count crash iterations run of $iterations total"
		echo "Overall Test Result: FAIL"		
	else
		let count--
		echo "$count crash iterations run of $iterations total"
		echo "Overall Test Result: PASS"
	fi
}

test_systemctl_status_running()
{
	service=$1
	systemctl_status=$(systemctl status $service | grep Active | awk '{print $2$3}')
	if [[ $service == "openvswitch" ]]; then
		if [[ "$systemctl_status" == "active(exited)" ]]; then
			echo "PASS" >> $result_file
		else
			echo "FAIL" >> $result_file
			check_final_result
			exit 1
		fi
	elif [[ $service == "ovs-vswitchd" ]] || [[ $service == "ovsdb-server" ]] && [[ "$systemctl_status" == "active(running)" ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
}

test_systemctl_status_stopped()
{
	service=$1
	systemctl_status=$(systemctl status $service | grep Active | awk '{print $2$3}')
	if [[ $service == "openvswitch" ]] && [[ "$systemctl_status" == "inactive(dead)" ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
}

test_process_crashed_restarted()
{
	process=$1
	vswitchd_pid1=$(pgrep vswitchd)
	ovsdb_pid1=$(pgrep vswitchd)
	if [[ $process == "vswitchd" ]]; then
		kill -SIGSEGV $vswitchd_pid1
	elif [[ $process == "ovsdb" ]]; then
		kill -SIGSEGV $ovsdb_pid1
	fi
	sleep 3
	vswitchd_pid2=$(pgrep vswitchd)
	ovsdb_pid2=$(pgrep vswitchd)
	if [[ -z $vswitchd_pid2 ]] || [[ $vswitchd_pid2 == $vswitchd_pid1 ]]; then
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	else
		echo "PASS" >> $result_file
	fi
	
	if [[ -z $ovsdb_pid2 ]] || [[ $ovsdb_pid2 == $ovsdb_pid1 ]]; then
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	else
		echo "PASS" >> $result_file
	fi
}

test_process_started()
{
	vswitchd_pid=$(pgrep vswitchd)
	ovsdb_pid=$(pgrep vswitchd)

	if [[ $vswitchd_pid ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
	
	if [[ $ovsdb_pid ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
}

test_process_stopped()
{
	vswitchd_pid=$(pgrep vswitchd)
	ovsdb_pid=$(pgrep vswitchd)

	if [[ -z $vswitchd_pid ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
	
	if [[ -z $ovsdb_pid ]]; then
		echo "PASS" >> $result_file
	else
		echo "FAIL" >> $result_file
		check_final_result
		exit 1
	fi
}

get_process_pid()
{
	process=$1
	"$process"_pid=$(pgrep $process)
}

systemctl stop openvswitch
sleep 3

### Tests
# start openvswitch
systemctl start openvswitch
sleep 3
test_systemctl_status_running openvswitch
test_systemctl_status_running ovs-vswitchd
test_systemctl_status_running ovsdb-server
test_process_started

# crash vswitchd
count=1
while [ $count -le $iterations ]; do
	echo "Running vswitchd crash iteration $count of $iterations"
	test_process_crashed_restarted vswitchd
	let count++
done

# crash ovsdb
count=1
while [ $count -le $iterations ]; do
	echo "Running ovsdb crash iteration $count of $iterations"
	test_process_crashed_restarted ovsdb
	let count++
done

check_final_result

#########################################################

Tested several times against openvswitch-2.9.0-83.el7fdp.1.x86_64.  Test never successfully proceeded beyond vswitch crash test.  Below is an example test run: 

[root@netqe20 ~]# ./bz1684477.sh 
Running vswitchd crash iteration 1 of 50
Running vswitchd crash iteration 2 of 50
Running vswitchd crash iteration 3 of 50
Running vswitchd crash iteration 4 of 50
Running vswitchd crash iteration 5 of 50
Running vswitchd crash iteration 6 of 50
Running vswitchd crash iteration 7 of 50
Running vswitchd crash iteration 8 of 50
Running vswitchd crash iteration 9 of 50
Running vswitchd crash iteration 10 of 50
Running vswitchd crash iteration 11 of 50
Running vswitchd crash iteration 12 of 50
Running vswitchd crash iteration 13 of 50
13 crash iterations run of 50 total
Overall Test Result: FAIL
[root@netqe20 ~]# rpm -q openvswitch
openvswitch-2.9.0-83.el7fdp.1.x86_64

Tested with openvswitch2.11-2.11.0-3.el7fdp.x86_64.  All tests passed.  Output below:

[root@netqe21 ~]# ./bz1684477.sh 
Running vswitchd crash iteration 1 of 50
Running vswitchd crash iteration 2 of 50
Running vswitchd crash iteration 3 of 50
Running vswitchd crash iteration 4 of 50
Running vswitchd crash iteration 5 of 50
Running vswitchd crash iteration 6 of 50
Running vswitchd crash iteration 7 of 50
Running vswitchd crash iteration 8 of 50
Running vswitchd crash iteration 9 of 50
Running vswitchd crash iteration 10 of 50
Running vswitchd crash iteration 11 of 50
Running vswitchd crash iteration 12 of 50
Running vswitchd crash iteration 13 of 50
Running vswitchd crash iteration 14 of 50
Running vswitchd crash iteration 15 of 50
Running vswitchd crash iteration 16 of 50
Running vswitchd crash iteration 17 of 50
Running vswitchd crash iteration 18 of 50
Running vswitchd crash iteration 19 of 50
Running vswitchd crash iteration 20 of 50
Running vswitchd crash iteration 21 of 50
Running vswitchd crash iteration 22 of 50
Running vswitchd crash iteration 23 of 50
Running vswitchd crash iteration 24 of 50
Running vswitchd crash iteration 25 of 50
Running vswitchd crash iteration 26 of 50
Running vswitchd crash iteration 27 of 50
Running vswitchd crash iteration 28 of 50
Running vswitchd crash iteration 29 of 50
Running vswitchd crash iteration 30 of 50
Running vswitchd crash iteration 31 of 50
Running vswitchd crash iteration 32 of 50
Running vswitchd crash iteration 33 of 50
Running vswitchd crash iteration 34 of 50
Running vswitchd crash iteration 35 of 50
Running vswitchd crash iteration 36 of 50
Running vswitchd crash iteration 37 of 50
Running vswitchd crash iteration 38 of 50
Running vswitchd crash iteration 39 of 50
Running vswitchd crash iteration 40 of 50
Running vswitchd crash iteration 41 of 50
Running vswitchd crash iteration 42 of 50
Running vswitchd crash iteration 43 of 50
Running vswitchd crash iteration 44 of 50
Running vswitchd crash iteration 45 of 50
Running vswitchd crash iteration 46 of 50
Running vswitchd crash iteration 47 of 50
Running vswitchd crash iteration 48 of 50
Running vswitchd crash iteration 49 of 50
Running vswitchd crash iteration 50 of 50
Running ovsdb crash iteration 1 of 50
Running ovsdb crash iteration 2 of 50
Running ovsdb crash iteration 3 of 50
Running ovsdb crash iteration 4 of 50
Running ovsdb crash iteration 5 of 50
Running ovsdb crash iteration 6 of 50
Running ovsdb crash iteration 7 of 50
Running ovsdb crash iteration 8 of 50
Running ovsdb crash iteration 9 of 50
Running ovsdb crash iteration 10 of 50
Running ovsdb crash iteration 11 of 50
Running ovsdb crash iteration 12 of 50
Running ovsdb crash iteration 13 of 50
Running ovsdb crash iteration 14 of 50
Running ovsdb crash iteration 15 of 50
Running ovsdb crash iteration 16 of 50
Running ovsdb crash iteration 17 of 50
Running ovsdb crash iteration 18 of 50
Running ovsdb crash iteration 19 of 50
Running ovsdb crash iteration 20 of 50
Running ovsdb crash iteration 21 of 50
Running ovsdb crash iteration 22 of 50
Running ovsdb crash iteration 23 of 50
Running ovsdb crash iteration 24 of 50
Running ovsdb crash iteration 25 of 50
Running ovsdb crash iteration 26 of 50
Running ovsdb crash iteration 27 of 50
Running ovsdb crash iteration 28 of 50
Running ovsdb crash iteration 29 of 50
Running ovsdb crash iteration 30 of 50
Running ovsdb crash iteration 31 of 50
Running ovsdb crash iteration 32 of 50
Running ovsdb crash iteration 33 of 50
Running ovsdb crash iteration 34 of 50
Running ovsdb crash iteration 35 of 50
Running ovsdb crash iteration 36 of 50
Running ovsdb crash iteration 37 of 50
Running ovsdb crash iteration 38 of 50
Running ovsdb crash iteration 39 of 50
Running ovsdb crash iteration 40 of 50
Running ovsdb crash iteration 41 of 50
Running ovsdb crash iteration 42 of 50
Running ovsdb crash iteration 43 of 50
Running ovsdb crash iteration 44 of 50
Running ovsdb crash iteration 45 of 50
Running ovsdb crash iteration 46 of 50
Running ovsdb crash iteration 47 of 50
Running ovsdb crash iteration 48 of 50
Running ovsdb crash iteration 49 of 50
Running ovsdb crash iteration 50 of 50
50 crash iterations run of 50 total for each process
Overall Test Result: PASS
[root@netqe21 ~]# rpm -q openvswitch2.11
openvswitch2.11-2.11.0-3.el7fdp.x86_64

#########################################################

Marking as Verified.


Note You need to log in before you can comment on or make changes to this bug.