Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1358316 - RHV self-hosted fails at 72%
Summary: RHV self-hosted fails at 72%
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Quickstart Cloud Installer
Classification: Red Hat
Component: Installation - RHEV
Version: 1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ga
: 1.0
Assignee: Fabian von Feilitzsch
QA Contact: Thom Carlin
Dan Macpherson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-20 13:45 UTC by Thom Carlin
Modified: 2016-09-13 16:32 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-13 16:32:52 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:1862 normal SHIPPED_LIVE Red Hat Quickstart Installer 1.0 2016-09-13 20:18:48 UTC
Red Hat Bugzilla 1362134 None None None Never

Description Thom Carlin 2016-07-20 13:45:15 UTC
Description of problem:

Self-hosted RHV fails deployment at 72%

Version-Release number of selected component (if applicable):

QCI-1.2-RHEL-7-20160718.t.0

How reproducible:

Uncertain

Steps to Reproduce:
1. Install and configure QCI
2. Deploy RHV with self-hosted

Actual results:

Error at 72% of Deploy Red Hat Enterprise Virtualization

Expected results:

No error, successful deployment

Additional info:

Host page on Sat 6 web UI:
* Status: Error
* Configuration: Error

Report Status:
* Failed: 6

"hosted-engine --deploy --config-append=/etc/qci/answers returned 1 instead of one of [0]"
"[ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue, fix and redeploy"
^ NOTE: Above is shown as Level "notice" although it is clearly an "err"

Logfile excerpts:
Cannot initialize minidnf
Checking configuration status...
multipath requires configuration
libvirt is not configured for vdsm yet
FAILED: conflicting vdsm and libvirt-qemu tls configuration
unable to write 'random state' (x 2)
2016-07-20 12:39:44 DEBUG otopi.plugins.otopi.network.hostname plugin.execute:941 execute-output: ('/sbin/ip', 'addr', 'show') stderr:


2016-07-20 12:39:44 DEBUG otopi.plugins.otopi.network.hostname hostname._validation:113 my addresses: [<< 3 192.168.252.* entries>>]
2016-07-20 12:39:44 DEBUG otopi.plugins.otopi.network.hostname hostname._validation:114 local addresses: [<<ipv4_address>>, <<ipv6_address>]
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.otopi.network.iptables.Plugin._validate
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.otopi.network.ssh.Plugin._validation
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.ovirt_hosted_engine_setup.engine.add_host.Plugin._validation
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.ovirt_hosted_engine_setup.network.bridge.Plugin._get_hostname_additional_hosts
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:148 condition False
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.ovirt_hosted_engine_setup.network.bridge.Plugin._get_hostname_from_bridge_if
2016-07-20 12:39:44 DEBUG otopi.context context._executeMethod:156 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/network/bridge.py", line 271, in _get_hostname_from_bridge_if
    ohostedcons.NetworkEnv.BRIDGE_IF
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/vds_info.py", line 89, in network
    'port' % device
RuntimeError: The selected device eth0 is not a supported bridge port

[NOTE: actual devices are consistent names]

Comment 1 John Matthews 2016-07-25 18:31:09 UTC
Please re-test with latest compose.

Comment 2 Thom Carlin 2016-08-01 11:00:09 UTC
Failed with QCI-1.0-RHEL-7-20160729.t.0

Child task shows:
====== Puppet run for host <<rhv_host_fqdn>> status reported as Error ======

Comment 3 Thom Carlin 2016-08-01 11:06:07 UTC
More logfile excerpts:

2016-08-01 10:37:35 DEBUG otopi.plugins.otopi.packagers.dnfpackager dnfpackager._boot:178 Cannot initialize minidnf
Traceback (most recent call last):
  File "/usr/share/otopi/plugins/otopi/packagers/dnfpackager.py", line 165, in _boot
    constants.PackEnv.DNF_DISABLED_PLUGINS
  File "/usr/share/otopi/plugins/otopi/packagers/dnfpackager.py", line 75, in _getMiniDNF
    from otopi import minidnf
  File "/usr/lib/python2.7/site-packages/otopi/minidnf.py", line 31, in <module>
    import dnf
ImportError: No module named dnf

2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage validation METHOD otopi.plugins.ovirt_hosted_engine_setup.network.bridge.Plugin._get_hostname_from_bridge_if
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:156 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/network/bridge.py", line 271, in _get_hostname_from_bridge_if
    ohostedcons.NetworkEnv.BRIDGE_IF
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/vds_info.py", line 89, in network
    'port' % device
RuntimeError: The selected device eth0 is not a supported bridge port
2016-08-01 10:38:01 ERROR otopi.context context._executeMethod:165 Failed to execute stage 'Setup validation': The selected device eth0 is not a supported bridge port
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage pre-terminate METHOD otopi.plugins.otopi.dialog.cli.Plugin._pre_terminate
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:148 condition False
2016-08-01 10:38:01 INFO otopi.context context.runSequence:427 Stage: Termination
2016-08-01 10:38:01 DEBUG otopi.context context.runSequence:431 STAGE terminate
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage terminate METHOD otopi.plugins.ovirt_hosted_engine_setup.core.misc.Plugin._terminate
2016-08-01 10:38:01 ERROR otopi.plugins.ovirt_hosted_engine_setup.core.misc misc._terminate:170 Hosted Engine deployment failed: this system is not reliable, please check the issue, fix and redeploy
2016-08-01 10:38:01 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND                 Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-201608
01103735-j2z4zj.log
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage terminate METHOD otopi.plugins.otopi.dialog.human.Plugin._terminate
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage terminate METHOD otopi.plugins.otopi.dialog.machine.Plugin._terminate
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:148 condition False
2016-08-01 10:38:01 DEBUG otopi.context context._executeMethod:142 Stage terminate METHOD otopi.plugins.otopi.core.log.Plugin._terminate

Comment 4 Thom Carlin 2016-08-01 11:07:31 UTC
FailedQA, Returning to ASSIGNED

Sosreport attached

Comment 6 Thom Carlin 2016-08-01 11:51:11 UTC
otopi is installed

ll -Z /usr/lib/python2.7/site-packages/otopi/minidnf.py
-rw-r--r--. root root system_u:object_r:lib_t:s0       /usr/lib/python2.7/site-packages/otopi/minidnf.py

Comment 7 Thom Carlin 2016-08-01 12:48:24 UTC
Workaround:
* Edit /etc/qci/answers
  * Change OVEHOSTED_NETWORK/bridgeIf=str:eth0 to 
OVEHOSTED_NETWORK/bridgeIf=str:<<consistent device name>>

Note that the consistent device name should be the configured [provisioning] NIC with an existing IP address

Comment 12 Thom Carlin 2016-08-03 11:10:05 UTC
Manually applying a fix resulted in:
Error: Could not back up /etc/qci/answers: Got passed new contents for sum {md5}09524f12687bcd5fc15b90b34337e717
Error: Could not back up /etc/qci/answers: Got passed new contents for sum {md5}09524f12687bcd5fc15b90b34337e717
Error: /Stage[main]/Ovirt::Self_hosted::Setup/File[/etc/qci/answers]/content: change from {md5}09524f12687bcd5fc15b90b34337e717 to {md5}505f5ae10c488dcba39cf188d5000584 failed: Could not back up /etc/qci/answers: Got passed new contents for sum {md5}09524f12687bcd5fc15b90b34337e717

Comment 13 Fabian von Feilitzsch 2016-08-04 20:37:43 UTC
https://github.com/fusor/fusor/pull/1124

Comment 14 John Matthews 2016-08-05 17:19:24 UTC
Will be in compose 8/5

Comment 15 John Matthews 2016-08-10 15:46:40 UTC
Moving back to ASSIGNED as John Kim saw issues with this on compose from 8/9.

Comment 16 Fabian von Feilitzsch 2016-08-15 14:32:27 UTC
Retry the deployment with "Ignore Puppet facts for provisioning" set to true in the Satellite settings. That might prevent the NIC identifier change we are seeing.

Comment 17 Thom Carlin 2016-08-15 14:43:31 UTC
Per devs: try
On Sat 6 web UI: 
Administer>Settings
Provisioning tab
Set Ignore Puppet facts for provisioning true

Comment 18 Thom Carlin 2016-08-29 03:33:39 UTC
Verified in QCI-1.0-RHEL-7-201608125.t.0 without changing settings.

Please reopen if this issue reoccurs

Comment 20 errata-xmlrpc 2016-09-13 16:32:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2016:1862


Note You need to log in before you can comment on or make changes to this bug.