Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1134873 - Report error as 'Hosted Engine HA service are already running on this system' when first configure hosted engine
Summary: Report error as 'Hosted Engine HA service are already running on this system'...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-setup
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 3.5.0
Assignee: Simone Tiraboschi
QA Contact: Artyom
URL:
Whiteboard: integration
Depends On:
Blocks: rhev3.5beta 1156165
TreeView+ depends on / blocked
 
Reported: 2014-08-28 11:46 UTC by wanghui
Modified: 2016-07-04 00:39 UTC (History)
19 users (show)

Fixed In Version: ovirt-hosted-engine-setup-1.2.0-0.3.beta.el7
Doc Type: Bug Fix
Doc Text:
Previously, hosted engine high availability services could run on the host even if they are unconfigured, which would prevent hosted engine deployment. Now, these services are stopped if they are running unconfigured and they do not interfere with hosted engine deployment.
Clone Of:
Environment:
Last Closed: 2015-02-11 20:40:46 UTC
oVirt Team: ---
Target Upstream Version:


Attachments (Terms of Use)
log infos and screen shot of error (deleted)
2014-08-28 11:46 UTC, wanghui
no flags Details
setup log (deleted)
2014-10-05 13:08 UTC, Artyom
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0161 normal SHIPPED_LIVE ovirt-hosted-engine-setup bug fix and enhancement update 2015-12-07 21:35:11 UTC
oVirt gerrit 32396 master MERGED packaging: setup: Stop ha services if they are running unconfigured Never
oVirt gerrit 32401 ovirt-hosted-engine-setup-1.2 MERGED packaging: setup: Stop ha services if they are running unconfigured Never
Red Hat Bugzilla 1001181 None None None Never

Internal Links: 1001181

Description wanghui 2014-08-28 11:46:10 UTC
Created attachment 931891 [details]
log infos and screen shot of error

Description of problem:
It reports error as follows when first configure hosted-engine in host.
===============================================================================
[ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services.

Version-Release number of selected component (if applicable):
rhev-hypervisor7-7.0-20140827.0.iso
ovirt-node-3.1.0-0.9.20140815gitee21d9b.el7.noarch
ovirt-node-plugin-hosted-engine-0.1.0-0.0.x86_64
ovirt-host-deploy-1.3.0-0.0.1.master.el7.noarch
ovirt-hosted-engine-setup-1.2.0-0.2.master.el7.noarch
ovirt-hosted-engine-ha-1.2.1-0.3.master.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. Install rhev-hypervisor7-7.0-20140827.0.iso on the host
2. Enabled network
3. ps -aux |grep ovirt-ha
4. Configure hosted-engine in shell
   #ovirt-hosted-engine-setup

Actual results:
1. After step3, the output is as follows.
   #ps -aux |grep ovirt-ha
   vdsm      1356  0.0  0.1 136204 12200 ?        S    03:56   0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
2. After step4, it will report error as follows.
   [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services.

Expected results:
It should run setup hosted-engine succeed.

Additional info:

Comment 2 Jiri Moskovcak 2014-08-28 13:05:36 UTC
The broker is running, so the code works as expected, do you have any idea what started it? Is this a clean system or it was used before to deploy the HE?

Comment 4 Ying Cui 2014-08-29 05:06:17 UTC
Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug.

Comment 5 Jiri Moskovcak 2014-08-29 07:33:00 UTC
> In reply to Ying Cui from comment #4)
> Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug.

Good catch! ovirt-node-plugin-hosted-engine enables it so it's started on boot, this should be handled by the hosted engine setup, so it need to be removed from the node spec. over to ovirt-node-plugin-hosted-engine

Comment 6 Sandro Bonazzola 2014-09-01 07:11:39 UTC
(In reply to Jiri Moskovcak from comment #5)
> > In reply to Ying Cui from comment #4)
> > Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug.
> 
> Good catch! ovirt-node-plugin-hosted-engine enables it so it's started on
> boot, this should be handled by the hosted engine setup, so it need to be
> removed from the node spec. over to ovirt-node-plugin-hosted-engine

+1
good catch!

Comment 7 Fabian Deutsch 2014-09-02 08:07:51 UTC
It is not possible to enable a service on Node at runtime.
That means that we normally enable all required services at build time - this is the reason why the HA service is already running.

IIUIC the engine setup tries to enable the HA service during the setup, is that correct?
If so, then this will fail, because enabling the service will not be persisted.

Is there a way to workaroundt his from the hosted-engine side?
Otherwise we could add a service which is enabled by default, and starts the HA service conditionally.

Comment 8 Douglas Schilling Landgraf 2014-09-02 18:16:38 UTC
Hi wanghui,

Can you please attach your /var/log/messages?

From my test with rhev-hypervisor7-7.0-20140827.0.iso

# ovirt-hosted-engine-setup 
[ INFO  ] Stage: Initializing
          Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards.
          Are you sure you want to continue? (Yes, No)[Yes]: Yes
          It has been detected that this program is executed through an SSH connection without using screen.
          Continuing with the installation may lead to broken installation if the network connection fails.
          It is highly recommended to abort the installation and run it inside a screen session using command "screen".
          Do you want to continue anyway? (Yes, No)[No]: Yes
[ INFO  ] Generating a temporary VNC password.
[ INFO  ] Stage: Environment setup
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20140902181447-x9whv9.log
          Version: otopi-1.3.0_master (otopi-1.3.0-0.0.master.el7)
[ INFO  ] Hardware supports virtualization
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ ERROR ] Failed to execute stage 'Environment setup': Command '/bin/systemctl' failed to execute
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'
[ INFO  ] Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination


Sep  2 17:50:13 localhost systemd: Starting Virtual Desktop Server Manager...
Sep  2 17:50:13 localhost systemd: vdsmd.service: control process exited, code=exited status=1
Sep  2 17:50:13 localhost systemd: Failed to start Virtual Desktop Server Manager.
Sep  2 17:50:13 localhost systemd: Unit vdsmd.service entered failed state.
Sep  2 17:50:13 localhost vdsmd_init_common.sh: vdsm: stopped during execute check_is_configured task (task returned with error code 1).
Sep  2 17:50:13 localhost vdsm-tool: module configfile could not load to vdsm-tool: Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 81, in load_modules
    mod_absp, mod_desc)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/configfile.py", line 30, in <module>
  File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas
  File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen
ImportError: Unable to import libpython2.7!
Sep  2 17:50:13 localhost vdsm-tool: module configurator could not load to vdsm-tool: Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 81, in load_modules
    mod_absp, mod_desc)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/configurator.py", line 31, in <module>
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/configfile.py", line 30, in <module>
  File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas
  File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen
ImportError: Unable to import libpython2.7!
Sep  2 17:50:13 localhost vdsm-tool: module upgrade_300_networks could not load to vdsm-tool: Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 81, in load_modules
    mod_absp, mod_desc)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/upgrade_300_networks.py", line 30, in <module>
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 41, in <module>
  File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module>
  File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas
  File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen
ImportError: Unable to import libpython2.7!
Sep  2 17:50:13 localhost vdsmd_init_common.sh: vdsm: Running mkdirs

Comment 9 Douglas Schilling Landgraf 2014-09-02 21:23:53 UTC
Ok, I have reproduced the report generating an ISO manually.

[ INFO  ] Generating a temporary VNC password.
[ INFO  ] Stage: Environment setup
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20140902212223-otzfhd.log
          Version: otopi-1.3.0_master (otopi-1.3.0-0.0.master.el7)
[ INFO  ] Hardware supports virtualization
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ ERROR ] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services.
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'
[ INFO  ] Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination


# ps auxww | grep ovirt-ha
vdsm      1489  0.0  0.4 132584 12396 ?        S    21:14   0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broke

Comment 10 Douglas Schilling Landgraf 2014-09-02 22:55:50 UTC
Hi Sandro,

Based on comment#7, can we add additional validation to such verification [1], for example: 

If ovirt-ha is running and there is no /etc/ovirt-hosted-engine/vm.conf can we continue the setup? or even similar validation in other .conf file?

I do believe, this kind of issue (enabling/starting the service) before the setup could be happened in a non ovirt-node environment as well.

Please let me know your thoughts, thanks!

[1] https://github.com/oVirt/ovirt-hosted-engine-setup/blob/master/src/plugins/ovirt-hosted-engine-setup/ha/ha_services.py#L62

Comment 11 Sandro Bonazzola 2014-09-03 13:09:33 UTC
I guess we can add a check in setup and automatically stop the ha services if we find them running but not configured instead of telling the user to stop them manually.

Comment 12 Douglas Schilling Landgraf 2014-09-03 13:24:29 UTC
(In reply to Sandro Bonazzola from comment #11)
> I guess we can add a check in setup and automatically stop the ha services
> if we find them running but not configured instead of telling the user to
> stop them manually.

Thanks a lot Sandro, that should help, I am going to verify!

Comment 13 wanghui 2014-09-05 04:35:17 UTC
Test version:
rhev-hypervisor7-7.0-20140904.0.iso
ovirt-node-3.1.0-0.10.20140904gitb828c37.el7.noarch
ovirt-hosted-engine-setup-1.2.0-0.2.master.el7.noarch
ovirt-hosted-engine-ha-1.2.1-0.3.master.el7.noarch
ovirt-node-plugin-hosted-engine-0.1.0-0.0.x86_64
ovirt-host-deploy-1.3.0-0.0.1.master.el7.noarch

Test steps:
1. Clean install rhev-hypervisor7-7.0-20140904.0.iso
2. Configure network
3. #ps -aux |grep ovirt-ha
4. #ovirt-hosted-engine-setup

Test result:
1. After step3, it still has Hosted Engine HA service running.
# ps -aux |grep ovirt-ha
vdsm      1377  0.0  0.1 136204 12188 ?        S    03:18   0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
2. After step4, it still report error as follows.
[ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services.

So this issue is not fixed in rhev-hypervisor7-7.0-20140904.0.iso. Change the status to assign.

Comment 14 Ying Cui 2014-09-05 04:58:18 UTC
Change this bug back to MODIFIED, because there is no updated ovirt-hosted-engine-setup package in downstream in build rhev-hypervisor7-7.0-20140904.0.iso.

Here needinfo Sandro, when are you going to rebuild downstream packages for QE testing? then RHEVH will pick them into.

Comment 21 Artyom 2014-10-01 09:27:04 UTC
Verified ovirt-hosted-engine-setup-1.2.0-1.el7.noarch
Finish deploy process and have running hosted-engine environment.

Comment 22 Artyom 2014-10-05 13:06:02 UTC
Ok installation was done on clean host, if I try to redeploy it failed with the above message(reboot) not help, after erase of all packages include vdsm and restart host, hosted-engine --deploy works again.

Comment 23 Artyom 2014-10-05 13:08:55 UTC
Created attachment 944040 [details]
setup log

Comment 24 Simone Tiraboschi 2014-10-24 09:38:06 UTC
I'm not sure it should fail the QA: preventing to redeploy on an host that has already been deployed it's just the aim of that check.

So it should work and a clean system while it should fail on an host witch has already been deployed with this error:
   [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services.

The problem is another: there is not an easy way to completely cleanup a deployed system and you have to manually remove packages to deconfigure them.
We have another RFE for that:
  please see https://bugzilla.redhat.com/1001181
till that we have to manually cleanup a deployed system before we can deploy it a second time.

Comment 25 Artyom 2014-10-26 11:13:51 UTC
Verified on ovirt-hosted-engine-setup-1.2.0-1.el7.noarch

Comment 27 errata-xmlrpc 2015-02-11 20:40:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0161.html


Note You need to log in before you can comment on or make changes to this bug.