Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1062987 - error message when live migration fails through GUI different from more verbose nova-client error message
Summary: error message when live migration fails through GUI different from more verbo...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: RDO
Classification: Community
Component: python-django-horizon
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: ---
Assignee: RHOS Maint
QA Contact: Ami Jeain
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-09 12:45 UTC by Ami Jeain
Modified: 2014-02-11 20:36 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-02-11 20:36:47 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1278066 None None None Never

Description Ami Jeain 2014-02-09 12:45:10 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. install a controller+2 computes, so you can live migrate from one server to another.
2. choose one instance and select live-migrate from the GUI, so the other compute
3.

Actual results:
you get the following in /var/log/horizon/horizon.log:
2014-02-09 10:06:18,753 20563 WARNING horizon.exceptions Recoverable error: Live migration of instance 0e2f047c-4fcc-4ba0-a9b4-79b6b755a357 to host server2 failed (HTTP 400) (Request-ID: req-1722e387-ab40-4228-b1c5-27164f0b45d5)



Expected results:

live migrate works 
Additional info:

some printouts from /var/log/nova/compute.log:
XML error: CPU feature `pdpe1gb' specified more than once

Refer to http://libvirt.org/html/libvirt-libvirt.html#virCPUCompareResult
2014-02-09 12:05:14.044 19968 DEBUG nova.openstack.common.rpc.amqp [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] Sending check_can_live_migrate_destination on notifications.error notify /usr/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py:621
2014-02-09 12:05:14.045 19968 DEBUG nova.openstack.common.rpc.amqp [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] UNIQUE_ID is 77379f740e58479aa977bed007535753. _add_unique_id /usr/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py:341
2014-02-09 12:05:14.071 19968 DEBUG nova.ceilometer.notifier [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] ignoring check_can_live_migrate_destination notify /usr/lib/python2.7/site-packages/ceilometer/compute/nova_notifier.py:143
2014-02-09 12:05:14.072 19968 ERROR nova.openstack.common.rpc.amqp [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] Exception during message handling
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     **args)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 90, in wrapped
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     payload)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 73, in wrapped
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4083, in check_can_live_migrate_destination
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     block_migration, disk_over_commit)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3968, in check_can_live_migrate_destination
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     self._compare_cpu(source_cpu_info)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4110, in _compare_cpu
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     LOG.error(m, {'ret': ret, 'u': u})
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4106, in _compare_cpu
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     ret = self._conn.compareCPU(cpu.to_xml(), 0)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     rv = execute(f,*args,**kwargs)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     rv = meth(*args,**kwargs)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2889, in compareCPU
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp     if ret == -1: raise libvirtError ('virConnectCompareCPU() failed', conn=self)
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp libvirtError: XML error: CPU feature `pdpe1gb' specified more than once
2014-02-09 12:05:14.072 19968 TRACE nova.openstack.common.rpc.amqp
2014-02-09 12:05:14.076 19968 ERROR nova.openstack.common.rpc.common [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] Returning exception XML error: CPU feature `pdpe1gb' specified more than once to caller
2014-02-09 12:05:14.076 19968 ERROR nova.openstack.common.rpc.common [req-1722e387-ab40-4228-b1c5-27164f0b45d5 55ebc145eb954c6f933f588407af2753 e433ae32fee7468fad9e80d96f4e92a0] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data\n    **args)\n', '  File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch\n    result = getattr(proxyobj, method)(ctxt, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/nova/exception.py", line 90, in wrapped\n    payload)\n', '  File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python2.7/site-packages/nova/exception.py", line 73, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4083, in check_can_live_migrate_destination\n    block_migration, disk_over_commit)\n', '  File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3968, in check_can_live_migrate_destination\n    self._compare_cpu(source_cpu_info)\n', '  File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4110, in _compare_cpu\n    LOG.error(m, {\'ret\': ret, \'u\': u})\n', '  File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4106, in _compare_cpu\n    ret = self._conn.compareCPU(cpu.to_xml(), 0)\n', '  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit\n    result = proxy_call(self._autowrap, f, *args, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call\n    rv = execute(f,*args,**kwargs)\n', '  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker\n    rv = meth(*args,**kwargs)\n', '  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2889, in compareCPU\n    if ret == -1: raise libvirtError (\'virConnectCompareCPU() failed\', conn=self)\n', "libvirtError: XML error: CPU feature `pdpe1gb' specified more than once\n"]

Comment 1 Ami Jeain 2014-02-09 14:34:06 UTC
# rpm -qa |grep horizon
python-django-horizon-2014.1-0.2b2.fc21.noarch

Comment 5 Matthias Runge 2014-02-11 07:32:01 UTC
Yes, the situation with qpid is just unfortunate. During my tests, I had the feeling, due qpid issues barely everything gets out of sync and I was not able to recover other than reinstalling including dropping the database.
The main issue is: how to reproduce that? (I can't)

Comment 6 Matthias Runge 2014-02-11 20:36:47 UTC
So, I understand, that the issue is the following:

The error message given in horizon differs from what is given via nova api, which is in this case more helpful.

This targets to a more general pattern in Horizon. No matter what exception is returned from a client, horizon catches that and sends an own (probably different) message to the user.

This has mainly two reasons:
- in some exception messages internal details of the cloud infrastructure is/will be disclosed. The returned messages are out of control from horizon
- messages directly returned from underlying services can not be translated.

A related blueprint is this one: https://blueprints.launchpad.net/horizon/+spec/improve-error-message-details-for-usability

forwarding error messages to the user has been on the openstack-dev mailing list and it's not wanted by design: http://lists.openstack.org/pipermail/openstack/2014-January/004651.html 

In this special case, an exception needs to be added to nova, to flag this error as: Live migrate only possible, when shared storage is configured.


Note You need to log in before you can comment on or make changes to this bug.