Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 591609 - Migrating guests in RHEVM generates exception traceback emails from Satellite
Summary: Migrating guests in RHEVM generates exception traceback emails from Satellite
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Virtualization
Version: 530
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Milan Zázrivec
QA Contact: Dimitar Yordanov
URL:
Whiteboard:
Depends On:
Blocks: sat531-triage
TreeView+ depends on / blocked
 
Reported: 2010-05-12 17:12 UTC by Martin Osvald 🛹
Modified: 2018-10-27 13:39 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, after migrating a Xen guest domain to KVM using Red Hat Enterprise Virtualization Manager, the following error may have occurred: While running 'registration.virt_notify': caught exceptions.KeyError : 'Unknown' This was caused by the use of invalid machine state, "Unknown". With this update, the incorrect state has been changed to "nostate".
Clone Of:
Environment:
Last Closed: 2010-10-11 14:34:50 UTC
Target Upstream Version:


Attachments (Terms of Use)
email with exception (deleted)
2010-05-12 17:12 UTC, Martin Osvald 🛹
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2010:0761 normal SHIPPED_LIVE rhncfg and rhn-virtualization bug fix update 2010-10-11 14:34:35 UTC

Description Martin Osvald 🛹 2010-05-12 17:12:40 UTC
Created attachment 413493 [details]
email with exception

Description of problem:

If vdsmd doesn't return state of the machine in domain info, poller.py sends "Unknown" machine state to Satellite which causes exception, because Satellite doesn't know about "Unknown" state.


Version-Release number of selected component (if applicable):

Satellite 5.3 - spacewalk-backend-0.5.28-54
rhn-virtualization-host-5.3.0-17


How reproducible:

always


Steps to Reproduce:

1. Migrate xen to kvm guest through RHEVM (vdsmd has to be enabled at host)

  
Actual results:

see attached email (sensitive data removed)


Expected results:

no exception emails should be sent by Satellite


Additional info:

related client side source code (related code is concatenated with '<<<---'):

===< snip from client/tools/rhn-virtualization/virtualization/poller.py ===
def poll_through_vdsm():
   """
    This method polls all the virt guests running on a VDSM enabled Host.
    Libvirt is disabled by default on RHEV-M managed clients.
    * Imports the localvdsm client that talks to the localhost
      and fetches the list of vms and their info.
    * Extract the data and construct the state to pass it to the 
      execution plan for guest polling.
    * The server should account for business rules similar to
      xen/kvm.
   """
   import localvdsm
   try:
       server = localvdsm.connect()
   except:
       # VDSM raised an exception we're done here
       return {}
   # Extract list of vm's. True returns full list
   try:
       domains = server.list(True)
   except:
       # Something went wrong in vdsm, exit
       return {}

   if not len(domains['vmList']):
       # No domains, exit.
       return

   state = {}
   for domain in domains['vmList']:
       #trim uuid
       uuid = domain['vmId'].lower().replace('-', '')
       # Map the VDSM status to libvirt for server compatibility
       status = "Unknown"          <<<---
       if VIRT_VDSM_STATUS_MAP.has_key(domain['status']):  <<<---
           status = VIRT_VDSM_STATUS_MAP[domain['status']]
       # This is gonna be fully virt as its managed by VDSM
       virt_type = VirtualizationType.FULLY

       #Memory
       memory = int(domain['memSize']) * 1024

       # vcpus
       if domain.has_key('smp'):
           vcpus = domain['smp']
       else:
           vcpus = '1'

       properties = {
           PropertyType.NAME   : domain['vmName'],
           PropertyType.UUID   : uuid,
           PropertyType.TYPE   : virt_type,
           PropertyType.MEMORY : memory, # current memory
           PropertyType.VCPUS  : vcpus,
           PropertyType.STATE  : status}

       state[uuid] = properties

   if state: _log_debug("Polled state: %s" % repr(state))

   return state
=== snip >===

There is also probably a bug in routine which migrate host from xen to kvm with not updating domain status correctly through vdsm during migration, but the fact that the above code sends/could send unknown state for Satellite is a bug. It shouldn't send such state or should send one of the known states:

NOSTATE     = 'nostate'
RUNNING     = 'running'
BLOCKED     = 'blocked'
PAUSED      = 'paused'
SHUTDOWN    = 'shutdown'
SHUTOFF     = 'shutoff'
CRASHED     = 'crashed'

or there should be implemented another state named "Unknown".

Traceback from Satellite side (see attached email for complete info):

Exception Handler Information
Traceback (most recent call last):
  File "/usr/share/rhn/server/apacheRequest.py", line 120, in call_function
    response = apply(func, params)
  File "/usr/share/rhn/server/handlers/xmlrpc/registration.py", line 957, in virt_notify
    rhnVirtualization._virt_notify(server_id, actions)
  File "/usr/share/rhn/server/rhnVirtualization.py", line 844, in _virt_notify
    handler.handle(server_id, action)
  File "/usr/share/rhn/server/rhnVirtualization.py", line 214, in handle
    self.__convert_properties(properties)
  File "/usr/share/rhn/server/rhnVirtualization.py", line 773, in __convert_properties
    properties[PropertyType.STATE] = CLIENT_SERVER_STATE_MAP[state]
KeyError: 'Unknown'

Comment 3 Matthew Booth 2010-06-03 10:10:21 UTC
There's a vocab nit which becomes important in this kind of discussion: we talk about 'converting' a guest from Xen to KVM, and 'migrating' a guest from hypervisor node A to hypervisor node B.

If I'm reading this correctly, a Xen guest has been converted to run on RHEV. It subsequently can't be migrated. Is this correct? How was it converted?

Comment 5 Alan Pevec 2010-06-07 15:55:26 UTC
Do you have a VM in this state? If yes, please run on the Hypervisor: vdsClient -s 0 list

Comment 8 Milan Zázrivec 2010-07-15 16:18:53 UTC
spacewalk.git master:

commit a4be5dcb417d1ad218c205f5cae6adc278f76bf6
Author: Milan Zazrivec <mzazrivec@redhat.com>
Date:   Thu Jul 15 18:14:35 2010 +0200

    591609 - 'Unknown' is not a valid virt. guest state
    
    'nostate' is the valid fallback state value

Comment 9 Milan Zázrivec 2010-07-15 16:21:57 UTC
satellite.git SATELLITE-5.3: 192a1ff8f74dab587d3fc0d21cd8b363d6dbe9c3

Comment 12 Dimitar Yordanov 2010-09-01 08:55:06 UTC
Hi Martin,

 I have just checked that the patch is included in  package rhn-virtualization-5.3.0-18.el5sat.src.rpm

+- 591609 - 'Unknown' is not a valid virt. guest state
-        status = "Unknown"
+        status = 'nostate'

 Unfortunately I faced a problem. I have no access to RHEV-M.
 I would like to ask you to if you can install  rhn-virtualization-common-5.3.0-18.el5sat.noarch.rpm  and  rhn-virtualization-common-5.3.0-18.el5sat.noarch.rpm  on the target machine and let us know if the issue is fixed correctly.
 Thank you in advance.

Regards
Dimitar Yordanov

Comment 13 Jaromir Hradilek 2010-09-03 15:53:15 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, after migrating a Xen guest domain to KVM using Red Hat Enterprise Virtualization Manager (RHEV-M), the following error may have occurred:

  While running 'registration.virt_notify': caught exceptions.KeyError : 'Unknown'

This was caused by the use of invalid machine state, "Unknown". With this update, the incorrect state has been changed to "nostate".

Comment 14 Jaromir Hradilek 2010-09-06 08:20:26 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,4 +1,4 @@
-Previously, after migrating a Xen guest domain to KVM using Red Hat Enterprise Virtualization Manager (RHEV-M), the following error may have occurred:
+Previously, after migrating a Xen guest domain to KVM using Red Hat Enterprise Virtualization Manager, the following error may have occurred:
 
   While running 'registration.virt_notify': caught exceptions.KeyError : 'Unknown'

Comment 21 errata-xmlrpc 2010-10-11 14:34:50 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0761.html


Note You need to log in before you can comment on or make changes to this bug.