Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1354685 - nodeinfo and siblings incorrectly reported in capabilities
Summary: nodeinfo and siblings incorrectly reported in capabilities
Keywords:
Status: ASSIGNED
Alias: None
Product: Virtualization Tools
Classification: Community
Component: libvirt
Version: unspecified
Hardware: ppc64le
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Andrea Bolognani
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-11 23:04 UTC by Rafael Folco
Modified: 2018-07-18 14:57 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Rafael Folco 2016-07-11 23:04:54 UTC
Description of problem:
Libvirt reports incomplete NUMA topology from getCapabilities(), which prevents applications that consume the topology info to effectively use NUMA with the cpu threads. Such applications, as OpenStack, relies in the provided NUMA topology to generate Libvirt XMLs when spawning instances. This prevents instances to spawn when number of requested vcpus is higher than the number of cores in a NUMA node: e.g.: requesting 8 vcpus (2 core, 4 threads) in one single NUMA node with 5 cpu cores (cannot fit 8 vcpus in 5 cpu node).

There is a initial proposal discussed here:
http://www.redhat.com/archives/libvir-list/2016-June/msg00687.html
 

Version-Release number of selected component (if applicable):


How reproducible:
virsh capabilities 

Steps to Reproduce:
Spawn an instance with NUMA flavor in OpenStack (which relies on getCapabilities)
 

Actual results:
5 cpus per node (online cpu cores exposed to the host)
        <cell id='17'>
 ...
          <cpus num='5'>
            <cpu id='80' socket_id='17' core_id='2208' siblings='80'/>
            <cpu id='88' socket_id='17' core_id='2224' siblings='88'/>
            <cpu id='96' socket_id='17' core_id='2272' siblings='96'/>
            <cpu id='104' socket_id='17' core_id='2280' siblings='104'/>
            <cpu id='112' socket_id='17' core_id='2288' siblings='112'/>
          </cpus>
...

Expected results:
FULL NUMA topology including CPU threads (not exposed to the host), including siblings info.
        <cell id='17'>
 ...
          <cpus num='40'>
            <cpu id='80' socket_id='17' core_id='2208' siblings='80'/>
...
            <cpu id='119' socket_id='17' core_id='xxxx' siblings='119'/>
          </cpus>
...



Additional info:

Comment 1 Rafael Folco 2016-07-19 13:56:11 UTC
n reply to Rafael Folco from comment #0)
> Description of problem:
> Libvirt reports incomplete NUMA topology from getCapabilities(), which
> prevents applications that consume the topology info to effectively use NUMA
> with the cpu threads. Such applications, as OpenStack, relies in the
> provided NUMA topology to generate Libvirt XMLs when spawning instances.
> This prevents instances to spawn when number of requested vcpus is higher
> than the number of cores in a NUMA node: e.g.: requesting 8 vcpus (2 core, 4
> threads) in one single NUMA node with 5 cpu cores (cannot fit 8 vcpus in 5
> cpu node).
> 
> There is a initial proposal discussed here:
> http://www.redhat.com/archives/libvir-list/2016-June/msg00687.html
> 
> Steps to Reproduce:
> Spawn an instance with NUMA flavor in OpenStack (which relies on
> getCapabilities)
> 
> Actual results:
> 5 cpus per node (online cpu cores exposed to the host)
>         <cell id='17'>
>  ...
>           <cpus num='5'>
>             <cpu id='80' socket_id='17' core_id='2208' siblings='80'/>
>             <cpu id='88' socket_id='17' core_id='2224' siblings='88'/>
>             <cpu id='96' socket_id='17' core_id='2272' siblings='96'/>
>             <cpu id='104' socket_id='17' core_id='2280' siblings='104'/>
>             <cpu id='112' socket_id='17' core_id='2288' siblings='112'/>
>           </cpus>
> ...
> 
> Expected results:
> FULL NUMA topology including CPU threads (not exposed to the host),
> including siblings info.
>         <cell id='17'>
>  ...
>           <cpus num='40'>
>             <cpu id='80' socket_id='17' core_id='2208' siblings='80'/>
> ...
>             <cpu id='119' socket_id='17' core_id='xxxx' siblings='119'/>
>           </cpus>
> ...

The actual proposal is

  https://www.redhat.com/archives/libvir-list/2016-May/msg00443.html

and the resulting topology information would actually end
up looking like

  <cell id='17'>
    ...
    <cpus num='5'>
      <cpu id='80' thread_capacity='8'
           socket_id='17' core_id='80' siblings='80'/>
      <cpu id='88' thread_capacity='8'
           socket_id='17' core_id='88' siblings='88'/>
      <cpu id='96' thread_capacity='8'
           socket_id='17' core_id='96' siblings='96'/>
      <cpu id='104' thread_capacity='8'
           socket_id='17' core_id='104' siblings='104'/>
      <cpu id='112' thread_capacity='8'
           socket_id='17' core_id='112' siblings='112'/>
    </cpus>
    ...

The reason we can't just list all CPUs, even thought we could
infer all required information by looking at the host, is
that there is an existing expectation that you will be able to
manage pCPUs listed there in a certain way, eg. by pinning
vCPUs to them; however, you can't pin vCPUs to secondary host
threads, as that would result in an error.

Would the proposed augmented output still satisfy your use
case? Please note that I've glossed over some details here, but
the mailing list message linked above contains more information.


Andrea's response ^^^

Comment 2 Rafael Folco 2016-07-19 14:10:30 UTC
Yes, I kind of expected that. This still implies in a special logic for handling NUMA topology consumed from getCapabilities(). Adding thread_capacity='1' for non-Power architectures helps the code to be more generic, though.
Thanks for working on this bug.

Comment 3 Daniel Berrange 2016-07-19 14:38:17 UTC
Responding to that upstream thread - the info in the node capabilities shouldn't  be restricted based on QEMU usage constraints. The node capabilities should always report the full set of sockets/cores/threads which physically exist. It could be augmented with state info to show which threads are online in the host. QEMU specific constraints meanwhile should be dealt with in the domain capabilities XML. https://www.redhat.com/archives/libvir-list/2016-July/msg00740.html


Note You need to log in before you can comment on or make changes to this bug.