Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1689202 - RFE: report limit on number of SEV guests in query-sev-capabilities QMP command
Summary: RFE: report limit on number of SEV guests in query-sev-capabilities QMP command
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: Laszlo Ersek
QA Contact: Guo, Zhiyi
URL:
Whiteboard:
Depends On:
Blocks: 1689195
TreeView+ depends on / blocked
 
Reported: 2019-03-15 12:09 UTC by Daniel Berrange
Modified: 2019-04-12 07:59 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)

Description Daniel Berrange 2019-03-15 12:09:05 UTC
Description of problem:
It was said that when using SEV, there is a limit of 15 guests that are able to use the feature concurrently.

https://www.redhat.com/archives/libvir-list/2019-January/msg00652.html

This kind of limit is important for a mgmt application to know about, so that it can place VMs on hosts which have suitable resource available, but this is not yet exposed anywhere.

Libvirt thus wants to report it, and would like to obtain this info from QEMU, which can then presumable ask the kernel for it. This might in turn require a kernel RFE if the info isn't already available in some way.

Version-Release number of selected component (if applicable):
qemu-kvm-3.1.0-11.el8

How reproducible:
Always

Steps to Reproduce:
1. Connect to a qmp console and run "query-sev-capabilities" command
2.
3.

Actual results:
No information on the guest limit is reported

Expected results:
The guest limit is reported

Additional info:

Comment 1 Daniel Berrange 2019-03-15 16:05:27 UTC
Turns out the limit can be obtained from CPUID results so doesn't need kernel support:

  "We can query the limit through the CPUID Fn0x8000_001f[ECX]."

https://www.redhat.com/archives/libvir-list/2019-March/msg01086.html

Comment 2 Laszlo Ersek 2019-04-08 14:26:14 UTC
(In reply to Daniel Berrange from comment #1)
> Turns out the limit can be obtained from CPUID results so doesn't need
> kernel support:
>
>   "We can query the limit through the CPUID Fn0x8000_001f[ECX]."
>
> https://www.redhat.com/archives/libvir-list/2019-March/msg01086.html

sev_get_capabilities() already does

    host_cpuid(0x8000001F, 0, NULL, &ebx, NULL, NULL);
    cap->cbitpos = ebx & 0x3f;

for determining the "C-bit location in page table entry" (see
@SevCapability in "qapi/target.json").

Hopefully it's a "straightforward" [*] addition to the above
host_cpuid() function call -- retrieve ECX too, and propagate the
information in a new field of @SevCapability.

[*] famous last words

--*--

... According to "devel/qapi-code-gen.txt", section "Struct types":

> On output structures (only mentioned in the 'returns' side of a
> command), [...] [c]hanging from optional to mandatory is safe.

and introducing a new (optional or mandatory) field in an output-only
struct is a special case of the above (it's like an optional field that
was never actually produced before that we're now making mandatory).

--*--

Brijesh: is ECX *guaranteed* to contain the needed information, even
with the earliest shipped version of SEV?

... AMD publication #55766 (rev 3.07) "Secure Encrypted Virtualization
API Version 0.17" <https://developer.amd.com/sev/> states,

> 6.17 ACTIVATE
> 6.17.1 Actions
>
> [...]
>
> If the guest is SEV-ES enabled, then the ASID must be at least 1h and
> at most (MIN_SEV_ASID- 1). If the guest is not SEV-ES enabled, then
> the ASID must be at least MIN_SEV_ASID and at most the maximum SEV
> ASID available. The MIN_SEV_ASID value is discovered by CPUID
> Fn8000_001F[EDX]. The maximum SEV ASID available is discovered by
> CPUID Fn8000_001F[ECX].

Based on this, it looks like we can get the simultaneous max guest
*count* from (ECX-EDX+1) -- however, that only applies to non-SEV-ES
guests.

I wonder if both the SEV-ES and non-SEV-ES guest counts should be
exposed at once in the new @SevCapability fields. What's the right level
of abstraction here? Let QEMU compute the differences and only give
those differences (i.e. counts) to libvirtd, or let QEMU expose the
actual ASID limits to libvirtd, and let libvirtd calculate the counts?

Based on the other fields in @SevCapability, I'd say we're already
exposing pretty low-level / raw details (such as "cbitpos"), so maybe we
should propagate MIN_SEV_ASID and MAX_SEV_ASID to libvirtd transparently
too.

Comment 3 BRIJESH SINGH 2019-04-08 15:36:01 UTC
Fn8000_001F[ECX] will contain the maximum number of ASID's in current and future hardware. So it's safe to use this in kernel and Qemu to calculate the maximum number of SEV guest. I am thinking in SEV-ES patches we will extend the qemu capabilities to expose the following new fields:

SEV Capabilities {
...
...  
  sev-es = true (bool)
  sev-es-max-guest = n (maximum num of simultaneous SEV-ES guest) [EDX]
  sev-max-guest = n    (maxium number of simultaneous SEV Guest) [ECX-sev-es-max-guest+1]
...
...
}

Comment 4 Laszlo Ersek 2019-04-08 16:32:33 UTC
Hi Brijesh,

(In reply to BRIJESH SINGH from comment #3)

> I am thinking in SEV-ES patches we will
> extend the qemu capabilities to expose the following new fields:
> 
> SEV Capabilities {
> ...
> ...  
>   sev-es = true (bool)
>   sev-es-max-guest = n (maximum num of simultaneous SEV-ES guest) [EDX]
>   sev-max-guest = n    (maxium number of simultaneous SEV Guest)
> [ECX-sev-es-max-guest+1]
> ...
> ...
> }

thanks for the info. Does that mean you are already tracking this feature (including the "sev-max-guest" field) in some upstream tracker item (or patch set even)?

Because then I could make this RHBZ dependent on that upstream tracker, and the RHBZ scope would be reduced to backporting AMD's upstream patches. Thanks.

Comment 5 BRIJESH SINGH 2019-04-08 18:18:32 UTC
Hi Laszlo,

I don't have BZ for this. Sometime back we had a discussion about this on libvirt ML and I created TODO task in our internal tracker. If you want then I can submit sev-max-guest field patch in QEMU. I am okay if you submit the patch and copy me for Ack. Let me know whatever works for you. The other fields can be added when we submit the SEV-ES patches.

thanks

Comment 6 Laszlo Ersek 2019-04-09 08:21:18 UTC
Hi Brijesh,

my plate is pretty full, and due to this change modifying a QAPI schema, I expect the upstream patch to go up to v2 at the least, with the review round-trip time that's usual for QEMU. If you can squeeze the upstream patch into your workload, so that I only have backport the upstream commit to RHEL8, I'd prefer that.

Thanks!
Laszlo

Comment 7 BRIJESH SINGH 2019-04-09 13:22:22 UTC
Hi Laszlo,

I will submit the patch this week (probably tomorrow) and copy you.

thanks

Comment 8 BRIJESH SINGH 2019-04-11 18:01:36 UTC
v1 posted https://patchwork.kernel.org/patch/10896557/


Note You need to log in before you can comment on or make changes to this bug.