Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1361172 - Disallow to attach upstream port to pxb-pcie if root-port is not attached to pxb-pcie
Summary: Disallow to attach upstream port to pxb-pcie if root-port is not attached to ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Laine Stump
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-28 12:38 UTC by yangyang
Modified: 2016-11-03 18:51 UTC (History)
4 users (show)

Fixed In Version: libvirt-2.0.0-5.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-03 18:51:04 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2577 normal SHIPPED_LIVE Moderate: libvirt security, bug fix, and enhancement update 2016-11-03 12:07:06 UTC

Description yangyang 2016-07-28 12:38:58 UTC
Description of problem:
I manually attach upstream port directly to pxb-pcie and attach 1 downstream port to the upstream port, like this, pxb-pcie--upstream port--downstream port. I plug 1 virtio disk (not bootable disk) into the downstream port. When I start vm, guest kernel panic. 
So in this case if there is no root-port, downstream port seems not usable

Version-Release number of selected component (if applicable):
libvirt-2.0.0-3.el7.x86_64
qemu-kvm-rhev-2.6.0-15.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1.add pxb-pcie--upstream port--downstream port in domain configuration, there is no root-port 

 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/mnt/nfs2/virtio1.img'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </disk>

<controller type='pci' index='0' model='pcie-root'/>

    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>

    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>

<controller type='pci' index='3' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>

    <controller type='pci' index='4' model='pcie-switch-upstream-port'>
      <model name='x3130-upstream'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>

    <controller type='pci' index='5' model='pcie-switch-downstream-port'>
      <model name='xio3130-downstream'/>
      <target chassis='5' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>

2. check qemu cli
#ps -ef|grep qemu
-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \
-device pxb-pcie,bus_nr=10,id=pci.3,bus=pcie.0,addr=0x2 \
-device x3130-upstream,id=pci.4,bus=pci.3,addr=0x0 \
-device xio3130-downstream,port=0x0,chassis=5,id=pci.5,bus=pci.4,addr=0x0 \
-drive file=/mnt/nfs2/RHEL-7.3-latest.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 \
-device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-drive file=/mnt/nfs2/virtio1.img,format=qcow2,if=none,id=drive-virtio-disk1 \
-device virtio-blk-pci,scsi=off,bus=pci.5,addr=0x0,drive=drive-virtio-disk1,id=virtio-disk1

3. start vm
# virsh start vm1-q35-pxb
Domain vm1-q35-pxb started

4.check the guest
guest kernel panic

Actual results:
guest kernel panic

Expected results:
disallow to attach upstream port w/o root-port to pxb-pcie

Additional info:

Comment 2 Laine Stump 2016-08-01 13:41:54 UTC
Marcel - libvirt allows connecting a pcie-switch-upstream-port (x3130-upstream) directly into pcie-expander-bus (pxb-pcie), but this test indicates it doesn't work properly. I think I recall asking you if that was allowable, and wrote the code in libvirt to allow it based on your reply. So there are 3 possibilities: 1) I am remembering wrong, and libvirt shouldn't allow this, 2) there is a bug in one of the devices in qemu, or 3) something else unrelated is causing the crash. Which is it?

Comment 3 Marcel Apfelbaum 2016-08-01 17:07:24 UTC
(In reply to Laine Stump from comment #2)
> Marcel - libvirt allows connecting a pcie-switch-upstream-port
> (x3130-upstream) directly into pcie-expander-bus (pxb-pcie), but this test
> indicates it doesn't work properly. I think I recall asking you if that was
> allowable, and wrote the code in libvirt to allow it based on your reply. So
> there are 3 possibilities: 1) I am remembering wrong, and libvirt shouldn't
> allow this, 2) there is a bug in one of the devices in qemu, or 3) something
> else unrelated is causing the crash. Which is it?

Hi Laine,
There is another possibility, that I didn't explain myself clear enough
and I am sorry if this is the case.
Regarding the problem, the PCIe root bus exposed by the pxb-pcie device
has the same rules as pcie.0 root bus.
Since you cannot add upstream ports directly into pcie.0 bus, you
cannot do that for the pxb-pcie bus either. (and this is why the kernel guest crashes)

You can argue we should prevent this configuration in QEMU, but there are
so many strange and not supported QEMU configurations that I thought
is better to enforce the sane ones in libvirt.

Thanks,
Marcel

Comment 4 Laine Stump 2016-08-10 15:02:18 UTC
Fix pushed upstream:

commit 10031fe5f218fe0acbf873a3063ce42a02fa83d9
Author: Laine Stump <laine@laine.org>
Date:   Fri Aug 5 21:19:27 2016 -0400

    conf: improve error log when PCI devices don't match requested controller

Comment 6 Laine Stump 2016-08-10 15:04:40 UTC
Sorry, disregard Comment 4 - *this* is the upstream commit:

commit b70e54342bbd1756234e07ed6b22bdd3cd12b689
Author: Laine Stump <laine@laine.org>
Date:   Thu Aug 4 13:04:12 2016 -0400

    conf: don't allow connecting upstream-port directly to pce-expander-bus

Comment 9 yangyang 2016-08-12 06:33:07 UTC
Verified on libvirt-2.0.0-5.el7.x86_64

define guest when attaching upstream port into pxb-pcie
<controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>

<controller type='pci' index='3' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='pci' index='4' model='pcie-switch-upstream-port'>
      <model name='x3130-upstream'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='5' model='pcie-switch-downstream-port'>
      <model name='xio3130-downstream'/>
      <target chassis='5' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>

# virsh define vm1-q35-pxb.xml
error: Failed to define domain from vm1-q35-pxb.xml
error: XML error: The device at PCI address 0000:03:00.0 cannot be plugged into the PCI controller with index='3'. It requires a controller that accepts a pci-switch-upstream-port.

If attaching root-port into pxb-pcie then attaching upstream-port, guest works

Comment 11 errata-xmlrpc 2016-11-03 18:51:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html


Note You need to log in before you can comment on or make changes to this bug.