Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1058321 - qemu-kvm-rhev: Qemu: Q35: hw: pci: use after free triggered via guest [rhel-7.2]
Summary: qemu-kvm-rhev: Qemu: Q35: hw: pci: use after free triggered via guest [rhel-7.2]
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Marcel Apfelbaum
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 983344
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-27 14:38 UTC by Markus Armbruster
Modified: 2015-12-04 16:14 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-rhev-2.3.0
Doc Type: Release Note
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-04 16:14:17 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2546 normal SHIPPED_LIVE qemu-kvm-rhev bug fix and enhancement update 2015-12-04 21:11:56 UTC

Description Markus Armbruster 2014-01-27 14:38:37 UTC
Description of problem:
When I unplug a virtio-blk-pci device sitting in a PCIe slot of q35's
xio3130-downstream bridge, the guest kernel warns.

Version-Release number of selected component (if applicable):
At least qemu-kvm-1.5.3-43.el7, older versions crash (bug 983344)
Guest kernel-3.10.0-71.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Boot a RHEL-7 guest with an additional, unused virtio-blk-pci
device connected to PCIe.  This requires q35.  Relevant part of
command line
-M q35 -device ioh3420,bus=pcie.0,id=root.2,slot=3 -device x3130-upstream,bus=root.2,id=upstream2 -device xio3130-downstream,bus=upstream2,id=downstream2,chassis=3 -drive if=none,id=foo,file=tmp.qcow2 -device virtio-blk-pci,id=bar,bus=downstream2,drive=foo
2. When the guest is up, unplug with "device_del bar"

Actual results:
Unplug succeeds, but guest kernel warns (details below).

Expected results:
Unplug succeeds, guest kernel doesn't warn.

Additional info:
Also observed with current upstream QEMU.

Older guest kernels crash, details at
https://bugzilla.redhat.com/show_bug.cgi?id=983344#c14

Guest dmesg:
[   37.674257] ------------[ cut here ]------------
[   37.674296] WARNING: at drivers/virtio/virtio.c:158 virtio_dev_remove+0x74/0x80 [virtio]()
[   37.674301] Modules linked in: ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg kvm_amd kvm pcspkr serio_raw i2c_i801 lpc_ich mfd_core mperf shpchp xfs libcrc32c sr_mod cdrom cirrus virtio_net virtio_blk syscopyarea sysfillrect sysimgblt drm_kms_helper ahci ttm libahci drm libata virtio_pci virtio_ring virtio i2c_core dm_mirror dm_region_hash dm_log dm_mod
[   37.674374] CPU: 0 PID: 22 Comm: kworker/0:1 Not tainted 3.10.0-71.el7.x86_64 #1
[   37.674380] Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
[   37.674393] Workqueue: pciehp-0 pciehp_power_thread
[   37.674397]  0000000000000009 ffff8802443dbb30 ffffffff815bd8c4 ffff8802443dbb68
[   37.674405]  ffffffff81059c61 ffff880241b0c400 ffff880241b0c408 ffffffffa0024000
[   37.674411]  ffff880244312098 0000000000000000 ffff8802443dbb78 ffffffff81059d3a
[   37.674418] Call Trace:
[   37.674431]  [<ffffffff815bd8c4>] dump_stack+0x19/0x1b
[   37.674443]  [<ffffffff81059c61>] warn_slowpath_common+0x61/0x80
[   37.674454]  [<ffffffff81059d3a>] warn_slowpath_null+0x1a/0x20
[   37.674465]  [<ffffffffa00220e4>] virtio_dev_remove+0x74/0x80 [virtio]
[   37.674476]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[   37.674484]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[   37.674491]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[   37.674498]  [<ffffffff81391485>] device_del+0x135/0x1d0
[   37.674505]  [<ffffffff8139153e>] device_unregister+0x1e/0x60
[   37.674516]  [<ffffffffa00224b6>] unregister_virtio_device+0x16/0x30 [virtio]
[   37.674527]  [<ffffffffa004c56b>] virtio_pci_remove+0x2b/0x70 [virtio_pci]
[   37.674537]  [<ffffffff812d252b>] pci_device_remove+0x3b/0xb0
[   37.674546]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[   37.674553]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[   37.674560]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[   37.674567]  [<ffffffff81391485>] device_del+0x135/0x1d0
[   37.674576]  [<ffffffff812cc064>] pci_stop_bus_device+0x94/0xa0
[   37.674583]  [<ffffffff812cc152>] pci_stop_and_remove_bus_device+0x12/0x20
[   37.674591]  [<ffffffff812e4bd8>] pciehp_unconfigure_device+0xa8/0x1b0
[   37.674599]  [<ffffffff812e4538>] pciehp_disable_slot+0x68/0x200
[   37.674607]  [<ffffffff812e4753>] pciehp_power_thread+0x83/0xf0
[   37.674616]  [<ffffffff8107862b>] process_one_work+0x17b/0x460
[   37.674623]  [<ffffffff810793db>] worker_thread+0x11b/0x400
[   37.674631]  [<ffffffff810792c0>] ? rescuer_thread+0x3e0/0x3e0
[   37.674638]  [<ffffffff8107fb90>] kthread+0xc0/0xd0
[   37.674646]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[   37.674653]  [<ffffffff815cd66c>] ret_from_fork+0x7c/0xb0
[   37.674659]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[   37.674665] ---[ end trace f5de3b0770382ce3 ]---

Comment 6 Yanhui Ma 2015-06-19 03:37:10 UTC
Reproduce:
Version of components:
qemu-kvm-1.5.3-43.el7
Guest kernel-3.10.0-71.el7.x86_64

steps:
1. Boot a RHEL-7 guest with an additional, unused virtio-blk-pci
device connected to PCIe.  

/usr/libexec/qemu-kvm -M q35 -m 4G -cpu Opteron_G3 -smp 4,sockets=4,cores=1,threads=1,maxcpus=4 -spice port=5931,disable-ticketing -monitor stdio -qmp tcp:0:6666,server,nowait -device ioh3420,bus=pcie.0,id=root.10,slot=1 -device x3130-upstream,bus=root.10,id=upstream10 -device xio3130-downstream,bus=upstream10,id=downstream10,chassis=0 -drive if=none,id=foo,file=/home/test.qcow2 -device virtio-blk-pci,id=bar,bus=downstream10,drive=foo -drive file=/home/rhel70-64-virtio-scsi.qcow2,if=none,id=drive-data-disk1,cache=writethrough,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi1,addr=0x13 -device scsi-hd,drive=drive-data-disk1,bus=scsi1.0,id=data-disk1,bootindex=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,id=virtio-net-pci0,mac=00:24:21:7f:b6:11,bus=pcie.0,addr=0x9

2. When the guest is up, unplug with "device_del bar"

Actual results:
Unplug succeeds, but guest kernel warns (details below).
[  203.171453] pciehp 0000:02:00.0:pcie24: Card not present on Slot(0)
[  203.174783] ------------[ cut here ]------------
[  203.175309] WARNING: at drivers/virtio/virtio.c:158 virtio_dev_remove+0x74/0x80 [virtio]()
[  203.176139] Modules linked in: fuse ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg i2c_i801 lpc_ich serio_raw shpchp kvm mfd_core mperf pcspkr uinput nfsd auth_rpcgss nfs_acl lockd xfs libcrc32c sd_mod crct10dif_generic crc_t10dif crct10dif_common cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper virtio_scsi virtio_blk ttm drm ahci libahci e1000 virtio_pci libata virtio_ring virtio i2c_core sunrpc dm_mirror dm_region_hash dm_log dm_mod
[  203.184857] CPU: 0 PID: 43 Comm: kworker/0:1 Not tainted 3.10.0-71.el7.x86_64 #1
[  203.185602] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[  203.186186] Workqueue: pciehp-0 pciehp_power_thread
[  203.186682]  0000000000000009 ffff88017966fb30 ffffffff815bd8c4 ffff88017966fb68
[  203.188574]  ffffffff81059c61 ffff8801723f1800 ffff8801723f1808 ffffffffa00ad000
[  203.189404]  ffff8801795b0098 0000000000000000 ffff88017966fb78 ffffffff81059d3a
[  203.190238] Call Trace:
[  203.190505]  [<ffffffff815bd8c4>] dump_stack+0x19/0x1b
[  203.191040]  [<ffffffff81059c61>] warn_slowpath_common+0x61/0x80
[  203.191664]  [<ffffffff81059d3a>] warn_slowpath_null+0x1a/0x20
[  203.192272]  [<ffffffffa00ab0e4>] virtio_dev_remove+0x74/0x80 [virtio]
[  203.192933]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[  203.193574]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[  203.194195]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[  203.194784]  [<ffffffff81391485>] device_del+0x135/0x1d0
[  203.195324]  [<ffffffff8139153e>] device_unregister+0x1e/0x60
[  203.195899]  [<ffffffffa00ab4b6>] unregister_virtio_device+0x16/0x30 [virtio]
[  203.196622]  [<ffffffffa008d56b>] virtio_pci_remove+0x2b/0x70 [virtio_pci]
[  203.197318]  [<ffffffff812d252b>] pci_device_remove+0x3b/0xb0
[  203.197891]  [<ffffffff8139535f>] __device_release_driver+0x7f/0xf0
[  203.198522]  [<ffffffff813953f3>] device_release_driver+0x23/0x30
[  203.199140]  [<ffffffff81394b88>] bus_remove_device+0x108/0x180
[  203.199723]  [<ffffffff81391485>] device_del+0x135/0x1d0
[  203.200264]  [<ffffffff812cc064>] pci_stop_bus_device+0x94/0xa0
[  203.200855]  [<ffffffff812cc152>] pci_stop_and_remove_bus_device+0x12/0x20
[  203.201559]  [<ffffffff812e4bd8>] pciehp_unconfigure_device+0xa8/0x1b0
[  203.202242]  [<ffffffff812e4538>] pciehp_disable_slot+0x68/0x200
[  203.202851]  [<ffffffff812e4753>] pciehp_power_thread+0x83/0xf0
[  203.203460]  [<ffffffff8107862b>] process_one_work+0x17b/0x460
[  203.204067]  [<ffffffff810793db>] worker_thread+0x11b/0x400
[  203.204626]  [<ffffffff810792c0>] ? rescuer_thread+0x3e0/0x3e0
[  203.205247]  [<ffffffff8107fb90>] kthread+0xc0/0xd0
[  203.205740]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[  203.206415]  [<ffffffff815cd66c>] ret_from_fork+0x7c/0xb0
[  203.206966]  [<ffffffff8107fad0>] ? kthread_create_on_node+0x110/0x110
[  203.207638] ---[ end trace 34b69ce9c6a3d31a ]---

As above show, this bz has been reproduce.


===================
Verify:
Version of components:
qemu-kvm-rhev-2.3.0-1.el7.x86_64
Guest kernel-3.10.0-71.el7.x86_64

Steps as above show, after step 2, unplug succeeds, no guest kernel warns
So this bz has been verified.

Comment 11 errata-xmlrpc 2015-12-04 16:14:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html


Note You need to log in before you can comment on or make changes to this bug.