Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1605098 - Get error message in qemu when boot guest with max number of virtio serial ports
Summary: Get error message in qemu when boot guest with max number of virtio serial ports
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.6
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: ---
Assignee: Gal Hammer
QA Contact: Qianqian Zhu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-20 07:39 UTC by Li Xiaohui
Modified: 2018-11-05 09:02 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-05 09:02:58 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Li Xiaohui 2018-07-20 07:39:21 UTC
Description of problem:
Get error message in qemu when boot guest with max number of virtio serial ports


Version-Release number of selected component (if applicable):
host info:
kernel-3.10.0-915.el7.x86_64 & qemu-img-rhev-2.12.0-7.el7.x86_64

guest info:
kernel-3.10.0-915.el7.x86_64


How reproducible:
100%


Steps to Reproduce:
1.Start Guest with 510 virtserialport on a virito-serial-pci,like following:
/usr/libexec/qemu-kvm -M pc \
-cpu SandyBridge -enable-kvm \
-m 8G \
-smp 4 -nodefconfig -nodefaults \
-rtc base=utc,clock=host,driftfix=slew \
-drive file=/home/rhel-image/rhel-7.6-915.qcow2,if=none,id=drive-disk1,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x7,id=scsi0 -device scsi-hd,drive=drive-disk1,id=disk1,bus=scsi0.0,lun=1 \
-netdev tap,id=net0,vhost=on -device virtio-net-pci,netdev=net0,mac=00:1b:21:7a:76:1c \
-vga qxl \
-monitor stdio \
-vnc :3 \
-boot menu=on \
-device virtio-serial-pci,id=virtio-serial0,max_ports=511 \
-chardev socket,id=channel0,path=/tmp/helloworld0,server,nowait -device virtserialport,chardev=channel0,name=com.redhat.rhevm.vdsm0,bus=virtio-serial0.0,id=port0 \
...
-chardev socket,id=channel509,path=/tmp/helloworld509,server,nowait -device virtserialport,chardev=channel509,name=com.redhat.rhevm.vdsm509,bus=virtio-serial0.0,id=port509 \


Actual results:
after step1, guest boot up quickly and successfully, but get some error in qemu
(qemu) qemu-kvm: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-kvm: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).

Expected results:
after step1, guest boot up quickly and successfully, and won't get error message in qemu


Additional info:
Bug 1483199 is closed and Bug 1528588 is similar with slow and fail to boot guest problem. 
and I think this bug is not same with Bug 1528588, and need one bug to trace this error, so I set this new bug

Comment 2 Ademar Reis 2018-07-25 19:42:22 UTC
(In reply to Li Xiaohui from comment #0)
> 
> Additional info:
> Bug 1483199 is closed and Bug 1528588 is similar with slow and fail to boot
> guest problem. 
> and I think this bug is not same with Bug 1528588, and need one bug to trace
> this error, so I set this new bug

Reassigning to Gal who's taking care of bug 1528588 already.

Comment 3 Gal Hammer 2018-08-08 13:05:06 UTC
This is the same bug as bug 1483199 and bug 1528588.

You see the "Too many open files (-24)" error message because you need more file handles when working with a lot of virtio-serial devices each with a lot of ports. Raising the allowed open files limit will solve this (# ulimit -n 30000). But once you solved the file handles limit, you'll be using the new ioeventfd feature which is slower unless you're using version newer than Linux 4.17 and QEMU 2.11.

Since this test doesn't apply to a real use case I don't think there is a need to try and back port the patches.

Comment 4 Li Xiaohui 2018-08-17 01:47:28 UTC
Hello Gal:
sorry for seeing this needinfo too later. 
In fact, I also reproduce this bug with two virtserialports, so I think this error has nothing with virtserialport num, less or more virtserialports all can cause this error. 
command like following:
/usr/libexec/qemu-kvm -M pc \
......
-device virtio-serial-pci,id=virtio-serial0,max_ports=511 \
-chardev socket,id=chardev1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=chardev1,id=serial1,name=com.redhat.rhevm.vdsm1,bus=virtio-serial0.0 \
-chardev socket,id=chardev2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=chardev2,id=serial2,name=com.redhat.rhevm.vdsm2,bus=virtio-serial0.0 \

My doubt here: 
(1)what's the allowed open files limit about our product? ----I don't think it can't support two virtio-serial device to open the files,  so do we still need to deal this bug?

Comment 5 Gal Hammer 2018-08-19 07:14:07 UTC
(In reply to Li Xiaohui from comment #4)
> Hello Gal:
> sorry for seeing this needinfo too later. 
> In fact, I also reproduce this bug with two virtserialports, so I think this
> error has nothing with virtserialport num, less or more virtserialports all
> can cause this error. 

The error is not related directly to the virtio-serial or its ports. It is a limit enforced by the kernel. It is virtio-serial when working with a lot of ports, open a file handle for each port. So using one or two devices each with 511 ports will reach your open files limit quite fast.

> command like following:
> /usr/libexec/qemu-kvm -M pc \
> ......
> -device virtio-serial-pci,id=virtio-serial0,max_ports=511 \
> -chardev socket,id=chardev1,path=/tmp/helloworld1,server,nowait -device
> virtserialport,chardev=chardev1,id=serial1,name=com.redhat.rhevm.vdsm1,
> bus=virtio-serial0.0 \
> -chardev socket,id=chardev2,path=/tmp/helloworld2,server,nowait -device
> virtserialport,chardev=chardev2,id=serial2,name=com.redhat.rhevm.vdsm2,
> bus=virtio-serial0.0 \
> 
> My doubt here: 
> (1)what's the allowed open files limit about our product? ----I don't think
> it can't support two virtio-serial device to open the files,  so do we still
> need to deal this bug?

This is not a bug. Run "ulimit -n" and you'll see your default/current open file limit. If you require more than this limit, you should increase it.

Comment 6 Li Xiaohui 2018-08-20 02:19:27 UTC
(In reply to Gal Hammer from comment #5)
> (In reply to Li Xiaohui from comment #4)
> > Hello Gal:
> > sorry for seeing this needinfo too later. 
> > In fact, I also reproduce this bug with two virtserialports, so I think this
> > error has nothing with virtserialport num, less or more virtserialports all
> > can cause this error. 
> 
> The error is not related directly to the virtio-serial or its ports. It is a
> limit enforced by the kernel. It is virtio-serial when working with a lot of
> ports, open a file handle for each port. So using one or two devices each
> with 511 ports will reach your open files limit quite fast.
> 
> > command like following:
> > /usr/libexec/qemu-kvm -M pc \
> > ......
> > -device virtio-serial-pci,id=virtio-serial0,max_ports=511 \
> > -chardev socket,id=chardev1,path=/tmp/helloworld1,server,nowait -device
> > virtserialport,chardev=chardev1,id=serial1,name=com.redhat.rhevm.vdsm1,
> > bus=virtio-serial0.0 \
> > -chardev socket,id=chardev2,path=/tmp/helloworld2,server,nowait -device
> > virtserialport,chardev=chardev2,id=serial2,name=com.redhat.rhevm.vdsm2,
> > bus=virtio-serial0.0 \
> > 
> > My doubt here: 
> > (1)what's the allowed open files limit about our product? ----I don't think
> > it can't support two virtio-serial device to open the files,  so do we still
> > need to deal this bug?
> 
> This is not a bug. Run "ulimit -n" and you'll see your default/current open
> file limit. If you require more than this limit, you should increase it.

Thank you, Gal, I got it. I will modify this case for better understand


Note You need to log in before you can comment on or make changes to this bug.