Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1515124 - qemu coredumps are not generated properly
Summary: qemu coredumps are not generated properly
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: 4.20.4
Hardware: Unspecified
OS: Unspecified
high
high vote
Target Milestone: ovirt-4.1.8
: 4.19.41
Assignee: Yaniv Bronhaim
QA Contact: Petr Matyáš
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-20 07:57 UTC by Yaniv Bronhaim
Modified: 2017-12-11 16:32 UTC (History)
7 users (show)

Fixed In Version: vdsm v4.19.41
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-11 16:32:01 UTC
oVirt Team: Infra
rule-engine: ovirt-4.1+
pmatyas: testing_plan_complete+


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
oVirt gerrit 84358 master MERGED Adding max_core = "unlimited" to libvirt qemu.conf 2017-11-23 12:49:55 UTC
oVirt gerrit 84389 ovirt-4.1 MERGED Adding max_core = "unlimited" to libvirt qemu.conf 2017-11-23 13:57:22 UTC
oVirt gerrit 84540 None None None 2017-11-23 14:05:09 UTC
oVirt gerrit 84580 ovirt-4.1 MERGED Raising libvirt CONF_VERSION to enable qemu cores 2017-11-23 13:57:52 UTC

Description Yaniv Bronhaim 2017-11-20 07:57:24 UTC
Description of problem:

kill -SIGSEGV [PID] for qemu process should generate core structure under abrt configured directory when the system configured properly.

ls /var/tmp/abrt
ccpp-2017-11-12-17:29:39-18108  ccpp-2017-11-15-16:19:01-3843  last-ccpp

In current version qemu processes are set with "max core file size 0" - which leads to no output at all.

If system is set with "ulimit -c unlimited" you will get the dumps. by default the os installations configured with that value set to 0, unless the process itself configured it, it will inherit the system value.
 

Version-Release number of selected component (if applicable):
v4.20.7

How reproducible:
100%

Steps to Reproduce:
1. run vm and kill its pid
2. check coredump output under /var/tmp/abrt

Actual results:
no core dumps output


Expected results:
qemu process core limit set to unlimited, and coredumps are generated under abrt folder

Comment 1 Martin Perina 2017-11-20 14:24:13 UTC
Setting as a blocker for 4.1.8 now, we need to find out since when qemu coredumps are not available.

Comment 2 Martin Perina 2017-11-21 13:52:55 UTC
Moving to 4.1.9 as we will need more time to provide fix especially around the upgrade. Also removing blocker flag as this doesn't affect normal flows, it just affects our ability to analyze issues, when coredumps (and that means also stacktraces for abort) are not created

Comment 3 Yaniv Bronhaim 2017-11-22 16:51:53 UTC
I posted https://gerrit.ovirt.org/84540 that raises the conf version, this will force re-configure on upgrade. I'm not sure we want to backport this to 4.1 to avoid running re-configure  (which involve also restart of libvirt) during minor version upgrade. introduce it in 4.2 will force re-configure during upgrade which I think is more appropriate if its not a regression. Do you agree?

Comment 5 Yaniv Bronhaim 2017-11-23 14:06:54 UTC
Without this change, abrt doesn't produce any output on kill. With this fix the following will be generated:

[root@dhcp-0-135 vdsm]# kill -SIGSEGV 21662
[root@dhcp-0-135 vdsm]# cd /var/tmp/abrt/
[root@dhcp-0-135 abrt]# ls
ccpp-2017-11-23-15:53:15-21662  last-ccpp
[root@dhcp-0-135 abrt]# cd ccpp-2017-11-23-15\:53\:15-21662/
[root@dhcp-0-135 ccpp-2017-11-23-15:53:15-21662]# ll
total 216
-rw-r-----. 1 root abrt     6 Nov 23 15:53 abrt_version
-rw-r-----. 1 root abrt     4 Nov 23 15:53 analyzer
-rw-r-----. 1 root abrt     6 Nov 23 15:53 architecture
-rw-r-----. 1 root abrt   724 Nov 23 15:53 cgroup
-rw-r-----. 1 root abrt  3119 Nov 23 15:53 cmdline
-rw-r-----. 1 root abrt    11 Nov 23 15:53 component
-rw-r-----. 1 root abrt     1 Nov 23 15:53 count
-rw-r-----. 1 root abrt  7933 Nov 23 15:53 dso_list
-rw-r-----. 1 root abrt    85 Nov 23 15:53 environ
-rw-r-----. 1 root abrt     0 Nov 23 15:53 event_log
-rw-r-----. 1 root abrt    21 Nov 23 15:53 executable
-rw-r-----. 1 root abrt     5 Nov 23 15:53 global_pid
-rw-r-----. 1 root abrt    25 Nov 23 15:53 hostname
-rw-r-----. 1 root abrt    25 Nov 23 15:53 kernel
-rw-r-----. 1 root abrt    10 Nov 23 15:53 last_occurrence
-rw-r-----. 1 root abrt  1323 Nov 23 15:53 limits
-rw-r-----. 1 root abrt   135 Nov 23 15:53 machineid
-rw-r-----. 1 root abrt 57208 Nov 23 15:53 maps
-rw-r-----. 1 root abrt  6359 Nov 23 15:53 open_fds
-rw-r-----. 1 root abrt   393 Nov 23 15:53 os_info
-rw-r-----. 1 root abrt    37 Nov 23 15:53 os_release
-rw-r-----. 1 root abrt    30 Nov 23 15:53 package
-rw-r-----. 1 root abrt     5 Nov 23 15:53 pid
-rw-r-----. 1 root abrt     6 Nov 23 15:53 pkg_arch
-rw-r-----. 1 root abrt     2 Nov 23 15:53 pkg_epoch
-rw-r-----. 1 root abrt    19 Nov 23 15:53 pkg_fingerprint
-rw-r-----. 1 root abrt    11 Nov 23 15:53 pkg_name
-rw-r-----. 1 root abrt    12 Nov 23 15:53 pkg_release
-rw-r-----. 1 root abrt     6 Nov 23 15:53 pkg_vendor
-rw-r-----. 1 root abrt     5 Nov 23 15:53 pkg_version
-rw-r-----. 1 root abrt  1180 Nov 23 15:53 proc_pid_status
-rw-r-----. 1 root abrt     1 Nov 23 15:53 pwd
-rw-r-----. 1 root abrt    26 Nov 23 15:53 reason
-rw-r-----. 1 root abrt     4 Nov 23 15:53 runlevel
-rw-r-----. 1 root abrt    10 Nov 23 15:53 time
-rw-r-----. 1 root abrt     4 Nov 23 15:53 type
-rw-r-----. 1 root abrt     3 Nov 23 15:53 uid
-rw-r-----. 1 root abrt     5 Nov 23 15:53 username
-rw-r-----. 1 root abrt    40 Nov 23 15:53 uuid
-rw-r-----. 1 root abrt  1256 Nov 23 15:53 var_log_messages

Comment 6 Petr Matyáš 2017-12-04 10:23:02 UTC
Verified on vdsm-4.19.41-1.el7ev.x86_64


Note You need to log in before you can comment on or make changes to this bug.