Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1548199 - _create_checksum:IOError: [Errno 2] No such file or directory: '/var/spool/abrt/.../sos.######/sosreport-*-*.tar.xz' [NEEDINFO]
Summary: _create_checksum:IOError: [Errno 2] No such file or directory: '/var/spool/ab...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sos
Version: 7.4
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Pavel Moravec
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-23 00:02 UTC by Peter Portante
Modified: 2018-11-06 07:48 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-06 07:48:01 UTC
pmoravec: needinfo? (pportant)


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github sosreport sos pull 1273 None None None 2018-04-20 15:22:58 UTC

Description Peter Portante 2018-02-23 00:02:50 UTC
reason:         sosreport.py:1461:_create_checksum:IOError: [Errno 2] No such file or directory: '/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799/sos.YoLoNP/sosreport-foobar.example.com-20180222200146.tar.xz'
cmdline:        /usr/bin/python /usr/sbin/sosreport --tmp-dir /var/spool/abrt/ccpp-2018-02-22-20:01:21-81799 --batch --only=anaconda --only=boot --only=devicemapper --only=filesys --only=hardware --only=kernel --only=libraries --only=memory --only=networking --only=nfsserver --only=pam --only=process --only=rpm -k rpm.rpmva=off --only=ssh --only=services --only=yum --only=general --only=x11 --only=cups --only=logs --only=grub2 --only=cron --only=pci --only=auditd --only=selinux --only=lvm2 --only=sar --only=processor
executable:     /usr/sbin/sosreport
package:        sos-3.4-13.el7_4
component:      sos
pid:            89283
hostname:       foobar.example.com
count:          7
abrt_version:   2.1.11
analyzer:       Python
architecture:   x86_64
duphash:        ee1fc6e6c7f908eeb6387817980804fc75e9bdfe
event_log:      
kernel:         3.10.0-693.17.1.el7.x86_64
last_occurrence: 1519339550
os_release:     Red Hat Enterprise Linux Server release 7.4 (Maipo)
pkg_arch:       noarch
pkg_epoch:      0
pkg_fingerprint: 199E 2F91 FD43 1D51
pkg_name:       sos
pkg_release:    13.el7_4
pkg_vendor:     Red Hat, Inc.
pkg_version:    3.4
runlevel:       N 3
time:           Thu 22 Feb 2018 08:03:24 PM UTC
type:           Python
uid:            0
username:       root
uuid:           ee1fc6e6c7f908eeb6387817980804fc75e9bdfe

backtrace:
:sosreport.py:1461:_create_checksum:IOError: [Errno 2] No such file or directory: '/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799/sos.YoLoNP/sosreport-foobar.example.com-20180222200146.tar.xz'
:
:Traceback (most recent call last):
:  File "/usr/sbin/sosreport", line 25, in <module>
:    main(sys.argv[1:])
:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1632, in main
:    sos.execute()
:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1611, in execute
:    return self.final_work()
:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1523, in final_work
:    checksum = self._create_checksum(archive, hash_name)
:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1461, in _create_checksum
:    archive_fp = open(archive, 'rb')
:IOError: [Errno 2] No such file or directory: '/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799/sos.YoLoNP/sosreport-foobar.example.com-20180222200146.tar.xz'
:
:Local variables in innermost frame:
:self: <sos.sosreport.SoSReport object at 0x2205990>
:hash_name: 'md5'
:archive: '/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799/sos.YoLoNP/sosreport-foobar.example.com-20180222200146.tar.xz'
environ:
:LANG=en_US.UTF-8
:GRUB_DISABLE_OS_PROBER=true
:ABRT_VERBOSE=0
:REPORT_CLIENT_SLAVE=1
:ABRT_SYSLOG=1
:SHLVL=1
:PWD=/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799
:REPORT_CLIENT_NONINTERACTIVE=1
:DUMP_DIR=/var/spool/abrt/ccpp-2018-02-22-20:01:21-81799
:LC_ALL=C
:PATH=/usr/sbin:/usr/bin:/root/bin:/usr/local/bin:/usr/local/sbin
:EVENT=post-create
:_=/usr/bin/nice
machineid:
:systemd=939e8e72f9fe4e789752f565ee80aaa6
:sosreport_uploader-dmidecode=4d5afca2532d5347dd48ac146924ebdcf7b299f01ace88e0f67dc0fbb501dd4b
os_info:
:NAME="Red Hat Enterprise Linux Server"
:VERSION="7.4 (Maipo)"
:ID="rhel"
:ID_LIKE="fedora"
:VARIANT="Server"
:VARIANT_ID="server"
:VERSION_ID="7.4"
:PRETTY_NAME="Employee SKU"
:ANSI_COLOR="0;31"
:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
:HOME_URL="https://www.redhat.com/"
:BUG_REPORT_URL="https://bugzilla.redhat.com/"
:
:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
:REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
:REDHAT_SUPPORT_PRODUCT_VERSION="7.4"

Comment 2 Peter Portante 2018-02-23 00:03:22 UTC
This bug has occurred 5 times in the last 3 hours.

Comment 3 Bryn M. Reeves 2018-02-23 09:04:51 UTC
Did ABRT remove the sos working directory while sosreport was still running? I thought that behaviour had already been fixed.

(the command line passed to sos is a giveaway that this was abrt running sosreport in response to some other detected crash: it passes that crazy range of hard-coded plugins, rather than either use the defaults as we've requested or using the profile mechanism to select a targeted set of data).

We can try to work around this and to handle the exceptions that pop up but this is not that simple: depending on exactly when abrt decides to trash the directory we can start seeing these exceptions in different parts of sos.

It's reasonable hardening but in the meantime it's better if abrt does not remove these paths without first killing the sosreport job.

Comment 4 Pavel Moravec 2018-03-03 13:32:55 UTC
(In reply to Peter Portante from comment #2)
> This bug has occurred 5 times in the last 3 hours.

Could you please provide some more background, i.e. system logs (or even sosreport that doesnt crach :) ), or if Bryn is right with abrtd removing dir?

(and re-scheduling for potential inclusion in 7.6)

Comment 5 Pavel Moravec 2018-03-03 16:52:30 UTC
Ah, this has been already reported and we have a reproducer..

*** This bug has been marked as a duplicate of bug 1511487 ***

Comment 9 Pavel Moravec 2018-11-06 07:48:01 UTC
This should have been fixed in RHEL7.6 due to previous updates done in sos 3.6 we rebased to in 7.6. Hence closing the BZ as fixed in current release:

package: sos-3.6-11

7.6.0 errata: https://access.redhat.com/errata/RHEA-2018:3144
7.6.0-day errata: https://access.redhat.com/errata/RHBA-2018:3338

(either one contains the fix)


Note You need to log in before you can comment on or make changes to this bug.