Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1354496 - Incorrect errata metadata in updateinfo.xml.gz
Summary: Incorrect errata metadata in updateinfo.xml.gz
Keywords:
Status: CLOSED DUPLICATE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: releng
Version: 7.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Lubos Kocman
QA Contact: Release Test Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-11 12:31 UTC by Silvio Moioli
Modified: 2017-02-03 12:07 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-03 12:07:32 UTC


Attachments (Terms of Use)

Description Silvio Moioli 2016-07-11 12:31:37 UTC
Description of problem:

We get a stack trace when reposyncing RHEL (CDN SSL repositories) on SUSE Manager/Spacewalk.

We believe that the updateinfo.xml.gz file from the CDN is not correct. We would expect:

<reference href="https://bugzilla.redhat.com/show_bug.cgi?id=1312911" type="bugzilla" id="1312911" title="Tracker: guest-images 6.8 release" />

but instead we get:

<reference href="https://bugzilla.redhat.com/show_bug.cgi?id=1159155" type="bugzilla" id="RHEA-2014:1888" title="Rebase of rhs-hadoop and rhs-hadoop-install packages in RH-Common Channel" />

Note that "id" is not an integer, but a RHEA string.

This results in the following error:

ValueError: invalid literal for int() with base 10: 'RHEA-2014:1174'


Version-Release number of selected component (if applicable):

RHEL 7.

How reproducible:

Always.

Steps to Reproduce:

Simply reposync https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rh-common/os

Actual results:

Python error as per above.


Expected results:

No errors.

Comment 2 Florin Samareanu 2016-08-15 07:45:30 UTC
I'd like to add a +1 to this bug for channel EL7 optional : ERROR: invalid literal for int() with base 10: 'RHBA-2014:2014'

Comment 3 Tony Coffman 2016-08-29 20:28:08 UTC
+1 for us as well.

ERROR: invalid literal for int() with base 10: 'RHSA-2015:0087' on https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os

Comment 4 stoupsi 2016-09-09 15:51:47 UTC
Also +2 for us!

repos rhel-7-server-rpms and rhel-7-server-optional-rpms have same issue when reposync ran!

RHBA-2014:2014 from rhel-7-server-optional-rpms 

RHBA-2014:1871 from rhel-7-server-rpms

Comment 5 stoupsi 2016-09-09 19:03:09 UTC
Pretty sure is cause by some accent used in french "À" 

Fist of all, if you check the file format of the updateinfo.xml, they use utf-8. File of repo where it's working correctly are in ASCII.

I've try to convert from utf-8 to ascii, then BOOM!

Erreur append and I found the letters with accent "Ã" who is not compatible with the script python used for spacewalk reposync.

Comment 6 PixelDrift.NET Support 2016-09-12 05:40:52 UTC
I had the exact same issue that Silvio describes attempting to use the current RHEL7 updateinfo.xml with spacewalk-repo-sync. This isn't caused by a character accent as stoupsi describes. I tested converting the file to ASCII and the error remained.

I found the following errata in the current RHEL7 updateinfo.xml with a non-numeric id specified for the bugzilla ID which was causing the 'ERROR: invalid literal for int() with base 10:' error.

RHBA-2014:1850
RHBA-2014:1871
RHBA-2014:1901
RHBA-2014:1996
RHSA-2014:1801
RHSA-2014:1827

The following is a hacky fix for the updateinfo.xml until either the tools/updateinfo.xml are updated.

sed 's/=\([0-9]*\)\(" type="bugzilla" id="\)RH[BS]A-[0-9]\{4\}:[0-9]\{4\}/=\1\2\1/' updateinfo.xml

Comment 7 stoupsi 2016-09-12 11:45:16 UTC
Could you tell me why from the updateinfo.xml file (repo  rhel-server-rhscl-7-rpms) I have the same structure of "ID" and the spacewalk-repo-sync working properly? This is a non-numeric caracters.

here's few lines from the updateinfo.xml file of rhel-server-rhscl-7-rpms repo.

<?xml version="1.0" encoding="UTF-8"?>
    <updates>
        <update status="final" from="release-engineering@redhat.com" version="1" type="enhancement" >
            <id>RHEA-2015:2038</id>
            <issued date="2015-11-16 00:00:00" />
            <title>new packages: devtoolset-4-dwz</title>
            <rights>Copyright 2015 Red Hat Inc</rights>
            <description>The dwz command-line utility attempts to optimize DWARF debugging information
contained in ELF shared libraries and ELF executables for size. It replaces
DWARF information representation with an equivalent smaller representation where
possible, and reduces the amount of duplication by using techniques from the
DWARF Standard, Appendix E. The devtoolset-4-dwz packages provide the Red Hat
Developer Toolset version of dwz.

This enhancement update adds the devtoolset-4-dwz packages to Red Hat Developer
Toolset 4.0 for Red Hat Enterprise Linux. These packages include dwz 0.12, which
provides the following bug fix over the version included in Red Hat Developer
Toolset 3.1:

* A bug which resulted in the possibility of dwz producing binaries or shared
libraries with unaligned non-allocated ELF sections has been fixed.

(BZ#1243063)

All users who require devtoolset-4-dwz are advised to install these new
packages.</description>
            <solution>Before applying this update, make sure all previously released errata
relevant to your system have been applied.

For details on how to apply this update, refer to:

https://access.redhat.com/articles/11258</solution>
            <summary>New devtoolset-4-dwz packages are now available as a part of Red Hat Developer
Toolset 4.0 for Red Hat Enterprise Linux.</summary>
            <pushcount>3</pushcount>
            <updated date="2015-11-16 00:00:00" />
            <references>
                <reference href="https://rhn.redhat.com/errata/RHEA-2015-2038.html" type="self" title="RHEA-2015:2038" />
            </references>
            <pkglist>
                <collection short="" >
                    <name>rhel-server-rhscl-7-rpms__7Server__x86_64</name>
                    <package src="devtoolset-4-dwz-0.12-1.el7.src.rpm" name="devtoolset-4-dwz" epoch="0" version="0.12" release="1.el7" arch="x86_64" >
                        <filename>devtoolset-4-dwz-0.12-1.el7.x86_64.rpm</filename>
                        <sum type="sha256" >69ec402c211690affae73661853ea00f76e4ac7bc5530d920378fcf27add4071</sum>
                    </package>
                </collection>
            </pkglist>
        </update>
        <update status="final" from="release-engineering@redhat.com" version="1" type="bugfix" >

........
 
(In reply to PixelDrift.NET Support from comment #6)
> I had the exact same issue that Silvio describes attempting to use the
> current RHEL7 updateinfo.xml with spacewalk-repo-sync. This isn't caused by
> a character accent as stoupsi describes. I tested converting the file to
> ASCII and the error remained.
> 
> I found the following errata in the current RHEL7 updateinfo.xml with a
> non-numeric id specified for the bugzilla ID which was causing the 'ERROR:
> invalid literal for int() with base 10:' error.
> 
> RHBA-2014:1850
> RHBA-2014:1871
> RHBA-2014:1901
> RHBA-2014:1996
> RHSA-2014:1801
> RHSA-2014:1827
> 
> The following is a hacky fix for the updateinfo.xml until either the
> tools/updateinfo.xml are updated.
> 
> sed 's/=\([0-9]*\)\(" type="bugzilla"
> id="\)RH[BS]A-[0-9]\{4\}:[0-9]\{4\}/=\1\2\1/' updateinfo.xml

Comment 8 PixelDrift.NET Support 2016-09-12 11:59:56 UTC
stoupsi,

Because it's not the ID of the errata in the <id></id> tags that is the issue, it's when the bugzilla ID in the 'reference' tag references a non-numeric ID for the 'id' value. What you have pasted doesn't include any bugzilla 'id' references.

The exact issue is described in Silvio's original post, specifically:

type="bugzilla" id="RHEA-2014:1888"

instead of:

type="bugzilla" id="1312911"

Comment 9 stoupsi 2016-09-12 12:37:28 UTC
You're right guys! I saw now exactly where the bug's from!

Thx!

(In reply to PixelDrift.NET Support from comment #8)
> stoupsi,
> 
> Because it's not the ID of the errata in the <id></id> tags that is the
> issue, it's when the bugzilla ID in the 'reference' tag references a
> non-numeric ID for the 'id' value. What you have pasted doesn't include any
> bugzilla 'id' references.
> 
> The exact issue is described in Silvio's original post, specifically:
> 
> type="bugzilla" id="RHEA-2014:1888"
> 
> instead of:
> 
> type="bugzilla" id="1312911"

Comment 10 PixelDrift.NET Support 2016-09-12 22:57:29 UTC
I think this bug needs reclassification, but it's difficult to say which product it should be assigned to.

Is there a definition/standard somewhere that states that the updateinfo.xml bugzilla id must always be numeric? or is this an assumption made by developers of the tools?

Why has the generation of updateinfo.xml all of a sudden changed on the Red Hat side? and interestingly.. why does this impact older errata?

I have now also found errata in the updateinfo.xml for RHEL6 breaking spacewalk-repo-sync:
RHBA-2015:0059
RHSA-2015:0087

So it's not RHEL7 specific.

Comment 11 Ivo K. 2016-10-11 22:14:23 UTC
Same problem... Bad IDs to date by RHEL repos

# for I in rhel-*; do echo $I:; zcat $I/*updateinfo.xml.gz | grep -o -P "id\=[0-9]*\" type\=\"bugzilla\" id\=\"[A-Z]{4}\-[0-9]{4}\:[0-9]{4}\""; done

rhel-6-server-extras-rpms:
id=1155514" type="bugzilla" id="RHBA-2014:1837"

rhel-7-server-extras-rpms:
id=1173272" type="bugzilla" id="RHEA-2014:1986"

rhel-7-server-optional-rpms:
id=1124472" type="bugzilla" id="RHBA-2014:1901"
id=1172146" type="bugzilla" id="RHBA-2014:1996"
id=1172147" type="bugzilla" id="RHBA-2014:1996"
id=1144289" type="bugzilla" id="RHSA-2014:1827"
id=1144291" type="bugzilla" id="RHSA-2014:1827"
id=1144293" type="bugzilla" id="RHSA-2014:1827"
id=1160685" type="bugzilla" id="RHBA-2014:2014"

rhel-7-server-rpms:
id=1124472" type="bugzilla" id="RHBA-2014:1901"
id=1172146" type="bugzilla" id="RHBA-2014:1996"
id=1172147" type="bugzilla" id="RHBA-2014:1996"
id=1139339" type="bugzilla" id="RHBA-2014:1871"
id=1148554" type="bugzilla" id="RHBA-2014:1871"
id=1148784" type="bugzilla" id="RHBA-2014:1871"
id=1144289" type="bugzilla" id="RHSA-2014:1827"
id=1144291" type="bugzilla" id="RHSA-2014:1827"
id=1144293" type="bugzilla" id="RHSA-2014:1827"
id=1158441" type="bugzilla" id="RHBA-2014:1850"
id=1148230" type="bugzilla" id="RHSA-2014:1801"
id=1148231" type="bugzilla" id="RHSA-2014:1801"
id=1148232" type="bugzilla" id="RHSA-2014:1801"

rhel-x86_64-server-6:
id=1179805" type="bugzilla" id="RHBA-2015:0059"
id=1113470" type="bugzilla" id="RHSA-2015:0087"
id=1163087" type="bugzilla" id="RHSA-2015:0087"

rhel-x86_64-server-6-rpms:
id=1179805" type="bugzilla" id="RHBA-2015:0059"
id=1113470" type="bugzilla" id="RHSA-2015:0087"
id=1163087" type="bugzilla" id="RHSA-2015:0087"

rhel-x86_64-server-7:
id=1124472" type="bugzilla" id="RHBA-2014:1901"
id=1172146" type="bugzilla" id="RHBA-2014:1996"
id=1172147" type="bugzilla" id="RHBA-2014:1996"
id=1139339" type="bugzilla" id="RHBA-2014:1871"
id=1148554" type="bugzilla" id="RHBA-2014:1871"
id=1148784" type="bugzilla" id="RHBA-2014:1871"
id=1144289" type="bugzilla" id="RHSA-2014:1827"
id=1144291" type="bugzilla" id="RHSA-2014:1827"
id=1144293" type="bugzilla" id="RHSA-2014:1827"
id=1158441" type="bugzilla" id="RHBA-2014:1850"
id=1148230" type="bugzilla" id="RHSA-2014:1801"
id=1148231" type="bugzilla" id="RHSA-2014:1801"
id=1148232" type="bugzilla" id="RHSA-2014:1801"

rhel-x86_64-server-optional-6:
id=1113470" type="bugzilla" id="RHSA-2015:0087"
id=1163087" type="bugzilla" id="RHSA-2015:0087"

Comment 12 Ivo K. 2016-10-30 21:49:48 UTC
Patch for Spacewalk 2.5

--- /usr/lib/python2.7/site-packages/spacewalk/satellite_tools/reposync.py.bak  2016-05-25 16:20:24.000000000 +0200
+++ /usr/lib/python2.7/site-packages/spacewalk/satellite_tools/reposync.py      2016-10-30 21:49:36.968311356 +0100
@@ -418,11 +418,18 @@
                 if len(bzs):
                     tmp = {}
                     for bz in bzs:
-                        if bz['id'] not in tmp:
+                        try:
+                            bz_id = int(bz['id'])
+                        # This can happen in some incorrectly generated updateinfo, let's be smart
+                        except ValueError:
+                            log_debug(1, "Bugzilla assigned to advisory %s has invalid id: %s, trying to get it from URL..."
+                                % (e['advisory_name'], bz['id']))
+                            bz_id = int(re.search("\d+$", bz['href']).group(0))
+                        if bz_id not in tmp:
                             bug = Bug()
-                            bug.populate({'bug_id': bz['id'], 'summary': bz['title'], 'href': bz['href']})
+                            bug.populate({'bug_id': bz_id, 'summary': bz['title'], 'href': bz['href']})
                             e['bugs'].append(bug)
-                            tmp[bz['id']] = None
+                            tmp[bz_id] = None
                 cves = [r for r in notice['references'] if r['type'] == 'cve']
                 if len(cves):
                     tmp = {}

Comment 13 Olivier Duquesne 2016-11-02 10:38:49 UTC
Hi, your patch was successfully applied and tested on spacewalk 2.4 too, thank you.

Comment 14 Matthew 2016-11-09 19:50:52 UTC
Running RHEL6.8, and Spacewalk 2.4. I added the above code and it doesn't seem to be working for me. Here's the referenced code I've updated in my reposync.py file:

                if len(bzs):
                    tmp = {}
                    for bz in bzs:
                        try:
                            bz_id = int(bz['id'])
                            # This can happen in some incorrectly generated updateinfo, let's be smart
                        except ValueError:
                            log_debug(1, "Bugzilla assigned to advisory %s has invalid id: %s, trying to get it from URL..." % (e['advisory_name'], bz['id']))
                            bz_id = int(re.search("\d+$", bz['href']).group(0))
                        if bz_id not in tmp:
                            bug = Bug()
                            bug.populate({'bug_id': bz['id'], 'summary': bz['title'], 'href': bz['href']})
                            e['bugs'].append(bug)
                            tmp['bz_id'] = None
                cves = [r for r in notice['references'] if r['type'] == 'cve']
                if len(cves):
                    tmp = {}
                    for cve in cves:
                        if cve['id'] not in tmp:
                            e['cve'].append(cve['id'])
                            tmp[cve['id']] = None

Comment 15 cyetil 2016-11-10 08:18:56 UTC
Hi,

Ivo K. 's patch was successfully applied on spacewalk 2.4 Thank you.

Comment 16 Matthew 2016-11-25 15:34:25 UTC
I've applied the code patch but I'm still seeing the below error when I do the repo sync (already running 2.4)

ERROR: invalid literal for int() with base 10: 'RHBA-2014:1996'

Comment 17 Lubos Kocman 2017-02-03 12:07:32 UTC

*** This bug has been marked as a duplicate of bug 1287605 ***


Note You need to log in before you can comment on or make changes to this bug.