Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1139391 - Satellite 5.6 postgres takes a long time to satellite-sync for initial sync
Summary: Satellite 5.6 postgres takes a long time to satellite-sync for initial sync
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 560
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: Stephen Herr
QA Contact: Jan Hutař
URL:
Whiteboard:
Depends On: 1023557
Blocks: sat560-triage
TreeView+ depends on / blocked
 
Reported: 2014-09-08 20:02 UTC by Stephen Herr
Modified: 2014-10-14 19:38 UTC (History)
9 users (show)

Fixed In Version: spacewalk-backend-2.0.3-30-sat spacewalk-schema-2.0.2-17-sat satellite-schema-5.6.0.19-3-sat
Doc Type: Bug Fix
Doc Text:
Clone Of: 1023557
Environment:
Last Closed: 2014-10-14 19:38:00 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1632 normal SHIPPED_LIVE Red Hat Satellite bug fix update 2014-10-14 23:37:20 UTC

Comment 2 Stephen Herr 2014-09-09 15:48:14 UTC
The root problem is that Postgresql does not support autonomous transactions, which we require. We work around this problem by using dblink_exec to execute the sql that should be autonomous in a separate connection. This works fine for the most part, but satellite-sync can require millions of autonomous transactions for an inital base channel sync, and the overhead of creating all those connections to the database was making it very slow. Note that these connections are only required when inserting new package capabilities or checksum information (or other things as well, but less commonly), so only the initial sync is affected.

I have filed pull request https://github.com/spacewalkproject/spacewalk/pull/120 for review that should (mostly) solve this issue. Instead of creating millions of separate connections to the db, this change would create one additional connection and reuse it for the most commonly used batches of autonomous sql. This has the effect of reducing the time required to the initial metadata import from a base channel from 20 hours 35 minutes to 5 hours 21 minutes, a 4x speed improvement. Sync logs below.

Before changes:
satellite-sync -c rhel-x86_64-server-6 --no-rpms --no-kickstarts
00:04:35 Spacewalk - live synchronization
00:04:35    url: https://satellite.rhn.redhat.com
00:04:35    debug/output level: 1
00:04:35    db:  spaceuser/<password>@spaceschema
00:04:35
00:04:35 Retrieving / parsing channel-families data
00:04:51 channel-families data complete
00:04:51
00:04:51 Retrieving / parsing product names data
00:04:51 product names data complete
00:04:55
00:04:55 Retrieving / parsing arches data
00:04:56 arches data complete
00:04:56
00:04:56 Retrieving / parsing additional arches data
00:04:56 additional arches data complete
00:04:56
00:04:56 Retrieving / parsing channel data
00:07:22    p = previously imported/synced channel
00:07:22    . = channel not yet imported/synced
00:07:22    base-channels:
00:07:22       p rhel-x86_64-server-6                     12659
00:07:22
00:07:23 Channel data complete
00:07:23
00:07:23 Retrieving short package metadata (used for indexing)
00:07:23    Retrieving / parsing short package metadata: rhel-x86_64-server-6 (12659)
00:07:32 Diffing package metadata (what's missing locally?): rhel-x86_64-server-6
            ________________________________________
Diffing:    ######################################## - complete
00:09:47
00:09:47 Downloading package metadata
00:11:53    Retrieving / parsing *relevant* package metadata: rhel-x86_64-server-6 (NONE RELEVANT)
00:13:56
00:13:56 Importing package metadata
00:13:56    Importing *relevant* package metadata: rhel-x86_64-server-6 (12659)
            ________________________________________
Importing:  ######################################## - complete
20:18:21
20:18:21 Linking packages to channels
20:19:34
20:19:34 Downloading errata data
20:19:34    Retrieving / parsing errata data: rhel-x86_64-server-6 (NONE RELEVANT)
20:19:34 Downloading errata data complete
20:19:34
20:19:34 Importing channel errata
20:20:06    Importing *relevant* errata: rhel-x86_64-server-6 (2407)
            ________________________________________
Downloading:######################################## - complete
    Import complete:
        Begin time: Thu Jul 10 00:04:35 2014
        End time:   Thu Jul 10 20:40:20 2014
        Elapsed:    20 hours, 35 minutes, 44 seconds



After changes:
satellite-sync -c rhel-x86_64-server-6 --no-rpms --no-kickstarts
17:50:46 Spacewalk - live synchronization
17:50:46    url: https://satellite.rhn.redhat.com
17:50:46    debug/output level: 1
17:50:47    db:  spaceuser/<password>@spaceschema
17:50:47
17:50:47 Retrieving / parsing channel-families data
17:51:03 channel-families data complete
17:51:03
17:51:03 Retrieving / parsing product names data
17:51:04 product names data complete
17:51:08
17:51:08 Retrieving / parsing arches data
17:51:08 arches data complete
17:51:08
17:51:08 Retrieving / parsing additional arches data
17:51:09 additional arches data complete
17:51:09
17:51:09 Retrieving / parsing channel data
17:53:35    p = previously imported/synced channel
17:53:35    . = channel not yet imported/synced
17:53:35    base-channels:
17:53:35       p rhel-x86_64-server-6                     12659
17:53:35
17:53:35 Channel data complete
17:53:35
17:53:35 Retrieving short package metadata (used for indexing)
17:53:36    Retrieving / parsing short package metadata: rhel-x86_64-server-6 (12659)
17:53:44 Diffing package metadata (what's missing locally?): rhel-x86_64-server-6
            ________________________________________
Diffing:    ######################################## - complete
17:55:59
17:55:59 Downloading package metadata
17:58:05    Retrieving / parsing *relevant* package metadata: rhel-x86_64-server-6 (NONE RELEVANT)
18:00:11
18:00:11 Importing package metadata
18:00:11    Importing *relevant* package metadata: rhel-x86_64-server-6 (12659)
            ________________________________________
Importing:  ######################################## - complete
23:02:21
23:02:21 Linking packages to channels
23:03:37
23:03:37 Downloading errata data
23:03:37    Retrieving / parsing errata data: rhel-x86_64-server-6 (2407)
            ________________________________________
Downloading:######################################## - complete
23:05:48 Downloading errata data complete
23:05:48
23:05:48 Importing channel errata
23:06:20    Importing *relevant* errata: rhel-x86_64-server-6 (2407)
            ________________________________________
Downloading:######################################## - complete
    Import complete:
        Begin time: Wed Jul  9 17:50:46 2014
        End time:   Wed Jul  9 23:11:50 2014
        Elapsed:    5 hours, 21 minutes, 4 seconds

Comment 17 errata-xmlrpc 2014-10-14 19:38:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1632.html


Note You need to log in before you can comment on or make changes to this bug.