Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 160125 - Unreasoable memory usage by rpm
Summary: Unreasoable memory usage by rpm
Keywords:
Status: CLOSED DUPLICATE of bug 155730
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 3
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Paul Nasrat
QA Contact: Mike McLean
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2005-06-10 23:11 UTC by H.J. Lu
Modified: 2007-11-30 22:11 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-08-16 15:03:33 UTC


Attachments (Terms of Use)
A testcase (deleted)
2005-06-10 23:14 UTC, H.J. Lu
no flags Details
A testcase (deleted)
2005-06-11 00:39 UTC, H.J. Lu
no flags Details

Description H.J. Lu 2005-06-10 23:11:00 UTC
After installing RC 3 on my EM64T machine with 2GB RAM, I tried
to install updates. The rpm used more than 1.6GB memory when doing
that.

Comment 1 H.J. Lu 2005-06-10 23:14:12 UTC
Created attachment 115317 [details]
A testcase

Here is the list of rpms I was trying to update. I did

# rpm -Uvh `cat /tmp/update.list`

The memory usage of rpm went from 10MB to 33MB, then jump to 600MB, then 1.8G:

  PID USER	PR  NI	VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13524 root	18   0 1168g 1.7g 1.7g R 51.9 89.3   0:15.31 rpm

I had to kill it.

Comment 2 Jeff Johnson 2005-06-10 23:56:04 UTC
This is possibly an attempt to calculate md5 on a sparse file marked with %config.
The fix for that cannot be handled through normal batch upgrades, as the already installed
rpm has the problem. The fix is also isolated to the /var/log/lastlog by strcmp on path

Hmmm, I don't see the setup package within the list of packages. Perhaps there's another sparse file 
marked with %config within the set. If so, that will need treatment like /var/log/lastlog.

Comment 3 H.J. Lu 2005-06-11 00:39:14 UTC
Created attachment 115321 [details]
A testcase

Here is a shorter list. I will keep searching.

Comment 4 Jeff Johnson 2005-06-11 01:24:12 UTC
Thanks. If the mechanism I think, look for a sparse file marked %config. The code path includes all files
on the file system from old packages as well as the install set that triggers.

strace shows an attempt to mmap() the sparse file, grep'ping out all the mmap calls
might confirm if attempting a digest on a sparse file is actually the cause. Perhaps
easier than binary search on the installed files.

Comment 5 H.J. Lu 2005-06-11 16:35:45 UTC
Yes, it is /var/log/lastlog:

-r--------  1 root root 1254130450140 Jun 11 09:28 /var/log/lastlog

Comment 6 Jeff Johnson 2005-06-11 18:09:05 UTC
Upgrading rpm to version that avoids md5 digest on sparse /var/log/lastlog
marked with %config is first step.

Upgrading to a setup package that removes the %config marker is the 2nd step.

The complexities come from attempting to do both these steps at once, as
the old version of rpmlib (wich does not avoid /var/log/lastlog digest) and the old
version of setup (which has the %config marker on /var/log/lastlog) are both
involved in a singles step upgrade.

Divide and conquer ...

Comment 7 H.J. Lu 2005-06-11 18:12:36 UTC
I couldn't find the new updated rpm for FC3. The only one I found for FC3 is
rpm-4.3.2-21.

Comment 8 Jeff Johnson 2005-06-11 18:31:54 UTC
No idea where the patch that fixes /var/log/lastlog has wandered. Verify with
    strings /usr/lib/librpm.a | grep lastlog
If the library contains the string "/var/log/lastlog", then you have the fix.

Comment 9 H.J. Lu 2005-06-11 18:38:53 UTC
There is no /usr/lib/librpm.a in rpm-4.3.2-21. BTW, rpm-4.3.2-21 is the only
rpm for FC3. There is no update since the initial FC3 release.

Comment 10 Jeff Johnson 2005-06-11 18:57:20 UTC
Then look at /usr/lib/librpm*.so

And if on an elf64 system, /usr/lib64 instead.



Comment 11 H.J. Lu 2005-06-11 19:02:11 UTC
I got

[hjl@gnu-9 kernel-FC3]$  strings /usr/lib64/librpm.so | grep lastlog
[hjl@gnu-9 kernel-FC3]$


Comment 12 Jeff Johnson 2005-06-11 19:06:43 UTC
Then that version of rpm does not have the fix.

Try rpm-4.4.1-21 packages from FC4, that almost certainly has the fix (can't check, sorry).

Comment 13 Jeff Johnson 2005-06-11 19:10:46 UTC
Renaming /var/log/lastlog while installing a version of setup that does
not include the %config marker.

The command "rpm -qc setup" or "rpm -qcp setup*.rpm" will spew
which files have %config markers.

Comment 14 Paul Nasrat 2005-08-16 15:03:33 UTC
An update has been released for this issue - please update rpm* in FC3.

*** This bug has been marked as a duplicate of 155730 ***


Note You need to log in before you can comment on or make changes to this bug.