Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 3519 - bug in patch causes rdist'ing files with multiple links to fail
Summary: bug in patch causes rdist'ing files with multiple links to fail
Keywords:
Status: CLOSED DUPLICATE of bug 3228
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: rdist
Version: 6.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jay Turner
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 1999-06-17 00:44 UTC by John Heidemann
Modified: 2015-01-07 23:37 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 1999-06-17 18:30:02 UTC


Attachments (Terms of Use)

Description John Heidemann 1999-06-17 00:44:32 UTC
X-url: http://www.isi.edu/~johnh/
To: rdist-dev@usc.edu
Subject: correction to rdist memory consumption problem
(with fix)
Fcc: week, outbox
--------

On Feb. 4th I posted a message about a problem:

	I've found a serious memory consumption problem for rdist
when running
	over directories that contain many hard-linked files.  The
attached
	patch fixes this problem at the cost of ~12 more lines of
code and a
	little more dynamic memory allocation.

	Rdist allocates a fixed-sized buffer (struct linkbuf in
defs.h) for
	each file that has more than one hard link.  This buffer
includes
	pre-allocated space for 3 strings, each of length BUFSIZ.
Undef
	Redhat 5.2/linux-2.0.36 BUFSIZ is 8KB, so a lot of this
buffer goes to
	waste.

	For my mh mailbox directory (mh---even worse than news
spools at
	stressing file systems!), I have ~54k directory entires,
~35k files
	with only one link and ~8300 with 2-7 links.  With typical
rdist this
	was resulting memory usage of >90MB and much swapping on my
poor
	laptop.  With the included fix memory for this case is
<3MB.

(This problem can also be found at
http://developer.redhat.com/bugzilla/show_bug.cgi?id=1046&BUGLIST=1046.)

Unfortuantely, there was a bug in the patch that I posted
that
breaks rdist'ing files with multiple links in certain
circumstances.
(If Tdest isn't set when linkinfo is called.)

Here's the fix, relative to my prior patch:
> --- client.c-   Mon Jun  7 15:54:59 1999
> +++ client.c    Mon Jun  7 15:58:12 1999
> @@ -347,8 +347,8 @@
>         if (Tdest)
>                 lp->target = strdup(Tdest);
>         else
> -               lp->target = NULL;
> -       if (!lp->pathname || !lp->src || !(Tdest &&
lp->target))
> +               lp->target = strdup("");
> +       if (!lp->pathname || !lp->src || !lp->target)
>                 fatalerr("Cannot malloc memory in
linkinfo.");
>
>         return((struct linkbuf *) NULL);

(Basically if Tdest was null, lp->target was always null and
the
fatalerr was always triggered.)


Alternatively, here's a replacement original patch (with the
bug
fixed):

----------------------------------------
--- src/client.c-	Wed Jun 16 17:34:57 1999
+++ src/client.c	Wed Jun 16 17:35:57 1999
@@ -309,6 +309,18 @@
 	return(0);
 }

+void freelinkinfo(lp)
+	struct linkbuf *lp;
+{
+	if (lp->pathname)
+		free(lp->pathname);
+	if (lp->src)
+		free(lp->src);
+	if (lp->target)
+		free(lp->target);
+	free(lp);
+}
+
 /*
  * Save and retrieve hard link info
  */
@@ -317,6 +329,7 @@
 {
 	struct linkbuf *lp;

+	/* xxx: linear search doesn't scale with many links */
 	for (lp = ihead; lp != NULL; lp = lp->nextp)
 		if (lp->inum == statp->st_ino && lp->devnum ==
statp->st_dev) {
 			lp->count--;
@@ -329,12 +342,14 @@
 	lp->inum = statp->st_ino;
 	lp->devnum = statp->st_dev;
 	lp->count = statp->st_nlink - 1;
-	(void) strcpy(lp->pathname, target);
-	(void) strcpy(lp->src, source);
+	lp->pathname = strdup(target);
+	lp->src = strdup(source);
 	if (Tdest)
-		(void) strcpy(lp->target, Tdest);
+		lp->target = strdup(Tdest);
 	else
-		*lp->target = CNULL;
+		lp->target = strdup("");
+	if (!lp->pathname || !lp->src || !lp->target)
+		fatalerr("Cannot malloc memory in linkinfo.");

 	return((struct linkbuf *) NULL);
 }
--- src/docmd.c-	Wed Jun 16 17:35:07 1999
+++ src/docmd.c	Wed Jun 16 17:35:18 1999
@@ -586,7 +586,7 @@
 	if (!nflag) {
 		register struct linkbuf *nextl, *l;

-		for (l = ihead; l != NULL; free((char *)l), l = nextl) {
+		for (l = ihead; l != NULL; freelinkinfo(l), l = nextl) {
 			nextl = l->nextp;
 			if (contimedout || IS_ON(opts, DO_IGNLNKS) ||
 			    l->count == 0)
--- include/defs.h-	Wed Jun 16 17:35:14 1999
+++ include/defs.h	Wed Jun 16 17:35:18 1999
@@ -276,9 +276,9 @@
 	ino_t	inum;
 	dev_t	devnum;
 	int	count;
-	char	pathname[BUFSIZ];
-	char	src[BUFSIZ];
-	char	target[BUFSIZ];
+	char	*pathname;
+	char	*src;
+	char	*target;
 	struct	linkbuf *nextp;
 };

----------------------------------------

This patch can replace RedHat's rdist-6.1.5-links.patch to
fix the
problem.

To reproduce the problem, create a directory like
/home/johnh/TMP/RDIST_TEST, populate it with the commands
	cd /home/johnh/TMP/RDIST_TEST
	touch a b
	ln b bb
	cat >distfile <<END

HOSTS = ( boreas.isi.edu )

master: /home/johnh/TMP/RDIST_TEST -> ${HOSTS}
        install -w -y;
END

then exercise the bug
with
	rdist -d HOSTS=otherhost.of.yours.com -f
/home/johnh/TMP/RDIST_TEST/distfile

with the incorrect fix rdist will exit with the error
"Cannot malloc memory in linkinfo.".
The correct version will run to completion.

My apologies for sending in a buggy first fix.

   -John Heidemann

Comment 1 Jeff Johnson 1999-06-17 15:27:59 UTC
Could you reply to this message with ascii patch? I hate HTML ...
Thanks.

------- Email Received From  John Heidemann <johnh@isi.edu> 06/17/99 12:17 -------

Comment 2 Jeff Johnson 1999-06-17 18:29:59 UTC
This has already been fixed in #3228 although in a slightly
different fashion. Please reopen this bug if there is still a problem.

Comment 3 Jeff Johnson 1999-06-17 18:30:59 UTC
*** This bug has been marked as a duplicate of 3228 ***


Note You need to log in before you can comment on or make changes to this bug.