Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 156602 - SCTP memory consumption, additional fixes
Summary: SCTP memory consumption, additional fixes
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel
Version: 4.3
Hardware: All
OS: Linux
Target Milestone: ---
: ---
Assignee: Neil Horman
QA Contact: Brian Brock
Depends On:
Blocks: 168429
TreeView+ depends on / blocked
Reported: 2005-05-02 12:11 UTC by Patrick C. F. Ernzer
Modified: 2007-11-30 22:07 UTC (History)
2 users (show)

Fixed In Version: RHSA-2006-0132
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2006-03-07 18:58:56 UTC
Target Upstream Version:

Attachments (Terms of Use)
patch to account for ulpevents in receive queue (deleted)
2005-09-16 12:18 UTC, Neil Horman
no flags Details | Diff
updated patch to account for remaining sctp skb allocations (deleted)
2005-09-20 20:38 UTC, Neil Horman
no flags Details | Diff
Updated patch taking upstream suggestions (deleted)
2005-09-27 18:06 UTC, Neil Horman
no flags Details | Diff
latest upstream proposal patch (deleted)
2005-10-18 17:23 UTC, Neil Horman
no flags Details | Diff
final upstream version of patch (deleted)
2005-10-20 13:50 UTC, Neil Horman
no flags Details | Diff
new version of patch (deleted)
2005-11-11 15:07 UTC, Neil Horman
no flags Details | Diff

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2005:808 normal SHIPPED_LIVE Important: kernel security update 2005-10-27 04:00:00 UTC
Red Hat Product Errata RHSA-2006:0132 qe-ready SHIPPED_LIVE Moderate: Updated kernel packages available for Red Hat Enterprise Linux 4 Update 3 2006-03-09 16:31:00 UTC

Description Patrick C. F. Ernzer 2005-05-02 12:11:51 UTC
Description of problem: This is split off BZ #146797.

This bug exists to track the non-critical memory consumption issues with sctp
(critical issues were fixed in BZ #146797, kernel 2.6.9-6.43, should be in RHEL4
U1 GA)

Version-Release number of selected component (if applicable):

Comment 6 Patrick C. F. Ernzer 2005-09-07 16:29:22 UTC
Update from IT:

In SIGTRAN's case, the smallest messages are at least 50 bytes. A realistic
message rate is probably 1000 messages/s in + 1000 messages/s out. We are
required to support over 2000 associations, but in reality currently there are
rarely more than 100 associations. This is lowmem we're talking about here and
there is always less than one GB of that to go around, that's the only static
limit there is.

The possibility that so many send buffers would fill up at the same time and
exhaust lowmem is totally theoretical at this point in time.

However, with 50 byte messages each socket uses 1.85 MB of lowmem to buffer
about 100KB of outbound data. That doesn't seem reasonable. One would only need
to fill the send buffer of 487 associations to exhaust 900MB of lowmem. That's
very uncomfortable considering we must support over 2000 concurrent
associations. The number of associations used will increase in the future as
server clusters and networks increase in size.

So this is not a problem in realistic situations yet, but it will be later on.

Comment 8 Neil Horman 2005-09-16 12:18:33 UTC
Created attachment 118890 [details]
patch to account for ulpevents in receive queue

This patch has been tested, and found to improve memory use slightly.  Its not
enough by any stretch, but will likely be part of the final solution.

Comment 9 Neil Horman 2005-09-20 20:38:21 UTC
Created attachment 119047 [details]
updated patch to account for remaining sctp skb allocations

This is an improvement on my previous patch, and seems to clear up all the
missing accounting pieces for me.

Comment 10 Neil Horman 2005-09-26 15:17:15 UTC
Customer reports that latest patch provides correct accounting.  I'll propose
this upstream later today.

Comment 13 Neil Horman 2005-09-27 18:06:01 UTC
Created attachment 119314 [details]
Updated patch taking upstream suggestions

This is an updated version of the patch taking into consideration some of the
suggestions provided by upstream.  Its functionally equivalent, and provides
the same accounting as the previous patch, but uses the skb->desctructor to do
its work, which in my view is a better solution, it also consolidates receive
buffer accounting so it doesn't need to occur both in sctp_rcv and
sctp_ulpevent_set_owner, and cleans up an inadvertent double accounting error.

Comment 14 Neil Horman 2005-10-18 17:23:42 UTC
Created attachment 120128 [details]
latest upstream proposal patch

This is the latest version of the upstream patch.  After going around several
times, we've come to the consensus that this is the best solution to the
immediate problem.  There are some outstanding issues with receive window size
that still need to be hashed out, but they aren't pertinent to this problem,
and the issues aren't RFC violators, nor do they have a real performance
impact. This patch passes all my tests, and as soon as I have upstream
acceptance, I'll build a test kernel for you to confirm

Comment 15 Neil Horman 2005-10-20 13:50:10 UTC
Created attachment 120191 [details]
final upstream version of patch

This is the version of the patch that now has a commitment for upstream
inclusion from the sctp maintainer.  Its identical to the previous patch, but
with a variable name change per request of the maintainer.  I'm going to build
a kernel with this patch against the latest RHEL4 kernel for you to test with,
and post internally for inclusion if it fixes the problem for you (it should,
it passes all the test cases I've been using).

Comment 16 Neil Horman 2005-11-11 15:07:21 UTC
Created attachment 120943 [details]
new version of patch

The receive buffer accounting patch uncovered an skb leak in the establishment
of stream style sockets, which the upstream maintainer rolled into the
accounting patch.  We should pick it up as well.  Same patch as before with the
additional leak fixing bits.

Comment 17 Neil Horman 2005-11-15 14:43:53 UTC
Ok, customer reports this corrects the remaining memeory accounting issues. 
Posting the above patch to rhkl

Comment 25 Red Hat Bugzilla 2006-03-07 18:58:56 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

Note You need to log in before you can comment on or make changes to this bug.