Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 156785

Summary: Missing SHUTDOWN notification with SCTP stream socket
Product: Red Hat Enterprise Linux 4 Reporter: Jere Leppanen <jere.leppanen>
Component: kernelAssignee: Neil Horman <nhorman>
Status: CLOSED ERRATA QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0CC: davej, pcfe, tao
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: RHSA-2006-0132 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-03-07 18:59:46 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 168429    
Attachments:
Description Flags
backport of my upstream proposal to allow tcp style sctp sockets get shutdown notices
none
corrected patch w/ upstream revisions none

Description Jere Leppanen 2005-05-04 09:03:58 UTC
Description of problem:
SCTP SOCK_STREAM sockets do not deliver SCTP_SHUTDOWN_EVENT notification when 
the peer closes connection. Instead recvmsg() returns 0, which it should do 
only after delivering the notification. With SCTP SOCK_SEQPACKET sockets 
SCTP_SHUTDOWN_EVENT is correctly delivered.

Version-Release number of selected component (if applicable):
kernel-2.6.9-5.EL

How reproducible:
Every time.


Steps to Reproduce:

1. Server: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_SCTP */) = 3
2. Server: bind(3, {sa_family=AF_INET, sin_port=htons(61000), sin_addr=inet_addr
("127.0.0.1")}, 16) = 0
3. Server: setsockopt(3, 0x84 /* IPPROTO_SCTP */, 11 /* SCTP_EVENTS */, "\1\1\1
\1\1\1\1\1", 8) = 0
4. Server: listen(3, 16)                           = 0

5. Client: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_SCTP */) = 3
6. Client: bind(3, {sa_family=AF_INET, sin_port=htons(61001), sin_addr=inet_addr
("127.0.0.1")}, 16) = 0
7. Client: connect(3, {sa_family=AF_INET, sin_port=htons(61000), 
sin_addr=inet_addr("127.0.0.1")}, 16) = 0

8. Server: accept(3, 0, NULL)                      = 4
9. Server: recvmsg(4, {msg_name(16)={sa_family=AF_INET, sin_port=htons
(61001),sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\0"..., 1048575}], 
msg_controllen=0, msg_flags=MSG_EOR|MSG_MORE}, MSG_NOSIGNAL) = 20

10. Client: <killed>

11. Server: recvmsg(4, {msg_name(0)={sa_family=AF_UNSPEC, sa_data="\0"}, msg_iov
(1)=[{"\0"..., 1048575}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 0

  
Actual results:
All SCTP events are subscribed in step 3. In step 9, the server gets an 
SCTP_ASSOC_CHANGE with COMM_UP notification and the association is up. The 
client is then killed in step 10, which causes the association to shut down. In 
step 11 recvmsg() returns 0 for the server.

Expected results:
In step 11, recvmsg() should deliver the SCTP_SHUTDOWN_EVENT notification. The 
next recvmsg() should return 0.

Additional info:
This is RFC draft non-compliance. The required behaviour is defined in "Sockets 
API Extensions for Stream Control Transmission Protocol" (draft-ietf-tsvwg-
sctpsocket-10.txt). The obvious workaround is to treat recvmsg()=0 as an 
SCTP_SHUTDOWN_EVENT.

Comment 5 Neil Horman 2005-09-09 20:34:11 UTC
As a personal test here, I modified the sctp_darn utility to subscribe to
shutdown notifications, so that when operating in receive mode, it would bail
out in test_verify_assoc_change, printing the notification type and exiting on
shutdown, when a shutdown event was recieved from it peer (assuming that the
notification was in fact delivered to userspace, which I did not expect, given
this bug report).  When I ran two interactive sctp_darn sessions (one in send
mode, the other in listen), with this modification, I sent a shutdown from the
send side, and was suprised to see the listening side bail out, printing an
indication that it had received an "unknown" notification (one other than the
SCTP_ASSOC_CHANGE that it expected).  The utility printed the notification type,
which was sn_type=32773.  This appears to be the defined value for
SCTP_SHUTDOWN_EVENT.  So it would appear that as long as the the peer socket is
shutdown cleanly, shutdown notifications are properly received.  If I repeat the
test and exit the sending peer with a ctrl-c or a kill -9, I get the same
results.  So I'm unable to reproduce this misssing shutdown notification using
my test setup.  Can you please confirm this?  I'm currently using the 2.6.9-17
RHEL4 kernel, and the sctp_darn utility from lksctp-tools-1.0.2-6.4E.1, modified
to subscribe to shutdown notifications in with this patch:
--- lksctp-tools-1.0.2/src/apps/sctp_darn.c.orig        2005-09-09
16:34:08.000000000 -0400
+++ lksctp-tools-1.0.2/src/apps/sctp_darn.c     2005-09-09 16:34:24.000000000 -0400
@@ -476,6 +476,7 @@ build_endpoint(char *argv0, int portnum)
                memset(&subscribe, 0, sizeof(subscribe));
                subscribe.sctp_data_io_event = 1;
                subscribe.sctp_association_event = 1;
+               subscribe.sctp_shutdown_event = 1;
                error = setsockopt(retval, SOL_SCTP, SCTP_EVENTS,
                                   (char *)&subscribe, sizeof(subscribe));
                if (error) {


Comment 6 Jere Leppanen 2005-09-14 09:37:35 UTC
Re-tested with kernel 2.6.9-17.TEST.IT79426smp (should be no different from 
plain 2.6.9-17 in this case, right?) and sctp_darn from lksctp-tools-1.0.2-
6.4E.1 with Neil's patch from comment #5. Results were identical with my first 
description.

Neil, if you were able to successfully send the shutdown, I guess you were 
using SOCK_SEQPACKET at least on the client side. Sending shutdown doesn't 
appear to be working with SOCK_STREAM sockets. Please double-check that you're 
using SOCK_STREAM on the server side. SHUTDOWN notification works correctly 
with SOCK_SEQPACKET, this problem exists only with SOCK_STREAM sockets.


Comment 7 Patrick C. F. Ernzer 2005-09-14 09:57:43 UTC
Jere,

2.6.9-17.TEST.IT79426smp is not a kernel I built, so I cannot comment. Please do
test with plain 2.6.9-17.

Kind regards,

Patrick

Comment 8 Jere Leppanen 2005-09-15 12:53:08 UTC
Re-tested with kernel 2.6.9-17.ELsmp and sctp_darn from lksctp-tools-1.0.2-
6.4E.1 with Neil's patch from comment #5. Again, results were identical with my 
first description.


Comment 9 Neil Horman 2005-09-21 19:30:22 UTC
Ok, I see the behavior now.  I think this may be because in
sctp_sf_do_9_2_shutdown, we add the command to transition state to
SCTP_STATE_SHUTDOWN_RECEIVED before we add the command to queue the SHUTDOWN
ulpevent to the recieve queue.  This causes sctp_cmd_new_state to run in the
state machine first, setting the RCV_SHUTDOWN state on the socket.  Later, for
tcp sytle sockets in sctp_ulpq_tail_event (where the ulpevent is queued), we
check for the RCV_SHUTDOWN flag, and if its set, we drop the frame.  I'm going
to work up a patch that adds the state transition command to the state machine
after the queue command for testing.

Comment 10 Neil Horman 2005-09-22 17:17:44 UTC
Created attachment 119153 [details]
backport of my upstream proposal to allow tcp style sctp sockets get shutdown notices

Heres a backport of the patch I'm proposing upstream to queue shutdown
notifications to the socket receive queue prior to shutting down the socket.

Comment 11 Neil Horman 2005-09-23 01:10:04 UTC
Created attachment 119171 [details]
corrected patch w/ upstream revisions

This is the final upstream version of the patch, taking into account sri's
suggestion changes for the things I broke in the process of fixing the
origional problem :)

Comment 12 Jere Leppanen 2005-09-26 15:13:37 UTC
Re-tested with the latest patch and it's working now, I'm getting SHUTDOWN 
correctly. Good job.


Comment 13 Neil Horman 2005-09-26 16:56:32 UTC
Thanks, This has been accepted upstream, so I'll post internally today.

Comment 20 Red Hat Bugzilla 2006-03-07 18:59:48 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2006-0132.html