Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1688848 - vsftpd sessions enabled causes continual growth of utmp and high system cpu%
Summary: vsftpd sessions enabled causes continual growth of utmp and high system cpu%
Keywords:
Status: ASSIGNED
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: vsftpd
Version: 7.6
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Ondřej Lysoněk
QA Contact: qe-baseos-daemons
URL:
Whiteboard:
Depends On:
Blocks: 1688852
TreeView+ depends on / blocked
 
Reported: 2019-03-14 15:11 UTC by Welterlen Benoit
Modified: 2019-03-29 06:50 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1688852 (view as bug list)
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)
patch (deleted)
2019-03-14 15:11 UTC, Welterlen Benoit
no flags Details | Diff
Fix SRPM (deleted)
2019-03-14 15:19 UTC, Welterlen Benoit
no flags Details

Description Welterlen Benoit 2019-03-14 15:11:23 UTC
Created attachment 1544113 [details]
patch

Description of problem:

When session_support=YES in vsftpd-3.0.2-25.el7.x86_64, as the connections come in, /var/run/utmp will grow indefinitely and the number of users logged into the system (via top) will continue to grow even though the number of connections still active are not changing.

If you execute `last -f /var/run/utmp` you can see thousands (depending on how long the system has been running) of users listed as "gone - no logout" .  

 last -f /var/run/utmp
...
user3    vsftpd:2244  192.168.122.1    Thu Mar  7 12:37    gone - no logout 
user2    vsftpd:2245  192.168.122.1    Thu Mar  7 12:37    gone - no logout 
user2    vsftpd:1878  192.168.122.1    Thu Mar  7 12:37    gone - no logout 
user6    vsftpd:1758  192.168.122.1    Thu Mar  7 12:37    gone - no logout 
user2    vsftpd:1701  192.168.122.1    Thu Mar  7 12:37    gone - no logout
...

From the utmpdump:
[7] [02245] [    ] [user2   ] [vsftpd:2245 ] [192.168.122.1       ] [0.0.0.0        ] [Thu Mar 07 12:37:23 2019 EST]
we can see that the entry was not updated during the logout.
We also see some logout entries that do not correspond to login entries.

Version-Release number of selected component (if applicable):
- RHEL 7
- vsftpd-3.0.2-25

How reproducible:
With a lot of simultaneous connections

Steps to Reproduce:
1. enable session in vsftpd.conf session_support=YES
2. remove the limitation per ip to reproduce the issue: max_per_ip=0
3. run the server and lot of clients:

#!/bin/bash

function ftpcommand () {
    HOST='192.168.122.155'
    PASSWD='user'
    for user in user1 user2 user3 user4 user5 user6; do
       ftp -inv $HOST << EOF
       quote USER $user
       quote PASS $PASSWD
       bye
EOF
    done
}
while true; do
    for i in {1..10}; do
        ftpcommand &
        #sleep 0.1
    done
    sleep .5
done


Actual results:
Entries in utmp broken, then the file size increases and performances fall

Expected results:
Clean utmp file

Additional info:
The issue is in the utmp management: sysdeputil.c.
The return code of pututxline is not used, then if it fails, the vsf_remove_uwtmp will add an entry instead of removing it.

One solution is to loop until it's successful (with maybe a max) or update s_uwtmp_inserted:

--- 1/sysdeputil.c      2019-03-14 07:11:33.404248709 -0400
+++ 2/sysdeputil.c      2019-03-14 07:14:26.947135914 -0400
@@ -1233,7 +1233,8 @@
                      sizeof(s_utent.ut_host));
   s_utent.ut_tv.tv_sec = vsf_sysutil_get_time_sec();
   setutxent();
-  (void) pututxline(&s_utent);
+  while (pututxline(&s_utent) == NULL)
+    setutxent();
   endutxent();
   updwtmpx(WTMPX_FILE, &s_utent);
 }
@@ -1251,7 +1252,8 @@
   vsf_sysutil_memclr(s_utent.ut_host, sizeof(s_utent.ut_host));
   s_utent.ut_tv.tv_sec = 0;
   setutxent();
-  (void) pututxline(&s_utent);
+  while (pututxline(&s_utent) == NULL)
+    setutxent();
   endutxent();
   s_utent.ut_tv.tv_sec = vsf_sysutil_get_time_sec();
   updwtmpx(WTMPX_FILE, &s_utent);

Comment 2 Welterlen Benoit 2019-03-14 15:19:07 UTC
Created attachment 1544114 [details]
Fix SRPM

Comment 3 Ondřej Lysoněk 2019-03-15 14:20:44 UTC
Thanks for the report.

Do you have some evidence that the pututxline() calls in fact fail? And if they do fail, what is the errno?

I was able to reproduce the 'gone - no logout' lines, however they disappear after some event (perhaps after the corresponding vsftpd processes terminate). And even though I got the 'gone - no logout' lines, none of the pututxline() calls in fact failed.

Comment 4 Welterlen Benoit 2019-03-15 14:26:29 UTC
Hello,

Yes, I was also in the situation where 'gone - no logout' disappear. But the issue is that sometimes, the entry stays in the utmp file, because the call does not success.

I made a ugly version where I print a message when the return is null:

  while (pututxline(&s_utent) == NULL) {
        str_alloc_text(&utmp_str, "pututxline return NULL, retrying \n");
        str_syslog(&utmp_str,0);
        setutxent();
}

And a previous version where I don't retry, and I can see the message when the utmp is broken.


Note You need to log in before you can comment on or make changes to this bug.