Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1511446 - sshd killed due to invoking a LDAP SSL connection in PAM plugin
Summary: sshd killed due to invoking a LDAP SSL connection in PAM plugin
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openssh
Version: 7.6-Alt
Hardware: x86_64
OS: Linux
medium
urgent
Target Milestone: rc
: ---
Assignee: Jakub Jelen
QA Contact: BaseOS QE Security Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-09 11:46 UTC by HenryGu
Modified: 2019-02-11 15:39 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-11 15:39:56 UTC


Attachments (Terms of Use)

Description HenryGu 2017-11-09 11:46:38 UTC
Description of problem:
We have a PAM plugin configured for sshd, in /etc/pam.d/sshd, like below:
...
#%PAM-1.0
auth       required     pam_additional_auth.so
# added start
#auth       required     pam_env.so
#auth       required     pam_deny.so
# added end
auth       required     pam_sepermit.so
auth       substack     password-auth
auth       include      postlogin
...

In our plugin, we create LDAP SSL connection like below:
...
    if ((rc = ldap_search_s(ld, SEARCH_BASE, LDAP_SCOPE_SUBTREE, ldapFilter, attrs, 0, &results)) != LDAP_SUCCESS) {
        ldap_perror(ld, "**Error: ldap_search_s");

		syslog(LOG_ERR, "invoke: ldap_search_s return %d.", rc);

	    /* Unbind/close connection to the LDAP server.. */
        ldap_unbind(ld);

		/* Free the memory allocated for the results.. */
        ldap_msgfree(results);

		return -3;
    }
...

But we encounter below errors while invoking the function sshd->PAM->LDAP SSL
The invoking failed at ldap_search_s without any return value.
We got error below:
============================================================================
   /var/log/audit/audit.log
    type=ANOM_ABEND msg=audit(1510072928.093:486): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=4892 comm="sshd" reason="memory violation" sig=11

   /var/log/messages
    Nov  8 00:40:41 localhost kernel: traps: sshd[4802] general protection ip:7f039a74807d sp:7ffe79c8f810 error:0 in libldap-2.4.so.2.10.7[7f039a733000+51000]
    Nov  8 00:42:08 localhost kernel: traps: sshd[4892] general protection ip:7f315e0ef07d sp:7fff2d0134f0 error:0 in libldap-2.4.so.2.10.7[7f315e0da000+51000]	
============================================================================


Version-Release number of selected component (if applicable):
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017

How reproducible:
contact me for reproducing

Steps to Reproduce:
1. install our PAM plugin
2. try it with ssh from client
3.

Actual results:
sshd failed

Expected results:
it should work and let login as design

Additional info:

Comment 2 Lukáš Nykrýn 2017-11-09 11:52:00 UTC
Hmm, to be honest I don't see a connection to systemd in this case.

Comment 3 HenryGu 2017-11-09 12:18:18 UTC
(In reply to Lukáš Nykrýn from comment #2)
> Hmm, to be honest I don't see a connection to systemd in this case.

:-), yes, it is a little complicated.

Comment 4 Michal Sekletar 2017-11-09 12:58:41 UTC
(In reply to HenryGu from comment #3)

> :-), yes, it is a little complicated.

Please be more specific and provide reasoning that makes you believe this systemd issue.

Comment 5 HenryGu 2017-11-09 13:41:59 UTC
(In reply to Michal Sekletar from comment #4)
> (In reply to HenryGu from comment #3)
> 
> > :-), yes, it is a little complicated.
> 
> Please be more specific and provide reasoning that makes you believe this
> systemd issue.

it should be sshd problem actually, is there a category of it? I put it in system, sorry.

Comment 6 Michal Sekletar 2017-11-09 14:14:43 UTC
(In reply to HenryGu from comment #5)

> it should be sshd problem actually, is there a category of it? I put it in
> system, sorry.

Yes. Reassigning to openssh.

Comment 7 Jakub Jelen 2017-11-09 14:27:49 UTC
> it should work and let login as design

Why do you think it should work? Did it work before? Do you have SELinux enforcing? How are you so sure that the problem is caused by the SSL connection? Can you share more debug logs with the context of the failure? Audit event type=ANOM_ABEND is kind of segfault. Do you have a coredump? There is really not enough information in this report to figure out what could be wrong.

Comment 8 HenryGu 2017-11-09 15:17:48 UTC
(In reply to Jakub Jelen from comment #7)
> > it should work and let login as design
> 
> Why do you think it should work? Did it work before? Do you have SELinux
> enforcing? How are you so sure that the problem is caused by the SSL
> connection? Can you share more debug logs with the context of the failure?
> Audit event type=ANOM_ABEND is kind of segfault. Do you have a coredump?
> There is really not enough information in this report to figure out what
> could be wrong.

We were lucky on RedHat 6.6. I mean the function with LDAP SSL connection works fine there. It's due to some security reasons, we have to use LDAP SSL connection, but we did a try with no SSL on Redhat7, it works good. We also tried the function with su, it work too. So, all the clues point us to sshd.

Comment 9 HenryGu 2017-11-09 15:21:21 UTC
Do you have test case like us? I mean sshd invokes a PAM plugin, and in the plugin, a function with LDAP SSL connection will be invoked.

Comment 10 HenryGu 2017-11-09 15:24:15 UTC
The strange thing is: there was no coredump. If there were one, I probably would not be here :-)

Comment 11 HenryGu 2017-11-09 15:27:34 UTC
Do you have any test cases like ours :-) 
So far, I do not have a good way to share the code to you.

Comment 12 Jakub Jelen 2017-11-10 07:33:33 UTC
Bugzilla is not a support tool. Please, open a Red Hat support ticket in customer portal with your support and prepare the answers to the questions pointed out in the comment #7. It will simplify the communication. Without most of them I am still not able to guess what went wrong in your case.

https://www.redhat.com/en/services/support

Comment 13 HenryGu 2017-11-14 19:13:50 UTC
(In reply to Jakub Jelen from comment #12)
> Bugzilla is not a support tool. Please, open a Red Hat support ticket in
> customer portal with your support and prepare the answers to the questions
> pointed out in the comment #7. It will simplify the communication. Without
> most of them I am still not able to guess what went wrong in your case.
> 
> https://www.redhat.com/en/services/support

we found the reason. one question, why does sshd needs to link openldap?
=================================================================
[root@localhost /]# ldd ./usr/sbin/sshd
        linux-vdso.so.1 =>  (0x00007ffd6718a000)
...
        libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f707e558000)
...
==========================================================================

Is there a version of sshd with no openldap needs to link on Redhat7?

Comment 14 Jakub Jelen 2017-11-15 08:16:43 UTC
OpenSSH server itself does not need libldap, but the way how the openssh-ldap subpackage is implemented, these libs got linked also to the ssh server.

That should not be intention. At this moment, there is no version without ldap, but if it is an issue for you, we can try to remove this bogus dependency.

Comment 15 Jakub Jelen 2017-11-15 09:20:15 UTC
Something like the following patch for Fedora should do the same job for RHEL7:

https://src.fedoraproject.org/rpms/openssh/c/e3f4c12?branch=master

Comment 16 HenryGu 2017-11-23 05:37:51 UTC
(In reply to Jakub Jelen from comment #15)
> Something like the following patch for Fedora should do the same job for
> RHEL7:
> 
> https://src.fedoraproject.org/rpms/openssh/c/e3f4c12?branch=master

thanks so much! you can close this ticket now.

Comment 17 Jakub Jelen 2017-11-23 09:19:02 UTC
Now, it is fixed in Fedora 27, not in RHEL. But this seems to me as a thing that would make sense to fix in RHEL too, but it will not happen from day to day.
If you want to prioritize it, please open a ticket with your Red Hat support with a link to this bugzilla.

Comment 18 Simo Sorce 2019-02-11 15:39:56 UTC
This issue was not selected to be included either in Red Hat Enterprise Linux 7.7 because it is seen either as low or moderate impact to a small amount of use-cases. The next release will be in Maintenance Support 1 Phase, which means that qualified Critical and Important Security errata advisories (RHSAs) and Urgent Priority Bug Fix errata advisories (RHBAs) may be released as they become available. We will now close this issue, but if you believe that it qualifies for the Maintenance Support 1 Phase, please re-open; otherwise we recommend moving the request to Red Hat Enterprise Linux 8 if applicable.


Note You need to log in before you can comment on or make changes to this bug.