Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1362630 - No way to enable remote execution SSH debug logging easily
Summary: No way to enable remote execution SSH debug logging easily
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Remote Execution
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
medium vote
Target Milestone: Unspecified
Assignee: Adam Ruzicka
QA Contact: Peter Ondrejka
Depends On:
TreeView+ depends on / blocked
Reported: 2016-08-02 17:48 UTC by Stephen Benjamin
Modified: 2019-04-01 20:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-10-16 18:57:30 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Foreman Issue Tracker 16991 None None None 2016-10-18 12:53:57 UTC

Description Stephen Benjamin 2016-08-02 17:48:03 UTC
Description of problem:
Net::SSH is able to have quite detailed logging, but we don't give the user a way to see it - options[:logger] seems empty, so it goes to STDERR.  We don't set the verbosity options either, maybe try to keep that the same as proxy's.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Look in /var/log/foreman-proxy/*

Actual results:
No ssh logs

Expected results:
See ssh logs

Additional info:

Manual way to view the logs:

1. Enable debug logging by editing /opt/theforeman/tfm/root/usr/share/gems/gems/smart_proxy_remote_execution_ssh_core-*/lib/smart_proxy_remote_execution_ssh_core/connector.rb

Line 133 change Net::SSH.start(@host, @user, ssh_options) to Net::SSH.start(@host, @user, ssh_options.merge(:verbose => :debug))

2. Stop smart_proxy_dynflow_core

3. Start as foreman-proxy user in interaactive mode:

  sudo -u foreman-proxy /usr/bin/smart_proxy_dynflow_core -p /var/run/foreman-proxy/

SSH log messages will show on STDERR

Comment 1 Stephen Benjamin 2016-08-02 17:50:39 UTC
With this enabled you'll see logs like this:

[root@sat-rhel7 smart_proxy_remote_execution_ssh_core]# sudo -u foreman-proxy /usr/bin/smart_proxy_dynflow_core -p /var/run/foreman-proxy/
D, [2016-08-02T17:49:16.375743 #12460] DEBUG -- net.ssh.transport.session[1d35f44]: establishing connection to
D, [2016-08-02T17:49:16.376066 #12460] DEBUG -- net.ssh.transport.session[1d35f44]: connection established
I, [2016-08-02T17:49:16.376127 #12460]  INFO -- net.ssh.transport.server_version[1d34e28]: negotiating protocol version
D, [2016-08-02T17:49:16.380906 #12460] DEBUG -- net.ssh.transport.server_version[1d34e28]: remote is `SSH-2.0-OpenSSH_6.6.1'
D, [2016-08-02T17:49:16.380982 #12460] DEBUG -- net.ssh.transport.server_version[1d34e28]: local is `SSH-2.0-Ruby/Net::SSH_2.9.2 x86_64-linux'
D, [2016-08-02T17:49:16.384481 #12460] DEBUG -- tcpsocket[1d35850]: read 1640 bytes
D, [2016-08-02T17:49:16.384630 #12460] DEBUG -- tcpsocket[1d35850]: received packet nr 0 type 20 len 1636
I, [2016-08-02T17:49:16.384695 #12460]  INFO -- net.ssh.transport.algorithms[1cf1b64]: got KEXINIT from server
I, [2016-08-02T17:49:16.384818 #12460]  INFO -- net.ssh.transport.algorithms[1cf1b64]: sending KEXINIT
D, [2016-08-02T17:49:16.384957 #12460] DEBUG -- tcpsocket[1d35850]: queueing packet nr 0 type 20 len 1684
D, [2016-08-02T17:49:16.385019 #12460] DEBUG -- tcpsocket[1d35850]: sent 1688 bytes
I, [2016-08-02T17:49:16.385039 #12460]  INFO -- net.ssh.transport.algorithms[1cf1b64]: negotiating algorithms
D, [2016-08-02T17:49:16.385138 #12460] DEBUG -- net.ssh.transport.algorithms[1cf1b64]: negotiated:
* kex: diffie-hellman-group-exchange-sha1
* host_key: ssh-rsa
* encryption_server: aes128-cbc
* encryption_client: aes128-cbc
* hmac_client: hmac-sha1
* hmac_server: hmac-sha1
* compression_client: none
* compression_server: none
* language_client: 
* language_server:

Comment 3 Adam Ruzicka 2016-10-18 12:53:55 UTC
Created redmine issue from this bug

Comment 4 2017-10-25 16:20:45 UTC
Upstream bug assigned to

Comment 5 2017-10-25 16:20:49 UTC
Upstream bug assigned to

Comment 6 2018-03-01 17:20:22 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue has been resolved.

Comment 7 Peter Ondrejka 2018-04-09 13:17:58 UTC
Verified upstream, pkgs:



set ssh_log_level in /etc/foreman-proxy/settings.d/remote_execution_ssh.yml, run katello-service restart

- set to nonsense, observe in /var/log/messages:
/var/log/messages:Apr  9 07:58:50 hostname smart_proxy_dynflow_core: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.1.1/lib/foreman_remote_execution_core.rb:31:in `validate_ssh_log_level!': Wrong value 'jkjkjj' for ssh_log_level, must be one of debug, info, warn, error, fatal (RuntimeError)

- set low level (e.g.), observe in /var/log/messages:
/var/log/messages:Apr  9 08:15:34 hostname smart_proxy_dynflow_core: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.1.1/lib/foreman_remote_execution_core.rb:46:in `validate_ssh_log_level!': ssh_log_level cannot be more verbose than regular log level (RuntimeError)

- lower the level /etc/smart_proxy_dynflow_core/settings.yml to match ssh_log_level, katello-service restart should succeed, observe ssh logs in var/log/foreman-proxy/proxy.log

Comment 8 Peter Ondrejka 2018-06-04 12:06:11 UTC
Verified on 6.4 snap 5

Comment 9 Bryan Kearney 2018-10-16 18:57:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

Note You need to log in before you can comment on or make changes to this bug.