Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1510212 - running oc adm diagnostics AggregatedLogging failed to launch if run from my localhost
Summary: running oc adm diagnostics AggregatedLogging failed to launch if run from my ...
Keywords:
Status: ASSIGNED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Command Line Interface
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 3.9.z
Assignee: Luke Meyer
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-06 23:32 UTC by Peter Ruan
Modified: 2019-02-13 23:24 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: The AggregatedLogging diagnostic now looks for logging deployments only via the API, rather than inspecting the master config for a logging URL. If logging is deployed in a non-standard project, that can be specified with a flag to the diagnostic. Reason: With OCP 3.9, the master config will no longer even supply this value. This forced the issue. In order to run the diagnostic, we needed to assume (or explicitly specify) where logging is deployed. Result: This diagnostic will run assuming logging is deployed in the project specified with --logging-project flag, or against the 'openshift-logging' or 'logging' projects if they exist. If none apply, it is skipped. The master config is not consulted, so the diagnostic can run from a pure client host.
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Peter Ruan 2017-11-06 23:32:57 UTC
Description of problem:
running oc adm diagnostics AggregatedLogging failed to launch if run from my localhost even though I specify the same config file which is SCPed over from the master.

oc adm diagnostics  --config=ose_admin.kubeconfig AggregatedLogging

Version-Release number of selected component (if applicable):


How reproducible:
always.

Steps to Reproduce:
1. have logging deployed
2. copy the /root/.kube/config file to localhost
3. oc adm diagnostics --config=config AggregatedLogging 


Actual results:
pruan@eos ~/junk $ oc adm diagnostics --config=config AggregatedLogging                      [ruby-2.4.1]
[Note] Determining if client configuration exists for client/cluster diagnostics
Info:  Successfully read a client config file at 'config'
Info:  Using context for cluster-admin access: 'default/openshift-123-lab-sjc-redhat-com:8443/system:admin'

[Note] Skipping diagnostic: AggregatedLogging
       Description: Check aggregated logging integration for proper configuration
       Because: No master config file was provided
       
[Note] Summary of diagnostics execution (version v3.6.173.0.65):
[Note] Completed with no errors or warnings seen.


Expected results:
When the same command is specified with the same config feed, the test is able to launch.

[root@openshift-123 ~]# oc adm diagnostics  --config=ose_admin.kubeconfig AggregatedLogging
[Note] Determining if client configuration exists for client/cluster diagnostics
Info:  Successfully read a client config file at 'ose_admin.kubeconfig'
Info:  Successfully read a client config file at '/root/.kube/config'
Info:  Using context for cluster-admin access: 'default/openshift-123-lab-sjc-redhat-com:8443/system:admin'

[Note] Running diagnostic: AggregatedLogging
       Description: Check aggregated logging integration for proper configuration
       
Info:  Found route 'logging-kibana' matching logging URL 'logs.apps.1106-o70.qe.rhcloud.com' in project: 'e31gp'

Info:  Did not find a DeploymentConfig to support optional component 'curator-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'es-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'kibana-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'mux'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Looked for 'logging-es-ops' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-es-ops-cluster' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-kibana-ops' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-mux' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
[Note] Summary of diagnostics execution (version v3.6.173.0.65):
[Note] Completed with no errors or warnings seen.


Additional info:

Comment 1 Luke Meyer 2017-11-10 14:45:33 UTC
So the reason it gives for not running is:

[Note] Skipping diagnostic: AggregatedLogging
       Description: Check aggregated logging integration for proper configuration
       Because: No master config file was provided

Obviously on your localhost the master-config.yaml is not available (though you could specify it with --master-config flag if you have it).

It looks like you ran the second command on the master, so the master config would automatically be found there.

So I'm tempted to say this is NOTABUG. However, this is understandably confusing. It comes back to the old question of "how do we know if logging is *supposed* to be deployed?" I have argued in the past that oadm diagnostics doesn't have any legitimate way to determine that (since it could theoretically be deployed in any project name, and the `logging` project could be used for something else entirely, and there is nothing that definitively identifies the logging project that couldn't be applied by a user in any other project), aside from looking in the master config to see if a URL points at it. All the same, it might be a good idea to apply a heuristic to find the logging project strictly using the API. Or a better approach might be to supply a parameter by which the diagnostics user can say "I really expect the `logging` project (or some other name) to be present and have logging deployed in it".

Comment 2 Luke Meyer 2018-02-23 12:24:24 UTC
The problem of "how do we know if logging is *supposed* to be deployed?" is being addressed in 3.9 via https://github.com/openshift/origin/pull/18437 and followup https://github.com/openshift/origin/pull/18714

Comment 6 Junqi Zhao 2018-04-19 07:01:07 UTC
Steps to Verify:
1. have logging deployed
2. copy the /root/.kube/config file to localhost
3. oc adm diagnostics --config=config AggregatedLogging 

Should see the following outputs
**************************************************************************
[Note] Determining if client configuration exists for client/cluster diagnostics
Info:  Successfully read a client config file at '/root/.kube/config'
Info:  Using context for cluster-admin access: 'default/172-16-120-10:8443/system:admin'

[Note] Running diagnostic: AggregatedLogging
       Description: Check aggregated logging integration for proper configuration
       
Info:  Did not find a DeploymentConfig to support optional component 'curator-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'es-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'kibana-ops'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Did not find a DeploymentConfig to support optional component 'mux'. If you require
       this component, please re-install or update logging and specify the appropriate
       variable to enable it.
       
Info:  Looked for 'logging-es-ops' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-es-ops-cluster' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-kibana-ops' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
Info:  Looked for 'logging-mux' among the logging services for the project but did not find it.
       This optional component may not have been specified by logging install options.
       
[Note] Summary of diagnostics execution (version v3.9.22):
[Note] Completed with no errors or warnings seen.

**************************************************************************

but get the following outputs
**************************************************************************
$ oc adm diagnostics --config=/home/juzhao/config AggregatedLogging
[Note] Determining if client configuration exists for client/cluster diagnostics
Info:  Successfully read a client config file at '/home/juzhao/config'

ERROR: [CED1008 from controller openshift/origin/pkg/oc/admin/diagnostics/cluster.go]
       Unknown error testing cluster-admin access for context 'default/172-16-120-10:8443/system:admin':
       Post https://172.16.120.10:8443/apis/authorization.k8s.io/v1/selfsubjectaccessreviews: dial tcp 172.16.120.10:8443: i/o timeout
       
[Note] Could not configure a client with cluster-admin permissions for the current server, so cluster diagnostics will be skipped

ERROR: [CED3001 from controller openshift/origin/pkg/oc/admin/diagnostics/util/util.go]
       Encountered fatal error while building diagnostics: Requested diagnostic(s) skipped; nothing to run. See --help and consider setting flags or providing config to enable running.
       
[Note] Summary of diagnostics execution (version v3.9.22):
[Note] Errors seen: 2
**************************************************************************

# openshift version
openshift v3.9.22
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.16

Comment 8 Junqi Zhao 2018-04-27 05:55:16 UTC
Changed to ON_QA by errata, assign it back


Note You need to log in before you can comment on or make changes to this bug.