Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1516487

Summary: Unable to authenticate via oauth-proxy and service account client secret on us-east-1,
Product: OpenShift Container Platform Reporter: Clayton Coleman <ccoleman>
Component: AuthAssignee: Simo Sorce <ssorce>
Status: CLOSED UPSTREAM QA Contact: Chuan Yu <chuyu>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: aos-bugs, mkhan, mrogers
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-03 17:45:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Clayton Coleman 2017-11-22 17:57:47 UTC
On us-east-1 the prometheus oauth-proxy returns a 403 server_error when the oauth dance completes against a server (in /oauth/callback).  Logging added to a local copy of oauth-proxy indicated the server returned:

2017/11/22 12:19:28 oauthproxy.go:571: [::1]:62960 Permission Denied: oauth form reported error: url.Values{"error":[]string{"server_error"}, "error_description":[]string{"The authorization server encountered an unexpected condition that prevented it from fulfilling the request."}}

although it's possible that the local copy was not configured exactly the same as production.  ca-central-1 runs the identical code and configuration and is able to successfully authenticate.  The service accounts were double checked: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"alerts"}}' '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"prometheus"}}'

and both had admitted routes.  The only known difference between ca-central-1 and us-east-1 is that us-east-1 is on 3.7.9 and ca-central is on a few week older version.

In local testing, I was unable to correctly authenticate to ca-central-1 (with the local binary) leading me to suspect that I had misconfigured the local proxy.  I was getting 

2017/11/22 12:22:50 oauthproxy.go:583: error redeeming code (client:[::1]:63071): got 400 from "" {"error":"unauthorized_client","error_description":"The client is not authorized to request a token using this method."}

but I had added https://localhost:8889

to the SA on both clusters.

Comment 1 Clayton Coleman 2017-11-22 17:58:37 UTC
This error prevents us from logging in to the devops prometheus on us-east-1, which is limiting debugging.  We *are* able to log in to a nearly identically configured prometheus on 3.7.9 on free-stg, so it's possible this is an environmental role issue.

Comment 3 Mo 2017-11-28 16:49:18 UTC
I opened 1518342 to track the client auth reaping fixes to be pushed to prod.

The following Trello cards and GH issues are tracking making this a non-issue even if online did not reap those objects:

Comment 4 Simo Sorce 2018-01-03 17:45:15 UTC
Closing as fixed.
Online has deployed fixed mgmt scripts and we have other bugs to track the better handling in future releases.