Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1516487 - Unable to authenticate via oauth-proxy and service account client secret on us-east-1,
Summary: Unable to authenticate via oauth-proxy and service account client secret on u...
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Auth
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Simo Sorce
QA Contact: Chuan Yu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-22 17:57 UTC by Clayton Coleman
Modified: 2019-03-29 15:48 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-01-03 17:45:15 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Clayton Coleman 2017-11-22 17:57:47 UTC
On us-east-1 the prometheus oauth-proxy returns a 403 server_error when the oauth dance completes against a server (in /oauth/callback).  Logging added to a local copy of oauth-proxy indicated the server returned:

2017/11/22 12:19:28 oauthproxy.go:571: [::1]:62960 Permission Denied: oauth form reported error: url.Values{"error":[]string{"server_error"}, "error_description":[]string{"The authorization server encountered an unexpected condition that prevented it from fulfilling the request."}}

although it's possible that the local copy was not configured exactly the same as production.  ca-central-1 runs the identical code and configuration and is able to successfully authenticate.  The service accounts were double checked:

    serviceaccounts.openshift.io/oauth-redirectreference.alerts: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"alerts"}}'
    serviceaccounts.openshift.io/oauth-redirectreference.prom: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"prometheus"}}'

and both had admitted routes.  The only known difference between ca-central-1 and us-east-1 is that us-east-1 is on 3.7.9 and ca-central is on a few week older version.

In local testing, I was unable to correctly authenticate to ca-central-1 (with the local binary) leading me to suspect that I had misconfigured the local proxy.  I was getting 

2017/11/22 12:22:50 oauthproxy.go:583: error redeeming code (client:[::1]:63071): got 400 from "https://api.starter-ca-central-1.openshift.com/oauth/token" {"error":"unauthorized_client","error_description":"The client is not authorized to request a token using this method."}

but I had added 

    serviceaccounts.openshift.io/oauth-redirecturi.test: https://localhost:8889

to the SA on both clusters.

Comment 1 Clayton Coleman 2017-11-22 17:58:37 UTC
This error prevents us from logging in to the devops prometheus on us-east-1, which is limiting debugging.  We *are* able to log in to a nearly identically configured prometheus on 3.7.9 on free-stg, so it's possible this is an environmental role issue.

Comment 3 Mo 2017-11-28 16:49:18 UTC
I opened 1518342 to track the client auth reaping fixes to be pushed to prod.

The following Trello cards and GH issues are tracking making this a non-issue even if online did not reap those objects:

https://trello.com/c/Tcabgp9F
https://trello.com/c/NEF8BnND
https://trello.com/c/PSjGN1LS
https://github.com/openshift/origin/issues/15119
https://github.com/openshift/origin/issues/15120

Comment 4 Simo Sorce 2018-01-03 17:45:15 UTC
Closing as fixed.
Online has deployed fixed mgmt scripts and we have other bugs to track the better handling in future releases.


Note You need to log in before you can comment on or make changes to this bug.