Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1100519 - Don't store the user's OpenStack credentials in Beaker
Summary: Don't store the user's OpenStack credentials in Beaker
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: scheduler
Version: develop
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: 24.0
Assignee: matt jia
QA Contact: tools-bugs
URL:
Whiteboard:
: 1360499 (view as bug list)
Depends On: 1040239
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-05-23 01:58 UTC by Nick Coghlan
Modified: 2018-02-06 00:41 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-21 18:48:29 UTC


Attachments (Terms of Use)

Description Nick Coghlan 2014-05-23 01:58:29 UTC
The initial implementation of the OpenStack integration in bug 1040239 stores the users' OpenStack credentials directly in Beaker. That's OK for experimental use with throwaway OpenStack accounts, but not acceptable in a production environment.

Instead, Beaker should likely be logging in to Keystone with a Beaker system account, and using the trust infrastructure created for OpenStack Heat and only storing a trust token rather than the raw login credentials.

As an initial UI, it seems reasonable to still collect the username/password combination once through the Beaker web UI, but use those to automate the creation of the appropriate trust delegation.

Comment 1 Dan Callaghan 2014-06-12 00:33:52 UTC
Another issue to consider here: Beaker needs to delete the instances many hours/days after they were created. At that point the user's credentials may have become invalid or been removed from Beaker (or, if using trusts, the user may have revoked the trust). However it's important that Beaker is still able to clean up the instances it created. Maybe it will *also* need its own set of privileged credentials which it can use for deletions.

Comment 2 Roman Joost 2016-07-27 05:10:04 UTC
*** Bug 1360499 has been marked as a duplicate of this bug. ***

Comment 3 Dan Callaghan 2016-07-27 07:13:17 UTC
*** Bug 1360495 has been marked as a duplicate of this bug. ***

Comment 4 matt jia 2016-08-21 23:02:32 UTC
To use the heat auth model, a user needs to provide his keystone authentication credentials in beaker. We could use a new element <openstack/> in the job xml and only store the trust id later. The approach I am looking at is:

1. create a Beaker system user on OpenStack and update beaker server.cfg to have possible the following line.

openstack.user_name=<username>
openstack.password=<password>
openstack.tenant_name=<tenant_name>
openstack.trusts_delegated_roles=<roles>

2. users specify their keystone login details in the job xml.

3. Beaker uses the keystone login details to get a token.

4. Beaker uses the token to create a trust between the user and the beaker system user, delegating a special role (or roles) as defined in the trusts_delegated_roles list in the server configuration file.

5. Beaker stores the trust id in the DB.

6. When Beaker needs to delete the instances many hours/days after they were created, it can use the trust id.

Comment 5 matt jia 2016-08-21 23:10:33 UTC
See http://hardysteven.blogspot.com.au/2014/04/heat-auth-model-updates-part-1-trusts.html for how the heat auth model works.

Comment 6 matt jia 2016-08-22 05:04:00 UTC
(In reply to matt jia from comment #4)
> 5. Beaker stores the trust id in the DB.
> 
> 6. When Beaker needs to delete the instances many hours/days after they were
> created, it can use the trust id.

Benefits of using this approach:


* no need to store the user's OpenStack credentials in Beaker which is more secure.

* a user can run beaker jobs in different places running OpenStack.

Comment 7 matt jia 2016-08-23 00:00:16 UTC
(In reply to matt jia from comment #4)
> To use the heat auth model, a user needs to provide his keystone
> authentication credentials in beaker. We could use a new element
> <openstack/> in the job xml and only store the trust id later. The approach
> I am looking at is:
> 
> 1. create a Beaker system user on OpenStack and update beaker server.cfg to
> have possible the following line.
> 
> openstack.user_name=<username>
> openstack.password=<password>
> openstack.tenant_name=<tenant_name>
> openstack.trusts_delegated_roles=<roles>

Correct this as the roles should be specified inside <openstack/>.

Comment 8 Dan Callaghan 2016-08-23 23:38:59 UTC
(In reply to matt jia from comment #6)
> * no need to store the user's OpenStack credentials in Beaker which is more
> secure.

... except if it's part of the job XML we *do* need to store it.

Our original idea was to add a new tab to the Preferences page, where the user does the Keystone delegation dance -- they type their OpenStack credentials into Beaker's web UI but then at the end of the dance, we just store the delegation token against their user account and then re-use that in future when provisioning their recipes.

This is how Heat does it, if I understood correctly (there is a one-off step for Heat to acquire a delegation token thing, then after that when you use Heat it uses the token it stored).

Can we do it that way instead?

Comment 9 Dan Callaghan 2016-08-23 23:40:32 UTC
(In reply to matt jia from comment #4)
> 4. Beaker uses the token to create a trust between the user and the beaker
> system user, delegating a special role (or roles) as defined in the
> trusts_delegated_roles list in the server configuration file.

What exactly are these roles? How does a user know what role they are supposed to type in? Why does the user need to figure that out and tell Beaker?

Wouldn't there just be some hardcoded list of roles like "ability to create instances", "ability to create networks", "ability to delete instances"... etc. Or is it something different?

Comment 10 matt jia 2016-08-24 00:05:24 UTC
(In reply to Dan Callaghan from comment #8)
> (In reply to matt jia from comment #6)
> > * no need to store the user's OpenStack credentials in Beaker which is more
> > secure.
> 
> ... except if it's part of the job XML we *do* need to store it.

If a user specifies his OpenStack account's username and password in the job xml, the beaker server will convert it to <trust_id/> when storing the job xml.
For example:

<openstack>
    <username value='foo'/>
    <password value='bar'/>
    <tenant value='beaker'/>
</openstack>

Beaker server returns the job xml as:

<openstack>
    <truest_id value='dfsffd'/>
</openstack>

> 
> Our original idea was to add a new tab to the Preferences page, where the
> user does the Keystone delegation dance -- they type their OpenStack
> credentials into Beaker's web UI but then at the end of the dance, we just
> store the delegation token against their user account and then re-use that
> in future when provisioning their recipes.
> 
> This is how Heat does it, if I understood correctly (there is a one-off step
> for Heat to acquire a delegation token thing, then after that when you use
> Heat it uses the token it stored).
> 
> Can we do it that way instead?

Hmm, I remember we discussed that there will be no web UI for asking users' OpenStack credentials. The downside of this approach is users can only run the jobs on one OpenStack server. If I want to have two recipes running on two different OpenStack servers in a job, this will not work, right?

Comment 11 matt jia 2016-08-24 00:06:18 UTC
(In reply to matt jia from comment #10)
> (In reply to Dan Callaghan from comment #8)
> > (In reply to matt jia from comment #6)
> > > * no need to store the user's OpenStack credentials in Beaker which is more
> > > secure.
> > 
> > ... except if it's part of the job XML we *do* need to store it.
> 
> If a user specifies his OpenStack account's username and password in the job
> xml, the beaker server will convert it to <trust_id/> when storing the job
> xml.
> For example:
> 
> <openstack>
>     <username value='foo'/>
>     <password value='bar'/>
>     <tenant value='beaker'/>
> </openstack>
> 
> Beaker server returns the job xml as:
> 
> <openstack>
>     <truest_id value='dfsffd'/>
> </openstack>
> 
> > 
> > Our original idea was to add a new tab to the Preferences page, where the
> > user does the Keystone delegation dance -- they type their OpenStack
> > credentials into Beaker's web UI but then at the end of the dance, we just
> > store the delegation token against their user account and then re-use that
> > in future when provisioning their recipes.
> > 
> > This is how Heat does it, if I understood correctly (there is a one-off step
> > for Heat to acquire a delegation token thing, then after that when you use
> > Heat it uses the token it stored).
> > 
> > Can we do it that way instead?
> 
> Hmm, I remember we discussed that there will be no web UI for asking users'
> OpenStack credentials. The downside of this approach is users can only run
> the jobs on one OpenStack server. If I want to have two recipes running on
> two different OpenStack servers in a job, this will not work, right?

Sorry, I mean two different projects.

Comment 12 matt jia 2016-08-24 00:12:43 UTC
(In reply to Dan Callaghan from comment #9)
> (In reply to matt jia from comment #4)
> > 4. Beaker uses the token to create a trust between the user and the beaker
> > system user, delegating a special role (or roles) as defined in the
> > trusts_delegated_roles list in the server configuration file.
> 
> What exactly are these roles? How does a user know what role they are
> supposed to type in? Why does the user need to figure that out and tell
> Beaker?

When consuming a trust, roles can be specified by the user to be granted to the trustor(the beaker system user). It is optional. I guess we could always set
impersonation = True, then the beaker system user will represent the user and do not collect the user's roles.

> 
> Wouldn't there just be some hardcoded list of roles like "ability to create
> instances", "ability to create networks", "ability to delete instances"...
> etc. Or is it something different?

The roles are set by the OpenStack admin. It could be 'admin' or 'member'..

Comment 13 Dan Callaghan 2016-08-24 00:45:59 UTC
(In reply to matt jia from comment #10)
> If a user specifies his OpenStack account's username and password in the job
> xml, the beaker server will convert it to <trust_id/> when storing the job
> xml.
> For example:
> 
> <openstack>
>     <username value='foo'/>
>     <password value='bar'/>
>     <tenant value='beaker'/>
> </openstack>
> 
> Beaker server returns the job xml as:
> 
> <openstack>
>     <truest_id value='dfsffd'/>
> </openstack>

The job XML is supposed to be round-trippable though (when you clone a job you get the same XML that you put in the first time). This would destroy that property.

> Hmm, I remember we discussed that there will be no web UI for asking users'
> OpenStack credentials. The downside of this approach is users can only run
> the jobs on one OpenStack server. If I want to have two recipes running on
> two different OpenStack servers in a job, this will not work, right?

Yeah I don't think we need to handle that scenario, at least for now. We can just assume there is a single Openstack instance (configured systemwide in Beaker's server.cfg) and each user is only in one tenant (or only wants to use one tenant for their Beaker recipes).

We can expand on that in future if need be, but I think for us right now those are reasonable limitations.

Comment 14 Dan Callaghan 2016-08-24 00:49:21 UTC
(In reply to matt jia from comment #12)
> When consuming a trust, roles can be specified by the user to be granted to
> the trustor(the beaker system user). It is optional.

Okay, but how does any old user like me know what are these roles I need to give to Beaker? Do I have to click around somewhere in OpenStack to find out what roles the admin has defined and what permissions they will grant, and then go back and tell Beaker I want to grant it those roles?

We can't expect people to figure that out and type in the role names by hand, Beaker should know what roles it needs and make that part of the request.

> I guess we could always
> set
> impersonation = True, then the beaker system user will represent the user
> and do not collect the user's roles.

Or yeah, I guess this is the blunter hammer which just gives Beaker full permission over their user account? This would be okay to start with, since it's basically equivalent to what we have now (collecting their username and password). We could refine it to request just specific roles in future.

Comment 15 matt jia 2016-09-06 01:43:57 UTC
(In reply to Dan Callaghan from comment #14)
> (In reply to matt jia from comment #12)
> > When consuming a trust, roles can be specified by the user to be granted to
> > the trustor(the beaker system user). It is optional.
> 
> Okay, but how does any old user like me know what are these roles I need to
> give to Beaker? Do I have to click around somewhere in OpenStack to find out
> what roles the admin has defined and what permissions they will grant, and
> then go back and tell Beaker I want to grant it those roles?
> 
> We can't expect people to figure that out and type in the role names by
> hand, Beaker should know what roles it needs and make that part of the
> request.
> 
> > I guess we could always
> > set
> > impersonation = True, then the beaker system user will represent the user
> > and do not collect the user's roles.
> 
> Or yeah, I guess this is the blunter hammer which just gives Beaker full
> permission over their user account? This would be okay to start with, since
> it's basically equivalent to what we have now (collecting their username and
> password). We could refine it to request just specific roles in future.

However the roles must be specified with the project id when creating a trust. As per the doc says, you cannot implicitly delegate all roles to a trustee. []1
Heat configures the roles through heat.conf and requires the user to have the pre-configured roles in the OpenStack Identity service.[2]. I guess we could do the same thing in beaker-server.conf.

[1] http://developer.openstack.org/api-ref/identity/v3-ext/
[2] http://docs.openstack.org/admin-guide/orchestration-auth-model.html

Comment 16 matt jia 2016-09-06 01:51:12 UTC
Another problem I am facing now is the Keystone Trusts is a feature in the Identity API v3 which requires python-keystoneclient >= 1.6.0(I get this requirement from heat). We have python-keystoneclient-0.7.1-2.el6eng on RHEL6 and it does not support API v3 really well. We could build 1.7.2-1 but it requires a lot work to build the dependencies.

Comment 17 Dan Callaghan 2016-09-06 02:10:19 UTC
(In reply to matt jia from comment #16)

There is probably a RHOS channel we can consume instead of building this stuff ourselves. Does RHOS7 have a sufficiently new version of keystoneclient?

Comment 19 matt jia 2016-09-07 05:20:02 UTC
(In reply to matt jia from comment #15)
> (In reply to Dan Callaghan from comment #14)
> > (In reply to matt jia from comment #12)
> > > When consuming a trust, roles can be specified by the user to be granted to
> > > the trustor(the beaker system user). It is optional.
> > 
> > Okay, but how does any old user like me know what are these roles I need to
> > give to Beaker? Do I have to click around somewhere in OpenStack to find out
> > what roles the admin has defined and what permissions they will grant, and
> > then go back and tell Beaker I want to grant it those roles?
> > 
> > We can't expect people to figure that out and type in the role names by
> > hand, Beaker should know what roles it needs and make that part of the
> > request.
> > 
> > > I guess we could always
> > > set
> > > impersonation = True, then the beaker system user will represent the user
> > > and do not collect the user's roles.
> > 
> > Or yeah, I guess this is the blunter hammer which just gives Beaker full
> > permission over their user account? This would be okay to start with, since
> > it's basically equivalent to what we have now (collecting their username and
> > password). We could refine it to request just specific roles in future.
> 
> However the roles must be specified with the project id when creating a
> trust. As per the doc says, you cannot implicitly delegate all roles to a
> trustee. []1
> Heat configures the roles through heat.conf and requires the user to have
> the pre-configured roles in the OpenStack Identity service.[2]. I guess we
> could do the same thing in beaker-server.conf.
> 
> [1] http://developer.openstack.org/api-ref/identity/v3-ext/
> [2] http://docs.openstack.org/admin-guide/orchestration-auth-model.html

After digging into the source code of Heat, I found a way to pass all the roles when creating a trust. As a result, we donnot need this approach.

Comment 20 matt jia 2016-09-09 01:47:48 UTC
To get python-novaclient support Keystone Identity API V3, we need to have

 python-novaclient >= 2.20 and python-keystoneclient >= 0.11

However, in the RH6-RHOS-5.0 repo, we only have python-novaclient-2.17.0-4 and python-keystoneclient-0.9.0-6. Both need to be updated in favour of using OpenStack integration on RHEL6.

Comment 21 matt jia 2016-09-09 03:48:39 UTC
(In reply to matt jia from comment #20)
> To get python-novaclient support Keystone Identity API V3, we need to have
> 
>  python-novaclient >= 2.20 and python-keystoneclient >= 0.11
> 
> However, in the RH6-RHOS-5.0 repo, we only have python-novaclient-2.17.0-4
> and python-keystoneclient-0.9.0-6. Both need to be updated in favour of
> using OpenStack integration on RHEL6.

The python-neutronclient also needs to be updated to 2.3.9-1.

Comment 22 Dan Callaghan 2016-09-12 05:55:15 UTC
So the problem is RHOS5 is too old, but RDO/CentOS only publishes RHOS5 for RHEL6, no newer RHOS releases?

Comment 23 matt jia 2016-09-13 03:29:50 UTC
For CentOS 6 users, people can get those required packages from 

https://wiki.centos.org/Cloud/OpenStack/JunoEL6QuickStart

We could use this repo for our beaker production as well.

Comment 24 matt jia 2016-09-13 03:30:27 UTC
(In reply to Dan Callaghan from comment #22)
> So the problem is RHOS5 is too old, but RDO/CentOS only publishes RHOS5 for
> RHEL6, no newer RHOS releases?

Right, there is no RHOS 7 for RHEL 6.

Comment 25 matt jia 2016-10-24 04:01:40 UTC
On Gerrit:

https://gerrit.beaker-project.org/#/c/5243/(use Keystone Trusts for OpenStack authentication)
https://gerrit.beaker-project.org/#/c/5252/15(HTTP APIs for creating/removing user Keystone trusts)
https://gerrit.beaker-project.org/#/c/5270/14(prefs page: add UI for editing Keystone trusts )

Comment 26 matt jia 2016-10-24 04:04:45 UTC
(In reply to matt jia from comment #25)
> On Gerrit:
> 
> https://gerrit.beaker-project.org/#/c/5243/(use Keystone Trusts for
> OpenStack authentication)
> https://gerrit.beaker-project.org/#/c/5252/15(HTTP APIs for
> creating/removing user Keystone trusts)
> https://gerrit.beaker-project.org/#/c/5270/14(prefs page: add UI for editing
> Keystone trusts )

Correct the last URL:

https://gerrit.beaker-project.org/#/c/5252(HTTP APIs for creating/removing user Keystone trusts)

https://gerrit.beaker-project.org/#/c/5270(prefs page: add UI for editing Keystone trusts )

Comment 29 Dan Callaghan 2017-02-21 18:48:29 UTC
Beaker 24.0 has been released.


Note You need to log in before you can comment on or make changes to this bug.