Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1691191 - Can join a cluster to the federation multi times
Summary: Can join a cluster to the federation multi times
Keywords:
Status: ASSIGNED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Federation
Version: 4.1
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.1.0
Assignee: Paul Morie
QA Contact: Qin Ping
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-21 05:39 UTC by Qin Ping
Modified: 2019-04-11 01:55 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Description Qin Ping 2019-03-21 05:39:07 UTC
Description of problem:
Can join a cluster to the federation multi times


Version-Release number of selected component (if applicable):
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.0.0-0.nightly-2019-03-19-004004   True        False         26h     Cluster version is 4.0.0-0.nightly-2019-03-19-004004

kubefed2 version: version.Info{Version:"0.0.6", GitCommit:"de6a0a909418f5ddf2d04d232b0ca55aa9cffb49", GitTreeState:"clean", BuildDate:"2019-03-14T00:43:37Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Federation v2 controller-manager version: version.Info{Version:"0.0.6", GitCommit:"de6a0a909418f5ddf2d04d232b0ca55aa9cffb49", GitTreeState:"clean", BuildDate:"2019-03-14T00:43:37Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

How reproducible:
100%

Steps to Reproduce:
1. Install a namespace-scoped federationv2 control plane
2. Join cluster with clustername "cluster1" to federation with cmd: kubefed2 join cluster1 --host-cluster-context=host-admin --cluster-context=cluster1-admin --federation-namespace=federation-llng --registry-namespace=federation-llng --add-to-registry --limited-scope=true
3. Join the same cluster with clustername "cluster2" to federation with cmd: kubefed2 join cluster2 --host-cluster-context=host-admin --cluster-context=cluster1-admin --federation-namespace=federation-llng --registry-namespace=federation-llng --add-to-registry --limited-scope=true


Actual results:
Join cluster to federation successfully.

Expected results:
The same cluster can not join federation multi times.

Additional info:
$ oc config get-contexts
CURRENT   NAME                  CLUSTER                                             AUTHINFO                                                       NAMESPACE
          cluster1-admin        cluster1-openshift-com:6443   testuser-0/cluster1-openshift-com:6443   default
          cluster1-testuser-1   cluster1-openshift-com:6443   testuser-1/cluster1-openshift-com:6443   
*         host-admin            cluster1-openshift-com:6443   testuser-0/cluster1-openshift-com:6443   default
          host-testuser-1       cluster1-openshift-com:6443   testuser-1/cluster1-openshift-com:6443


$ oc describe cluster -n federation-llng|sed 's/api.*:6443/cluster1.openshift.com:6443/'
Name:         cluster1
Namespace:    federation-llng
Labels:       <none>
Annotations:  <none>
API Version:  clusterregistry.k8s.io/v1alpha1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2019-03-21T05:13:57Z
  Generation:          1
  Resource Version:    1138845
  Self Link:           /apis/clusterregistry.k8s.io/v1alpha1/namespaces/federation-llng/clusters/cluster1
  UID:                 2119b2e5-4b98-11e9-8d00-06ff955a0906
Spec:
  Auth Info:
  Kubernetes API Endpoints:
    Server Endpoints:
      Client CIDR:     0.0.0.0/0
      Server Address:  https://cluster1.openshift.com:6443
Status:
Events:  <none>


Name:         cluster2
Namespace:    federation-llng
Labels:       <none>
Annotations:  <none>
API Version:  clusterregistry.k8s.io/v1alpha1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2019-03-21T05:15:44Z
  Generation:          1
  Resource Version:    1139957
  Self Link:           /apis/clusterregistry.k8s.io/v1alpha1/namespaces/federation-llng/clusters/cluster2
  UID:                 61393f00-4b98-11e9-8d00-06ff955a0906
Spec:
  Auth Info:
  Kubernetes API Endpoints:
    Server Endpoints:
      Client CIDR:     0.0.0.0/0
      Server Address:  https://cluster1.openshift.com:6443
Status:
Events:  <none>

$ oc describe federatedcluster -n federation-llng
Name:         cluster1
Namespace:    federation-llng
Labels:       <none>
Annotations:  <none>
API Version:  core.federation.k8s.io/v1alpha1
Kind:         FederatedCluster
Metadata:
  Creation Timestamp:  2019-03-21T05:14:03Z
  Generation:          1
  Resource Version:    1152032
  Self Link:           /apis/core.federation.k8s.io/v1alpha1/namespaces/federation-llng/federatedclusters/cluster1
  UID:                 24aaa39f-4b98-11e9-8d00-06ff955a0906
Spec:
  Cluster Ref:
    Name:  cluster1
  Secret Ref:
    Name:  cluster1-8b8pr
Status:
  Conditions:
    Last Probe Time:       2019-03-21T05:36:27Z
    Last Transition Time:  2019-03-21T05:14:25Z
    Message:               /healthz responded with ok
    Reason:                ClusterReady
    Status:                True
    Type:                  Ready
  Region:                  us-east-2
  Zone:                    us-east-2a
Events:                    <none>


Name:         cluster2
Namespace:    federation-llng
Labels:       <none>
Annotations:  <none>
API Version:  core.federation.k8s.io/v1alpha1
Kind:         FederatedCluster
Metadata:
  Creation Timestamp:  2019-03-21T05:15:51Z
  Generation:          1
  Resource Version:    1152034
  Self Link:           /apis/core.federation.k8s.io/v1alpha1/namespaces/federation-llng/federatedclusters/cluster2
  UID:                 651cc026-4b98-11e9-8d00-06ff955a0906
Spec:
  Cluster Ref:
    Name:  cluster2
  Secret Ref:
    Name:  cluster2-ncb76
Status:
  Conditions:
    Last Probe Time:       2019-03-21T05:36:27Z
    Last Transition Time:  2019-03-21T05:16:25Z
    Message:               /healthz responded with ok
    Reason:                ClusterReady
    Status:                True
    Type:                  Ready
  Region:                  us-east-2
  Zone:                    us-east-2a
Events:                    <none>

Comment 1 Paul Morie 2019-03-21 15:28:52 UTC
This is correct and expected; for example, a user may want to use the same cluster with different service accounts depending on the context. Does that make sense? If so, I think this would be good to capture in doc somewhere, possibly in the CSV description.

Comment 2 Qin Ping 2019-03-22 00:34:27 UTC
If I understand correctly, there are 2 types of users. One type of users are the end users, they create federated objects; the second type of users are service accounts created by `kubefed2 join` command, they respond to distribute non-federated objects to multiple clusters per federated objects.

When joining a cluster to a federation multiple times, multiple the second type of users are created. I think this is unnecessary, one the second type of user is enough, he can distribute all the federated objects created by multiple end users.

Comment 3 Qin Ping 2019-04-02 09:10:24 UTC
If a cluster can join a federation multiple times, the openshift objects created by federationv2 controller manager will keep deleting and creating.

$ oc get svc -w
NAME           TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
test-service   NodePort   172.30.144.175   <none>        80:31173/TCP   8s
test-service   NodePort   172.30.144.175   <none>   80:31173/TCP   11s
test-service   NodePort   172.30.70.8   <none>   80:31959/TCP   1s
test-service   NodePort   172.30.70.8   <none>   80:31959/TCP   11s
test-service   NodePort   172.30.106.99   <none>   80:31827/TCP   1s
test-service   NodePort   172.30.106.99   <none>   80:31827/TCP   11s

Comment 4 Paul Morie 2019-04-02 11:50:45 UTC
> When joining a cluster to a federation multiple times, multiple the second type of users are created. I think this is unnecessary, one the second type of user is enough, he can distribute all the federated objects created by multiple end users.

In the long term, users will likely demand a 'least privilege' scheme for permissioning service accounts that federation uses. It's valid to want to join the same cluster multiple times using different service accounts.

I'm having trouble seeing the bug behavior here, can you elaborate on exactly what that is?

Comment 5 Qin Ping 2019-04-03 01:46:34 UTC
Hi Paul,

I increased the bug to high priority cause if we add a cluster to a federation multiple time(uses the name cluster1 and cluster2), then we create a federatedservice object "test-service", it's spec.placement.clusterNames is cluster1(not include cluster2), then federation v2 controller manager will create test-service service object for cluster1, and delete test-service service object for cluster2, so in the cluster we will see the test-service service object keeps be deleted and created.

Comment 8 Maru Newby 2019-04-10 22:33:10 UTC
I've filed an upstream issue to raise the visibility of the problem: https://github.com/kubernetes-sigs/federation-v2/issues/749

This problem was accidentally introduced when join became idempotent: https://github.com/kubernetes-sigs/federation-v2/issues/555

Previous to join idempotency, it was not possible to join a cluster multiple times.


Note You need to log in before you can comment on or make changes to this bug.