Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1509919 - Mediawiki main page can't be visited after PostgreSQL plan upgrade
Summary: Mediawiki main page can't be visited after PostgreSQL plan upgrade
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.9.0
Assignee: Jason Montleon
QA Contact: Qixuan Wang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-06 10:50 UTC by Qixuan Wang
Modified: 2018-06-18 18:26 UTC (History)
4 users (show)

Fixed In Version: openshift v3.9.0-0.19.0
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2018-06-13 14:40:31 UTC


Attachments (Terms of Use)

Description Qixuan Wang 2017-11-06 10:50:55 UTC
Description of problem:
Provision Mediawiki and PostgreSQL in a namespace, create postgresql binding and add to mediawiki, the mediawiki main page can be visited. Upgrade postgresql plan from dev to prod, the main page can't be visited, web shows "A database query error has occurred. This may indicate a bug in the software."


Version-Release number of selected component (if applicable):
openshift v3.7.0-0.191.0
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-service-catalog:v3.7.0-0.194.0.0
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-ansible-service-broker:v3.7.0-0.194.0.0


How reproducible:
Always

Steps to Reproduce:
1. Provision a Mediawiki and PostgreSQL APB in the same project on web UI.
2. Create bind to inject credentials add to Mediawiki. 
3. Access http://mediawiki123-qwang2.apps.1106-n6r.qe.rhcloud.com
4. Update PostgreSQL ServiceInstance plan from dev to prod
5. Access route again.
6. Check ClusterServiceBroker, ServiceInstance, APB pods.


Actual results:
3. Can access Mediawiki main page
4. Update successfully. Pods run again.
5. Can't access Mediawiki. Web shows "A database query error has occurred. This may indicate a bug in the software."
6. ServiceInstance: PostgreSQL (InstanceUpdatedSuccessfully). 
Mediawiki ( ProvisionedSuccessfully).
APB pods run.
Successfully fetched catalog entries from broker.
[root@host-172-16-120-50 ~]# oc describe serviceinstance rh-rhscl-postgresql-apb-m5g2l
Name:		rh-rhscl-postgresql-apb-m5g2l
Namespace:	qwang2
Labels:		<none>
Annotations:	<none>
API Version:	servicecatalog.k8s.io/v1beta1
Kind:		ServiceInstance
Metadata:
  Creation Timestamp:	2017-11-06T10:20:34Z
  Finalizers:
    kubernetes-incubator/service-catalog
  Generate Name:	rh-rhscl-postgresql-apb-
  Generation:		2
  Resource Version:	12812
  Self Link:		/apis/servicecatalog.k8s.io/v1beta1/namespaces/qwang2/serviceinstances/rh-rhscl-postgresql-apb-m5g2l
  UID:			20154c87-c2dc-11e7-b68b-0a580a810003
Spec:
  Cluster Service Class External Name:	rh-rhscl-postgresql-apb
  Cluster Service Class Ref:
    Name:				29ed8bc1f3a28e1b4bf4ab87c6c9b728
  Cluster Service Plan External Name:	prod
  Cluster Service Plan Ref:
    Name:	82c9bfe3fe8b5a6939af33eac15ec0cc
  External ID:	eff76ca8-e3bf-4648-9172-79364ac744f2
  Parameters From:
    Secret Key Ref:
      Key:		parameters
      Name:		rh-rhscl-postgresql-apb-parameters633it
  Update Requests:	0
  User Info:
    Groups:
      system:cluster-admins
      system:authenticated
    UID:	
    Username:	system:admin
Status:
  Async Op In Progress:	false
  Conditions:
    Last Transition Time:	2017-11-06T10:27:25Z
    Message:			The instance was updated successfully
    Reason:			InstanceUpdatedSuccessfully
    Status:			True
    Type:			Ready
  Deprovision Status:		Required
  External Properties:
    Cluster Service Plan External ID:	82c9bfe3fe8b5a6939af33eac15ec0cc
    Cluster Service Plan External Name:	prod
    Parameter Checksum:			72273327c9c404feee442eb4079b477f53a29027f1b73bec90e54e8e72061e3f
    Parameters:
      Postgresql _ Database:	<redacted>
      Postgresql _ Password:	<redacted>
      Postgresql _ User:	<redacted>
      Postgresql _ Version:	<redacted>
    User Info:
      Groups:
        system:cluster-admins
        system:authenticated
      UID:				
      Username:				system:admin
  Orphan Mitigation In Progress:	false
  Reconciled Generation:		2
Events:
  FirstSeen	LastSeen	Count	From					SubObjectPath	Type		Reason				Message
  ---------	--------	-----	----					-------------	--------	------				-------
  27m		27m		1	service-catalog-controller-manager			Warning		ErrorWithParameters		Failed to prepare ServiceInstance parameters nil: secrets "rh-rhscl-postgresql-apb-parameters633it" not found
  27m		27m		1	service-catalog-controller-manager			Normal		Provisioning			The instance is being provisioned asynchronously
  27m		27m		1	service-catalog-controller-manager			Normal		ProvisionedSuccessfully		The instance was provisioned successfully
  21m		21m		1	service-catalog-controller-manager			Normal		UpdatingInstance		The instance is being updated asynchronously
  20m		20m		1	service-catalog-controller-manager			Normal		InstanceUpdatedSuccessfully	The instance was updated successfully

[root@host-172-16-120-50 ~]# oc describe serviceinstance rh-mediawiki123-apb-dmd2w
Name:		rh-mediawiki123-apb-dmd2w
Namespace:	qwang2
Labels:		<none>
Annotations:	<none>
API Version:	servicecatalog.k8s.io/v1beta1
Kind:		ServiceInstance
Metadata:
  Creation Timestamp:	2017-11-06T10:20:06Z
  Finalizers:
    kubernetes-incubator/service-catalog
  Generate Name:	rh-mediawiki123-apb-
  Generation:		1
  Resource Version:	11852
  Self Link:		/apis/servicecatalog.k8s.io/v1beta1/namespaces/qwang2/serviceinstances/rh-mediawiki123-apb-dmd2w
  UID:			0f20fc77-c2dc-11e7-b68b-0a580a810003
Spec:
  Cluster Service Class External Name:	rh-mediawiki123-apb
  Cluster Service Class Ref:
    Name:				740897399fff88869106c8d7cdfd3997
  Cluster Service Plan External Name:	default
  Cluster Service Plan Ref:
    Name:	6cdcb34d5f8165244322d12d14e3719d
  External ID:	e3fd5385-0515-4d60-8e97-9db433d8d268
  Parameters From:
    Secret Key Ref:
      Key:		parameters
      Name:		rh-mediawiki123-apb-parameters2c27q
  Update Requests:	0
  User Info:
    Extra:
      Scopes . Authorization . Openshift . Io:
        user:full
    Groups:
      system:authenticated:oauth
      system:authenticated
    UID:	
    Username:	qwang
Status:
  Async Op In Progress:	false
  Conditions:
    Last Transition Time:	2017-11-06T10:21:08Z
    Message:			The instance was provisioned successfully
    Reason:			ProvisionedSuccessfully
    Status:			True
    Type:			Ready
  Deprovision Status:		Required
  External Properties:
    Cluster Service Plan External ID:	6cdcb34d5f8165244322d12d14e3719d
    Cluster Service Plan External Name:	default
    Parameter Checksum:			983e9b190fbc3bf20214a9c0921784b40222fee5025f8b8a6bb5b780a00da683
    Parameters:
      Mediawiki _ Admin _ Pass:	<redacted>
      Mediawiki _ Admin _ User:	<redacted>
      Mediawiki _ Db _ Schema:	<redacted>
      Mediawiki _ Site _ Lang:	<redacted>
      Mediawiki _ Site _ Name:	<redacted>
    User Info:
      Extra:
        Scopes . Authorization . Openshift . Io:
          user:full
      Groups:
        system:authenticated:oauth
        system:authenticated
      UID:				
      Username:				qwang
  Orphan Mitigation In Progress:	false
  Reconciled Generation:		1
Events:
  FirstSeen	LastSeen	Count	From					SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----					-------------	--------	------			-------
  27m		27m		1	service-catalog-controller-manager			Warning		ErrorWithParameters	Failed to prepare ServiceInstance parameters nil: secrets "rh-mediawiki123-apb-parameters2c27q" not found
  27m		27m		1	service-catalog-controller-manager			Normal		Provisioning		The instance is being provisioned asynchronously
  26m		26m		1	service-catalog-controller-manager			Normal		ProvisionedSuccessfully	The instance was provisioned successfully


[root@host-172-16-120-50 ~]# oc get all
NAME                             REVISION   DESIRED   CURRENT   TRIGGERED BY
deploymentconfigs/mediawiki123   2          1         1         config
deploymentconfigs/postgresql     2          1         1         config

NAME                  HOST/PORT                                          PATH      SERVICES       PORT      TERMINATION   WILDCARD
routes/mediawiki123   mediawiki123-qwang2.apps.1106-n6r.qe.rhcloud.com             mediawiki123   web                     None

NAME                      READY     STATUS    RESTARTS   AGE
po/mediawiki123-2-bq8pp   1/1       Running   0          21m
po/postgresql-2-9zlbx     1/1       Running   0          17m

NAME                DESIRED   CURRENT   READY     AGE
rc/mediawiki123-1   0         0         0         23m
rc/mediawiki123-2   1         1         1         21m
rc/postgresql-1     0         0         0         23m
rc/postgresql-2     1         1         1         17m

NAME               CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
svc/mediawiki123   172.30.148.209   <none>        8080/TCP   23m
svc/postgresql     172.30.71.70     <none>        5432/TCP   23m


[root@host-172-16-120-50 ~]# oc describe clusterservicebroker ansible-service-broker 
Name:		ansible-service-broker
Namespace:	
Labels:		<none>
Annotations:	<none>
API Version:	servicecatalog.k8s.io/v1beta1
Kind:		ClusterServiceBroker
Metadata:
  Creation Timestamp:	2017-11-06T09:16:15Z
  Finalizers:
    kubernetes-incubator/service-catalog
  Generation:		1
  Resource Version:	15019
  Self Link:		/apis/servicecatalog.k8s.io/v1beta1/clusterservicebrokers/ansible-service-broker
  UID:			23aa4034-c2d3-11e7-b68b-0a580a810003
Spec:
  Auth Info:
    Bearer:
      Secret Ref:
        Name:		asb-client
        Namespace:	openshift-ansible-service-broker
  Ca Bundle:		xxx
  Relist Behavior:	Duration
  Relist Duration:	15m0s
  Relist Requests:	0
  URL:			https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker
Status:
  Conditions:
    Last Transition Time:	2017-11-06T09:18:12Z
    Message:			Successfully fetched catalog entries from broker.
    Reason:			FetchedCatalog
    Status:			True
    Type:			Ready
  Last Catalog Retrieval Time:	2017-11-06T10:46:09Z
  Reconciled Generation:	1
Events:
  FirstSeen	LastSeen	Count	From					SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----					-------------	--------	------			-------
  1h		1h		14	service-catalog-controller-manager			Warning		ErrorFetchingCatalog	Error getting broker catalog: Get https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog: dial tcp 172.30.160.184:1338: getsockopt: no route to host
  1h		3m		6	service-catalog-controller-manager			Normal		FetchedCatalog		Successfully fetched catalog entries from broker.




Expected results:
5. Mediawiki main page can be visited after PostgreSQL plan upgrade.


Additional info:
[2017-11-06T10:20:06.397Z] [INFO] ASYNC provisioning in progress
[2017-11-06T10:20:06.397Z] [NOTICE] ============================================================
[2017-11-06T10:20:06.397Z] [NOTICE]                        PROVISIONING                         
[2017-11-06T10:20:06.397Z] [NOTICE] ============================================================
[2017-11-06T10:20:06.397Z] [NOTICE] Spec.ID: 740897399fff88869106c8d7cdfd3997
[2017-11-06T10:20:06.397Z] [NOTICE] Spec.Name: rh-mediawiki123-apb
[2017-11-06T10:20:06.397Z] [NOTICE] Spec.Image: docker.io/ansibleplaybookbundle/mediawiki123-apb:latest
[2017-11-06T10:20:06.397Z] [NOTICE] Spec.Description: Mediawiki123 apb implementation
[2017-11-06T10:20:06.397Z] [NOTICE] ============================================================
[2017-11-06T10:20:06.397Z] [INFO] Checking if project qwang2 exists...
10.129.0.1 - - [06/Nov/2017:10:20:06 +0000] "PUT /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268?accepts_incomplete=true HTTP/1.1" 202 58
[2017-11-06T10:20:06.601Z] [INFO] Provision requested for instance e3fd5385-0515-4d60-8e97-9db433d8d268, but job is already in progress
10.129.0.1 - - [06/Nov/2017:10:20:06 +0000] "PUT /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268?accepts_incomplete=true HTTP/1.1" 202 58
[2017-11-06T10:20:06.671Z] [INFO] Successfully wrote resources to /tmp/asb-resource-files/apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7.yaml
10.129.0.1 - - [06/Nov/2017:10:20:06 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 29
[2017-11-06T10:20:06.915Z] [INFO] Successfully created apb sandbox: [ apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 ], with edit permissions in namespace rh-mediawiki123-apb-prov-kqq8s
[2017-11-06T10:20:06.915Z] [NOTICE] Creating pod "apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7" in the rh-mediawiki123-apb-prov-kqq8s namespace
[2017-11-06T10:20:07.12Z] [INFO] error: unable to upgrade connection: container not found ("apb")

[2017-11-06T10:20:07.12Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 1: Failed to exec into the container
[2017-11-06T10:20:07.12Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 1: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
10.129.0.1 - - [06/Nov/2017:10:20:07 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 29
10.129.0.1 - - [06/Nov/2017:10:20:11 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 29
[2017-11-06T10:20:12.38Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:12.381Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 2: Bind credentials not available yet
[2017-11-06T10:20:12.381Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 2: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
[2017-11-06T10:20:17.637Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:17.637Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 3: Bind credentials not available yet
[2017-11-06T10:20:17.637Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 3: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
10.129.0.1 - - [06/Nov/2017:10:20:19 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 29
[2017-11-06T10:20:22.874Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:22.874Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 4: Bind credentials not available yet
[2017-11-06T10:20:22.874Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 4: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
[2017-11-06T10:20:28.144Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:28.144Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 5: Bind credentials not available yet
[2017-11-06T10:20:28.144Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 5: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
[2017-11-06T10:20:33.379Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:33.379Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 6: Bind credentials not available yet
[2017-11-06T10:20:33.38Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 6: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
[2017-11-06T10:20:34.793Z] [INFO] ASYNC provisioning in progress
[2017-11-06T10:20:34.793Z] [NOTICE] ============================================================
[2017-11-06T10:20:34.793Z] [NOTICE]                        PROVISIONING                         
[2017-11-06T10:20:34.793Z] [NOTICE] ============================================================
[2017-11-06T10:20:34.793Z] [NOTICE] Spec.ID: 29ed8bc1f3a28e1b4bf4ab87c6c9b728
[2017-11-06T10:20:34.793Z] [NOTICE] Spec.Name: rh-rhscl-postgresql-apb
[2017-11-06T10:20:34.793Z] [NOTICE] Spec.Image: docker.io/ansibleplaybookbundle/rhscl-postgresql-apb:latest
[2017-11-06T10:20:34.793Z] [NOTICE] Spec.Description: SCL PostgreSQL apb implementation
[2017-11-06T10:20:34.793Z] [NOTICE] ============================================================
[2017-11-06T10:20:34.793Z] [INFO] Checking if project qwang2 exists...
10.129.0.1 - - [06/Nov/2017:10:20:34 +0000] "PUT /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2?accepts_incomplete=true HTTP/1.1" 202 58
[2017-11-06T10:20:34.994Z] [INFO] Successfully wrote resources to /tmp/asb-resource-files/apb-605b35d3-73da-4226-b4bd-5e165262d235.yaml
[2017-11-06T10:20:34.996Z] [INFO] Provision requested for instance eff76ca8-e3bf-4648-9172-79364ac744f2, but job is already in progress
10.129.0.1 - - [06/Nov/2017:10:20:34 +0000] "PUT /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2?accepts_incomplete=true HTTP/1.1" 202 58
10.129.0.1 - - [06/Nov/2017:10:20:35 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=4f21178a-28ae-4f37-bff1-0192cc7d89d7&plan_id=fe6afa23d897dc9538d4a373f7fcb933&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:20:35.225Z] [INFO] Successfully created apb sandbox: [ apb-605b35d3-73da-4226-b4bd-5e165262d235 ], with edit permissions in namespace rh-rhscl-postgresql-apb-prov-p64nt
[2017-11-06T10:20:35.225Z] [NOTICE] Creating pod "apb-605b35d3-73da-4226-b4bd-5e165262d235" in the rh-rhscl-postgresql-apb-prov-p64nt namespace
[2017-11-06T10:20:35.438Z] [INFO] error: unable to upgrade connection: container not found ("apb")

[2017-11-06T10:20:35.438Z] [WARNING] [apb-605b35d3-73da-4226-b4bd-5e165262d235] Retry attempt 1: Failed to exec into the container
[2017-11-06T10:20:35.438Z] [WARNING] [apb-605b35d3-73da-4226-b4bd-5e165262d235] Retry attempt 1: exec into apb-605b35d3-73da-4226-b4bd-5e165262d235 failed
10.129.0.1 - - [06/Nov/2017:10:20:35 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 29
10.129.0.1 - - [06/Nov/2017:10:20:35 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=4f21178a-28ae-4f37-bff1-0192cc7d89d7&plan_id=fe6afa23d897dc9538d4a373f7fcb933&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:20:38.643Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:38.643Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 7: Bind credentials not available yet
[2017-11-06T10:20:38.643Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 7: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
10.129.0.1 - - [06/Nov/2017:10:20:39 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=4f21178a-28ae-4f37-bff1-0192cc7d89d7&plan_id=fe6afa23d897dc9538d4a373f7fcb933&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:20:40.724Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:40.724Z] [WARNING] [apb-605b35d3-73da-4226-b4bd-5e165262d235] Retry attempt 2: Bind credentials not available yet
[2017-11-06T10:20:40.724Z] [WARNING] [apb-605b35d3-73da-4226-b4bd-5e165262d235] Retry attempt 2: exec into apb-605b35d3-73da-4226-b4bd-5e165262d235 failed
[2017-11-06T10:20:43.89Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:20:43.89Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 8: Bind credentials not available yet
[2017-11-06T10:20:43.89Z] [WARNING] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] Retry attempt 8: exec into apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7 failed
[2017-11-06T10:20:45.965Z] [NOTICE] [apb-605b35d3-73da-4226-b4bd-5e165262d235] Bind credentials found
[2017-11-06T10:20:45.965Z] [INFO] Destroying APB sandbox...
10.129.0.1 - - [06/Nov/2017:10:20:47 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=4f21178a-28ae-4f37-bff1-0192cc7d89d7&plan_id=fe6afa23d897dc9538d4a373f7fcb933&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 27
[2017-11-06T10:20:49.058Z] [ERROR] [apb-10e9efc8-f838-44f9-8ce2-5c0be92b73b7] APB completed
[2017-11-06T10:20:49.058Z] [INFO] Destroying APB sandbox...
10.129.0.1 - - [06/Nov/2017:10:21:07 +0000] "GET /ansible-service-broker/v2/service_instances/e3fd5385-0515-4d60-8e97-9db433d8d268/last_operation?operation=637eb464-7eaf-4595-8a81-b6ccb146e71b&plan_id=6cdcb34d5f8165244322d12d14e3719d&service_id=740897399fff88869106c8d7cdfd3997 HTTP/1.1" 200 27
[2017-11-06T10:21:57.771Z] [WARNING] Broker configured to *NOT* launch and run APB bind
10.129.0.1 - - [06/Nov/2017:10:21:57 +0000] "PUT /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/service_bindings/939e6ca6-5209-445f-8b21-7faa65726aaf HTTP/1.1" 201 181
[2017-11-06T10:22:40.775Z] [WARNING] Broker configured to *NOT* launch and run APB bind
10.129.0.1 - - [06/Nov/2017:10:22:40 +0000] "PUT /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/service_bindings/53c71f98-4675-42b6-8aad-55de8f554fd4 HTTP/1.1" 201 181
[2017-11-06T10:26:28.164Z] [INFO] ASYNC update in progress
[2017-11-06T10:26:28.164Z] [NOTICE] ============================================================
[2017-11-06T10:26:28.164Z] [NOTICE]                        UPDATING                             
[2017-11-06T10:26:28.164Z] [NOTICE] ============================================================
[2017-11-06T10:26:28.164Z] [NOTICE] Spec.ID: 29ed8bc1f3a28e1b4bf4ab87c6c9b728
[2017-11-06T10:26:28.164Z] [NOTICE] Spec.Name: rh-rhscl-postgresql-apb
[2017-11-06T10:26:28.164Z] [NOTICE] Spec.Image: docker.io/ansibleplaybookbundle/rhscl-postgresql-apb:latest
[2017-11-06T10:26:28.164Z] [NOTICE] Spec.Description: SCL PostgreSQL apb implementation
[2017-11-06T10:26:28.164Z] [NOTICE] ============================================================
[2017-11-06T10:26:28.164Z] [INFO] Checking if project qwang2 exists...
10.129.0.1 - - [06/Nov/2017:10:26:28 +0000] "PATCH /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2?accepts_incomplete=true HTTP/1.1" 202 58
[2017-11-06T10:26:28.398Z] [INFO] Successfully wrote resources to /tmp/asb-resource-files/apb-7d361006-51b9-4548-96a3-addc8ec97bb4.yaml
10.129.0.1 - - [06/Nov/2017:10:26:28 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:26:28.644Z] [INFO] Successfully created apb sandbox: [ apb-7d361006-51b9-4548-96a3-addc8ec97bb4 ], with edit permissions in namespace rh-rhscl-postgresql-apb-upda-nrkz2
[2017-11-06T10:26:28.644Z] [NOTICE] Creating pod "apb-7d361006-51b9-4548-96a3-addc8ec97bb4" in the rh-rhscl-postgresql-apb-upda-nrkz2 namespace
10.129.0.1 - - [06/Nov/2017:10:26:28 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:26:28.857Z] [INFO] error: unable to upgrade connection: container not found ("apb")

[2017-11-06T10:26:28.857Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 1: Failed to exec into the container
[2017-11-06T10:26:28.857Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 1: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
10.129.0.1 - - [06/Nov/2017:10:26:29 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:26:34.115Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:26:34.115Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 2: Bind credentials not available yet
[2017-11-06T10:26:34.115Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 2: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
10.129.0.1 - - [06/Nov/2017:10:26:37 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:26:39.37Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:26:39.37Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 3: Bind credentials not available yet
[2017-11-06T10:26:39.37Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 3: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
[2017-11-06T10:26:44.628Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:26:44.628Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 4: Bind credentials not available yet
[2017-11-06T10:26:44.628Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 4: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
[2017-11-06T10:26:49.871Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:26:49.871Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 5: Bind credentials not available yet
[2017-11-06T10:26:49.871Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 5: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
10.129.0.1 - - [06/Nov/2017:10:26:53 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 29
[2017-11-06T10:26:55.11Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:26:55.111Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 6: Bind credentials not available yet
[2017-11-06T10:26:55.111Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 6: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
[2017-11-06T10:27:00.369Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:27:00.37Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 7: Bind credentials not available yet
[2017-11-06T10:27:00.37Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 7: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
[2017-11-06T10:27:05.625Z] [INFO] Bind credentials not available.
command terminated with exit code 2

[2017-11-06T10:27:05.625Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 8: Bind credentials not available yet
[2017-11-06T10:27:05.625Z] [WARNING] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Retry attempt 8: exec into apb-7d361006-51b9-4548-96a3-addc8ec97bb4 failed
[2017-11-06T10:27:10.89Z] [NOTICE] [apb-7d361006-51b9-4548-96a3-addc8ec97bb4] Bind credentials found
[2017-11-06T10:27:10.89Z] [INFO] Destroying APB sandbox...
[2017-11-06T10:27:19.059Z] [INFO] Broker configured to refresh specs every 10m0s seconds
[2017-11-06T10:27:19.059Z] [INFO] Attempting bootstrap at 2017-11-06 10:27:19.059155151 +0000 UTC
[2017-11-06T10:27:19.059Z] [INFO] AnsibleBroker::Bootstrap
[2017-11-06T10:27:22.704Z] [INFO] Didn't find encoded Spec label. Assuming image is not APB and skiping
[2017-11-06T10:27:23.432Z] [INFO] Validating specs...
[2017-11-06T10:27:23.432Z] [NOTICE] All specs passed validation!
[2017-11-06T10:27:25.077Z] [ERROR] Failed to bootstrap
[2017-11-06T10:27:25.077Z] [ERROR] client: etcd cluster is unavailable or misconfigured; error #0: client: endpoint http://0.0.0.0:2379 exceeded header timeout

[2017-11-06T10:27:25.077Z] [NOTICE] Broker successfully bootstrapped
10.129.0.1 - - [06/Nov/2017:10:27:25 +0000] "GET /ansible-service-broker/v2/service_instances/eff76ca8-e3bf-4648-9172-79364ac744f2/last_operation?operation=756b7ac9-9853-4c34-82ec-e4edb5090ccb&plan_id=82c9bfe3fe8b5a6939af33eac15ec0cc&service_id=29ed8bc1f3a28e1b4bf4ab87c6c9b728 HTTP/1.1" 200 27
[2017-11-06T10:28:54.48Z] [INFO] AnsibleBroker::Catalog
10.129.0.1 - - [06/Nov/2017:10:28:54 +0000] "GET /ansible-service-broker/v2/catalog HTTP/1.1" 200 74264
[2017-11-06T10:37:19.059Z] [INFO] Broker configured to refresh specs every 10m0s seconds
[2017-11-06T10:37:19.059Z] [INFO] Attempting bootstrap at 2017-11-06 10:37:19.059182688 +0000 UTC
[2017-11-06T10:37:19.059Z] [INFO] AnsibleBroker::Bootstrap
[2017-11-06T10:37:21.528Z] [INFO] Didn't find encoded Spec label. Assuming image is not APB and skiping
[2017-11-06T10:37:24.189Z] [INFO] Validating specs...
[2017-11-06T10:37:24.189Z] [NOTICE] All specs passed validation!
[2017-11-06T10:37:26.095Z] [NOTICE] Broker successfully bootstrapped

Comment 1 Jason Montleon 2017-11-06 13:55:56 UTC
I would expect this. 

When you flip the database from dev to prod after the mediawiki instance is created you're wiping out the database. We don't currently have anything that's migrating the data in the process of the upgrade.

I'll investigate if there is a way we can handle this better with the mediawiki entrypoint script and recreate the database.

Down the road we'd also like to find a way to migrate the database when going from non-persistent to persistent storage, but this was not something we were able to fit in for 3.7.

Comment 2 John Matthews 2017-11-06 14:31:01 UTC
For 3.7 we did not address migration of data in our database APB examples for updating a plan.

We are planning to revisit in 3.8 to address how we want to handle migration with an update action.

Comment 3 Jason Montleon 2018-01-09 14:34:49 UTC
This works in upstream latest now.

Comment 5 Qixuan Wang 2018-01-15 07:01:59 UTC
Tested on OCP 3.9 (openshift v3.9.0-0.19.0, kubernetes v1.9.0-beta1, etcd 3.2.8, ose-ansible-service-broker:v3.9, ose-service-catalog:v3.9), the bug has been fixed, thanks.


Note You need to log in before you can comment on or make changes to this bug.