Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1692384 - Smoke test failing: Managed cluster should ensure pods use images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel]
Summary: Smoke test failing: Managed cluster should ensure pods use images from our re...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Test Infrastructure
Version: 4.1
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Steve Kuznetsov
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-25 13:33 UTC by Jessica Forrester
Modified: 2019-03-29 18:01 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-29 18:01:09 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift origin pull 22402 None None None 2019-03-25 14:23:47 UTC

Description Jessica Forrester 2019-03-25 13:33:12 UTC
The test:
openshift-tests [Feature:Platform][Smoke] Managed cluster should ensure pods use images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel] 

appears to be failing 100% of the time with this error:


Mar 25 09:19:17.214: INFO: >>> kubeConfig: /tmp/admin.kubeconfig
Mar 25 09:19:17.217: INFO: Waiting up to 30m0s for all (but 100) nodes to be schedulable
Mar 25 09:19:17.312: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready
Mar 25 09:19:17.374: INFO: 3 / 3 pods in namespace 'kube-system' are running and ready (0 seconds elapsed)
Mar 25 09:19:17.374: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready.
Mar 25 09:19:17.374: INFO: Waiting up to 5m0s for all daemonsets in namespace 'kube-system' to start
Mar 25 09:19:17.394: INFO: e2e test version: v1.12.4+d78770f
Mar 25 09:19:17.408: INFO: kube-apiserver version: v1.12.4+d78770f
[BeforeEach] [Top Level]
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:67
[BeforeEach] [Feature:Platform][Smoke] Managed cluster
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:147
STEP: Creating a kubernetes client
Mar 25 09:19:17.411: INFO: >>> kubeConfig: /tmp/admin.kubeconfig
[It] should ensure pods use images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel]
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/operators/images.go:21
Mar 25 09:19:18.296: INFO: Running 'oc adm release info --config= --namespace= --pullspecs -o json'
Mar 25 09:19:18.613: INFO: Error running &{/usr/bin/oc [oc adm release info --config= --namespace= --pullspecs -o json] []   error: unable to read image registry.svc.ci.openshift.org/ocp/release@sha256:5dbcd3f6d1e1d28927a955709de6d9dfe2bc11b31ecd7541e8dd347c2e792c34: unauthorized: authentication required
 error: unable to read image registry.svc.ci.openshift.org/ocp/release@sha256:5dbcd3f6d1e1d28927a955709de6d9dfe2bc11b31ecd7541e8dd347c2e792c34: unauthorized: authentication required
 [] <nil> 0xc422698de0 exit status 1 <nil> <nil> true [0xc422692390 0xc4226923b8 0xc4226923b8] [0xc422692390 0xc4226923b8] [0xc422692398 0xc4226923b0] [0x8b8ab0 0x8b8bb0] 0xc4232e5020 <nil>}:
error: unable to read image registry.svc.ci.openshift.org/ocp/release@sha256:5dbcd3f6d1e1d28927a955709de6d9dfe2bc11b31ecd7541e8dd347c2e792c34: unauthorized: authentication required
Mar 25 09:19:18.613: INFO: unable to read release payload with error: exit status 1
[AfterEach] [Feature:Platform][Smoke] Managed cluster
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/cli.go:107
[AfterEach] [Feature:Platform][Smoke] Managed cluster
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:148
Mar 25 09:19:18.614: INFO: Waiting up to 3m0s for all (but 100) nodes to be ready
Mar 25 09:19:18.646: INFO: Running AfterSuite actions on all node
Mar 25 09:19:18.646: INFO: Running AfterSuite actions on node 1
fail [github.com/openshift/origin/test/extended/operators/images.go:25]: Mar 25 09:19:18.613: unable to read release payload with error: exit status 1




Example run:
https://openshift-gce-devel.appspot.com/build/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-4.0/6113#openshift-tests-featureplatformsmoke-managed-cluster-should-ensure-pods-use-images-from-our-release-image-with-proper-imagepullpolicy-suiteopenshiftconformanceparallel

Comment 1 W. Trevor King 2019-03-26 04:02:57 UTC
I dunno what the fix was, but we seem to be passing now:

$ curl -s https://storage.googleapis.com/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-4.0/6147/build-log.txt | grep 'Managed cluster should ensure pods use images from our release image with proper ImagePullPolicy'
started: (0/3/906) "[Feature:Platform][Smoke] Managed cluster should ensure pods use images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel]"
passed: (2.9s) 2019-03-25T23:05:54 "[Feature:Platform][Smoke] Managed cluster should ensure pods use images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel]"

I guess we can revert origin#22402, ideally link the fix (whatever it was), and mark this MODIFIED?

Comment 2 W. Trevor King 2019-03-26 04:03:30 UTC
Or maybe we're only "passing" because of origin#22402 ducking?

Comment 3 Derek Carr 2019-03-26 20:14:03 UTC
Posted PR to pull image pull secret from cluster and use that to read the release payload.

https://github.com/openshift/origin/pull/22408

Comment 4 Steve Kuznetsov 2019-03-29 18:01:09 UTC
Above PR merged and job looks relatively healthy now.


Note You need to log in before you can comment on or make changes to this bug.