Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1695440 - Inspecting containers fails with KeyError: 'docker-content-digest'
Summary: Inspecting containers fails with KeyError: 'docker-content-digest'
Keywords:
Status: ON_QA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 15.0 (Stein)
Hardware: All
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Steve Baker
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
Depends On:
Blocks: 1667964
TreeView+ depends on / blocked
 
Reported: 2019-04-03 06:10 UTC by Tony Breeds
Modified: 2019-04-15 22:06 UTC (History)
4 users (show)

Fixed In Version: openstack-tripleo-common-containers-10.6.2-0.20190412150355.0ec6518.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1822915 None None None 2019-04-03 06:12:35 UTC
OpenStack gerrit 642028 None None None 2019-04-03 21:13:55 UTC

Description Tony Breeds 2019-04-03 06:10:34 UTC
When trying to deploy with:

+ openstack overcloud deploy --templates /home/stack/templates -r /home/stack/templates/roles_data.yaml --validation-errors-nonfatal --libvirt-type kvm --ntp-server clock01.util.phx2.redhat.com -e /home/stack/templates/node-info.yaml -e /home/stack/templates/container-image-prepare.yaml -e /home/stack/templates/customization.yaml --config-download --stack-only

the deploy failed with:
<SNIP>
Deploying templates in the directory /tmp/tripleoclient-ffh1i7ef/tripleo-heat-templates
Initializing overcloud plan deployment
'docker-content-digest'

Trying: openstack tripleo container image prepare --dry-run -e ~/templates/container-image-prepare.yaml

we see:
(undercloud) [stack@director ~]$ openstack tripleo container image prepare --dry-run -e ~/templates/container-image-prepare.yaml
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 30, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py", line 680, in take_action
    cleanup=parsed_args.cleanup)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 188, in container_images_prepare_multi
    mirrors=mirrors
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 286, in container_images_prepare
    images, tag_from_label)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 518, in discover_image_tags
    discover_args):
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1566, in discover_tag_from_inspect
    i = self._inspect(image_url, session=session)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1436, in _inspect
    image_url, session=session)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 319, in iter
    return fut.result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 409, in _inspect
    digest = manifest_r.headers['Docker-Content-Digest']
  File "/usr/lib/python3.6/site-packages/requests/structures.py", line 52, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'docker-content-digest'
'docker-content-digest'

It seems that some? images are missing that header?

Comment 1 Alex Schultz 2019-04-03 15:19:01 UTC
Please provide /home/stack/templates/container-image-prepare.yaml

Comment 2 Steve Baker 2019-04-03 21:13:56 UTC
container-image-prepare.yaml: http://pastebin.test.redhat.com/748740

It looks like brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 doesn't set the docker-content-digest, which the inspect call depends on. There is an upstream fix.


Note You need to log in before you can comment on or make changes to this bug.