Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1357046 - Updating a repo without specifying checksum_type causes key_error in Pulp
Summary: Updating a repo without specifying checksum_type causes key_error in Pulp
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Pulp
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
high
medium vote
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Roman Plevka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-15 14:55 UTC by Roman Plevka
Modified: 2019-04-01 20:27 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-02-21 16:51:07 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Pulp Redmine 2134 High CLOSED - CURRENTRELEASE Updating a repo without specifying checksum_type causes KeyError 2016-09-19 21:31:30 UTC

Description Roman Plevka 2016-07-15 14:55:38 UTC
Description of problem:

On updating a repo without touching checksum_type attribute, pulp raises an unexpected KeyError:

<pre>
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) Task pulp.server.tasks.repository.distributor_update[36edf787-70ca-437a-b840-a3aebba3fd4e] raised unexpected: KeyError(u'checksum_type',)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) Traceback (most recent call last):
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     R = retval = fun(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     return super(Task, self).__call__(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     return super(PulpTask, self).__call__(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     return self.run(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 211, in update
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     distributor.config.pop(k)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)   File "/usr/lib/python2.7/site-packages/mongoengine/base/datastructures.py", line 73, in pop
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352)     return super(BaseDict, self).pop(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) KeyError: u'checksum_type'
</pre>

Not specifying a checksum_type on creating repo seems to be no problem.


Version-Release number of selected component (if applicable):
6.2.0 snap20

How reproducible:
on updating repo

Steps to Reproduce:
1. hammer --csv -u admin -p changeme repository create --name foo_repo_cli_1 --product-id 10 --content-type yum --url https://foo/bar/
2. hammer --csv -u admin -p changeme repository update --name foo_repo_cli_1 --product-id 10 --url https://bar/foo/ --organization-id 1 
3. watch /var/log/messages for pulp logs

Actual results:
KeyError(u'checksum_type',)

Expected results:
no error

Additional info:

Comment 2 Brad Buckingham 2016-07-18 21:06:22 UTC
The error observed in the log file is not fatal and does not appear to affect the creation or updating of a repository.  That said, we should look to address this error to minimize confusion.

Note: Although it is a difference scenario, the behavior here is very similar to the issue raised in bug 1356111.

Comment 4 Michael Hrivnak 2016-08-05 20:22:43 UTC
I suspect that this is more fatal than it may initially appear. The operation is likely updating the importer first, and then any distributors. If you only were trying to change an importer setting, that would work, and the error would appear to not cause any problems. But if you tried to update a distributor setting in that operation, I bet this error would prevent it from working.

Comment 5 Filip Dobrovolny 2016-08-08 15:31:21 UTC
I was unable to reproduce this with pulp-admin on pulp-2.8.4-1. Can someone please provide either more specific reproducer steps with pulp, or a machine that exhibits the problem?

Comment 6 pulp-infra@redhat.com 2016-08-09 10:30:25 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2016-08-09 10:30:28 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2016-08-15 12:00:27 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2016-08-18 13:30:25 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 10 pulp-infra@redhat.com 2016-08-30 19:00:25 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 11 Bryan Kearney 2016-09-12 16:03:48 UTC
Moving to POST since this is fixed in Pulp.

Comment 12 pulp-infra@redhat.com 2016-09-13 14:31:45 UTC
The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug.

Comment 13 pulp-infra@redhat.com 2016-09-19 21:31:31 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 14 Roman Plevka 2017-01-09 15:17:35 UTC
VERIFIED
on sat6.2.7 snap 1

the error seems to be gone:

2017-01-09 10:13:46 [app] [I] Started GET "/katello/api/products/1/repositories?name=foo_repo_cli_1&organization_id=1" for ::1 at 2017-01-09 10:13:46 -0500
2017-01-09 10:13:46 [app] [I] Processing by Katello::Api::V2::RepositoriesController#index as JSON
2017-01-09 10:13:46 [app] [I]   Parameters: {"name"=>"foo_repo_cli_1", "organization_id"=>"1", "api_version"=>"v2", "product_id"=>"1", "repository"=>{}}
2017-01-09 10:13:46 [app] [I] Authorized user admin(Admin User)
2017-01-09 10:13:46 [app] [I]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.92/app/views/katello/api/v2/repositories/index.json.rabl within katello/api/v2/layouts/collection (22.1ms)
2017-01-09 10:13:46 [app] [I] Completed 200 OK in 58ms (Views: 19.7ms | ActiveRecord: 11.0ms)
2017-01-09 10:13:46 [app] [I] Started PUT "/katello/api/repositories/4" for ::1 at 2017-01-09 10:13:46 -0500
2017-01-09 10:13:46 [app] [I] Processing by Katello::Api::V2::RepositoriesController#update as JSON
2017-01-09 10:13:46 [app] [I]   Parameters: {"name"=>"foo_repo_cli_1", "url"=>"https://bar/foo/", "api_version"=>"v2", "id"=>"4", "repository"=>{"name"=>"foo_repo_cli_1", "url"=>"https://bar/foo/"}}
2017-01-09 10:13:46 [app] [I] Authorized user admin(Admin User)
2017-01-09 10:13:47 [app] [I]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.92/app/views/katello/api/v2/repositories/show.json.rabl within katello/api/v2/layouts/resource (58.3ms)
2017-01-09 10:13:47 [app] [I] Completed 200 OK in 1584ms (Views: 23.5ms | ActiveRecord: 54.6ms)

Comment 15 Roman Plevka 2017-01-10 17:13:33 UTC
actually - FAILEDQA:

was looking into a wrong log file. The pulp still logs this into the syslog:

Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912) Traceback (most recent call last):
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     R = retval = fun(*args, **kwargs)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     return super(Task, self).__call__(*args, **kwargs)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     return super(PulpTask, self).__call__(*args, **kwargs)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     return self.run(*args, **kwargs)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 211, in update
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     distributor.config.pop(k)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)   File "/usr/lib/python2.7/site-packages/mongoengine/base/datastructures.py", line 73, in pop
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912)     return super(BaseDict, self).pop(*args, **kwargs)
Jan  9 10:13:47 bkr-hv02-guest27 pulp: celery.worker.job:ERROR: (641-14912) KeyError: u'checksum_type'



..as a bonus, I'm no longer able to create new repos after this:

Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008) Exception adding importer to repo [ksvxWsZvER-uSkDmNwcJy-IJKiwmPEvU]; the repo will be deleted
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008) Traceback (most recent call last):
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 411, in create_repo
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     importer_controller.set_importer(repo_id, importer_type_id, importer_repo_plugin_config)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/celery/local.py", line 167, in <lambda>
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     __call__ = lambda x, *a, **kw: x._get_current_object()(*a, **kw)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     return super(Task, self).__call__(*args, **kwargs)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     return super(PulpTask, self).__call__(*args, **kwargs)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/celery/app/task.py", line 420, in __call__
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     return self.run(*args, **kwargs)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/importer.py", line 85, in set_importer
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     validate_importer_config(repo.repo_id, importer_type_id, repo_plugin_config)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/importer.py", line 173, in validate_importer_config
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008)     raise exceptions.PulpCodedValidationException(validation_errors=message)
Jan 10 11:44:17 bkr-hv02-guest27 pulp: pulp.server.controllers.repository:ERROR: (19882-31008) PulpCodedValidationException: A validation error occurred.


I need a fresh sat6.2.7 installation to prove the latter is caused by this BZ.

Comment 16 Roman Plevka 2017-01-10 17:14:47 UTC
please disregard - I didn't notice this is ON_QA for 6.3.0

Comment 17 Roman Plevka 2017-08-30 12:45:01 UTC
VERIFIED
on sat6.3.0 snap#13

no traceback present anymore

Comment 18 pm-sat@redhat.com 2018-02-21 16:51:07 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> 
> For information on the advisory, and where to find the updated files, follow the link below.
> 
> If the solution does not work for you, open a new bug report.
> 
> https://access.redhat.com/errata/RHSA-2018:0336


Note You need to log in before you can comment on or make changes to this bug.