Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1690054 - Manifest import fails with "Pool with id <id> could not be found" [NEEDINFO]
Summary: Manifest import fails with "Pool with id <id> could not be found"
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Candlepin
Version: 6.3.5
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: Unspecified
Assignee: Barnaby Court
QA Contact: jcallaha
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-18 16:49 UTC by Danny Synk
Modified: 2019-03-19 12:47 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-19 12:46:49 UTC
Target Upstream Version:
bcourt: needinfo? (dsynk)


Attachments (Terms of Use)

Description Danny Synk 2019-03-18 16:49:43 UTC
Description of problem:

When a customer attempts to import a new manifest from the Satellite webUI, the import hangs for nearly 3 hours on Actions::Candlepin::Owner::ImportProducts, then fails with the following error:

`Katello::Errors::CandlepinError

Pool with id 8a348af969350cd601695963578a26a4 could not be found.`

The customer reports that the manifest being imported is the same as the manifest existing on the Satellite except for the addition of a RHEL 8 Beta entitlement. The customer also reports that they are unable to find above pool ID on the manifest being imported or on any previously-imported manifests.


Version-Release number of selected component (if applicable):
Satellite 6.3.5

How reproducible:
Customer is able to consistently reproduce the issue. Unable to reproduce the issue outside customer environment thus far.


Additional info:

Data available from linked support case:

- Task export including this error 
- Sosreport generated after debug logging enabled on all Satellite components and issue reproduced by customer 

Full traceback from ImportProducts task error:

Error:

Katello::Errors::CandlepinError

Pool with id 8a348af969350cd601695963578a26a4 could not be found.

---
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in
  `rescue in propagate_candlepin_errors'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/propagate_candlepin_errors.rb:18:in
  `propagate_candlepin_errors'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/propagate_candlepin_errors.rb:9:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/remote_action.rb:16:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/remote_action.rb:40:in
  `block in as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/models/katello/concerns/user_extensions.rb:21:in
  `cp_config'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/remote_action.rb:27:in
  `as_cp_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/remote_action.rb:39:in
  `as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/remote_action.rb:16:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:30:in
  `with_progress_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:16:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/keep_locale.rb:11:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/keep_locale.rb:22:in
  `with_locale'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.87/app/lib/actions/middleware/keep_locale.rb:11:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:31:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/world.rb:30:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:517:in
  `block (2 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:516:in
  `catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:516:in
  `block in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in
  `block in with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in
  `catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in
  `with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:511:in
  `execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:268:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in
  `block (2 levels) in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `with_meta_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in
  `block in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in
  `open_action'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director.rb:42:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/executors/parallel/worker.rb:11:in
  `on_message'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/context.rb:46:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/executes_context.rb:7:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/actor.rb:26:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/awaits.rb:15:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:38:in
  `process_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:31:in
  `process_envelopes?'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:20:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/termination.rb:55:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/removes_child.rb:10:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in
  `on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:161:in
  `process_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:95:in
  `block in on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:118:in
  `block (2 levels) in schedule_execution'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `block in synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:115:in
  `block in schedule_execution'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:18:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:96:in
  `work'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:77:in
  `block in call_job'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in
  `run_task'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in
  `block (3 levels) in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in
  `loop'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in
  `block (2 levels) in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in
  `catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in
  `block in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in
  `block in create_with_logging_context'"

Comment 3 Barnaby Court 2019-03-19 12:46:49 UTC
Danny, I see a lot of concurrent traffic with the manifest import. The two recommendations are 
1) Firewall off client traffic during a manifest import. This should prevent the deadlock that is evident in the candlepin log & allow the manifest import to complete. 
2) Upgrade to Satellite 6.4 - This will speed up the overall time needed for import. Please note this will not eliminate the need to use firewall rules to prevent deadlocks during import.


Note You need to log in before you can comment on or make changes to this bug.