Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1357866 - Helper rake tasks not fully updated for Host Unification and Scoped search
Summary: Helper rake tasks not fully updated for Host Unification and Scoped search
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Hosts
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: 6.2
Assignee: Justin Sherrill
QA Contact: jcallaha
URL: http://projects.theforeman.org/issues...
Whiteboard:
: 1354555 (view as bug list)
Depends On:
Blocks: 1405500
TreeView+ depends on / blocked
 
Reported: 2016-07-19 12:39 UTC by Justin Sherrill
Modified: 2018-12-06 20:45 UTC (History)
11 users (show)

Fixed In Version: rubygem-katello-3.0.0.91-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1405500 (view as bug list)
Environment:
Last Closed: 2017-01-26 10:40:42 UTC


Attachments (Terms of Use)
still consumed subs (deleted)
2017-01-17 22:07 UTC, jcallaha
no flags Details
before clean (deleted)
2017-01-18 18:34 UTC, jcallaha
no flags Details
after clean (deleted)
2017-01-18 18:34 UTC, jcallaha
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0197 normal SHIPPED_LIVE Satellite 6.2.7 Async Bug Release 2017-01-26 15:38:38 UTC
Foreman Issue Tracker 15721 None None None 2016-07-19 13:05:56 UTC

Description Justin Sherrill 2016-07-19 12:39:18 UTC
these two rake tasks:

rake katello:reindex
rake katello:clean_backend_objects

have not been fully updated to support host unification and scoped search.

Also, the katello db is now the canonical source for information about hosts, no reindex on them needed

Comment 1 Justin Sherrill 2016-07-19 12:39:20 UTC
Created from redmine issue http://projects.theforeman.org/issues/15721

Comment 2 Justin Sherrill 2016-07-19 12:39:25 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 4 Bryan Kearney 2016-07-29 14:16:26 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 5 Bryan Kearney 2016-07-29 14:16:29 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 6 Bryan Kearney 2016-08-03 14:17:07 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/15721 has been closed

Comment 8 Eric Helms 2016-12-01 23:02:12 UTC
*** Bug 1354555 has been marked as a duplicate of this bug. ***

Comment 9 Justin Sherrill 2016-12-16 16:12:28 UTC
Created attachment 1232583 [details]
updated 6.2 copy of clean_backend_objects

Comment 12 Justin Sherrill 2017-01-11 18:33:59 UTC
there are a few things:

1.  foreman-rake katello:reimport   (this was formally reindex).

 This pulls in content from pulp into the db that is missing.  You can likely test this by running in the foreman-rake console:

[Katello::Erratum,
Katello::PackageGroup,
Katello::PuppetModule,
Katello::Rpm,
Katello::DockerManifest,
Katello::DockerTag].each{|content| content.destroy_all }

with various repos synced.  Then run the rake task and check to see that all content listings for repos and content views is accurate.

2.  foreman-rake katello:clean_backend_objects

This handles two cases:

  a) Some consumer was deleted in candlepin but not in katello/foreman.  You can reproduce this by registering a client and running this in the console:

   User.current = User.anonymous_api_admin
   Katello::Resources::Candlepin::Consumer.destroy("UUID")   

where UUID is the uuid of some registered client.

  b) Some consumer was deleted in katello/foreman but exists in candlepin. You can reproduce this by registering a client, subscribing it to some subscription, and running this in the console:

  host = Host.find_by(:name => "foo.example.com")
  host.subscription_facet.destroy
  host.content_facet.destroy
  host.destroy

the host should dissappear from the webUI, but it should still show as consuming a subscription on the subscriptions page.  Then run the clean_backend_objects rake task and it should correct it.

Comment 13 jcallaha 2017-01-17 22:07:21 UTC
Justin, I tried the steps you provided, but am seeing some questionable results. 

After running katello:reimport, i see significant count reductions throughout packages and errata. 

Content view before
24793 Packages
1892 Errata ( 379  1220  293  )

Content view after
128 Packages
128 Errata ( 13  93  22  )

After running the second scenario with clean_backend_objects, I'm still seeing the consumed subscriptions that the deleted host held. (See attached screenshot for after).

-bash-4.2# hammer -u admin -p changeme host list --organization-id 1
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
ID | NAME                                                        | OPERATING SYSTEM | HOST GROUP | IP            | MAC              
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
18 | cloud-qe-06.idmqe.lab.eng.bos.redhat.com                    | RHEL Server 7.3  |            | 10.19.34.35   | 34:40:b5:8a:43:ff
13 | ibm-x3250m4-06.lab.eng.rdu2.redhat.com                      | RHEL Server 7.3  |            | 10.8.0.71     | 40:f2:e9:32:1c:ac
10 | ibm-x3550m3-09.lab.eng.brq.redhat.com                       | RHEL Server 7.3  |            | 10.34.36.133  | 34:40:b5:a0:13:44
17 | mgmt3.rhq.lab.eng.bos.redhat.com                            | RHEL Server 7.3  |            | 10.16.4.13    | 00:50:45:5f:3e:6f
14 | short                                                       | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
15 | test                                                        | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
9  | tom-temp                                                    |                  |            |               |                  
7  | unused                                                      | RHEL Server 7.3  |            | 10.12.211.202 | 00:50:56:ae:89:e9
3  | virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1                |                  |            |               |                  
8  | virt-who-dhcp-8-101-249.satellite.lab.eng.rdu2.redhat.com-1 |                  |            |               |                  
4  | virt-who-inorbit.hq.gsslab.rdu.redhat.com-1                 |                  |            |               |                  
5  | virt-who-ironforge.satellite.lab.eng.rdu2.redhat.com-1      |                  |            |               |                  
6  | virt-who-nightwing.hq.gsslab.rdu.redhat.com-1               |                  |            |               |                  
2  | virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1   |                  |            |               |                  
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
-bash-4.2# foreman-rake console
/usr/share/foreman/lib/tasks/console.rake:4: warning: already initialized constant ARGV
Loading production environment (Rails 4.1.5)
irb(main):001:0> host = Host.find_by(:name => "ibm-x3550m3-09.lab.eng.brq.redhat.com")
=> #<Host::Managed id: 10, name: "ibm-x3550m3-09.lab.eng.brq.redhat.com", last_compile: "2017-01-17 19:52:48", last_report: "2017-01-17 19:48:55", updated_at: "2017-01-17 19:52:58", created_at: "2017-01-16 14:40:04", root_pass: nil, architecture_id: 1, operatingsystem_id: 1, environment_id: 1, ptable_id: nil, medium_id: nil, build: false, comment: nil, disk: nil, installed_at: nil, model_id: 1, hostgroup_id: nil, owner_id: 1, owner_type: "User", enabled: true, puppet_ca_proxy_id: 2, managed: false, use_image: nil, image_file: nil, uuid: nil, compute_resource_id: nil, puppet_proxy_id: 2, certname: "ibm-x3550m3-09.lab.eng.brq.redhat.com", image_id: nil, organization_id: 1, location_id: 2, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: nil, provision_method: nil, content_source_id: nil, grub_pass: "", content_view_id: nil, lifecycle_environment_id: nil, global_status: 1, lookup_value_matcher: "fqdn=ibm-x3550m3-09.lab.eng.brq.redhat.com", discovery_rule_id: nil, openscap_proxy_id: nil>
irb(main):002:0> host.subscription_facet.destroy
=> #<Katello::Host::SubscriptionFacet id: 20, host_id: 10, uuid: "8fc5bcb6-5b27-4c98-b887-b8a306d90994", last_checkin: "2017-01-17 19:52:57", service_level: nil, release_version: nil, autoheal: true, registered_at: "2017-01-17 19:52:50">
irb(main):003:0> host.content_facet.destroy
=> #<Katello::Host::ContentFacet id: 3, host_id: 10, uuid: "8fc5bcb6-5b27-4c98-b887-b8a306d90994", content_view_id: 5, lifecycle_environment_id: 1, kickstart_repository_id: nil>
irb(main):004:0> host.destroy
=> #<Host::Managed id: 10, name: "ibm-x3550m3-09.lab.eng.brq.redhat.com", last_compile: "2017-01-17 19:52:48", last_report: "2017-01-17 19:48:55", updated_at: "2017-01-17 19:52:58", created_at: "2017-01-16 14:40:04", root_pass: nil, architecture_id: 1, operatingsystem_id: 1, environment_id: 1, ptable_id: nil, medium_id: nil, build: false, comment: nil, disk: nil, installed_at: nil, model_id: 1, hostgroup_id: nil, owner_id: 1, owner_type: "User", enabled: true, puppet_ca_proxy_id: 2, managed: false, use_image: nil, image_file: nil, uuid: nil, compute_resource_id: nil, puppet_proxy_id: 2, certname: "ibm-x3550m3-09.lab.eng.brq.redhat.com", image_id: nil, organization_id: 1, location_id: 2, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: nil, provision_method: nil, content_source_id: nil, grub_pass: "", content_view_id: nil, lifecycle_environment_id: nil, global_status: 1, lookup_value_matcher: "fqdn=ibm-x3550m3-09.lab.eng.brq.redhat.com", discovery_rule_id: nil, openscap_proxy_id: nil>
irb(main):005:0> exit
-bash-4.2# hammer -u admin -p changeme host list --organization-id 1
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
ID | NAME                                                        | OPERATING SYSTEM | HOST GROUP | IP            | MAC              
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
18 | cloud-qe-06.idmqe.lab.eng.bos.redhat.com                    | RHEL Server 7.3  |            | 10.19.34.35   | 34:40:b5:8a:43:ff
13 | ibm-x3250m4-06.lab.eng.rdu2.redhat.com                      | RHEL Server 7.3  |            | 10.8.0.71     | 40:f2:e9:32:1c:ac
17 | mgmt3.rhq.lab.eng.bos.redhat.com                            | RHEL Server 7.3  |            | 10.16.4.13    | 00:50:45:5f:3e:6f
14 | short                                                       | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
15 | test                                                        | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
9  | tom-temp                                                    |                  |            |               |                  
7  | unused                                                      | RHEL Server 7.3  |            | 10.12.211.202 | 00:50:56:ae:89:e9
3  | virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1                |                  |            |               |                  
8  | virt-who-dhcp-8-101-249.satellite.lab.eng.rdu2.redhat.com-1 |                  |            |               |                  
4  | virt-who-inorbit.hq.gsslab.rdu.redhat.com-1                 |                  |            |               |                  
5  | virt-who-ironforge.satellite.lab.eng.rdu2.redhat.com-1      |                  |            |               |                  
6  | virt-who-nightwing.hq.gsslab.rdu.redhat.com-1               |                  |            |               |                  
2  | virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1   |                  |            |               |                  
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
-bash-4.2# foreman-rake katello:clean_backend_objects
The following changes will not actually be performed.  Rerun with COMMIT=true to apply the changes
1 orphaned consumer id(s) found.
-bash-4.2# foreman-rake katello:clean_backend_objects COMMIT=true
1 orphaned consumer id(s) found.
-bash-4.2# foreman-rake katello:clean_backend_objects COMMIT=true
0 orphaned consumer id(s) found.

Comment 14 jcallaha 2017-01-17 22:07:47 UTC
Created attachment 1241971 [details]
still consumed subs

Comment 15 Justin Sherrill 2017-01-17 22:13:15 UTC
Jake, the content counts issue was reported in https://bugzilla.redhat.com/show_bug.cgi?id=1412486 (actually introduced as part of https://bugzilla.redhat.com/show_bug.cgi?id=1399294 )

For the 2nd issue, is your ListenOnCandlepinEvents process running properly?

Comment 16 jcallaha 2017-01-18 18:33:33 UTC
Justin, that'll do it! I don't know why that task keeps dropping, but with it running, the second scenario went off flawlessly. Will attach screenshots of the subscription's associations before and after running clean_backend_objects.

Since the issue I saw with the first scenario is already covered in another BZ, I will mark this one as Verified in Satellite 6.2.7 Snap 2.

-bash-4.2# hammer -u admin -p changeme host list --organization-id 1
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
ID | NAME                                                        | OPERATING SYSTEM | HOST GROUP | IP            | MAC              
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
18 | cloud-qe-06.idmqe.lab.eng.bos.redhat.com                    | RHEL Server 7.3  |            | 10.19.34.35   | 34:40:b5:8a:43:ff
13 | ibm-x3250m4-06.lab.eng.rdu2.redhat.com                      | RHEL Server 7.3  |            | 10.8.0.71     | 40:f2:e9:32:1c:ac
17 | mgmt3.rhq.lab.eng.bos.redhat.com                            | RHEL Server 7.3  |            | 10.16.4.13    | 00:50:45:5f:3e:6f
14 | short                                                       | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
15 | test                                                        | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
9  | tom-temp                                                    |                  |            |               |                  
7  | unused                                                      | RHEL Server 7.3  |            | 10.12.211.202 | 00:50:56:ae:89:e9
3  | virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1                |                  |            |               |                  
8  | virt-who-dhcp-8-101-249.satellite.lab.eng.rdu2.redhat.com-1 |                  |            |               |                  
4  | virt-who-inorbit.hq.gsslab.rdu.redhat.com-1                 |                  |            |               |                  
5  | virt-who-ironforge.satellite.lab.eng.rdu2.redhat.com-1      |                  |            |               |                  
6  | virt-who-nightwing.hq.gsslab.rdu.redhat.com-1               |                  |            |               |                  
2  | virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1   |                  |            |               |                  
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
-bash-4.2# foreman-rake console
/usr/share/foreman/lib/tasks/console.rake:4: warning: already initialized constant ARGV
Loading production environment (Rails 4.1.5)
irb(main):001:0> host = Host.find_by(:name => "ibm-x3550m3-09.lab.eng.brq.redhat.com")
=> nil
irb(main):002:0> host = Host.find_by(:name => "short")
=> #<Host::Managed id: 14, name: "short", last_compile: "2017-01-16 20:23:46", last_report: nil, updated_at: "2017-01-16 20:23:47", created_at: "2017-01-16 19:44:05", root_pass: nil, architecture_id: 1, operatingsystem_id: 2, environment_id: nil, ptable_id: nil, medium_id: nil, build: false, comment: nil, disk: nil, installed_at: nil, model_id: 1, hostgroup_id: nil, owner_id: 1, owner_type: "User", enabled: true, puppet_ca_proxy_id: nil, managed: false, use_image: nil, image_file: nil, uuid: nil, compute_resource_id: nil, puppet_proxy_id: nil, certname: nil, image_id: nil, organization_id: 1, location_id: 2, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: nil, provision_method: nil, content_source_id: nil, grub_pass: "", content_view_id: nil, lifecycle_environment_id: nil, global_status: 1, lookup_value_matcher: "fqdn=short", discovery_rule_id: nil, openscap_proxy_id: nil>
irb(main):003:0> host.subscription_facet.destroy
=> #<Katello::Host::SubscriptionFacet id: 16, host_id: 14, uuid: "dce969ad-3bfa-4299-af3e-9b9c5ef39903", last_checkin: "2017-01-16 19:47:38", service_level: nil, release_version: nil, autoheal: true, registered_at: "2017-01-16 19:47:33">
irb(main):004:0> host.content_facet.destroy
=> #<Katello::Host::ContentFacet id: 7, host_id: 14, uuid: "dce969ad-3bfa-4299-af3e-9b9c5ef39903", content_view_id: 2, lifecycle_environment_id: 1, kickstart_repository_id: nil>
irb(main):005:0> host.destroy
=> #<Host::Managed id: 14, name: "short", last_compile: "2017-01-16 20:23:46", last_report: nil, updated_at: "2017-01-16 20:23:47", created_at: "2017-01-16 19:44:05", root_pass: nil, architecture_id: 1, operatingsystem_id: 2, environment_id: nil, ptable_id: nil, medium_id: nil, build: false, comment: nil, disk: nil, installed_at: nil, model_id: 1, hostgroup_id: nil, owner_id: 1, owner_type: "User", enabled: true, puppet_ca_proxy_id: nil, managed: false, use_image: nil, image_file: nil, uuid: nil, compute_resource_id: nil, puppet_proxy_id: nil, certname: nil, image_id: nil, organization_id: 1, location_id: 2, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: nil, provision_method: nil, content_source_id: nil, grub_pass: "", content_view_id: nil, lifecycle_environment_id: nil, global_status: 1, lookup_value_matcher: "fqdn=short", discovery_rule_id: nil, openscap_proxy_id: nil>
irb(main):006:0> exit
-bash-4.2# hammer -u admin -p changeme host list --organization-id 1
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
ID | NAME                                                        | OPERATING SYSTEM | HOST GROUP | IP            | MAC              
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
18 | cloud-qe-06.idmqe.lab.eng.bos.redhat.com                    | RHEL Server 7.3  |            | 10.19.34.35   | 34:40:b5:8a:43:ff
13 | ibm-x3250m4-06.lab.eng.rdu2.redhat.com                      | RHEL Server 7.3  |            | 10.8.0.71     | 40:f2:e9:32:1c:ac
17 | mgmt3.rhq.lab.eng.bos.redhat.com                            | RHEL Server 7.3  |            | 10.16.4.13    | 00:50:45:5f:3e:6f
15 | test                                                        | RedHat 7.2       |            | 172.17.0.2    | 02:42:ac:11:00:02
9  | tom-temp                                                    |                  |            |               |                  
7  | unused                                                      | RHEL Server 7.3  |            | 10.12.211.202 | 00:50:56:ae:89:e9
3  | virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1                |                  |            |               |                  
8  | virt-who-dhcp-8-101-249.satellite.lab.eng.rdu2.redhat.com-1 |                  |            |               |                  
4  | virt-who-inorbit.hq.gsslab.rdu.redhat.com-1                 |                  |            |               |                  
5  | virt-who-ironforge.satellite.lab.eng.rdu2.redhat.com-1      |                  |            |               |                  
6  | virt-who-nightwing.hq.gsslab.rdu.redhat.com-1               |                  |            |               |                  
2  | virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1   |                  |            |               |                  
---|-------------------------------------------------------------|------------------|------------|---------------|------------------
-bash-4.2# foreman-rake katello:clean_backend_objects COMMIT=true
1 orphaned consumer id(s) found.

Comment 17 jcallaha 2017-01-18 18:34:11 UTC
Created attachment 1242252 [details]
before clean

Comment 18 jcallaha 2017-01-18 18:34:29 UTC
Created attachment 1242253 [details]
after clean

Comment 20 errata-xmlrpc 2017-01-26 10:40:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0197


Note You need to log in before you can comment on or make changes to this bug.