Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1695022 - Compliance data not uploaded with insights-client
Summary: Compliance data not uploaded with insights-client
Keywords:
Status: NEW
Alias: None
Product: Red Hat Insights
Classification: Red Hat
Component: Pilot
Version: unspecified
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
Assignee: Mohit Goyal
QA Contact: Rob Williams
Kevin Blake
URL:
Whiteboard:
Depends On:
Blocks: 1122832
TreeView+ depends on / blocked
 
Reported: 2019-04-02 10:11 UTC by Peter Vreman
Modified: 2019-04-03 11:06 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Description Peter Vreman 2019-04-02 10:11:09 UTC
Description of problem:
Compliant report not uploaded

I created a compliance report according to the instructions in /var/lib/insights/latest-compliance-report.xml

------------
sudo yum install -y openscap-scanner scap-security-guide
sudo oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_standard --results-arf /var/lib/insights/latest-compliance-report.xml /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml
------------

It is genereatated but not uploaded, see
------------
vrempet@li-lc-1443 ~
$ ls -l /var/lib/insights/
total 22488
-rw-rw-r--+ 1 root root   783328 Apr  2 10:05 last_stable.egg
-rw-rw-r--+ 1 root root      811 Apr  2 10:05 last_stable.egg.asc
-rw-rw-r--+ 1 root root 22234712 Apr  2 10:00 latest-compliance-report.xml

vrempet@li-lc-1443 ~
$ grep latest-compliance-report.xml /var/log/insights-client/insights-client.log

vrempet@li-lc-1443 ~
$ sudo insights-client --version
Client: 3.0.3-9
Core: 3.0.87-1
-------



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Mohit Goyal 2019-04-02 12:22:40 UTC
Peter, can you please try running the command as follows with `--results` not `--results-arf`. We need to get the Getting Started guide updated to fix this issue. Please do confirm if this works change enables you to successfully load the reports.

Comment 2 Peter Vreman 2019-04-02 16:43:23 UTC
Mohit, it does not make a difference.
I checked the insights archive and it does not have either /var/lib/insights nor any file containing the word 'compliance' included.
then i grepped the contents of the latest egg for 'compliance' and it also did not show any hit.
--------
crash/LI] root@li-lc-1443:~/1# grep -R -i compli
insights/configtree/__init__.py:        grandchild nodes in the tree. This allows more complicated queries,
insights/contrib/toposort.py:# you may not use this file except in compliance with the License.
insights/core/__init__.py:        grandchild nodes in the tree. This allows more complicated queries,
insights/core/plugins.py:           # ... some complicated logic
insights/parsers/docker_list.py:            # complicated for words :-)
insights/parsers/foreman_log.py:        10.181.73.211 - rhcapkdc.example2.com [27/Mar/2017:13:34:52 -0400] "GET /rhsm/consumers/385e688f-43ad-41b2-9fc7-593942ddec78/compliance HTTP/1.1" 200 5527
[crash/LI] root@li-lc-1443:~/1#
------------

Is the insights-client already prepared for being able to upload compliance data?

Comment 3 Andrew Kofink 2019-04-02 18:38:54 UTC
Peter,

To use insights-client to upload your SCAP report, please try the following:

sudo insights-client --payload <report output XML from scap command> --content-type application/vnd.redhat.compliance.something+tgz

There is no logic specific to compliance in the insights archive - the content type tells the insights upload service how to handle the payload.

Let me know if this works for you.

- Andrew

Comment 4 Peter Vreman 2019-04-03 07:26:19 UTC
According to --help there is no --payload option:

-------------
vrempet@li-lc-1443 ~
$ sudo insights-client --version
Client: 3.0.3-9
Core: 3.0.87-1

vrempet@li-lc-1443 ~
$ sudo insights-client --help | grep -E '(payload|content)'

vrempet@li-lc-1443 ~
$
-------------


For a go-live for all customers the client part for compliance shall have a simple and fluent out-of-the-box experience like standard Insights client has (install rpm and start service) .

My recommendation:

Create an additional rpm insights-client-compliance
- depends on openscap-scanner and security guide
- create a wrapper script with config file insights-compliance with config file that selects the policies to run (sequential)
- systemd timer to run insights-compliance daily
- systemd insights-compliance must run before the daily insights timer is run to have up-to-date results


It must be a simple straight forward experience out-of-the-box also for users that are not scap-gurus like me.

Peter

Comment 5 Peter Vreman 2019-04-03 11:06:25 UTC
The following command with the latest insights client 3.0.87 worked successful:

-------
vrempet@li-lc-1443 ~
$ sudo insights-client --payload /var/lib/insights/latest-compliance-report.xml --content-type application/vnd.redhat.compliance.something+tgz --verbose
2019-04-03 07:26:37,956    DEBUG insights.client.client Logging initialized
...
2019-04-03 07:26:41,534    DEBUG insights.client.connection Canonical facts collected:
{"bios_uuid": "4217b589-1550-a8d0-92b6-...9d"}
2019-04-03 07:26:41,534    DEBUG insights.client.connection Uploading /var/lib/insights/latest-compliance-report.xml to https://li-lc-1017.hag.hilti.com:443/redhat_access/r/insights/platform/upload/api/v1/upload
2019-04-03 07:26:46,647    DEBUG insights.client.connection Upload status: 202 Accepted
2019-04-03 07:26:46,647    DEBUG insights.client.connection Request ID: None
2019-04-03 07:26:46,647    DEBUG insights.client.connection
-------

Sadly the web insights site is not working (redirected to Openshift Preview cluster installer page) to confirm there is now data to visualize


Please always make explicit also which insights client core version is required for customers like me that have set auto_update=false

Peter


Note You need to log in before you can comment on or make changes to this bug.