Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1511961 - Middleware - Error on generating reports on middleware servers
Summary: Middleware - Error on generating reports on middleware servers
Keywords:
Status: CLOSED EOL
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.9.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.10.0
Assignee: Xavier Canal
QA Contact: Sunil Kondkar
URL:
Whiteboard:
Depends On:
Blocks: 1514124
TreeView+ depends on / blocked
 
Reported: 2017-11-10 14:08 UTC by Sunil Kondkar
Modified: 2018-03-30 17:21 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1514124 (view as bug list)
Environment:
Last Closed: 2018-01-08 14:25:51 UTC
Category: ---
Cloudforms Team: Middleware


Attachments (Terms of Use)
Screenshot_report-Error (deleted)
2017-11-10 14:08 UTC, Sunil Kondkar
no flags Details
evm.log (deleted)
2017-11-10 14:11 UTC, Sunil Kondkar
no flags Details

Description Sunil Kondkar 2017-11-10 14:08:49 UTC
Created attachment 1350520 [details]
Screenshot_report-Error

Description of problem:

Cloud Intel >> Reports >> Reports >> Performance by Asset Type->Middleware servers:
All reports under Middleware servers show error in UI and in evm.log

Please find attached screenshot and errors in evm.log

Version-Release number of selected component (if applicable):

Version 5.9.0.8.20171109215303_ed87902

How reproducible:

Always

Steps to Reproduce:

1. Login to CFME
2. Add a middleware provider which has middleware server.
2. Navigate to cloud Intel >> Reports
3. Click Reports >> Performance by Asset Type >> Middleware servers
4. Click on the link for 'JVM Heap Usage - daily over last week'
(Any report link under Middleware Servers)
​​5. Click on 'Queue' button at the top left on the screen to generate the report

Actual results:

Errors in UI and evm.log on generating middleware server reports

Expected results: Reports should be generated on middleware server.


Additional info:

Below is in evm.log:


evm.log:

[----] I, [2017-11-10T08:31:44.768101 #13353:158dbac]  INFO -- : <AuditSuccess> MIQ(Generator.queue_generate_table) userid: [admin] - Generate Report: 'JVM Non Heap Usage - daily over last week', successfully initiated
[----] I, [2017-11-10T08:31:44.768206 #13353:158dbac]  INFO -- : MIQ(MiqTask#update_status) Task: [5] [Queued] [Ok] [Task has been queued]
[----] I, [2017-11-10T08:31:44.777567 #13353:158dbac]  INFO -- : MIQ(MiqQueue.put) Message id: [199],  id: [], Zone: [default], Role: [], Server: [], Ident: [reporting], Target id: [], Instance id: [121], Task id: [], Command: [MiqReport._async_generate_table], Timeout: [3600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [5, {:userid=>"admin", :mode=>"async", :report_source=>"Requested by user"}]
[----] I, [2017-11-10T08:31:45.833462 #2077:11f713c]  INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=reporting, wcount=2, priority=200
[----] I, [2017-11-10T08:31:48.580837 #13343:11f713c]  INFO -- : MIQ(MiqReportingWorker::Runner#get_message_via_drb) Message id: [199], MiqWorker id: [41], Zone: [default], Role: [], Server: [], Ident: [reporting], Target id: [], Instance id: [121], Task id: [], Command: [MiqReport._async_generate_table], Timeout: [3600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [5, {:userid=>"admin", :mode=>"async", :report_source=>"Requested by user"}], Dequeued in: [3.806931086] seconds
[----] I, [2017-11-10T08:31:48.581042 #13343:11f713c]  INFO -- : MIQ(MiqQueue#deliver) Message id: [199], Delivering...
[----] I, [2017-11-10T08:31:48.583113 #13343:11f713c]  INFO -- : MIQ(MiqTask#update_status) Task: [5] [Active] [Ok] [Generating report]
[----] I, [2017-11-10T08:31:48.592811 #13343:11f713c]  INFO -- : MIQ(ManageIQ::Providers::Hawkular::MiddlewareManager#with_provider_connection) Connecting through ManageIQ::Providers::Hawkular::MiddlewareManager: [Hawkular-Provider]
[----] I, [2017-11-10T08:31:48.595494 #13293:11f713c]  INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 4.
[----] I, [2017-11-10T08:31:48.601346 #13293:11f713c]  INFO -- : MIQ(MiqQueue.put) Message id: [200],  id: [], Zone: [default], Role: [], Server: [9648b08e-e110-4c0d-9bf7-4ea640130607], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: []
[----] I, [2017-11-10T08:31:48.606763 #13293:11f713c]  INFO -- : MIQ(MiqQueue.put) Message id: [201],  id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [ready], Deliver On: [], Data: [], Args: []
[----] E, [2017-11-10T08:31:48.687147 #13343:11f713c] ERROR -- : [NoMethodError]: undefined method `invert' for nil:NilClass  Method:[block in method_missing]
[----] E, [2017-11-10T08:31:48.687374 #13343:11f713c] ERROR -- : /var/www/miq/vmdb/app/models/mixins/live_metrics_mixin.rb:66:in `block in live_metrics_config

Comment 2 Sunil Kondkar 2017-11-10 14:11:00 UTC
Created attachment 1350521 [details]
evm.log

Comment 4 Xavier Canal 2017-11-13 08:33:12 UTC
Did you follow the instructions to enable the specific metrics needed for that report?
In the PR are listed: https://github.com/ManageIQ/manageiq/pull/16174

Comment 5 Sunil Kondkar 2017-11-13 12:11:05 UTC
@Xavier yes.

The statistics for undertow subsystem are enabled in eap configuration.

The issue is not only with EAP web metrics, the error appears for all types of reports that come under middleware servers. Below reports show error which can be seen in the attached screenshot:

-EAP Web Metrics - every minute over the last hour
-EAP Web Metrics - hourly over the last day
-JVM Garbage Collection - daily over last week
-JVM Heap Usage - daily over last week
-JVM Non Heap Usage - daily over last week
-Transactions Pool - every minute over the last hour
-Transactions Pool - hourly over the last day

Additional info:  There is an existing bug which may be related where user is not able to see the metrics on Middleware Server Summary->Monitoring->Utilization page. It shows" No Utilization data available". below is the bug for reference:
https://bugzilla.redhat.com/show_bug.cgi?id=1510548

Comment 6 Xavier Canal 2017-11-14 16:34:25 UTC
I see, this is happening because the middleware servers you are using are MiddlewareServerWildfly or MiddlewareServerEap instead of MiddlewareServer.

Everything points that https://github.com/ManageIQ/manageiq-providers-hawkular/pull/60 breaks middleware server reports.

It is missing the live metrics definition yaml file. There is only the definition for the MiddlewareServer, not for the wildfly/eap ones.
https://github.com/ManageIQ/manageiq/blob/master/app/models/mixins/live_metrics_mixin.rb#L72

`config_file` param it now has values as middleware_server, middleware_server_wildfly and middleware_server_eap. For the last two options, file doesn't exist.

I am digging into the scenario in order to have a proper solution.

Comment 7 Xavier Canal 2017-11-15 17:41:57 UTC
Got a merge in here: https://github.com/ManageIQ/manageiq-providers-hawkular/pull/106

Comment 8 Greg Blomquist 2017-11-15 22:37:05 UTC
Hi Xavier,

the BZ status should be POST when the associated PR is merged.

Marking the BZ at ON_QA assumes the PR has been included in a build and is available for QE to test.


Note You need to log in before you can comment on or make changes to this bug.