Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1512712 - Dashboard not updating statistics
Summary: Dashboard not updating statistics
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-dwh
Version: 4.1.6
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Shirly Radco
QA Contact: Pavel Stehlik
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-13 22:10 UTC by Christopher Wawak
Modified: 2017-11-15 12:18 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-15 12:18:05 UTC
oVirt Team: Metrics


Attachments (Terms of Use)

Description Christopher Wawak 2017-11-13 22:10:35 UTC
Description of problem:
I replaced the motherboard in a personal testing RHV 4.1.6 manager, and after I did so, data collection for the dashboard ceased. I suspect I may have munged something in the database when the system initially started up with the wrong time (the date/time was set to some un-sane setting when initially booting the system, 1971 or some such). 

2017-11-13 16:46:00|WflvmA|PSKxwq|b1zenN|OVIRT_ENGINE_DWH|SampleTimeKeepingJob|Default|5|tWarn|tWarn_1|Can not sample data, oVirt Engine is not updating the statistics. Please check your oVirt Engine status.|9704

I don't see much in /var/log/ovirt-engine/*.log that would give me a pointer to what's going on, nor do the postgresql logs shed any light. 

A logcollector is located at:
http://file.rdu.redhat.com/~cwawak/share/sosreport-LogCollector-20171113170236.tar.xz

Version-Release number of selected component (if applicable):
# rpm -qv ovirt-engine
ovirt-engine-4.1.6.2-0.1.el7.noarch

How reproducible:
Every time.

Steps to Reproduce:
1. Reinstall motherboard, don't set time
2. Boot RHVM


Actual results:
Data collection for dashboard stops

Expected results:
Data collection for dashboard continues

Additional info:

Comment 2 Shirly Radco 2017-11-14 07:15:48 UTC
Hi,

Please attach the history_configuration table 

Select * from history_configuration;

And the date you run this query at.

If now is for example '2017-11-13 17:20:56 EST'
then 'lastHourAggr' should be '2017-11-13 16:00:00 EST'
and  'lastDayAggr' should be '2017-11-12 00:00:00 EST'

This issue should be resolved in ovirt 4.1.7 release.

Comment 3 Shirly Radco 2017-11-14 09:50:16 UTC
For clarification, I do not need the actual 'history_configuration' table.
Just running the above query on the ov_engine_history db and attach here the result along with the timestamp of the time you run the query at for reference.

Comment 4 Christopher Wawak 2017-11-14 11:34:24 UTC
Hi Shirly, thanks for the quick response. Let me know if this is what you were looking for:

-bash-4.2$ date
Tue Nov 14 06:32:33 EST 2017
-bash-4.2$ psql
psql (9.2.23)
Type "help" for help.

postgres=# \c ovirt_engine_history
You are now connected to database "ovirt_engine_history" as user "postgres".
ovirt_engine_history=# Select * from history_configuration;
     var_name      | var_value |      var_datetime
-------------------+-----------+------------------------
 default_language  | en_US     |
 MinimalETLVersion | 4.1.0     |
 firstSync         | false     | 2017-11-05 19:58:00-05
 lastHourAggr      |           | 2017-11-14 05:44:55-05
 HourlyAggFailed   | false     |
 lastDayAggr       |           | 2017-11-13 15:45:08-05
(6 rows)

Comment 5 Shirly Radco 2017-11-14 13:01:51 UTC
I can see that the lastHourAggr and lastDayAggr are corrupted.

Please update them like this :

ovirt_engine_history=# UPDATE history_configuration set var_datetime = date_trunc('hour', now())- interval '2 hour' WHERE var_name = 'lastHourAggr';

ovirt_engine_history=# UPDATE history_configuration set var_datetime = cast(now() as date)- interval '1 day' WHERE var_name = 'lastDayAggr';

Second thing is that the error you are seeing 
16:46:00|WflvmA|PSKxwq|b1zenN|OVIRT_ENGINE_DWH|SampleTimeKeepingJob|Default|5|tWarn|tWarn_1|Can not sample data, oVirt Engine is not updating the statistics. Please check your oVirt Engine status.|9704

Is also documented in the ovirt-engine-dwhd.log file.

If it does not repeat every 20 or 60 second but in a non consistent interval it is probably due to the change of the collection interval to 20 second.
We have updated the collection back to 60 second in ovirt-engine-dwh-4.1.7.
See bug bug #1478859.

Please check you ovirt-engine-dwh package version and upgrade if needed.
You can also just add a conf file to /etc/ovirt-engine-dwh/ovirt-engine-dwhd.conf.d/:
with the parameter
DWH_SAMPLING=60

and restart ovirt-engine-dwhd service.

Comment 6 Christopher Wawak 2017-11-14 14:37:31 UTC
Shirly - those SQL statements seem to have gotten things moving again. I now see the dashboard updating. Sincere thanks!

Do you think this corruption was related to the inconsistent time when the motherboard was swapped?

Comment 9 Shirly Radco 2017-11-15 09:48:26 UTC
I'm happy to hear this solved the issue. Can we close this bug?

Comment 10 Christopher Wawak 2017-11-15 12:18:05 UTC
Things are running well on my end. If this is fixed in the next release Iā€™m happy to close. Thanks again!


Note You need to log in before you can comment on or make changes to this bug.