Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1517238 - No alert when volume becomes degraded or partial
Summary: No alert when volume becomes degraded or partial
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: web-admin-tendrl-notifier
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Shubhendu Tripathi
QA Contact: Filip Balák
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-24 11:25 UTC by Filip Balák
Modified: 2017-12-18 04:38 UTC (History)
6 users (show)

Fixed In Version: tendrl-gluster-integration-1.5.4-13.el7rhgs.noarch tendrl-notifier-1.5.4-6.el7rhgs.noarch
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-18 04:38:26 UTC


Attachments (Terms of Use)
gluster cli (deleted)
2017-12-05 04:44 UTC, Shubhendu Tripathi
no flags Details
events sequence (deleted)
2017-12-05 04:44 UTC, Shubhendu Tripathi
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:3478 normal SHIPPED_LIVE RHGS Web Administration packages 2017-12-18 09:34:49 UTC
Red Hat Bugzilla 1520832 None None None Never

Internal Links: 1520832

Description Filip Balák 2017-11-24 11:25:25 UTC
Description of problem:
There are no alerts when volume's status becomes `degraded` or `partial`. There are only alerts related to bricks that are down when nodes are disconnected.

Version-Release number of selected component (if applicable):
tendrl-ansible-1.5.4-1.el7rhgs.noarch
tendrl-ui-1.5.4-4.el7rhgs.noarch
tendrl-grafana-plugins-1.5.4-5.el7rhgs.noarch
tendrl-selinux-1.5.3-2.el7rhgs.noarch
tendrl-commons-1.5.4-4.el7rhgs.noarch
tendrl-api-1.5.4-2.el7rhgs.noarch
tendrl-api-httpd-1.5.4-2.el7rhgs.noarch
tendrl-monitoring-integration-1.5.4-5.el7rhgs.noarch
tendrl-grafana-selinux-1.5.3-2.el7rhgs.noarch
tendrl-node-agent-1.5.4-5.el7rhgs.noarch
tendrl-notifier-1.5.4-3.el7rhgs.noarch
tendrl-gluster-integration-1.5.4-4.el7rhgs.noarch
glusterfs-3.8.4-52.el7rhgs.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Import cluster with disperse volume into tendrl.
2. Ensure that tendrl user has valid mail and enabled notifications.
3. Mount volume on client.
4. Start checking snmp messages:
# journalctl -u snmptrapd -fe
5. Start disconnecting hosts and wait after each change is in grafana.
6. Disconnect enough hosts so the volume becomes `Degraded` and after that `Partial`
6. Check for mails.
7. Check alerts in tendrl.

Actual results:
There are no messages related to the volume's status. There are only alerts related to stopped bricks. No notification is sent also when the volume is healthy again.

Expected results:
There should be notifications related to change of volume's status.

Additional info:

Comment 2 Filip Balák 2017-12-04 19:44:43 UTC
There are notifications when the volume status is changed to Degraded or Partial but the messages look like this:
``[Tendrl Alert] Volume State, WARNING: status changed-State of volume: volume_gama_disperse_4_plus_2x2 changed from (degraded) to (partial)``
``[Tendrl Alert] Volume State, WARNING: status changed-State of volume: volume_gama_disperse_4_plus_2x2 changed from up to (degraded)``
I think that degraded and partial status should not be in brackets as up and down status is also not in brackets.

There is also no notification when status of volume changes from partial to up (when I start all nodes). I have waited for more then 15 minutes and all nodes are up in grafana but there is no notification.

--> ASSIGNED

Tested with:
tendrl-node-agent-1.5.4-8.el7rhgs.noarch
tendrl-notifier-1.5.4-5.el7rhgs.noarch
tendrl-selinux-1.5.4-1.el7rhgs.noarch
tendrl-commons-1.5.4-5.el7rhgs.noarch
tendrl-api-1.5.4-3.el7rhgs.noarch
tendrl-api-httpd-1.5.4-3.el7rhgs.noarch
tendrl-monitoring-integration-1.5.4-8.el7rhgs.noarch
tendrl-grafana-selinux-1.5.4-1.el7rhgs.noarch
tendrl-ansible-1.5.4-2.el7rhgs.noarch
tendrl-ui-1.5.4-4.el7rhgs.noarch
tendrl-grafana-plugins-1.5.4-8.el7rhgs.noarch

Comment 4 Shubhendu Tripathi 2017-12-05 04:44:26 UTC
Created attachment 1362977 [details]
gluster cli

Comment 5 Shubhendu Tripathi 2017-12-05 04:44:53 UTC
Created attachment 1362978 [details]
events sequence

Comment 10 Filip Balák 2017-12-11 15:24:24 UTC
Looks ok. --> VERIFIED

Tested with:
tendrl-ansible-1.5.4-6.el7rhgs.noarch
tendrl-ui-1.5.4-6.el7rhgs.noarch
tendrl-grafana-plugins-1.5.4-14.el7rhgs.noarch
tendrl-selinux-1.5.4-1.el7rhgs.noarch
tendrl-commons-1.5.4-9.el7rhgs.noarch
tendrl-api-1.5.4-4.el7rhgs.noarch
tendrl-api-httpd-1.5.4-4.el7rhgs.noarch
tendrl-monitoring-integration-1.5.4-14.el7rhgs.noarch
tendrl-grafana-selinux-1.5.4-1.el7rhgs.noarch
tendrl-node-agent-1.5.4-15.el7rhgs.noarch
tendrl-notifier-1.5.4-6.el7rhgs.noarch

Comment 12 errata-xmlrpc 2017-12-18 04:38:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3478


Note You need to log in before you can comment on or make changes to this bug.