Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1677319 (Gluster_5_Affecting_oVirt_4.3) - [Tracker] Gluster 5 issues affecting oVirt 4.3
Summary: [Tracker] Gluster 5 issues affecting oVirt 4.3
Keywords:
Status: POST
Alias: Gluster_5_Affecting_oVirt_4.3
Product: ovirt-distribution
Classification: oVirt
Component: General
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
high
high with 2 votes vote
Target Milestone: ovirt-4.3.4
: ---
Assignee: Sahina Bose
QA Contact: SATHEESARAN
URL:
Whiteboard:
Depends On: 1672318 1677160 1682925 1649054 1651246 1671556 1672711 1673058 1674406 1678570 1679406 1679965 1679968 1683008 1683900 1684385
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-14 14:23 UTC by Sandro Bonazzola
Modified: 2019-04-05 09:17 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
oVirt Team: Gluster
sbonazzo: ovirt-4.3?


Attachments (Terms of Use)

Description Sandro Bonazzola 2019-02-14 14:23:34 UTC
Tracking Gluster 5 bugs reported affecting oVirt 4.3

Comment 1 James 2019-02-14 14:48:48 UTC
I'm also having problems with Gluster bricks going offline since upgrading to oVirt 4.3 yesterday (previously I've never had a single issue with gluster nor have had a brick ever go down).  I suspect this will continue to happen daily as some other users on this group have suggested.  I was able to pull some logs from engine and gluster from around the time the brick dropped.  My setup is 3 node HCI and I was previously running the latest 4.2 updates (before upgrading to 4.3).  My hardware is has a lot of overhead and I'm on 10Gbe gluster backend (the servers were certainly not under any significant amount of load when the brick went offline).  To recover I had to place the host in maintenance mode and reboot (although I suspect I could have simply unmounted and remounted gluster mounts). 

grep "2019-02-14" engine.log-20190214 | grep "GLUSTER_BRICK_STATUS_CHANGED"
2019-02-14 02:41:48,018-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler1) [5ff5b093] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/non_prod_b/non_prod_b of volume non_prod_b of cluster Default from UP to DOWN via cli.
2019-02-14 03:20:11,189-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/engine/engine of volume engine of cluster Default from DOWN to UP via cli.
2019-02-14 03:20:14,819-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/prod_b/prod_b of volume prod_b of cluster Default from DOWN to UP via cli.
2019-02-14 03:20:19,692-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/isos/isos of volume isos of cluster Default from DOWN to UP via cli.
2019-02-14 03:20:25,022-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/prod_a/prod_a of volume prod_a of cluster Default from DOWN to UP via cli.
2019-02-14 03:20:29,088-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/non_prod_b/non_prod_b of volume non_prod_b of cluster Default from DOWN to UP via cli.
2019-02-14 03:20:34,099-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler3) [760f7851] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/non_prod_a/non_prod_a of volume non_prod_a of cluster Default from DOWN to UP via cli

glusterd.log

# grep -B20 -A20 "2019-02-14 02:41" glusterd.log
[2019-02-14 02:36:49.585034] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_b
[2019-02-14 02:36:49.597788] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2 times between [2019-02-14 02:36:49.597788] and [2019-02-14 02:36:49.900505]
[2019-02-14 02:36:53.437539] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a
[2019-02-14 02:36:53.452816] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:36:53.864153] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a
[2019-02-14 02:36:53.875835] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:36:30.958649] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2019-02-14 02:36:35.322129] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_b
[2019-02-14 02:36:39.639645] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume isos
[2019-02-14 02:36:45.301275] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_a
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2 times between [2019-02-14 02:36:53.875835] and [2019-02-14 02:36:54.180780]
[2019-02-14 02:37:59.193409] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:38:44.065560] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2019-02-14 02:38:44.072680] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume isos
[2019-02-14 02:38:44.077841] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a
[2019-02-14 02:38:44.082798] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_b
[2019-02-14 02:38:44.088237] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_a
[2019-02-14 02:38:44.093518] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_b
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2 times between [2019-02-14 02:37:59.193409] and [2019-02-14 02:38:44.100494]
[2019-02-14 02:41:58.649683] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 6 times between [2019-02-14 02:41:58.649683] and [2019-02-14 02:43:00.286999]
[2019-02-14 02:43:46.366743] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2019-02-14 02:43:46.373587] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume isos
[2019-02-14 02:43:46.378997] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a
[2019-02-14 02:43:46.384324] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_b
[2019-02-14 02:43:46.390310] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_a
[2019-02-14 02:43:46.397031] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_b
[2019-02-14 02:43:46.404083] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:45:47.302884] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2019-02-14 02:45:47.309697] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume isos
[2019-02-14 02:45:47.315149] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a
[2019-02-14 02:45:47.320806] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_b
[2019-02-14 02:45:47.326865] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_a
[2019-02-14 02:45:47.332192] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_b
[2019-02-14 02:45:47.338991] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:46:47.789575] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_b
[2019-02-14 02:46:47.795276] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_a
[2019-02-14 02:46:47.800584] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prod_b
[2019-02-14 02:46:47.770601] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2019-02-14 02:46:47.778161] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume isos
[2019-02-14 02:46:47.784020] I [MSGID: 106499] [glusterd-handler.c:4389:__glusterd_handle_status_volume] 0-management: Received status volume req for volume non_prod_a

engine.log

# grep -B20 -A20 "2019-02-14 02:41:48" engine.log-20190214
2019-02-14 02:41:43,495-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 172c9ee8
2019-02-14 02:41:43,609-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@479fcb69, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6443e68f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2b4cf035, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5864f06a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6119ac8c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1a9549be, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5614cf81, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@290c9289, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5dd26e8, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@35355754, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@452deeb4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@8f8b442, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@647e29d3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7bee4dff, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@511c4478, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1c0bb0bd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@92e325e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@260731, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@33aaacc9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@72657c59, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@aa10c89], log id: 172c9ee8
2019-02-14 02:41:43,610-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 3a0e9d63
2019-02-14 02:41:43,703-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@5ca4a20f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@57a8a76, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@7bd1b14], log id: 3a0e9d63
2019-02-14 02:41:43,704-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVDOVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 49966b05
2019-02-14 02:41:44,213-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 49966b05
2019-02-14 02:41:44,214-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 30db0ce2
2019-02-14 02:41:44,311-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@61a309b5, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@ea9cb2e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@749d57bd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1c49f9d0, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@655eb54d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@256ee273, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3bd079dc, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6804900f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@78e0a49f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2acfbc8a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@12e92e96, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5ea1502c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2398c33b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7464102e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2f221daa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7b561852, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1eb29d18, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a030b80, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@75739027, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3eac8253, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@34fc82c3], log id: 30db0ce2
2019-02-14 02:41:44,312-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 6671d0d7
2019-02-14 02:41:44,329-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:44,345-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:44,374-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:44,405-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@f6a9696, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@558e3332, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@5b449da], log id: 6671d0d7
2019-02-14 02:41:44,406-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVDOVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 6d2bc6d3
2019-02-14 02:41:44,908-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 6d2bc6d3
2019-02-14 02:41:44,909-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = Host0, GlusterVolumeAdvancedDetailsVDSParameters:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5', volumeName='non_prod_b'}), log id: 36ae23c6
2019-02-14 02:41:47,336-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:47,351-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:47,379-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler3) [7b9bd2d] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:47,979-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@7a4a787b, log id: 36ae23c6
2019-02-14 02:41:48,018-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler1) [5ff5b093] EVENT_ID: GLUSTER_BRICK_STATUS_CHANGED(4,086), Detected change in status of brick host2.replaced.domain.com:/gluster_bricks/non_prod_b/non_prod_b of volume non_prod_b of cluster Default from UP to DOWN via cli.
2019-02-14 02:41:48,046-04 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler1) [5ff5b093] EVENT_ID: GLUSTER_BRICK_STATUS_DOWN(4,151), Status of brick host2.replaced.domain.com:/gluster_bricks/non_prod_b/non_prod_b of volume non_prod_b on cluster Default is down.
2019-02-14 02:41:48,139-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler1) [5ff5b093] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:48,140-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler3) [7b9bd2d] START, GlusterServersListVDSCommand(HostName = Host0, VdsIdVDSCommandParametersBase:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5'}), log id: e1fb23
2019-02-14 02:41:48,911-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler3) [7b9bd2d] FINISH, GlusterServersListVDSCommand, return: [10.12.0.220/24:CONNECTED, host1.replaced.domain.com:CONNECTED, host2.replaced.domain.com:CONNECTED], log id: e1fb23
2019-02-14 02:41:48,930-04 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler1) [5ff5b093] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[a45fe964-9989-11e8-b3f7-00163e4bf18a=GLUSTER]', sharedLocks=''}'
2019-02-14 02:41:48,931-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler3) [7b9bd2d] START, GlusterVolumesListVDSCommand(HostName = Host0, GlusterVolumesListVDSParameters:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5'}), log id: 68f1aecc
2019-02-14 02:41:49,366-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler3) [7b9bd2d] FINISH, GlusterVolumesListVDSCommand, return: {6c05dfc6-4dc0-41e3-a12f-55b4767f1d35=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1952a85, 3f8f6a0f-aed4-48e3-9129-18a2a3f64eef=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2f6688ae, 71ff56d9-79b8-445d-b637-72ffc974f109=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@730210fb, 752a9438-cd11-426c-b384-bc3c5f86ed07=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c3be510c, c3e7447e-8514-4e4a-9ff5-a648fe6aa537=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@450befac, 79e8e93c-57c8-4541-a360-726cec3790cf=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1926e392}, log id: 68f1aecc
2019-02-14 02:41:49,489-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = Host0, VdsIdVDSCommandParametersBase:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5'}), log id: 38debe74
2019-02-14 02:41:49,581-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5e5a7925, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2cdf5c9e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@443cb62, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@49a3e880, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@443d23c0, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1250bc75, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@8d27d86, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5e6363f4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@73ed78db, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@64c9d1c7, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7fecbe95, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3a551e5f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2266926e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@88b380c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1209279e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3c6466, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@16df63ed, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@47456262, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1c2b88c3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7f57c074, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@12fa0478], log id: 38debe74
2019-02-14 02:41:49,582-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = Host0, VdsIdVDSCommandParametersBase:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5'}), log id: 7ec02237
2019-02-14 02:41:49,660-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@3eedd0bc, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@7f78e375, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@3d63e126], log id: 7ec02237
2019-02-14 02:41:49,661-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVDOVolumeListVDSCommand(HostName = Host0, VdsIdVDSCommandParametersBase:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5'}), log id: 42cdad27
2019-02-14 02:41:50,142-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 42cdad27
2019-02-14 02:41:50,143-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 12f5fdf2
2019-02-14 02:41:50,248-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2aaed792, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@8e66930, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@276d599e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1aca2aec, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@46846c60, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7d103269, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@30fc25fc, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7baae445, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1ea8603c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@62578afa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@33d58089, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1f71d27a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4205e828, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1c5bbac8, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@395a002, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@12664008, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7f4faec4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3e03d61f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1038e46d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@307e8062, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@32453127], log id: 12f5fdf2
2019-02-14 02:41:50,249-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 1256aa5e
2019-02-14 02:41:50,338-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@459a2ff5, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@123cab4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@1af41fbe], log id: 1256aa5e
2019-02-14 02:41:50,339-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVDOVolumeListVDSCommand(HostName = Host1, VdsIdVDSCommandParametersBase:{hostId='fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d'}), log id: 3dd752e4
2019-02-14 02:41:50,847-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 3dd752e4
2019-02-14 02:41:50,848-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 29a6272c
2019-02-14 02:41:50,954-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@364f3ec6, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@c7cce5e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@b3bed47, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@13bc244b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5cca81f4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@36aeba0d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@62ab384a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1047d628, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@188a30f5, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5bb79f3b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@60e5956f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4e3df9cd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7796567, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@60d06cf4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2cd2d36c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@d80a4aa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@411eaa20, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@22cac93b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@18b927bd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@101465f4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@246f927c], log id: 29a6272c
2019-02-14 02:41:50,955-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 501814db
2019-02-14 02:41:51,044-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@1cd55aa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@32c5aba2, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@6ae123f4], log id: 501814db
2019-02-14 02:41:51,045-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVDOVolumeListVDSCommand(HostName = Host2, VdsIdVDSCommandParametersBase:{hostId='fd0752d8-2d41-45b0-887a-0ffacbb8a237'}), log id: 7acf4cbf
2019-02-14 02:41:51,546-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 7acf4cbf
2019-02-14 02:41:51,547-04 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler1) [5ff5b093] START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = Host0, GlusterVolumeAdvancedDetailsVDSParameters:{hostId='771c67eb-56e6-4736-8c67-668502d4ecf5', volumeName='non_prod_a'}), log id: 11c42649

Comment 2 James 2019-02-14 16:17:15 UTC
Brick log from around the time the brick went down:


[2019-02-14 02:20:29.611099] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:20:29.611131] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:ee716e24-e187-4b57-a371-cab544f41162-GRAPH_ID:0-PID:30671-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:20:29.619521] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:ee716e24-e187-4b57-a371-cab544f41162-GRAPH_ID:0-PID:30671-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:20:29.619867] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:ee716e24-e187-4b57-a371-cab544f41162-GRAPH_ID:0-PID:30671-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 19988 times between [2019-02-14 02:19:31.377315] and [2019-02-14 02:21:14.033991]
[2019-02-14 02:21:30.303440] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 10 times between [2019-02-14 02:21:30.303440] and [2019-02-14 02:23:20.421140]
[2019-02-14 02:23:33.142281] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 34 times between [2019-02-14 02:23:33.142281] and [2019-02-14 02:25:29.115156]
[2019-02-14 02:25:30.326469] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:25:53.973830] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.222"
[2019-02-14 02:25:53.973896] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:25:53.973928] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:4a6b8860-8274-4b3b-b400-d66cbfb97349-GRAPH_ID:0-PID:33522-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:25:54.627728] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:4a6b8860-8274-4b3b-b400-d66cbfb97349-GRAPH_ID:0-PID:33522-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:25:54.628149] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:4a6b8860-8274-4b3b-b400-d66cbfb97349-GRAPH_ID:0-PID:33522-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:25:56.396855] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.220"
[2019-02-14 02:25:56.396926] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:25:56.396957] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:963c2196-108c-485d-aca6-a236906d2acf-GRAPH_ID:0-PID:33635-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:25:56.404566] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:963c2196-108c-485d-aca6-a236906d2acf-GRAPH_ID:0-PID:33635-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:25:56.404866] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:963c2196-108c-485d-aca6-a236906d2acf-GRAPH_ID:0-PID:33635-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 25 times between [2019-02-14 02:25:30.326469] and [2019-02-14 02:27:25.965601]
[2019-02-14 02:28:10.538374] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 38 times between [2019-02-14 02:28:10.538374] and [2019-02-14 02:29:22.622679]
[2019-02-14 02:29:48.891040] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:29:56.026002] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:31:22.494824] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.221"
[2019-02-14 02:31:22.494914] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:31:22.494946] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:210b837a-1c98-4d88-aab8-5ce895398d81-GRAPH_ID:0-PID:18765-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:31:22.503920] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:210b837a-1c98-4d88-aab8-5ce895398d81-GRAPH_ID:0-PID:18765-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:31:22.504205] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:210b837a-1c98-4d88-aab8-5ce895398d81-GRAPH_ID:0-PID:18765-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 15301 times between [2019-02-14 02:29:48.891040] and [2019-02-14 02:31:28.963054]
[2019-02-14 02:31:31.016063] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 628 times between [2019-02-14 02:31:31.016063] and [2019-02-14 02:33:27.936726]
[2019-02-14 02:33:35.878655] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 1776 times between [2019-02-14 02:33:35.878655] and [2019-02-14 02:35:28.143659]
[2019-02-14 02:35:29.607848] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:36:00.483996] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.220"
[2019-02-14 02:36:00.484066] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:36:00.484098] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:9f2cd861-23c9-412b-be2c-8c45658bb155-GRAPH_ID:0-PID:39820-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:36:00.779203] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:9f2cd861-23c9-412b-be2c-8c45658bb155-GRAPH_ID:0-PID:39820-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:36:00.779662] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:9f2cd861-23c9-412b-be2c-8c45658bb155-GRAPH_ID:0-PID:39820-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:36:49.892372] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.222"
[2019-02-14 02:36:49.892451] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:36:49.892481] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:91cc19de-30ae-4639-a407-c6fce4682fa4-GRAPH_ID:0-PID:39353-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:36:49.900530] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:91cc19de-30ae-4639-a407-c6fce4682fa4-GRAPH_ID:0-PID:39353-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:36:49.900863] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:91cc19de-30ae-4639-a407-c6fce4682fa4-GRAPH_ID:0-PID:39353-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 15772 times between [2019-02-14 02:35:29.607848] and [2019-02-14 02:37:24.862502]
[2019-02-14 02:38:11.704945] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 23 times between [2019-02-14 02:38:11.704945] and [2019-02-14 02:39:24.469485]
[2019-02-14 02:39:47.550186] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 34 times between [2019-02-14 02:39:47.550186] and [2019-02-14 02:41:28.642587]
[2019-02-14 02:41:29.746302] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:42:15.811205] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.220"
[2019-02-14 02:42:15.811284] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:42:15.811317] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:6de46e97-ef84-43c8-a472-4e942471a8f6-GRAPH_ID:0-PID:43752-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:42:15.819841] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:6de46e97-ef84-43c8-a472-4e942471a8f6-GRAPH_ID:0-PID:43752-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:42:15.820156] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:6de46e97-ef84-43c8-a472-4e942471a8f6-GRAPH_ID:0-PID:43752-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 5147 times between [2019-02-14 02:41:29.746302] and [2019-02-14 02:43:28.777825]
[2019-02-14 02:43:35.252750] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 17909 times between [2019-02-14 02:43:35.252750] and [2019-02-14 02:45:29.331082]
[2019-02-14 02:45:29.497786] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:46:08.243167] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.222"
[2019-02-14 02:46:08.243230] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:46:08.243259] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:56fb6f47-3e05-4f07-abdf-cb49213cf5c0-GRAPH_ID:0-PID:44348-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:46:09.512844] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:56fb6f47-3e05-4f07-abdf-cb49213cf5c0-GRAPH_ID:0-PID:44348-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:46:09.513302] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:56fb6f47-3e05-4f07-abdf-cb49213cf5c0-GRAPH_ID:0-PID:44348-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2892 times between [2019-02-14 02:45:29.497786] and [2019-02-14 02:47:23.957871]
[2019-02-14 02:47:30.087716] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:47:42.722682] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.221"
[2019-02-14 02:47:42.722779] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:47:42.722810] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:34bb8b68-3483-4744-bd33-7a47cd4214b2-GRAPH_ID:0-PID:27248-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:47:42.731133] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:34bb8b68-3483-4744-bd33-7a47cd4214b2-GRAPH_ID:0-PID:27248-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:47:42.731447] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:34bb8b68-3483-4744-bd33-7a47cd4214b2-GRAPH_ID:0-PID:27248-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 564 times between [2019-02-14 02:47:30.087716] and [2019-02-14 02:49:28.480491]
[2019-02-14 02:49:32.129061] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 15058 times between [2019-02-14 02:49:32.129061] and [2019-02-14 02:51:26.667738]
[2019-02-14 02:52:35.362594] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:53:08.127443] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.222"
[2019-02-14 02:53:08.127528] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:53:08.127562] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:3fbc7931-7ac7-4dfd-9169-6271a18fc0f9-GRAPH_ID:0-PID:48068-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:53:08.135384] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:3fbc7931-7ac7-4dfd-9169-6271a18fc0f9-GRAPH_ID:0-PID:48068-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:53:08.135682] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:3fbc7931-7ac7-4dfd-9169-6271a18fc0f9-GRAPH_ID:0-PID:48068-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 11 times between [2019-02-14 02:52:35.362594] and [2019-02-14 02:53:24.927967]
[2019-02-14 02:53:56.793284] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 38 times between [2019-02-14 02:53:56.793284] and [2019-02-14 02:55:29.101365]
[2019-02-14 02:55:30.445077] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:56:14.259883] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.222"
[2019-02-14 02:56:14.259948] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:56:14.259980] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:d7f3996c-4c68-4e15-b0e2-0197bc358ddd-GRAPH_ID:0-PID:885-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:56:14.336000] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:d7f3996c-4c68-4e15-b0e2-0197bc358ddd-GRAPH_ID:0-PID:885-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:56:14.336327] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:d7f3996c-4c68-4e15-b0e2-0197bc358ddd-GRAPH_ID:0-PID:885-HOST:host2.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 19 times between [2019-02-14 02:55:30.445077] and [2019-02-14 02:57:28.589205]
[2019-02-14 02:57:35.071899] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 02:58:33.855724] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.220"
[2019-02-14 02:58:33.855805] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 02:58:33.855838] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:868382a8-7e38-481c-b5a3-80c1bdd21f28-GRAPH_ID:0-PID:4631-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)
[2019-02-14 02:58:33.864814] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-non_prod_b-server: disconnecting connection from CTX_ID:868382a8-7e38-481c-b5a3-80c1bdd21f28-GRAPH_ID:0-PID:4631-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
[2019-02-14 02:58:33.865124] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-non_prod_b-server: Shutting down connection CTX_ID:868382a8-7e38-481c-b5a3-80c1bdd21f28-GRAPH_ID:0-PID:4631-HOST:host0.replaced.domain.com-PC_NAME:non_prod_b-client-2-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 35 times between [2019-02-14 02:57:35.071899] and [2019-02-14 02:59:29.398638]
[2019-02-14 02:59:33.501166] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 03:00:01.622537] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 21104 times between [2019-02-14 02:59:33.501166] and [2019-02-14 03:01:29.325701]
[2019-02-14 03:01:30.635130] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 640 times between [2019-02-14 03:01:30.635130] and [2019-02-14 03:03:29.020755]
[2019-02-14 03:03:31.238543] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-02-14 03:04:00.722382] I [addr.c:54:compare_addr_and_update] 0-/gluster_bricks/non_prod_b/non_prod_b: allowed = "*", received addr = "10.11.0.221"
[2019-02-14 03:04:00.722466] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 7b741fe4-72ca-41ba-8efb-7add1e4fe6f3
[2019-02-14 03:04:00.722499] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-non_prod_b-server: accepted client from CTX_ID:ca4e5703-7129-4242-9fbf-9b119de9e0eb-GRAPH_ID:0-PID:35692-HOST:host1.replaced.domain.comm-PC_NAME:non_prod_b-client-2-RECON_NO:-0 (version: 5.3)

Comment 3 Netbulae 2019-02-18 11:43:20 UTC
Can we add https://bugzilla.redhat.com/show_bug.cgi?id=1306562 to this list? 

We would like to start using libgfapi but without live migration this doesn't work for us.

Comment 4 Netbulae 2019-02-18 12:07:00 UTC
(In reply to Netbulae from comment #3)
> Can we add https://bugzilla.redhat.com/show_bug.cgi?id=1306562 to this list? 
> 
> We would like to start using libgfapi but without live migration this
> doesn't work for us.

Ah sorry, this is a libvirt issue not GlusterFS. Please ignore


Note You need to log in before you can comment on or make changes to this bug.