Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1365034 - [rbd-mirror] - failed to commit journal event
Summary: [rbd-mirror] - failed to commit journal event
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RBD
Version: 2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: 3.0
Assignee: Jason Dillaman
QA Contact: Parikshith
URL:
Whiteboard:
: 1421311 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-08 11:53 UTC by Hemanth Kumar
Modified: 2017-12-05 23:31 UTC (History)
8 users (show)

Fixed In Version: RHEL: ceph-12.1.2-1.el7cp Ubuntu: ceph_12.1.2-2redhat1xenial
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-05 23:31:14 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:3387 normal SHIPPED_LIVE Red Hat Ceph Storage 3.0 bug fix and enhancement update 2017-12-06 03:03:45 UTC
Ceph Project Bug Tracker 16962 None None None 2016-08-08 20:03:04 UTC

Comment 2 Jason Dillaman 2016-08-08 14:18:31 UTC
The errors on the "remote" side are actually just errors recorded within the image journals. You had some event on the primary side (unrelated to mirroring) where you witnessed the following errors:

2016-08-05 08:02:00.903776 7fb903fff700 -1 librbd::SnapshotProtectRequest: image must support layering
2016-08-05 08:02:00.903786 7fb903fff700 -1 librbd::SnapshotProtectRequest: encountered error: (38) Function not implemented
2016-08-05 08:02:01.004965 7fcecafcdd80 -1 librbd: parent image must support layering
2016-08-05 08:02:01.053367 7fe6a7e46d80 -1 librbd: rbd image dataset2 already exists
2016-08-05 08:04:28.369797 7ff3fa7fc700 -1 librbd::SnapshotProtectRequest: image must support layering
2016-08-05 08:04:28.369808 7ff3fa7fc700 -1 librbd::SnapshotProtectRequest: encountered error: (38) Function not implemented
2016-08-05 08:04:28.454584 7f1207490d80 -1 librbd: parent image must support layering
2016-08-05 08:04:28.502443 7fc272e36d80 -1 librbd: rbd image dataset3 already exists
2016-08-05 08:06:59.738275 7f0289ffb700 -1 librbd::SnapshotProtectRequest: image must support layering
2016-08-05 08:06:59.738287 7f0289ffb700 -1 librbd::SnapshotProtectRequest: encountered error: (38) Function not implemented
2016-08-05 08:06:59.820645 7fa1e352bd80 -1 librbd: parent image must support layering
2016-08-05 08:06:59.869198 7f9799d11d80 -1 librbd: rbd image dataset4 already exists
2016-08-05 08:09:20.475211 7f28e1ffb700 -1 librbd::SnapshotProtectRequest: image must support layering
2016-08-05 08:09:20.475222 7f28e1ffb700 -1 librbd::SnapshotProtectRequest: encountered error: (38) Function not implemented

Therefore, you have images where you attempted to protect snapshot that didn't support layering. This is unrelated to IPv4 / IPv6 and instead is just a split-brain caused by an unhandled error code.

Comment 3 Neil Levine 2016-08-08 18:38:20 UTC
Not clear on what the customer impact here is?

Comment 4 Jason Dillaman 2016-08-08 19:47:21 UTC
An illegal operation (attempting to protect a snapshot against an image that doesn't support that feature), resulted in a split-brain for the affected image. The customer would need to request a full resync of the affected image to get mirroring started again for the image.

In general, if a maintenance operation fails (except in a few white listed cases), we need to treat the result as a potential delta between the primary and non-primary image. This is an example of a case that should be white listed.

Comment 5 Neil Levine 2016-08-08 20:28:41 UTC
Is this a release note for 2.0 and a re-assign to 2.1?

Comment 8 Jason Dillaman 2017-01-04 21:17:49 UTC
This fix is included in Ceph v10.2.6

Comment 10 Jason Dillaman 2017-02-12 17:42:38 UTC
*** Bug 1421311 has been marked as a duplicate of this bug. ***

Comment 17 errata-xmlrpc 2017-12-05 23:31:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3387


Note You need to log in before you can comment on or make changes to this bug.