Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1518969 - Failure to detach volume from a controller node after a backup of a cinder volume with LVM signature
Summary: Failure to detach volume from a controller node after a backup of a cinder vo...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Eric Harney
QA Contact: Avi Avraham
URL:
Whiteboard:
Depends On: 1261083
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-29 21:35 UTC by Mark Jones
Modified: 2018-02-14 14:51 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-02-14 14:50:28 UTC
tshefi: automate_bug-


Attachments (Terms of Use)

Description Mark Jones 2017-11-29 21:35:42 UTC
Description of problem:

When backing up a cinder volume where the volume has a LVM signature, the source volume can get stuck in detaching on the controller node causing the backup to fail:

+-----------------------+-----------------------------------------------------------------------------------------------------------------+
| Field                 | Value                                                                                                           |
+-----------------------+-----------------------------------------------------------------------------------------------------------------+
| availability_zone     | nova                                                                                                            |
| container             | 84/8c/848cbb6d-9dce-4cd3-9340-29575500d3ce                                                                      |
| created_at            | 2017-11-29T17:24:04.000000                                                                                      |
| data_timestamp        | 2017-11-29T17:22:27.000000                                                                                      |
| description           |                                                                |
| fail_reason           | Unexpected error while running command.                                                                         |
|                       | Command: multipath -f 36006016052503e00e633511a2ad5e711                                                         |
|                       | Exit code: 1                                                                                                    |
|                       | Stdout: u'Nov 29 17:31:40 | 36006016052503e00e633511a2ad5e711: map in use\nNov 29 17:31:40 | failed to remove   |
|                       | multipath map 36006016052503e00e6335                                                                            |
| has_dependent_backups | False                                                                                                           |
| id                    | 848cbb6d-9dce-4cd3-9340-29575500d3ce                                                                            |
| is_incremental        | False                                                                                                           |
| name                  | **********_volume                                                                                              |
| object_count          | 4                                                                                                               |
| size                  | 5                                                                                                               |
| snapshot_id           | e750aae5-089f-4d13-a053-aedb7edb8c2a                                                                            |
| status                | error                                                                                                           |
| updated_at            | 2017-11-29T17:32:20.000000                                                                                      |
| volume_id             | 7579ead8-3551-4b90-8485-848d84e43b00                                                                            |
+-----------------------+-----------------------------------------------------------------------------------------------------------------+
 
/var/log/cinder/backup.log:

2017-11-29 17:29:19.140 898476 WARNING oslo.privsep.daemon [req-39ee4da3-6358-404e-aa7e-8e6d9b1313de db1e6fb0f4d94a5f8f23695479780f45 7ddb6f70931a49e29811fbc1b85ba410 - default default] privsep log: 2017-11-29 17:28:42.291 946368 ERROR oslo.privsep.daemon Exit code: 1

2017-11-29 17:29:19.140 898476 WARNING oslo.privsep.daemon [req-39ee4da3-6358-404e-aa7e-8e6d9b1313de db1e6fb0f4d94a5f8f23695479780f45 7ddb6f70931a49e29811fbc1b85ba410 - default default] privsep log: 2017-11-29 17:28:42.291 946368 ERROR oslo.privsep.daemon Stdout: u'Nov 29 17:28:02 | 36006016052503e00e633511a2ad5e711: map in use\nNov 29 17:28:02 | failed to remove multipath map 36006016052503e00e633511a2ad5e711\n'

Version-Release number of selected component (if applicable):


How reproducible:

Repeatedly with Cinder volumes that contain LVM signatures

Steps to Reproduce:
1. Create a Cinder volume and write a LVM signature on it by creating a VG and LV
2. Perform a Cinder Backup operation for that volume


Actual results:

Backup errors out

Expected results:

Backup succeeds

Additional info:

This appears to be very similar to the issue in BZ 1499044 and documented in https://access.redhat.com/solutions/3213311. The customer is testing whether the use of LVM filter on the controller nodes inhibits this issue.

Comment 1 Mark Jones 2017-12-03 17:47:00 UTC
The customer has completed their testing where the LVM filter is defined as per https://access.redhat.com/solutions/3213311 and this has resolved this specific error during the Cinder Backup operation.


Note You need to log in before you can comment on or make changes to this bug.