Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 969026 - [engine] Storage Domain that was destroyed from GUI did not get completely removed from DB
Summary: [engine] Storage Domain that was destroyed from GUI did not get completely re...
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.2.0
Hardware: All
OS: Linux
Target Milestone: ---
: 3.5.0
Assignee: Liron Aravot
QA Contact: Ori Gofen
Whiteboard: storage
Depends On:
Blocks: rhev3.5beta 1156165
TreeView+ depends on / blocked
Reported: 2013-05-30 13:58 UTC by Gadi Ickowicz
Modified: 2016-05-26 01:48 UTC (History)
12 users (show)

Fixed In Version: ovirt-3.5.0-alpha2
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-02-16 19:10:13 UTC
oVirt Team: Storage
Target Upstream Version:
amureini: Triaged+

Attachments (Terms of Use)
vdsm + engine logs (deleted)
2013-05-30 13:58 UTC, Gadi Ickowicz
no flags Details

System ID Priority Status Summary Last Updated
oVirt gerrit 25231 None None None Never
oVirt gerrit 25232 master MERGED core: proceedStorageDomainTreatmentByDomainType - proper compensation Never

Description Gadi Ickowicz 2013-05-30 13:58:51 UTC
Created attachment 754868 [details]
vdsm + engine logs

Description of problem:
After creating a new DC with 1 host and 1 storage domain, and 1 additional domain which is not attached, all storages were destroyed (directly on the storage server). To clean the RHEVM the host was moved to maintenance, DC was force-removed and additional domain was destroyed. However the domain remained in the storage_domain_static table. Now it is not possible to add another domain to the RHEVM with the same name as that one.

Queries to the DB:
# psql -d engine -U postgres -c "select * from storage_domain_static;"
                 id                  |                storage                 | storage_name | storage_domain_type | storage_type | storage_domain_format_type |         _create_date          |         _update_dat
 feb6aa0e-4302-42b9-86a7-84d902fea07e | AxHc8t-CRgE-FKft-qrSl-PG8O-Svs5-9m5dL3 | iscsi_2      |                   1 |            3 | 3                          | 2013-05-26 14:39:33.223723+03 |                    
 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0      |                   0 |            3 | 3                          | 2013-05-30 14:25:00.450573+03 | 2013-05-30 14:25:04
 f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1      |                   1 |            3 | 0                          | 2013-05-30 14:26:22.152056+03 | 2013-05-30 14:26:22

# # psql -d engine -U postgres -c "select * from storage_domains;"
                  id                  |                storage                 | storage_name |           storage_pool_id            | available_disk_size | used_disk_size | commited_disk_size | status | storage_p
 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0      | 7cc34b9a-ea43-49ba-a73d-f667bb93f664 |                     |                |                  0 |      4 | TestDC   
 f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1      |                                      |                  10 |              4 |                  0 |        |          
(2 rows)

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Create datacenter, cluster and add 1 host
2. Create storage domain and attach it to DC
3. Create additonal storage domain (not attached to DC)
4. Destroy storage (directly on the storage server)
5. When host is in non-operational move it to maintenance and force-remove DC
6. Destroy remaining storage domain

Actual results:
After repeating this procedure several times I was suddenly unable to re-add a storage domain with the same name as the additional domain. Checking the DB showed that the storage domain entry remained in the storage_domain_static table, but not in the storage_domains table

Expected results:
destroying a domain should remove it completely from the DB

Additional info:

Comment 1 Alissa 2013-07-09 12:40:03 UTC
What did you mean by storage_domains table? there is no such table in the db... storage_domains is a view. in which specific table did you see the records remained?

Also, you wrote "After repeating this procedure several times" -is this consistent? How many times the procedure needs to be repeated for reproduction?

Comment 3 Alissa 2013-08-19 14:47:22 UTC
Some information and the cause for the problem in the bug:
relevant storage domain id is feb6aa0e-4302-42b9-86a7-84d902fea07e
1. ReconstructMasterDomainCommand starts, compensates storage_domain_static. 
2. ForceRemoveStorageDomainCommand starts.
It removes the storage domain from the db.
3. ReconstructMasterDomainCommand fails.
It triggers the compensation of the storage domain. Since it was removed from db by step 2 (ForceRemoveStorageDomainCommand), the compensation gets storage domain inserted  again to storage_domain_static table.

Comment 5 Allon Mureinik 2014-02-16 09:09:22 UTC
Liron, was this solved as part of your on reconstruct?

Comment 6 Liron Aravot 2014-03-02 13:51:29 UTC
Allon, no.
I provided patches to solve the issue, added to the tracker.

Comment 7 Kevin Alon Goldblatt 2014-07-07 07:19:48 UTC
All instances of the storage domains were removed from the database after performing the above scenario ONCE. Is this sufficient to move this defect to Verified? Gadi?

Comment 8 Gadi Ickowicz 2014-07-08 08:16:41 UTC
(In reply to Kevin Alon Goldblatt from comment #7)
> All instances of the storage domains were removed from the database after
> performing the above scenario ONCE. Is this sufficient to move this defect
> to Verified? Gadi?

If it worked, then I guess so

Comment 9 Ori Gofen 2014-07-29 07:51:16 UTC
verified on oVirt beta.2

Comment 11 Allon Mureinik 2015-02-16 19:10:13 UTC
RHEV-M 3.5.0 has been released, closing this bug.

Note You need to log in before you can comment on or make changes to this bug.