Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 452121 - cmirror fails to report fully synced after creation on one node in cluster
Summary: cmirror fails to report fully synced after creation on one node in cluster
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Cluster Suite
Classification: Retired
Component: cmirror-kernel
Version: 4
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
Assignee: Jonathan Earl Brassow
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-06-19 14:50 UTC by Corey Marthaler
Modified: 2013-09-23 15:32 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-23 15:32:43 UTC


Attachments (Terms of Use)
log from taft-03 (deleted)
2008-06-19 15:20 UTC, Corey Marthaler
no flags Details
log (with extra debugging) from taft-03 when this happened (deleted)
2008-06-25 14:47 UTC, Corey Marthaler
no flags Details
log (with extra debugging) from taft-04 when this happened (deleted)
2008-06-25 14:48 UTC, Corey Marthaler
no flags Details
log (with extra debugging) from taft-01 when this happened (deleted)
2008-06-25 14:58 UTC, Corey Marthaler
no flags Details
log (with extra debugging) from taft-02 when this happened (deleted)
2008-06-25 15:51 UTC, Corey Marthaler
no flags Details

Description Corey Marthaler 2008-06-19 14:50:13 UTC
Description of problem:
This may be related to bz 240864.
I created two cmirrors and one of them is reported as 0.00% synced on taft-03.


Creating mirror(s) on taft-04...
taft-04: lvcreate -m 2 -n syncd_secondary_3legs_1 -L 800M helter_skelter
/dev/sdh1:0-1000 /dev/sde1:0-1000 /dev/sdf1:0-1000 /dev/sdg1:0-150
taft-04: lvcreate -m 2 -n syncd_secondary_3legs_2 -L 800M helter_skelter
/dev/sdh1:0-1000 /dev/sde1:0-1000 /dev/sdf1:0-1000 /dev/sdg1:0-150

Waiting until all mirrors become fully syncd...
        0/2 mirror(s) are fully synced: ( 1=8.00% 2=0.50% )
        0/2 mirror(s) are fully synced: ( 1=25.00% 2=16.50% )
        0/2 mirror(s) are fully synced: ( 1=42.50% 2=32.50% )
        0/2 mirror(s) are fully synced: ( 1=59.00% 2=48.00% )
        0/2 mirror(s) are fully synced: ( 1=76.50% 2=64.00% )
        0/2 mirror(s) are fully synced: ( 1=93.50% 2=81.00% )
        1/2 mirror(s) are fully synced: ( 1=100.00% 2=98.50% )
        2/2 mirror(s) are fully synced: ( 1=100.00% 2=100.00% )
syncd_secondary_3legs_2 is not fully sync'ed on taft-03 (0.00%)

[root@taft-01 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00

[root@taft-02 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00

[root@taft-03 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog   0.00

[root@taft-04 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00


2.6.9-71.ELsmp

lvm2-2.02.37-3.el4    BUILT: Thu Jun 12 10:09:19 CDT 2008
lvm2-cluster-2.02.37-3.el4    BUILT: Thu Jun 12 10:22:07 CDT 2008
device-mapper-1.02.25-2.el4    BUILT: Mon Jun  9 09:28:41 CDT 2008
cmirror-1.0.1-1    BUILT: Tue Jan 30 17:28:02 CST 2007
cmirror-kernel-2.6.9-41.4    BUILT: Tue Jun  3 13:54:29 CDT 2008

Comment 1 Corey Marthaler 2008-06-19 15:20:03 UTC
Created attachment 309850 [details]
log from taft-03

Comment 2 Corey Marthaler 2008-06-25 13:31:09 UTC
Hit this last night while attempting to reproduce bz 450939.

Creating mirror(s) on taft-04...
taft-04: lvcreate --corelog -m 1 -n syncd_primary_core_2legs_1 -L 800M
helter_skelter /dev/sdf1:0-1000 /dev/sdh1:0-1000
taft-04: lvcreate --corelog -m 1 -n syncd_primary_core_2legs_2 -L 800M
helter_skelter /dev/sdf1:0-1000 /dev/sdh1:0-1000

Waiting until all mirrors become fully syncd...
        0/2 mirror(s) are fully synced: ( 1=11.00% 2=0.00% )
        0/2 mirror(s) are fully synced: ( 1=33.00% 2=22.00% )
        0/2 mirror(s) are fully synced: ( 1=53.50% 2=42.50% )
        0/2 mirror(s) are fully synced: ( 1=74.50% 2=63.50% )
        0/2 mirror(s) are fully synced: ( 1=95.50% 2=84.00% )
        2/2 mirror(s) are fully synced: ( 1=100.00% 2=100.00% )
syncd_primary_core_2legs_2 is not fully sync'ed on taft-01 (0.00%)


[root@taft-01 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
    0.00                  


[root@taft-02 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
  100.00                  


[root@taft-03 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
    0.00                  


[root@taft-04 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
  100.00                  


Comment 3 Corey Marthaler 2008-06-25 14:47:25 UTC
Created attachment 310272 [details]
log (with extra debugging) from taft-03 when this happened

Comment 4 Corey Marthaler 2008-06-25 14:48:00 UTC
Created attachment 310273 [details]
log (with extra debugging) from taft-04 when this happened

Comment 5 Corey Marthaler 2008-06-25 14:58:31 UTC
Created attachment 310277 [details]
log (with extra debugging) from taft-01 when this happened

Comment 6 Corey Marthaler 2008-06-25 15:51:57 UTC
Created attachment 310279 [details]
log (with extra debugging) from taft-02 when this happened

Comment 7 Corey Marthaler 2008-07-09 19:01:53 UTC
Hit this after an upconvert.

Up converting linear(s) back to mirror(s) on taft-04...
taft-04: lvconvert --corelog -m 1 helter_skelter/nonsyncd_secondary_core_2legs_1
/dev/sde1:0-1000 /dev/sdf1:0-1000
taft-04: lvconvert --corelog -m 1 helter_skelter/nonsyncd_secondary_core_2legs_2
/dev/sde1:0-1000 /dev/sdf1:0-1000


[root@taft-01 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
       100.00


[root@taft-02 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00


[root@taft-03 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00


[root@taft-04 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00


Comment 8 Corey Marthaler 2008-07-09 19:12:47 UTC
These are the only "odd" messages during this failure, on taft-01 (the only node
that reports the 2nd mirror as 100%):

dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change


Comment 11 Corey Marthaler 2011-01-17 20:24:32 UTC
FWIW, this issue still exists and continues to cause basic cmirror regression
tests to fail.

2.6.9-94.ELsmp

lvm2-2.02.42-9.el4    BUILT: Thu Oct 21 15:49:57 CDT 2010
lvm2-cluster-2.02.42-9.el4    BUILT: Thu Oct 21 15:46:55 CDT 2010
device-mapper-1.02.28-3.el4    BUILT: Thu Mar  4 14:48:16 CST 2010
cmirror-1.0.2-1.el4    BUILT: Thu Feb 26 15:29:27 CST 2009
cmirror-kernel-2.6.9-43.14.el4    BUILT: Wed Dec 22 16:24:19 CST 2010

Comment 12 Lon Hohberger 2013-09-23 15:32:43 UTC
The Red Hat Cluster Suite product is past end-of-life; closing.


Note You need to log in before you can comment on or make changes to this bug.