Bug #58852: [rbd-mirror] snapshot-based mirroring doesn't propagate discards - rbd - Ceph

Actions

Copy link

Bug #58852

open

[rbd-mirror] snapshot-based mirroring doesn't propagate discards

Added by Ilya Dryomov about 1 year ago. Updated 2 months ago.

Status:

Fix Under Review

Priority:

Normal

Assignee:

Nithya Balachandran

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

52358

Crash signature (v1):

Crash signature (v2):

Description

$ rbd --cluster site-a create -s 100M data/img 
$ rbd --cluster site-a mirror image enable data/img snapshot
Mirroring enabled
$ sudo rbd map --cluster site-a -t nbd data/img
/dev/nbd0
$ xfs_io -d -c 'pwrite -b 4M 0 100M' /dev/nbd0
wrote 104857600/104857600 bytes at offset 0
100 MiB, 25 ops; 0:00:01.42 (69.952 MiB/sec and 17.4879 ops/sec)
$ rbd --cluster site-a du data/img
NAME  PROVISIONED  USED
img       100 MiB  100 MiB

$ rbd --cluster site-a mirror image snapshot data/img
Snapshot ID: 20
< ... snapshot sync ... >
$ rbd --cluster site-b du data/img
NAME  PROVISIONED  USED   
img       100 MiB  100 MiB

$ blkdiscard -l 80M /dev/nbd0
$ rbd --cluster site-a du data/img
NAME  PROVISIONED  USED  
img       100 MiB  20 MiB

$ rbd --cluster site-a mirror image snapshot data/img
Snapshot ID: 22
< ... snapshot sync ... >
$ rbd --cluster site-b du data/img
NAME  PROVISIONED  USED
img       100 MiB  100 MiB

It's not just that the area remains allocated on site-b, it still contains data at those offsets. The checksums don't match, of course:

$ xfs_io -d -c 'pread -v 0 512' /dev/nbd0
00000000:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000010:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000020:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000030:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000040:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000050:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000060:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000070:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000080:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000090:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000a0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000b0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000c0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000d0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000e0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000f0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000100:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000110:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000120:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000130:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000140:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000150:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000160:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000170:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000180:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000190:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001a0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001b0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001c0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001d0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001e0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000001f0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
read 512/512 bytes at offset 0
512.000000 bytes, 1 ops; 0.0016 sec (309.789 KiB/sec and 619.5787 ops/sec)
$ md5sum /dev/nbd0
067122afb788aa6349f7780dadbf9811  /dev/nbd0
$ sudo rbd unmap --cluster site-a -t nbd data/img
$ rbd --cluster site-a mirror image demote data/img
Image demoted to non-primary
$ rbd --cluster site-b mirror image promote data/img
Image promoted to primary
$ sudo rbd map --cluster site-b -t nbd data/img
/dev/nbd0
$ xfs_io -d -c 'pread -v 0 512' /dev/nbd0
00000000:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000010:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000020:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000030:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000040:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000050:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000060:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000070:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000080:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000090:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000a0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000b0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000c0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000d0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000e0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000000f0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000100:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000110:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000120:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000130:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000140:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000150:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000160:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000170:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000180:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
00000190:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001a0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001b0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001c0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001d0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001e0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
000001f0:  cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  ................
read 512/512 bytes at offset 0
512.000000 bytes, 1 ops; 0.0014 sec (353.357 KiB/sec and 706.7138 ops/sec)
$ md5sum /dev/nbd0
a5547703c21e4855c5e3e4abc5a668d6  /dev/nbd0