Ceph » rbd

Steps :
Set up 2 ceph clusters for rbd mirroring.

$ bin/rbd --cluster site-a create -s 8M data/src1
$ bin/rbd --cluster site-a map -t nbd data/src1

$ xfs_io -d -c 'pwrite -S 0xadad -b 4M 0 5M' /dev/nbd0

$ bin/rbd --cluster site-a snap create data/src1@snap1
$ bin/rbd --cluster site-a snap protect data/src1@snap1
$ bin/rbd --cluster site-a clone data/src1@snap1 data/dst1

$ bin/rbd --cluster site-a unmap -t nbd data/src1
$ bin/rbd --cluster site-a map -t nbd data/dst1

#Enable mirroring on both parent and child images
$ bin/rbd --cluster site-a mirror image enable data/src1 snapshot
$ bin/rbd --cluster site-a mirror image enable data/dst1 snapshot

#Wait until synced

$ xfs_io -d -c 'pwrite -S 0x11 -b 4M 7M 1M' /dev/nbd0
$ xfs_io -d -c 'pread -v 4M 512' /dev/nbd0
00400000: ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ................
00400010: ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ................
00400020: ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ................
00400030: ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ................
00400040: ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ad ................
...

$ bin/rbd --cluster site-a mirror image snapshot data/dst1
$ bin/rbd --cluster site-a unmap -t nbd data/dst1
$ bin/rbd --cluster site-a mirror image demote data/dst1
$ bin/rbd --cluster site-b mirror image promote data/dst1
$ bin/rbd --cluster site-b map -t nbd data/dst1
/dev/nb0

#Data written directly to the clone has been copied
$ xfs_io -d -c 'pread -v 7M 512' /dev/nbd0
00700000: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700010: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700020: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700030: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700040: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700050: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
00700060: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ................
...

#The parent data has not been copied up to object 1:
$ xfs_io -d -c 'pread -v 4M 512' /dev/nbd0
00400000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00400060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...