Ceph » rbd

Hi,

I believe that the fix introduced in response to https://tracker.ceph.com/issues/42248 has broken the 'whole-object' parameter switch for rbd diff when comparing two snapshots.

The parameter is described in the man page as:

  Specifies that the diff should be limited to the extents of a  full  ob-
  ject instead of showing intra-object deltas. When the object map feature
  is enabled on an image, limiting the diff to  the  object  extents  will
  dramatically  improve  performance since the differences can be computed
  by examining the in-memory object map instead of querying RADOS for each
  object within the image.

I'm using the KRBD client with kernel 5.11.7 and Ceph Octopus 15.2.11 as part of Proxmox PVE 6.4 which is based on Debian 10. Images have the following features and I've performed offline object map checks and rebuilds (no errors reported).

To reproduce my issue I first create a new RBD image (default features are 63), map it using KRBD, write some data, create first snapshot, write a single object (4 MiB), create a second snapshot and then list the differences:

[admin@kvm1a ~]# rbd create rbd_hdd/test --size 40G
[admin@kvm1a ~]# rbd info rbd_hdd/test
rbd image 'test':
        size 40 GiB in 10240 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 73363f8443987b
        block_name_prefix: rbd_data.73363f8443987b
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
        op_features:
        flags:
        create_timestamp: Wed May 12 23:01:11 2021
        access_timestamp: Wed May 12 23:01:11 2021
        modify_timestamp: Wed May 12 23:01:11 2021
[admin@kvm1a ~]# rbd map rbd_hdd/test
/dev/rbd18
[admin@kvm1a ~]# dd if=/dev/zero of=/dev/rbd18 bs=64M count=1
1+0 records in
1+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 0.668701 s, 100 MB/s
[admin@kvm1a ~]# sync
[admin@kvm1a ~]# rbd snap create rbd_hdd/test@snap1
[admin@kvm1a ~]# dd if=/dev/zero of=/dev/rbd18 bs=4M count=1
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.265691 s, 15.8 MB/s
[admin@kvm1a ~]# sync
[admin@kvm1a ~]# rbd snap create rbd_hdd/test@snap2
[admin@kvm1a ~]# rbd diff --from-snap snap1 rbd_hdd/test@snap2 --format=json
[{"offset":0,"length":4194304,"exists":"true"}]

This is all 100% perfect, when one however then adds the '--whole-object' switch it lists all allocated regions of the snapshot instead of only the differences:

[admin@kvm1b ~]# rbd diff --from-snap snap1 rbd_hdd/test@snap2 --format=json --whole-object
[{"offset":0,"length":4194304,"exists":"true"},
{"offset":4194304,"length":4194304,"exists":"true"},
{"offset":8388608,"length":4194304,"exists":"true"},
{"offset":12582912,"length":4194304,"exists":"true"},
{"offset":16777216,"length":4194304,"exists":"true"},
{"offset":20971520,"length":4194304,"exists":"true"},
{"offset":25165824,"length":4194304,"exists":"true"},
{"offset":29360128,"length":4194304,"exists":"true"},
{"offset":33554432,"length":4194304,"exists":"true"},
{"offset":37748736,"length":4194304,"exists":"true"},
{"offset":41943040,"length":4194304,"exists":"true"},
{"offset":46137344,"length":4194304,"exists":"true"},
{"offset":50331648,"length":4194304,"exists":"true"},
{"offset":54525952,"length":4194304,"exists":"true"},
{"offset":58720256,"length":4194304,"exists":"true"},
{"offset":62914560,"length":4194304,"exists":"true"}]

[admin@kvm1a ~]# rbd du rbd_hdd/test
NAME        PROVISIONED  USED
test@snap1       40 GiB   64 MiB
test@snap2       40 GiB   64 MiB
test             40 GiB    4 MiB
<TOTAL>          40 GiB  132 MiB

Regards
David Herselman