Ceph » RADOS

Lets imagine we have a pool TEST.
In the PG we have object OBJ of size 1M.

We create snap SNAP-1 and write some 4K to OBJ.
As result we get OBJ.1 that takes 1M and OBJ.head that reuses all but 4K.
The total data usage is 1M + 4K.

Now we move PG to other OSD.
In some cases OBJ.head + OBJ.1 will take 2M.

The example of this happening is in attachment snap-pg-move-history.sh.
When data is on original PG on OSD.0:

ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS
0 ssd 0.09859 1.00000 101 GiB 1.1 GiB 101 MiB 0 B 21 MiB 100 GiB 1.09 1.05 2 up
1 ssd 0.09859 1.00000 101 GiB 1.0 GiB 740 KiB 0 B 20 MiB 100 GiB 0.99 0.95 1 up
TOTAL 202 GiB 2.1 GiB 101 MiB 0 B 41 MiB 200 GiB 1.04
MIN/MAX VAR: 0.95/1.05 STDDEV: 0.05

And after forcibly moving PG to OSD.

ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS
0 ssd 0.09859 1.00000 101 GiB 1.0 GiB 756 KiB 0 B 21 MiB 100 GiB 0.99 0.91 1 up
1 ssd 0.09859 1.00000 101 GiB 1.2 GiB 201 MiB 0 B 21 MiB 100 GiB 1.18 1.09 2 up
TOTAL 202 GiB 2.2 GiB 201 MiB 0 B 42 MiB 200 GiB 1.09
MIN/MAX VAR: 0.91/1.09 STDDEV: 0.10

The script was tested on Reef, but I do not believe it is limited to it.