Ceph » mgr

Our application use object size mostly 512 bytes or lower, with 4K overhead we can't use standard stored or stored_raw metrics for pool usage

# ceph df
RAW STORAGE:
    CLASS       SIZE        AVAIL       USED        RAW USED     %RAW USED
    hdd_pg      4.3 PiB     1.0 PiB     3.2 PiB      3.2 PiB         75.91
    nvme_pg     161 TiB      44 TiB      96 TiB      117 TiB         72.65
    TOTAL       4.4 PiB     1.1 PiB     3.3 PiB      3.4 PiB         75.79

POOLS:
    POOL     ID     PGS       STORED      OBJECTS     USED        %USED     MAX AVAIL
    meta     17      2307     3.6 TiB       8.62G      96 TiB     85.58       5.4 TiB
    data     18     16384     1.1 PiB       3.16G     3.2 PiB     81.76       245 TiB

As we can see for store 3.6TiB we use 96TiB

We need expose avail_raw metrics and revert bytes_used metric back

(((8620017891 * 512) * 3) = 13240347480576 (12.04TiB)) - (((8620017891 * 4096) * 3) = 105922779844608 (96.34TiB)) = 92682432364032 (84.29TiB) that we can't see