Unable to create rbd image format 2 on Hammer cluster from Jewel client
I'm Unable to create an rbd image with format 2 on a Ceph Hammer cluster from a Ceph Jewel client:
On a Ubuntu Xenial client with Ceph jewel version 10.2.2-1xenial installed I issue the following command:
rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240
2016-07-13 12:01:31.865308 7f01fe293d40 -1 librbd: error writing header:
(38) Function not implemented
rbd: create error: (38) Function not implemented
If I run the same command on a Ubuntu Xenial client with Hammer 0.94.7-1xenial installed this works fine and the image is created.
The Ceph cluster on where the image gets created is a Fujitsu CD10K system running Ceph version ceph-0.94.3.3-0.el6.x86_64 on CentOS 6.7
Note: creating the image with format "1" works in both cases.
It seems that the newer client is trying to use a function which is not available on the cluster.
#1 Updated by Igor Podoski about 3 years ago
This is probably rbd default feature problem. Hammer suports: layering, strippingV2, exclusive_Lock and object_map, Jewel has additional: fast_diff, deep_flatten and journalin.
Looking on default parameters:
rbd_default_features = 3
rbd_default_features = 61
You can fix your issue in to ways:
First, choose what features do you want to enable, to be as much similar to Jewel defaults, I would propose
1. Proivde propper rbd feature set to rbd when creating image (by name):
rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240 --image-feature layering,object-map,exclusive-lock
2. Add line below to your
/etc/ceph_cd10k/ceph.conf file (by number):
rbd default format = 2
rbd default features = 13
Then you could create you image simply by:
rbd --conf /etc/ceph_cd10k/ceph.conf create CD10K_ONE/stefan-test-v2 --size 10240
"13" is a bit representation of enabled features, in hammer you have:
LAYERING = 1
STRIPINGV2 = 2
EXCLUSIVE_LOCK = 4
OBJECT_MAP = 8
"13" = (1 + 4 + 8), layering + exclusive-lock + object-map.
Looking on Jewel default: 61 = (1 + 4 + 8 + 16 + 32), additional fast_diff and deep_flatten is enabled
FAST_DIFF = 16
DEEP_FLATTEN = 32
JOURNALIN = 64