Bug #16677
closedUnable to create rbd image format 2 on Hammer cluster from Jewel client
0%
Description
I'm Unable to create an rbd image with format 2 on a Ceph Hammer cluster from a Ceph Jewel client:
On a Ubuntu Xenial client with Ceph jewel version 10.2.2-1xenial installed I issue the following command:
rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240
which returns:
2016-07-13 12:01:31.865308 7f01fe293d40 -1 librbd: error writing header:
(38) Function not implemented
rbd: create error: (38) Function not implemented
If I run the same command on a Ubuntu Xenial client with Hammer 0.94.7-1xenial installed this works fine and the image is created.
The Ceph cluster on where the image gets created is a Fujitsu CD10K system running Ceph version ceph-0.94.3.3-0.el6.x86_64 on CentOS 6.7
Note: creating the image with format "1" works in both cases.
It seems that the newer client is trying to use a function which is not available on the cluster.
Updated by Igor Podoski almost 8 years ago
Hello Stefan,
This is probably rbd default feature problem. Hammer suports: layering, strippingV2, exclusive_Lock and object_map, Jewel has additional: fast_diff, deep_flatten and journalin.
Looking on default parameters:
Hammer: rbd_default_features = 3
Jewel: rbd_default_features = 61
You can fix your issue in to ways:
First, choose what features do you want to enable, to be as much similar to Jewel defaults, I would propose "13"
.
1. Proivde propper rbd feature set to rbd when creating image (by name):rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240 --image-feature layering,object-map,exclusive-lock
2. Add line below to your /etc/ceph_cd10k/ceph.conf
file (by number):[client]
rbd default format = 2
rbd default features = 13
Then you could create you image simply by:rbd --conf /etc/ceph_cd10k/ceph.conf create CD10K_ONE/stefan-test-v2 --size 10240
The number "13"
is a bit representation of enabled features, in hammer you have:LAYERING = 1
STRIPINGV2 = 2
EXCLUSIVE_LOCK = 4
OBJECT_MAP = 8
So, for "13"
= (1 + 4 + 8), layering + exclusive-lock + object-map.
Looking on Jewel default: 61 = (1 + 4 + 8 + 16 + 32), additional fast_diff and deep_flatten is enabled
FAST_DIFF = 16
DEEP_FLATTEN = 32
JOURNALIN = 64
Regards,
Igor.
Updated by Stefan Kooman almost 8 years ago
@Igor Gajowiak: I can confirm that your suggestion (adjusting feature bit client side) fixes the issue. Thanks for the quick response and explanation!
Updated by Igor Podoski over 7 years ago
- Status changed from New to Rejected
- Assignee set to Igor Podoski
Not a bug