Project

General

Profile

Bug #16677

Unable to create rbd image format 2 on Hammer cluster from Jewel client

Added by Stefan Kooman about 3 years ago. Updated almost 3 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
07/13/2016
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

I'm Unable to create an rbd image with format 2 on a Ceph Hammer cluster from a Ceph Jewel client:

On a Ubuntu Xenial client with Ceph jewel version 10.2.2-1xenial installed I issue the following command:

rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240

which returns:

2016-07-13 12:01:31.865308 7f01fe293d40 -1 librbd: error writing header:
(38) Function not implemented
rbd: create error: (38) Function not implemented

If I run the same command on a Ubuntu Xenial client with Hammer 0.94.7-1xenial installed this works fine and the image is created.

The Ceph cluster on where the image gets created is a Fujitsu CD10K system running Ceph version ceph-0.94.3.3-0.el6.x86_64 on CentOS 6.7

Note: creating the image with format "1" works in both cases.

It seems that the newer client is trying to use a function which is not available on the cluster.

History

#1 Updated by Igor Podoski about 3 years ago

Hello Stefan,

This is probably rbd default feature problem. Hammer suports: layering, strippingV2, exclusive_Lock and object_map, Jewel has additional: fast_diff, deep_flatten and journalin.

Looking on default parameters:
Hammer: rbd_default_features = 3
Jewel: rbd_default_features = 61

You can fix your issue in to ways:

First, choose what features do you want to enable, to be as much similar to Jewel defaults, I would propose "13".

1. Proivde propper rbd feature set to rbd when creating image (by name):
rbd --conf /etc/ceph_cd10k/ceph.conf create --image-format 2 CD10K_ONE/stefan-test-v2 --size 10240 --image-feature layering,object-map,exclusive-lock

2. Add line below to your /etc/ceph_cd10k/ceph.conf file (by number):
[client]
rbd default format = 2
rbd default features = 13

Then you could create you image simply by:
rbd --conf /etc/ceph_cd10k/ceph.conf create CD10K_ONE/stefan-test-v2 --size 10240

The number "13" is a bit representation of enabled features, in hammer you have:
LAYERING = 1
STRIPINGV2 = 2
EXCLUSIVE_LOCK = 4
OBJECT_MAP = 8

So, for "13" = (1 + 4 + 8), layering + exclusive-lock + object-map.

Looking on Jewel default: 61 = (1 + 4 + 8 + 16 + 32), additional fast_diff and deep_flatten is enabled
FAST_DIFF = 16
DEEP_FLATTEN = 32
JOURNALIN = 64

Regards,
Igor.

#2 Updated by Stefan Kooman about 3 years ago

@Igor: I can confirm that your suggestion (adjusting feature bit client side) fixes the issue. Thanks for the quick response and explanation!

#3 Updated by Igor Podoski almost 3 years ago

  • Status changed from New to Rejected
  • Assignee set to Igor Podoski

Not a bug

Also available in: Atom PDF