Bug #10671
closed
rados cppool don't work if target is a ecpool
Added by Italo Santos over 9 years ago.
Updated over 8 years ago.
Description
I'm trying to copy a ec pool to another ecpool but when I try I face the error bellow:
root@ceph0001:~# rados cppool ecpool newecpool
.rgw.buckets:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117 => newecpool:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117
error copying object: (2) No such file or directory
error copying pool .rgw.buckets => newecpool: (95) Operation not supported
In other hand if try copy a replicated pool to ecpool everything goes fine:
root@ceph0001:~# rados cppool ecpool replicated
.rgw.buckets:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117 => replicated:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117
.rgw.buckets:default.770771.56_mysql/2015-01/teste_backup-3-1421773772.sql.gz => replicated:default.770771.56_mysql/2015-01/teste_backup-3-1421773772.sql.gz
.rgw.buckets:default.4797.1__shadow_.gR7_-CzFxl5-1hMe60nb2qtUSQpb2h8_129 => replicated:default.4797.1__shadow_.gR7_-CzFxl5-1hMe60nb2qtUSQpb2h8_129
.rgw.buckets:default.4797.1_GP010004.MP4 => replicated:default.4797.1_GP010004.MP4
[...]
.rgw.buckets:default.4797.1_IPS-LWB-CONFIGURACAO/LWBUtilities/usr/local/sas/utilities/backupcontrol => replicated:default.4797.1_IPS-LWB-CONFIGURACAO/LWBUtilities/usr/local/sas/utilities/backupcontrol
.rgw.buckets:default.4797.1_IPS-LWB-CONFIGURACAO/LWBScripts/root/scripts/aggr_mtu_conf.sh => replicated:default.4797.1_IPS-LWB-CONFIGURACAO/LWBScripts/root/scripts/aggr_mtu_conf.sh
.rgw.buckets:default.770771.56__shadow_.pv59KVmTTFGzfAwoZA4ixRBu6HRrju3_5 => replicated:default.770771.56__shadow_.pv59KVmTTFGzfAwoZA4ixRBu6HRrju3_5
.rgw.buckets:default.4797.1__shadow_.gR7_-CzFxl5-1hMe60nb2qtUSQpb2h8_84 => replicated:default.4797.1__shadow_.gR7_-CzFxl5-1hMe60nb2qtUSQpb2h8_84
successfully copied pool ecpool
To ensure I made another test, coping a replicated pool to a ecpool and I face with the same error, coping ecpool to another ecpool:
root@ceph0001:~# rados cppool replicated newecpool
replicated:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117 => newecpool:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117
error copying object: (2) No such file or directory
error copying pool replicated => newecpool: (95) Operation not supported
cppool should figure out the write unit size based on the EC pool properties.
To ensure I made another test, coping a replicated pool to a ecpool and I face with the same error, coping ecpool to another ecpool:
root@ceph0001:~# rados cppool replicated newecpool
replicated:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117 => newecpool:default.4797.1__shadow_.guzLcfIwb-wC3-cFRZtnkW-5TFxvzLg_117
error copying object: (2) No such file or directory
error copying pool replicated => newecpool: (95) Operation not supported
I haven't tried the other tests you did, but I'm seeing this exact thing right now, cppool can't copy a replicated pool to an EC one.
rados cppool .rgw.buckets .rgw.buckets.new
.rgw.buckets:default.2995076.1__shadow_.i74hPhdtSEbT_jRVcaIB-CY_rbAgPuW_43 => .rgw.buckets.new:default.2995076.1__shadow_.i74hPhdtSEbT_jRVcaIB-CY_rbAgPuW_43
error copying object: (0) Success
error copying pool .rgw.buckets => .rgw.buckets.new: (95) Operation not supported
i am not able to reproduce this issue. could you let me know the version of ceph you are using ? i am using a84ae7b14e1aaa1c6f65d09a78230b54d0c25110 from the master branch. thanks.
$ ./ceph osd pool get rbd all
constructing Rados: ceph
constructing Rados: None
size: 3
min_size: 1
crash_replay_interval: 0
pg_num: 8
pgp_num: 8
crush_ruleset: 0
auid: 0
write_fadvise_dontneed: false
$ ./ceph osd pool create newecpool 12 12 erasure
constructing Rados: ceph
constructing Rados: None
pool 'newecpool' created
$ ./rados cppool rbd newecpool
rbd:test.cc => newecpool:test.cc
successfully copied pool rbd
$ ./ceph osd pool get newecpool all
constructing Rados: ceph
constructing Rados: None
size: 3
min_size: 2
crash_replay_interval: 0
pg_num: 12
pgp_num: 12
crush_ruleset: 1
auid: 0
erasure_code_profile: default
write_fadvise_dontneed: false
Ceph version
# ceph -v
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
Replicated pool
# ceph osd pool get .rgw.buckets size
size: 3
# ceph osd pool get .rgw.buckets min_size
min_size: 2
# ceph osd pool get .rgw.buckets crush_ruleset
crush_ruleset: 0
EC pool
# ceph osd pool get .rgw.buckets.new size
size: 18
# ceph osd pool get .rgw.buckets.new min_size
min_size: 16
# ceph osd pool get .rgw.buckets.new crush_ruleset
crush_ruleset: 14
# ceph osd pool get .rgw.buckets.new erasure_code_profile
erasure_code_profile: 16_2_r6_RS_jerasure
CRUSH rulesets
ruleset 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
ruleset 14
type erasure
min_size 3
max_size 20
step set_chooseleaf_tries 5
step take default
step chooseleaf indep 0 type host
step emit
h3. Erasure code profile
# ceph osd erasure-code-profile get 16_2_r6_RS_jerasure
directory=/usr/lib64/ceph/erasure-code
k=16
m=2
plugin=jerasure
technique=reed_sol_r6_op
@rados cppool
@
# rados cppool .rgw.buckets .rgw.buckets.new
.rgw.buckets:default.2995076.1__shadow_.i74hPhdtSEbT_jRVcaIB-CY_rbAgPuW_43 => .rgw.buckets.new:default.2995076.1__shadow_.i74hPhdtSEbT_jRVcaIB-CY_rbAgPuW_43
error copying object: (0) Success
error copying pool .rgw.buckets => .rgw.buckets.new: (95) Operation not supported
- Priority changed from Normal to High
note that we probably want to rewrite this eventaully to use the export|import path, which will be more complete/robust. cppool does not preserve snapshots for rbd images, for example
It would appear that this issue (sometimes) still exists in 9.2.0. I recently attempted to migrate 11 pools from replicated to erasure coded. Of these, 8 cppool operations completed successfully while the remaining 3 failed with the error above. Is there any workaround to this issue? Has a fix been further discussed?
Craig McElroy wrote:
It would appear that this issue (sometimes) still exists in 9.2.0. I recently attempted to migrate 11 pools from replicated to erasure coded. Of these, 8 cppool operations completed successfully while the remaining 3 failed with the error above. Is there any workaround to this issue? Has a fix been further discussed?
The problem is probably that the objects in question have omap data associated with them. rgw can now keep those objects in a separate replciated pool, but there is no tool to do this sort of migration currently.
- Status changed from New to Rejected
Yeah, omap is fundamentally not supported on ec at the moment (probably ever). Trying to do a cppool from a pool with omap objects to an ec pool should fail -- handling that correctly (using a config where rgw puts those objects in a replicated pool) is beyond the scope of rados cppool.
Also available in: Atom
PDF