Project

General

Profile

Bug #43339

bucket sync init command failed

Added by Chang Liu 4 months ago. Updated 2 months ago.

Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

[root@VM_71_139_centos build]# ./bin/radosgw-admin bucket sync init --bucket=test
ERROR: source zone not specified
[root@VM_71_139_centos build]# ./bin/radosgw-admin bucket sync init --bucket=test --source-zone=us-east
019-12-17T04:42:57.646+0000 7f86bbb0eec0  0 bucket sync zone:bb1d9f59 bucket:test connection object to zone bb1d9f59-4282-4576-8bbd-1b7a7adeb045 does not exist
ERROR: sync.init() returned ret=-22
[root@VM_71_139_centos build]# ./bin/radosgw-admin bucket sync init --bucket=test --source-zone=us-east --rgw-zone=es
could not get bucket info for bucket=test
ERROR: could not init bucket: (2) No such file or directory

bucket is created in sourze zone. rgw couldn't get bucket info in elastic search zone. here is a conflict in `rgw-zone` param.

History

#1 Updated by Chang Liu 4 months ago

elastic search zone does not sync metadate, so all those `bucket sync *` commands are not able to used in elastic zone ?

#2 Updated by Chang Liu 4 months ago

Chang Liu wrote:

elastic search zone does not sync metadate, so all those `bucket sync *` commands are not able to used in elastic zone ?

I found that bucket info will be synced in here https://github.com/ceph/ceph/blob/0f0ae55a0d1dd92612dab81f1ed0cc0ded703bbe/src/rgw/rgw_data_sync.cc#L3309-L3315

#3 Updated by Chang Liu 4 months ago

I add some debug log, it shows that sync module could get bucket instance info. but radosgw-admin could not . as following:

2019-12-17T09:49:45.490+0000 7f82004c3ec0  0 RGW-SYNC:data:sync:shard[69]:entry[test:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.1]:bucket[test:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.1]: debuglc: get bucket instance info: 0
2019-12-17T09:49:45.498+0000 7f82004c3ec0  0 RGW-SYNC:data:sync:shard[120]:entry[test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2]:bucket[test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2]: debuglc: get bucket instance info: 0
2019-12-17T09:49:45.670+0000 7f82004c3ec0  0 RGW-SYNC:data:sync:shard[120]:entry[test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2]:bucket[test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2]: debuglc: get bucket instance info: 0
[root@VM_71_139_centos build]# ./bin/radosgw-admin bucket sync status --bucket=test --source-zone=us-east --rgw-zone=es
could not get bucket info for bucket=test
[root@VM_71_139_centos build]# ./bin/radosgw-admin bucket sync status --bucket=test2 --source-zone=us-east --rgw-zone=es
could not get bucket info for bucket=test2

    yield call(new RGWGetBucketInstanceInfoCR(sync_env->async_rados, sync_env->store, bs.bucket, &bucket_info));
    tn->log(0, SSTR("debuglc: get bucket instance info: " << retcode));   <<-- add logging here
    if (retcode == -ENOENT) {
      /* bucket instance info has not been synced in yet, fetch it now */

#4 Updated by Chang Liu 4 months ago

2019-12-17T11:41:03.306+0000 7f593cc54ec0 20 get_system_obj_state: rctx=0x7ffc6cc59fd0 obj=es.rgw.meta:root:test2 state=0x7f593ef0cf80 s->prefetch_data=0
could not get bucket info for bucket=test2

[root@VM_71_139_centos build]# ./bin/rados ls -p es.rgw.meta --all 2>/dev/null
root    .bucket.meta.test:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.1
root    .bucket.meta.test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2
[root@VM_71_139_centos build]# ./bin/rados ls -p us-east.rgw.meta --all 2>/dev/null
root    .bucket.meta.test:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.1
users.keys    BXAQKB0F10U19M903MND
users.keys    KBZYKLMUIG00XV6SFEO1
users.uid    test
users.uid    test2.buckets
users.keys    0555b35654ad1656d804
root    test
users.uid    test2
root    .bucket.meta.test2:bb1d9f59-4282-4576-8bbd-1b7a7adeb045.4578.2
root    test2
users.uid    synchronization-user

#5 Updated by Casey Bodley 4 months ago

Chang Liu wrote:

Chang Liu wrote:

elastic search zone does not sync metadate, so all those `bucket sync *` commands are not able to used in elastic zone ?

I found that bucket info will be synced in here https://github.com/ceph/ceph/blob/0f0ae55a0d1dd92612dab81f1ed0cc0ded703bbe/src/rgw/rgw_data_sync.cc#L3309-L3315

data sync is only fetching the bucket instance object, and not its bucket entrypoint

commands like 'bucket sync init' that take --bucket have to look up its bucket entrypoint object by name. the bucket entrypoint stores the current bucket instance id, so we can use that to find the right bucket instance object. if metadata sync hasn't fetched the bucket entrypoint object yet, commands with --bucket will fail this way

#6 Updated by Casey Bodley 3 months ago

  • Status changed from New to Need More Info

#7 Updated by Casey Bodley 2 months ago

  • Status changed from Need More Info to Can't reproduce

Also available in: Atom PDF