Project

General

Profile

Bug #57804

Enabling sync on bucket not working

Added by Anonymous 4 months ago. Updated 28 days ago.

Status:
Need More Info
Priority:
Normal
Assignee:
-
Target version:
% Done:

0%

Source:
Tags:
multisite
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hello,

I'm having a problem when trying to enable sync on one of our buckets (multi-site) from master zone.

Here's part of the log from the RGW :

debug 2022-10-10T06:29:47.033+0000 7fce3d144700 0 RGW-SYNC:data:sync:shard87:entry:bucket_sync_sources[target=:[]):source_bucket=:[]):source_zone=<zone_2>]:bucket[<bucket>:<bucket_id>:34227<-<bucket>:<bucket_id>:34227]: bucket sync disabled

This command (from master zone) :

radosgw-admin bucket sync enable --bucket=<our_bucket>

does not seem to work after a few minutes running. Bucket still not enabled.

This command (from master zone) :

radosgw-admin bucket sync status --bucket=<our_bucket>

returns :

_ realm <realm id> (<realm>)
zonegroup <zonegroup id> (<zonegroup>)
zone <zone id> (<zone>)
bucket :<bucket>[<bucket id>])

source zone &lt;zone 2&gt; (&lt;zone 2&gt;)
source bucket &lt;bucket&gt;:&lt;bucket id&gt;
full sync: 0/60000 shards
incremental sync: 0/60000 shards
bucket is behind on 60000 shards_

This command :

radosgw-admin bucket sync init --bucket=<our_bucket> --source-zone=<zone_2>

returns :

2022-10-10T06:45:19.654+0000 7f49137fe700 0 ERROR: curl error: Failure when receiving data from the peer, maybe network unstable
2022-10-10T06:45:19.658+0000 7f49f64723c0 0 data sync: ERROR: failed to fetch bucket index status

And does not start initialisation of the bucket.

Would you help us in starting enabling and starting sync on this bucket?

Thanks by advance for your help.

History

#1 Updated by Casey Bodley 4 months ago

  • Status changed from New to Need More Info
  • Tags set to multisite

did the `bucket sync enable` command finish? i imagine it would take a while with 60000 index shards..

#2 Updated by Anonymous 4 months ago

Hello Casey,

The init command ended after 60 minutes running.

Unfortunately the two errors are returned constantly during this time :

"ERROR: curl error: Failure when receiving data from the peer, maybe network unstable"

and

"data sync: ERROR: failed to fetch bucket index status"

The status remained the same unfortunately after this.

Thank you for your help.

#3 Updated by Casey Bodley 3 months ago

i can only recommend running the command until it succeeds

#4 Updated by Pawel Stefanski 28 days ago

@Allexandre can you fill more data about your env - what was exact version you tested and hit this?

Also available in: Atom PDF