Project

General

Profile

Actions

Bug #57804

open

Enabling sync on bucket not working

Added by Anonymous over 1 year ago. Updated about 1 year ago.

Status:
Need More Info
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
multisite
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hello,

I'm having a problem when trying to enable sync on one of our buckets (multi-site) from master zone.

Here's part of the log from the RGW :

debug 2022-10-10T06:29:47.033+0000 7fce3d144700 0 RGW-SYNC:data:sync:shard87:entry:bucket_sync_sources[target=:[]):source_bucket=:[]):source_zone=<zone_2>]:bucket[<bucket>:<bucket_id>:34227<-<bucket>:<bucket_id>:34227]: bucket sync disabled

This command (from master zone) :

radosgw-admin bucket sync enable --bucket=<our_bucket>

does not seem to work after a few minutes running. Bucket still not enabled.

This command (from master zone) :

radosgw-admin bucket sync status --bucket=<our_bucket>

returns :

_ realm <realm id> (<realm>)
zonegroup <zonegroup id> (<zonegroup>)
zone <zone id> (<zone>)
bucket :<bucket>[<bucket id>])

source zone &lt;zone 2&gt; (&lt;zone 2&gt;)
source bucket &lt;bucket&gt;:&lt;bucket id&gt;
full sync: 0/60000 shards
incremental sync: 0/60000 shards
bucket is behind on 60000 shards_

This command :

radosgw-admin bucket sync init --bucket=<our_bucket> --source-zone=<zone_2>

returns :

2022-10-10T06:45:19.654+0000 7f49137fe700 0 ERROR: curl error: Failure when receiving data from the peer, maybe network unstable
2022-10-10T06:45:19.658+0000 7f49f64723c0 0 data sync: ERROR: failed to fetch bucket index status

And does not start initialisation of the bucket.

Would you help us in starting enabling and starting sync on this bucket?

Thanks by advance for your help.

Actions #1

Updated by Casey Bodley over 1 year ago

  • Status changed from New to Need More Info
  • Tags set to multisite

did the `bucket sync enable` command finish? i imagine it would take a while with 60000 index shards..

Actions #2

Updated by Anonymous over 1 year ago

Hello Casey,

The init command ended after 60 minutes running.

Unfortunately the two errors are returned constantly during this time :

"ERROR: curl error: Failure when receiving data from the peer, maybe network unstable"

and

"data sync: ERROR: failed to fetch bucket index status"

The status remained the same unfortunately after this.

Thank you for your help.

Actions #3

Updated by Casey Bodley over 1 year ago

i can only recommend running the command until it succeeds

Actions #4

Updated by Pawel Stefanski over 1 year ago

@Allexandre can you fill more data about your env - what was exact version you tested and hit this?

Actions #5

Updated by Ilya Dryomov about 1 year ago

  • Target version deleted (v16.2.11)
Actions

Also available in: Atom PDF