Project

General

Profile

Bug #14208

rgw: 500 returned on Put Bucket ACL (ECANCELLED/-125)

Added by Robin Johnson almost 4 years ago. Updated about 3 years ago.

Status:
Need More Info
Priority:
Normal
Assignee:
Target version:
-
Start date:
12/30/2015
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
hammer rgw
Backport:
Regression:
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature:

Description

Put Bucket ACL is failing, returning ECANCELLED (-125)

<Error><Code>UnknownError</Code></Error>

The new ACL is just adding publicly readable to a given bucket, cannot reproduce on other buckets/users.

2015-12-30 22:56:07.042754 7fc718ce9700 1 -- [2607:f298:4:d006::5781]:0/1008664 --> [2607:f298:4:d007::8853]:6863/1453551 -- osd_op(client.170008394.0:4652663043 .bucket.meta.USER-BUCKET-NAME:default.170008394.70001 [call version.check_conds,call version.set,setxattr user.rgw.acl (197)] 13.101bba61 ondisk+write+known_if_redirected e838310) v5 -- ?+0 0x7fc92883d5b0 con 0x7fc94f4630f0
2015-12-30 22:56:07.043773 7fc718ce9700 0 WARNING: set_req_state_err err_no=125 resorting to 500

RGW logs at 20/20 attached (redacted private info).

The object .bucket.meta.USER-BUCKET-NAME:default.170008394.70001 seems fine.

put-bucket-acl-ECANCELLED.txt View - put-bucket-acl-ECANCELLED.txt (11.1 KB) Robin Johnson, 12/30/2015 11:21 PM

container creation failure - Logs captured with debug_ms set to 1/5 and debug_rgw to 20/20 (148 KB) Pavan Rallabhandi, 09/10/2016 03:15 PM

History

#1 Updated by Yehuda Sadeh almost 4 years ago

Could be related to metadata cache disagreement between different rgws (as we've seen in your environment).

#2 Updated by Loic Dachary over 3 years ago

  • Target version deleted (v0.94.6)

#3 Updated by Warren Wang about 3 years ago

Yehuda Sadeh wrote:

Could be related to metadata cache disagreement between different rgws (as we've seen in your environment).

FWIW, we just saw this occur on a dev build of Jewel. Resolved by restarting all RGWs. Unable to reproduce after a restart.

#4 Updated by Matt Benjamin about 3 years ago

  • Status changed from New to Need More Info
  • Assignee set to Matt Benjamin

Assigning myself. I think we need more context to guide in reproducing (or a reproducer?).

#5 Updated by Pavan Rallabhandi about 3 years ago

Matt,

We ran into this on Jewel 10.2.2 clusters, while creating a container (bizarrely reproducible only with a specified named container). Am attaching RGW logs captured with debug_ms set to 1/5 and debug_rgw set to 20/20, please let me know if you need any further info, the cluster is in a reproducible state.

Thanks,
-Pavan.

#6 Updated by Yehuda Sadeh about 3 years ago

@prallabh issue looks similar, but not the same problem afaict

Also available in: Atom PDF