Project

General

Profile

Actions

Bug #14025

closed

Bucket DELETE fails, 404 NoSuchKey, but cannot reproduce on other bucket names or users

Added by Robin Johnson over 8 years ago. Updated almost 8 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
-
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

This is a weird one, I can't reproduce the problem with any other bucket name, or the same bucket name but on a different user.

bucket=www.basicpacific.com-cdn

Test process:
- (bucket does not exist, state is clean)
- s3cmd mb s3://${bucket}
- s3cmd ls # confirm bucket exists to user
- rgw-admin bucket list --uid=${uid} # confirm bucket exists to admin
- s3cmd rb s3://${bucket} # this FAILs
- rgw-admin bucket rm --uid=${uid} --bucket=${bucket} # this passes
- s3cmd ls # confirm bucket was removed
- rgw-admin bucket list --uid=${uid} # confirm bucket was removed

Attached is two passes RGW debug log 20 showing the DELETE failing, separated by time (the sequence was run multiple times inbetween, along with other attempts to find why). Other log content trimmed out.


Files

rgw-delete-fail.txt (19.9 KB) rgw-delete-fail.txt Robin Johnson, 12/08/2015 07:55 PM
Actions #1

Updated by Yehuda Sadeh over 8 years ago

looking at the logs, it seems to me that the bucket entry point still exists but the bucket instance metadata does not. We probably don't handle that gracefully.

Actions #2

Updated by Yehuda Sadeh over 8 years ago

  • Priority changed from Normal to High
Actions #3

Updated by Loïc Dachary about 8 years ago

  • Target version deleted (v0.94.6)
Actions #4

Updated by Yehuda Sadeh about 8 years ago

There are a few operations that happen when removing a bucket:
- remove bucket instance
- remove bucket entry point
- delete bucket entry from user's list of buckets

If rgw goes down in the middle of the bucket removal then we might end up with some inconsistency.

Actions #5

Updated by Orit Wasserman almost 8 years ago

  • Status changed from New to Can't reproduce
Actions

Also available in: Atom PDF