Project

General

Profile

Bug #22985

Buckets missing from rgw_read_user_buckets output

Added by Robin Johnson about 6 years ago. Updated almost 6 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
rgw, bucket-list
Backport:
luminous
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Some buckets that are known to exist have vanished from the listing of buckets.

Dreamhost tracking DHO-1722.

Here's an example from of our tech support staff, who noticed it on their own buckets.

# radosgw-admin --uid jerhec  bucket list |grep jerhec.dreamhosters.com
(no output)
# radosgw-admin --uid jerhec  metadata get bucket:jerhec.dreamhosters.com
{
    "key": "bucket:jerhec.dreamhosters.com",
    "ver": {
        "tag": "_pzGKFON9ecR6xvrlneQd4Kv",
        "ver": 3
    },
    "mtime": "2018-02-05 05:58:32.519748Z",
    "data": {
        "bucket": {
            "name": "jerhec.dreamhosters.com",
            "marker": "11164460.37390",
            "bucket_id": "default.506986875.1",
            "tenant": "",
            "explicit_placement": {
                "data_pool": ".rgw.data.1",
                "data_extra_pool": "",
                "index_pool": ".rgw.data.1" 
            }
        },
        "owner": "jerhec",
        "creation_time": "0.000000",
        "linked": "true",
        "has_bucket_info": "false" 
    }
}

History

#1 Updated by Robin Johnson about 6 years ago

In the case of account jerhec, at least two buckets are known to be missing; let me know what other debug/data exports to provide for that user; i'm not immediately sure where the map of buckets actually is.

#2 Updated by Robin Johnson about 6 years ago

  • Subject changed from Bucket missing from rgw_read_user_buckets output to Buckets missing from rgw_read_user_buckets output

#3 Updated by Robin Johnson about 6 years ago

This output contains the buckets that are fine, but not the missing ones.
The omap header is non-empty.

# rados -p .users.uid listomapkeys jerhec.buckets

#4 Updated by Orit Wasserman almost 6 years ago

  • Assignee set to Yehuda Sadeh

#5 Updated by Matt Benjamin almost 6 years ago

  • Status changed from New to 12

#6 Updated by Yehuda Sadeh almost 6 years ago

@robin does re-linking the buckets solve the issue?

#7 Updated by Robin Johnson almost 6 years ago

Yes, I re-linked all of the buckets by scanning the entire pool for bucket names, and comparing what buckets weren't linked.

Good candidate for rgw scrubbing tooling.

#8 Updated by Yehuda Sadeh almost 6 years ago

Sadly we don't have enough info to determine the culprit.

#9 Updated by Yehuda Sadeh almost 6 years ago

  • Status changed from 12 to Can't reproduce

Feel free to reopen if it happens again and/or you find more info.

Also available in: Atom PDF