Project

General

Profile

Bug #11984

Error with `radosgw-admin check --check-head-obj-locator --bucket=$bucket --fix` command

Added by Francois Lafont over 4 years ago. Updated about 4 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
-
Target version:
-
Start date:
06/12/2015
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:

Description

Hi,

I have a cluster in Hammer 0.94.1 which I have upgraded to 0.94.2. In my radosgw host, I wanted to fix the problem with the rgw objects that start with underscore. But the command with the --fix option displays some errors which I don't understand. Here is a (troncated) paste of my shell below.

Regards.

~# radosgw-admin --id=radosgw.gw2 bucket check --check-head-obj-locator --bucket=$bucket
{
    "bucket": "moodles-poc-registry",
    "check_objects": [
        {
            "key": {
                "name": "_multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
                "instance": "" 
            },
            "oid": "default.763616.1___multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
            "locator": "default.763616.1__multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
            "needs_fixing": true,
            "status": "needs_fixing" 
        },

[snip]

        {
            "key": {
                "name": "_multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
                "instance": "" 
            },
            "oid": "default.763616.1___multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
            "locator": "default.763616.1__multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
            "needs_fixing": true,
            "status": "needs_fixing" 
        }

    ]
}

~# radosgw-admin --id=radosgw.gw2 bucket check --check-head-obj-locator --bucket=$bucket --fix
2015-06-12 03:01:33.197984 7f3c9130d840 -1 ERROR: ioctx.operate(oid=default.763616.1___multipart_registry/images/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta) returned ret=-2
ERROR: fix_head_object_locator() returned ret=-2
2015-06-12 03:01:33.200428 7f3c9130d840 -1 ERROR: ioctx.operate(oid=default.763616.1___multipart_registry/images/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909/layer.2~poMH-PQKCLstUWpMQpji7JuGaBT53Th.meta) returned ret=-2
ERROR: fix_head_object_locator() returned ret=-2
ERROR: fix_head_object_locator() returned ret=-2
2015-06-12 03:01:33.206875 7f3c9130d840 -1 ERROR: ioctx.operate(oid=default.763616.1___multipart_registry/images/c5a7fc74211188aabf3429539674275645b07717d003c390a943acc44f35c6d0/layer.2~Bg6bkbSOE8GCtV4Mxr0t56vSfTQTCx9.1) returned ret=-2
2015-06-12 03:01:33.209293 7f3c9130d840 -1 ERROR: ioctx.operate(oid=default.763616.1___multipart_registry/images/c5a7fc74211188aabf3429539674275645b07717d003c390a943acc44f35c6d0/layer.2~Bg6bkbSOE8GCtV4Mxr0t56vSfTQTCx9.2) returned ret=-2
ERROR: fix_head_object_locator() returned ret=-2
ERROR: fix_head_object_locator() returned ret=-2

[snip]

2015-06-12 03:01:33.301101 7f3c9130d840 -1 ERROR: ioctx.operate(oid=default.763616.1___multipart_registry/images/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta) returned ret=-2
{
    "bucket": "moodles-poc-registry",
    "check_objects": [
        {
            "key": {
                "name": "_multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
                "instance": "" 
            },
            "oid": "default.763616.1___multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
            "locator": "default.763616.1__multipart_registry\/images\/1483a2ea4c3f5865d4d583fb484bbe11afe709a6f3d1baef102904d4d9127909\/layer.2~QorD8QaGiDc4HPUP7VVpx4LS-e_7f0u.meta",
            "needs_fixing": true,
            "status": "needs_fixing" 
        },

[snip]

        {
            "key": {
                "name": "_multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
                "instance": "" 
            },
            "oid": "default.763616.1___multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
            "locator": "default.763616.1__multipart_registry\/images\/fa4fd76b09ce9b87bfdc96515f9a5dd5121c01cc996cf5379050d8e13d4a864b\/layer.2~TSdIpafsfGXJ7kKMOVqJ-hn8Aog4ETF.meta",
            "needs_fixing": true,
            "status": "needs_fixing" 
        }

    ]
}

Related issues

Related to rgw - Bug #11974: Fix tool for #11442 does not correctly fix objects created via multipart uploads Resolved 06/12/2015

Associated revisions

Revision 81039085 (diff)
Added by Yehuda Sadeh about 4 years ago

rgw: only scan for objects not in namespace

Fixes: #11984
The tool should only work on the head objects, and these are not inside
any namespace.

Signed-off-by: Yehuda Sadeh <>

Revision 28d32f60 (diff)
Added by Yehuda Sadeh about 4 years ago

rgw: only scan for objects not in namespace

Fixes: #11984
The tool should only work on the head objects, and these are not inside
any namespace.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit 8103908548bf7d6c9fa47fb181cd450670bae8d6)

History

#1 Updated by Francois Lafont over 4 years ago

Sorry I have forgotten to give some info about my environment. For each node of my cluster (2 "radosgw" servers, 2 "monitor+OSDs+mds" servers and 1 "monitor" server) :

- OS: Ubuntu 14.04
- Kernel: 3.19.0-20-generic (after a apt-get install linux-image-generic-lts-vivid)
- Ceph version: 0.94.2

#2 Updated by Sage Weil over 4 years ago

  • Priority changed from Normal to Urgent

these objects will go away on their own, so it's not critical.

we should fix the tool to ignore them.

#3 Updated by Yehuda Sadeh about 4 years ago

  • Status changed from New to Duplicate

This is not a real problem, but is also handled in the fix for #12079.

Thanks!

Also available in: Atom PDF