Project

General

Profile

Bug #22714

Existing AccessKeys with leading underscore broke in upgrade

Added by Robin Johnson about 6 years ago. Updated about 6 years ago.

Status:
Triaged
Priority:
Normal
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
luminous
Regression:
Yes
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Pre-existing AccessKeys that start with an underscore broke after the upgrade to Luminous. Newly created keys even identical name are fine.

Workaround:

radosgw-admin key rm --uid=$U --access-key=$AK --secret-key=$SK && \
radosgw-admin key create --uid=$U --access-key=$AK --secret-key=$SK

Notes:

  • New access keys with a leading underscore created in Luminous work fine.
  • Recreating the key fixes it.
  • Problem might exist in librados layer.
# Broken key, not fixed yet:
$ rados -p .users ls |grep _FtBhGSAZABCDEF01234
__FtBhGSAZABCDEF01234    _FtBhGSAZABCDEF01234

# Broken key, after rm & ecreate
$ rados -p .users ls |grep _yWcZHmCjgABCDEF01234
__yWcZHmCjgABCDEF01234    _yWcZHmCjgABCDEF01234
_yWcZHmCjgABCDEF01234

# Newly created key with leading underscore:
$ rados -p .users ls |grep _LyfrysABCDEF01234
_LyfrysABCDEF01234

Related issues

Duplicated by rgw - Bug #23373: Problem with UID starting with underscores Closed 03/15/2018

History

#1 Updated by Robin Johnson about 6 years ago

Missed a part in my debug output:

# broken key
$ rados -p .users get _FtBhGSAZABCDEF01234 - |xxd
error getting .users/_FtBhGSAZABCDEF01234: (2) No such file or directory

# fixed key
$ rados -p .users get _yWcZHmCjgABCDEF01234 - |xxd
(correct output)

#2 Updated by Matt Benjamin about 6 years ago

  • Status changed from New to Triaged

#3 Updated by Yehuda Sadeh about 6 years ago

Try running:

$ rados cp -p .users __FtBhGSAZABCDEF01234 --object_locator _FtBhGSAZABCDEF01234 _FtBhGSAZABCDEF01234

#4 Updated by Robin Johnson about 6 years ago

Yehuda: I do confirm that your rados cp command causes the key to work again, but it does not explain why it broke in the Jewel->Luminous upgrade, also now there are two objects in the users pool for that key, and probably won't be cleaned up properly.

#5 Updated by Orit Wasserman about 6 years ago

  • Assignee set to Orit Wasserman

#6 Updated by Yehuda Sadeh about 6 years ago

@robbat2 there was some encoding change that we did between jewel and luminous and apparently there was a case we missed. The workaround is to recreate the new objects with the new name. You can remove the old object, the command should be something like this:

rados rm -p .users __FtBhGSAZABCDEF01234 --object_locator _FtBhGSAZABCDEF01234

#7 Updated by Yehuda Sadeh about 6 years ago

  • Priority changed from High to Normal

#8 Updated by Casey Bodley about 6 years ago

  • Duplicated by Bug #23373: Problem with UID starting with underscores added

Also available in: Atom PDF