Actions
Bug #47451
openRGW appends control character to etags in bucket index
% Done:
0%
Source:
Tags:
etag
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
We've encountered a semi-rare bug where rgw appends a control character to objects' etag fields in the bucket index. When s3 clients try to list such an object, the control character invalidates the xml response. We are running both nautilus and luminous clusters, and this only seems to happening on nautilus (14.2.5 and 14.2.8).
As far as mitigation goes, `radosgw-admin bucket check --fix` removes the control character in the bucket index entry. However, I don't know how to reproduce this condition, as it seems to effect at most 1 in a million objects.
Example hex of the etag before:
00000090 25 0f 23 00 00 00 37 63 39 65 32 62 39 36 63 63 |%.#...7c9e2b96cc| 000000a0 37 65 63 36 31 35 39 37 38 65 35 66 63 61 39 33 |7ec615978e5fca93| 000000b0 35 31 35 61 33 61 2d 31 0e 07 00 00 00 36 34 34 |515a3a-1.....644|
And after running bucket check:
00000090 25 0f 22 00 00 00 37 63 39 65 32 62 39 36 63 63 |%."...7c9e2b96cc| 000000a0 37 65 63 36 31 35 39 37 38 65 35 66 63 61 39 33 |7ec615978e5fca93| 000000b0 35 31 35 61 33 61 2d 31 07 00 00 00 36 34 34 39 |515a3a-1....6449|
This report about a luminous bug, now closed, seems like it might be related: https://tracker.ceph.com/issues/23188
Actions