Bug #22758
rgw sends garbage meta.compression to ElasticSearch
% Done:
0%
Source:
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Once compression is enabled for a zone, metadata indexing on elasticsearch stops working.
Here is how to reproduce:
- Set up two zones as described on https://ceph.com/rgw/new-luminous-rgw-metadata-search/
- Enable compression: radosgw-admin zone placement modify --rgw-zone=us-east-1 --placement-id=default-placement --compression=zlib
- Configure custom metadata fields
- Upload object with metadata
Look at ElasticSearch log file (/var/log/elasticsearch/elasticsearch.log), you will see something like the following:
[2018-01-20T23:13:43,587][DEBUG][o.e.a.b.TransportShardBulkAction] [uSX41lj] [rgw-us-natick-04b97c18][6] failed to execute bulk item (index) BulkShardRequest [[rgw-us-natick-04b97c18][6]] containing [index {[rgw-us-natick-04b97c18][object][880837de-f383-4d5c-aa6f-8080518ca8f0.264524.1:top123:null], source[{"bucket":"buck","name":"top123","instance":"null","versioned_epoch":0,"owner":{"id":"yyang","display_name":"yyang"},"permissions":["yyang"], "meta":{"size":109064,"mtime":"2018-01-21T04:07:55.248Z","compression":"\u0001\u00012\u0000\u0000\u0000\u0004\u0000\u0000\u0000zlib\u0008�\u0001\u0000\u0000\u0000\u0000\u0000\u0001\u0000\u0000 \u0000\u0001\u0001\u0018\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000H�\u0000 \u0000\u0000\u0000\u0000","content_type":"binary/octet-stream","etag":"d08b14ff8f3c6326191d934c5668b1a0", "tail_tag":"880837de-f383-4d5c-aa6f-8080518ca8f0.264519.204"}}]}] org.elasticsearch.index.mapper.MapperParsingException: failed to parse [meta.compression] at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:298) ~[elasticsearch-5.6.6.jar:5.6.6]
Related issues
History
#1 Updated by John Spray about 6 years ago
- Project changed from Ceph to rgw
#2 Updated by Abhishek Lekshmanan about 6 years ago
- Assignee set to Abhishek Lekshmanan
#3 Updated by Abhishek Lekshmanan about 6 years ago
- Status changed from New to In Progress
#4 Updated by Abhishek Lekshmanan about 6 years ago
- Status changed from In Progress to Fix Under Review
- Backport set to luminous
#5 Updated by Abhishek Lekshmanan about 6 years ago
- Target version deleted (
v12.2.2)
#6 Updated by Casey Bodley almost 6 years ago
- Status changed from Fix Under Review to Pending Backport
#7 Updated by Nathan Cutler almost 6 years ago
- Copied to Backport #23869: luminous: rgw sends garbage meta.compression to ElasticSearch added
#8 Updated by Nathan Cutler almost 6 years ago
- Status changed from Pending Backport to Resolved