Project

General

Profile

Bug #22758

rgw sends garbage meta.compression to ElasticSearch

Added by YOUZHONG YANG about 1 year ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Target version:
-
Start date:
01/23/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

Once compression is enabled for a zone, metadata indexing on elasticsearch stops working.

Here is how to reproduce:

- Set up two zones as described on https://ceph.com/rgw/new-luminous-rgw-metadata-search/
- Enable compression: radosgw-admin zone placement modify --rgw-zone=us-east-1 --placement-id=default-placement --compression=zlib
- Configure custom metadata fields
- Upload object with metadata

Look at ElasticSearch log file (/var/log/elasticsearch/elasticsearch.log), you will see something like the following:

[2018-01-20T23:13:43,587][DEBUG][o.e.a.b.TransportShardBulkAction] [uSX41lj] [rgw-us-natick-04b97c18][6] 
failed to execute bulk item (index) BulkShardRequest 
[[rgw-us-natick-04b97c18][6]] containing [index {[rgw-us-natick-04b97c18][object][880837de-f383-4d5c-aa6f-8080518ca8f0.264524.1:top123:null], 
source[{"bucket":"buck","name":"top123","instance":"null","versioned_epoch":0,"owner":{"id":"yyang","display_name":"yyang"},"permissions":["yyang"],
"meta":{"size":109064,"mtime":"2018-01-21T04:07:55.248Z","compression":"\u0001\u00012\u0000\u0000\u0000\u0004\u0000\u0000\u0000zlib\u0008�\u0001\u0000\u0000\u0000\u0000\u0000\u0001\u0000\u0000
\u0000\u0001\u0001\u0018\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000H�\u0000
\u0000\u0000\u0000\u0000","content_type":"binary/octet-stream","etag":"d08b14ff8f3c6326191d934c5668b1a0",
"tail_tag":"880837de-f383-4d5c-aa6f-8080518ca8f0.264519.204"}}]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [meta.compression]
        at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:298) ~[elasticsearch-5.6.6.jar:5.6.6]

Related issues

Copied to rgw - Backport #23869: luminous: rgw sends garbage meta.compression to ElasticSearch Resolved

History

#1 Updated by John Spray about 1 year ago

  • Project changed from Ceph to rgw

#2 Updated by Abhishek Lekshmanan 12 months ago

  • Assignee set to Abhishek Lekshmanan

#3 Updated by Abhishek Lekshmanan 12 months ago

  • Status changed from New to In Progress

#4 Updated by Abhishek Lekshmanan 11 months ago

  • Status changed from In Progress to Need Review
  • Backport set to luminous

#5 Updated by Abhishek Lekshmanan 11 months ago

  • Target version deleted (v12.2.2)

#6 Updated by Casey Bodley 9 months ago

  • Status changed from Need Review to Pending Backport

#7 Updated by Nathan Cutler 9 months ago

  • Copied to Backport #23869: luminous: rgw sends garbage meta.compression to ElasticSearch added

#8 Updated by Nathan Cutler 8 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF