Project

General

Profile

Actions

Bug #45607

open

Ceph Elasticsearch sync module not working with rgw_keystone_implicit_tenants=true

Added by Luca Cervigni almost 4 years ago. Updated almost 4 years ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
elasticsearch
Backport:
Regression:
No
Severity:
2 - major
Reviewed:

Description

rgw_keystone_implicit_tenants = true implies that a bucket is saved as PROJECT_ID$BUCKET_NAME.

the payload of RGW tier ES that sends a search request to ES is formed and comes out like this:

{
"query": {
"bool": {
"must": [ {
"term": {
"bucket": "test"
}
}, {
"bool": {
"must": [ {
"term": {
"permissions": "xxxxxxxx$yyyyyyyy"
}
}, {
"term": {
"name": "little023"
}
}
]
}
}
]
}
}
}
'

The problem with this is that the standard ES analyzer breaks that string into two words, xxxxxxx and yyyyyyy. There is no way (that I found) to configure properly ES because "permissions" is a array of texts, so I was not able to do a proper mapping and relative explicit analyzer or search_analyzer. Putting the full index as analyzer keyword did not work for me.

I think the only solution is to modify the call to elasticsearch that instead of permission, calls permission.keyword as such:

curl -X GET "localhost:9200/objectstorage-metadata/_search?pretty" -H 'Content-Type: application/json' -d' {
"query": {
"bool": {
"must": [ {
"term": {
"bucket": "obo5"
}
}, {
"bool": {
"must": [ {
"term": {
"permissions.keyword": "xxxxxxxxx$yyyyyyyyyy"
}
}, {
"term": {
"name": "little023"
}
}
]
}
}
]
}
}
}
'

Using permission.keyword solves the $ problem and correctly finds the permissions. As it is, the ES Sync module cannot work with rgw_keystone_implicit_tenants=true.

Any chance someone could have a look at this and eventually providing a patch? It should be a very very easy change.

Thanks,
Regards

Actions #1

Updated by Brad Hubbard almost 4 years ago

  • Project changed from Ceph to rgw
Actions #2

Updated by Matthew Oliver almost 4 years ago

I like the sound of a very very easy change. I'll take a crack, I haven't played with the EC side, so I'm guessing the testing will take longer then the patching. Famous. last. words :P

Actions #3

Updated by Matthew Oliver almost 4 years ago

here is my first attempt at a fix: https://github.com/ceph/ceph/pull/35224

I've tested it using the radosgw-es binary, but haven't actually tested it in an ES environment. if you could great, otherwise I'll next have to attempt to set it up on my dev machine.. probably a good learning experience anyway :)

Actions #4

Updated by Matthew Oliver almost 4 years ago

Oh I could probably just pull up a single node ES cluster using docker.. I'll give that a go :)

Actions #5

Updated by Casey Bodley almost 4 years ago

  • Status changed from New to Fix Under Review
  • Tags set to elasticsearch
  • Pull request ID set to 35224
Actions #6

Updated by Luca Cervigni almost 4 years ago

Any news of a patch?
I tried to build CEPH from source with the change with no luck.

Are ubuntu deb builds available from somewhere for me to give it a try?

Actions

Also available in: Atom PDF