Bug #45607
openCeph Elasticsearch sync module not working with rgw_keystone_implicit_tenants=true
0%
Description
rgw_keystone_implicit_tenants = true implies that a bucket is saved as PROJECT_ID$BUCKET_NAME.
the payload of RGW tier ES that sends a search request to ES is formed and comes out like this:
{
"query": {
"bool": {
"must": [ {
"term": {
"bucket": "test"
}
}, {
"bool": {
"must": [ {
"term": {
"permissions": "xxxxxxxx$yyyyyyyy"
}
}, {
"term": {
"name": "little023"
}
}
]
}
}
]
}
}
}
'
The problem with this is that the standard ES analyzer breaks that string into two words, xxxxxxx and yyyyyyy. There is no way (that I found) to configure properly ES because "permissions" is a array of texts, so I was not able to do a proper mapping and relative explicit analyzer or search_analyzer. Putting the full index as analyzer keyword did not work for me.
I think the only solution is to modify the call to elasticsearch that instead of permission, calls permission.keyword as such:
curl -X GET "localhost:9200/objectstorage-metadata/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{
"term": {
"bucket": "obo5"
}
},
{
"bool": {
"must": [
{
"term": {
"permissions.keyword": "xxxxxxxxx$yyyyyyyyyy"
}
},
{
"term": {
"name": "little023"
}
}
]
}
}
]
}
}
}
'
Using permission.keyword solves the $ problem and correctly finds the permissions. As it is, the ES Sync module cannot work with rgw_keystone_implicit_tenants=true.
Any chance someone could have a look at this and eventually providing a patch? It should be a very very easy change.
Thanks,
Regards
Updated by Matthew Oliver almost 4 years ago
I like the sound of a very very easy change. I'll take a crack, I haven't played with the EC side, so I'm guessing the testing will take longer then the patching. Famous. last. words :P
Updated by Matthew Oliver almost 4 years ago
here is my first attempt at a fix: https://github.com/ceph/ceph/pull/35224
I've tested it using the radosgw-es binary, but haven't actually tested it in an ES environment. if you could great, otherwise I'll next have to attempt to set it up on my dev machine.. probably a good learning experience anyway :)
Updated by Matthew Oliver almost 4 years ago
Oh I could probably just pull up a single node ES cluster using docker.. I'll give that a go :)
Updated by Casey Bodley almost 4 years ago
- Status changed from New to Fix Under Review
- Tags set to elasticsearch
- Pull request ID set to 35224
Updated by Luca Cervigni almost 4 years ago
Any news of a patch?
I tried to build CEPH from source with the change with no luck.
Are ubuntu deb builds available from somewhere for me to give it a try?