Bug #52669
closedBucket index not always created when new object added
0%
Description
BACKGROUND：
Ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable)
Ceph health is OK
Deployed using Rook
Using EC pool with 3 RGW instances
DESCRIBE:
When use S3 LIST_OBJECT() to list, some uploaded objects cannot be listed.
I am sure that the object has been uploaded because I can get it with the exact key.
Moreover I experience this only with RGW instances > 1, with a single RGW instance the issue is not reported.
SOME ELSE：
I have the same issue with REP pool.
Investigating more I find out that:
radosgw-admin bi list --bucket=ceph-bkt-31c9b571-6495-4edd-a0f1-8f2cb9ff97cb | grep part-00005-0bb55c0f-755f-4299-80ee-142d65e38eb4-c000.snappy.parquet
Does not return anything (index missing)
Same for:
rados listomapkeys -p my-store.rgw.buckets.index .dir.1cabf71f-2fe9-4187-a928-537277185b62.51427781.2.1 | grep part-00005-0bb55c0f-755f-4299-80ee-142d65e38eb4-c000.snappy.parquet
These files are uploaded by S3a file committer. I doubt this has something to do with the issue because the http requests to the RGW are identical both in case of object uploaded correctly and not
Logs even at debug-level=20 do not report any error/warning
Updated by Pietro Di Marco over 2 years ago
BACKGROUND：
Ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable)
Ceph health is OK
Deployed using Rook
Using EC pool with 3 RGW instances
DESCRIBE:
When use S3 LIST_OBJECT() to list, some uploaded objects cannot be listed.
I am sure that the object has been uploaded because I can get it with the exact key.
Moreover I experience this only with RGW instances > 1, with a single RGW instance the issue is not reported.
SOME ELSE：
I have the same issue with REP pool.
Investigating more I find out that:
radosgw-admin bi list --bucket=ceph-bkt-31c9b571-6495-4edd-a0f1-8f2cb9ff97cb | grep part-00005-0bb55c0f-755f-4299-80ee-142d65e38eb4-c000.snappy.parquet
Does not return anything (index missing)
Same for:
rados listomapkeys -p my-store.rgw.buckets.index .dir.1cabf71f-2fe9-4187-a928-537277185b62.51427781.2.1 | grep part-00005-0bb55c0f-755f-4299-80ee-142d65e38eb4-c000.snappy.parquet
These files are uploaded by S3a file committer. I doubt this has something to do with the issue because the http requests to the RGW are identical both in case of object uploaded correctly and not
Logs even at debug-level=20 do not report any error/warning
Updated by Casey Bodley over 2 years ago
- Is duplicate of Bug #52671: Bucket index not always created when new object added added