Bug #47919
list object versions returned multiple 'IsLatest true' entries
0%
Description
problem¶
- list object versions returned two entries with 'IsLatest' is 'true'
{
"Versions": [
{
"ETag": "\"e5b02961dccf9faf6198287e2f7bbf50\"",
"Size": 8,
"StorageClass": "STANDARD",
"Key": "heheobj",
"VersionId": "WL6LZRcqE2w0GROaHrdl6EnwLqQk4kK",
"IsLatest": true,
"LastModified": "2020-10-20T03:49:43.919Z",
"Owner": {
"DisplayName": "M. Tester",
"ID": "testid"
}
},
{
"ETag": "\"36ea73317e5cecde5720ffdf8f952c89\"",
"Size": 12,
"StorageClass": "STANDARD",
"Key": "heheobj",
"VersionId": "null",
"IsLatest": true,
"LastModified": "2020-10-20T03:49:43.048Z",
"Owner": {
"DisplayName": "M. Tester",
"ID": "testid"
}
},
{
"ETag": "\"cf1c5e62517cea8d80cc134370cf31e3\"",
"Size": 7,
"StorageClass": "STANDARD",
"Key": "heheobj",
"VersionId": "gTWeEyETVL1FIqp7YpEzyjzHQ9rq3Mg",
"IsLatest": false,
"LastModified": "2020-10-20T03:49:41.706Z",
"Owner": {
"DisplayName": "M. Tester",
"ID": "testid"
}
}
]
}
reproduce¶
#!/bin/bash
bucket="whybucket"
object="heheobj"
endpoint="http://127.0.0.1:8000"
#debug="--debug"
v1obj="v1-obj"
nullobj="null-objjjj"
v2obj="v2-objj"
set -e
echo $v1obj > $v1obj
echo $v2obj > $v2obj
echo $nullobj > $nullobj
aws $debug --endpoint=$endpoint s3 mb s3://$bucket
echo ">> bucket is: $bucket"
echo ">> object is: $object"
echo ""
# enable bucket versioning, put objcet
aws --endpoint=$endpoint s3api put-bucket-versioning --bucket $bucket --versioning-configuration Status=Enabled
echo ">> set bucket versioning enabled"
echo ">> put object to bucket"
aws --endpoint=$endpoint s3api put-object --bucket $bucket --key $object --body $v1obj
echo ""
# suspend bucket versioning put object and set object acl
aws --endpoint=$endpoint s3api put-bucket-versioning --bucket $bucket --versioning-configuration Status=Suspended
echo ">> set bucket versioning suspended"
echo ">> put object to bucket"
aws --endpoint=$endpoint s3api put-object --bucket $bucket --key $object --body $nullobj
echo ">> set object acl or tagging: "
#aws --endpoint=$endpoint s3api put-object-acl --bucket $bucket --key=$object --acl public-read-write
aws --endpoint=$endpoint s3api put-object-tagging --bucket $bucket --key=$object --tagging '{"TagSet": [{"Key": "kkk", "Value": "vvvvv"}]}'
echo ""
# enabled bucket versiong, put object
aws --endpoint=$endpoint s3api put-bucket-versioning --bucket $bucket --versioning-configuration Status=Enabled
echo ">> set bucket versioning enabled"
echo ">> put object to bucket"
aws --endpoint=$endpoint s3api put-object --bucket $bucket --key $object --body $v2obj
echo ""
# list object instances
echo ">>>> list: "
aws --endpoint=$endpoint s3api list-object-versions --bucket $bucket
analysis¶
bool is_current() {
int test_flags = RGW_BUCKET_DIRENT_FLAG_VER | RGW_BUCKET_DIRENT_FLAG_CURRENT;
return (flags & RGW_BUCKET_DIRENT_FLAG_VER) == 0 ||
(flags & test_flags) == test_flags;
}
As shown in `is_current()`, entry in the following two cases is 'current':
- non-versioned index entry
- versioned and current index entry
After checked raw bi entries of this bucket, I find
- flag of version 'WL6LZRcqE2w0GROaHrdl6EnwLqQk4kK' is 3(current and versioned)
- flag of version 'null' is 0(non-versioned index entry), so `is_current()` returned true
...
{
"type": "plain",
"idx": "heheobj\u0000v911\u0000iWL6LZRcqE2w0GROaHrdl6EnwLqQk4kK",
"entry": {
"name": "heheobj",
"instance": "WL6LZRcqE2w0GROaHrdl6EnwLqQk4kK",
"ver": {
"pool": 6,
"epoch": 1
},
"locator": "",
"exists": "true",
"meta": {
"category": 1,
"size": 8,
"mtime": "2020-10-20 03:49:43.919636Z",
"etag": "e5b02961dccf9faf6198287e2f7bbf50",
"storage_class": "",
"owner": "testid",
"owner_display_name": "M. Tester",
"content_type": "",
"accounted_size": 8,
"user_data": "",
"appendable": "false"
},
"tag": "46e0724e-6987-47a4-9950-0090ab80c3c5.4153.33",
"flags": 3,
"pending_map": [],
"versioned_epoch": 4
}
},
{
"type": "plain",
"idx": "heheobj\u0000v912\u0000i",
"entry": {
"name": "heheobj",
"instance": "",
"ver": {
"pool": 6,
"epoch": 7
},
"locator": "",
"exists": "true",
"meta": {
"category": 1,
"size": 12,
"mtime": "2020-10-20 03:49:43.048160Z",
"etag": "36ea73317e5cecde5720ffdf8f952c89",
"storage_class": "",
"owner": "testid",
"owner_display_name": "M. Tester",
"content_type": "",
"accounted_size": 12,
"user_data": "",
"appendable": "false"
},
"tag": "_5SQz70NH7aVO97CD8xAI-60mv0qxffK",
"flags": 0,
"pending_map": [],
"versioned_epoch": 3
}
},
Actually, the flag of 'null' entry's instance entry is changed from 3 to 0 after put-object-tagging/put-object-acl is excuted.
Then the bucket versioning is turned on, and I put a v2obj into this object which generated a new version WL6LZRcqE2w0GROaHrdl6EnwLqQk4kK.
And 'null' entry's instance entry is read out, then its 'current' bit flag is cleared(from 0 to 0), and is written back.
Also the 'null' entry is updated with 0 flag.
solution¶
- keep the `FLAG_VER` bit of 'null' entry (`rgw_bucket_complete_op()`)
diff --git a/src/cls/rgw/cls_rgw.cc b/src/cls/rgw/cls_rgw.cc
index c36e31c301..6b815ade7b 100644
--- a/src/cls/rgw/cls_rgw.cc
+++ b/src/cls/rgw/cls_rgw.cc
@@ -981,9 +981,7 @@ int rgw_bucket_complete_op(cls_method_context_t hctx, bufferlist *in, bufferlist
entry.index_ver = header.ver;
/* resetting entry flags, entry might have been previously a delete
* marker */
- entry.flags = (entry.key.instance.empty() ?
- 0 :
- rgw_bucket_dir_entry::FLAG_VER);
+ entry.flags &= rgw_bucket_dir_entry::FLAG_VER;
if (op.tag.size()) {
auto pinter = entry.pending_map.find(op.tag);
Related issues
History
#1 Updated by Honggang Yang over 3 years ago
#2 Updated by Kefu Chai over 3 years ago
- Status changed from New to Fix Under Review
- Assignee set to Honggang Yang
- Pull request ID set to 37733
#3 Updated by Matt Benjamin over 3 years ago
Is target version Nautilus really correct (linked fix is against master). I.e., do we need backports for Octopus and Nautilus?
Matt
#4 Updated by Casey Bodley over 3 years ago
- Tags set to versioning
- Backport set to nautilus octopus
#5 Updated by Honggang Yang over 3 years ago
Matt Benjamin wrote:
Is target version Nautilus really correct (linked fix is against master). I.e., do we need backports for Octopus and Nautilus?
Matt
Yes, this should be backported to O and N
#6 Updated by Kefu Chai over 3 years ago
- Backport changed from nautilus octopus to nautilus, octopus
#7 Updated by Nathan Cutler over 3 years ago
- Copied to Backport #48238: nautilus: list object versions returned multiple 'IsLatest true' entries added
#8 Updated by Nathan Cutler over 3 years ago
- Related to Backport #48239: octopus: list object versions returned multiple 'IsLatest true' entries added
#9 Updated by Nathan Cutler over 3 years ago
- Related to deleted (Backport #48239: octopus: list object versions returned multiple 'IsLatest true' entries)
#10 Updated by Nathan Cutler over 3 years ago
- Copied to Backport #48239: octopus: list object versions returned multiple 'IsLatest true' entries added
#11 Updated by J. Eric Ivancich over 3 years ago
- Status changed from Fix Under Review to Pending Backport
#12 Updated by Nathan Cutler about 3 years ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".