Actions
Bug #40153
closedmultisite: bucket index log can omit multipart entries
Status:
Duplicate
Priority:
Normal
Assignee:
-
Target version:
-
% Done:
0%
Source:
Tags:
multisite multisite-backlog
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
data sync does not replicate parts, so the bucket index log doesn't need to store a {write prepare, write complete, and delete complete} entry for every part and .meta.
the (abbreviated) bilog list for a 4-part multipart upload looks like this:
[ { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.meta", "state": "pending", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.meta", "state": "complete", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.1", "state": "pending", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.1", "state": "complete", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.2", "state": "pending", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.2", "state": "complete", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.3", "state": "pending", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.3", "state": "complete", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.4", "state": "pending", }, { "op": "write", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.4", "state": "complete", }, { "op": "write", "object": "128m.iso", "state": "pending", }, { "op": "write", "object": "128m.iso", "state": "complete", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.1", "state": "complete", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.2", "state": "complete", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.3", "state": "complete", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.4", "state": "complete", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.meta", "state": "pending", }, { "op": "del", "object": "_multipart_128m.iso.2~Asko0xU3EFWlRdRJwTpgr0tD9AuWrHl.meta", "state": "complete", }, ]
but the only entry required to sync the object is:
[ { "op": "write", "object": "128m.iso", "state": "complete", }, ]
Updated by Casey Bodley 12 months ago
- Tags changed from multisite to multisite multisite-backlog
Updated by Jiffin Tony Thottan 6 months ago
In the latest version, I was not able to reproduce the issue
upload_id=$aws s3api create-multipart-upload --no-verify-ssl --endpoint-url http://localhost:8001 --bucket bkt --key my-object) upload file 5 files of 5MB size aws s3api complete-multipart-upload --no-verify-ssl --endpoint-url http://localhost:8001 --bucket bkt --key my-object --upload-id $upload_id --multipart-upload file://file.json { "Location": "http://localhost:8001/bkt/my-object", "Bucket": "bkt", "Key": "my-object", "ETag": "" } aws s3 ls --no-verify-ssl --endpoint-url http://localhost:8001 s3://bkt 2023-10-23 09:27:52 26214400 my-object
../src/mrun c1 radosgw-admin bilog list --bucket=bkt 2023-10-23T09:30:06.701-0400 7ff52b410780 -1 WARNING: all dangerous and experimental features are enabled. 2023-10-23T09:30:06.704-0400 7ff52b410780 -1 WARNING: all dangerous and experimental features are enabled. 2023-10-23T09:30:06.705-0400 7ff52b410780 -1 WARNING: all dangerous and experimental features are enabled. [ { "op_id": "6#00000000004.18.6", "op_tag": "_E77zWfL8f_mTK2sQ-HM_kXiVHPIo71o", "op": "write", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.3", "instance": "", "state": "complete", "index_ver": 4, "timestamp": "2023-10-23T13:04:18.041789126Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] }, { "op_id": "6#00000000005.20.6", "op_tag": "_102h5pLo_crPKSIMy6Id-livvD8-ztX", "op": "write", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.4", "instance": "", "state": "complete", "index_ver": 5, "timestamp": "2023-10-23T13:04:24.643495411Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] }, { "op_id": "6#00000000006.22.6", "op_tag": "_ObdDtBBieTwRlovQbJIhvJUcTYqKhxT", "op": "write", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.5", "instance": "", "state": "complete", "index_ver": 6, "timestamp": "2023-10-23T13:04:30.349749107Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] }, { "op_id": "6#00000000007.25.6", "op_tag": "33374512-b9e7-459d-85a5-2804d61f1d92.4231.12034979229018231025", "op": "write", "object": "my-object", "instance": "", "state": "complete", "index_ver": 7, "timestamp": "2023-10-23T13:27:52.186580707Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] }, { "op_id": "6#00000000008.25.8", "op_tag": "", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.1", "instance": "", "state": "complete", "index_ver": 8, "timestamp": "2023-10-23T13:04:03.336950682Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [] }, { "op_id": "6#00000000009.25.11", "op_tag": "", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.2", "instance": "", "state": "complete", "index_ver": 9, "timestamp": "2023-10-23T13:04:11.685823424Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [] }, { "op_id": "6#00000000010.25.14", "op_tag": "", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.3", "instance": "", "state": "complete", "index_ver": 10, "timestamp": "2023-10-23T13:04:18.041789126Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [] }, { "op_id": "6#00000000011.25.17", "op_tag": "", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.4", "instance": "", "state": "complete", "index_ver": 11, "timestamp": "2023-10-23T13:04:24.643495411Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [] }, { "op_id": "6#00000000012.25.20", "op_tag": "", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.5", "instance": "", "state": "complete", "index_ver": 12, "timestamp": "2023-10-23T13:04:30.349749107Z", "ver": { "pool": 11, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [] }, { "op_id": "6#00000000013.27.6", "op_tag": "_Q9P7Yd2HpwjX6bIZzAmcaA7q1U6MoxK", "op": "del", "object": "_multipart_my-object.2~s1bO1s8i0e5Ihjpodq5lm68xGUGkguk.meta", "instance": "", "state": "complete", "index_ver": 13, "timestamp": "2023-10-23T13:27:52.165706660Z", "ver": { "pool": 10, "epoch": 0 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] } ] [root@localhost build]# ../src/mrun c2 radosgw-admin bilog list --bucket=bkt 2023-10-23T09:30:42.153-0400 7f1459403780 -1 WARNING: all dangerous and experimental features are enabled. 2023-10-23T09:30:42.156-0400 7f1459403780 -1 WARNING: all dangerous and experimental features are enabled. 2023-10-23T09:30:42.157-0400 7f1459403780 -1 WARNING: all dangerous and experimental features are enabled. [ { "op_id": "6#00000000001.12.6", "op_tag": "_ZTqBE-J7PMP9CqUbtmQyMySh6S50iwE", "op": "write", "object": "my-object", "instance": "", "state": "complete", "index_ver": 1, "timestamp": "2023-10-23T13:27:52.186580707Z", "ver": { "pool": 9, "epoch": 7 }, "bilog_flags": 0, "versioned": false, "owner": "", "owner_display_name": "", "zones_trace": [ { "entry": "33374512-b9e7-459d-85a5-2804d61f1d92:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" }, { "entry": "a24706c2-772d-48e1-883b-a3f3457c40d9:bkt:33374512-b9e7-459d-85a5-2804d61f1d92.4233.1" } ] } ]
Updated by Casey Bodley 6 months ago
In the latest version, I was not able to reproduce the issue
thanks for looking, Jiffin. the point of this tracker is to stop writing those bilog entries for _multipart_
objects. i see they're still present in the bilog list
output on the source zone
Updated by Shilpa MJ 5 months ago
- Status changed from New to Duplicate
duplicate of https://tracker.ceph.com/issues/21800
Actions