Project

General

Profile

Actions
Copy link

Bug #42429

open

LC process falls into an infinite loop during processing transition

Added by Ilsoo Byun over 4 years ago. Updated over 4 years ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
Matt Benjamin
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
31214
Crash signature (v1):
Crash signature (v2):

Description

If two LC processes tries to transition the same object by chance, one of them falls into an infinite loop.
The following logs shows what happened when two lc processes conflict at the myobjects_1815916180 object.

2019-10-19 17:28:29.727 7ff8f6f74700 20 lifecycle: bucket_lc_process(): key=lifecycle/myobjects_1815916180
2019-10-19 17:28:29.727 7ff8f6f74700 20 check(): key=lifecycle/myobjects_1815916180: current version, skipping
2019-10-19 17:28:29.727 7ff8f6f74700 20 obj_has_expired(): mtime=2019-10-19 16:10:21.0.480875s days=1 base_time=2019-10-19 17:28:29.728551 timediff=4688.73 cmp=600
2019-10-19 17:28:29.727 7ff8f6f74700 20 check(): key=lifecycle/myobjects_1815916180: is_expired=1
2019-10-19 17:28:29.727 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.942 7ff8f6f74700 10 manifest: total_size = 939000
2019-10-19 17:28:29.942 7ff8f6f74700 20 get_obj_state: setting s->obj_tag to 11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3381854.29704537
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.acl
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.content_type
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.etag
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.idtag
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.manifest
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.pg_ver
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.source_zone
2019-10-19 17:28:29.942 7ff8f6f74700 20 Read xattr: user.rgw.tail_tag
2019-10-19 17:28:29.942 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.942 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.942 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.942 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.942 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.942 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.942 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.943 7ff8f6f74700 20 rados->read r=0 bl.length=0
2019-10-19 17:28:29.943 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.943 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.943 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.943 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.943 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.944 7ff8f6f74700 20 rados->read r=0 bl.length=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.944 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.944 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.944 7ff8f6f74700 20 rados->read r=0 bl.length=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.944 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.944 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.944 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.945 7ff8f6f74700 20 rados->read r=0 bl.length=0
2019-10-19 17:28:29.945 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.945 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.945 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.945 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.945 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.946 7ff8f6f74700 20 rados->read r=0 bl.length=0
2019-10-19 17:28:29.946 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.946 7ff8f6f74700 20 max_chunk_size=4194304
2019-10-19 17:28:29.946 7ff8f6f74700 20 get_obj_state: rctx=0x7ff8f6f70f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x559ee08e8ea0 s->prefetch_data=0
2019-10-19 17:28:29.946 7ff8f6f74700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.946 7ff8f6f74700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.947 7ff8f6f74700 20 rados->read r=0 bl.length=0
…(infinite loop)


2019-10-19 17:28:29.719 7f17c4f4c700 20 lifecycle: bucket_lc_process(): key=lifecycle/myobjects_1815916180
2019-10-19 17:28:29.719 7f17c4f4c700 20 check(): key=lifecycle/myobjects_1815916180: current version, skipping
2019-10-19 17:28:29.719 7f17c4f4c700 20 obj_has_expired(): mtime=2019-10-19 16:10:21.0.480875s days=1 base_time=2019-10-19 17:28:29.720686 timediff=4688.72 cmp=600
2019-10-19 17:28:29.719 7f17c4f4c700 20 check(): key=lifecycle/myobjects_1815916180: is_expired=1
2019-10-19 17:28:29.719 7f17c4f4c700 20 get_obj_state: rctx=0x7f17c4f48f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x555665f1b1a0 s->prefetch_data=0
2019-10-19 17:28:29.720 7f17c4f4c700 10 manifest: total_size = 939000
2019-10-19 17:28:29.720 7f17c4f4c700 20 get_obj_state: setting s->obj_tag to 11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3381854.29704537
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.acl
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.content_type
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.etag
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.idtag
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.manifest
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.pg_ver
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.source_zone
2019-10-19 17:28:29.720 7f17c4f4c700 20 Read xattr: user.rgw.tail_tag
2019-10-19 17:28:29.720 7f17c4f4c700 20 max_chunk_size=4194304
2019-10-19 17:28:29.720 7f17c4f4c700 20 max_chunk_size=4194304
2019-10-19 17:28:29.720 7f17c4f4c700 20 get_obj_state: rctx=0x7f17c4f48f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x555665f1b1a0 s->prefetch_data=0
2019-10-19 17:28:29.720 7f17c4f4c700 20 max_chunk_size=4194304
2019-10-19 17:28:29.720 7f17c4f4c700 20 get_obj_state: rctx=0x7f17c4f48f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x555665f1b1a0 s->prefetch_data=0
2019-10-19 17:28:29.720 7f17c4f4c700 20 state for obj=bucket-lc939:lifecycle/myobjects_1815916180 is not atomic, not appending atomic test
2019-10-19 17:28:29.720 7f17c4f4c700 20 rados->read obj-ofs=0 read_ofs=0 read_len=939000
2019-10-19 17:28:29.726 7f17c4f4c700 20 rados->read r=0 bl.length=939000
2019-10-19 17:28:29.727 7f17c4f4c700 20 max_chunk_size=4194304
2019-10-19 17:28:29.753 7f17c4f4c700 20 get_obj_state: rctx=0x7f17c4f48f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x555665f1b1a0 s->prefetch_data=0
2019-10-19 17:28:29.753 7f17c4f4c700 20 get_obj_state: rctx=0x7f17c4f48f98 obj=bucket-lc939:lifecycle/myobjects_1815916180 state=0x555665f1b1a0 s->prefetch_data=0
2019-10-19 17:28:29.753 7f17c4f4c700 10 setting object write_tag=_FRx-D5MQFpyGyOUGKVzI_enI8-TBAk9
2019-10-19 17:28:29.753 7f17c4f4c700 20 reading from default.rgw.meta:root:.bucket.meta.bucket-lc939:11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791
2019-10-19 17:28:29.753 7f17c4f4c700 20 get_system_obj_state: rctx=0x7f17c4f47100 obj=default.rgw.meta:root:.bucket.meta.bucket-lc939:11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791 state=0x5556666b76c0 s->prefetch_data=0
2019-10-19 17:28:29.753 7f17c4f4c700 10 cache get: name=default.rgw.meta+root+.bucket.meta.bucket-lc939:11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791 : hit (requested=0x16, cached=0x17)
2019-10-19 17:28:29.753 7f17c4f4c700 20 get_system_obj_state: s->obj_tag was set empty
2019-10-19 17:28:29.753 7f17c4f4c700 10 cache get: name=default.rgw.meta+root+.bucket.meta.bucket-lc939:11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791 : hit (requested=0x11, cached=0x17)
2019-10-19 17:28:29.753 7f17c4f4c700 20  bucket index object: .dir.11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791.0
2019-10-19 17:28:29.782 7f17c4f4c700  2 TRANSITIONED:bucket-lc939[11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791]:lifecycle/myobjects_1815916180 -> STANDARD_IA
2019-10-19 17:28:29.782 7f17c4f4c700 20 processed:bucket-lc939[11fd7df3-e916-4ac4-af14-7dca2dd1ee0d.3402367.3791]:lifecycle/myobjects_1815916180
Actions
Copy link
 #1

Updated by Brad Hubbard over 4 years ago

  • Project changed from Ceph to rgw
Actions
Copy link
 #2

Updated by Casey Bodley over 4 years ago

cls locks should be preventing multiple gateways from duplicating lifecycle processing, so I wonder where the gap is here

Actions
Copy link
 #3

Updated by Abhishek Lekshmanan over 4 years ago

How were the 2 processes run in this case? was one of the processes a radosgw-admin process or are these both radosgw processes that happened to process the same object

Actions
Copy link
 #4

Updated by Ilsoo Byun over 4 years ago

It is the latter. We run multiple gateways and the each gateway is configured to run lc.

Actions
Copy link
 #5

Updated by Ilsoo Byun over 4 years ago

I found that’s because one of two lc processes read the stale head object. I created a related PR(https://github.com/ceph/ceph/pull/31214).

When transitioning object, even though the new write tag is assigned, the old tag remains as the following RGW log shows.
The reason is that the transaction setting the previous tag was added together(refer to the OSD log).

Therefore I modified two parts.

1. add atomic test to read_op to avoid reading from the stale head object.
2. before calling copy_obj_data, remove tag information from the attribute set

<<RGW log>>

(Put request)
2019-10-28 19:52:15.970 7fd5a5d59700  1 ====== starting new request req=0x5650cce428d0 =====
….
2019-10-28 19:52:15.980 7fd5a5d59700  0 RGWRados::Object::prepare_atomic_modification ptag=a2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
2019-10-28 19:52:15.980 7fd5a5d59700  0 setting object write_tag=a2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
….
2019-10-28 19:52:16.007 7fd5a5d59700  1 ====== req done req=0x5650cce428d0 op status=0 http_status=200 latency=0.0370014s ======

(Transition)
2019-10-28 19:52:41.412 7fd5a9d61700  0 setting object write_tag=__VmLBnnX6aBaPYDr7DCaBsJk-NR-dXo

(Get request)
2019-10-28 20:00:03.575 7fd59653a700  1 ====== starting new request req=0x5650cce928d0 =====
…
2019-10-28 20:00:03.578 7fd59653a700  0 s->obj_tag=a2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
…
2019-10-28 20:00:03.585 7fd59653a700  1 ====== req done req=0x5650cce928d0 op status=0 http_status=200 latency=0.0100004s ======

<<OSD log>>
Please pay attention to the '_user.rgw.idtag' attribute.

2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.idtag value=__VmLBnnX6aBaPYDr7DCaBsJk-NR-dXo
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.tail_tag value=__VmLBnnX6aBaPYDr7DCaBsJk-NR-dXo
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.manifest value==
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.acl value=
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.content_type value=application/octet-stream
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.etag value=b6d81b360a5672d80c27430f39153e2cuser.rgw.idtaga2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.idtag value=a2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.pg_ver value=
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.source_zone value=
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.storage_class value=STANDARDuser.rgw.tail_taga2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.tail_tag value=a2b78a35-3263-43fc-9884-2b02e802281e.2034102.2
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.x-amz-date value=Mon, 28 Oct 2019 10:52:15 +0000
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.x-amz-meta-s3cmd-attrs value=atime:1569893162/ctime:1571739046/gid:11000/gname:irteamsu/md5:b6d81b360a5672d80c27430f39153e2c/mode:33188/mtime:1571739046/uid:11000/uname:irteamsu
2019-10-28 19:52:41.418 7f9bb1dea700  0 osd.0 pg_epoch: 605 pg[6.4( v 605'2442 (0'0,605'2442] local-lis/les=604/605 n=1 ec=78/78 lis/c 604/604 les/c/f 605/605/0 604/604/604) [0] r=0 lpr=604 crt=605'2442 lcod 599'2441 mlcod 599'2441 active+clean] CEPH_OSD_OP_SETXATTR 6:25c57b01:::a2b78a35-3263-43fc-9884-2b02e802281e.1314125.1_test1m:head name=_user.rgw.storage_class value=STANDARD_IA
Actions
Copy link
 #6

Updated by Casey Bodley over 4 years ago

  • Status changed from New to Fix Under Review
  • Assignee set to Matt Benjamin
  • Pull request ID set to 31214
Actions
Copy link
 #7

Updated by Matt Benjamin over 4 years ago

  • Backport set to nautilus
Actions
Copy link

Also available in: Atom PDF