Project

General

Profile

Bug #22834 ยป prim.in.peer_info.log

David Zafman, 01/30/2018 11:43 PM

 
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats actingset 0,2 upset 0,2 acting_recovery_backfill 0,2
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats acting [2,0] up [2,0]
2018-01-29 22:25:12.800 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats initial missing_target_objects size = 1
2018-01-29 22:25:12.800 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats target = 2
2018-01-29 22:25:12.800 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing_target_objects size = 2
2018-01-29 22:25:12.800 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing_target_objects = 2,0,2,2
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing shard 0 missing= 2
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing shard 2 missing= 2
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats degraded 4
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats misplaced 0
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] publish_stats_to_osd reporting purged_snaps []
2018-01-29 22:25:12.800 7fb3a25ec700 15 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] publish_stats_to_osd 27:1285
2018-01-29 22:25:12.800 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] removing repgather(0x563df0d86e80 27'1268 rep_tid=7501 committed?=1 applied?=1 r=0)
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] q front is repgather(0x563df0d86e80 27'1268 rep_tid=7501 committed?=1 applied?=1 r=0)
2018-01-29 22:25:12.800 7fb3a25ec700 15 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_osd_op_effects client.4357 con 0x563de8153800
2018-01-29 22:25:12.800 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] remove_repop repgather(0x563df0d86e80 27'1268 rep_tid=7501 committed?=1 applied?=1 r=0)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] all_unfound_are_queried_or_lost all of might_have_unfound 0,1 have been queried or are marked lost
2018-01-29 22:25:12.808 7fb3a0de9700 3 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] mark_all_unfound_lost l_revert
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:70b3bf12:::existing_4:head 0'0 on osd.2 (local)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:70b3bf12:::existing_4:head 17'1 on osd.0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:70b3bf12:::existing_4:head 17'1 (newest)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] 27'1269 (19'3) l_revert 2:70b3bf12:::existing_4:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:78623b1d:::existing_6:head 0'0 on osd.2 (local)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:78623b1d:::existing_6:head 17'2 on osd.0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] pick_newest_available 2:78623b1d:::existing_6:head 17'2 (newest)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] 27'1270 (19'4) l_revert 2:78623b1d:::existing_6:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] submit_log_entries 27'1269 (19'3) l_revert 2:70b3bf12:::existing_4:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0,27'1270 (19'4) l_revert 2:78623b1d:::existing_6:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] new_repop: repgather(0x563deeb33480 27'1270 rep_tid=7511 committed?=0 applied?=0 r=0)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries 27'1269 (19'3) l_revert 2:70b3bf12:::existing_4:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0,27'1270 (19'4) l_revert 2:78623b1d:::existing_6:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] append_log_entries_update_missing info.last_backfill MAX
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_missing before for peer 0 = {2:70b3bf12:::existing_4:head=19'3(17'1) flags = none,2:78623b1d:::existing_6:head=19'4(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_info before for peer 0 = 2.e( v 27'1268 lc 17'2 (0'0,27'1268] local-lis/les=26/27 n=2 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26)
2018-01-29 22:25:12.808 7fb3a0de9700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_missing for 0 = missing(2 may_include_deletes = 1)
2018-01-29 22:25:12.808 7fb3a0de9700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] missing {2:70b3bf12:::existing_4:head=27'1269(17'1) flags = none,2:78623b1d:::existing_6:head=19'4(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a0de9700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] missing {2:70b3bf12:::existing_4:head=27'1269(17'1) flags = none,2:78623b1d:::existing_6:head=27'1270(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_missing after for peer 0 = {2:70b3bf12:::existing_4:head=27'1269(17'1) flags = none,2:78623b1d:::existing_6:head=27'1270(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_info after for peer 0 = 2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=2 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_missing before for peer 2 = {}
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] merge_new_log_entries peer_info before for peer 2 = 0.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c 0/0 les/c/f 0/0/0 0/0/0)
2018-01-29 22:25:12.808 7fb3a0de9700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] op_applied version 27'1270
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 27 dequeue_op 0x563ded0df480 prio 63 cost 4096 latency 0.000077 osd_op(client.4357.0:20199 2.e 2.1baae84e (undecoded) ondisk+write+known_if_redirected e27) v9 pg pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2]
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _handle_message: 0x563ded0df480
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_op: op osd_op(client.4357.0:20199 2.e 2:721755d8:::benchmark_data_smithi169_31450_object20198:head [set-alloc-hint object_size 4096 write_size 4096,write 0~4096] snapc 0=[] ondisk+write+known_if_redirected e27) v9
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] op_has_sufficient_caps session=0x563defc37480 pool=2 (unfound_pool ) owner=0 pool_app_metadata={rados={}} need_read_cap=0 need_write_cap=1 classes=[] -> yes
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_op osd_op(client.4357.0:20199 2.e 2:721755d8:::benchmark_data_smithi169_31450_object20198:head [set-alloc-hint object_size 4096 write_size 4096,write 0~4096] snapc 0=[] ondisk+write+known_if_redirected e27) v9 may_write -> write-ordered flags ondisk+write+known_if_redirected
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] get_object_context: obc NOT found in cache: 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] get_object_context: no obc for soid 2:721755d8:::benchmark_data_smithi169_31450_object20198:head but can_create
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] create_object_context 0x563decf93480 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] populate_obc_watchers 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] PrimaryLogPG::check_blacklisted_obc_watchers for obc 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] get_object_context: 0x563decf93480 2:721755d8:::benchmark_data_smithi169_31450_object20198:head rwstate(none n=0 w=0) oi: 2:721755d8:::benchmark_data_smithi169_31450_object20198:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0]) ssc: 0x563dec8b8ec0 snapset: 0=[]:{}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] find_object_context 2:721755d8:::benchmark_data_smithi169_31450_object20198:head @head oi=2:721755d8:::benchmark_data_smithi169_31450_object20198:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0])
2018-01-29 22:25:12.808 7fb3a65f4700 25 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_op oi 2:721755d8:::benchmark_data_smithi169_31450_object20198:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0])
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_op obc obc(2:721755d8:::benchmark_data_smithi169_31450_object20198:head(dne) rwstate(write n=1 w=0))
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] execute_ctx 0x563de8e67800
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] execute_ctx 2:721755d8:::benchmark_data_smithi169_31450_object20198:head [set-alloc-hint object_size 4096 write_size 4096,write 0~4096] ov 0'0 av 27'1271 snapc 0=[] snapset 0=[]:{}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_osd_op 2:721755d8:::benchmark_data_smithi169_31450_object20198:head [set-alloc-hint object_size 4096 write_size 4096,write 0~4096]
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_osd_op set-alloc-hint object_size 4096 write_size 4096
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_osd_op write 0~4096
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] make_writeable 2:721755d8:::benchmark_data_smithi169_31450_object20198:head snapset=0=[]:{} snapc=0=[]
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] setting DIRTY flag
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] make_writeable 2:721755d8:::benchmark_data_smithi169_31450_object20198:head done, snapset=0=[]:{}
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] finish_ctx 2:721755d8:::benchmark_data_smithi169_31450_object20198:head 0x563de8e67800 op modify
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] set mtime to 2018-01-29 22:25:12.812575
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] final snapset 0=[]:{} in 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] zeroing write result code 0
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] op order client.4357 tid 20199 (first)
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] new_repop rep_tid 7513 on osd_op(client.4357.0:20199 2.e 2:721755d8:::benchmark_data_smithi169_31450_object20198:head [set-alloc-hint object_size 4096 write_size 4096,write 0~4096] snapc 0=[] ondisk+write+known_if_redirected e27) v9
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] new_repop: repgather(0x563df0360300 0'0 rep_tid=7513 committed?=0 applied?=0 r=0)
2018-01-29 22:25:12.808 7fb3a65f4700 7 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] issue_repop rep_tid 7513 o 2:721755d8:::benchmark_data_smithi169_31450_object20198:head
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] issue_repop shard 0 before missing {2:70b3bf12:::existing_4:head=27'1269(17'1) flags = none,2:78623b1d:::existing_6:head=27'1270(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] issue_repop shard 2 before missing {}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] issue_repop shard 0 after add next missing {2:70b3bf12:::existing_4:head=27'1269(17'1) flags = none,2:78623b1d:::existing_6:head=27'1270(17'2) flags = none}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1266 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] issue_repop shard 2 after add next missing {}
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1270 lc 17'2 (0'0,27'1270] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] append_log log((0'0,27'1270], crt=27'1268) [27'1271 (0'0) modify 2:721755d8:::benchmark_data_smithi169_31450_object20198:head by client.4357.0:20199 2018-01-29 22:25:12.812575 0]
2018-01-29 22:25:12.808 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1268 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] add_log_entry 27'1271 (0'0) modify 2:721755d8:::benchmark_data_smithi169_31450_object20198:head by client.4357.0:20199 2018-01-29 22:25:12.812575 0
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] rollforward: entry=27'1269 (19'3) l_revert 2:70b3bf12:::existing_4:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] rollforward: entry=27'1270 (19'4) l_revert 2:78623b1d:::existing_6:head by unknown.0.0:0 2018-01-29 22:25:12.810245 0
2018-01-29 22:25:12.808 7fb3a65f4700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] rollforward: entry=27'1271 (0'0) modify 2:721755d8:::benchmark_data_smithi169_31450_object20198:head by client.4357.0:20199 2018-01-29 22:25:12.812575 0
2018-01-29 22:25:12.812 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] op_applied: 7513
2018-01-29 22:25:12.812 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 lua=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] op_applied version 27'1271
2018-01-29 22:25:12.812 7fb3a65f4700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] eval_repop repgather(0x563df0360300 27'1271 rep_tid=7513 committed?=0 applied?=0 r=0)
2018-01-29 22:25:12.812 7fb3a25ec700 10 osd.2 27 dequeue_op 0x563dedb98140 prio 196 cost 0 latency 0.001055 pg_update_log_missing_reply(2.e epoch 27/26 rep_tid 7511) v2 pg pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2]
2018-01-29 22:25:12.812 7fb3a25ec700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _handle_message: 0x563dedb98140
2018-01-29 22:25:12.812 7fb3a25ec700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] do_update_log_missing_reply got reply from 0
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1268 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] repop_all_committed: repop tid 7511 all committed
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] eval_repop repgather(0x563deeb33480 27'1270 rep_tid=7511 committed?=1 applied?=1 r=0) (no op)
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] commit: repgather(0x563deeb33480 27'1270 rep_tid=7511 committed?=1 applied?=1 r=0)
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] applied: repgather(0x563deeb33480 27'1270 rep_tid=7511 committed?=1 applied?=1 r=0)
2018-01-29 22:25:12.812 7fb3b160a700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats actingset 0,2 upset 0,2 acting_recovery_backfill 0,2
2018-01-29 22:25:12.812 7fb3b160a700 20 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats acting [2,0] up [2,0]
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats initial missing_target_objects size = 1
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats target = 2
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing_target_objects size = 3
2018-01-29 22:25:12.812 7fb3b160a700 10 osd.2 pg_epoch: 27 pg[2.e( v 27'1271 lc 17'2 (0'0,27'1271] local-lis/les=26/27 n=1267 ec=12/12 lis/c 26/16 les/c/f 27/17/0 26/26/26) [2,0] r=0 lpr=26 pi=[12,26)/2 luod=27'1270 crt=27'1271 mlcod 17'2 active+recovery_unfound+degraded m=2 u=2] _update_calc_stats missing_target_objects = 0,2,2,0,2,2
2018-01-29 22:25:12.812 7fb3b160a700 -1 /build/ceph-13.0.0-4965-g1d0e064/src/osd/PG.cc: In function 'void PG::_update_calc_stats()' thread 7fb3b160a700 time 2018-01-29 22:25:12.814656
/build/ceph-13.0.0-4965-g1d0e064/src/osd/PG.cc: 2884: FAILED assert(target >= missing_target_objects.size())

ceph version 13.0.0-4965-g1d0e064 (1d0e064b5384f091d462a0d0e2e97aef10db2b60) mimic (dev)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xf5) [0x7fb3c8f42f75]
2: (PG::_update_calc_stats()+0x146a) [0x563de5bbf5fa]
3: (PG::publish_stats_to_osd()+0x10f) [0x563de5bdbb3f]
4: (PrimaryLogPG::eval_repop(PrimaryLogPG::RepGather*)+0x245) [0x563de5c86b65]
5: (()+0x4dd772) [0x563de5c87772]
6: (Context::complete(int)+0x9) [0x563de5b56e19]
7: (Finisher::finisher_thread_entry()+0x31f) [0x7fb3c8f4158f]
8: (()+0x76ba) [0x7fb3c7a196ba]
9: (clone()+0x6d) [0x7fb3c72423dd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
    (1-1/1)