Project

General

Profile

Bug #40522

on_local_recover doesn't touch?

Added by Sage Weil almost 5 years ago. Updated over 3 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2019-06-22T11:45:01.779+0000 7f87a234e700 10 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=4}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] on_local_recover: 2:b022980f:::smithi04715146-47:head
2019-06-22T11:45:01.779+0000 7f87a234e700 10 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=4}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] got missing 2:b022980f:::smithi04715146-47:head v 1351'676
2019-06-22T11:45:01.779+0000 7f87a234e700 10 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=4}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] last_complete now 1350'674 log.complete_to 1351'675
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats actingset 5,7 upset 5,7 acting_recovery_backfill 5,7
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats shard 5 primary objects 4 missing 3
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats shard 1 objects 4 missing 0
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats shard 7 objects 4 missing 0
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats object_location_counts {1,5,7=1,1,7=3}
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats ml 3 upset size 2 up 1
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats ml 1 upset size 2 up 2
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats missing based degraded 0
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats missing based misplaced 3
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats degraded 0
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] update_calc_stats misplaced 3
2019-06-22T11:45:01.779+0000 7f87a234e700 20 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] prepare_stats_for_publish reporting purged_snaps [322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]
2019-06-22T11:45:01.779+0000 7f87a234e700 15 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] publish_stats_to_osd 1355:4108
2019-06-22T11:45:01.779+0000 7f87a234e700 10 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] release_backoffs [2:b022980f:::smithi04715146-47:head,2:b022980f:::smithi04715146-47:head)
2019-06-22T11:45:01.779+0000 7f87a234e700 10 osd.5 pg_epoch: 1355 pg[2.d( v 1351'678 lc 1350'674 (997'542,1351'678] local-lis/les=1354/1355 n=4 ec=917/21 lis/c 1354/1349 les/c/f 1355/1350/0 1353/1354/1354) [5,7] r=0 lpr=1354 pi=[1166,1354)/2 crt=1351'678 mlcod 0'0 active+recovering rops=1 m=1 mbc={255={(1+1)=3,(2+1)=1}} ps=[322~2,327~2,32b~1,32d~1,32f~2,332~1,334~1,336~2]] readable_with_acting: locs:1,5,7
...
 -3556> 2019-06-22T11:45:01.779+0000 7f87a234e700 10 bluestore(/var/lib/ceph/osd/ceph-5) queue_transactions ch 0xd359200 2.d_head
 -3552> 2019-06-22T11:45:01.779+0000 7f87a234e700 20 bluestore(/var/lib/ceph/osd/ceph-5) _txc_create osr 0xd36eb40 = 0x10e6d000 seq 12
 -3544> 2019-06-22T11:45:01.779+0000 7f87a234e700 10 bluestore(/var/lib/ceph/osd/ceph-5) _txc_add_transaction op 28 got ENOENT on #2:b022980f:::smithi04715146-47:head#
 -3541> 2019-06-22T11:45:01.779+0000 7f87a234e700 10 bluestore(/var/lib/ceph/osd/ceph-5) _txc_add_transaction op 12 got ENOENT on #2:b022980f:::smithi04715146-47:head#
 -3538> 2019-06-22T11:45:01.779+0000 7f87a234e700 10 bluestore(/var/lib/ceph/osd/ceph-5) _txc_add_transaction op 15 got ENOENT on #2:b022980f:::smithi04715146-47:head#
 -3528> 2019-06-22T11:45:01.779+0000 7f87a234e700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _txc_add_transaction error (2) No such file or directory not handled on operation 15 (op 2, counting from 0)
 -3527> 2019-06-22T11:45:01.779+0000 7f87a234e700 -1 bluestore(/var/lib/ceph/osd/ceph-5) unexpected error code
 -3462> 2019-06-22T11:45:01.779+0000 7f87a234e700  0 _dump_transaction transaction dump:
{
    "ops": [
        {
            "op_num": 0,
            "op_name": "rmattrs",
            "collection": "2.d_head",
            "oid": "#2:b022980f:::smithi04715146-47:head#" 
        },
        {
            "op_num": 1,
            "op_name": "truncate",
            "collection": "2.d_head",
            "oid": "#2:b022980f:::smithi04715146-47:head#",
            "offset": 2672643
        },
        {
            "op_num": 2,
            "op_name": "setattrs",
            "collection": "2.d_head",
            "oid": "#2:b022980f:::smithi04715146-47:head#",
            "attr_lens": {
                "_": 272,
                "__header": 58,
                "snapset": 131
            }
        },
        {
            "op_num": 3,
            "op_name": "omap_rmkeys",
            "collection": "2.d_head",
            "oid": "#2:b0000000::::head#",
            "attrs": [
                "missing/0000000000000002.D044910F.head.smithi04715146-47.." 
            ]
        },
        {
            "op_num": 4,
            "op_name": "omap_setkeys",
            "collection": "2.d_head",
            "oid": "#2:b0000000::::head#",
            "attr_lens": {
                "_info": 992
            }
        }
    ]
}

/a/sage-2019-06-21_16:36:51-rados:thrash-wip-sage2-testing-2019-06-20-2154-distro-basic-smithi/4057772

Related issues

Related to RADOS - Bug #37969: ENOENT on setattrs Can't reproduce 01/18/2019
Related to RADOS - Bug #22624: filestore: 3180: FAILED assert(0 == "unexpected error"): error (2) No such file or directory not handled on operation 0x55e1ce80443c (21888.1.0, or op 0, counting from 0) Duplicate 01/08/2018
Related to RADOS - Bug #42175: _txc_add_transaction error (2) No such file or directory not handled on operation 15 Can't reproduce

History

#1 Updated by jianpeng ma almost 5 years ago

2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] _handle_message: 0xff7a6e0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_op: op osd_op(client.4312.0:185 2.d 2:b022980f:::smithi04715146-47:head [write 773122~559572,s
tat] snapc 0=[] ondisk+write+known_if_redirected e24) v8
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] op_has_sufficient_caps session=0x100a8500 pool=2 (unique_pool_0 ) pool_app_metadata={rados={}} ne
ed_read_cap=1 need_write_cap=1 classes=[] -> yes
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_op osd_op(client.4312.0:185 2.d 2:b022980f:::smithi04715146-47:head [write 773122~559572,stat] snapc 0=[] ondisk+write+known_if_redirected e24) v8 may_write may_read -> write-ordered flags ondisk+write+known_if_redirected
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] get_object_context: obc NOT found in cache: 2:b022980f:::smithi04715146-47:head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 15 bluestore(/var/lib/ceph/osd/ceph-5) getattr 2.d_head #2:b022980f:::smithi04715146-47:head# _
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5).collection(2.d_head 0xfe41a00) get_onode oid #2:b022980f:::smithi04715146-47:head# key 0x7f8000000000000002b022980f21736d'ithi04715146-47!='0xfffffffffffffffeffffffffffffffff'o'
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5).collection(2.d_head 0xfe41a00)  r -2 v.len 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 bluestore(/var/lib/ceph/osd/ceph-5) getattr 2.d_head #2:b022980f:::smithi04715146-47:head# _ = -2
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] get_object_context: no obc for soid 2:b022980f:::smithi04715146-47:head but can_create
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] create_object_context 0x1000a3c0 2:b022980f:::smithi04715146-47:head 
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] populate_obc_watchers 2:b022980f:::smithi04715146-47:head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] PrimaryLogPG::check_blacklisted_obc_watchers for obc 2:b022980f:::smithi04715146-47:head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] get_object_context: 0x1000a3c0 2:b022980f:::smithi04715146-47:head rwstate(none n=0 w=0) oi: 2:b022980f:::smithi04715146-47:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0]) ssc: 0xff7bce0 snapset: 0=[]:{}
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] find_object_context 2:b022980f:::smithi04715146-47:head @head oi=2:b022980f:::smithi04715146-47:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0])
2019-06-22T11:19:10.964+0000 7fcbe4c17700 25 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_op oi 2:b022980f:::smithi04715146-47:head(0'0 unknown.0.0:0 s 0 uv 0 alloc_hint [0 0 0])
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_op obc obc(2:b022980f:::smithi04715146-47:head(dne) rwstate(excl n=1 w=0))
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] execute_ctx 0xfc38400
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] execute_ctx 2:b022980f:::smithi04715146-47:head [write 773122~559572,stat] ov 0'0 av 24'4 snapc 0=[] snapset 0=[]:{}
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_osd_op 2:b022980f:::smithi04715146-47:head [write 773122~559572,stat]
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_osd_op  write 773122~559572
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] clean_regions modifiedclean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] do_osd_op  stat
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] stat oi has 1332694 0.000000
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] make_writeable 2:b022980f:::smithi04715146-47:head snapset=0=[]:{}  snapc=0=[]
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean]  setting DIRTY flag
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] make_writeable 2:b022980f:::smithi04715146-47:head done, snapset=0=[]:{}
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] finish_ctx 2:b022980f:::smithi04715146-47:head 0xfc38400 op modify
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean]  set mtime to 2019-06-22T11:19:10.968879+0000
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean]  final snapset 0=[]:{} in 2:b022980f:::smithi04715146-47:head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] finish_ctx object 2:b022980f:::smithi04715146-47:head marks clean_regions clean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean]  zeroing write result code 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] calc_trim_to_aggressive limit = 24'3
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean]  op order client.4312 tid 185 (first)
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] new_repop rep_tid 15 on osd_op(client.4312.0:185 2.d 2:b022980f:::smithi04715146-47:head [write 773122~559572,stat] snapc 0=[] ondisk+write+known_if_redirected e24) v8
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] new_repop: repgather(0xc65d780 0'0 rep_tid=15 committed?=0 r=0)
2019-06-22T11:19:10.964+0000 7fcbe4c17700  7 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=1 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] issue_repop rep_tid 15 o 2:b022980f:::smithi04715146-47:head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 -- v2:172.21.15.19:6807/13238 connect_to v2:172.21.15.47:6801/13114 existing 0xfcb7680
2019-06-22T11:19:10.964+0000 7fcbe4c17700  1 -- v2:172.21.15.19:6807/13238 --> v2:172.21.15.47:6801/13114 -- osd_repop(client.4312.0:185 2.d e24/21 2:b022980f:::smithi04715146-47:head v 24'4) v2 -- 0x100e8000 con 0xfcb7680
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 --2- v2:172.21.15.19:6807/13238 >> v2:172.21.15.47:6801/13114 conn(0xfcb7680 0xfcc0680 crc :-1 s=READY pgs=4 cs=0 l=0 rx=0 tx=0).prepare_send_message m=osd_repop(client.4312.0:185 2.d e24/21 2:b022980f:::smithi04715146-47:head v 24'4) v2
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 --2- v2:172.21.15.19:6807/13238 >> v2:172.21.15.47:6801/13114 conn(0xfcb7680 0xfcc0680 crc :-1 s=READY pgs=4 cs=0 l=0 rx=0 tx=0).prepare_send_message encoding features 4540138292836696063 0x100e8000 osd_repop(client.4312.0:185 2.d e24/21 2:b022980f:::smithi04715146-47:head v 24'4) v2
2019-06-22T11:19:10.964+0000 7fcbe4c17700  5 --2- v2:172.21.15.19:6807/13238 >> v2:172.21.15.47:6801/13114 conn(0xfcb7680 0xfcc0680 crc :-1 s=READY pgs=4 cs=0 l=0 rx=0 tx=0).send_message enqueueing message m=0x100e8000 type=112 osd_repop(client.4312.0:185 2.d e24/21 2:b022980f:::smithi04715146-47:head v 24'4) v2
2019-06-22T11:19:10.964+0000 7fcbe4c17700 15 --2- v2:172.21.15.19:6807/13238 >> v2:172.21.15.47:6801/13114 conn(0xfcb7680 0xfcc0680 crc :-1 s=READY pgs=4 cs=0 l=0 rx=0 tx=0).send_message inline write is denied, reschedule m=0x100e8000
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 Event(0xc6474c0 nevent=5000 time_id=201).wakeup
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 earliest_dup_version = 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 trim 24'4 (0'0) modify   2:b022980f:::smithi04715146-47:head by client.4312.0:185 2019-06-22T11:19:10.968879+0000 0 ObjectCleanRegions clean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'3 (0'0,24'3] local-lis/les=21/22 n=2 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] append_log log((0'0,24'3], crt=24'3) [24'4 (0'0) modify   2:b022980f:::smithi04715146-47:head by client.4312.0:185 2019-06-22T11:19:10.968879+0000 0 ObjectCleanRegions clean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0]
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'4 (0'0,24'4] local-lis/les=21/22 n=2 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 luod=24'3 lua=24'3 crt=24'3 lcod 24'2 mlcod 24'2 active+clean] add_log_entry 24'4 (0'0) modify   2:b022980f:::smithi04715146-47:head by client.4312.0:185 2019-06-22T11:19:10.968879+0000 0 ObjectCleanRegions clean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 osd.5 pg_epoch: 24 pg[2.d( v 24'4 (0'0,24'4] local-lis/les=21/22 n=2 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 luod=24'3 lua=24'3 crt=24'4 lcod 24'2 mlcod 24'2 active+clean] rollforward: entry=24'4 (0'0) modify   2:b022980f:::smithi04715146-47:head by client.4312.0:185 2019-06-22T11:19:10.968879+0000 0 ObjectCleanRegions clean_offsets: [0~773122,1332694~18446744073708218921], clean_omap: 1, object_new: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'4 (0'0,24'4] local-lis/les=21/22 n=2 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 luod=24'3 lua=24'3 crt=24'4 lcod 24'2 mlcod 24'2 active+clean] append_log approx pg log length =  4
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 osd.5 pg_epoch: 24 pg[2.d( v 24'4 (0'0,24'4] local-lis/les=21/22 n=2 ec=21/21 lis/c 21/21 les/c/f 22/22/0 21/21/21) [5,1] r=0 lpr=21 luod=24'3 lua=24'3 crt=24'4 lcod 24'2 mlcod 24'2 active+clean] append_log transaction_applied = 1
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 trim proposed trim_to = 0'0
2019-06-22T11:19:10.964+0000 7fcbe4c17700  6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 24'4, trimmed: , trimmed_dups: , clear_divergent_priors: 0
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 bluestore(/var/lib/ceph/osd/ceph-5) queue_transactions ch 0xfe41a00 2.d_head
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5) _txc_create osr 0xfed8120 = 0xfef7c00 seq 9
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5).collection(2.d_head 0xfe41a00) get_onode oid #2:b022980f:::smithi04715146-47:head# key 0x7f8000000000000002b022980f21736d'ithi04715146-47!='0xfffffffffffffffeffffffffffffffff'o'
*2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5).collection(2.d_head 0xfe41a00)  r -2 v.len 0*
2019-06-22T11:19:10.964+0000 7fcbe4c17700 15 bluestore(/var/lib/ceph/osd/ceph-5) _touch 2.d_head #2:b022980f:::smithi04715146-47:head#
2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5) _assign_nid 1100
2019-06-22T11:19:10.964+0000 7fcbe4c17700 10 bluestore(/var/lib/ceph/osd/ceph-5) _touch 2.d_head #2:b022980f:::smithi04715146-47:head# = 0

The above message is the firstly create for 2:b022980f:::smithi04715146-47:head. In Using OP_CREATE, it shuold not print "*2019-06-22T11:19:10.964+0000 7fcbe4c17700 20 bluestore(/var/lib/ceph/osd/ceph-5).collection(2.d_head 0xfe41a00) r -2 v.len 0*". I don't know why OP_CREATE don't used.
Sage: could you give some suggestions to debug this? Thanks.

#2 Updated by Greg Farnum over 4 years ago

  • Priority changed from Urgent to High

Ping Sage, it sounds like you know what the issue is?

jianping, does your comment have anything to do with this starting bug? It seems different.

#3 Updated by Greg Farnum over 4 years ago

#4 Updated by Greg Farnum over 4 years ago

  • Related to Bug #22624: filestore: 3180: FAILED assert(0 == "unexpected error"): error (2) No such file or directory not handled on operation 0x55e1ce80443c (21888.1.0, or op 0, counting from 0) added

#5 Updated by David Zafman over 4 years ago

I see this as a hang in running standalone tests in particular qa/standalone/osd/divergent-priors.sh. The test hangs caused by OSD crash:

   -13> 2019-08-28T17:51:26.386+0000 7f6f2e9f3700 -1 bluestore(td/divergent-priors/1) _txc_add_transaction error (2) No such file or directory not handled on operation 15 (op 2, counting from 0)
   -12> 2019-08-28T17:51:26.386+0000 7f6f2e9f3700 -1 bluestore(td/divergent-priors/1) unexpected error code
   -11> 2019-08-28T17:51:26.386+0000 7f6f2e9f3700  0 _dump_transaction transaction dump:
{
    "ops": [
        {
            "op_num": 0,
            "op_name": "rmattrs",
            "collection": "1.0_head",
            "oid": "#1:d858c035:::existing_1:head#" 
        },
        {
            "op_num": 1,
            "op_name": "truncate",
            "collection": "1.0_head",
            "oid": "#1:d858c035:::existing_1:head#",
            "offset": 89
        },
        {
            "op_num": 2,
            "op_name": "setattrs",
            "collection": "1.0_head",
            "oid": "#1:d858c035:::existing_1:head#",
            "attr_lens": {
                "_": 265,
                "snapset": 35
            }
        },
        {
            "op_num": 3,
            "op_name": "omap_rmkeys",
            "collection": "1.0_head",
            "oid": "#1:00000000::::head#",
            "attrs": [
                "missing/0000000000000001.B1A130CA.head.existing%u1.." 
            ]
        },
        {
            "op_num": 4,
            "op_name": "omap_setkeys",
            "collection": "1.0_head",
            "oid": "#1:00000000::::head#",
            "attr_lens": {
                "_info": 1041
            }
        }
    ]
}

   -10> 2019-08-28T17:51:26.406+0000 7f6f3f809700  5 prioritycache tune_memory target: 4294967296 mapped: 57155584 unmapped: 31326208 heap: 88481792 old mem: 2845352098 new mem: 2845414984
    -9> 2019-08-28T17:51:26.402+0000 7f6f46a23700 10 osd.1 43 tick_without_osd_lock
    -8> 2019-08-28T17:51:26.406+0000 7f6f46a23700 20 osd.1 43 can_inc_scrubs_pending 0 -> 1 (max 1, active 0)
    -7> 2019-08-28T17:51:26.406+0000 7f6f46a23700 20 osd.1 43 sched_scrub not scheduling scrubs due to active recovery
    -6> 2019-08-28T17:51:26.406+0000 7f6f46a23700 10 osd.1 43 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec
    -5> 2019-08-28T17:51:26.406+0000 7f6f46a23700 20 osd.1 43 promote_throttle_recalibrate  new_prob 1000
    -4> 2019-08-28T17:51:26.406+0000 7f6f46a23700 10 osd.1 43 promote_throttle_recalibrate  actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000
    -3> 2019-08-28T17:51:26.410+0000 7f6f46a23700 20 osd.1 43 send_beacon sending
    -2> 2019-08-28T17:51:26.410+0000 7f6f46a23700 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7115/0
    -1> 2019-08-28T17:51:26.418+0000 7f6f2e9f3700 -1 /home/dzafman/ceph/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)' thread 7f6f2e9f3700 time 2019-08-28T17:51:26.392795+0000
/home/dzafman/ceph/src/os/bluestore/BlueStore.cc: 11864: ceph_abort_msg("unexpected error")

 ceph version v15.0.0-4215-g38ecc4e92c (38ecc4e92c8982763a7073503c8f5740cd712fec) octopus (dev)
 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xf6) [0x5640293b5cdc]
 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x1f6d) [0x564029166277]
 3: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x5e8) [0x5640291637e4]
 4: (ObjectStore::queue_transaction(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ceph::os::Transaction&&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x96) [0x564028981508]
 5: (PrimaryLogPG::queue_transaction(ceph::os::Transaction&&, boost::intrusive_ptr<OpRequest>)+0x78) [0x564028bee9dc]
 6: (ReplicatedBackend::_do_pull_response(boost::intrusive_ptr<OpRequest>)+0x6d8) [0x564028f52b98]
 7: (ReplicatedBackend::do_push(boost::intrusive_ptr<OpRequest>)+0x56) [0x564028f66ede]
 8: (ReplicatedBackend::_handle_message(boost::intrusive_ptr<OpRequest>)+0x16c) [0x564028f4d83c]
 9: (PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)+0x78) [0x564028cb59a0]
 10: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0xc76) [0x564028b46eb2]
 11: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x390) [0x56402895e2e4]
 12: (PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x6e) [0x564028e00180]
 13: (OpQueueItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x4b) [0x56402898b4fb]
 14: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x3631) [0x56402896bd39]
 15: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x59c) [0x5640293a4f70]
 16: (ShardedThreadPool::WorkThreadSharded::entry()+0x25) [0x5640293a6953]
 17: (Thread::entry_wrapper()+0x78) [0x564029391214]
 18: (Thread::_entry_func(void*)+0x18) [0x564029391192]
 19: (()+0x76db) [0x7f6f4eb3d6db]
 20: (clone()+0x3f) [0x7f6f4d88688f]

#6 Updated by Neha Ojha over 4 years ago

  • Priority changed from High to Urgent

#7 Updated by David Zafman over 4 years ago

#8 Updated by Kefu Chai over 4 years ago

/a/pdonnell-2019-09-14_22:40:03-rados-master-distro-basic-smithi/4307679/
/a/kchai-2019-09-15_15:37:26-rados-wip-kefu-testing-2019-09-15-1533-distro-basic-mira/4311113/

#9 Updated by David Zafman over 4 years ago

  • Related to Bug #42175: _txc_add_transaction error (2) No such file or directory not handled on operation 15 added

#10 Updated by Patrick Donnelly over 4 years ago

  • Status changed from 12 to New

#11 Updated by Sage Weil about 4 years ago

  • Priority changed from Urgent to High

#12 Updated by Neha Ojha over 3 years ago

  • Status changed from New to Can't reproduce

Also available in: Atom PDF