Bug #36667
Updated by Patrick Donnelly over 5 years ago
i deploy a cephfs and the used the vdbench tool to wirte data in cephfs mount point,after a while osd appears down. and i manually restart osd and the following log appears: <pre> 2018-11-01 11:01:50.482203 b6867000 5 osd.10 pg_epoch: 975 pg[2.25( v 954'2580 (0'0,954'2580] local-les=959 n=3 ec=66 les/c/f 959/959/0 975/975/714) [9,5,10]/[9,5] r=-1 lpr=0 pi=708-974/40 crt=954'2580 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset -8> 2018-11-01 11:01:50.483487 b6867000 5 osd.10 pg_epoch: 975 pg[0.10b(unlocked)] enter Initial -7> 2018-11-01 11:01:50.520894 b6867000 5 osd.10 pg_epoch: 975 pg[0.10b( v 937'6036 (150'3000,937'6036] local-les=959 n=798 ec=77 les/c/f 959/959/0 975/975/736) [6,10]/[6] r=-1 lpr=0 pi=721-974/19 crt=937'6036 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.037408 0 0.000000 -6> 2018-11-01 11:01:50.520972 b6867000 5 osd.10 pg_epoch: 975 pg[0.10b( v 937'6036 (150'3000,937'6036] local-les=959 n=798 ec=77 les/c/f 959/959/0 975/975/736) [6,10]/[6] r=-1 lpr=0 pi=721-974/19 crt=937'6036 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset -5> 2018-11-01 11:01:50.522519 b6867000 5 osd.10 pg_epoch: 975 pg[1.15b(unlocked)] enter Initial -4> 2018-11-01 11:01:50.560698 b6867000 5 osd.10 pg_epoch: 975 pg[1.15b( v 959'27450 (873'24391,959'27450] local-les=959 n=4307 ec=63 les/c/f 959/959/0 975/975/973) [10,1]/[1] r=-1 lpr=0 pi=958-974/8 crt=959'27450 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.038179 0 0.000000 -3> 2018-11-01 11:01:50.560773 b6867000 5 osd.10 pg_epoch: 975 pg[1.15b( v 959'27450 (873'24391,959'27450] local-les=959 n=4307 ec=63 les/c/f 959/959/0 975/975/973) [10,1]/[1] r=-1 lpr=0 pi=958-974/8 crt=959'27450 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset -2> 2018-11-01 11:01:50.562743 b6867000 5 osd.10 pg_epoch: 975 pg[2.5d(unlocked)] enter Initial -1> 2018-11-01 11:01:50.581308 af60eb10 -1 filestore(/var/lib/ceph/osd/ceph-10) object_map sync got (1) Operation not permitted 0> 2018-11-01 11:01:50.583579 af60eb10 -1 os/filestore/FileStore.cc: In function 'void FileStore::sync_entry()' thread af60eb10 time 2018-11-01 11:01:50.581359 os/filestore/FileStore.cc: 3796: FAILED assert(0 == "object_map sync returned error") </pre> --------------------- below is my ceph cluster node env: OS: Distributor ID: Ubuntu Description: Ubuntu 14.04.4 LTS Release: 14.04 Codename: trusty Kernel: Linux arm242 4.4.8-armada-17.02.2 who has a good ideas? thx.