Ceph » CephFS

Zheng, does this look familiar to you?

http://pulpito.ceph.com/teuthology-2016-03-09_17:10:02-knfs-jewel-testing-basic-smithi/49840/

2016-03-10T12:52:13.465994+00:00 smithi026 kernel: ===============================
2016-03-10T12:52:13.466007+00:00 smithi026 kernel: [ INFO: suspicious RCU usage. ]
2016-03-10T12:52:13.475445+00:00 smithi026 kernel: 4.5.0-rc3-ceph-00787-g062cc0a #1 Not tainted
2016-03-10T12:52:13.475465+00:00 smithi026 kernel: -------------------------------
2016-03-10T12:52:13.479632+00:00 smithi026 kernel: fs/ceph/inode.c:772 suspicious rcu_dereference_check() usage!
2016-03-10T12:52:13.486424+00:00 smithi026 kernel: #012other info that might help us debug this:
2016-03-10T12:52:13.500942+00:00 smithi026 kernel: #012rcu_scheduler_active = 1, debug_locks = 0
2016-03-10T12:52:13.500965+00:00 smithi026 kernel: 6 locks held by kworker/6:2/286:
2016-03-10T12:52:13.515666+00:00 smithi026 kernel: #0:  ("ceph-msgr"){.+.+.+}, at: [<ffffffff810c2ee5>] process_one_work+0x1e5/0x5e0
2016-03-10T12:52:13.525217+00:00 smithi026 kernel: #1:  ((&(&con->work)->work)){+.+.+.}, at: [<ffffffff810c2ee5>] process_one_work+0x1e5/0x5e0
2016-03-10T12:52:13.533998+00:00 smithi026 kernel: #2:  (&s->s_mutex){+.+...}, at: [<ffffffffa0556a46>] dispatch+0x1a56/0x1ba0 [ceph]
2016-03-10T12:52:13.543209+00:00 smithi026 kernel: #3:  (&mdsc->snap_rwsem){+.+...}, at: [<ffffffffa0555fac>] dispatch+0xfbc/0x1ba0 [ceph]
2016-03-10T12:52:13.552587+00:00 smithi026 kernel: #4:  (&req->r_fill_mutex){+.+...}, at: [<ffffffffa055600a>] dispatch+0x101a/0x1ba0 [ceph]
2016-03-10T12:52:13.565953+00:00 smithi026 kernel: #5:  (&(&ci->i_ceph_lock)->rlock){+.+...}, at: [<ffffffffa052cc06>] fill_inode.isra.18+0x1a6/0xde0 [ceph]
2016-03-10T12:52:13.565971+00:00 smithi026 kernel: #012stack backtrace:
2016-03-10T12:52:13.565987+00:00 smithi026 kernel: CPU: 6 PID: 286 Comm: kworker/6:2 Not tainted 4.5.0-rc3-ceph-00787-g062cc0a #1
2016-03-10T12:52:13.730222+00:00 smithi026 kernel: Hardware name: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 1.0c 09/07/2015
2016-03-10T12:52:13.730245+00:00 smithi026 kernel: Workqueue: ceph-msgr ceph_con_workfn [libceph]
2016-03-10T12:52:13.730255+00:00 smithi026 kernel: 0000000000000286 000000007a559750 ffff88085491b998 ffffffff813f8661
2016-03-10T12:52:13.730264+00:00 smithi026 kernel: ffff8800745ecd80 0000000000000001 ffff88085491b9c8 ffffffff810fcd37
2016-03-10T12:52:13.730272+00:00 smithi026 kernel: ffff88085646d218 0000000000000000 0000000000000000 ffff8807e1070528
2016-03-10T12:52:13.730280+00:00 smithi026 kernel: Call Trace:
2016-03-10T12:52:13.730288+00:00 smithi026 kernel: [<ffffffff813f8661>] dump_stack+0x85/0xc4
2016-03-10T12:52:13.730296+00:00 smithi026 kernel: [<ffffffff810fcd37>] lockdep_rcu_suspicious+0x107/0x110
2016-03-10T12:52:13.730304+00:00 smithi026 kernel: [<ffffffffa052cebc>] fill_inode.isra.18+0x45c/0xde0 [ceph]
2016-03-10T12:52:13.730315+00:00 smithi026 kernel: [<ffffffff81277af3>] ? unlock_new_inode+0x23/0x80
2016-03-10T12:52:13.730323+00:00 smithi026 kernel: [<ffffffffa052dd0c>] ceph_fill_trace+0x4cc/0xc10 [ceph]
2016-03-10T12:52:13.730331+00:00 smithi026 kernel: [<ffffffff810fdde3>] ? trace_hardirqs_on_caller+0x163/0x1d0
2016-03-10T12:52:13.730339+00:00 smithi026 kernel: [<ffffffff810fde5d>] ? trace_hardirqs_on+0xd/0x10
2016-03-10T12:52:13.730346+00:00 smithi026 kernel: [<ffffffffa055600a>] ? dispatch+0x101a/0x1ba0 [ceph]
2016-03-10T12:52:13.730354+00:00 smithi026 kernel: [<ffffffffa0556038>] dispatch+0x1048/0x1ba0 [ceph]
2016-03-10T12:52:13.730362+00:00 smithi026 kernel: [<ffffffff817d7c2e>] ? mutex_unlock+0xe/0x10
2016-03-10T12:52:13.730370+00:00 smithi026 kernel: [<ffffffffa0390659>] ceph_con_workfn+0x1629/0x2f00 [libceph]
2016-03-10T12:52:13.730378+00:00 smithi026 kernel: [<ffffffff810d2f83>] ? finish_task_switch+0x1c3/0x2e0
2016-03-10T12:52:13.730387+00:00 smithi026 kernel: [<ffffffff810c2ff1>] process_one_work+0x2f1/0x5e0
2016-03-10T12:52:13.730395+00:00 smithi026 kernel: [<ffffffff810c2ee5>] ? process_one_work+0x1e5/0x5e0
2016-03-10T12:52:13.730403+00:00 smithi026 kernel: [<ffffffff810c3587>] worker_thread+0x2a7/0x490
2016-03-10T12:52:13.730412+00:00 smithi026 kernel: [<ffffffff810c32e0>] ? process_one_work+0x5e0/0x5e0
2016-03-10T12:52:13.730420+00:00 smithi026 kernel: [<ffffffff810c9eca>] kthread+0x10a/0x120
2016-03-10T12:52:13.730428+00:00 smithi026 kernel: [<ffffffff810c9dc0>] ? kthread_stop+0x220/0x220
2016-03-10T12:52:13.730436+00:00 smithi026 kernel: [<ffffffff817da81f>] ret_from_fork+0x3f/0x70
2016-03-10T12:52:13.730444+00:00 smithi026 kernel: [<ffffffff810c9dc0>] ? kthread_stop+0x220/0x220