Actions
Bug #3342
closedcon_work crash on bad con pointer or mutex (kclient ffsb + thrashing)
Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
libceph
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
[0]kdb> rd ax: 0000000000000000 bx: ffff880223db6830 cx: 0000000000000000 dx: ffff880223db69f0 si: ffff880223db69f0 di: 0000000000000090 bp: ffff88020d7efc40 sp: ffff88020d7efb50 r8: 2222222222222222 r9: 2222222222222222 r10: 2222222222222222 r11: 0000000000000000 r12: 0000000000000000 r13: 0000000000000000 r14: ffff880223db69b0 r15: ffff880223db6c20 ip: ffffffffa047e7ed flags: 00010293 cs: 00000010 ss: 00000018 ds: 00000018 es: 00000018 fs: 00000018 gs: 00000018 [0]kdb> bt Stack traceback for pid 62 0xffff88020d793ec0 62 2 1 0 R 0xffff88020d794320 *kworker/0:2 ffff88020d7efb50 0000000000000018 0000000000000000 0000000000000246 0000000000000000 0000000000000000 0000000000000000 ffff880223db6a18 0000000000000000 ffff880223db69b0 ffff880223db69b0 0000000000000246 Call Trace: [<ffffffff8163274c>] ? mutex_lock_nested+0x25c/0x320 [<ffffffffa047eedf>] ? con_work+0x3f/0x16b0 [libceph] [<ffffffffa047efd3>] ? con_work+0x133/0x16b0 [libceph] [<ffffffff810848a8>] ? finish_task_switch+0x48/0x110 [<ffffffff81071776>] ? process_one_work+0x136/0x5f0 [<ffffffff810717e0>] ? process_one_work+0x1a0/0x5f0 [<ffffffff81071776>] ? process_one_work+0x136/0x5f0 [<ffffffffa047eea0>] ? try_read+0x1920/0x1920 [libceph] [<ffffffff810735fd>] ? worker_thread+0x18d/0x4f0 [<ffffffff81073470>] ? manage_workers+0x320/0x320 [<ffffffff810791fe>] ? kthread+0xae/0xc0 [<ffffffff810b393d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff8163f3c4>] ? kernel_thread_helper+0x4/0x10 [<ffffffff816360b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81079150>] ? flush_kthread_work+0x1a0/0x1a0 [<ffffffff8163f3c0>] ? gs_change+0x13/0x13
that was
kernel: &id001 kdb: true sha1: 6285bc231277419255f3498d3eb5ddc9f8e7fe79 nuke-on-error: true overrides: ceph: conf: mds: debug mds: 1/20 fs: btrfs log-whitelist: - slow request sha1: 2528b5ee105b16352c91af064af5c0b5a7d45d7c s3tests: branch: testing workunit: sha1: 2528b5ee105b16352c91af064af5c0b5a7d45d7c roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mds.a - osd.3 - osd.4 - osd.5 - - client.0 targets: ubuntu@plana72.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCwa+tIQskKvsvQ1/J9QOYGeunl0M+mIAvxqGeKFawdMKoXCNNb5YLRL6Fr9y/smJVCBIGugSb8LBrjVWF14gUOXAk1j16qJ6rsvIF8176L8tsIMxhXx3dw7dCacaGEHCrmK6KO8YhFm6ky1qftGCu7amfyzeJTj8kvrY4tl1ifwH0sv2M0iEzLXx63xn2UpAMAIvo5v+eqPjo+1w2PFXe+r2dViDN5wjlwQTKOXAPFRewmDy2o6K/rW/iRRg0tHLO4atCZr5Y8XlXYQkIBQVlXrL9CRwe8rmrxHyH8wnFqYcvpzLFk1oKw93sFNauy4mCxVntvIm3WT1S8nQuEE6HJ ubuntu@plana73.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxoJnvRI1V0OJuQI9SosOedC7mj9O627LjoQWPKilJiBbHduPe1byBaKrgwTeEghl43VNf+EBs1+MwVH7zlDolnwN4tAlW9bRpC2SzURJfhZskp2CSQY3l8ca7a5f0J3hdOhx47oSSapN7O2cqmPzwlL/+MrFKGi+ITT613nUtzCjduZRPdhjyqZ0cQWeb0p1neDw5hbDBKd+HAH+ek/E6DK2PaqN6YAtmIgP76q0fQ85Omd0oDlmGXpKe3jlxlPT0W/5KD1+mpobPsh/EF2qar7IG/WqHHJ6NZAcXbdZ4KiMf9erP+Pk4KkD5SJ+e3GF7OEOwXtahKIIR1An4P2GD ubuntu@plana74.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCbl3n0OItF2T7uE/9SNA9YV+pq3nFmmr3EBjf2aMo7Bkyy6e9MC3kaK64vzlZxppbpxEhbrgWo7GsVHxphZqwtMt5Ek3RDbLUpQ8dPckbhOZWBhoZe9S8uCynDIZaF3Yq/b1//OBufSdM0TRYK4Vf4rtQnpyWPEn48HaairNhQvKCxY6XUImZc7Lh4CIaMkT0yyZkBlksJeAkTs4UgxhkSoSc9IAlAxwwPK5JpRL7GOowT+hsqYCVJM5+NVqIOGS66Vam/m9xHqecQtfYIEFoVFZiF/xi+b2kstxJzD39UTNP/XB5J2CtJOujW9J1jsdPKSz1L9XoDIkAFHzQjdhyx tasks: - internal.lock_machines: 3 - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.check_conflict: null - kernel: *id001 - internal.base: null - internal.archive: null - internal.coredump: null - internal.syslog: null - internal.timer: null - chef: null - clock: null - ceph: log-whitelist: - wrongly marked me down - objects unfound and apparently lost - thrashosds: null - kclient: null - workunit: clients: all: - suites/ffsb.sh
ubuntu@teuthology:/a/teuthology-2012-10-17_02:00:05-regression-testing-master-basic/2759
Files
Actions