Project

General

Profile

Actions

Bug #40862

closed

kclient: crashed after evicted twice

Added by Erqi Chen almost 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
fs/ceph
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

After kernel mount client evicted twice due to not responded to mds cap revoke, it's crashed.
kernel version is 3.10 with latest patch ported.

vmcore-dmesg.txt
[4058680.824852] libceph: mds0 10.48.148.42:6800 socket closed (con state OPEN)
[4058683.098344] libceph: mds0 10.48.148.42:6800 connection reset
[4058683.098634] libceph: reset on mds0
[4058683.098635] ceph: mds0 closed our session
[4058683.098637] ceph: mds0 reconnect start
[4058683.106593] ceph: mds0 reconnect denied
[4058683.106608] ceph: dropping dirty XxFw state for ffff88262e694b90 1101208305803
[4058683.107115] libceph: mds0 10.48.148.42:6800 socket closed (con state NEGOTIATING)
[4058710.463132] ceph: get_quota_realm: ino (1001a35bfb1.fffffffffffffffe) null i_snap_realm
[4058710.463672] ceph: get_quota_realm: ino (1001a35bfb1.fffffffffffffffe) null i_snap_realm
[4058717.231387] libceph: mds0 10.48.148.42:6800 socket closed (con state OPEN)
[4058717.233585] libceph: mds0 10.48.148.42:6800 connection reset
[4058717.233915] libceph: reset on mds0
[4058717.233917] ceph: mds0 closed our session
[4058717.233918] ceph: mds0 reconnect start
[4058717.236841] ceph: mds0 reconnect denied
[4058717.236846] ceph: dropping dirty+flushing XxFw state for ffff88262e6bf280 1101208305806
[4058717.236849] ceph: dropping dirty+flushing Fw state for ffff882329bbd6b0 1101208305807
[4058717.237776] libceph: mds0 10.48.148.42:6800 socket closed (con state NEGOTIATING)
[4058749.968287] ------------[ cut here ]------------
[4058749.968581] kernel BUG at fs/ceph/addr.c:82!
[4058749.968807] invalid opcode: 0000 [#1] SMP
[4058749.969011] Modules linked in: kpatch_ceph_request(OEK) kpatch(OE) ipmi_watchdog ipmi_poweroff nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) tcp_diag inet_diag veth ceph libceph dns_resolver br_netfilter bridge stp llc overlay(T) nf_conntrack_netlink xt_statistic xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 xt_mark xt_comment xt_conntrack ip_set nfnetlink xt_addrtype iptable_nat nf_nat_ipv4 nf_nat iptable_filter nf_conntrack_ipv4 nf_defrag_ipv4 vxlan ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip6_udp_tunnel nf_conntrack udp_tunnel libcrc32c binfmt_misc sb_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support mei_me mei lpc_ich pcspkr
[4058749.970945] i2c_i801 sg ioatdma ipmi_si shpchp ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci libata ixgbe igb crct10dif_pclmul crct10dif_common crc32c_intel mdio ptp pps_core i2c_algo_bit i2c_core dca megaraid_sas mxm_wmi wmi
[4058749.972736] CPU: 31 PID: 14668 Comm: python Tainted: P OE K------------ T 3.10.0-1.0.2.el7.x86_64 #1
[4058749.973248] Hardware name: Supermicro SYS-4028GR-TR2/X10DRG-O+-CPU, BIOS 2.0a 04/24/2017
[4058749.973775] task: ffff881f70ea0fd0 ti: ffff881fd3a40000 task.ti: ffff881fd3a40000
[4058749.974342] RIP: 0010:[<ffffffffc073687e>] [<ffffffffc073687e>] ceph_set_page_dirty+0x1ce/0x250 [ceph]
[4058749.974924] RSP: 0018:ffff881fd3a43b08 EFLAGS: 00010246
[4058749.975499] RAX: 002fffff00000039 RBX: ffffea0043e429c0 RCX: 0000000001000000
[4058749.976093] RDX: ffffffff81226bd0 RSI: 0000000000400000 RDI: ffffea0043e429c0
[4058749.976698] RBP: ffff881fd3a43b68 R08: 0000000000001000 R09: ffffea0043e429c0
[4058749.977311] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000001000
[4058749.977963] R13: ffff882329bbd800 R14: 0000000000000000 R15: 0000000000001000
[4058749.978587] FS: 00007f601ef33700(0000) GS:ffff881fffa40000(0000) knlGS:0000000000000000
[4058749.979233] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[4058749.979884] CR2: 00007f5ea4327928 CR3: 0000003fb0e5a000 CR4: 00000000003407e0
[4058749.980553] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[4058749.981235] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[4058749.981947] Stack:
[4058749.982629] 0000100029bbd800 0000000000001000 0000000000401000 ffff881fd3a43b38
[4058749.983350] ffffffff8117f712 ffffea0043e429c0 ffff881fd3a43b78 ffffea0043e429c0
[4058749.984080] 0000000000001000 ffff882329bbd6b0 0000000000000000 0000000000001000
[4058749.984823] Call Trace:
[4058749.985570] [<ffffffff8117f712>] ? wait_for_stable_page+0x12/0x40
[4058749.986393] [<ffffffff8117f687>] set_page_dirty+0x37/0x60
[4058749.987169] [<ffffffffc0736403>] ceph_write_end+0xd3/0x140 [ceph]
[4058749.987960] [<ffffffff81174599>] generic_file_buffered_write+0x189/0x2a0
[4058749.988757] [<ffffffffc07337ca>] ceph_aio_write+0xb0a/0xd40 [ceph]
[4058749.989562] [<ffffffff811833ee>] ? lru_cache_add+0xe/0x10
[4058749.990414] [<ffffffff81187c22>] ? putback_lru_page+0x52/0xb0
[4058749.991234] [<ffffffff811d88a7>] ? migrate_pages+0x2d7/0x7a0
[4058749.992067] [<ffffffff811d5f00>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[4058749.992919] [<ffffffff811a2909>] ? do_numa_page+0x159/0x1e0
[4058749.993766] [<ffffffff811f1c8d>] do_sync_write+0x8d/0xd0
[4058749.994631] [<ffffffff811f274d>] vfs_write+0xbd/0x1e0
[4058749.995453] [<ffffffff811f355f>] SyS_write+0x7f/0xe0
[4058749.996285] [<ffffffff81692f09>] system_call_fastpath+0x16/0x1b
[4058749.997102] Code: c0 e9 9f fe ff ff 4c 8b 47 10 49 8b 55 00 48 89 f9 48 c7 c6 f8 c5 75 c0 48 c7 c7 70 d0 76 c0 31 c0 e8 07 58 c0 c0 e9 6c fe ff ff <0f> 0b 0f 0b be 7c 00 00 00 48 c7 c7 33 51 76 c0 89 45 d0 e8 da
[4058749.998892] RIP [<ffffffffc073687e>] ceph_set_page_dirty+0x1ce/0x250 [ceph]
[4058749.999759] RSP <ffff881fd3a43b08>

Actions

Also available in: Atom PDF