Actions
Bug #58732
openquincy - kernel BUG at fs/ceph/inode.c:1376
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
02/21/2023
Affected Versions:
ceph-qa-suite:
fs
Crash signature (v1):
Crash signature (v2):
Description
Crash of the kernel right after a snap of the CephFS of the / directory (cephfs-shell 'snap create backup /')
Crash logs:
Feb 14 15:02:57 silae-rbx-str1 kernel: ------------[ cut here ]------------
Feb 14 15:02:57 silae-rbx-str1 kernel: kernel BUG at fs/ceph/inode.c:1376!
Feb 14 15:02:57 silae-rbx-str1 kernel: invalid opcode: 0000 [#1] SMP NOPTI
Feb 14 15:02:58 silae-rbx-str1 kernel: CPU: 9 PID: 2116367 Comm: kworker/9:0 Not tainted 5.10.0-20-amd64 #1 Debian 5.10.158-2
Feb 14 15:02:58 silae-rbx-str1 kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X470D4U2-2T, BIOS L4.03G 12/12/2012
Feb 14 15:02:58 silae-rbx-str1 kernel: Workqueue: ceph-msgr ceph_con_workfn [libceph]
Feb 14 15:02:58 silae-rbx-str1 kernel: RIP: 0010:ceph_fill_trace+0x912/0xac0 [ceph]
Feb 14 15:02:58 silae-rbx-str1 kernel: Code: c1 e8 92 cc 86 dc e9 41 fe ff ff 0f 0b 0f 0b 4c 89 fa 48 c7 c6 dd 7d 2a c1 48 c7 c7 f8 02 2c c1 e8 73 cc 86 dc e9 32 fe ff ff <0f> 0b 49 8b 56 40 48 c7 c6 18 b4 29 c1 44 88 54 24 08 48 c7 c7 68
Feb 14 15:02:58 silae-rbx-str1 kernel: RSP: 0018:ffff9f3683433c68 EFLAGS: 00010212
Feb 14 15:02:58 silae-rbx-str1 kernel: RAX: ffff8c5085551819 RBX: ffff8c53e3078360 RCX: ffff8c5085551800
Feb 14 15:02:58 silae-rbx-str1 kernel: RDX: 0000000000000022 RSI: 0000000000000001 RDI: ffff8c53b3dc3d68
Feb 14 15:02:58 silae-rbx-str1 kernel: RBP: ffff8c53b3dc3ce0 R08: ffffffff9ec06ed0 R09: ffff9f3683433c00
Feb 14 15:02:58 silae-rbx-str1 kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
Feb 14 15:02:58 silae-rbx-str1 kernel: R13: ffff8c51afbbe000 R14: ffff8c526f07fb00 R15: ffff8c51a4290e40
Feb 14 15:02:58 silae-rbx-str1 kernel: FS: 0000000000000000(0000) GS:ffff8c579ec40000(0000) knlGS:0000000000000000
Feb 14 15:02:58 silae-rbx-str1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 14 15:02:58 silae-rbx-str1 kernel: CR2: 00007fe7b03e2a20 CR3: 00000001060a6000 CR4: 0000000000350ee0
Feb 14 15:02:58 silae-rbx-str1 kernel: Call Trace:
Feb 14 15:02:58 silae-rbx-str1 kernel: dispatch+0x7da/0x1540 [ceph]
Feb 14 15:02:58 silae-rbx-str1 kernel: ceph_con_workfn+0x1a5f/0x2850 [libceph]
Feb 14 15:02:58 silae-rbx-str1 kernel: ? __switch_to_asm+0x3a/0x60
Feb 14 15:02:58 silae-rbx-str1 kernel: ? __switch_to+0x114/0x460
Feb 14 15:02:58 silae-rbx-str1 kernel: process_one_work+0x1b6/0x350
Feb 14 15:02:58 silae-rbx-str1 kernel: worker_thread+0x53/0x3e0
Feb 14 15:02:58 silae-rbx-str1 kernel: ? process_one_work+0x350/0x350
Feb 14 15:02:58 silae-rbx-str1 kernel: kthread+0x11b/0x140
Feb 14 15:02:58 silae-rbx-str1 kernel: ? __kthread_bind_mask+0x60/0x60
Feb 14 15:02:58 silae-rbx-str1 kernel: ret_from_fork+0x22/0x30
Feb 14 15:02:58 silae-rbx-str1 kernel: Modules linked in: cbc ceph libceph rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc nfs_ssc fscache dm_mod overlay xt_state bonding ipmi_ssif nft_chain_nat xt_MASQUERADE nf_nat nft_counter xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables nfnetlink amd64_edac_mod edac_mce_amd kvm_amd snd_hda_intel snd_intel_dspcfg soundwire_intel soundwire_generic_allocation kvm snd_soc_core snd_compress soundwire_cadence snd_hda_codec irqbypass snd_hda_core snd_hwdep ghash_clmulni_intel soundwire_bus acpi_ipmi nls_ascii nls_cp437 aesni_intel drm_vram_helper snd_pcm libaes drm_ttm_helper vfat crypto_simd fat tpm_crb ttm cdc_ether snd_timer ipmi_si tpm_tis sp5100_tco drm_kms_helper snd cryptd glue_helper tpm_tis_core cec rapl isofs efi_pstore wmi_bmof usbnet k10temp watchdog i2c_algo_bit soundcore ipmi_devintf evdev ccp joydev mii tpm sg ipmi_msghandler rng_core button acpi_cpufreq fuse drm configfs efivarfs ip_tables x_tables autofs4
Feb 14 15:02:58 silae-rbx-str1 kernel: ext4 crc16 mbcache jbd2 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx sr_mod xor cdrom raid6_pq libcrc32c crc32c_generic raid0 multipath linear hid_generic uas raid1 usbhid hid md_mod usb_storage sd_mod ahci nvme libahci ixgbe xhci_pci xfrm_algo libata nvme_core dca mdio_devres xhci_hcd libphy t10_pi crc_t10dif crct10dif_generic crc32_pclmul ptp crct10dif_pclmul i2c_piix4 crc32c_intel usbcore scsi_mod crct10dif_common pps_core wmi usb_common mdio gpio_amdpt gpio_generic
Feb 14 15:02:58 silae-rbx-str1 kernel: ---[ end trace b4fcb18811dcb093 ]---
Feb 14 15:02:58 silae-rbx-str1 kernel: RIP: 0010:ceph_fill_trace+0x912/0xac0 [ceph]
Feb 14 15:02:58 silae-rbx-str1 kernel: Code: c1 e8 92 cc 86 dc e9 41 fe ff ff 0f 0b 0f 0b 4c 89 fa 48 c7 c6 dd 7d 2a c1 48 c7 c7 f8 02 2c c1 e8 73 cc 86 dc e9 32 fe ff ff <0f> 0b 49 8b 56 40 48 c7 c6 18 b4 29 c1 44 88 54 24 08 48 c7 c7 68
Feb 14 15:02:58 silae-rbx-str1 kernel: RSP: 0018:ffff9f3683433c68 EFLAGS: 00010212
Feb 14 15:02:58 silae-rbx-str1 kernel: RAX: ffff8c5085551819 RBX: ffff8c53e3078360 RCX: ffff8c5085551800
Feb 14 15:02:58 silae-rbx-str1 kernel: RDX: 0000000000000022 RSI: 0000000000000001 RDI: ffff8c53b3dc3d68
Feb 14 15:02:58 silae-rbx-str1 kernel: RBP: ffff8c53b3dc3ce0 R08: ffffffff9ec06ed0 R09: ffff9f3683433c00
Feb 14 15:02:58 silae-rbx-str1 kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
Feb 14 15:02:58 silae-rbx-str1 kernel: R13: ffff8c51afbbe000 R14: ffff8c526f07fb00 R15: ffff8c51a4290e40
Feb 14 15:02:58 silae-rbx-str1 kernel: FS: 0000000000000000(0000) GS:ffff8c579ec40000(0000) knlGS:0000000000000000
Feb 14 15:02:58 silae-rbx-str1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 14 15:02:58 silae-rbx-str1 kernel: CR2: 00007fe7b03e2a20 CR3: 00000001060a6000 CR4: 0000000000350ee0
Updated by Xiubo Li about 1 year ago
- Status changed from New to Need More Info
- Assignee set to Xiubo Li
Antoine,
Thanks for reporting this.
BTW, I am not familiar with the Debian community, do you know where could I get the Debian kernel source code ?
And could you reproduce this ? If so could you provide the detail steps ?
Thanks,
Updated by Xiubo Li about 1 year ago
- Project changed from CephFS to Linux kernel client
Updated by Antoine Dheygers about 1 year ago
You can get it from a Debian by using the
apt-get source linux-image-5.10.0-21-amd64I only had this bug once and did not manage to reproduce it.
Here are the two commands that where played just before the failure (i ded not played those by hand so they are played very fast one after the other)
cephfs-shell 'snap create backup /'
tar -cvzf - {{ path to my mounting point }}/.snap/backup/{{ Some folder }} | gpg -e -r '' > /{{ some destination file }}.tar.gz.gpg
Actions