Project

General

Profile

Actions

Bug #44695

open

client: enable fscache will hit "CacheFiles: Assertion failed" with large size read

Added by Xiubo Li about 4 years ago. Updated about 4 years ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Crash signature (v1):
Crash signature (v2):

Description

<7>[  333.227204] ceph:  page/inode not in cache
<7>[  333.571182] ceph:  page/inode not in cache
<7>[  333.741097] ceph:  page/inode not in cache
<7>[  337.977233] ceph:  all-page read submitted
<7>[  339.649714] ceph:  all-page read submitted
<3>[  339.654882] CacheFiles: 
<3>[  339.654952] CacheFiles: Assertion failed
<4>[  339.655124] ------------[ cut here ]------------
<2>[  339.655126] kernel BUG at fs/cachefiles/rdwr.c:434!
<4>[  339.655225] invalid opcode: 0000 [#1] SMP KASAN PTI
<4>[  339.655309] CPU: 0 PID: 3004 Comm: python2 Tainted: G            E     5.6.0-rc6+ #77
<4>[  339.655436] Hardware name: Red Hat RHEV Hypervisor, BIOS 1.11.0-2.el7 04/01/2014
<4>[  339.655578] RIP: 0010:cachefiles_read_or_alloc_page.cold.19+0xa2/0xcd [cachefiles]
<4>[  339.655706] Code: 65 08 41 be 97 ff ff ff e8 27 ef 04 00 e9 48 cc ff ff 48 c7 c7 c0 83 9c c0 e8 af a9 7c ce 48 c7 c7 00 84 9c c0 e8 a3 a9 7c ce <0f> 0b 48 c7 c7 c0 83 9c c0 e8 95 a9 7c ce 48 c7 c7 00 84 9c c0 e8
<4>[  339.656007] RSP: 0018:ffff88838110f610 EFLAGS: 00010282
<4>[  339.656095] RAX: 000000000000001c RBX: ffff88836e564600 RCX: 0000000000000001
<4>[  339.656210] RDX: 0000000000000000 RSI: 0000000000000007 RDI: ffff888389c22bd0
<4>[  339.656326] RBP: ffff88837b502e48 R08: ffffed107138457b R09: ffffed107138457b
<4>[  339.656441] R10: ffffed107138457a R11: ffff888389c22bd3 R12: ffffea000d2dffc0
<4>[  339.656563] R13: 0000000000000000 R14: ffff88835ef44700 R15: ffff88835ef44600
<4>[  339.656679] FS:  00007fbb546c3740(0000) GS:ffff888389c00000(0000) knlGS:0000000000000000
<4>[  339.656809] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[  339.656902] CR2: 0000561ecd9c5108 CR3: 000000036c20a000 CR4: 00000000000406f0
<4>[  339.657020] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[  339.657136] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
<4>[  339.657251] Call Trace:
<4>[  339.657322]  ? fscache_run_op.isra.10+0x80/0x150 [fscache]
<4>[  339.657421]  ? cachefiles_read_copier+0x920/0x920 [cachefiles]
<4>[  339.657529]  ? fscache_wait_for_operation_activation+0xe2/0x310 [fscache]
<4>[  339.657659]  __fscache_read_or_alloc_page+0x3d2/0x520 [fscache]
<4>[  339.657797]  ceph_readpage_from_fscache+0x80/0x110 [ceph]
<4>[  339.657915]  ceph_do_readpage+0x1f3/0x660 [ceph]
<4>[  339.657997]  ? scan_shadow_nodes+0x50/0x50
<4>[  339.658094]  ? writepages_finish+0x700/0x700 [ceph]
<4>[  339.658178]  ? add_to_page_cache_lru+0x10e/0x190
<4>[  339.658255]  ? add_to_page_cache_locked+0x10/0x10
<4>[  339.658361]  ceph_readpage+0x12/0x40 [ceph]
<4>[  339.658433]  generic_file_buffered_read+0xa4e/0x1160
<4>[  339.658517]  ? read_cache_page_gfp+0x20/0x20
<4>[  339.658599]  ? down_read+0xe0/0x190
<4>[  339.658661]  ? unaccount_page_cache_page+0x2f0/0x2f0
<4>[  339.658743]  ? generic_file_read_iter+0x3b/0x220
<4>[  339.658846]  ceph_read_iter+0x321/0x1430 [ceph]
<4>[  339.658923]  ? avc_has_perm_noaudit+0x147/0x200
<4>[  339.659027]  ? ceph_direct_read_write+0xee0/0xee0 [ceph]
<4>[  339.659115]  ? perf_event_fork+0x20/0x20
<4>[  339.659182]  ? vma_wants_writenotify+0x6f/0x240
<4>[  339.659257]  ? __ia32_sys_mmap_pgoff+0x80/0x80
<4>[  339.659334]  ? __rb_insert_augmented+0x34/0x3f0
<4>[  339.662451]  ? vm_get_page_prot+0x60/0x60
<4>[  339.665560]  ? vma_set_page_prot+0xe5/0x100
<4>[  339.668668]  ? mmap_region+0x370/0xa60
<4>[  339.671773]  ? userfaultfd_unmap_complete+0x89/0x1e0
<4>[  339.674900]  ? new_sync_read+0x286/0x3b0
<4>[  339.678011]  new_sync_read+0x286/0x3b0
<4>[  339.681102]  ? __ia32_sys_llseek+0x1d0/0x1d0
<4>[  339.684196]  ? __fsnotify_parent+0x8d/0x1f0
<4>[  339.687242]  ? fsnotify+0x5c5/0x600
<4>[  339.690224]  ? __fsnotify_inode_delete+0x20/0x20
<4>[  339.693188]  ? avc_policy_seqno+0x22/0x30
<4>[  339.696043]  ? security_file_permission+0xe1/0x180
<4>[  339.698821]  vfs_read+0xaf/0x1b0
<4>[  339.701518]  ksys_read+0xb4/0x150
<4>[  339.704177]  ? kernel_write+0xb0/0xb0
<4>[  339.706822]  do_syscall_64+0x73/0x260
<4>[  339.709427]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
<4>[  339.712055] RIP: 0033:0x7fbb534eb950
<4>[  339.714661] Code: 0b 31 c0 48 83 c4 08 e9 be fe ff ff 48 8d 3d 87 c2 09 00 e8 b2 8b 02 00 66 90 83 3d 4d d6 2d 00 00 75 10 b8 00 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 4e cc 01 00 48 89 04 24
<4>[  339.720166] RSP: 002b:00007fffb8506c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
<4>[  339.722950] RAX: ffffffffffffffda RBX: 0000000001408a20 RCX: 00007fbb534eb950
<4>[  339.725767] RDX: 00000000c0000000 RSI: 00007fba0661e034 RDI: 0000000000000004
<4>[  339.728630] RBP: 00007fbb537c0380 R08: ffffffffffffffff R09: 0000000000000000
<4>[  339.731452] R10: 0000000000000022 R11: 0000000000000246 R12: 00000000c0001000
<4>[  339.734272] R13: 0000000000000d68 R14: 00007fbb537bf840 R15: 00007fba0661e034
<4>[  339.737084] Modules linked in: ceph(E) libceph dns_resolver tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle xt_MASQUERADE iptable_nat nf_nat xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iscsi_target_mod target_core_user target_core_mod uio iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi rfkill cachefiles fscache sunrpc intel_rapl_msr intel_rapl_common crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd sg virtio_rng i2c_piix4 virtio_balloon cryptd pcspkr glue_helper dm_multipath binfmt_misc ip_tables xfs libcrc32c sd_mod t10_pi crc_t10dif crct10dif_generic virtio_net net_failover virtio_console failover virtio_scsi sr_mod cdrom ata_generic pata_acpi qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm ata_piix serio_raw libata crct10dif_pclmul crct10dif_common crc32c_intel
<4>[  339.737183]  virtio_pci virtio_ring i2c_core virtio floppy dm_mirror dm_region_hash dm_log dm_mod
<4>[  339.767456] ---[ end trace df142463377d0492 ]---
<4>[  339.771239] RIP: 0010:cachefiles_read_or_alloc_page.cold.19+0xa2/0xcd [cachefiles]
<4>[  339.775133] Code: 65 08 41 be 97 ff ff ff e8 27 ef 04 00 e9 48 cc ff ff 48 c7 c7 c0 83 9c c0 e8 af a9 7c ce 48 c7 c7 00 84 9c c0 e8 a3 a9 7c ce <0f> 0b 48 c7 c7 c0 83 9c c0 e8 95 a9 7c ce 48 c7 c7 00 84 9c c0 e8
<4>[  339.783189] RSP: 0018:ffff88838110f610 EFLAGS: 00010282
<4>[  339.787257] RAX: 000000000000001c RBX: ffff88836e564600 RCX: 0000000000000001
<4>[  339.791384] RDX: 0000000000000000 RSI: 0000000000000007 RDI: ffff888389c22bd0
<4>[  339.795514] RBP: ffff88837b502e48 R08: ffffed107138457b R09: ffffed107138457b
<4>[  339.799667] R10: ffffed107138457a R11: ffff888389c22bd3 R12: ffffea000d2dffc0
<4>[  339.803818] R13: 0000000000000000 R14: ffff88835ef44700 R15: ffff88835ef44600
<4>[  339.807986] FS:  00007fbb546c3740(0000) GS:ffff888389c00000(0000) knlGS:0000000000000000
<4>[  339.812219] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[  339.816474] CR2: 0000561ecd9c5108 CR3: 000000036c20a000 CR4: 00000000000406f0
<4>[  339.820811] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[  339.825135] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Actions #1

Updated by Xiubo Li about 4 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Xiubo Li about 4 years ago

The fixing v1 patch, which under reviewing: https://lkml.org/lkml/2020/3/20/399

Actions #3

Updated by Xiubo Li about 4 years ago

  • Status changed from In Progress to Fix Under Review
Actions

Also available in: Atom PDF