Bug #14086: Ceph File System logging warning about ceph_set_page_dirty - Linux kernel client - Ceph

Actions

Copy link

Bug #14086

closed

Ceph File System logging warning about ceph_set_page_dirty

Added by Eric Eastman over 8 years ago. Updated over 8 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Community (user)

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Crash signature (v1):

Crash signature (v2):

Description

While testing Linux Target SCSI, LIO, with a Ceph File System file backstore, that is kernel mounted, I am seeing this warning on my LIO export gateway out of dmesg -T:

[Tue Dec 15 00:46:55 2015] ------------[ cut here ]------------
[Tue Dec 15 00:46:55 2015] WARNING: CPU: 0 PID: 1123421 at /home/kernel/COD/linux/fs/ceph/addr.c:125 ceph_set_page_dirty+0x230/0x240 [ceph]()
[Tue Dec 15 00:46:55 2015] Modules linked in: iptable_filter ip_tables x_tables xfs rbd iscsi_target_mod vhost_scsi tcm_qla2xxx ib_srpt tcm_fc tcm_usb_gadget tcm_loop target_core_file target_core_iblock target_core_pscsi target_core_user target_core_mod ipmi_devintf vhost qla2xxx ib_cm ib_sa ib_mad ib_core ib_addr libfc scsi_transport_fc libcomposite udc_core uio configfs ipmi_ssif ttm drm_kms_helper gpio_ich drm i2c_algo_bit fb_sys_fops coretemp syscopyarea ipmi_si sysfillrect ipmi_msghandler sysimgblt kvm acpi_power_meter 8250_fintek irqbypass hpilo shpchp input_leds serio_raw lpc_ich i7core_edac edac_core mac_hid ceph libceph libcrc32c fscache bonding lp parport mlx4_en vxlan ip6_udp_tunnel udp_tunnel ptp pps_core hid_generic usbhid hid hpsa mlx4_core psmouse bnx2 scsi_transport_sas fjes [last unloaded: target_core_mod]
[Tue Dec 15 00:46:55 2015] CPU: 0 PID: 1123421 Comm: iscsi_trx Tainted: G        W I     4.4.0-040400rc4-generic #201512061930
[Tue Dec 15 00:46:55 2015] Hardware name: HP ProLiant DL360 G6, BIOS P64 01/22/2015
[Tue Dec 15 00:46:55 2015]  0000000000000000 00000000fdc0ce43 ffff880bf38c38c0 ffffffff813c8ab4
[Tue Dec 15 00:46:55 2015]  0000000000000000 ffff880bf38c38f8 ffffffff8107d772 ffffea00127a8680
[Tue Dec 15 00:46:55 2015]  ffff8804e52c1448 ffff8804e52c15b0 ffff8804e52c10f0 0000000000000200
[Tue Dec 15 00:46:55 2015] Call Trace:
[Tue Dec 15 00:46:55 2015]  [<ffffffff813c8ab4>] dump_stack+0x44/0x60
[Tue Dec 15 00:46:55 2015]  [<ffffffff8107d772>] warn_slowpath_common+0x82/0xc0
[Tue Dec 15 00:46:55 2015]  [<ffffffff8107d8ba>] warn_slowpath_null+0x1a/0x20
[Tue Dec 15 00:46:55 2015]  [<ffffffffc01fadb0>] ceph_set_page_dirty+0x230/0x240 [ceph]
[Tue Dec 15 00:46:55 2015]  [<ffffffff81188770>] ? pagecache_get_page+0x150/0x1c0
[Tue Dec 15 00:46:55 2015]  [<ffffffffc01fe338>] ? ceph_pool_perm_check+0x48/0x700 [ceph]
[Tue Dec 15 00:46:55 2015]  [<ffffffff8119301d>] set_page_dirty+0x3d/0x70
[Tue Dec 15 00:46:55 2015]  [<ffffffffc01fcd7e>] ceph_write_end+0x5e/0x180 [ceph]
[Tue Dec 15 00:46:55 2015]  [<ffffffff813dc006>] ? iov_iter_copy_from_user_atomic+0x156/0x220
[Tue Dec 15 00:46:55 2015]  [<ffffffff81187bc4>] generic_perform_write+0x114/0x1c0
[Tue Dec 15 00:46:55 2015]  [<ffffffffc01f818a>] ceph_write_iter+0xf8a/0x1050 [ceph]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc0205983>] ? ceph_put_cap_refs+0x143/0x320 [ceph]
[Tue Dec 15 00:46:55 2015]  [<ffffffff810b10ba>] ? check_preempt_wakeup+0xfa/0x220
[Tue Dec 15 00:46:55 2015]  [<ffffffff811a7eec>] ? zone_statistics+0x7c/0xa0
[Tue Dec 15 00:46:55 2015]  [<ffffffff813dd2ee>] ? copy_page_to_iter+0x5e/0xa0
[Tue Dec 15 00:46:55 2015]  [<ffffffff816e5d22>] ? skb_copy_datagram_iter+0x122/0x250
[Tue Dec 15 00:46:55 2015]  [<ffffffff812053f6>] vfs_iter_write+0x76/0xc0
[Tue Dec 15 00:46:55 2015]  [<ffffffffc02cbf88>] fd_do_rw.isra.5+0xd8/0x1e0 [target_core_file]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc02cc155>] fd_execute_rw+0xc5/0x2a0 [target_core_file]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc04696f2>] sbc_execute_rw+0x22/0x30 [target_core_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc04681ef>] __target_execute_cmd+0x1f/0x70 [target_core_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc0468da5>] target_execute_cmd+0x195/0x2a0 [target_core_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc05db89a>] iscsit_execute_cmd+0x20a/0x270 [iscsi_target_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc05e4aea>] iscsit_sequence_cmd+0xda/0x190 [iscsi_target_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffffc05eafbd>] iscsi_target_rx_thread+0x51d/0xe30 [iscsi_target_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffff8101566c>] ? __switch_to+0x1dc/0x5a0
[Tue Dec 15 00:46:55 2015]  [<ffffffffc05eaaa0>] ? iscsi_target_tx_thread+0x1e0/0x1e0 [iscsi_target_mod]
[Tue Dec 15 00:46:55 2015]  [<ffffffff8109c8b8>] kthread+0xd8/0xf0
[Tue Dec 15 00:46:55 2015]  [<ffffffff8109c7e0>] ? kthread_create_on_node+0x1a0/0x1a0
[Tue Dec 15 00:46:55 2015]  [<ffffffff817fc58f>] ret_from_fork+0x3f/0x70
[Tue Dec 15 00:46:55 2015]  [<ffffffff8109c7e0>] ? kthread_create_on_node+0x1a0/0x1a0
[Tue Dec 15 00:46:55 2015] ---[ end trace 4079437668c77cbb ]---
[Tue Dec 15 00:47:45 2015] ABORT_TASK: Found referenced iSCSI task_tag: 95784927
[Tue Dec 15 00:47:45 2015] ABORT_TASK: ref_tag: 95784927 already complete, skipping

In the 12 hours since I started the ESXi test, I have seen this message in dmesg about 20 times.

I am using Ubuntu Trusty, the 4.4rc4 kernel and ceph 9.2.0 from http://ceph.com/debian-infernalis/
ceph -v
ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
uname -a
Linux dfgw01 4.4.0-040400rc4-generic #201512061930 SMP Mon Dec 7 00:32:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Additional information can be found in the dev mail list:
http://article.gmane.org/gmane.comp.file-systems.ceph.devel/28664

The exported file is 900GB, to be less then the 1TB default max size of a Ceph File System file.

I have attached the output of dmesg -T

Files

Download all files

dmesg.txt.gz (10.1 KB) dmesg.txt.gz		Eric Eastman, 12/15/2015 04:10 PM
dmesg-17Dec15.txt.gz (4.32 KB) dmesg-17Dec15.txt.gz		Eric Eastman, 12/17/2015 08:50 AM
dmesg.17Dec15a.txt.gz (2.84 KB) dmesg.17Dec15a.txt.gz		Eric Eastman, 12/17/2015 07:44 PM
dmesg.18Dec15.txt.gz (2.91 KB) dmesg.18Dec15.txt.gz		Eric Eastman, 12/18/2015 03:49 PM

Actions

Copy link

Updated by Eric Eastman over 8 years ago

File dmesg-17Dec15.txt.gz dmesg-17Dec15.txt.gz added

I patched the Linux 4.4rc4 kernel with the patch supplied by Yan, Zheng on the dev mail list, and shortly after rebooting with the new kernel and restarting the ESXi test, I started seeing the new WARNING

[Thu Dec 17 03:29:55 2015] WARNING: CPU: 0 PID: 2547 at fs/ceph/addr.c:1162 ceph_write_begin+0xfb/0x120 [ceph]()
[Thu Dec 17 03:29:55 2015] Modules linked in: iscsi_target_mod vhost_scsi tcm_qla2xxx ib_srpt tcm_fc tcm_usb_gadget tcm_loop target_core_file target_core_iblock target_core_pscsi target_core_user target_core_mod ipmi_devintf vhost qla2xxx ib_cm ib_sa ib_mad ib_core ib_addr libfc scsi_transport_fc libcomposite udc_core uio configfs ttm ipmi_ssif drm_kms_helper drm coretemp kvm gpio_ich i2c_algo_bit i7core_edac fb_sys_fops syscopyarea edac_core sysfillrect sysimgblt ipmi_si input_leds hpilo ipmi_msghandler shpchp acpi_power_meter irqbypass serio_raw 8250_fintek lpc_ich mac_hid ceph bonding libceph lp parport libcrc32c fscache mlx4_en vxlan ip6_udp_tunnel udp_tunnel ptp pps_core hid_generic usbhid hid mlx4_core hpsa psmouse bnx2 fjes scsi_transport_sas [last unloaded: target_core_mod]
[Thu Dec 17 03:29:55 2015] CPU: 0 PID: 2547 Comm: iscsi_trx Tainted: G        W I     4.4.0-rc4-ede1 #1
[Thu Dec 17 03:29:55 2015] Hardware name: HP ProLiant DL360 G6, BIOS P64 01/22/2015
[Thu Dec 17 03:29:55 2015]  ffffffffc020cd47 ffff8805f1e97958 ffffffff813ad644 0000000000000000
[Thu Dec 17 03:29:55 2015]  ffff8805f1e97990 ffffffff81079702 ffff8805f1e97a50 00000000015dd000
[Thu Dec 17 03:29:55 2015]  ffff880c034df800 0000000000000200 ffffea0000b26a80 ffff8805f1e979a0
[Thu Dec 17 03:29:55 2015] Call Trace:
[Thu Dec 17 03:29:55 2015]  [<ffffffff813ad644>] dump_stack+0x44/0x60
[Thu Dec 17 03:29:55 2015]  [<ffffffff81079702>] warn_slowpath_common+0x82/0xc0
[Thu Dec 17 03:29:55 2015]  [<ffffffff810797fa>] warn_slowpath_null+0x1a/0x20
[Thu Dec 17 03:29:55 2015]  [<ffffffffc01e53bb>] ceph_write_begin+0xfb/0x120 [ceph]
[Thu Dec 17 03:29:55 2015]  [<ffffffff8117c8df>] generic_perform_write+0xbf/0x1a0
[Thu Dec 17 03:29:55 2015]  [<ffffffffc01dff9c>] ceph_write_iter+0xf5c/0x1010 [ceph]
[Thu Dec 17 03:29:55 2015]  [<ffffffff810a888c>] ? __enqueue_entity+0x6c/0x70
[Thu Dec 17 03:29:55 2015]  [<ffffffff813c0003>] ? iov_iter_get_pages+0x113/0x210
[Thu Dec 17 03:29:55 2015]  [<ffffffff816b6802>] ? skb_copy_datagram_iter+0x122/0x250
[Thu Dec 17 03:29:55 2015]  [<ffffffff811f6c93>] vfs_iter_write+0x63/0xa0
[Thu Dec 17 03:29:55 2015]  [<ffffffffc03c3f29>] fd_do_rw.isra.5+0xc9/0x1b0 [target_core_file]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc03c40d5>] fd_execute_rw+0xc5/0x2a0 [target_core_file]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc0445e72>] sbc_execute_rw+0x22/0x30 [target_core_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc04449cf>] __target_execute_cmd+0x1f/0x70 [target_core_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc0445525>] target_execute_cmd+0x195/0x2a0 [target_core_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc05c978a>] iscsit_execute_cmd+0x20a/0x270 [iscsi_target_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc05d28da>] iscsit_sequence_cmd+0xda/0x190 [iscsi_target_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffffc05d8c4d>] iscsi_target_rx_thread+0x51d/0xe30 [iscsi_target_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffff8101463d>] ? __switch_to+0x1cd/0x570
[Thu Dec 17 03:29:55 2015]  [<ffffffffc05d8730>] ? iscsi_target_tx_thread+0x1c0/0x1c0 [iscsi_target_mod]
[Thu Dec 17 03:29:55 2015]  [<ffffffff810974c9>] kthread+0xc9/0xe0
[Thu Dec 17 03:29:55 2015]  [<ffffffff81097400>] ? kthread_create_on_node+0x180/0x180
[Thu Dec 17 03:29:55 2015]  [<ffffffff817c794f>] ret_from_fork+0x3f/0x70
[Thu Dec 17 03:29:55 2015]  [<ffffffff81097400>] ? kthread_create_on_node+0x180/0x180
[Thu Dec 17 03:29:55 2015] ---[ end trace 382a45986961da4e ]---

I have attached the dmesg output showing the WARNINGs

There are WARNINGs on both line 125 and 1162.

I wanted to note that file system snapshots are enabled and being used on this file system.

Actions

Copy link

Updated by Eric Eastman over 8 years ago

File dmesg.17Dec15a.txt.gz dmesg.17Dec15a.txt.gz added

I have applied both cephfs.patch and cephfs1.patch provided on the dev mail list and re-ran the test. I am now seeing:


[Thu Dec 17 14:27:59 2015] ------------[ cut here ]------------
[Thu Dec 17 14:27:59 2015] WARNING: CPU: 0 PID: 3036 at fs/ceph/addr.c:1171 ceph_write_begin+0xfb/0x120 [ceph]()
[Thu Dec 17 14:27:59 2015] Modules linked in: iscsi_target_mod vhost_scsi tcm_qla2xxx ib_srpt tcm_fc tcm_usb_gadget tcm_loop target_core_file target_core_iblock target_core_pscsi target_core_user target_core_mod ipmi_devintf vhost qla2xxx ib_cm ib_sa ib_mad ib_core ib_addr libfc scsi_transport_fc libcomposite udc_core uio configfs ttm drm_kms_helper drm ipmi_ssif coretemp gpio_ich i2c_algo_bit kvm fb_sys_fops syscopyarea sysfillrect sysimgblt shpchp input_leds ceph irqbypass i7core_edac serio_raw hpilo edac_core ipmi_si ipmi_msghandler 8250_fintek lpc_ich acpi_power_meter libceph mac_hid libcrc32c fscache bonding lp parport mlx4_en vxlan ip6_udp_tunnel udp_tunnel ptp pps_core hid_generic usbhid hid mlx4_core hpsa psmouse bnx2 fjes scsi_transport_sas [last unloaded: target_core_mod]
[Thu Dec 17 14:27:59 2015] CPU: 0 PID: 3036 Comm: iscsi_trx Tainted: G        W I     4.4.0-rc4-ede2 #1
[Thu Dec 17 14:27:59 2015] Hardware name: HP ProLiant DL360 G6, BIOS P64 01/22/2015
[Thu Dec 17 14:27:59 2015]  ffffffffc02b2e37 ffff880c0289b958 ffffffff813ad644 0000000000000000
[Thu Dec 17 14:27:59 2015]  ffff880c0289b990 ffffffff81079702 ffff880c0289ba50 0000000846c21000
[Thu Dec 17 14:27:59 2015]  ffff880c009ea200 0000000000001000 ffffea00122ed700 ffff880c0289b9a0
[Thu Dec 17 14:27:59 2015] Call Trace:
[Thu Dec 17 14:27:59 2015]  [<ffffffff813ad644>] dump_stack+0x44/0x60
[Thu Dec 17 14:27:59 2015]  [<ffffffff81079702>] warn_slowpath_common+0x82/0xc0
[Thu Dec 17 14:27:59 2015]  [<ffffffff810797fa>] warn_slowpath_null+0x1a/0x20
[Thu Dec 17 14:27:59 2015]  [<ffffffffc028b41b>] ceph_write_begin+0xfb/0x120 [ceph]
[Thu Dec 17 14:27:59 2015]  [<ffffffff8117c8df>] generic_perform_write+0xbf/0x1a0
[Thu Dec 17 14:27:59 2015]  [<ffffffffc0285f9c>] ceph_write_iter+0xf5c/0x1010 [ceph]
[Thu Dec 17 14:27:59 2015]  [<ffffffff817c3396>] ? __schedule+0x386/0x9c0
[Thu Dec 17 14:27:59 2015]  [<ffffffff817c3a05>] ? schedule+0x35/0x80
[Thu Dec 17 14:27:59 2015]  [<ffffffff811d7c65>] ? __slab_free+0xb5/0x290
[Thu Dec 17 14:27:59 2015]  [<ffffffff813c0003>] ? iov_iter_get_pages+0x113/0x210
[Thu Dec 17 14:27:59 2015]  [<ffffffff811f6c93>] vfs_iter_write+0x63/0xa0
[Thu Dec 17 14:27:59 2015]  [<ffffffffc02d2f29>] fd_do_rw.isra.5+0xc9/0x1b0 [target_core_file]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc02d30d5>] fd_execute_rw+0xc5/0x2a0 [target_core_file]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc0430e72>] sbc_execute_rw+0x22/0x30 [target_core_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc042f9cf>] __target_execute_cmd+0x1f/0x70 [target_core_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc0430525>] target_execute_cmd+0x195/0x2a0 [target_core_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc05b778a>] iscsit_execute_cmd+0x20a/0x270 [iscsi_target_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc05c08da>] iscsit_sequence_cmd+0xda/0x190 [iscsi_target_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffffc05c6c4d>] iscsi_target_rx_thread+0x51d/0xe30 [iscsi_target_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffff8101463d>] ? __switch_to+0x1cd/0x570
[Thu Dec 17 14:27:59 2015]  [<ffffffffc05c6730>] ? iscsi_target_tx_thread+0x1c0/0x1c0 [iscsi_target_mod]
[Thu Dec 17 14:27:59 2015]  [<ffffffff810974c9>] kthread+0xc9/0xe0
[Thu Dec 17 14:27:59 2015]  [<ffffffff81097400>] ? kthread_create_on_node+0x180/0x180
[Thu Dec 17 14:27:59 2015]  [<ffffffff817c794f>] ret_from_fork+0x3f/0x70
[Thu Dec 17 14:27:59 2015]  [<ffffffff81097400>] ? kthread_create_on_node+0x180/0x180
[Thu Dec 17 14:27:59 2015] ---[ end trace 8346192e3f29ed5d ]---

Each of the WARNING on line 1171 is followed by a WARNING on line 125. I have attached the dmesg -T output

Actions

Copy link

Updated by Zheng Yan over 8 years ago

Status changed from New to 7

https://lkml.org/lkml/2015/12/12/113

Actions

Copy link

Updated by Eric Eastman over 8 years ago

File dmesg.18Dec15.txt.gz dmesg.18Dec15.txt.gz added

With the 4.4rc4 kernel and the cephfs_new.patch and CONFIG_DEBUG_VM=y I hit a BUG in mm/filemap.c.

Fri Dec 18 01:14:39 2015] kernel BUG at mm/filemap.c:812!
[Fri Dec 18 01:14:39 2015] invalid opcode: 0000 [#1] SMP 
[Fri Dec 18 01:14:39 2015] Modules linked in: iscsi_target_mod vhost_scsi tcm_qla2xxx ib_srpt tcm_fc tcm_usb_gadget tcm_loop target_core_file target_core_iblock target_core_pscsi target_core_user target_core_mod ipmi_devintf vhost qla2xxx ib_cm ib_sa ib_mad ib_core ib_addr libfc scsi_transport_fc libcomposite udc_core uio configfs ttm drm_kms_helper coretemp drm kvm ipmi_ssif gpio_ich ceph i2c_algo_bit fb_sys_fops syscopyarea input_leds sysfillrect sysimgblt irqbypass shpchp hpilo serio_raw acpi_power_meter ipmi_si lpc_ich i7core_edac ipmi_msghandler edac_core 8250_fintek libceph mac_hid libcrc32c fscache bonding lp parport mlx4_en vxlan ip6_udp_tunnel udp_tunnel ptp pps_core hid_generic usbhid hid mlx4_core psmouse hpsa bnx2 fjes scsi_transport_sas [last unloaded: target_core_mod]
[Fri Dec 18 01:14:39 2015] CPU: 0 PID: 2147 Comm: iscsi_trx Tainted: G        W I     4.4.0-rc4-ede3-DEBUG_VM #1
[Fri Dec 18 01:14:39 2015] Hardware name: HP ProLiant DL360 G6, BIOS P64 01/22/2015
[Fri Dec 18 01:14:39 2015] task: ffff880c02077080 ti: ffff880bfce5c000 task.ti: ffff880bfce5c000
[Fri Dec 18 01:14:39 2015] RIP: 0010:[<ffffffff8117d041>]  [<ffffffff8117d041>] unlock_page+0x81/0x90
[Fri Dec 18 01:14:39 2015] RSP: 0018:ffff880bfce5f9b8  EFLAGS: 00010282
[Fri Dec 18 01:14:39 2015] RAX: 0000000000000021 RBX: ffffea0015d36ac0 RCX: 0000000000000000
[Fri Dec 18 01:14:40 2015] RDX: 0000000000000021 RSI: ffff880607a0dc78 RDI: ffff880607a0dc78
[Fri Dec 18 01:14:40 2015] RBP: ffff880bfce5f9b8 R08: 0000000000000000 R09: 000000000000041e
[Fri Dec 18 01:14:40 2015] R10: 0000000000000246 R11: 000000000000041e R12: 0000000000000000
[Fri Dec 18 01:14:40 2015] R13: 0000000000001000 R14: ffff8800dad39f88 R15: 0000000000001000
[Fri Dec 18 01:14:40 2015] FS:  0000000000000000(0000) GS:ffff880607a00000(0000) knlGS:0000000000000000
[Fri Dec 18 01:14:40 2015] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[Fri Dec 18 01:14:40 2015] CR2: 00000000008e6f90 CR3: 0000000001c0a000 CR4: 00000000000006f0
[Fri Dec 18 01:14:40 2015] Stack:
[Fri Dec 18 01:14:40 2015]  ffff880bfce5fa00 ffffffffc02d4ac6 ffff880bfce5fa00 ffffffff813c5456
[Fri Dec 18 01:14:40 2015]  0000000000001000 000000081e838000 ffff880bfce5fc80 0000000000000000
[Fri Dec 18 01:14:40 2015]  ffff8800dad3a0f0 ffff880bfce5fa88 ffffffff8117cb95 ffff88002cd5ec80
[Fri Dec 18 01:14:40 2015] Call Trace:
[Fri Dec 18 01:14:40 2015]  [<ffffffffc02d4ac6>] ceph_write_end+0x66/0x180 [ceph]
[Fri Dec 18 01:14:40 2015]  [<ffffffff813c5456>] ? iov_iter_copy_from_user_atomic+0x156/0x220
[Fri Dec 18 01:14:40 2015]  [<ffffffff8117cb95>] generic_perform_write+0x105/0x1a0
[Fri Dec 18 01:14:40 2015]  [<ffffffffc02cff9c>] ceph_write_iter+0xf5c/0x1010 [ceph]
[Fri Dec 18 01:14:40 2015]  [<ffffffff817c8af6>] ? __schedule+0x386/0x9c0
[Fri Dec 18 01:14:40 2015]  [<ffffffff817c9165>] ? schedule+0x35/0x80
[Fri Dec 18 01:14:40 2015]  [<ffffffff813c0003>] ? insn_get_immediate.part.8+0x293/0x300
[Fri Dec 18 01:14:40 2015]  [<ffffffff816bbeb2>] ? skb_copy_datagram_iter+0x122/0x250
[Fri Dec 18 01:14:40 2015]  [<ffffffff811fbe23>] vfs_iter_write+0x63/0xa0
[Fri Dec 18 01:14:40 2015]  [<ffffffffc0252f29>] fd_do_rw.isra.5+0xc9/0x1b0 [target_core_file]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc02530d5>] fd_execute_rw+0xc5/0x2a0 [target_core_file]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc0352e72>] sbc_execute_rw+0x22/0x30 [target_core_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc03519cf>] __target_execute_cmd+0x1f/0x70 [target_core_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc0352525>] target_execute_cmd+0x195/0x2a0 [target_core_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc05bf78a>] iscsit_execute_cmd+0x20a/0x270 [iscsi_target_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc05c88da>] iscsit_sequence_cmd+0xda/0x190 [iscsi_target_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffffc05cec4d>] iscsi_target_rx_thread+0x51d/0xe30 [iscsi_target_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffff8101463d>] ? __switch_to+0x1cd/0x570
[Fri Dec 18 01:14:40 2015]  [<ffffffffc05ce730>] ? iscsi_target_tx_thread+0x1c0/0x1c0 [iscsi_target_mod]
[Fri Dec 18 01:14:40 2015]  [<ffffffff81097859>] kthread+0xc9/0xe0
[Fri Dec 18 01:14:40 2015]  [<ffffffff81097790>] ? kthread_create_on_node+0x180/0x180
[Fri Dec 18 01:14:40 2015]  [<ffffffff817cd0cf>] ret_from_fork+0x3f/0x70
[Fri Dec 18 01:14:40 2015]  [<ffffffff81097790>] ? kthread_create_on_node+0x180/0x180
[Fri Dec 18 01:14:40 2015] Code: b8 00 00 00 48 8b 80 a8 00 00 00 48 d3 ea 48 8d 14 52 48 8d 3c d0 31 d2 e8 2d cb f3 ff 5d c3 48 c7 c6 a0 12 ad 81 e8 3f c5 02 00 <0f> 0b 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 
[Fri Dec 18 01:14:40 2015] RIP  [<ffffffff8117d041>] unlock_page+0x81/0x90
[Fri Dec 18 01:14:40 2015]  RSP <ffff880bfce5f9b8>
[Fri Dec 18 01:14:40 2015] ---[ end trace d2dd732cc24afbf8 ]---

I have attached the output of dmesg -T. I will start a test with the 4.4rc5 kernel.

Actions

Copy link

Updated by Eric Eastman over 8 years ago

The latest test with 4.4rc5 with CONFIG_DEBUG_VM=y has ran for over 36 hours with no ERRORS or WARNINGS. My plan is to install the 4.4rc6 kernel from the Ubuntu kernel-ppa site once it is available, and rerun the tests.

Actions

Copy link