https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2021-02-18T21:46:43Z
Ceph
Linux kernel client - Bug #49365: octopus: qa: "Cannot write to 'pjd-fstest-20090130-RC-aclfixes.tgz' (Invalid argument)."
https://tracker.ceph.com/issues/49365?journal_id=185454
2021-02-18T21:46:43Z
Patrick Donnelly
pdonnell@redhat.com
<ul></ul><p>I think this might be because of inline feature getting turned on. Same failure/error for dbench:</p>
<p>/ceph/teuthology-archive/yuriw-2021-02-16_18:11:32-kcephfs-octopus-testing-basic-smithi/5888328/teuthology.log</p>
Linux kernel client - Bug #49365: octopus: qa: "Cannot write to 'pjd-fstest-20090130-RC-aclfixes.tgz' (Invalid argument)."
https://tracker.ceph.com/issues/49365?journal_id=185455
2021-02-18T22:01:14Z
Patrick Donnelly
pdonnell@redhat.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><pre>
2021-02-18T02:24:58.661159+00:00 smithi163 kernel: [ 558.670041] ------------[ cut here ]------------
2021-02-18T02:24:58.661205+00:00 smithi163 kernel: [ 558.670069] ceph: write_begin called on still-inlined inode!
2021-02-18T02:24:58.661208+00:00 smithi163 kernel: [ 558.670099] WARNING: CPU: 2 PID: 11937 at fs/ceph/addr.c:1236 ceph_write_begin+0x237/0x290 [ceph]
2021-02-18T02:24:58.661209+00:00 smithi163 kernel: [ 558.670129] Modules linked in: ceph libceph fscache netfs xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge stp llc xfs overlay intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal ipmi_ssif intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper joydev lpc_ich mei_me mei wmi acpi_ipmi ipmi_si kvm_intel ipmi_devintf ipmi_msghandler acpi_power_meter ioatdma acpi_pad kvm irqbypass sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi lp parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs blake2b_generic xor zstd_compress raid6_pq libcrc32c hid_generic usbhid hid igb i2c_algo_bit ixgbe dca nvme ahci ptp libahci nvme_core pps_core mdio
2021-02-18T02:24:58.661228+00:00 smithi163 kernel: [ 558.670348] CPU: 2 PID: 11937 Comm: wget Not tainted 5.11.0-ceph-ge25079bca433 #1
2021-02-18T02:24:58.661230+00:00 smithi163 kernel: [ 558.670353] Hardware name: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0a 08/02/2016
2021-02-18T02:24:58.661232+00:00 smithi163 kernel: [ 558.670356] RIP: 0010:ceph_write_begin+0x237/0x290 [ceph]
2021-02-18T02:24:58.661233+00:00 smithi163 kernel: [ 558.670375] Code: ff ff ff 41 bc ea ff ff ff e9 5c ff ff ff 48 c7 c7 10 82 14 c1 44 89 44 24 0c 48 89 34 24 c6 05 2f cb 05 00 01 e8 0a a3 97 e4 <0f> 0b 48 8b 34 24 44 8b 44 24 0c 48 85 f6 0f 85 10 fe ff ff 48 8b
2021-02-18T02:24:58.661235+00:00 smithi163 kernel: [ 558.670379] RSP: 0018:ffffa973c1277c30 EFLAGS: 00010286
2021-02-18T02:24:58.661247+00:00 smithi163 kernel: [ 558.670385] RAX: 0000000000000000 RBX: ffffa973c1277ca0 RCX: 0000000000000027
2021-02-18T02:24:58.661249+00:00 smithi163 kernel: [ 558.670388] RDX: 0000000000000027 RSI: 0000000000000000 RDI: ffff8df11fc98ff8
2021-02-18T02:24:58.661251+00:00 smithi163 kernel: [ 558.670391] RBP: ffff8dea80eb9038 R08: ffff8df11fc98ff0 R09: 0000000000000001
2021-02-18T02:24:58.661252+00:00 smithi163 kernel: [ 558.670394] R10: 0000000000000001 R11: 0000000000000001 R12: ffffa973c1277e60
2021-02-18T02:24:58.661254+00:00 smithi163 kernel: [ 558.670398] R13: ffff8dea80eb9038 R14: ffff8de9c312ce00 R15: 0000000000001000
2021-02-18T02:24:58.661255+00:00 smithi163 kernel: [ 558.670401] FS: 00007f58ce5492c0(0000) GS:ffff8df11fc80000(0000) knlGS:0000000000000000
2021-02-18T02:24:58.661267+00:00 smithi163 kernel: [ 558.670405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2021-02-18T02:24:58.661269+00:00 smithi163 kernel: [ 558.670408] CR2: 0000559f1ed18f98 CR3: 000000010852c005 CR4: 00000000003706e0
2021-02-18T02:24:58.661271+00:00 smithi163 kernel: [ 558.670412] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
2021-02-18T02:24:58.661272+00:00 smithi163 kernel: [ 558.670415] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
2021-02-18T02:24:58.661273+00:00 smithi163 kernel: [ 558.670418] Call Trace:
2021-02-18T02:24:58.661275+00:00 smithi163 kernel: [ 558.670427] generic_perform_write+0xc3/0x1e0
2021-02-18T02:24:58.661277+00:00 smithi163 kernel: [ 558.670446] ceph_write_iter+0xb2f/0xd50 [ceph]
2021-02-18T02:24:58.661289+00:00 smithi163 kernel: [ 558.670492] ? new_sync_write+0x11c/0x1b0
2021-02-18T02:24:58.661290+00:00 smithi163 kernel: [ 558.670498] new_sync_write+0x11c/0x1b0
2021-02-18T02:24:58.661292+00:00 smithi163 kernel: [ 558.670513] vfs_write+0x243/0x390
2021-02-18T02:24:58.661293+00:00 smithi163 kernel: [ 558.670523] ksys_write+0x68/0xe0
2021-02-18T02:24:58.661295+00:00 smithi163 kernel: [ 558.670531] do_syscall_64+0x2d/0x70
2021-02-18T02:24:58.661296+00:00 smithi163 kernel: [ 558.670537] entry_SYSCALL_64_after_hwframe+0x44/0xa9
2021-02-18T02:24:58.661308+00:00 smithi163 kernel: [ 558.670544] RIP: 0033:0x7f58cd058224
2021-02-18T02:24:58.661310+00:00 smithi163 kernel: [ 558.670548] Code: 89 02 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 8d 05 c1 07 2e 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 f3 c3 66 90 41 54 55 49 89 d4 53 48 89 f5
2021-02-18T02:24:58.661312+00:00 smithi163 kernel: [ 558.670552] RSP: 002b:00007ffd48b0c738 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
2021-02-18T02:24:58.661313+00:00 smithi163 kernel: [ 558.670558] RAX: ffffffffffffffda RBX: 0000000000002000 RCX: 00007f58cd058224
2021-02-18T02:24:58.661314+00:00 smithi163 kernel: [ 558.670561] RDX: 0000000000002000 RSI: 0000559f1ed14f10 RDI: 0000000000000004
2021-02-18T02:24:58.661316+00:00 smithi163 kernel: [ 558.670563] RBP: 0000559f1ed14f10 R08: 0000000000002000 R09: 0000000000000000
2021-02-18T02:24:58.661317+00:00 smithi163 kernel: [ 558.670566] R10: 0000559f1ed0f010 R11: 0000000000000246 R12: 0000559f1ed104a0
2021-02-18T02:24:58.661329+00:00 smithi163 kernel: [ 558.670569] R13: 0000000000002000 R14: 00007f58cd32f760 R15: 0000000000002000
2021-02-18T02:24:58.661330+00:00 smithi163 kernel: [ 558.670587] irq event stamp: 14161
2021-02-18T02:24:58.661332+00:00 smithi163 kernel: [ 558.670590] hardirqs last enabled at (14167): [<ffffffffa5115cef>] console_unlock+0x4bf/0x5a0
2021-02-18T02:24:58.661333+00:00 smithi163 kernel: [ 558.670599] hardirqs last disabled at (14172): [<ffffffffa5115c62>] console_unlock+0x432/0x5a0
2021-02-18T02:24:58.661334+00:00 smithi163 kernel: [ 558.670604] softirqs last enabled at (13322): [<ffffffffa5e00306>] __do_softirq+0x306/0x42c
2021-02-18T02:24:58.661335+00:00 smithi163 kernel: [ 558.670610] softirqs last disabled at (13317): [<ffffffffa5c010af>] asm_call_irq_on_stack+0xf/0x20
2021-02-18T02:24:58.661348+00:00 smithi163 kernel: [ 558.670616] ---[ end trace 9112c35e13586d71 ]---
</pre>
<p>From: /ceph/teuthology-archive/yuriw-2021-02-16_18:11:32-kcephfs-octopus-testing-basic-smithi/5888305/remote/smithi163/syslog/kern.log.gz</p>
<p>Caused by new code: <a class="external" href="https://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git/tree/fs/ceph/addr.c?h=ceph-netfs-lib#n1237">https://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git/tree/fs/ceph/addr.c?h=ceph-netfs-lib#n1237</a></p>
<p>Jeff is looking into it.</p>
Linux kernel client - Bug #49365: octopus: qa: "Cannot write to 'pjd-fstest-20090130-RC-aclfixes.tgz' (Invalid argument)."
https://tracker.ceph.com/issues/49365?journal_id=185689
2021-02-22T16:54:48Z
Jeff Layton
jlayton@redhat.com
<ul><li><strong>Project</strong> changed from <i>CephFS</i> to <i>Linux kernel client</i></li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>The problem was the write_begin patch in the netfs lib patchset in testing branch. I resolved it in my own testing and pushed an updated branch to ceph-client/testing.</p>
<p>The main issue was that I needed to handle the case of writing to a new inode with no data.</p>
Linux kernel client - Bug #49365: octopus: qa: "Cannot write to 'pjd-fstest-20090130-RC-aclfixes.tgz' (Invalid argument)."
https://tracker.ceph.com/issues/49365?journal_id=185857
2021-02-24T15:28:51Z
Patrick Donnelly
pdonnell@redhat.com
<ul></ul><p>Tests in flight:</p>
<p><a class="external" href="https://pulpito.ceph.com/pdonnell-2021-02-24_15:27:46-kcephfs-octopus-distro-basic-smithi/">https://pulpito.ceph.com/pdonnell-2021-02-24_15:27:46-kcephfs-octopus-distro-basic-smithi/</a><br /><a class="external" href="https://pulpito.ceph.com/pdonnell-2021-02-24_15:28:12-kcephfs-octopus-distro-basic-smithi/">https://pulpito.ceph.com/pdonnell-2021-02-24_15:28:12-kcephfs-octopus-distro-basic-smithi/</a></p>
Linux kernel client - Bug #49365: octopus: qa: "Cannot write to 'pjd-fstest-20090130-RC-aclfixes.tgz' (Invalid argument)."
https://tracker.ceph.com/issues/49365?journal_id=185874
2021-02-24T21:01:04Z
Patrick Donnelly
pdonnell@redhat.com
<ul></ul><p>Patrick Donnelly wrote:</p>
<blockquote>
<p>Tests in flight:</p>
<p><a class="external" href="https://pulpito.ceph.com/pdonnell-2021-02-24_15:27:46-kcephfs-octopus-distro-basic-smithi/">https://pulpito.ceph.com/pdonnell-2021-02-24_15:27:46-kcephfs-octopus-distro-basic-smithi/</a><br /><a class="external" href="https://pulpito.ceph.com/pdonnell-2021-02-24_15:28:12-kcephfs-octopus-distro-basic-smithi/">https://pulpito.ceph.com/pdonnell-2021-02-24_15:28:12-kcephfs-octopus-distro-basic-smithi/</a></p>
</blockquote>
<p>Seems the fix worked.</p>