Bug #54743: crash: Client::_get_vino(Inode*) - CephFS - Ceph

Actions

Copy link

Bug #54743

open

crash: Client::_get_vino(Inode*)

Added by Telemetry Bot about 2 years ago. Updated over 1 year ago.

Status:

Triaged

Priority:

Normal

Assignee:

Milind Changire

Category:

Target version:

% Done:

Source:

Telemetry

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v15.2.13, Ceph - v16.2.7, Ceph - v17.2.0

ceph-qa-suite:

Component(FS):

Labels (FS):

Pull request ID:

Crash signature (v1):

fe356f5a5b2f149a5f06f53f59b7d42330c1bdb241c5a3e2bbcca6194b3d9700
41037d704a658873c64eb18d13de2172a5c06e512f18e0c57c0e5ded3ccef94a
6c7e840cd9244b8bb137d964e941eebf2f74077456c9330d9a6424a8d1050cd2
9c701b9743ed46630b3bf973e8244bac4eff6c3351ba464531af58ac427a058d
b680a903695b7b6fc40b5c3fa66258c3b26f8ad09e710ae266eeb5de9f08e41b

Crash signature (v2):

7160918e1db8c7e75cc34967bf8771e62ffd3bae95aa2630ac142e6de138962c
40f5abbee44ea6074376240bf7bcd505f895b21c4b0c01a141427fd0637279d8

Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7160918e1db8c7e75cc34967bf8771e62ffd3bae95aa2630ac142e6de138962c

Sanitized backtrace:

    Client::_get_vino(Inode*)
    Client::_ll_getattr(Inode*, int, UserPerm const&)
    Client::ll_getattr(Inode*, stat*, UserPerm const&)

Crash dump sample:

{
    "backtrace": [
        "__kernel_rt_sigreturn()",
        "(Client::_get_vino(Inode*)+0) [0xaaaab01f0d78]",
        "(Client::_ll_getattr(Inode*, int, UserPerm const&)+0x3c) [0xaaaab02370fc]",
        "(Client::ll_getattr(Inode*, stat*, UserPerm const&)+0xd8) [0xaaaab0237480]",
        "ceph-fuse(+0x98714) [0xaaaab01ea714]",
        "/lib/aarch64-linux-gnu/libfuse.so.2(+0x14d7c) [0xffff84b6ad7c]",
        "/lib/aarch64-linux-gnu/libfuse.so.2(+0x15064) [0xffff84b6b064]",
        "/lib/aarch64-linux-gnu/libfuse.so.2(+0x12158) [0xffff84b68158]",
        "/lib/aarch64-linux-gnu/libpthread.so.0(+0x84fc) [0xffff7bed74fc]",
        "/lib/aarch64-linux-gnu/libc.so.6(+0xd467c) [0xffff7bc1367c]" 
    ],
    "ceph_version": "16.2.7",
    "crash_id": "2022-02-22T23:50:00.338728Z_0e9cc298-8bc6-496c-815d-9b6f1110a901",
    "entity_name": "client.779720a56c617f4713d46a0389a5f0b5c78d2903",
    "os_id": "ubuntu",
    "os_name": "Ubuntu",
    "os_version": "20.04.4 LTS (Focal Fossa)",
    "os_version_id": "20.04",
    "process_name": "ceph-fuse",
    "stack_sig": "fe356f5a5b2f149a5f06f53f59b7d42330c1bdb241c5a3e2bbcca6194b3d9700",
    "timestamp": "2022-02-22T23:50:00.338728Z",
    "utsname_machine": "aarch64",
    "utsname_release": "5.4.0-1052-raspi",
    "utsname_sysname": "Linux",
    "utsname_version": "#58-Ubuntu SMP PREEMPT Mon Feb 7 16:52:35 UTC 2022" 
}

Actions

Copy link

Updated by Telemetry Bot about 2 years ago

Crash signature (v1) updated (diff)
Crash signature (v2) updated (diff)
Affected Versions v16.2.7 added

Actions

Copy link

Updated by Venky Shankar about 2 years ago

Status changed from New to Triaged
Assignee set to Milind Changire

Actions

Copy link

Updated by Milind Changire about 2 years ago

This seems to be the real culprit:

        {
            "os_id": "ubuntu",
            "os_name": "Ubuntu",
            "archived": "2022-02-21 18:45:32.756343",
            "crash_id": "2022-02-18T08:20:12.455080Z_abeac706-7c0a-4857-8fd9-0689023db9af",
            "backtrace": [
                "__kernel_rt_sigreturn()",
                "gsignal()",
                "abort()",
                "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x194) [0xffff883ed958]",
                "(ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0xffff883edac8]",
                "(ObjectCacher::bh_write_commit(long, sobject_t, std::vector<std::pair<long, unsigned long>, std::allocator<std::pair<long, unsigned long> > >&, unsigned long, int)+0x1468) [0xaaaaabcfcb50]",
                "(ObjectCacher::C_WriteCommit::finish(int)+0x78) [0xaaaaabd01648]",
                "(Context::complete(int)+0x1c) [0xaaaaabc9c624]",
                "(C_Lock::finish(int)+0x50) [0xaaaaabc9f368]",
                "(Context::complete(int)+0x1c) [0xaaaaabc9c624]",
                "(Finisher::finisher_thread_entry()+0x164) [0xffff8844ecfc]",
                "/lib/aarch64-linux-gnu/libpthread.so.0(+0x84fc) [0xffff881674fc]",
                "/lib/aarch64-linux-gnu/libc.so.6(+0xd467c) [0xffff87ea367c]" 
            ],
            "stack_sig": "b680a903695b7b6fc40b5c3fa66258c3b26f8ad09e710ae266eeb5de9f08e41b",
            "timestamp": "2022-02-18T08:20:12.455080Z",
            "assert_msg": "/build/ceph-16.2.7/src/osdc/ObjectCacher.cc: In function 'void ObjectCacher::bh_write_commit(int64_t, sobject_t, std::vector<std::pair<long int, long unsigned int> >&, ceph_tid_t, int)' thread ffff6fffd880 time 2022-02-18T09:20:12.410517+0100\n/build/ceph-16.2.7/src/osdc/ObjectCacher.cc: 1254: FAILED ceph_assert(ob->last_commit_tid < tid)\n",
            "os_version": "20.04.3 LTS (Focal Fossa)",
            "assert_file": "/build/ceph-16.2.7/src/osdc/ObjectCacher.cc",
            "assert_func": "void ObjectCacher::bh_write_commit(int64_t, sobject_t, std::vector<std::pair<long int, long unsigned int> >&, ceph_tid_t, int)",
            "assert_line": 1254,
            "entity_name": "client.779720a56c617f4713d46a0389a5f0b5c78d2903",
            "ceph_version": "16.2.7",
            "process_name": "ceph-fuse",
            "os_version_id": "20.04",
            "utsname_machine": "aarch64",
            "utsname_release": "5.4.0-1050-raspi",
            "utsname_sysname": "Linux",
            "utsname_version": "#56-Ubuntu SMP PREEMPT Thu Jan 13 13:09:35 UTC 2022",
            "assert_condition": "ob->last_commit_tid < tid",
            "assert_thread_name": "fn_anonymous" 
        }

Actions

Copy link