Project

General

Profile

Actions

Bug #58305

open

src/mon/AuthMonitor.cc: FAILED ceph_assert(version > keys_ver)

Added by yite gu over 1 year ago. Updated over 1 year ago.

Status:
Need More Info
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

# ceph crash info 2022-12-17T19:15:22.811185Z_de06b1be-451b-4128-a06b-fcbc582877b8
{
    "assert_condition": "version > keys_ver",
    "assert_file": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.7/rpm/el8/BUILD/ceph-15.2.7/src/mon/AuthMonitor.cc",
    "assert_func": "virtual void AuthMonitor::update_from_paxos(bool*)",
    "assert_line": 250,
    "assert_msg": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.7/rpm/el8/BUILD/ceph-15.2.7/src/mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos(bool*)' thread 7f7f438ec700 time 2022-12-17T19:15:22.808254+0000\n/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.7/rpm/el8/BUILD/ceph-15.2.7/src/mon/AuthMonitor.cc: 250: FAILED ceph_assert(version > keys_ver)\n",
    "assert_thread_name": "fn_monstore",
    "backtrace": [
        "(()+0x12dd0) [0x7f7f50755dd0]",
        "(gsignal()+0x10f) [0x7f7f4f3be70f]",
        "(abort()+0x127) [0x7f7f4f3a8b25]",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f7f52c54bdd]",
        "(()+0x279da6) [0x7f7f52c54da6]",
        "(AuthMonitor::update_from_paxos(bool*)+0x2c59) [0x560fdad64189]",
        "(PaxosService::refresh(bool*)+0x10e) [0x560fdae0a83e]",
        "(Monitor::refresh_from_paxos(bool*)+0x18c) [0x560fdacdb60c]",
        "(Paxos::do_refresh()+0x57) [0x560fdadf8317]",
        "(Paxos::commit_finish()+0x793) [0x560fdae05173]",
        "(C_Committed::finish(int)+0x45) [0x560fdae09725]",
        "(Context::complete(int)+0xd) [0x560fdad168dd]",
        "(MonitorDBStore::C_DoTransaction::finish(int)+0x98) [0x560fdae09448]",
        "(Context::complete(int)+0xd) [0x560fdad168dd]",
        "(Finisher::finisher_thread_entry()+0x1a5) [0x7f7f52ce4e65]",
        "(()+0x82de) [0x7f7f5074b2de]",
        "(clone()+0x43) [0x7f7f4f482e83]" 
    ],
    "ceph_version": "15.2.7",
    "crash_id": "2022-12-17T19:15:22.811185Z_de06b1be-451b-4128-a06b-fcbc582877b8",
    "entity_name": "mon.a",
    "os_id": "centos",
    "os_name": "CentOS Linux",
    "os_version": "8 (Core)",
    "os_version_id": "8",
    "process_name": "ceph-mon",
    "stack_sig": "f59fe39a08e437790b56f254ed0cbddcac88811155e0779478f47c8509ed746b",
    "timestamp": "2022-12-17T19:15:22.811185Z",
    "utsname_hostname": "n6-058-030",
    "utsname_machine": "x86_64",
    "utsname_release": "5.4.56.bsk.9-amd64",
    "utsname_sysname": "Linux",
    "utsname_version": "#5.4.56.bsk.9 SMP Debian 5.4.56.bsk.9 Wed Aug 25 03:42:38 UTC 20" 
}


Files

log (749 KB) log yite gu, 12/20/2022 11:55 AM

Related issues 2 (2 open0 closed)

Related to RADOS - Bug #40777: hit assert in AuthMonitor::update_from_paxosNew

Actions
Related to RADOS - Bug #51945: qa/workunits/mon/caps.sh: Error: Expected return 13, got 0NewRadoslaw Zarzynski

Actions
Actions #1

Updated by Neha Ojha over 1 year ago

  • Subject changed from mon: mon crash to src/mon/AuthMonitor.cc: FAILED ceph_assert(version > keys_ver)
Actions #2

Updated by Radoslaw Zarzynski over 1 year ago

  • Related to Bug #40777: hit assert in AuthMonitor::update_from_paxos added
Actions #3

Updated by Radoslaw Zarzynski over 1 year ago

  • Related to Bug #51945: qa/workunits/mon/caps.sh: Error: Expected return 13, got 0 added
Actions #4

Updated by Radoslaw Zarzynski over 1 year ago

  • Status changed from New to Need More Info

Thanks for the report! Do have a corresponding log or coredump by any chance?

Actions #5

Updated by yite gu over 1 year ago

Radoslaw Zarzynski wrote:

Thanks for the report! Do have a corresponding log or coredump by any chance?

This log file come from /opt/rook/rook-ceph/crash/posted/2022-12-17T19:15:22.811185Z_de06b1be-451b-4128-a06b-fcbc582877b8

Actions #6

Updated by Radoslaw Zarzynski over 1 year ago

Thanks for the log file! Would you be able to try to replicate with higher debugs levels?

Perhaps something like: debug_paxos=20, debug_mon=20, debug_auth+20, debug_ms=1.

Actions #7

Updated by yite gu over 1 year ago

Radoslaw Zarzynski wrote:

Thanks for the log file! Would you be able to try to replicate with higher debugs levels?

Perhaps something like: debug_paxos=20, debug_mon=20, debug_auth+20, debug_ms=1.

I already set higher debugs levels, but problem have no replicate.

Actions

Also available in: Atom PDF