Project

General

Profile

Bug #56192

crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty())

Added by Telemetry Bot almost 2 years ago. Updated 3 months ago.

Status:
Pending Backport
Priority:
Normal
Category:
-
Target version:
% Done:

0%

Source:
Telemetry
Tags:
backport_processed
Backport:
pacific quincy reef
Regression:
No
Severity:
3 - minor
Reviewed:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):

0037073cc3ac263b34152d0d00e30b5f8ee6ff25d309aa22de81e99b33c18e5b
01fe407aa36177ebade5c10cd1ca1b8a7e214dc44b7c3314f3a05c82d96070b2
02b9953dd602d50e803b32065040392754234059b7c78dbc52c05b2cf2480c82
0a4ae0b1fe10851f11f6119256e4526088bc54ae8a871695a79859a5d9bf6cdd
0c3729fca27d98d58d528e908dbd0e0f4b7ac708e5778cf41976ba2dc992d280
0d31619405b90977aa8ba93d1aa0de302f38b31523f5b24750f7f734101749cf
0f71e7c40c0d7a571deb6d0293f20c9f9e86954651cd7a3e670b6cfddec2e64a
0f9f5cdbdedac2f97ac4cfb856f5363ade0d5442d19905dadc0d8cc7c5a1c203
1006527e6b73331da4af4f329a2c91ad60fc1a1e42ed371668c2e70a1900c38c
171bb3b4c4a5cddd807d880095b932d6136043f45d52a6479c845b412e251ebf
1774b4da6e43135a7cbc32baece2a1e6e32d5e0f4988feaaa25bad7ea499152d
1b851a647fd99035eae4617269c162ddc0ca8088f0e45a42b9f3d917dd9ffbac
1cc2c183c985a12ffd90fcc22301ee5fc8924cdbaac7bb86aa9b3f605c35abcc
1e19e924a858513e6aa3312fb6c0ffe0aba3f7ff4a30f15579eb0c9745ca9f0d
1e38abda93a6eb21a156dfdad370df3d9aa07fce82bc0ae6f1db39ce35a58520
1f5f72ffb7ec30455b1238ff046a80619957b4a1ea90d6381df4f7260ced3cf8
2288ee26e57c0d86ab231111fb0853c2879ee6c2244f706cd754fc1f4aa4f6df
24d2e457991a4b25d11701ef84b0aa1a701b0a8bbe3931f9c3906c314755fd91
27e43265a72bd977462170c1aaa6eb587b69863be57e32bd9d96d5658d1a6c91
28e96f48d734371f3de2ce562c28ebc591a27763f6b3f721002cef5a43a370c8
2b70a0749b6df7529a37abed998dc5ac904d0bb59057b3a5af7242bb277c41a0
2c4c2a8df54c774c379179d243ba3f3b63948555b8e85bfb4a0bbee688116485
2e57dd0d327fecc770fcc99f20dcbf57dbffcc3ab1ba00d488136a011c638163
3235d384e0b57dd21b629902401560c997fae3630d1a5e5d44f6c7dc23e70b24
33da0e0b37012332a8e4fd115e84893bfa2708ab00aa3e40b44eedd345b4f904
37d2da5484651ba0bc0aa425b2a3029217a642f2f592f3ffb6dbb108b2216ea8
3804ce6bd27df78d651ae85fc5659bb303913ef600ecf91cabfbd3c9c797182f
399c6f4d1900f90b5d32e1e5ddfaf0739b2f069cec6aa84114771231a43aa08f
3a662d0984c17fbeaf5198bd6b20be5866810c652cca481c894f7cdc721d6340
442cb77a4b48fe3793975d545b71b1eeacc1df8fda34aee20a16f2db6e9758f3
444bbd61bba8eaedcc3fcf88d69982332d54f0b64cc7a2c1420cd365b11688b2
46616716f881b169815ad1afe5ec3cdebcc89bf3ae50637fd0b5fb8ac01217af
477a9f47c1c91cddfcdb38bd4f4b68e95fd99a04276d84dbd382620150915ec3
49e33478f626549cdb2c000a820af6e34705f1cbe8b623daa3952854134282c9
50ab467369eeda4735cc2c2219da44016ed98552e8699e20991538571b305388
52364301bebf6da1fb9330f7bfd29c1bd45df1202ea8a70121eb483ea3a481a1
53808cea7fc2878d50b4f983cf23e0aa685b169c2090b5eb3f41e4b5a1e55b76
56053032bcc45b55a16d484e58c0947a58ce158e1dc089dba7552500d49ad4ea
5614ee01349834181c19990869a934cd459b960d250443800b648a6066e07877
5782d6d8fd10557472172a01104b5f377ce91d57ff1b38bb3c983212e3cf8747
5854e2b9ad73a03e32c718cd35e8245960facba999ca431ed47b4298d85c0f27
58e4d5fe6ce12be326e8a45f4b65b471041e2937277038d6dc5bef2c4840be09
599feb545b52e9426933c8bbea51bfdf817c45ac92a5c7c90be45c76b6b5ee2b
5b672d3969d42e32f3575fa1962291d057545c40b00f5fd200ffe68be83eaee2
5d3237426b682a9d0e501eb9b29651d7021758ba763acef287ba8b65ae4aa10c
5de79f6af8fceb6d07b74a2c0086040b1290b9ddfe1485c7bc3a35c9e734868c
601cd339ce16e4f5d90403fdf6facf938310770e0e6c4fe33ab145f0829c0b8b
651712564ef713504b84edd17288d949dc1819923a3d8d4d454e6568b8edec46
653733cbc179da1bfb82ec5a73c4dadc94f4fa16a4a208fb3f3cd1d3d4a2c5fd
6971b431afc9a2507cf9c61f75b6b6d3c7f37f985523af08d6ff1ebeff6db6b6
69ea3e5e6912d49c32c7958efa0d9af843dff3b41a061f95ed32ef73efdcb7d7
6b4d9327010e2a32fbe8592545c480d3b56f55c035bc965adee4feda68a65dae
6b9731c07a7be7b7a70a360846e4c226a3f3697e6c3574c68a125b64b8fddbaa
6c1cb309b68664d00282027d1ba1a2acf957b1efecca681b4c4eac5bd71e66f2
6d6faaa1380cc76166e1de40fca37961ee556cfe7829a7ce0b4768206fc254d4
713793a0fd9f314787ab0987df569ebca008a8cef81556382e2b142ac54f9faf
72909647829fe10423ed58cb6b1bb25cf4361a8a30ce089b8bbcc56cef5e2385
7600e2c4dff3b2745d43e62675bfed1d676cccb48d230e55e6eeb46278f6259f
76e9418d6773677225fcbe95066de253a805150c03ef12353fc0db9b1977897d
76f85fe836a932a6d1431fcdd4afd6f22fef14d6d4ff72b7eec92f6fdb8b0009
76fad4931c66319a5751ce8747761c9dd56a2ade4d907b784f55749982245f29
789d789696a0daa5932cdbd5b021ebefd702026e6f9fdd10c60487cecf050129
7e7e314e491555efd254bc82aa1b8249c2d5565d5c2c3e048087588ea80997f9
7f30774fdeff3299c20c49dfe32fface87fbaf5cf0ff88f87db05c6b85307b89
85d9d0551f1c5bc8ab885b25e713f95695dffc22a9e3feb329902231896efed0
869811946de06c60d23a983592e607ec6568ea9d58332b96a703eb99cca708fe
8adcfb87b088c80f48f4ebf8f4d48a4fa179f2f29303d16bd695173fbb1c83b9
8be4972a4ff33bc9472dac276963d89669ecd937f2555a66691b567e40b50dbc
8c0c708b30c7b150d192dc3774327092e0fa3fb14956334c1e3d45c7a0a7d180
8c5ccf65bc699e123490e8acf8058effd49e839bed779f73b2fc2ebe0047d8c1
904c128123cd9b50d6196d9eecaf2b48c094ec2120a7b514ecfbba12ee620d44
93f4cb57f250cf6b8c784029b3c1de1e9c487284bdadd6df0de78ff7bba57d3f
9c8794042eb1d0677892b6a951a5d0f2c58a6886618add013631426358a5d933
a35620c323a01c1ac660eee1042235afda85316d49cc27522f17ae5255862990
a8249d527fa8b1ac3df0c46b5bb94360b0bedd04ff96d334580744199bcb1fdd
aef8fbf142955ddf1007a8f60d4839f38e5fd4380e4b99faf0b864341b141a94
af2cd66391faea2b5e52d9db535990a6164530a554d42557b62e5ece2e325bae
b2f926ff53c12f90e56c8d507bd201607aa2bc82c8316a0397042bd3b72a2c28
b3c6b23926968cf461baa5dd8ea945eb053261c7c27d6294ba725cb020ddb6a8
b573f573e49aa6653cd0698715c21169d2c4f9bf371f725283e69b575107135f
b76d37d5658cb5fae9eeba1ec5af8c716811b017ee4724545d37ce9b2a72a277
b7df6df15efe4c849c74fadd44c7115187ea01ce976bc608738821d7a0b4e73d
ba67bb3a2c61b9e25af95d2acb9d80e337c41fd60ffdb747f22df9f63b50a3ee
beed93a7f6b8abd789185f0d6a73bf8856848f2aa403f29e42b4ddf8f7fae9bf
c44c794bfe1ed9b017e4c93a7031dde8893d4aaaef68f4f9bbfe9121b3f73b78
c7d7213859ab7cdabcc40049aff5482ebbf1b9e92d6e65a376ea1d5e89787cf6
c7dd13490be7c7e638bbd16cc89f6099486b1778ee17f530116b660a405f5e6c
cd97d88a6d6863362ec58bec80a97ed89fc94292b4c2325a2439060b9fc6f2e3
d325b85126e4565493175a6cb48a60524bfeafe12a86af91c8a5946640758ad0
d95742e5f7278bd7d1a68ed42c74e03a9a84b736f83d1ea847b5a325af2d35f6
dad1ebbed3ea9cf6c701953e66b600f7355b8b43e0659e1fda718bda3502a647
dfabce7a9eda4d0e0f2f65e643eb9bc5054b82e58dba7206943c62a9d22229db
e1dcde46df2a08231398b4c3abdd316deb41af98b9522f3b03291f09f189a6b1
e2ec02436f52889d93736082fd68aab1968aefad43126248a35a08df4ec9ac3c
e439219388e62c8828274c4bf2fdc57b8f045dc893181c473def02aa20cd9ca0
e688e3029b5a0c8e7a2b26d3eafc90cc4256208cdf3fd42ef3afe4fbdea367cf
e713a6ef3b71d08b098d997dfb751be11092bb9cb9d5f9f355a70fef521adf9d
e7ae808388c228eab64a80b06f0ac3de0d5c436cf8e614fff8f22d848c68543d
e8d16c9ff6b75e2f25f60c19928e13173834600b3eb8a40066b18e5ac337aad1
eda591d1e7428ff097fd5ee27dcdb530cf85125c053ba7d0e0204e5fdb9f4a17
ee8c42c4eae844f6a790ec53e8371bd0bd3fa690f360fb4bd220539f78b9112a
eefd2f54edd310e2481ab0a0f4c2bee7ddaa126a6f04f4b4bdf05e8ac99ea3a5
f27e6ecc2ef24e60aafa709e2c7b9c096beedb5933431e04435ee39cc51acaa7
f401d58951145efcbc1b4a36ce439273068bad073f28b459ae767186c65477ee
f4b631574d3a65262f2fef7adaff609d8a476e66c09d53cca0802e7846c5bada
f91c2ad8dc044afabdaf82169a6c9fb16fa5a52c3818cb7fa4c987ebfb666b26
fb779840c44a694e9e53599633dbecc5b0ac5522a9e8478ed7d0f6bd02c776d4
fea18c11258f3ec945cfe4f500a055bcb5f0a6182df6186ca428c6fda911f59e
feed246b5b8fd7b55bd6b4ad942e3de80c1f04fc1b8446a615ae8a7310c9b3ed
3be5ec11aac7a2213113ce639fa2f581ec0a51dc1ec9c0ac13fa1c535e9e6d6c
9de0912121a4d1279baa2468fa0be4d8c9c9217c34c2595bd6ec4ae92e4cca0b
a2783ac3b6009b084c11101331c52be8935fb254a9d2b9dc086bd2e08be77790
2223405dfc53b72708e828d994c4c3745c61fd613531c3fa43151f377fea93a5
2e544c59b11a35057ebc368feedead48af1c3b58f188189a688450b55f951aa3
4dda60dced00b4b5488239129fbdafd938c0cea07ed01a21eebf339e6f2e5717
758012c55a6f557e378a1dd5842914e358a15a4286b10c49965ecb52a9872eb6
78627d24ed1d7361c1adee9b0845c58523c14f403d1174f58d5237e79365eea6
82a6591948c848c76849aaeea04c3f1b1db0f72c3a7218dc6d5b253c8e89eee3
a0f2559d94bc364d9b23665ada275e66b2664da25e3d4dac4ac3918a1d14d48c
a3ce4b7d417324de07b95d7451f0c8b29eeaaa7507039cc15fa8b3caa9d31990
c3afe8811e7a97b352da9e3f0c031daf2c5297f45eda4d5acefc83c305af39de
c948234f48b3e204a2d4a391083dff18124ea6c89a95b984f9de70dd72f3adb5
e9a7ec580606736f5e8049bbc7cb49fc1fc1d12977b41062a61ce335c03c8cf3
fb0406154764d4981354731ab8c4c675b42b1f2826c6f3a610808f69b1740b68


Description

New crash events were reported via Telemetry with newer versions (['16.2.9', '17.2.0']) than encountered in Tracker (16.2.7).

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=4f44183919f5fa8a88e924626a837b3ab94969ad75e38ce757ac8552938223f8

Assert condition: session_map.sessions.empty()
Assert function: virtual Monitor::~Monitor()

Sanitized backtrace:

    Monitor::~Monitor()
    Monitor::~Monitor()

Crash dump sample:
{
    "assert_condition": "session_map.sessions.empty()",
    "assert_file": "mon/Monitor.cc",
    "assert_func": "virtual Monitor::~Monitor()",
    "assert_line": 267,
    "assert_msg": "mon/Monitor.cc: In function 'virtual Monitor::~Monitor()' thread 7fe2f703d3c0 time 2022-06-23 20:34:09.659176\nmon/Monitor.cc: 267: FAILED ceph_assert(session_map.sessions.empty())",
    "assert_thread_name": "ceph-mon",
    "backtrace": [
        "(()+0x12890) [0x7fe2ed089890]",
        "(gsignal()+0xc7) [0x7fe2ec181e97]",
        "(abort()+0x141) [0x7fe2ec183801]",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x7fe2ee210483]",
        "(ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7fe2ee21060d]",
        "(Monitor::~Monitor()+0x9a6) [0x563352aaced6]",
        "(Monitor::~Monitor()+0x9) [0x563352aacf29]",
        "(main()+0x28b5) [0x563352a39ef5]",
        "(__libc_start_main()+0xe7) [0x7fe2ec164b97]",
        "(_start()+0x2a) [0x563352a69eaa]" 
    ],
    "ceph_version": "14.2.22",
    "crash_id": "2022-06-23_19:34:09.667889Z_ff5737b2-8529-4764-9f3c-24a76d9788ed",
    "entity_name": "mon.73f0abc217e3e5a0d8d1e7e4feef48a12b221b44",
    "os_id": "ubuntu",
    "os_name": "Ubuntu",
    "os_version": "18.04.4 LTS (Bionic Beaver)",
    "os_version_id": "18.04",
    "process_name": "ceph-mon",
    "stack_sig": "e7ae808388c228eab64a80b06f0ac3de0d5c436cf8e614fff8f22d848c68543d",
    "timestamp": "2022-06-23 19:34:09.667889Z",
    "utsname_machine": "x86_64",
    "utsname_release": "5.3.0-62-generic",
    "utsname_sysname": "Linux",
    "utsname_version": "#56~18.04.1-Ubuntu SMP Wed Jun 24 16:17:03 UTC 2020" 
}


Related issues

Related to RADOS - Bug #39150: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum Resolved
Related to RADOS - Bug #52142: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) Duplicate
Related to RADOS - Bug #52199: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) Duplicate
Related to RADOS - Bug #51882: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) Duplicate
Related to RADOS - Bug #52198: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) Duplicate
Related to RADOS - Backport #53659: pacific: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum Resolved
Related to RADOS - Backport #53660: octopus: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum Resolved
Copied to RADOS - Backport #63980: pacific: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) Resolved
Copied to RADOS - Backport #63981: reef: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) In Progress
Copied to RADOS - Backport #63982: quincy: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) In Progress

History

#1 Updated by Telemetry Bot almost 2 years ago

  • Related to Bug #39150: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum added

#2 Updated by Telemetry Bot almost 2 years ago

  • Related to Bug #52142: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#3 Updated by Telemetry Bot almost 2 years ago

  • Related to Bug #52199: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#4 Updated by Telemetry Bot almost 2 years ago

  • Related to Bug #51882: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#5 Updated by Telemetry Bot almost 2 years ago

  • Related to Bug #52198: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#6 Updated by Telemetry Bot almost 2 years ago

  • Related to Backport #53659: pacific: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum added

#7 Updated by Telemetry Bot almost 2 years ago

  • Related to Backport #53660: octopus: mon: "FAILED ceph_assert(session_map.sessions.empty())" when out of quorum added

#8 Updated by Telemetry Bot almost 2 years ago

  • Crash signature (v1) updated (diff)
  • Crash signature (v2) updated (diff)
  • Affected Versions v14.2.0, v14.2.1, v14.2.10, v14.2.11, v14.2.13, v14.2.16, v14.2.2, v14.2.22, v14.2.4, v14.2.5, v14.2.6, v14.2.7, v14.2.8, v15.2.0, v15.2.10, v15.2.11, v15.2.12, v15.2.13, v15.2.14, v15.2.15, v15.2.2, v15.2.3, v15.2.4, v15.2.5, v15.2.6, v15.2.7, v15.2.8, v15.2.9, v16.2.0, v16.2.2, v16.2.4, v16.2.5, v16.2.6, v16.2.7, v16.2.9, v17.2.0 added

#10 Updated by Telemetry Bot over 1 year ago

  • Crash signature (v1) updated (diff)
  • Affected Versions v15.2.16, v17.2.1 added

#12 Updated by Laura Flores about 1 year ago

/a/lflores-2023-03-27_02:17:31-rados-wip-aclamk-bs-elastic-shared-blob-save-25.03.2023-a-distro-default-smithi/7220990

#13 Updated by Telemetry Bot 11 months ago

  • Crash signature (v1) updated (diff)
  • Affected Versions v16.2.10, v17.2.3, v17.2.5 added

#14 Updated by Nitzan Mordechai 10 months ago

  • Assignee set to Nitzan Mordechai
  • Crash signature (v1) updated (diff)

New session connection are not checked before adding new session if the monitor is on shutdown.

#15 Updated by Nitzan Mordechai 10 months ago

  • Status changed from New to In Progress

#16 Updated by Nitzan Mordechai 10 months ago

  • Pull request ID set to 51868

#17 Updated by Nitzan Mordechai 10 months ago

  • Status changed from In Progress to Fix Under Review

#18 Updated by Konstantin Shalygin 3 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Target version set to v19.0.0
  • Backport set to pacific quincy reef

#19 Updated by Backport Bot 3 months ago

  • Copied to Backport #63980: pacific: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#20 Updated by Backport Bot 3 months ago

  • Copied to Backport #63981: reef: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#21 Updated by Backport Bot 3 months ago

  • Copied to Backport #63982: quincy: crash: virtual Monitor::~Monitor(): assert(session_map.sessions.empty()) added

#22 Updated by Backport Bot 3 months ago

  • Tags set to backport_processed

Also available in: Atom PDF