Project

General

Profile

Bug #20683

mon: HealthMonitor.cc: 216: FAILED assert(store_size > 0)

Added by Patrick Donnelly over 4 years ago. Updated over 4 years ago.

Status:
Rejected
Priority:
High
Assignee:
-
Category:
Monitor
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
multimds
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Assertion: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.1.0-1131-g5f5647a/rpm/el7/BUILD/ceph-12.1.0-1131-g5f5647a/src/mon/HealthMonitor.cc: 216: FAILED assert(store_size > 0)
ceph version 12.1.0-1131-g5f5647a (5f5647afc2ede502fb21e600268a5c2a2b6f8e9a) luminous (rc)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x6cd3a0]
 2: (HealthMonitor::check_member_health()+0x1481) [0x5152b1]
 3: (HealthMonitor::tick()+0xbe) [0x51649e]
 4: (Monitor::tick()+0x80) [0x49e250]
 5: (C_MonContext::finish(int)+0x37) [0x47e817]
 6: (Context::complete(int)+0x9) [0x4ba1a9]
 7: (SafeTimer::timer_thread()+0x104) [0x6c9c44]
 8: (SafeTimerThread::entry()+0xd) [0x6cb66d]
 9: (()+0x7dc5) [0xa7cbdc5]
 10: (clone()+0x6d) [0xd20776d]
1 jobs: ['1412993']
suites: ['clusters/fixed-2-ucephfs.yaml', 'frag_enable.yaml', 'fs/verify/{begin.yaml', 'mon-debug.yaml', 'mount/fuse.yaml', 'objectstore/filestore-xfs.yaml', 'overrides/{debug.yaml', 'tasks/cfuse_workunit_suites_fsstress.yaml', 'validater/valgrind.yaml}', 'whitelist_wrongly_marked_down.yaml}']

From: /ceph/teuthology-archive/pdonnell-2017-07-18_16:38:38-fs-master-distro-basic-smithi/1412993/teuthology.log


Related issues

Duplicated by RADOS - Bug #52150: crash: bool HealthMonitor::check_member_health(): assert(store_size > 0) Won't Fix

History

#1 Updated by Joao Eduardo Luis over 4 years ago

  • Status changed from New to Need More Info

monitor logs for the timeframe where the assert is triggered (and logged to teuthology.log) are missing. We have a few warnings from gzip stating the `file size changed while zipping`, so maybe this is why the logs seem to be missing?

#2 Updated by Jason Dillaman over 4 years ago

Hit this on my local dev box as well -- think it was from me deleting the store directory before killing off the monitor process between test runs. I can see that it would return zero if RocksDBStore::get_estimated_size attempts to open a non-existent directory.

#3 Updated by Joao Eduardo Luis over 4 years ago

  • Status changed from Need More Info to Rejected

If that's the root cause, I'm simply rejecting the ticket. I don't think we want to handle the store suddenly disappearing, and asserting seems an adequate response.

Feel free to reopen if this is reproduced on a different scenario.

#4 Updated by Neha Ojha about 2 months ago

  • Duplicated by Bug #52150: crash: bool HealthMonitor::check_member_health(): assert(store_size > 0) added

Also available in: Atom PDF