Project

General

Profile

Actions

Bug #36758

closed

aborts in rocksdb::TableFileName() in mimic-x upgrade test suite

Added by Kefu Chai over 5 years ago. Updated over 5 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
Category:
Correctness/Safety
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description


2018-11-10 16:33:54.245 7f1225c05180  0 ceph version 13.2.2-366-g3b1f636 (3b1f63653ad98126ed45b4ce6cb986de461d2d3a) mimic (stable), process ceph-mon, pid 7901
2018-11-10 16:33:54.249 7f1225c05180  4 rocksdb: RocksDB version: 5.13.0

2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb: Compression algorithms supported:
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kZSTDNotFinalCompression supported: 0
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kZSTD supported: 0
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kXpressCompression supported: 0
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kLZ4HCCompression supported: 1
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kLZ4Compression supported: 1
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kBZip2Compression supported: 0
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kZlibCompression supported: 1
2018-11-10 16:33:53.377 7f2a8b439180  4 rocksdb:        kSnappyCompression supported: 1
2018-11-10 16:33:54.249 7f1225c05180  4 rocksdb: Fast CRC32 supported: Supported on x86
...
2018-11-10 16:35:50.261 7f0e6989cec0  0 ceph version 14.0.1-732-g56412bf (56412bf6d2586a0178bb8eaca58b31702ae0e27e) nautilus (dev), process ceph-mon, pid 11167
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb: RocksDB version: 5.14.0
...
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb: [/build/ceph-14.0.1-732-g56412bf/src/rocksdb/db/version_set.cc:3096] Recovering from manifest file: MANIFEST-000005
...
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:   Options.memtable_huge_page_size: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                           Options.bloom_locality: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                    Options.max_successive_merges: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                Options.optimize_filters_for_hits: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                Options.paranoid_file_checks: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                Options.force_consistency_checks: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                Options.report_bg_io_stats: 0
2018-11-10 16:35:50.265 7f0e6989cec0  4 rocksdb:                               Options.ttl: 0
2018-11-10 16:35:50.269 7f0e57e44700 -1 *** Caught signal (Aborted) **
 in thread 7f0e57e44700 thread_name:ceph-mon

 ceph version 14.0.1-732-g56412bf (56412bf6d2586a0178bb8eaca58b31702ae0e27e) nautilus (dev)
 1: (()+0x12890) [0x7f0e5f9e9890]
 2: (gsignal()+0xc7) [0x7f0e5eadce97]
 3: (abort()+0x141) [0x7f0e5eade801]
 4: (()+0x3039a) [0x7f0e5eace39a]
 5: (()+0x30412) [0x7f0e5eace412]
 6: (rocksdb::TableFileName[abi:cxx11](std::vector<rocksdb::DbPath, std::allocator<rocksdb::DbPath> > const&, unsigned long, unsigned int)+0xf2) [0x55c22c88b022]
 7: (rocksdb::TableCache::GetTableReader(rocksdb::EnvOptions const&, rocksdb::InternalKeyComparator const&, rocksdb::FileDescriptor const&, bool, unsigned long, bool, rocksdb::HistogramImpl*, std::unique_ptr<rocksdb::TableReader, std::default_delete<rocksdb::TableReader> >*, rocksdb::SliceTransform const*, bool, int, bool, bool)+0xb4) [0x55c22c7b1614]
 8: (rocksdb::TableCache::FindTable(rocksdb::EnvOptions const&, rocksdb::InternalKeyComparator const&, rocksdb::FileDescriptor const&, rocksdb::Cache::Handle**, rocksdb::SliceTransform const*, bool, bool, rocksdb::HistogramImpl*, bool, int, bool)+0x34e) [0x55c22c7b209e]
 9: (std::_Function_handler<void (), rocksdb::VersionBuilder::Rep::LoadTableHandlers(rocksdb::InternalStats*, int, bool, rocksdb::SliceTransform const*)::{lambda()#1}>::_M_invoke(std::_Any_data const&)+0x92) [0x55c22c8ed692]
 10: (()+0xbe733) [0x7f0e5f503733]
 11: (()+0x76db) [0x7f0e5f9de6db]
 12: (clone()+0x3f) [0x7f0e5ebbf88f]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

http://pulpito.ceph.com/kchai-2018-11-10_15:23:32-upgrade:mimic-x-master-distro-basic-smithi/3245876/


Related issues 1 (0 open1 closed)

Is duplicate of RADOS - Bug #25146: "rocksdb: Corruption: Can't access /000000.sst" in upgrade:mimic-x:parallel-master-distro-basic-smithiResolvedRadoslaw Zarzynski07/28/2018

Actions
Actions #1

Updated by Kefu Chai over 5 years ago

  • Priority changed from Normal to Urgent

marking it "urgent", as it can be consistently reproducible. and it renders the cluster unusable after upgrading from mimic to nautilus.

Actions #2

Updated by Kefu Chai over 5 years ago

i think it's a dup of #25146

Actions #3

Updated by Brad Hubbard over 5 years ago

  • Status changed from New to Duplicate
  • Assignee set to Brad Hubbard
Actions #4

Updated by Brad Hubbard over 5 years ago

  • Is duplicate of Bug #25146: "rocksdb: Corruption: Can't access /000000.sst" in upgrade:mimic-x:parallel-master-distro-basic-smithi added
Actions

Also available in: Atom PDF