Project

General

Profile

Activity

From 07/17/2022 to 08/15/2022

08/15/2022

12:59 PM Bug #53002: crash BlueStore::Onode::put from BlueStore::TransContext::~TransContext
4) Quincy case from Telemetry: https://tracker.ceph.com/issues/56382
Igor Fedotov
12:57 PM Bug #53002 (New): crash BlueStore::Onode::put from BlueStore::TransContext::~TransContext
Looks like this hasn't been completely fixed yet.
We've got a bunch of new tickets from Telemetry bot which indicate...
Igor Fedotov

08/11/2022

04:16 PM Bug #53002: crash BlueStore::Onode::put from BlueStore::TransContext::~TransContext
according to https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/PPWIFPEI3EVBU3GQYYO6ABGF23WR5SGZ/ this ... Anonymous
11:23 AM Bug #56424 (Resolved): bluestore_cache_other mempool entry leak
Igor Fedotov
11:23 AM Backport #56599 (Resolved): pacific: octopus : bluestore_cache_other pool memory leak ?
Igor Fedotov
10:22 AM Bug #52705 (Resolved): pg scrub stat mismatch with special objects that have hash 'ffffffff'
Konstantin Shalygin

08/10/2022

08:07 AM Backport #49980 (Rejected): nautilus: BlueRocksEnv::GetChildren may pass trailing slashes to Blue...
Nautilus is EOL Konstantin Shalygin

08/09/2022

08:25 AM Bug #56503: Deleting large (~850gb) objects causes OSD to crash
Ok,
so here's what I did:
1. I moved .rgw.gc pool to different crush tree to isolate affected OSDs
2. I increased ...
Marcin Gibula
06:30 AM Bug #51682 (Resolved): bluestore repair might cause invalid write
Konstantin Shalygin
06:30 AM Backport #52766 (Rejected): nautilus: bluestore repair might cause invalid write
Nautilus is EOL Konstantin Shalygin

08/08/2022

08:32 AM Bug #47740 (Duplicate): OSD crash when increase pg_num
Igor Fedotov
08:30 AM Bug #54650 (Duplicate): crash: BlueStore::Onode::put()
Igor Fedotov
08:28 AM Bug #56200 (Duplicate): crash: ceph::buffer::ptr::release()
Igor Fedotov
08:26 AM Bug #54727 (Duplicate): crash: __pthread_mutex_lock()
Igor Fedotov

08/03/2022

03:40 PM Backport #57028 (Resolved): quincy: Bluefs might put an orpan op_update record in the log
Backport Bot
03:40 PM Backport #57027 (Resolved): pacific: Bluefs might put an orpan op_update record in the log
Backport Bot
03:35 PM Bug #56533 (Pending Backport): Bluefs might put an orpan op_update record in the log
Igor Fedotov
01:25 PM Bug #57015 (New): bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file)
/a/yuriw-2022-07-28_15:03:58-rados-wip-yuri3-testing-2022-07-27-1134-distro-default-smithi/6952123/... Kamoltat (Junior) Sirivadhna

08/02/2022

04:55 PM Backport #56669: pacific: Deferred writes might cause "rocksdb: Corruption: Bad table magic number"
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47296
merged
Yuri Weinstein

08/01/2022

05:23 PM Backport #56668: quincy: Deferred writes might cause "rocksdb: Corruption: Bad table magic number"
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47297
merged
Yuri Weinstein
08:34 AM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
This morning, I have :
PG_NOT_DEEP_SCRUBBED: 11 pgs not deep-scrubbed in time
Never had before Pacific.
Could it...
Gilles Mocellin

07/29/2022

02:12 PM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
I tried for a couple of hours, but it was worst than with 512k. I had a latency plateau and IOPS drops and rollback.
...
Gilles Mocellin
11:33 AM Bug #56503: Deleting large (~850gb) objects causes OSD to crash
Radoslaw Zarzynski wrote:
> Do you have logs with @debug_bluestore=20@ maybe?
>
> Mark, can this be related to th...
Igor Fedotov
11:18 AM Bug #56503: Deleting large (~850gb) objects causes OSD to crash
Ok, I'll try to reproduce this Marcin Gibula

07/28/2022

04:39 PM Backport #56598: quincy: octopus : bluestore_cache_other pool memory leak ?
https://github.com/ceph/ceph/pull/47339 Igor Fedotov
04:27 PM Backport #56598 (In Progress): quincy: octopus : bluestore_cache_other pool memory leak ?
Igor Fedotov
04:28 PM Backport #56599 (In Progress): pacific: octopus : bluestore_cache_other pool memory leak ?
https://github.com/ceph/ceph/pull/47337 Igor Fedotov
04:20 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
Correction, each test had site1 w/out BSZBD and the other with it enabled. The second round of tests were the same bu... Tim Wilkinson
04:18 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
Two rounds of testing have occurred, one with osd debug_bluestore=20 but this one did not reproduce the perf boost se... Tim Wilkinson
02:49 PM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Gilles Mocellin wrote:
> Increasing bluestore_prefer_deferred_size_hdd to 128k and even 512k helped us with slow ops...
Igor Fedotov
01:28 PM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Increasing bluestore_prefer_deferred_size_hdd to 128k and even 512k helped us with slow ops and reccurring high laten... Gilles Mocellin
02:23 AM Bug #56943 (New): crash: boost::intrusive::bstree_impl<boost::intrusive::bhtraits<BlueStore::Exte...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=46670c1166822eff93a5c43e...
Telemetry Bot
02:23 AM Bug #56942 (New): crash: rocksdb::DecodeFixed32(char const*)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=fff872db27320085d5bd3c4e...
Telemetry Bot
02:23 AM Bug #56941 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d483d1bd67a50d29a49a9c2f...
Telemetry Bot
02:23 AM Bug #56940 (New): crash: virtual int BlueStore::write_meta(const string&, const string&): assert(...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b90ad279d7f5c361a824a09d...
Telemetry Bot
02:23 AM Bug #56939 (New): crash: rocksdb::DataBlockIter::SeekToFirstImpl()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=455aa69e50b040c101f55d15...
Telemetry Bot
02:23 AM Bug #56936 (New): crash: rocksdb::DataBlockIter::SeekToFirstImpl()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=8590b7fc8f5415e7225d962b...
Telemetry Bot
02:23 AM Bug #56935 (New): crash: virtual void KernelDevice::aio_submit(IOContext*): assert(r == 0)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=0a624fe2e97fa1ffd88d50dc...
Telemetry Bot
02:23 AM Bug #56933 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5f0c1131ba4800f063920642...
Telemetry Bot
02:23 AM Bug #56932 (New): crash: rocksdb::WriteBatchInternal::Put(rocksdb::WriteBatch*, unsigned int, roc...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=f6f3ea231fc56e74e4e6d6bb...
Telemetry Bot
02:23 AM Bug #56931 (New): crash: rocksdb::MemTableIterator::NextAndGetResult(rocksdb::IterateResult*)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=8926109eb0d8ab91146ec193...
Telemetry Bot
02:23 AM Bug #56930 (New): crash: base::internal::SpinLockDelay(int volatile*, int, int)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b42e6a1680ce22b480b052d6...
Telemetry Bot
02:23 AM Bug #56929 (New): crash: cfree()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b89501efbb4c8751cb1ce6c5...
Telemetry Bot
02:23 AM Bug #56928 (New): crash: void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectSto...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5315149c952c81404277d576...
Telemetry Bot
02:23 AM Bug #56927 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5602ffe6853c07359f35c069...
Telemetry Bot
02:23 AM Bug #56924 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=f38c97119a825fde03f0ce8e...
Telemetry Bot
02:23 AM Bug #56922 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=33474006b60691e46972d8f4...
Telemetry Bot
02:23 AM Bug #56921 (New): crash: rocksdb::DecodeEntry::operator()(char const*, char const*, unsigned int*...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5d3dc9333da319125cf186c8...
Telemetry Bot
02:23 AM Bug #56918 (New): crash: pthread_mutex_lock()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=9b2e91f34aa28e7988d2fc0e...
Telemetry Bot
02:23 AM Bug #56917 (New): crash: pread64()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=ec0f636281801bfa8f889580...
Telemetry Bot
02:23 AM Bug #56916 (New): crash: syscall()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b768be65fbd8841719dcb117...
Telemetry Bot
02:23 AM Bug #56915 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=73d8a758380b04ecd772b14d...
Telemetry Bot
02:23 AM Bug #56914 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d535add5e715a2e23e813913...
Telemetry Bot
02:23 AM Bug #56913 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=56806c971618016752c15b1c...
Telemetry Bot
02:23 AM Bug #56912 (New): crash: int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=6b137275d89d84d4c60afd16...
Telemetry Bot
02:23 AM Bug #56911 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=3aeea5d607ac5d4ab0d80543...
Telemetry Bot
02:22 AM Bug #56910 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d171fa279d74eacb36d7da3c...
Telemetry Bot
02:22 AM Bug #56909 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=85ed5133055b8aa9722528b9...
Telemetry Bot
02:22 AM Bug #56908 (New): crash: virtual int BlueStore::write_meta(const string&, const string&): assert(...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d2e0dd05fe7eeb5c6d2902f6...
Telemetry Bot
02:22 AM Bug #56907 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5348e00696e4a251e00fb579...
Telemetry Bot
02:22 AM Bug #56905 (New): crash: LZ4_decompress_safe()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=0ab37cb2e4890193279d234d...
Telemetry Bot
02:22 AM Bug #56902 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=3eeb235b1f1fc2558adfd76b...
Telemetry Bot
02:22 AM Bug #56899 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=e5ca6db6653c7b124e7ee407...
Telemetry Bot
02:22 AM Bug #56893 (New): crash: ceph::buffer::ptr::iterator_impl<true>::operator

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a74bc1e9c988858f79475e99...
Telemetry Bot
02:22 AM Bug #56891 (New): crash: rocksdb::DBImpl::GetIntProperty(rocksdb::ColumnFamilyHandle*, rocksdb::S...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=93e4ff70cee45151fce093c7...
Telemetry Bot
02:22 AM Bug #56885 (New): crash: pread64()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=dcd9097d27b34cfe2a6cef37...
Telemetry Bot
02:22 AM Bug #56884 (New): crash: int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a33947a3ddbb26d8cf02c253...
Telemetry Bot
02:22 AM Bug #56882 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5aeccf665a5a7739826e62db...
Telemetry Bot
02:22 AM Bug #56881 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=456b14e92628089886ace9d4...
Telemetry Bot
02:22 AM Bug #56880 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=374d78417debf1f10bc9a262...
Telemetry Bot
02:21 AM Bug #56879 (New): crash: rocksdb::IteratorWrapperBase<rocksdb::Slice>::Next()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=58891b4c629de3f8d2b35794...
Telemetry Bot
02:21 AM Bug #56877 (New): crash: rocksdb::IteratorWrapperBase<rocksdb::Slice>::NextAndGetResult(rocksdb::...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7cd4caca1d19b0ab20067b88...
Telemetry Bot
02:21 AM Bug #56876 (New): crash: pread64()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=df09937ed2deb0251d707038...
Telemetry Bot
02:21 AM Bug #56875 (New): crash: BlueStore::Onode::put()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7362aab90e47739fdd28931f...
Telemetry Bot
02:21 AM Bug #56868 (New): crash: BlueStore::log_latency(char const*, int, std::chrono::duration<unsigned ...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=54840d56a916bf7899715821...
Telemetry Bot
02:21 AM Bug #56866 (New): crash: syscall()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=749895d8e1a0bce5973293f0...
Telemetry Bot
02:21 AM Bug #56859 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b1e23039e715e2ecdb7205a9...
Telemetry Bot
02:21 AM Bug #56857 (New): crash: BlueStore::Onode::put()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=2ecac47698645f1119e47c05...
Telemetry Bot
02:21 AM Bug #56854 (New): crash: std::unique_ptr<rocksdb::Block, std::default_delete<rocksdb::Block> >::g...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=32476aa8c69d6b64f8892f80...
Telemetry Bot
02:21 AM Bug #56853 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=f3d118a9511f61db150d57f1...
Telemetry Bot
02:21 AM Bug #56852 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7cc03cdc46ada57683950f95...
Telemetry Bot
02:21 AM Bug #56851 (New): crash: int BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::rea...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=825b54b05f03883919a608a9...
Telemetry Bot
02:20 AM Bug #56846 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=3785747dae8308b15b6d1ba6...
Telemetry Bot
02:20 AM Bug #56845 (New): crash: int64_t BlueFS::_read(BlueFS::FileReader*, uint64_t, size_t, ceph::buffe...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=be248808680ca921b9f51d1c...
Telemetry Bot
02:20 AM Bug #56838 (New): crash: rocksdb_cache::BinnedLRUHandleTable::FindPointer(rocksdb::Slice const&, ...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=66fd05516a21298e1ffcdfae...
Telemetry Bot
02:20 AM Bug #56834 (New): crash: ceph::PluginRegistry::get_with_load(std::basic_string<char, std::char_tr...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b386053a897d1eda5fc2198d...
Telemetry Bot
02:20 AM Bug #56829 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=f22692305dea7c0ae3001472...
Telemetry Bot
02:20 AM Bug #56828 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=5085905d74b5df92d8f9b9ee...
Telemetry Bot
02:20 AM Bug #56827 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=09acbd6c37d2afeb74cb077d...
Telemetry Bot
02:20 AM Bug #56825 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7b983065a737287fbb375b9f...
Telemetry Bot
02:20 AM Bug #56824 (New): crash: BlueStore::_txc_create(BlueStore::Collection*, BlueStore::OpSequencer*, ...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=6e74e61390a32760454390e7...
Telemetry Bot
02:20 AM Bug #56823 (New): crash: rocksdb::crc32c::Extend(unsigned int, char const*, unsigned long)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=b4832aa999ec40f147de90f6...
Telemetry Bot
02:20 AM Bug #56820 (New): crash: ceph::buffer::ptr::iterator_impl<true>::operator

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=4d59e458cfce7d94cf5b78cc...
Telemetry Bot
02:20 AM Bug #56819 (New): crash: pread64()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=4aba058b736b569ffe07a2cc...
Telemetry Bot
02:19 AM Bug #56818 (New): crash: pread64()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=2a14d56e3767b24d0345d846...
Telemetry Bot
02:19 AM Bug #56817 (New): crash: pthread_rwlock_rdlock()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=35078a6b8d06a938ebabc486...
Telemetry Bot
02:19 AM Bug #56812 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=bb50bfecaa2cde07d1f60942...
Telemetry Bot
02:19 AM Bug #56807 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=959ca98b98161f22904cba6d...
Telemetry Bot
02:19 AM Bug #56806 (New): crash: void BlueStore::_do_write_small(BlueStore::TransContext*, BlueStore::Col...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=c6aab93a57fed4eff9ca5a21...
Telemetry Bot
02:19 AM Bug #56805 (New): crash: void BlueStore::_do_write_small(BlueStore::TransContext*, BlueStore::Col...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=1c077f32e70f08fce239b6d3...
Telemetry Bot
02:19 AM Bug #56803 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=1562c3fb88628d327cf7bdad...
Telemetry Bot
02:19 AM Bug #56801 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=36aad439aac04165e757a34d...
Telemetry Bot
02:19 AM Bug #56800 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=0988c6e1fa90274309e110f0...
Telemetry Bot
02:18 AM Bug #56799 (New): crash: virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IO...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=275b1186a0c5a700ff350229...
Telemetry Bot
02:18 AM Bug #56798 (New): crash: HybridAllocator::init_rm_free(uint64_t, uint64_t)::<lambda(uint64_t, uin...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=ad1067445d53dcc60397ecf3...
Telemetry Bot
02:18 AM Bug #56795 (New): crash: void BlueStore::_do_write_small(BlueStore::TransContext*, BlueStore::Col...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=8bcd735b03b19a0102739f37...
Telemetry Bot
02:17 AM Bug #55187: ceph_abort_msg(\"bluefs enospc\")

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=80c8aa5fc17d3e51500a97e14...
Telemetry Bot
02:17 AM Bug #56791 (New): crash: bool SimpleBitmap::set(uint64_t, uint64_t): assert(offset + length < m_n...

*New crash events were reported via Telemetry with newer versions (['17.1.0']) than encountered in Tracker (0.0.0)....
Telemetry Bot
02:17 AM Bug #56790 (New): crash: virtual int KernelDevice::flush(): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=9aab05f010f83cdac4badc19...
Telemetry Bot
02:17 AM Bug #56788 (New): crash: void KernelDevice::_aio_thread(): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=c1fd5f4de1d415baece7e119...
Telemetry Bot
02:17 AM Bug #56784 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=e3abc74b74df1ddc00fb39b4...
Telemetry Bot
02:16 AM Bug #56783 (New): crash: virtual void KernelDevice::aio_submit(IOContext*): assert(r == 0)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=2005126bb96e53ddaba3cd74...
Telemetry Bot
02:16 AM Bug #56782 (New): crash: bool rocksdb::InlineSkipList<rocksdb::MemTableRep::KeyComparator const&>...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=4e383a9aa9a9b5ee75e87e66...
Telemetry Bot
02:16 AM Bug #56777 (New): crash: BlueStore::ExtentMap::fault_range(KeyValueDB*, uint32_t, uint32_t)::<lam...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=993546d443261ef33ab16d28...
Telemetry Bot
02:16 AM Bug #56775 (New): crash: void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectSto...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a854bd98ce38205d9909358c...
Telemetry Bot
02:16 AM Bug #56771 (New): crash: rocksdb::crc32c::crc32c_3way(unsigned int, char const*, unsigned long)

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d56c8a2c9634cec7ad800505...
Telemetry Bot
02:16 AM Bug #56769 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=76e8becc0b90eacf59295290...
Telemetry Bot
02:16 AM Bug #56768 (New): crash: void BlueStore::_do_write_small(BlueStore::TransContext*, BlueStore::Col...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7c8a3b629e9ce58810f204de...
Telemetry Bot
02:16 AM Bug #56767 (New): crash: int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=21ef9f7b9207c8d4a97c92c5...
Telemetry Bot
02:16 AM Bug #56766 (New): crash: int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=de4f4fe5831504c2902d7467...
Telemetry Bot
02:15 AM Bug #56765 (New): crash: int64_t BlueFS::_read_random(BlueFS::FileReader*, uint64_t, uint64_t, ch...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=61793a7ae77a9f18bb1bc4d3...
Telemetry Bot
02:15 AM Bug #56763 (New): crash: int BlueStore::_do_gc(BlueStore::TransContext*, BlueStore::CollectionRef...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=003e6c8b8938e88311f0a28c...
Telemetry Bot
02:15 AM Bug #56762 (New): crash: virtual int RocksDBStore::get(const string&, const char*, size_t, ceph::...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=821d6920aa8c1012875df41d...
Telemetry Bot
02:15 AM Bug #56760 (New): crash: virtual int KernelDevice::flush(): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a142a97ca2b72bcecf8e9bb3...
Telemetry Bot
02:14 AM Bug #56758 (New): crash: int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t): abort

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a3bf1e9df33bea667de16747...
Telemetry Bot
02:14 AM Bug #56757 (New): crash: void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectSto...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=a85cb534c8bc7cd818e51dda...
Telemetry Bot
02:14 AM Bug #56753 (New): crash: virtual int RocksDBStore::get(const string&, const char*, size_t, ceph::...

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=d666fd63a7dfdb77830d510a...
Telemetry Bot
02:14 AM Bug #56751 (New): crash: pthread_cond_wait()

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=c1f4d2190a3e2785c3d243a5...
Telemetry Bot
12:25 AM Bug #55328: OSD crashed due to checksum error
> After my last report, I was suffering from our testing environment changes,
Unfortunately, since our test envir...
Shinya Hayashi

07/27/2022

10:43 AM Backport #56668 (In Progress): quincy: Deferred writes might cause "rocksdb: Corruption: Bad tabl...
Adam Kupczyk

07/26/2022

06:49 PM Bug #56706 (Closed): _do_thrash:test_bluestore_reshard:Exception: ceph-bluestore-tool resharding...
closed because this is yet merged to main Kamoltat (Junior) Sirivadhna
03:55 PM Bug #48819: fsck error: found stray (per-pg) omap data on omap_head
/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6944045/ Kamoltat (Junior) Sirivadhna

07/25/2022

09:09 PM Bug #56706: _do_thrash:test_bluestore_reshard:Exception: ceph-bluestore-tool resharding failed
Dead job:
/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6943775
Kamoltat (Junior) Sirivadhna
08:10 PM Bug #56706 (Closed): _do_thrash:test_bluestore_reshard:Exception: ceph-bluestore-tool resharding...
/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6944099... Kamoltat (Junior) Sirivadhna
07:46 PM Bug #48819: fsck error: found stray (per-pg) omap data on omap_head
/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6943842/ Kamoltat (Junior) Sirivadhna

07/22/2022

04:18 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Sébastien Bernard wrote:
> We spent full last day to recover 4 osds that wouldn't come back up.
> We saw on the mac...
Igor Fedotov
03:16 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
We spent full last day to recover 4 osds that wouldn't come back up.
We saw on the machines a lot of errors like thi...
Sébastien Bernard
11:45 AM Bug #56488 (Fix Under Review): BlueStore doesn't defer small writes for pre-pacific hdd osds
Igor Fedotov

07/21/2022

10:55 PM Backport #56669 (Resolved): pacific: Deferred writes might cause "rocksdb: Corruption: Bad table ...
https://github.com/ceph/ceph/pull/47296 Backport Bot
10:55 PM Backport #56668 (Resolved): quincy: Deferred writes might cause "rocksdb: Corruption: Bad table m...
https://github.com/ceph/ceph/pull/47297 Backport Bot
10:54 PM Bug #54547 (Pending Backport): Deferred writes might cause "rocksdb: Corruption: Bad table magic ...
Neha Ojha
10:48 PM Bug #54547: Deferred writes might cause "rocksdb: Corruption: Bad table magic number"
https://github.com/ceph/ceph/pull/46890 merged Yuri Weinstein

07/20/2022

06:31 PM Bug #56503: Deleting large (~850gb) objects causes OSD to crash
Do you have logs with @debug_bluestore=20@ maybe?
Mark, can this be related to the RocksDB's tombstones issues?
Radoslaw Zarzynski
04:58 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
There are two test cases that would be executed to find more details on what is going on with this feature and how it... Vikhyat Umrao

07/19/2022

09:07 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
Vikhyat Umrao wrote:
> This has been already verified that by default COSbench uses random not zero.
>
> COSBench...
Vikhyat Umrao
09:06 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
The `bluestore_zero_block_detection` was set to false in 17.2.1. For more details please check:
https://tracker.ce...
Vikhyat Umrao
08:55 PM Bug #56640: RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17.2.1
This has been already verified that by default COSbench uses random not zero.
COSBench document - page 50 - https:...
Vikhyat Umrao
08:53 PM Bug #56640 (New): RGW S3 workload has a huge performance boost in quincy 17.2.0 as compared to 17...
RGW S3 *small object* workload has a huge performance boost in the quincy 17.2.0 as compared to 17.2.1 due to bluesto... Vikhyat Umrao
04:33 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Well, this doesn't look like a bluestore issue anymore. Neither I can't see any clues it's the same as the original o... Igor Fedotov
12:53 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Igor Fedotov wrote:
> Hi!
>
> May I have some clarifications then.
> I can see multiple backtraces for .107 like...
Aurélien Le Clainche
10:33 AM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Hi!
May I have some clarifications then.
I can see multiple backtraces for .107 like the last one. All are happen...
Igor Fedotov
08:29 AM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Aurélien Le Clainche wrote:
> Igor Fedotov wrote:
> > Aurélien Le Clainche wrote:
> > > Igor Fedotov wrote:
> > >...
Aurélien Le Clainche

07/18/2022

02:24 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Igor Fedotov wrote:
> Aurélien Le Clainche wrote:
> > Igor Fedotov wrote:
> > > Aurélien Le Clainche wrote:
> > >...
Aurélien Le Clainche
01:58 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Aurélien Le Clainche wrote:
> Igor Fedotov wrote:
> > Aurélien Le Clainche wrote:
> > > do you have a procedure or...
Igor Fedotov
01:01 PM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Igor Fedotov wrote:
> Aurélien Le Clainche wrote:
> > do you have a procedure or example to do the fsck?
>
> Not...
Aurélien Le Clainche
09:56 AM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Aurélien Le Clainche wrote:
> do you have a procedure or example to do the fsck?
Not sure how to do that with Roo...
Igor Fedotov
09:17 AM Bug #56456: rook-ceph-v1.9.5: ceph-osd crash randomly
Sébastien Bernard wrote:
> Regarding this problem, /var filesystem is an xfs filesystem.
> osd are setup by rook.
...
Aurélien Le Clainche
09:10 AM Backport #56600 (Rejected): octopus: octopus : bluestore_cache_other pool memory leak ?
Backport Bot
09:10 AM Backport #56599 (Resolved): pacific: octopus : bluestore_cache_other pool memory leak ?
Backport Bot
09:10 AM Backport #56598 (Resolved): quincy: octopus : bluestore_cache_other pool memory leak ?
Backport Bot
09:07 AM Bug #56424 (Pending Backport): bluestore_cache_other mempool entry leak
Igor Fedotov
09:05 AM Bug #56424: bluestore_cache_other mempool entry leak
alexandre derumier wrote:
> Hi,
>
> I think it's fixing the problem.
>
> I'll keep it running, and I'll send n...
Igor Fedotov
 

Also available in: Atom