General

Profile

Igor Fedotov's activity

From 10/30/2020 to 11/28/2020

11/28/2020

12:00 AM bluestore Bug #48389: _do_read bdev-read failed
Seena Fallah wrote:
> You are right. It seems the disk has read error by itself and this occurs 3 times today and I'...
Igor Fedotov

11/27/2020

11:29 PM bluestore Bug #48389: _do_read bdev-read failed
Thanks for sharing!
Unfortunately too low debug level for bdev hence not much useful info.
Wondering if you're ab...
Igor Fedotov
09:47 PM bluestore Bug #48389: _do_read bdev-read failed
I think this is another form of https://tracker.ceph.com/issues/48276
And the root cause is presumably pretty the sa...
Igor Fedotov
04:04 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Seena Fallah wrote:
> Igor Fedotov wrote:
> > Seena Fallah wrote:
> > > Did QA and QE run on bitmap allocator too ...
Igor Fedotov
03:34 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Seena Fallah wrote:
> Did QA and QE run on bitmap allocator too in nautilus 14.2.14?
Sorry I'm not getting the qu...
Igor Fedotov
02:06 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Seena Fallah wrote:
> Just to prioritize this issue another OSD from my SSD tier fails :(
Mind switching to bitma...
Igor Fedotov

11/26/2020

07:31 PM bluestore Backport #47669 (In Progress): nautilus: Some structs aren't bound to mempools properly
https://github.com/ceph/ceph/pull/38310 Igor Fedotov
06:40 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Seena Fallah wrote:
> I faced this issue again in nautilus 14.2.14 and there is a log about the HybridAllocator
> [...
Igor Fedotov
06:32 PM bluestore Bug #48036: bluefs corrupted in a OSD
To troubleshoot 2) one might try the following:
- Create two containers that access a single shared folder from a ho...
Igor Fedotov

11/25/2020

12:11 PM bluestore Bug #48036: bluefs corrupted in a OSD
May be multiple containers attached to the same volume by some chance? Igor Fedotov
12:04 PM bluestore Bug #48036: bluefs corrupted in a OSD
You can double check the above by trying to run multiple OSD-0 instance in parallel manually. Highly likely they will... Igor Fedotov
12:02 PM bluestore Bug #48036: bluefs corrupted in a OSD
Hence presumable we have multiple ceph-osd instances using the same bluefs.
I can see at least two issues here. Both...
Igor Fedotov
11:58 AM bluestore Bug #48036: bluefs corrupted in a OSD
So my hypothesis about multiple kv_sync_thread-s is confirmed. Here is the log snippet from OSD log:
Thread 7faf0e...
Igor Fedotov

11/24/2020

06:00 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Bastian, thanks!
Igor Fedotov
11:57 AM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
It would be great if one try to grep failing OSD's logs (prior to the assertion) for "constructing fallback allocator... Igor Fedotov

11/23/2020

06:32 PM bluestore Bug #48036: bluefs corrupted in a OSD
@Satoru,
could you please reproduce the issue once again, now with both debug_bluefs set to 20 and debug_bluestore s...
Igor Fedotov
06:06 PM bluestore Bug #48036: bluefs corrupted in a OSD
Satoru Takeuchi wrote:
> @Igor
>
> Do you have any progress?
Hi Satoru,
sorry for a long response.
At the se...
Igor Fedotov
04:13 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
v14.2.11 has got hybrid allocator enabled but bluestore_volume_selection_policy was still at original there. Hence th... Igor Fedotov
02:06 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Thanks everybody for updates. Yeah I understand all the complexities for the debugging this sort of issues in a produ... Igor Fedotov
12:45 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
Meanwhile I see no way to troubleshoot this unless one is able to repro the issue with debug-bdev set to 20. Igor Fedotov
12:43 PM bluestore Bug #48276: OSD Crash with ceph_assert(is_valid_io(off, len))
The following patch once merged [and backported] will provide more insight on the issue's root cause.
https://gith...
Igor Fedotov

11/12/2020

05:21 PM bluestore Bug #48216: Spanning blobs list might have zombie blobs that aren't of use any more
Related PR to detect leaked spanning blobs and fix with fsck: https://github.com/ceph/ceph/pull/38050 Igor Fedotov
05:15 PM bluestore Bug #48216 (New): Spanning blobs list might have zombie blobs that aren't of use any more
As reported at https://tracker.ceph.com/issues/40449#note-9 users are still facing "no blob id" assertion. Provided l... Igor Fedotov
05:17 PM bluestore Backport #40449: nautilus: "no available blob id" assertion might occur
Nathan Cutler wrote:
> @Alexander - it might make sense to open a new bug in the Bluestore project for that, since t...
Igor Fedotov

11/02/2020

03:34 PM bluestore Bug #48036: bluefs corrupted in a OSD
@Satoru,
given you're able to reproduce the issue locally would you be able to collect OSD log (with debug-bluefs = ...
Igor Fedotov
12:10 PM bluestore Bug #48036: bluefs corrupted in a OSD
Hi Satoru,
thanks for the update.
Nevertheless I'm not completely sure whether bluefs-bdev-expand is a trigger for ...
Igor Fedotov
12:26 PM bluestore Bug #47751 (Pending Backport): Hybrid allocator might segfault when fallback allocator is present
Igor Fedotov
11:35 AM bluestore Bug #48025: osd start up failed when osd superblock crc fail
Bo Zhang wrote:
> Another bug also appears on the same node.(https://tracker.ceph.com/issues/48061)
This another ...
Igor Fedotov

10/30/2020

10:56 AM bluestore Bug #48036: bluefs corrupted in a OSD
Igor Fedotov wrote:
>
> Please set debug-bluestore & debug-bluefs to 20 and collect OSD startup log.
Never mind...
Igor Fedotov
10:41 AM bluestore Bug #48036: bluefs corrupted in a OSD
As far as I can see you're attempting to expand DB volume, weren't you? Any rationale for that?
Wasn't that a volum...
Igor Fedotov
10:41 AM bluestore Bug #48036: bluefs corrupted in a OSD
Both
https://tracker.ceph.com/issues/46886
and https://github.com/ceph/ceph/pull/36745
were following up the http...
Igor Fedotov
10:28 AM bluestore Bug #48025: osd start up failed when osd superblock crc fail
Bo Jang, I haven't got your last commends on disabled WAL, please elaborate.
From RocksDB config line I don't see ...
Igor Fedotov
10:24 AM RADOS Bug #47673: cephfs 4k randwrite + EC pool(2+1) + single node all OSDs OOM
鑫 王 wrote:
> *A slow IO will occur during execution.*
> I have another question why is the field buffer_anon also g...
Igor Fedotov
10:13 AM bluestore Bug #48047: osd: fix bluestore stupid allocator
IMO bdev_block_size should be marked with FLAG_STARTUP (or even FLAG_CREATE) and hence protected from the modificatio... Igor Fedotov
 

Also available in: Atom