Project

General

Profile

Activity

From 11/03/2022 to 12/02/2022

12/02/2022

01:30 AM Feature #57785: fragmentation score in metrics
I didn't know it was a problem until I tripped across it. The warning I think does more help then harm. Having a docu... Kevin Fox
12:43 AM Bug #58022: Fragmentation score rising by seemingly stuck thread
These run away osds are not on the heavy delete workload cluster. Its a relatively lightly loaded cluster. though I c... Kevin Fox

12/01/2022

08:44 PM Bug #58022: Fragmentation score rising by seemingly stuck thread
Igor/Adam - "But the behavior stops immediately on restart. So feels like some thread in the osd is doing something u... Vikhyat Umrao
03:14 PM Bug #58099 (Need More Info): ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixPreferDeferred/2 ...
Adam Kupczyk
07:55 AM Backport #58102 (In Progress): pacific: BlueStore doesn't defer small writes for pre-pacific hdd ...
Adam Kupczyk

11/30/2022

11:53 AM Bug #58113 (In Progress): BLK/Kernel: Improve protection against running one OSD twice
Igor Fedotov

11/29/2022

05:07 PM Bug #58113 (Resolved): BLK/Kernel: Improve protection against running one OSD twice
Vikhyat Umrao

11/28/2022

09:57 PM Backport #58103 (Resolved): quincy: BlueStore doesn't defer small writes for pre-pacific hdd osds
https://github.com/ceph/ceph/pull/49333 Backport Bot
09:56 PM Backport #58102 (Resolved): pacific: BlueStore doesn't defer small writes for pre-pacific hdd osds
https://github.com/ceph/ceph/pull/49170 Backport Bot
09:56 PM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Konstantin Shalygin wrote:
> Igor, PR should be replaced to 48490?
yep! done.
Igor Fedotov
09:56 PM Bug #56488 (Pending Backport): BlueStore doesn't defer small writes for pre-pacific hdd osds
Igor Fedotov
07:38 PM Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Igor, PR should be replaced to 48490? Konstantin Shalygin
07:17 PM Bug #58022: Fragmentation score rising by seemingly stuck thread
Newer picture, after I had just restarted the current batch of runaways. Kevin Fox
05:24 PM Bug #58022: Fragmentation score rising by seemingly stuck thread
11 more osds started doing this over the holiday weekend. Kevin Fox
06:23 PM Bug #58099 (Need More Info): ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixPreferDeferred/2 ...
/a/yuriw-2022-10-19_18:35:19-rados-wip-yuri10-testing-2022-10-19-0810-distro-default-smithi/7074995/... Laura Flores

11/27/2022

10:10 PM Feature #57785: fragmentation score in metrics
I think having the metric available opens the door for monitoring escalation for prometheus and less frequently used ... Paul Cuzner

11/23/2022

04:20 PM Bug #48216: Spanning blobs list might have zombie blobs that aren't of use any more
Gilles Mocellin wrote:
> Hello,
>
> No news on that ?
> Does someone knows if the problem also happens on Quincy...
Igor Fedotov
09:46 AM Bug #48216: Spanning blobs list might have zombie blobs that aren't of use any more
Hello,
No news on that ?
Does someone knows if the problem also happens on Quincy ?
Gilles Mocellin
04:12 PM Bug #54019: OSD::mkfs: ObjectStore::mkfs failed with error (5) Input/output error
Just made a topic for potential fix discussion at https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/CHVBMPENHO... Igor Fedotov

11/22/2022

07:01 PM Feature #57785: fragmentation score in metrics
❤️ Kevin Fox
06:52 PM Feature #57785: fragmentation score in metrics
After syncing with Adam Kupczyk today: 
In the shorter term we will make the fragmentation score, both for bluefs ...
Yaarit Hatuka

11/21/2022

08:17 PM Bug #58022: Fragmentation score rising by seemingly stuck thread
Saw this on 8 more osds over the weekend. Kevin Fox

11/17/2022

10:37 PM Feature #57785: fragmentation score in metrics
We have a meeting scheduled for next week to discuss this topic. Laura Flores
06:30 PM Feature #57785: fragmentation score in metrics
❤️ Kevin Fox
06:28 PM Feature #57785: fragmentation score in metrics
Thanks, Kevin. Let me talk this over with Adam and Paul, and we will decide a course of action. Laura Flores
06:15 PM Feature #57785: fragmentation score in metrics
A ceph warning for it would also be quite useful I think.
https://access.redhat.com/documentation/fr-fr/red_hat_ceph...
Kevin Fox
06:09 PM Feature #57785: fragmentation score in metrics
Thanks for sharing this, Kevin. We discussed this Tracker more in the Telemetry huddle, and we are curious if you wou... Laura Flores
05:11 PM Feature #57785: fragmentation score in metrics
We've had to hack a script together to monitor one of our clusters, and it has been useful to catch an issue:
https:...
Kevin Fox
04:25 PM Feature #57785: fragmentation score in metrics
@Kevin I have asked Paul Cuzner to take a look at this tracker and offer his opinion, as he has done a lot of work fo... Laura Flores

11/15/2022

09:56 AM Bug #48827: Ceph Bluestore OSDs fail to start on WAL corruption
In the customer case running luminous when OSD process was run twice (let's skip how), the assert
'file->fnode.ino' ...
Adam Kupczyk

11/14/2022

05:06 PM Bug #58022 (Pending Backport): Fragmentation score rising by seemingly stuck thread
Due to issue https://tracker.ceph.com/issues/57672 we've been monitoring our clusters closely ensure it doesn't run i... Kevin Fox
12:10 PM Bug #53466 (Fix Under Review): OSD is unable to allocate free space for BlueFS
Igor Fedotov

11/08/2022

09:20 PM Feature #57785: fragmentation score in metrics
@Vikhyat, no worries. Based on Kevin's comment, I think this metric might be better suited for Prometheus than Teleme... Laura Flores
06:37 PM Feature #57785: fragmentation score in metrics
Laura - sorry I missed the update. Can you please ping Adam and Igor? Vikhyat Umrao
07:37 PM Fix #54299 (Need More Info): osd error restart
Igor Fedotov
07:34 PM Bug #57672 (Duplicate): SSD OSD won't start after high framentation score!
Igor Fedotov
07:27 PM Bug #53466 (In Progress): OSD is unable to allocate free space for BlueFS
Igor Fedotov
 

Also available in: Atom