Project

General

Profile

Activity

From 09/11/2018 to 10/10/2018

10/10/2018

07:57 PM Bug #36364: Bluestore OSD IO Hangs near Flush (flush in 90.330556)
To clarify the behaviour I see on iostat... The disk %util of a hung disk goes to 100%, while the average queue lengt... Gavin Baker
09:06 AM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
Reporting back, increasing min_free_kbytes has not appeared to have helped. Swap usage is only a couple of MB out of ... Nick Fisk

10/09/2018

08:04 PM Bug #36364: Bluestore OSD IO Hangs near Flush (flush in 90.330556)
We used the default configuration for the Bluestore OSD creation, so all DB/WAL on the OSDs. Our cluster does not con... Gavin Baker
07:49 PM Bug #36364: Bluestore OSD IO Hangs near Flush (flush in 90.330556)
Gavin,
some questions, please.
What kind of disks are used for BlueStore?
What's the HW config for OSD nodes?
...
Igor Fedotov
07:42 PM Bug #36364 (Can't reproduce): Bluestore OSD IO Hangs near Flush (flush in 90.330556)
Our Mimic Ceph cluster has been having an issue with intermittent slow OPs (metadata, etc) that seem to hang various ... Gavin Baker

10/08/2018

11:49 PM Backport #36146 (In Progress): mimic: fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24480 Prashant D
09:42 PM Feature #36231 (In Progress): cli options for ceph journal migration to different ssd/nvme
Igor Fedotov
10:02 AM Bug #35971 (Resolved): bloom filter num entry miscalculation in bluestore repairer
Nathan Cutler
10:02 AM Backport #36130 (Resolved): mimic: bloom filter num entry miscalculation in bluestore repairer
Nathan Cutler

10/06/2018

12:07 AM Bug #36331 (Can't reproduce): FAILED ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixNoCsum/2...
... Neha Ojha

10/05/2018

09:19 PM Backport #36130: mimic: bloom filter num entry miscalculation in bluestore repairer
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24339
merged
Yuri Weinstein
09:19 PM Bug #36268: Unable to recover from ENOSPC in BlueFS
https://github.com/ceph/ceph/pull/24352 merged Yuri Weinstein

10/04/2018

08:01 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
Update from our side: we've been running the patch I've posted above since Mimic for all our production clusters. No ... Paul Emmerich

10/03/2018

09:56 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
I've just bit hit by a wave of these after upgrading to Mimic, everything else remains the same, no reboot was carrie... Nick Fisk
03:59 PM Bug #36303 (Duplicate): luminous: 12.2.8 - FAILED assert(0 == "put on missing extent (nothing bef...
Same/similar as reported on http://tracker.ceph.com/issues/24715, after removing several rbd snapshots I'm getting an... Ricardo Barberis

10/01/2018

09:05 PM Bug #36284 (Duplicate): Bluestore might be hanging OSD

/a/dzafman-2018-09-26_22:31:44-rados-wip-zafman-testing-distro-basic-smithi/3074689
Bluetooth stuck in a loop st...
David Zafman
01:58 PM Bug #36268 (Resolved): Unable to recover from ENOSPC in BlueFS
Under heavy load and full DB volume BlueStore might fall into the state where it lacks additional space for BlueFS ev... Igor Fedotov

09/28/2018

08:07 PM Backport #36130 (In Progress): mimic: bloom filter num entry miscalculation in bluestore repairer
Igor Fedotov
06:47 PM Backport #36130: mimic: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24339 Igor Fedotov

09/26/2018

11:25 PM Feature #36231 (Resolved): cli options for ceph journal migration to different ssd/nvme
we dont have procedure on how to move the journal/wal in case of bluestore/filestore from local hdd to different ssd/... Vasu Kulkarni

09/25/2018

09:43 AM Bug #24906: fio with bluestore crushed
Minghao Cong wrote:
> I also met this bug on master branch.
>
> Have you solved it?
>
> 0.0
not yet
Honggang Yang

09/24/2018

11:01 AM Backport #36146 (Resolved): mimic: fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24480 Nathan Cutler
11:01 AM Backport #36145 (Resolved): luminous: fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24705 Nathan Cutler
11:00 AM Backport #36130 (Resolved): mimic: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24339 Nathan Cutler

09/22/2018

04:25 PM Bug #36099 (Resolved): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueSt...
Kefu Chai

09/21/2018

01:49 PM Bug #36108 (Duplicate): Assertion due to ENOENT result on clonerange2
All my OSDs started crashing after addding a couple of new OSDs into the cluster. They just keep coming up and down. ... Vladimír Kincl
12:01 PM Bug #36099 (Fix Under Review): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestor...
https://github.com/ceph/ceph/pull/24220 Kefu Chai
12:00 PM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
Jianpeng, thanks for the analysis. i don't think you missed anything. it's just good timing =) Kefu Chai
10:16 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
Add in hobjec_t.h... jianpeng ma
10:13 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
using thi command "./bin/ceph_test_objectstore --gtest_catch_exceptions=0 --debug-bluestore=20 --log-to-stderr=true ... jianpeng ma
04:29 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
could be a regression introduced by https://github.com/ceph/ceph/pull/22739 Kefu Chai

09/20/2018

07:14 PM Bug #25050: osd: OSD Failed to Start In function 'int BlueStore::_do_alloc_write
Issue is still unresolved, OSDs keep crushing when compression is enabled, and it is not coming, only when we disable... Yohay Azulay
01:28 PM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
/a/sage-2018-09-19_18:44:57-rados-wip-sage4-testing-2018-09-19-1054-distro-basic-smithi/3043638 Sage Weil
01:27 PM Bug #36099 (Resolved): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueSt...
... Sage Weil

09/19/2018

10:06 AM Bug #35971 (Pending Backport): bloom filter num entry miscalculation in bluestore repairer
Kefu Chai
10:02 AM Bug #32731 (Pending Backport): fsck: cid is improperly matched to oid
Kefu Chai

09/18/2018

10:15 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
After more dig into, I think it still a bluestore issue.
The suicide thread which timed out stuck when call BlueStor...
Michael Yang

09/14/2018

06:12 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I'm sorry but no the segfaults do not contain more lines or informations.
i started to monitor all segfaults now. ...
Stefan Priebe
08:44 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I think I got the reason about this issue:
1). One pg scrub start
2). The scrub job doesn't finished in 15s(osd_op_...
Michael Yang
04:02 AM Bug #24906: fio with bluestore crushed

I also met this bug on master branch.
Have you solved it?
0.0
Minghao Cong

09/13/2018

06:32 PM Bug #32731 (Fix Under Review): fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24085 Sage Weil
02:23 PM Bug #35971: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24076 Igor Fedotov
02:22 PM Bug #35971 (Resolved): bloom filter num entry miscalculation in bluestore repairer
This could cause an assertion due to an access to uninitialized bloom
filter. This happened when detected errors inv...
Igor Fedotov
01:41 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I've just submitted a PR to fix a bug in BlueStore repairer which Troy faced a while ago (This isn't this ticket fix ... Igor Fedotov
01:39 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Thanks, Stefan.
But it looks like call stack is a bit different in you case:
BlueStore::_txc_write_nodes function v...
Igor Fedotov
01:31 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Sure. Currently i don't get a new segault.
Here we go:
Sep 07 18:46:53 cloud1-1468 ceph-osd[11765]: *** Caught si...
Stefan Priebe
11:09 AM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Stefan, just in case - would you mind to share existing crash logs, please? Given that fsck detects no error it might... Igor Fedotov
10:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I found it's easy to reproduce this issue in my cluster with below steps:
1. Choose one pg which had caused this iss...
Michael Yang
06:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
Sage Weil wrote:
> Is there anything in the kernel log? These sorts of timeouts usually are caused by a media error...
Michael Yang
07:57 AM Bug #20870: OSD compression: incorrect display of the used disk space
Sage Weil wrote:
> The problem is that currently the RAW USED stats is just USED * (replications or ec factor).
>...
Lei Liu

09/12/2018

03:31 PM Bug #34526 (Need More Info): OSD crash in KernelDevice::direct_read_unaligned while scrubbing
Is there anything in the kernel log? These sorts of timeouts usually are caused by a media error or other hardware i... Sage Weil

09/11/2018

06:13 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I see. What about in-memory logging (debug_bluestore=0/20)? Only predefined number of recent debugs is stored and the... Radoslaw Zarzynski
05:31 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
i don't know how to crash the OSD it happens once out of nothing... and i think i can't run debug_bluestore for a few... Stefan Priebe
05:22 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
fsck has found nothing:... Radoslaw Zarzynski
10:33 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
After I first set osd nodown, then do osd scrub, everything work fine.
I have no idea about why this happen...
Be...
Michael Yang
06:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I found this dump related to scrub thread timeout, not related to bluestore.
There are such log in the osd log:
201...
Michael Yang
 

Also available in: Atom