Project

General

Profile

Activity

From 09/04/2018 to 10/03/2018

10/03/2018

09:56 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
I've just bit hit by a wave of these after upgrading to Mimic, everything else remains the same, no reboot was carrie... Nick Fisk
03:59 PM Bug #36303 (Duplicate): luminous: 12.2.8 - FAILED assert(0 == "put on missing extent (nothing bef...
Same/similar as reported on http://tracker.ceph.com/issues/24715, after removing several rbd snapshots I'm getting an... Ricardo Barberis

10/01/2018

09:05 PM Bug #36284 (Duplicate): Bluestore might be hanging OSD

/a/dzafman-2018-09-26_22:31:44-rados-wip-zafman-testing-distro-basic-smithi/3074689
Bluetooth stuck in a loop st...
David Zafman
01:58 PM Bug #36268 (Resolved): Unable to recover from ENOSPC in BlueFS
Under heavy load and full DB volume BlueStore might fall into the state where it lacks additional space for BlueFS ev... Igor Fedotov

09/28/2018

08:07 PM Backport #36130 (In Progress): mimic: bloom filter num entry miscalculation in bluestore repairer
Igor Fedotov
06:47 PM Backport #36130: mimic: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24339 Igor Fedotov

09/26/2018

11:25 PM Feature #36231 (Resolved): cli options for ceph journal migration to different ssd/nvme
we dont have procedure on how to move the journal/wal in case of bluestore/filestore from local hdd to different ssd/... Vasu Kulkarni

09/25/2018

09:43 AM Bug #24906: fio with bluestore crushed
Minghao Cong wrote:
> I also met this bug on master branch.
>
> Have you solved it?
>
> 0.0
not yet
Honggang Yang

09/24/2018

11:01 AM Backport #36146 (Resolved): mimic: fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24480 Nathan Cutler
11:01 AM Backport #36145 (Resolved): luminous: fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24705 Nathan Cutler
11:00 AM Backport #36130 (Resolved): mimic: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24339 Nathan Cutler

09/22/2018

04:25 PM Bug #36099 (Resolved): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueSt...
Kefu Chai

09/21/2018

01:49 PM Bug #36108 (Duplicate): Assertion due to ENOENT result on clonerange2
All my OSDs started crashing after addding a couple of new OSDs into the cluster. They just keep coming up and down. ... Vladimír Kincl
12:01 PM Bug #36099 (Fix Under Review): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestor...
https://github.com/ceph/ceph/pull/24220 Kefu Chai
12:00 PM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
Jianpeng, thanks for the analysis. i don't think you missed anything. it's just good timing =) Kefu Chai
10:16 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
Add in hobjec_t.h... jianpeng ma
10:13 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
using thi command "./bin/ceph_test_objectstore --gtest_catch_exceptions=0 --debug-bluestore=20 --log-to-stderr=true ... jianpeng ma
04:29 AM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
could be a regression introduced by https://github.com/ceph/ceph/pull/22739 Kefu Chai

09/20/2018

07:14 PM Bug #25050: osd: OSD Failed to Start In function 'int BlueStore::_do_alloc_write
Issue is still unresolved, OSDs keep crushing when compression is enabled, and it is not coming, only when we disable... Yohay Azulay
01:28 PM Bug #36099: ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueStore.cc: 589...
/a/sage-2018-09-19_18:44:57-rados-wip-sage4-testing-2018-09-19-1054-distro-basic-smithi/3043638 Sage Weil
01:27 PM Bug #36099 (Resolved): ObjectStore/StoreTest.BluestoreRepairTest/2 fails with os/bluestore/BlueSt...
... Sage Weil

09/19/2018

10:06 AM Bug #35971 (Pending Backport): bloom filter num entry miscalculation in bluestore repairer
Kefu Chai
10:02 AM Bug #32731 (Pending Backport): fsck: cid is improperly matched to oid
Kefu Chai

09/18/2018

10:15 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
After more dig into, I think it still a bluestore issue.
The suicide thread which timed out stuck when call BlueStor...
Michael Yang

09/14/2018

06:12 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I'm sorry but no the segfaults do not contain more lines or informations.
i started to monitor all segfaults now. ...
Stefan Priebe
08:44 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I think I got the reason about this issue:
1). One pg scrub start
2). The scrub job doesn't finished in 15s(osd_op_...
Michael Yang
04:02 AM Bug #24906: fio with bluestore crushed

I also met this bug on master branch.
Have you solved it?
0.0
Minghao Cong

09/13/2018

06:32 PM Bug #32731 (Fix Under Review): fsck: cid is improperly matched to oid
https://github.com/ceph/ceph/pull/24085 Sage Weil
02:23 PM Bug #35971: bloom filter num entry miscalculation in bluestore repairer
https://github.com/ceph/ceph/pull/24076 Igor Fedotov
02:22 PM Bug #35971 (Resolved): bloom filter num entry miscalculation in bluestore repairer
This could cause an assertion due to an access to uninitialized bloom
filter. This happened when detected errors inv...
Igor Fedotov
01:41 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I've just submitted a PR to fix a bug in BlueStore repairer which Troy faced a while ago (This isn't this ticket fix ... Igor Fedotov
01:39 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Thanks, Stefan.
But it looks like call stack is a bit different in you case:
BlueStore::_txc_write_nodes function v...
Igor Fedotov
01:31 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Sure. Currently i don't get a new segault.
Here we go:
Sep 07 18:46:53 cloud1-1468 ceph-osd[11765]: *** Caught si...
Stefan Priebe
11:09 AM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Stefan, just in case - would you mind to share existing crash logs, please? Given that fsck detects no error it might... Igor Fedotov
10:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I found it's easy to reproduce this issue in my cluster with below steps:
1. Choose one pg which had caused this iss...
Michael Yang
06:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
Sage Weil wrote:
> Is there anything in the kernel log? These sorts of timeouts usually are caused by a media error...
Michael Yang
07:57 AM Bug #20870: OSD compression: incorrect display of the used disk space
Sage Weil wrote:
> The problem is that currently the RAW USED stats is just USED * (replications or ec factor).
>...
Lei Liu

09/12/2018

03:31 PM Bug #34526 (Need More Info): OSD crash in KernelDevice::direct_read_unaligned while scrubbing
Is there anything in the kernel log? These sorts of timeouts usually are caused by a media error or other hardware i... Sage Weil

09/11/2018

06:13 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I see. What about in-memory logging (debug_bluestore=0/20)? Only predefined number of recent debugs is stored and the... Radoslaw Zarzynski
05:31 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
i don't know how to crash the OSD it happens once out of nothing... and i think i can't run debug_bluestore for a few... Stefan Priebe
05:22 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
fsck has found nothing:... Radoslaw Zarzynski
10:33 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
After I first set osd nodown, then do osd scrub, everything work fine.
I have no idea about why this happen...
Be...
Michael Yang
06:41 AM Bug #34526: OSD crash in KernelDevice::direct_read_unaligned while scrubbing
I found this dump related to scrub thread timeout, not related to bluestore.
There are such log in the osd log:
201...
Michael Yang

09/10/2018

07:09 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
# ceph-bluestore-tool fsck --path /var/lib/ceph/osd/ceph-36
fsck success
there was no more output?
Stefan Priebe
06:05 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Let's start from the plain fsck on the affected OSD and 12.2.7. Radoslaw Zarzynski
06:00 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Yes 12.2.2 and 12.2.5. So i sould just run fsck on the affected osd? Do i need 12.2.8 before? Stefan Priebe
05:53 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
Stefan, has the crashing OSD seen anything older than 12.2.6?
fsck is supposed to help in such cases. Take a look ...
Radoslaw Zarzynski
05:35 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
+1 i have the same crashes running 12.2.7. Is there anything i can do now? Stefan Priebe

09/07/2018

06:58 PM Bug #25180 (Resolved): ObjectStore/StoreTest.CompressionTest/2 fail
fixed by 044ce83b1e032b5996cad10bc1cf83918f991550 Sage Weil
01:55 AM Bug #24639: [segfault] segfault in BlueFS::read
I think i have the exact same problem with one of my OSD's, which were created with ceph-deploy on Ceph v12.2.7 and f... Compile Nix

09/06/2018

04:49 PM Bug #25098 (In Progress): Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(p...
Adding the check to BlueStore.
IIUC ceph-volume started creating the OSD's directory dynamically and this applies al...
Radoslaw Zarzynski
04:00 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
We discussed this in standup. I think there are 2 things going on here:
1. the symlink disappeared in Alfredo's ...
Sage Weil
06:07 AM Backport #26944 (Resolved): mimic: os/bluestore/BlueStore.cc: 1025: FAILED assert(buffer_bytes >=...
Nathan Cutler

09/05/2018

09:30 PM Backport #26944: mimic: os/bluestore/BlueStore.cc: 1025: FAILED assert(buffer_bytes >= b->length)...
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/23748
merged
Yuri Weinstein
05:09 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Looks like BlueStore did the mkfs stage with the <code class="cpp">block.db</code></pre> on a separated device which ... Radoslaw Zarzynski
04:13 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Was able to narrow down this problem: the OSD was deployed with a block.db in a different device that didn't come up ... Alfredo Deza
02:08 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Just a thought - may be superblock has been overwritten by the parallel deployment or something? Igor Fedotov
02:07 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
The problem appeared during decoding of BlueFS' superblock deeply im the low-level decoding bits.... Radoslaw Zarzynski

09/04/2018

11:38 PM Bug #25001: Crashing OSDs after going from 12.2.5 -> 12.2.6 -> 13.2.0
I'm afraid the corruption can be caused by "the racy SharedBlob::put()":http://tracker.ceph.com/issues/24211 (fixed s... Radoslaw Zarzynski
11:28 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Full log output from our latest tests:... Alfredo Deza
11:03 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Radoslaw Zarzynski
11:02 PM Bug #25098: Bluestore OSD failed to start with `bluefs_types.h: 54: FAILED assert(pos <= end)`
Reported also by Alredo Deza on 14.0.0-2808-gccbb825 (ccbb82553b5f130a109835907d4e09cd6e9cd993). Log is available: ht... Radoslaw Zarzynski
06:21 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
I've run it with
bluestore cache size hdd = 134217728
and it still hasn't happened even once after I changed. B...
Dennis Björklund
01:06 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
Dennis Björklund wrote:
> I lowered the memory usage 6 weeks ago by setting "bluestore cache size hdd" to a lower va...
Alfredo Rezinovsky
04:54 PM Bug #24715 (Duplicate): FAILED assert(0 == "put on missing extent (nothing before)")
Radoslaw Zarzynski
 

Also available in: Atom