Project

General

Profile

Activity

From 03/11/2018 to 04/09/2018

04/09/2018

10:53 PM Bug #23577: Inconsistent PG refusing to deep-scrub or repair
I attempted to upload a log file with debug_osd = 20/20 for this with upload tag e6d4f641-3006-4ee9-86eb-359f569de6ed... David Turner
05:47 PM Bug #23577: Inconsistent PG refusing to deep-scrub or repair
I have a second PG in the same cluster doing this exact same thing. One of it's 11 copies is on Bluestore, the rest ... David Turner
06:54 PM Bug #23333: bluestore: ENODATA on aio
Radoslaw Zarzynski
06:54 PM Bug #23333 (Fix Under Review): bluestore: ENODATA on aio
PR: https://github.com/ceph/ceph/pull/21306. Radoslaw Zarzynski
06:51 PM Support #23433: Ceph cluster doesn't start - ERROR: error creating empty object store in /data/ce...
What is the filesystem underneath _/data/ceph/build/dev/osd0_? Radoslaw Zarzynski
01:19 PM Bug #23540: FAILED assert(0 == "can't mark unloaded shard dirty") with compression enabled
Hello,
We are running for a few days without problem (with compression disabled), to get a debug i need to enable ...
Francisco Freire

04/06/2018

08:39 PM Bug #22616 (Resolved): bluestore_cache_data uses too much memory
Nathan Cutler
08:38 PM Backport #23226 (Resolved): luminous: bluestore_cache_data uses too much memory
Nathan Cutler
07:27 PM Backport #23226: luminous: bluestore_cache_data uses too much memory
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/21059
merged
Yuri Weinstein
08:29 PM Bug #22678: block checksum mismatch from rocksdb
debug_bluefs=20
Uploaded with id 5f2ee681-a9d7-4923-899e-5852b3fe18cb
Sergey Malinin
02:46 PM Bug #22678: block checksum mismatch from rocksdb
Sergey, can you generate a similar log, but also with 'debug bluefs = 20' enabled? You can even turn down 'debug blu... Sage Weil
04:28 PM Bug #23577 (Can't reproduce): Inconsistent PG refusing to deep-scrub or repair
This is an issue brought over from the ceph-users Mailing List for a thread titled "Have an inconsistent PG, repair n... David Turner
02:49 PM Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim
Sage, to be honest I'm still uncertain if that patch is related to this issue.
I haven't managed to reproduce this ...
Igor Fedotov
02:41 PM Bug #21259 (Resolved): bluestore: segv in BlueStore::TwoQCache::_trim
Igor deduced this was missing backports in luminous. THey're merged now, will be in 12.2.5:
https://github.com/ce...
Sage Weil
02:17 PM Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim
The same issue here - segfaults on 12.2.4 when recovery:... Konstantin Shalygin

04/05/2018

06:12 PM Bug #22678: block checksum mismatch from rocksdb
After crash OSD restarts and goes on running for a few hours before it crashes again.
I have uploaded log file wit...
Sergey Malinin
12:08 PM Bug #22678: block checksum mismatch from rocksdb
Is the error permanent in the sense that an affected OSD doesn't start and must be recreated?
Could you please pro...
Radoslaw Zarzynski
02:49 PM Bug #22102: BlueStore crashed on rocksdb checksum mismatch
Hrm, this run got a crc *near* EOF, but not past it.... Sage Weil

04/04/2018

07:49 PM Bug #22102: BlueStore crashed on rocksdb checksum mismatch
Scratch that, mark didn't hit the assert for num_readers ==0 , and the core indicates file isn't deleted.
_read_ra...
Sage Weil
03:40 PM Bug #22678: block checksum mismatch from rocksdb
I have similar issue with two OSDs (12.2.4) running on the same host. Recreating OSDs did not have any effect, I get ... Sergey Malinin
03:12 PM Bug #23540: FAILED assert(0 == "can't mark unloaded shard dirty") with compression enabled
Francisco,
thanks for the update, very appreciated.
Curious if you can collect a log for the crushing OSD, with d...
Igor Fedotov
02:53 PM Bug #23540: FAILED assert(0 == "can't mark unloaded shard dirty") with compression enabled
I disabled the compression for a while and no OSD's get the error, after enabled it again they back to get the proble... Francisco Freire

04/03/2018

09:36 PM Bug #22102: BlueStore crashed on rocksdb checksum mismatch
Current theory: bluefs is not protecting against a file open for read that is deleted. Mark observes that he sees th... Sage Weil

04/02/2018

11:50 PM Bug #23540: FAILED assert(0 == "can't mark unloaded shard dirty") with compression enabled
Yeah. The whole cluster has compression enabled Francisco Freire
09:56 PM Bug #23540: FAILED assert(0 == "can't mark unloaded shard dirty") with compression enabled
Hi Francisco,
wondering if you have compression enabled for any of your pools or the whole bluestore?
Igor Fedotov
08:22 PM Bug #23540 (Resolved): FAILED assert(0 == "can't mark unloaded shard dirty") with compression ena...
We are using the latest ceph luminous version (12.2.4), and we have a SATA pool tiered by an SSD pool. All using blue... Francisco Freire
08:25 AM Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim
We have also had this fault a number of times.
This was during a migration to bluestore - so we were backfilling for...
Shane Voss

03/29/2018

06:38 PM Bug #23040 (Resolved): bluestore: statfs available can go negative
Nathan Cutler
06:38 PM Backport #23074 (Resolved): luminous: bluestore: statfs available can go negative
Nathan Cutler
01:19 PM Backport #23074: luminous: bluestore: statfs available can go negative
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/20554
merged
Yuri Weinstein
12:43 PM Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim
Igor Fedotov
08:17 AM Bug #23141 (Resolved): BlueFS reports rotational journals if BDEV_WAL is not set
Nathan Cutler
08:17 AM Backport #23173 (Resolved): luminous: BlueFS reports rotational journals if BDEV_WAL is not set
Nathan Cutler

03/28/2018

10:27 PM Backport #23173: luminous: BlueFS reports rotational journals if BDEV_WAL is not set
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/20651
merged
Yuri Weinstein
04:14 AM Backport #23226 (In Progress): luminous: bluestore_cache_data uses too much memory
https://github.com/ceph/ceph/pull/21059 Prashant D

03/26/2018

03:29 PM Bug #23463 (Can't reproduce): src/os/bluestore/StupidAllocator.cc: 336: FAILED assert(rm.empty())
The ceph-volume nightly tests have seen this failure on one run so far (March 25th) with 2 out of 6 OSDs deployed. We... Alfredo Deza
03:58 AM Bug #23459: BlueStore kv_sync_thread() crash
crash dump attached Alex Gorbachev
03:55 AM Bug #23459 (Can't reproduce): BlueStore kv_sync_thread() crash
2018-03-25 06:49:02.894926 7ff4fdc97700 -1 *** Caught signal (Aborted) **
in thread 7ff4fdc97700 thread_name:bstore...
Alex Gorbachev

03/22/2018

08:21 PM Documentation #23443 (Resolved): doc: object -> file -> disk is wrong for bluestore
http://docs.ceph.com/docs/master/architecture/#storing-data
object -> file -> disk
is wrong now (for bluesto...
Марк Коренберг
10:54 AM Bug #23372: osd: segfault
Nokia ceph-users wrote:
> We are having 5 node cluster with 5 mons and 120 OSDs.
>
> One of the OSD (osd.7) crash...
Nokia ceph-users

03/21/2018

08:45 PM Bug #23246 (Fix Under Review): [OSD bug] KernelDevice.cc: 539: FAILED assert(r == 0)
Pull request: https://github.com/ceph/ceph/pull/20996. Radoslaw Zarzynski
04:33 PM Support #23433 (New): Ceph cluster doesn't start - ERROR: error creating empty object store in /d...
After running make vstart. When I try to start a ceph cluster with
@MON=3 OSD=1 MDS=1 MGR=1 RGW=1 ../src/vstart.sh ...
Neha Gupta

03/20/2018

11:28 PM Bug #23426: aio thread got No space left on device
Yeah, "the assertion came from _aio_t::get_return_value_":https://github.com/ceph/ceph/blob/820dac980e9416fe05998d50c... Radoslaw Zarzynski
10:33 PM Bug #23426: aio thread got No space left on device
might be dupe of #23333 Yuri Weinstein
10:32 PM Bug #23426 (Won't Fix): aio thread got No space left on device
Seems reproducible on all distros
Runs:
http://pulpito.ceph.com/teuthology-2018-03-20_05:02:01-smoke-master-tes...
Yuri Weinstein
10:08 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
I'm seeing the same problem here.
When I get the notification about the deep scrub error, I don't need to do "repa...
Brian Marcotte
12:57 PM Bug #23333 (In Progress): bluestore: ENODATA on aio
> Mar 13 15:55:45 ceph02 kernel: [362540.919407] print_req_error: critical medium error, dev sde, sector 5245986552
...
Radoslaw Zarzynski

03/19/2018

09:25 PM Bug #23333: bluestore: ENODATA on aio
Robert Sander wrote:
> It's a little bit strange, but it seems to have healed itself.
Looking around I just fou...
Robert Sander
09:22 PM Bug #23333: bluestore: ENODATA on aio
Radoslaw Zarzynski wrote:
> Instead, could you please:
> * provide log with increased debug log levels (_debug_osd ...
Robert Sander
08:55 PM Bug #23333: bluestore: ENODATA on aio
Errata, the issue is related to _aio_t::get_return_value_, not to _io_getevents_. Our debug message is misleading. I'... Radoslaw Zarzynski
05:17 PM Bug #23333: bluestore: ENODATA on aio
Hi Robert,
thanks for providing the info! I've took a look on the implementation of _io_getevents_ in your kernel ...
Radoslaw Zarzynski
03:01 PM Bug #23333: bluestore: ENODATA on aio
Radoslaw Zarzynski wrote:
> Could you please provide more information about used OS and hardware? Especially kerne...
Robert Sander
01:50 PM Bug #23333 (Need More Info): bluestore: ENODATA on aio
This looks really interesting. The assertion failure came from the _io_getevents_ (called by one of the _bstore_aio_ ... Radoslaw Zarzynski
10:17 AM Bug #21259 (Fix Under Review): bluestore: segv in BlueStore::TwoQCache::_trim
https://github.com/ceph/ceph/pull/20956
Sage Weil

03/16/2018

02:45 PM Bug #23390 (Resolved): Identifying NVMe via PCI serial isn't sufficient (Bluestore/SPDK)
Hi,
the manual requires a serial to put in to be able to use the NVMe with spdk.
http://docs.ceph.com/docs/master/r...
Andreas Merk
02:42 PM Bug #23246 (In Progress): [OSD bug] KernelDevice.cc: 539: FAILED assert(r == 0)
Sage Weil

03/15/2018

02:40 PM Bug #23266 (Won't Fix): "terminate called after throwing an instance of 'std::bad_alloc'" in upgr...
don't care about kraken Sage Weil
05:51 AM Bug #23372 (Can't reproduce): osd: segfault
We are having 5 node cluster with 5 mons and 120 OSDs.
One of the OSD (osd.7) crashed with following logs:...
Nokia ceph-users

03/14/2018

01:57 PM Bug #23333: bluestore: ENODATA on aio
Sage Weil
08:29 AM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
Hi Eric,
I am trying to pinpoint whether problem is related to AIO. Have you tested on official ceph builds, or tri...
Adam Kupczyk

03/13/2018

04:27 PM Bug #22464: Bluestore: many checksum errors, always 0x6706be76 (which matches a zero block)
I am seeing this issue too.
We were running Proxmox 4.x with CEPH 12.2.2 until a few weeks ago, never had a problem....
Eric Blevins
02:05 PM Bug #23333 (Resolved): bluestore: ENODATA on aio
Since 3 days one of 18 BlueStore OSDs is constantly crashing:
2018-03-10 04:01:45.366202 mon.ceph01 mon.0 192.168....
Robert Sander

03/12/2018

10:40 AM Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim
Another one fired:
https://tracker.ceph.com/issues/23283
Igor Fedotov
 

Also available in: Atom