Activity
From 02/09/2019 to 03/10/2019
03/10/2019
- 08:58 PM Bug #38656 (New): scrub reservation leak?
- /a/sage-2019-03-10_18:54:11-rados-wip-sage2-testing-2019-03-10-1053-distro-basic-smithi/3705804
pg 1.0 scrub does ... - 04:01 PM Bug #38655 (Resolved): osd: missing, size mismatch, snap mapper errors
- ...
- 03:57 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- /a/sage-2019-03-10_01:08:05-rados-master-distro-basic-smithi/3703837
description: rados/thrash/{0-size-min-size-ov...
03/09/2019
- 07:23 PM Bug #38631 (Resolved): osd-scrub-repair.sh fails due to num_objects wrong
- 05:00 PM Bug #38633 (Resolved): /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Segmentatio...
- 01:53 PM Bug #38579: osd: should not mark cluster_messenger when commited new osdmap
- Sage Weil wrote:
> So in step 5, the primary hasn't seen osdmap 20, right? Only the replica has? The part I don't ... - 12:54 AM Feature #38653: Enhance health message when pool quota fills up
- 12:27 AM Backport #38316 (In Progress): luminous: filestore: fsync(2) return value not checked
- https://github.com/ceph/ceph/pull/26871
03/08/2019
- 11:34 PM Feature #38653 (In Progress): Enhance health message when pool quota fills up
- 11:00 PM Feature #38653 (New): Enhance health message when pool quota fills up
- https://bugzilla.redhat.com/show_bug.cgi?id=1481306...
- 08:46 PM Feature #22147 (In Progress): Set multiple flags in a single command line
- https://github.com/ceph/ceph/pull/26785
- 08:09 PM Backport #38646 (In Progress): mimic: OpTracker destruct assert when OSD destruct
- 02:46 PM Backport #38646 (Resolved): mimic: OpTracker destruct assert when OSD destruct
- https://github.com/ceph/ceph/pull/26862
- 03:00 PM Bug #38649 (Can't reproduce): [ERR] full status failsafe engaged, dropping updates, now -21474836...
- /a/sage-2019-03-08_07:14:13-rados-wip-sage2-testing-2019-03-07-2213-distro-basic-smithi/3682171
- 02:48 PM Bug #38377: OpTracker destruct assert when OSD destruct
- master is still being merged into nautilus AFAICT
- 04:12 AM Bug #38377 (Pending Backport): OpTracker destruct assert when OSD destruct
- 01:15 PM Bug #38579 (Need More Info): osd: should not mark cluster_messenger when commited new osdmap
- So in step 5, the primary hasn't seen osdmap 20, right? Only the replica has? The part I don't understand is that i...
- 10:37 AM Backport #38610: luminous: mon: osdmap prune
- https://github.com/ceph/ceph/pull/26834
- 10:36 AM Backport #38561 (In Progress): mimic: mgr deadlock
- https://github.com/ceph/ceph/pull/26833
- 08:19 AM Bug #38124: OSD down on snaptrim.
- Hello,
any updates regarding this bug? I would love a patch to resolve this issue ASAP. One of my monitors just... - 08:12 AM Bug #38307 (Resolved): ceph-osd fails to bind to IPv6 interface for public_network
- The PR https://github.com/ceph/ceph/pull/26692 enforces pick_addresses to fail when ms_bind_ipv4 and ms_bind_ipv6 opt...
- 04:56 AM Bug #38633 (Fix Under Review): /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Seg...
- 01:18 AM Bug #38633 (Resolved): /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Segmentatio...
- a1539b118ed6372c19f321c94e2246f4fd130a33...
- 04:26 AM Backport #38562 (In Progress): luminous: mgr deadlock
- https://github.com/ceph/ceph/pull/26830
- 04:13 AM Bug #38598 (Resolved): osdmap may include only v1 address while osd binds to v2; mon drops messages
- 04:03 AM Subtask #37732: qa/suites/rados/thrash-erasure-code*: coverage review tasks
- https://github.com/ceph/ceph/pull/26417
Addresses
- Leveldb mons no longer relevant
- Fast-read could be added t... - 03:57 AM Cleanup #38635: bluestore: test osd_memory_target
- We want to test with different values of osd_memory_target.
Also, create tests that necessarily go beyond the osd_me... - 03:50 AM Cleanup #38635 (In Progress): bluestore: test osd_memory_target
03/07/2019
- 10:26 PM Bug #38358: short pg log + cache tier ceph_test_rados out of order reply
- /a/yuriw-2019-03-07_00:04:47-rados-wip_yuri_nautilus_3.6.19-distro-basic-smithi/3675857/
- 09:36 PM Bug #38631 (Resolved): osd-scrub-repair.sh fails due to num_objects wrong
A status check for "1/1 objects unfound" is coming back as "1/2 objects unfound"
Can be reproduced easily with:
...- 03:03 PM Bug #36546 (Duplicate): common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_flight_list.back...
- 03:02 PM Bug #38592 (Duplicate): mon,osd: src/common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_fli...
- 02:56 PM Bug #38623 (Fix Under Review): 2.8s2 past_intervals [6539,6541) start interval does not contain t...
- https://github.com/ceph/ceph/pull/26822
- 12:25 PM Bug #38623 (Resolved): 2.8s2 past_intervals [6539,6541) start interval does not contain the requi...
- ...
- 01:52 PM Bug #38579: osd: should not mark cluster_messenger when commited new osdmap
- Greg Farnum wrote:
> Do you have a reproducer for this?
>
> I get nervous when people want to remove mark_down ca... - 12:56 PM Bug #38604 (Resolved): mon logs not getting reopened after rotation
- 12:47 PM Bug #38624 (New): crush: get_rule_weight_osd_map does not handle multi-take rules
- CrushWrapper::get_rule_weight_osd_map() does not handle multi-take rules well. for example, a take 1 (primary) and t...
- 07:55 AM Backport #38565 (In Progress): mimic: Code to strip | from core pattern isn't right
- 06:43 AM Feature #38603: mon: osdmap prune
- @Nathan, I developed and tested code, I will open PR in the next couple of days soon. Please assign this to me :)
- 02:12 AM Feature #38616: Improvements to auto repair
OSD stats might have to be in meta collection- 01:29 AM Feature #38617 (Resolved): osd: Better error message when OSD count is less than osd_pool_default...
- Clearly indicate when number of OSDs is less than osd_pool_default_size, to avoid users from setting up clusters inco...
03/06/2019
- 10:38 PM Feature #38616 (Resolved): Improvements to auto repair
We should allow auto repair for bluestore pools since it has built in checksums. Currently, we are limited to er...- 10:18 PM Feature #38458: Ceph does not have command to show current osd primary-affinity
- So this is dumped as part of the osdmap output, but you want a way to see it for a particular OSD? Do we have any out...
- 10:13 PM Bug #38579: osd: should not mark cluster_messenger when commited new osdmap
- Do you have a reproducer for this?
I get nervous when people want to remove mark_down calls, as they are generally... - 07:50 PM Bug #38604 (Fix Under Review): mon logs not getting reopened after rotation
- aha, ceph-mgr and ceph-mds expliiclty set the thread name on startup.
https://github.com/ceph/ceph/pull/26797 - 07:40 PM Bug #38604: mon logs not getting reopened after rotation
- this appears to be because of /proc/$pid/stat. before,...
- 01:00 PM Bug #38604 (Resolved): mon logs not getting reopened after rotation
- ...
- 07:42 PM Bug #38219: rebuild-mondb hangs
- rados:singleton/{all/rebuild-mondb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rad...
- 06:40 PM Bug #38598 (Fix Under Review): osdmap may include only v1 address while osd binds to v2; mon drop...
- 03:10 AM Bug #38598: osdmap may include only v1 address while osd binds to v2; mon drops messages
- Proposed OSD fix:
- if we get an osdmap with require_osd_release < nautilus, and are bound to v2+v1, we rebind to ... - 03:08 AM Bug #38598 (Resolved): osdmap may include only v1 address while osd binds to v2; mon drops messages
- - osd binds to v2+v1
- osd sends osd_boot to mon
- mon adds v1 addr to osdmap only (due to require_osd_release < na... - 06:05 PM Bug #38555: scrub error on ec pg, got 6579891/0 or 7569408/6832128 bytes
- "2019-03-06 15:21:41.756014 osd.5 (osd.5) 287 : cluster [ERR] 2.2s0 scrub : stat mismatch, got 2/2 objects, 1/1 clone...
- 05:59 PM Backport #38610 (Need More Info): luminous: mon: osdmap prune
- Feature backport assumed to be non-trivial. Assigning to Joao, author of the feature, for now.
- 05:58 PM Backport #38610 (Rejected): luminous: mon: osdmap prune
- https://github.com/ceph/ceph/pull/26834
- 05:58 PM Feature #38603 (Pending Backport): mon: osdmap prune
- 10:18 AM Feature #38603 (Resolved): mon: osdmap prune
- Tracker to enable backport of this feature to luminous:
https://github.com/ceph/ceph/pull/19331 - 04:54 PM Backport #38274 (In Progress): luminous: Fix recovery and backfill priority handling
- 02:30 AM Bug #38592: mon,osd: src/common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_flight_list.bac...
- Is this related to http://tracker.ceph.com/issues/36546?
03/05/2019
- 11:17 PM Bug #26971: failed to become clean before timeout expired
- Reproduced on master in 1 of 10 duplicate runs:
dzafman-2019-03-05_10:43:39-rados:thrash-master-distro-basic-smith... - 07:01 PM Bug #26971: failed to become clean before timeout expired
- Seen recently in luminous.
yuriw-2019-02-28_14:42:05-rados-wip-yuri4-testing-2019-02-27-2159-luminous-distro-basi... - 10:37 PM Bug #38484 (Resolved): osd: InvalidRead, PG use-after-free putting ref
- 10:09 PM Bug #38525 (Resolved): qa/standalone/osd/pg-split-merge.sh fails
- 08:48 PM Bug #38594 (New): mimic: common/Mutex.cc: 110: FAILED assert(r == 0) in powercycle
- ...
- 08:22 PM Bug #38592 (Duplicate): mon,osd: src/common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_fli...
- ...
- 06:03 PM Bug #38499 (Need More Info): ceph-mon segfaults at startup
- 03:12 PM Feature #21073 (Resolved): mgr: ceph/rgw: show hostnames and ports in ceph -s status output
- 10:51 AM Bug #38582 (New): Pool storage MAX AVAIL reduction seems higher when single OSD reweight is done
i have a 5 node ceph 11.2.0 cluster with 335 osds. Each OSD is a 4TB HDD. It has one EC 4+1 pool.
Due to high st...- 03:28 AM Backport #38511 (In Progress): mimic: ceph CLI ability to change file ownership
- https://github.com/ceph/ceph/pull/26760
- 01:52 AM Backport #38510 (In Progress): luminous: ceph CLI ability to change file ownership
- https://github.com/ceph/ceph/pull/26758
- 01:49 AM Backport #38507 (In Progress): mimic: ENOENT on setattrs (obj was recently deleted)
- https://github.com/ceph/ceph/pull/26709
- 01:36 AM Bug #38579 (Need More Info): osd: should not mark cluster_messenger when commited new osdmap
-
when we run some fault test in Luminous 12.2.10, got coredump like ...
03/04/2019
- 10:14 PM Support #38475: PG stuck in creating state
- Support tickets will get a lot more eyes if you email the issue to ceph-users. :)
- 10:07 PM Bug #38499: ceph-mon segfaults at startup
- This must be running the labs or something, where's a log Abhi?
- 09:23 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- Argh, I can't find it now but I'm pretty sure I saw a PR go by that purported to fix this. The claimed issue is that ...
- 05:36 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- the rgw/multisite suite has been reproducing this reliably - probably because it runs with 'wait-for-scrub: false'
- 03:14 PM Bug #38484 (Fix Under Review): osd: InvalidRead, PG use-after-free putting ref
- https://github.com/ceph/ceph/pull/26742
- 02:31 PM Bug #38219: rebuild-mondb hangs
- /a/sage-2019-03-03_23:01:07-rados-wip-sage3-testing-2019-03-03-1043-distro-basic-smithi/3664297
- 12:14 PM Backport #38567 (Resolved): luminous: osd_recovery_priority is not documented (but osd_recovery_o...
- https://github.com/ceph/ceph/pull/27471
- 12:14 PM Backport #38566 (Resolved): mimic: osd_recovery_priority is not documented (but osd_recovery_op_p...
- https://github.com/ceph/ceph/pull/26901
- 12:13 PM Backport #38565 (Resolved): mimic: Code to strip | from core pattern isn't right
- https://github.com/ceph/ceph/pull/26811
- 12:12 PM Backport #38562 (Resolved): luminous: mgr deadlock
- https://github.com/ceph/ceph/pull/26830
- 12:12 PM Backport #38561 (Resolved): mimic: mgr deadlock
- https://github.com/ceph/ceph/pull/26833
- 10:51 AM Bug #38322: luminous: mons do not trim maps until restarted
- seen this issue with 10.2.4
03/03/2019
- 04:52 PM Documentation #38558 (New): doc: osd [test-]reweight-by-utilization is not properly documented in...
- Looks like:...
- 02:11 AM Bug #38537 (Pending Backport): mgr deadlock
03/02/2019
- 02:31 PM Bug #38484: osd: InvalidRead, PG use-after-free putting ref
- /a/sage-2019-03-02_01:13:07-rados-wip-sage2-testing-2019-03-01-1553-distro-basic-smithi/3656299
- 02:29 PM Bug #38555 (Can't reproduce): scrub error on ec pg, got 6579891/0 or 7569408/6832128 bytes
- ...
- 01:53 AM Bug #38525 (Fix Under Review): qa/standalone/osd/pg-split-merge.sh fails
- 01:45 AM Documentation #23999 (Pending Backport): osd_recovery_priority is not documented (but osd_recover...
- 01:33 AM Backport #38552 (Resolved): mimic: core: lazy omap stat collection
- https://github.com/ceph/ceph/pull/29189
- 01:33 AM Backport #38551 (Resolved): luminous: core: lazy omap stat collection
- https://github.com/ceph/ceph/pull/29190
03/01/2019
- 11:06 PM Bug #23875 (Need More Info): Removal of snapshot with corrupt replica crashes osd
- 11:04 PM Bug #38325 (Pending Backport): Code to strip | from core pattern isn't right
- 09:46 PM Bug #38484 (Can't reproduce): osd: InvalidRead, PG use-after-free putting ref
- i think i must have mixed up my test branches or something. i can't reproduce this.
- 09:45 PM Bug #38483 (In Progress): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_...
- 06:22 PM Feature #38550: osd: Implement lazy omap usage statistics per osd
- From - https://tracker.ceph.com/issues/38136
- 06:21 PM Feature #38550 (Duplicate): osd: Implement lazy omap usage statistics per osd
- This https://github.com/ceph/ceph/pull/26614 implements per pg and it would be good to summarize them per osd.
- 06:02 PM Feature #38136 (Pending Backport): core: lazy omap stat collection
- 05:01 PM Bug #38537 (Fix Under Review): mgr deadlock
- https://github.com/ceph/ceph/pull/26723
- 02:29 PM Bug #38537 (Resolved): mgr deadlock
- ...
- 03:40 PM Backport #38507 (New): mimic: ENOENT on setattrs (obj was recently deleted)
- 04:13 AM Backport #38507 (In Progress): mimic: ENOENT on setattrs (obj was recently deleted)
- -https://github.com/ceph/ceph/pull/26708-
- 03:28 PM Bug #36306 (Resolved): monstore tool rebuild does not generate creating_pgs
- 03:27 PM Backport #36434 (Resolved): luminous: monstore tool rebuild does not generate creating_pgs
- 03:26 PM Bug #36497 (Resolved): FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in ProtocolV1::repla...
- 03:26 PM Backport #37905 (Resolved): luminous: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in Pr...
- 03:26 PM Backport #37904 (Resolved): mimic: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in Proto...
- 03:25 PM Bug #24676 (Resolved): FreeBSD/Linux integration - monitor map with wrong sa_family
- 03:24 PM Backport #37972 (Resolved): luminous: FreeBSD/Linux integration - monitor map with wrong sa_family
- 02:40 PM Documentation #23999: osd_recovery_priority is not documented (but osd_recovery_op_priority is)
- https://github.com/ceph/ceph/pull/26705/commits/9475acb9805abeb6ab631df912cdbce0a7f34d3d
- 10:29 AM Bug #38053 (Resolved): Add hashinfo testing for dump command of ceph-objectstore-tool
- 10:28 AM Backport #38140 (Resolved): luminous: Add hashinfo testing for dump command of ceph-objectstore-tool
- 10:28 AM Backport #38141 (Resolved): mimic: Add hashinfo testing for dump command of ceph-objectstore-tool
- 10:14 AM Bug #38295 (Resolved): luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- 10:14 AM Backport #38342 (Resolved): mimic: luminous->(mimic,nautilus): PGMapDigest decode error on lumino...
- 05:28 AM Bug #38525: qa/standalone/osd/pg-split-merge.sh fails
- looks like the test is broken. we aren't reliably making a gap, so it would usually pass for the wrong reason.
- 05:27 AM Bug #38525 (Resolved): qa/standalone/osd/pg-split-merge.sh fails
- ...
- 03:10 AM Bug #38077: Marking all OSDs as "out" does not trigger a HEALTH_ERR state
- Hi,I don't know whether my opinion is right or not, but I think the status should be HEALTH_WARN when OSDs being mark...
- 02:24 AM Backport #38506 (In Progress): luminous: ENOENT on setattrs (obj was recently deleted)
- https://github.com/ceph/ceph/pull/26706
- 12:10 AM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- /a/sage-2019-02-28_12:30:17-rados-wip-sage-testing-2019-02-27-1720-distro-basic-smithi/3649931
description: rados/...
02/28/2019
- 07:35 PM Backport #36434: luminous: monstore tool rebuild does not generate creating_pgs
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25825
merged - 07:29 PM Backport #37905: luminous: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in ProtocolV1::r...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25956
merged - 07:24 PM Backport #37972: luminous: FreeBSD/Linux integration - monitor map with wrong sa_family
- Mykola Golub wrote:
> https://github.com/ceph/ceph/pull/26042
merged - 07:24 PM Backport #38140: luminous: Add hashinfo testing for dump command of ceph-objectstore-tool
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26284
merged - 04:32 PM Bug #38358: short pg log + cache tier ceph_test_rados out of order reply
- This is on luminous:
/a/teuthology-2019-02-23_01:30:03-rados-luminous-distro-basic-smithi/3627561/
We recently ... - 03:25 PM Bug #38184: osd: recovery does not preserve copy-on-write allocations between object clones after...
- This is indeed the current behavior. The OSD isn't clever enough to preserve the shared allocations across recovery....
- 01:54 PM Bug #38513 (Rejected): luminous: "AsyncReserver.h: 190: FAILED assert(!queue_pointers.count(item)...
- Run: http://pulpito.ceph.com/yuriw-2019-02-27_17:20:44-rados-wip-yuri3-testing-2019-02-25-2101-luminous-distro-basic-...
- 12:36 PM Backport #38511 (Resolved): mimic: ceph CLI ability to change file ownership
- https://github.com/ceph/ceph/pull/26760
- 12:36 PM Backport #38510 (Resolved): luminous: ceph CLI ability to change file ownership
- https://github.com/ceph/ceph/pull/26758
- 12:36 PM Backport #38507 (Resolved): mimic: ENOENT on setattrs (obj was recently deleted)
- https://github.com/ceph/ceph/pull/26709
- 12:36 PM Backport #38506 (Resolved): luminous: ENOENT on setattrs (obj was recently deleted)
- https://github.com/ceph/ceph/pull/26706
02/27/2019
- 11:50 PM Feature #38136: core: lazy omap stat collection
- Brad Hubbard wrote:
> Backporting https://github.com/ceph/ceph/pull/26614 may be easier Vikhyat if/when it merges?
... - 10:59 PM Bug #38431 (Resolved): osd: leaked pg refs on shutdown
- 10:59 PM Bug #38477 (Resolved): upgrade to nautilus leaves v1: osd blacklist entries
- 04:29 PM Backport #38342: mimic: luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26451
merged - 04:24 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- /a/rdias-2019-02-26_22:35:27-rados-wip-rdias2-testing-distro-basic-smithi/3642422
description: rados/thrash/{0-siz... - 03:35 PM Bug #38307: ceph-osd fails to bind to IPv6 interface for public_network
- Jesse, what's the value of the ms_bind_ipv6 and ms_bind_ipv4 in your configuration when you hit this problem?
My t... - 01:55 PM Bug #38499 (Need More Info): ceph-mon segfaults at startup
- ...
- 08:54 AM Feature #38496 (New): ceph.in: use same units for displaying ceph osd df
- ...
- 12:03 AM Bug #38184: osd: recovery does not preserve copy-on-write allocations between object clones after...
- Anyone?
02/26/2019
- 09:01 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- /ceph/teuthology-archive/pdonnell-2019-02-26_07:49:50-multimds-wip-pdonnell-testing-20190226.051327-distro-basic-smit...
- 02:35 PM Bug #38484 (Resolved): osd: InvalidRead, PG use-after-free putting ref
- ...
- 02:34 PM Bug #38403: osd: leaked from OSDMap::apply_incremental
- /a/sage-2019-02-26_12:41:21-rados:verify-wip-sage-testing-2019-02-25-1642-distro-basic-smithi/3641678
- 01:21 PM Bug #38483 (Fix Under Review): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_...
- https://github.com/ceph/ceph/pull/26651
- 12:40 PM Bug #38483 (Resolved): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_spl...
- ...
02/25/2019
- 11:08 PM Bug #38433 (Duplicate): rados/test.sh timeout
- #38238
- 10:55 PM Bug #38477 (Fix Under Review): upgrade to nautilus leaves v1: osd blacklist entries
- https://github.com/ceph/ceph/pull/26640
- 07:13 PM Bug #38477 (Resolved): upgrade to nautilus leaves v1: osd blacklist entries
- after a mimic -> nautilus upgrade,...
- 08:37 PM Backport #38141: mimic: Add hashinfo testing for dump command of ceph-objectstore-tool
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26283
merged - 06:53 PM Bug #38377 (Fix Under Review): OpTracker destruct assert when OSD destruct
- 06:41 PM Bug #38295 (Fix Under Review): luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- Follow-up fix: https://github.com/ceph/ceph/pull/26636
- 02:44 PM Feature #38370 (Pending Backport): ceph CLI ability to change file ownership
- 02:43 PM Bug #38432 (Pending Backport): ENOENT on setattrs (obj was recently deleted)
- 12:15 PM Support #38475 (New): PG stuck in creating state
- Hi,
After one big fail of my ceph cluster I would like force create PGs (beacause old pgs are lost definitively).
... - 11:21 AM Feature #38136: core: lazy omap stat collection
- Backporting https://github.com/ceph/ceph/pull/26614 may be easier Vikhyat if/when it merges?
- 04:08 AM Backport #38443 (In Progress): mimic: osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- -https://github.com/ceph/ceph/pull/26618-
- 01:56 AM Bug #37772: unittest_seastar_messenger fails with debug build
- Hi,I tested both ceph master and ceph14.0.1 on X86 and the unittest_seastar_messenger passed with debug build. So I d...
- 01:03 AM Backport #38442 (In Progress): luminous: osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- https://github.com/ceph/ceph/pull/26616
02/24/2019
- 03:30 PM Bug #24320: out of order reply and/or osd assert with set-chunks-read.yaml
- ...
- 03:28 PM Bug #21592: LibRadosCWriteOps.CmpExt got 0 instead of -4095-1
- /a/sage-2019-02-23_23:02:18-rados-wip-sage2-testing-2019-02-23-1354-distro-basic-smithi/3631993
- 03:27 PM Bug #24990: api_watch_notify: LibRadosWatchNotify.Watch3Timeout failed
- ...
- 03:26 PM Bug #38358: short pg log + cache tier ceph_test_rados out of order reply
- /a/sage-2019-02-23_23:02:18-rados-wip-sage2-testing-2019-02-23-1354-distro-basic-smithi/3631889
- 07:52 AM Feature #38462 (New): Store comments to config options stored in monitors (i.e. ceph config dump)
- It will be nice to have ability to add arbitrary comments to any option stored. In Ceph.conf it is possible. I see it...
- 06:47 AM Bug #38461 (New): Ceph osd out is the same as ceph osd reweight 0 (result in same bucket weights)
- http://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-osds says:...
02/23/2019
- 04:43 PM Feature #38458 (New): Ceph does not have command to show current osd primary-affinity
- It will be nice to have ability to show current primary-affinity value for an osd.
02/22/2019
- 09:19 PM Backport #38423: luminous: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- Thanks for the tidy up Nathan.
- 05:45 PM Bug #22525 (Resolved): auth: ceph auth add does not sanity-check caps
- 05:45 PM Backport #23670 (Resolved): luminous: auth: ceph auth add does not sanity-check caps
- 05:27 PM Feature #37597 (Resolved): ceph-objectstore-tool: Add HashInfo to object dump output
- 05:27 PM Backport #37690 (Resolved): luminous: ceph-objectstore-tool: Add HashInfo to object dump output
- 05:26 PM Bug #37776 (Resolved): workunits/rados/test_health_warnings.sh fails with <9 osds down
- 05:26 PM Backport #37815 (Resolved): luminous: workunits/rados/test_health_warnings.sh fails with <9 osds ...
- 05:26 PM Bug #24601 (Resolved): FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- 05:26 PM Backport #37833 (Resolved): luminous: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- 05:17 PM Bug #38431 (Fix Under Review): osd: leaked pg refs on shutdown
- https://github.com/ceph/ceph/pull/26595
- 05:06 PM Bug #38431: osd: leaked pg refs on shutdown
- This appears to be as simple as a queued write in progress when shutdown happens:...
- 12:55 PM Bug #38431 (Resolved): osd: leaked pg refs on shutdown
- /a/sage-2019-02-21_21:52:17-rados-wip-sage3-testing-2019-02-21-1359-distro-basic-smithi/3622562
w/ pg ref logs - 04:47 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- /a/sage-2019-02-22_15:54:54-rados-wip-sage2-testing-2019-02-22-0711-distro-basic-smithi/3626248
- 04:17 PM Bug #38070 (Resolved): A PG repairing doesn't mean PG is damaged
- 04:17 PM Backport #38207 (Resolved): luminous: A PG repairing doesn't mean PG is damaged
- 04:07 PM Backport #38317 (Resolved): mimic: filestore: fsync(2) return value not checked
- 04:07 PM Bug #37593 (Resolved): ec pool lost data due to snap clone
- 04:06 PM Backport #37993 (Resolved): luminous: ec pool lost data due to snap clone
- 04:06 PM Cleanup #38025 (Resolved): qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries
- 04:06 PM Backport #38046 (Resolved): luminous: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log...
- 04:06 PM Bug #37919 (Resolved): osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- 04:06 PM Backport #38105 (Resolved): luminous: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing...
- 03:41 PM Backport #38450 (In Progress): mimic: src/osd/OSDMap.h: 1065: FAILED assert(__null != pool)
- 02:36 PM Backport #38450 (Resolved): mimic: src/osd/OSDMap.h: 1065: FAILED assert(__null != pool)
- https://github.com/ceph/ceph/pull/29976
- 03:39 PM Backport #38243 (Resolved): mimic: scrub warning check incorrectly uses mon scrub interval
- 03:36 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- ceph-client.admin.19974.log.gz is aio_pp
it starts, gets a few tests in, then the log stops unexpectedly...... - 03:03 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- another instance:
/a/sage-2019-02-21_21:52:17-rados-wip-sage3-testing-2019-02-21-1359-distro-basic-smithi/3622638
... - 03:09 PM Backport #38162 (In Progress): luminous: maybe_remove_pg_upmaps incorrectly cancels valid pending...
- 02:57 PM Bug #38432 (Fix Under Review): ENOENT on setattrs (obj was recently deleted)
- https://github.com/ceph/ceph/pull/26591
- 12:59 PM Bug #38432 (Resolved): ENOENT on setattrs (obj was recently deleted)
- ...
- 02:34 PM Backport #38443 (Resolved): mimic: osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- https://github.com/ceph/ceph/pull/27907
- 02:34 PM Backport #38442 (Resolved): luminous: osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- https://github.com/ceph/ceph/pull/26616 (merged for v12.2.12)
backport of follow-up fix: https://github.com/ceph/cep... - 02:33 PM Backport #38437 (Resolved): mimic: crc cache should be invalidated when posting preallocated rx b...
- https://github.com/ceph/ceph/pull/29247
- 02:33 PM Backport #38436 (Resolved): luminous: crc cache should be invalidated when posting preallocated r...
- https://github.com/ceph/ceph/pull/29248
- 01:18 PM Bug #38425 (Duplicate): mon: segmentation fault in AuthMonitor::create_pending
- just fixed this, #38372
- 04:45 AM Bug #38425: mon: segmentation fault in AuthMonitor::create_pending
- Here's a different stack trace that's probably related:
/ceph/teuthology-archive/pdonnell-2019-02-19_07:16:18-fs-w... - 04:42 AM Bug #38425 (Duplicate): mon: segmentation fault in AuthMonitor::create_pending
- ...
- 01:14 PM Bug #38372 (Resolved): segfault in "AuthMonitor::increase_max_global_id()"
- 01:11 PM Bug #38416 (Pending Backport): crc cache should be invalidated when posting preallocated rx buffers
- 01:10 PM Bug #36337: OSDs crash with failed assertion in PGLog::merge_log as logs do not overlap
- ...
- 01:06 PM Bug #38433 (Duplicate): rados/test.sh timeout
- ...
- 06:03 AM Backport #38398 (In Progress): mimic: rados_shutdown hang forever in ~objecter()
- https://github.com/ceph/ceph/pull/26583
- 03:35 AM Backport #38400 (In Progress): luminous: rados_shutdown hang forever in ~objecter()
- https://github.com/ceph/ceph/pull/26579
02/21/2019
- 11:29 PM Backport #23670: luminous: auth: ceph auth add does not sanity-check caps
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24906
merged - 11:26 PM Backport #37690: luminous: ceph-objectstore-tool: Add HashInfo to object dump output
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25722
merged - 11:25 PM Backport #37815: luminous: workunits/rados/test_health_warnings.sh fails with <9 osds down
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25851
merged - 11:24 PM Backport #37833: luminous: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25853
merged - 11:24 PM Backport #38424 (In Progress): mimic: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- https://github.com/ceph/ceph/pull/26578
- 11:15 PM Backport #38424 (Resolved): mimic: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- https://github.com/ceph/ceph/pull/26578
- 11:23 PM Backport #38207: luminous: A PG repairing doesn't mean PG is damaged
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26305
merged - 11:19 PM Backport #38423 (In Progress): luminous: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- 11:10 PM Backport #38423: luminous: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- https://github.com/ceph/ceph/pull/26574
- 11:10 PM Backport #38423 (Resolved): luminous: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- https://github.com/ceph/ceph/pull/26574
- 11:05 PM Bug #38406: osd/TestPGLog.cc: Verify that dup_index is being trimmed
- https://github.com/ceph/ceph/pull/26533
- 08:27 PM Bug #38406 (Pending Backport): osd/TestPGLog.cc: Verify that dup_index is being trimmed
- 10:52 PM Cleanup #38422 (New): qa/suites/rados/verify: coverage review tasks
- Tracker for verify coverage review tasks
- 10:50 PM Cleanup #38421 (New): qa/suites/rados/objectstore: coverage review tasks
- Tracker for objectstore coverage review tasks.
- 10:11 PM Backport #38317: mimic: filestore: fsync(2) return value not checked
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26438
merged - 09:54 PM Backport #37993: luminous: ec pool lost data due to snap clone
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26078
merged - 09:53 PM Backport #38046: luminous: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26148
merged - 09:53 PM Backport #38105: luminous: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26236
merged - 09:35 PM Bug #38083 (Resolved): mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- 08:24 PM Bug #38083: mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- https://github.com/ceph/ceph/pull/26573
- 08:23 PM Bug #38083 (Fix Under Review): mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- 08:31 PM Bug #26970 (Pending Backport): src/osd/OSDMap.h: 1065: FAILED assert(__null != pool)
- 07:19 PM Bug #26970: src/osd/OSDMap.h: 1065: FAILED assert(__null != pool)
- Seen in mimic: /a/yuriw-2019-02-20_20:24:46-rados-wip-yuri3-testing-2019-02-20-1755-mimic-distro-basic-smithi/3619026...
- 06:59 PM Bug #26970: src/osd/OSDMap.h: 1065: FAILED assert(__null != pool)
- This was fixed as cfe6ca82ed800e81e286396b5b7783ca299325f0 in larger pull request https://github.com/ceph/ceph/pull/2...
- 05:34 PM Backport #38243: mimic: scrub warning check incorrectly uses mon scrub interval
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26493
merged - 01:15 PM Bug #38358: short pg log + cache tier ceph_test_rados out of order reply
- /a/sage-2019-02-21_06:38:51-rados-wip-sage-testing-2019-02-20-2138-distro-basic-smithi/3620775
- 09:13 AM Bug #23320 (Resolved): OSD suicide itself because of a firewall rule but reports a received signal
- 06:33 AM Bug #38377: OpTracker destruct assert when OSD destruct
- Greg Farnum wrote:
> Is this a custom build? Where did it come from?
aha,ceph version is Luminous 12.2.10,
see [... - 03:36 AM Bug #24866 (Resolved): FAILED assert(0 == "past_interval start interval mismatch") in check_past_...
- 03:34 AM Bug #38417 (New): ceph tell mon.a help timeout
- ...
- 02:04 AM Bug #23879: test_mon_osdmap_prune.sh fails
- http://pulpito.ceph.com/gregf-2019-02-19_21:22:48-rados:monthrash-master-distro-basic-mira/3614157/
- 12:49 AM Feature #38136: core: lazy omap stat collection
- Vikhyat Umrao wrote:
> I am checking if we can backport only this to luminous - https://github.com/ceph/ceph/pull/18... - 12:40 AM Feature #38136: core: lazy omap stat collection
- I am checking if we can backport only this to luminous - https://github.com/ceph/ceph/pull/18096/commits/13295a38839d...
02/20/2019
- 11:44 PM Bug #38416 (Fix Under Review): crc cache should be invalidated when posting preallocated rx buffers
- https://github.com/ceph/ceph/pull/26552
- 11:27 PM Bug #38416 (Resolved): crc cache should be invalidated when posting preallocated rx buffers
- Otherwise, if the buffer was previously checksummed, the client will spin on "bad crc in data" errors forever.
- 11:28 PM Bug #27988 (Need More Info): Warn if queue of scrubs ready to run exceeds some threshold
- This is put on the back burner until we decide what to do next
- 11:16 PM Bug #38333 (Duplicate): mon crash in AuthMonitor::Incremental::encode buffer code
- oh, i bet it was the same auth thing i just fixed: #38372
- 10:05 PM Bug #38333: mon crash in AuthMonitor::Incremental::encode buffer code
- Is it possible this is a result of some of the buffer list stuff we know was broken?
- 11:16 PM Bug #38372: segfault in "AuthMonitor::increase_max_global_id()"
- actually, the original crash here was slightly different than I thought: the old assign_global_id() was passed false ...
- 11:10 PM Bug #38372 (Fix Under Review): segfault in "AuthMonitor::increase_max_global_id()"
- https://github.com/ceph/ceph/pull/26550
- 10:27 PM Bug #38372 (In Progress): segfault in "AuthMonitor::increase_max_global_id()"
- Monitor.cc auth methods protected by auth_lock, but AuthMontior' assign_global_id() is under the normal mon->lock
- 10:27 PM Bug #38372: segfault in "AuthMonitor::increase_max_global_id()"
- I see the last log lines from the bad thread are...
- 10:20 PM Bug #38372: segfault in "AuthMonitor::increase_max_global_id()"
- Sage just rewrote part of this and I see it's under the ProtocolV2 stack, so giving it to him...
- 10:59 PM Backport #38244 (In Progress): luminous: scrub warning check incorrectly uses mon scrub interval
- 10:09 AM Backport #38244 (Need More Info): luminous: scrub warning check incorrectly uses mon scrub interval
- see the mimic backport for why this is non-trivial
- 10:51 PM Bug #38212: Rare qa/standalone/osd/osd-markdown.sh mon start-up error
- So the problem is that the test checks for a failure return, and it's getting a failure return in the wrong place? Do...
- 10:46 PM Bug #38248: qa/standalone/osd/pg-split-merge.sh TEST_import_after_merge_and_gap() test fails
- Do we know what PRs this test branch covered? Three jobs hitting a new bug is certainly suspicious!
- 10:43 PM Bug #38306: ceph-mon: "*** Caught signal (Segmentation fault) **" in upgrade:luminous-x-mimic
- Is it possible we did a naughty backport that made these both pop up close together? :/
- 10:40 PM Bug #38356 (Duplicate): standalone/osd/osd-markdown.sh fails in TEST_markdown_boot
- 10:37 PM Bug #38377: OpTracker destruct assert when OSD destruct
- Is this a custom build? Where did it come from?
- 10:31 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- Ah, /a/sage-2019-02-07_22:38:18-rados-master-distro-basic-smithi/3561540 appears to be it
- 10:28 PM Bug #38238: rados/test.sh: api_aio_pp doesn't seem to start
- The referenced log doesn't seem to be this bug, as it got stuck on a crashed OSD...
- 10:17 PM Bug #38381: Rados.get_fsid() returning bytes in python3
- https://github.com/ceph/ceph/pull/26514
- 10:14 PM Bug #38402: ceph-objectstore-tool on down osd w/ not enough in osds
- I see we have a bunch of live OSDs which are not marked in; perhaps this is the same as #37439?
- 12:58 PM Bug #38402 (Can't reproduce): ceph-objectstore-tool on down osd w/ not enough in osds
- crush couldn't quite do it:...
- 10:12 PM Bug #38172: segv in rocksdb NewIterator
- We can bump back up if it appears again or close it if we don't see it reappear.
- 07:19 PM Bug #38344 (Resolved): src/osd/PG.h: 1245: FAILED ceph_assert(prev >= 0)
- 03:17 PM Bug #38406 (Resolved): osd/TestPGLog.cc: Verify that dup_index is being trimmed
- Modify existing tests to verify that dup_index is getting trimmed.
- 01:11 PM Bug #20909: Error ETIMEDOUT: crush test failed with -110: timed out during smoke test (5 seconds)
- ...
- 01:10 PM Bug #38403 (Duplicate): osd: leaked from OSDMap::apply_incremental
- ...
- 01:08 PM Bug #36739: ENOENT in collection_move_rename on EC backfill target
- /a/sage-2019-02-19_23:03:51-rados-wip-sage3-testing-2019-02-19-1008-distro-basic-smithi/3614320
- 01:06 PM Bug #20798: LibRadosLockECPP.LockExclusiveDurPP gets EEXIST
- /a/sage-2019-02-19_23:03:51-rados-wip-sage3-testing-2019-02-19-1008-distro-basic-smithi/3614261
- 01:04 PM Bug #38359 (Pending Backport): osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- 01:02 PM Bug #36304 (Resolved): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_spl...
- 01:00 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- 12:17 PM Bug #16890: rbd diff outputs nothing when the image is layered and with a writeback cache tier
- rebased PR posted at https://github.com/ceph/ceph/pull/26542
- 10:44 AM Backport #38400 (Resolved): luminous: rados_shutdown hang forever in ~objecter()
- https://github.com/ceph/ceph/pull/26579
- 10:43 AM Backport #38398 (Resolved): mimic: rados_shutdown hang forever in ~objecter()
- https://github.com/ceph/ceph/pull/26583
- 10:08 AM Backport #38243: mimic: scrub warning check incorrectly uses mon scrub interval
- First attempted backport, https://github.com/ceph/ceph/pull/26356, was closed.
- 08:57 AM Backport #38162: luminous: maybe_remove_pg_upmaps incorrectly cancels valid pending upmaps
- https://github.com/ceph/ceph/pull/26127
02/19/2019
- 06:17 PM Bug #38384 (Resolved): ceph_mon --help mon broken
- 06:08 PM Bug #38384 (Resolved): ceph_mon --help mon broken
http://qa-proxy.ceph.com/teuthology/dzafman-2019-02-19_08:38:36-rados-wip-zafman-testing-38344-distro-basic-smithi/...- 05:36 PM Bug #38372: segfault in "AuthMonitor::increase_max_global_id()"
- http://qa-proxy.ceph.com/teuthology/jdillaman-2019-02-19_09:50:43-rbd-wip-jd-testing-distro-basic-smithi/3612490/teut...
- 04:05 PM Bug #38381 (Fix Under Review): Rados.get_fsid() returning bytes in python3
- 03:24 PM Bug #38381 (In Progress): Rados.get_fsid() returning bytes in python3
- 03:24 PM Bug #38381: Rados.get_fsid() returning bytes in python3
- 03:13 PM Bug #38381 (Resolved): Rados.get_fsid() returning bytes in python3
- Hello,
I've found a bug in librados which causing big issues in Openstack projects (glance_store, cinder, nova).
... - 03:36 PM Bug #24866: FAILED assert(0 == "past_interval start interval mismatch") in check_past_interval_bo...
- ...
- 03:34 PM Bug #24866 (Fix Under Review): FAILED assert(0 == "past_interval start interval mismatch") in che...
- https://github.com/ceph/ceph/pull/26513
- 02:44 PM Bug #37766 (Pending Backport): rados_shutdown hang forever in ~objecter()
- 02:30 PM Bug #38359: osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- /a/sage-2019-02-18_23:58:27-rados-wip-sage-testing-2019-02-18-1341-distro-basic-smithi/3609330
- 09:21 AM Bug #38377 (Resolved): OpTracker destruct assert when OSD destruct
- coredump...
- 02:46 AM Bug #38375 (Need More Info): OSD segmentation fault on rbd create
- Random segfault when attempting to create an rbd on an erasure encoded pool.
ceph version 12.2.10 (177915764b7528...
02/18/2019
- 10:15 PM Bug #38372 (Resolved): segfault in "AuthMonitor::increase_max_global_id()"
- ...
- 10:06 PM Feature #38370 (Fix Under Review): ceph CLI ability to change file ownership
- 06:21 PM Feature #38370 (Resolved): ceph CLI ability to change file ownership
- When creating/fetching a key/monmap/crushmap it might be useful (especially for file used by a process later) to appl...
- 10:02 PM Bug #38307: ceph-osd fails to bind to IPv6 interface for public_network
- You going to open a PR for this, Jesse?
- 09:31 PM Bug #36304 (Fix Under Review): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_...
- https://github.com/ceph/ceph/pull/26492
- 09:30 PM Bug #36304: FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_split_child(PG*)
- aha:
during startup, we load pg 2.fs1, but fail to prime it from init():... - 05:32 PM Bug #36304: FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_split_child(PG*)
- We have logs here: /a/nojha-2019-02-11_18:58:45-rados:thrash-erasure-code-wip-test-revert-distro-basic-smithi/3575122
- 09:03 PM Bug #38320 (Resolved): leak in OSD::ms_handle_authentication()
- 09:02 PM Bug #38359 (Fix Under Review): osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- https://github.com/ceph/ceph/pull/26489
- 06:57 PM Feature #38029 (Fix Under Review): [RFE] If the nodeep-scrub/noscrub flags are set in pools inste...
- 06:55 PM Feature #38029: [RFE] If the nodeep-scrub/noscrub flags are set in pools instead of global cluste...
- https://github.com/ceph/ceph/pull/26488
- 02:30 PM Bug #38322: luminous: mons do not trim maps until restarted
- I have a feeling this is actually due to what led me to open this PR: https://github.com/ceph/ceph/pull/19076
The ...
02/17/2019
- 02:16 PM Bug #36304: FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_wake_split_child(PG*)
- reproduced, but without logs.....
- 02:05 PM Bug #20909: Error ETIMEDOUT: crush test failed with -110: timed out during smoke test (5 seconds)
- ...
- 02:04 AM Bug #38344 (Fix Under Review): src/osd/PG.h: 1245: FAILED ceph_assert(prev >= 0)
02/16/2019
- 07:45 PM Bug #38359 (Resolved): osd-markdown.sh can fail with CLI_DUP_COMMAND=1
- ...
- 03:49 PM Bug #38358 (Resolved): short pg log + cache tier ceph_test_rados out of order reply
- the combination of
- *1-pg-log-overrides/short_pg_log.yaml*
and
- *workloads/cache-agent-small.yaml*
and ... - 03:37 PM Bug #38357 (Pending Backport): ClsLock.TestExclusiveEphemeralStealEphemeral failed
- ...
- 03:34 PM Bug #38356 (Duplicate): standalone/osd/osd-markdown.sh fails in TEST_markdown_boot
- ...
- 10:58 AM Backport #38239 (Resolved): mimic: radosbench tests hit ENOSPC
- 10:55 AM Backport #38352 (Rejected): luminous: Limit loops waiting for force-backfill/force-recovery to ha...
- 10:55 AM Backport #38351 (Resolved): mimic: Limit loops waiting for force-backfill/force-recovery to happen
- https://github.com/ceph/ceph/pull/29245
02/15/2019
- 11:19 PM Bug #38294 (Rejected): osd/PG.cc: 6141: FAILED ceph_assert(info.history.same_interval_since != 0)...
- no logs
- 11:17 PM Bug #38330 (Resolved): osd/OSD.cc: 1515: abort() in Service::build_incremental_map_msg
- 02:45 PM Bug #38330 (Fix Under Review): osd/OSD.cc: 1515: abort() in Service::build_incremental_map_msg
- https://github.com/ceph/ceph/pull/26448
- 01:33 PM Bug #38330 (Resolved): osd/OSD.cc: 1515: abort() in Service::build_incremental_map_msg
- ...
- 10:46 PM Backport #38208 (Resolved): mimic: A PG repairing doesn't mean PG is damaged
- 10:39 PM Backport #38208: mimic: A PG repairing doesn't mean PG is damaged
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26304
merged - 10:37 PM Backport #38239: mimic: radosbench tests hit ENOSPC
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26354
merged - 10:26 PM Bug #38309 (Pending Backport): Limit loops waiting for force-backfill/force-recovery to happen
- 10:06 PM Bug #38345: mon: segv in MonOpRequest::~MonOpRequest OpHistory::cleanup
- /a/sage-2019-02-15_20:30:53-upgrade:mimic-x-wip-mimic-upgrade-distro-basic-smithi/3595562
- 06:31 PM Bug #38345 (Resolved): mon: segv in MonOpRequest::~MonOpRequest OpHistory::cleanup
- ...
- 05:56 PM Bug #38344: src/osd/PG.h: 1245: FAILED ceph_assert(prev >= 0)
- I have seen a similar failure here: /a/nojha-2019-02-13_14:45:41-rados:thrash-erasure-code-wip-test-revert-2-distro-b...
- 05:47 PM Bug #38344: src/osd/PG.h: 1245: FAILED ceph_assert(prev >= 0)
- Just prior to the crash pg 4.13 ended up with a local_num_bytes of -42624KiB....
- 05:22 PM Bug #38344 (Resolved): src/osd/PG.h: 1245: FAILED ceph_assert(prev >= 0)
http://pulpito.ceph.com/dzafman-2019-02-14_22:11:34-rados:thrash-master-distro-basic-smithi/3593928...- 04:58 PM Backport #38342 (In Progress): mimic: luminous->(mimic,nautilus): PGMapDigest decode error on lum...
- 04:49 PM Backport #38342 (Resolved): mimic: luminous->(mimic,nautilus): PGMapDigest decode error on lumino...
- https://github.com/ceph/ceph/pull/26451
- 04:44 PM Bug #37965 (Can't reproduce): rados/upgrade test fails
- 04:43 PM Bug #38295 (Pending Backport): luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- the commit to backport to mimic is e4ae368ff7a5396194f8bdd5692429af5457998b
- 04:42 PM Bug #38283 (Resolved): max-pg-per-osd tests failing
- 04:40 PM Bug #38320 (In Progress): leak in OSD::ms_handle_authentication()
- https://github.com/ceph/ceph/pull/26353
- 04:39 PM Bug #24667 (Can't reproduce): osd: SIGSEGV in MMgrReport::encode_payload
- 04:06 PM Backport #38341 (Resolved): mimic: pg stuck in backfill_wait with plenty of disk space
- https://github.com/ceph/ceph/pull/28201
- 03:13 PM Bug #38333 (Duplicate): mon crash in AuthMonitor::Incremental::encode buffer code
- ...
- 10:53 AM Bug #38322: luminous: mons do not trim maps until restarted
- Here's an example on v12.2.8. The mon db is normally trim at around 700MB, but after some backfilling its currently a...
- 02:06 AM Bug #38012: osd bad crc cause the whole cluster stop accepting new request.
- Josh Durgin wrote:
> It seems the first step would be reporting the crc mismatches via a perfcounter. Then the mgr c... - 01:14 AM Backport #38317 (In Progress): mimic: filestore: fsync(2) return value not checked
- https://github.com/ceph/ceph/pull/26438
- 12:56 AM Bug #38325: Code to strip | from core pattern isn't right
- I tried openSUSE and it doesn't add back the pipe character when doing sysctl -w, so it should never have been stripp...
02/14/2019
- 11:46 PM Bug #38309 (Fix Under Review): Limit loops waiting for force-backfill/force-recovery to happen
- 01:00 AM Bug #38309 (Resolved): Limit loops waiting for force-backfill/force-recovery to happen
while loops in qa/standalone/osd/osd-backfill-prio.sh and qa/standalone/osd/osd-recovery-prio.sh can loop until teu...- 09:00 PM Bug #38325 (Fix Under Review): Code to strip | from core pattern isn't right
- 08:57 PM Bug #38325 (Resolved): Code to strip | from core pattern isn't right
- 07:25 PM Bug #37875: osdmaps aren't being cleaned up automatically on healthy cluster
- We have seen the same behavior with luminous and +2500 osds in late 2017. Analyzed with Joao then, but he could not r...
- 06:37 PM Bug #38034 (Pending Backport): pg stuck in backfill_wait with plenty of disk space
- 04:02 PM Bug #38282 (Resolved): cephtool/test.sh failure in test_mon_osd_pool_set
- 11:58 AM Bug #38282: cephtool/test.sh failure in test_mon_osd_pool_set
- /a/kchai-2019-02-14_06:27:37-rados-wip-kefu2-testing-2019-02-14-1156-distro-basic-smithi/3590390
- 03:56 PM Bug #38293 (Duplicate): qa/standalone/osd/osd-backfill-prio.sh failed
- 01:31 PM Bug #38322 (Closed): luminous: mons do not trim maps until restarted
- Reported by several users, most recently at https://marc.info/?l=ceph-devel&m=154955388914036&w=2
- 11:23 AM Bug #38320 (Resolved): leak in OSD::ms_handle_authentication()
- ...
- 11:03 AM Backport #38317 (Resolved): mimic: filestore: fsync(2) return value not checked
- https://github.com/ceph/ceph/pull/26438
- 11:02 AM Backport #38316 (Resolved): luminous: filestore: fsync(2) return value not checked
- https://github.com/ceph/ceph/pull/26871
- 09:40 AM Bug #21194: mon clock skew test is fragile
- /a/kchai-2019-02-14_06:27:37-rados-wip-kefu2-testing-2019-02-14-1156-distro-basic-smithi/3590540/
02/13/2019
- 11:36 PM Bug #38034 (Fix Under Review): pg stuck in backfill_wait with plenty of disk space
- 11:09 PM Bug #38307 (Resolved): ceph-osd fails to bind to IPv6 interface for public_network
When trying to start ceph-osd configured for IPv6, interface binding fails.
Configure in ceph.conf:
[global]
p...- 09:24 PM Bug #38282: cephtool/test.sh failure in test_mon_osd_pool_set
- I have a feeling #38283 has the same root cause...
- 09:02 PM Bug #38282 (Fix Under Review): cephtool/test.sh failure in test_mon_osd_pool_set
- https://github.com/ceph/ceph/pull/26413
- 08:59 PM Bug #38282: cephtool/test.sh failure in test_mon_osd_pool_set
- ...
- 08:19 PM Bug #38293: qa/standalone/osd/osd-backfill-prio.sh failed
This test shouldn't loop forever so that teuthology has to time-out. I'll fix that elsewhere.
The mon received ...- 12:37 PM Bug #38293 (Duplicate): qa/standalone/osd/osd-backfill-prio.sh failed
- ...
- 06:09 PM Subtask #37731: upgrade/luminous-x - add "require-osd-release nautilus" and clean up
- https://github.com/ceph/ceph/pull/26302
- 04:49 PM Bug #38306 (New): ceph-mon: "*** Caught signal (Segmentation fault) **" in upgrade:luminous-x-mimic
- Run: http://pulpito.ceph.com/teuthology-2019-02-13_02:25:02-upgrade:luminous-x-mimic-distro-basic-smithi/
Job: ['358... - 01:31 PM Bug #38295 (Fix Under Review): luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- fixed by a patch to MMonMgrDigest in https://github.com/ceph/ceph/pull/26389
- 01:18 PM Bug #38295: luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- This appears to be broken since mimic, and triggers if you upgrade a mgr before all mons are upgrade.
We call enco... - 12:49 PM Bug #38295 (Resolved): luminous->(mimic,nautilus): PGMapDigest decode error on luminous end
- ...
- 12:59 PM Bug #38296 (Resolved): segv in fgets() in collect_sys_info reading /proc/cpuinfo
- ...
- 12:48 PM Bug #38294 (Resolved): osd/PG.cc: 6141: FAILED ceph_assert(info.history.same_interval_since != 0)...
- ...
- 12:40 PM Bug #38258 (Pending Backport): filestore: fsync(2) return value not checked
- 12:40 PM Bug #37910 (Resolved): segv during crc of incoming message front
- https://github.com/ceph/ceph/pull/26391
- 12:39 PM Bug #38024 (Resolved): segv, heap corruption in ec encode_and_write
- https://github.com/ceph/ceph/pull/26391
- 12:38 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- /a/sage-2019-02-13_00:42:53-rados-wip-sage2-testing-2019-02-12-1700-distro-basic-smithi/3581795
- 12:34 PM Bug #24866: FAILED assert(0 == "past_interval start interval mismatch") in check_past_interval_bo...
- /a/sage-2019-02-13_00:42:53-rados-wip-sage2-testing-2019-02-12-1700-distro-basic-smithi/3581667
description: rados... - 01:15 AM Bug #23937: FAILED assert(info.history.same_interval_since != 0)
- I think it was resolved by this https://github.com/ceph/ceph/pull/20571
and the related issue is http://tracker.ceph...
02/12/2019
- 11:38 PM Bug #38083: mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- This seems to reliably fail in mimic only with rhel_latest.yaml.
http://pulpito.ceph.com/nojha-2019-02-12_21:14:17... - 10:51 PM Bug #38283 (Resolved): max-pg-per-osd tests failing
- ...
- 10:49 PM Bug #38282 (Resolved): cephtool/test.sh failure in test_mon_osd_pool_set
- ...
- 05:47 PM Feature #36737 (Resolved): Allow multi instances of "make tests" on the same machine
- 08:06 AM Feature #36737 (Pending Backport): Allow multi instances of "make tests" on the same machine
- 05:47 PM Backport #38266 (Resolved): mimic: test: Allow multi instances of "make tests" on the same machine
- 08:07 AM Backport #38266: mimic: test: Allow multi instances of "make tests" on the same machine
- h3. original description
Currently it's only possible to run `...make; make tests -j8; ctest ...` on the same mach... - 08:04 AM Backport #38266 (In Progress): mimic: test: Allow multi instances of "make tests" on the same mac...
- https://github.com/ceph/ceph/pull/26376
- 07:58 AM Backport #38266 (Resolved): mimic: test: Allow multi instances of "make tests" on the same machine
- https://github.com/ceph/ceph/pull/26376
- 04:10 PM Backport #38277 (Resolved): mimic: osd_map_message_max default is too high?
- https://github.com/ceph/ceph/pull/29242
- 04:10 PM Backport #38276 (Resolved): luminous: osd_map_message_max default is too high?
- https://github.com/ceph/ceph/pull/28640
- 04:10 PM Backport #38275 (Resolved): mimic: Fix recovery and backfill priority handling
- https://github.com/ceph/ceph/pull/27081
- 04:09 PM Backport #38274 (Resolved): luminous: Fix recovery and backfill priority handling
- https://github.com/ceph/ceph/pull/26793
- 03:32 PM Bug #38040 (Pending Backport): osd_map_message_max default is too high?
- 03:10 PM Backport #38096 (Resolved): mimic: doc/rados/configuration: refresh osdmap section
- 03:05 AM Backport #38096 (In Progress): mimic: doc/rados/configuration: refresh osdmap section
- https://github.com/ceph/ceph/pull/26373
- 03:07 PM Backport #37689 (Resolved): mimic: ceph-objectstore-tool: Add HashInfo to object dump output
- 12:03 AM Backport #37689: mimic: ceph-objectstore-tool: Add HashInfo to object dump output
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25721
merged - 03:07 PM Backport #37992 (Resolved): mimic: ec pool lost data due to snap clone
- 12:02 AM Backport #37992: mimic: ec pool lost data due to snap clone
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26077
merged - 03:01 PM Backport #38106 (Resolved): mimic: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(ho...
- 12:00 AM Backport #38106: mimic: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26239
merged - 02:49 PM Bug #27985 (Resolved): force-backfill sets forced_recovery instead of forced_backfill in 13.2.1
- 02:49 PM Backport #38111 (Resolved): mimic: force-backfill sets forced_recovery instead of forced_backfill...
- 12:06 PM Bug #23937: FAILED assert(info.history.same_interval_since != 0)
- huang jun wrote:
> @rafal what the status of this issue now? did you resolve the problem?
I had to redeploy this ... - 04:21 AM Bug #23937: FAILED assert(info.history.same_interval_since != 0)
- @rafal what the status of this issue now? did you resolve the problem?
- 08:02 AM Backport #38073 (Resolved): luminous: build/ops: Allow multi instances of "make tests" on the sam...
- 03:56 AM Bug #38034: pg stuck in backfill_wait with plenty of disk space
I created a simple fix for this scenario in https://github.com/ceph/ceph/pull/20933. The result is probably this s...- 02:58 AM Backport #38095 (In Progress): luminous: doc/rados/configuration: refresh osdmap section
- https://github.com/ceph/ceph/pull/26372
- 12:05 AM Bug #36517: client crashes osd with empty object name
- Ok, thank you Noah! Much appreciated. I'll have a look at this soon.
02/11/2019
- 09:58 PM Bug #36517: client crashes osd with empty object name
- Jesse,
This is still a bug in the latest master. Also, it appears to be worse than before--it seems as though raw ... - 09:07 PM Backport #38111: mimic: force-backfill sets forced_recovery instead of forced_backfill in 13.2.1
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26324
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
m... - 05:13 PM Bug #38172: segv in rocksdb NewIterator
- I'm guessing this is a dup of #38024
- 05:11 PM Bug #38258 (Fix Under Review): filestore: fsync(2) return value not checked
- https://github.com/ceph/ceph/pull/26366
- 02:33 PM Bug #38258 (Resolved): filestore: fsync(2) return value not checked
- WBThrottle is the main one, but there are also fsync(2) calls in the write guard code that should be checked.
- 02:48 PM Bug #24685: config options: possible inconsistency between flag 'can_update_at_runtime' and 'flag...
- Yes, I can edit them now. Thanks Nathan :)
- 02:36 PM Bug #24685: config options: possible inconsistency between flag 'can_update_at_runtime' and 'flag...
- @Tatjana - I added you to the "ceph developers" group so you should be able to change Status etc. fields.
- 02:35 PM Bug #24685 (Resolved): config options: possible inconsistency between flag 'can_update_at_runtime...
- 11:05 AM Bug #24685: config options: possible inconsistency between flag 'can_update_at_runtime' and 'flag...
- I retested it and the issue is gone from my point of view. Unfortunately I can't tell which pull request fixed it.
... - 12:35 PM Backport #38256 (Need More Info): luminous: OSD crashes when loading pgs with "FAILED assert(inte...
- 12:35 PM Backport #38256 (New): luminous: OSD crashes when loading pgs with "FAILED assert(interval.last >...
- Sage writes in https://github.com/ceph/ceph/pull/25800
We're having a hard time root causing http://tracker.ceph.c... - 12:31 PM Backport #38256 (In Progress): luminous: OSD crashes when loading pgs with "FAILED assert(interva...
- 12:30 PM Backport #38256 (Duplicate): luminous: OSD crashes when loading pgs with "FAILED assert(interval....
- 12:30 PM Bug #21142 (Pending Backport): OSD crashes when loading pgs with "FAILED assert(interval.last > l...
- 03:08 AM Backport #38243 (In Progress): mimic: scrub warning check incorrectly uses mon scrub interval
- https://github.com/ceph/ceph/pull/26356
- 01:15 AM Backport #38240 (In Progress): luminous: radosbench tests hit ENOSPC
- https://github.com/ceph/ceph/pull/26355
- 01:12 AM Backport #38239 (In Progress): mimic: radosbench tests hit ENOSPC
- https://github.com/ceph/ceph/pull/26354
02/09/2019
- 02:13 AM Bug #36739 (Fix Under Review): ENOENT in collection_move_rename on EC backfill target
- 01:49 AM Bug #38195: osd-backfill-space.sh exposes rocksdb hang
Also, seen in http://qa-proxy.ceph.com/teuthology/dzafman-2019-02-07_22:34:38-rados-wip-dzafman-testing3-distro-bas...- 01:44 AM Bug #38248 (New): qa/standalone/osd/pg-split-merge.sh TEST_import_after_merge_and_gap() test fails
http://pulpito.ceph.com/dzafman-2019-02-07_22:34:38-rados-wip-dzafman-testing3-distro-basic-smithi/
3564438 356444...- 12:42 AM Bug #38041 (Pending Backport): Fix recovery and backfill priority handling
- 12:39 AM Bug #37393 (Resolved): mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
Also available in: Atom