Activity
From 01/09/2019 to 02/07/2019
02/07/2019
- 11:57 PM Bug #36748 (Can't reproduce): ms_deliver_verify_authorizer no AuthAuthorizeHandler found for prot...
- 07:19 PM Bug #23031: FAILED assert(!parent->get_log().get_missing().is_missing(soid))
- This should be assigned to http://tracker.ceph.com/users/3114 but doesn't show up on assignee list.
- 07:08 PM Bug #37264 (Pending Backport): scrub warning check incorrectly uses mon scrub interval
- 04:46 PM Bug #36494 (Resolved): Change osd_objectstore default to bluestore
- 03:36 PM Bug #36494: Change osd_objectstore default to bluestore
- @Nathan Let's not backport this to luminous and mimic.
- 04:46 PM Backport #37995 (Rejected): luminous: Change osd_objectstore default to bluestore
- 04:46 PM Backport #37994 (Rejected): mimic: Change osd_objectstore default to bluestore
- 04:36 PM Bug #37665 (Resolved): ceph-objectstore-tool export from luminous, import to master clears same_i...
- 04:36 PM Backport #37821 (Resolved): mimic: ceph-objectstore-tool export from luminous, import to master c...
- 04:31 PM Backport #38111 (In Progress): mimic: force-backfill sets forced_recovery instead of forced_backf...
- 03:48 AM Bug #38219 (Resolved): rebuild-mondb hangs
- http://pulpito.ceph.com/sage-2019-02-06_23:33:50-rados-master-distro-basic-smithi/...
02/06/2019
- 10:46 PM Backport #38207 (In Progress): luminous: A PG repairing doesn't mean PG is damaged
- 12:33 PM Backport #38207 (Resolved): luminous: A PG repairing doesn't mean PG is damaged
- https://github.com/ceph/ceph/pull/26305
- 10:44 PM Backport #38208 (In Progress): mimic: A PG repairing doesn't mean PG is damaged
- 12:33 PM Backport #38208 (Resolved): mimic: A PG repairing doesn't mean PG is damaged
- https://github.com/ceph/ceph/pull/26304
- 10:42 PM Bug #38124: OSD down on snaptrim.
- I was theorizing in a bug scrub that maybe the PG was running behind on OSDMaps and so missing the nosnaptrim flag up...
- 10:15 PM Bug #38024: segv, heap corruption in ec encode_and_write
- related? submit_transaction and bufferlist::rebuild()...
/a/sage-2019-02-06_15:56:08-rados-wip-sage-testing-2019-... - 10:09 PM Bug #38198 (Duplicate): ceph-mon sometimes fails to start (only seen in odd-markdown.sh)
- 10:06 PM Bug #38195 (Resolved): osd-backfill-space.sh exposes rocksdb hang
- 04:44 PM Bug #38195: osd-backfill-space.sh exposes rocksdb hang
- Workaround merged, so changed priority from Urgent to High.
- 10:01 PM Bug #37804 (Closed): "monmaptool: too many arguments" in perf siute
- Seems better now http://pulpito.ceph.com/teuthology-2019-01-25_03:57:03-perf-basic-master-distro-basic-smithi/
- 09:37 PM Backport #37821: mimic: ceph-objectstore-tool export from luminous, import to master clears same_...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25856
merged - 09:29 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- /a/sage-2019-02-06_15:56:42-rados-wip-msgr2-peer-addr-distro-basic-smithi/3557216
rados/singleton-flat/valgrind-le... - 09:12 PM Bug #37797 (Pending Backport): radosbench tests hit ENOSPC
- looks like we are hitting this in mimic as well: /a/yuriw-2019-02-06_16:30:03-rados-wip-yuri4-testing-2019-02-05-1539...
- 08:59 PM Feature #38215 (New): Add bulk operation (--op bulk) to ceph-objectstore-tool
Instead of adding an individual bulk operation for rm-omap like in https://github.com/ceph/ceph/pull/22379, I sugge...- 07:58 PM Bug #37393 (In Progress): mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
- 06:15 PM Bug #38041 (Fix Under Review): Fix recovery and backfill priority handling
- 04:42 PM Bug #38027 (Resolved): osd/osd-backfill-space.sh fails
- 04:38 PM Bug #38212 (New): Rare qa/standalone/osd/osd-markdown.sh mon start-up error
http://pulpito.ceph.com/dzafman-2019-02-05_11:42:47-rados-wip-zafman-testing2-distro-basic-smithi/3553445
It hap...- 12:45 PM Bug #37618 (Resolved): Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-ra...
- 12:45 PM Backport #37688 (Resolved): mimic: Command failed on smithi191 with status 1: '\n sudo yum -y ins...
- 04:09 AM Backport #37688: mimic: Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-r...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26201
merged - 12:44 PM Bug #36686 (Resolved): osd: pg log hard limit can cause crash during upgrade
- 12:44 PM Backport #37902 (Resolved): mimic: osd: pg log hard limit can cause crash during upgrade
- 04:08 AM Backport #37902: mimic: osd: pg log hard limit can cause crash during upgrade
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26206
merged - 12:33 PM Backport #38206 (Resolved): mimic: osds allows to partially start more than N+2
- https://github.com/ceph/ceph/pull/29241
- 12:33 PM Backport #38205 (Resolved): luminous: osds allows to partially start more than N+2
- https://github.com/ceph/ceph/pull/31858
- 11:02 AM Bug #38076 (Pending Backport): osds allows to partially start more than N+2
- 09:02 AM Bug #37404 (Resolved): OSD mkfs might assert when working agains bluestore disk that already has ...
- 08:57 AM Backport #37496 (Resolved): mimic: OSD mkfs might assert when working agains bluestore disk that ...
- 04:09 AM Backport #37496: mimic: OSD mkfs might assert when working agains bluestore disk that already has...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25385
merged - 01:04 AM Documentation #23999: osd_recovery_priority is not documented (but osd_recovery_op_priority is)
- I don't see osd_recovery_op_priority documentation, so I propose adding these lines to doc/rados/configuration/pool-p...
02/05/2019
- 11:27 PM Bug #38195: osd-backfill-space.sh exposes rocksdb hang
https://github.com/ceph/ceph/pull/26290 has a workaround for this issue.- 07:03 PM Bug #38195: osd-backfill-space.sh exposes rocksdb hang
After adding code to send a SEGV on kill_daemon timeout I got the following stack traces....- 07:01 PM Bug #38195 (New): osd-backfill-space.sh exposes rocksdb hang
After increasing the timeout for backfills finish for http://tracker.ceph.com/issues/38027 we see this kill_daemons...- 11:22 PM Bug #38198 (Duplicate): ceph-mon sometimes fails to start (only seen in odd-markdown.sh)
http://pulpito.ceph.com/dzafman-2019-02-05_11:42:47-rados-wip-zafman-testing2-distro-basic-smithi/3553445
<pre...- 11:17 PM Bug #38070 (Pending Backport): A PG repairing doesn't mean PG is damaged
- 07:04 PM Bug #38011 (Closed): [Mimic version]extra null list in json output of command: ceph osd crush tre...
- 06:32 PM Bug #38011: [Mimic version]extra null list in json output of command: ceph osd crush tree --forma...
- Looks like this was from running an old monitor, and should be fixed in newer release.
- 04:56 PM Bug #37886 (Resolved): Adding back the IOPS line for client and recovery IO in cluster logs
- 04:48 PM Bug #38184 (New): osd: recovery does not preserve copy-on-write allocations between object clones...
- Hi. I've already reported it in issue 36614, but here is a more concrete case.
- Start with a bluestore Ceph clust... - 04:22 PM Backport #38140 (In Progress): luminous: Add hashinfo testing for dump command of ceph-objectstor...
- 04:20 PM Backport #38141 (In Progress): mimic: Add hashinfo testing for dump command of ceph-objectstore-tool
02/04/2019
- 10:47 PM Bug #38027: osd/osd-backfill-space.sh fails
dzafman-2019-02-04_11:24:54-rados-wip-zafman-testing2-distro-basic-smithi/3549933
Before mon shutdowns it appear...- 10:23 PM Bug #38124: OSD down on snaptrim.
- Hello,
I have collected additional information Sage asked. Attached log has debug_osd=20 set.
How this happ... - 08:49 PM Backport #38107 (Resolved): mimic: Adding back the IOPS line for client and recovery IO in cluste...
- https://github.com/ceph/ceph/pull/26208
- 08:47 PM Bug #37886: Adding back the IOPS line for client and recovery IO in cluster logs
- merged https://github.com/ceph/ceph/pull/26208
- 08:04 PM Bug #38172 (New): segv in rocksdb NewIterator
- ...
- 07:58 PM Bug #37393: mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
- /a/yuriw-2019-02-02_14:56:39-rados-wip-yuri4-testing-2019-01-31-2315-mimic-distro-basic-smithi/3542409/
- 07:55 PM Bug #38083: mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- /a/yuriw-2019-02-02_14:56:39-rados-wip-yuri4-testing-2019-01-31-2315-mimic-distro-basic-smithi/3542404/
- 06:09 PM Backport #38108 (Resolved): luminous: Adding back the IOPS line for client and recovery IO in clu...
- https://github.com/ceph/ceph/pull/26207
- 11:22 AM Backport #38163 (Resolved): mimic: maybe_remove_pg_upmaps incorrectly cancels valid pending upmaps
- Included in https://github.com/ceph/ceph/pull/27963
- 11:22 AM Backport #38162 (Resolved): luminous: maybe_remove_pg_upmaps incorrectly cancels valid pending up...
- https://github.com/ceph/ceph/pull/26127
- 05:50 AM Bug #38159 (New): ec does not recover below min_size
- ...
- 05:26 AM Bug #17257: ceph_test_rados_api_lock fails LibRadosLockPP.LockExclusiveDurPP
- /a/sage-2019-02-03_18:58:17-rados-wip-sage2-testing-2019-02-03-1047-distro-basic-smithi/3545716
02/03/2019
- 04:59 PM Bug #38023: segv on FileJournal::prepare_entry in bufferlist
- ...
- 04:43 PM Bug #24320: out of order reply and/or osd assert with set-chunks-read.yaml
- /a/kchai-2019-02-03_02:07:02-rados-wip-kefu2-testing-2019-02-03-0001-distro-basic-smithi/3543791
rados/thrash/{0-s... - 04:40 PM Bug #20798: LibRadosLockECPP.LockExclusiveDurPP gets EEXIST
- /a/kchai-2019-02-03_02:07:02-rados-wip-kefu2-testing-2019-02-03-0001-distro-basic-smithi/3543664/
- 03:24 AM Bug #38155 (Duplicate): PG stuck in undersized+degraded+remapped+backfill_toofull+peered
dzafman-2019-02-02_15:37:09-rados-wip-zafman-testing2-distro-basic-smithi/3542711
Something like this happened b...- 03:12 AM Bug #38027: osd/osd-backfill-space.sh fails
After increasing the time out, saw a different failure. As expected all 4 PG backfills completed and 4 PGs are in ...
02/02/2019
- 09:25 AM Bug #37968: maybe_remove_pg_upmaps incorrectly cancels valid pending upmaps
- https://github.com/ceph/ceph/pull/26179
- 09:25 AM Bug #37968 (Pending Backport): maybe_remove_pg_upmaps incorrectly cancels valid pending upmaps
02/01/2019
- 11:11 PM Bug #38027: osd/osd-backfill-space.sh fails
- http://pulpito.ceph.com/dzafman-2019-01-30_18:54:50-rados-wip-zafman-testing-distro-basic-smithi/3528763
- 06:08 PM Bug #38151 (New): cephx: service ticket validity dobuled
- ...
- 09:18 AM Backport #38141 (Resolved): mimic: Add hashinfo testing for dump command of ceph-objectstore-tool
- https://github.com/ceph/ceph/pull/26283
- 09:18 AM Backport #38140 (Resolved): luminous: Add hashinfo testing for dump command of ceph-objectstore-tool
- https://github.com/ceph/ceph/pull/26284
- 06:49 AM Backport #38106 (In Progress): mimic: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing...
- https://github.com/ceph/ceph/pull/26239
- 03:49 AM Backport #38105 (In Progress): luminous: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_miss...
- https://github.com/ceph/ceph/pull/26236
- 12:29 AM Feature #38136 (Resolved): core: lazy omap stat collection
- In Nautlis this PR - https://github.com/ceph/ceph/pull/18096 will bring very good support for all sizes in `ceph osd ...
- 12:12 AM Bug #38135: Ceph is in HEALTH_ERR status with inconsistent PG after some rbd snapshot creating/re...
- 1, create_rbd.sh, this is for creating rbds
2, create_snapshot.sh, this is for creating snapshots
3, delete_random_...
01/31/2019
- 11:53 PM Bug #38135 (New): Ceph is in HEALTH_ERR status with inconsistent PG after some rbd snapshot creat...
- We observe Ceph is in HEALTH_ERR status with inconsistent PG after some rbd snapshot creating/removing task. Here are...
- 06:57 PM Backport #37688: mimic: Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-r...
- @Nathan That sounds right.
- 12:05 PM Bug #38124 (Resolved): OSD down on snaptrim.
- All of ceph cluster OSD's crash when ceph runs snaptrim.
The particular error osd is throwing before crashing ... - 10:43 AM Bug #24531: Mimic MONs have slow/long running ops
- I have restarted mon.node3 and now everything is OK.
- 10:38 AM Bug #24531: Mimic MONs have slow/long running ops
- Seems, the same:...
- 10:29 AM Bug #37443 (Resolved): crushtool: add --reclassify operation to convert legacy crush maps to use ...
- 10:28 AM Backport #37437 (Resolved): mimic: crushtool: add --reclassify operation to convert legacy crush ...
- 10:27 AM Bug #37653 (Resolved): list-inconsistent-obj output truncated, causing osd-scrub-repair.sh failure
- 10:27 AM Backport #37686 (Resolved): mimic: list-inconsistent-obj output truncated, causing osd-scrub-repa...
- 10:22 AM Backport #37832 (Resolved): mimic: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- 10:19 AM Backport #38045 (Resolved): mimic: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_en...
01/30/2019
- 10:21 PM Bug #37975 (Resolved): assert failure in OSDService::shutdown()
- 10:20 PM Bug #38012: osd bad crc cause the whole cluster stop accepting new request.
- It seems the first step would be reporting the crc mismatches via a perfcounter. Then the mgr could look at those to ...
- 06:38 PM Backport #38107 (In Progress): mimic: Adding back the IOPS line for client and recovery IO in clu...
- 12:57 PM Backport #38107 (Need More Info): mimic: Adding back the IOPS line for client and recovery IO in ...
- @Vikhyat - assigning backport to you, since you volunteered to do it in https://tracker.ceph.com/issues/37886#note-10
- 12:56 PM Backport #38107 (Resolved): mimic: Adding back the IOPS line for client and recovery IO in cluste...
- https://github.com/ceph/ceph/pull/26208
- 06:25 PM Bug #38057: "ceph -s" hangs indefinitely when a machine running a monitor has failed storage.
- The node that had the failed SSD is "hoenir"
The node that I'm trying to use ceph commands from is "mimir".
I've ... - 03:37 PM Bug #38057: "ceph -s" hangs indefinitely when a machine running a monitor has failed storage.
- Is the dead node the one that isn't in quorum?
What's the ceph.conf on the client that can't complete "ceph -s"?
... - 06:16 PM Backport #38108 (In Progress): luminous: Adding back the IOPS line for client and recovery IO in ...
- 12:58 PM Backport #38108 (Need More Info): luminous: Adding back the IOPS line for client and recovery IO ...
- @Vikhyat - assigning backport to you, since you volunteered to do it in https://tracker.ceph.com/issues/37886#note-10
- 12:56 PM Backport #38108 (Resolved): luminous: Adding back the IOPS line for client and recovery IO in clu...
- https://github.com/ceph/ceph/pull/26207
- 06:03 PM Backport #37902 (In Progress): mimic: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/26206
- 05:36 PM Bug #38053 (Pending Backport): Add hashinfo testing for dump command of ceph-objectstore-tool
- 05:30 PM Feature #37935 (Pending Backport): Add clear-data-digest command to objectstore tool
- 05:07 PM Backport #37437: mimic: crushtool: add --reclassify operation to convert legacy crush maps to use...
- Mykola Golub wrote:
> https://github.com/ceph/ceph/pull/25306
merged - 05:03 PM Backport #37686: mimic: list-inconsistent-obj output truncated, causing osd-scrub-repair.sh failure
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25603
merged - 05:00 PM Backport #37832: mimic: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25852
merged - 04:56 PM Backport #38045: mimic: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/26147
merged - 01:15 PM Backport #37688: mimic: Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-r...
- @Kefu, @Neha - I rejected the luminous backport because there is no "rados/thrash-old-clients" suite in luminous. Ple...
- 01:08 PM Backport #37688 (In Progress): mimic: Command failed on smithi191 with status 1: '\n sudo yum -y ...
- 03:11 AM Backport #37688 (New): mimic: Command failed on smithi191 with status 1: '\n sudo yum -y install ...
- @Neha, sorry for the latency. and, yes, it's ready for the backport.
- 01:06 PM Backport #37687 (Rejected): luminous: Command failed on smithi191 with status 1: '\n sudo yum -y ...
- rados/thrash-old-clients does not exist in luminous
- 01:00 PM Bug #37507 (Resolved): osd_memory_target: failed assert when options mismatch
- 12:58 PM Backport #38111 (Resolved): mimic: force-backfill sets forced_recovery instead of forced_backfill...
- https://github.com/ceph/ceph/pull/26324
- 12:56 PM Backport #38106 (Resolved): mimic: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(ho...
- https://github.com/ceph/ceph/pull/26239
- 12:56 PM Backport #38105 (Resolved): luminous: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing...
- https://github.com/ceph/ceph/pull/26236
- 12:55 PM Backport #38096 (Resolved): mimic: doc/rados/configuration: refresh osdmap section
- https://github.com/ceph/ceph/pull/26373
- 12:55 PM Backport #38095 (Resolved): luminous: doc/rados/configuration: refresh osdmap section
- https://github.com/ceph/ceph/pull/26372
- 03:10 AM Bug #37618: Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-radosgw\n '
- just a note: https://github.com/ceph/teuthology/pull/1246 should be able to address the issue completely.
- 12:19 AM Bug #37919 (Pending Backport): osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
01/29/2019
- 09:31 PM Bug #38057: "ceph -s" hangs indefinitely when a machine running a monitor has failed storage.
- I'll be performing maintenance on this machine soon.
This'll be the only chance anyone gets to get more debugging ... - 06:44 PM Bug #38083 (Resolved): mimic: test_kvstore_tool.sh: mkfs failed: (22) Invalid argument
- ...
- 06:39 PM Bug #37393: mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
- /a/nojha-2019-01-29_03:40:43-rados-wip-37902-mimic-2019-01-28-distro-basic-smithi/3522520/
- 06:38 PM Bug #38082: mimic: mon/caps.sh fails with "Expected return 0, got 110"
- /a/teuthology-2018-12-29_02:30:02-rados-mimic-distro-basic-smithi/3403799/
- 06:36 PM Bug #38082 (New): mimic: mon/caps.sh fails with "Expected return 0, got 110"
- ...
- 06:18 PM Bug #23879: test_mon_osdmap_prune.sh fails
- Seen in mimic /a/nojha-2019-01-29_03:40:43-rados-wip-37902-mimic-2019-01-28-distro-basic-smithi/3522485/
- 05:01 PM Bug #38077 (New): Marking all OSDs as "out" does not trigger a HEALTH_ERR state
- Just tested this on my local 5 OSD dev environment, but this likely applies to any given cluster: when setting the cl...
- 04:37 PM Bug #38076: osds allows to partially start more than N+2
- https://github.com/ceph/ceph/pull/26177
- 04:37 PM Bug #38076 (Resolved): osds allows to partially start more than N+2
- - jewel osds
- install mimic
- try to start osds. they fail because of compatset checks etc
- ... but mimic rocks... - 04:21 PM Bug #38034: pg stuck in backfill_wait with plenty of disk space
During preemption what ensures that backfill node processes the following messages from primary in order?
Primar...- 06:04 AM Bug #38034: pg stuck in backfill_wait with plenty of disk space
I think this is where things went wrong. We've seen something like this in the past, I think. Here the osd.6 rese...- 02:58 AM Bug #38034: pg stuck in backfill_wait with plenty of disk space
Here are the enter/exit lines on the primary where we entered backfilling and then went to backfill_wait for the la...- 02:53 AM Bug #38034: pg stuck in backfill_wait with plenty of disk space
Analysis so far:
Maybe this is a backfill preemption issue. The pg is in backfill_wait state after getting Remo...- 12:23 PM Backport #38073 (In Progress): luminous: build/ops: Allow multi instances of "make tests" on the ...
- https://github.com/ceph/ceph/pull/26186
- 12:21 PM Backport #38073 (Resolved): luminous: build/ops: Allow multi instances of "make tests" on the sam...
- https://github.com/ceph/ceph/pull/26186
01/28/2019
- 10:35 PM Bug #38070 (Fix Under Review): A PG repairing doesn't mean PG is damaged
- 10:35 PM Bug #38070: A PG repairing doesn't mean PG is damaged
- https://github.com/ceph/ceph/pull/26178
- 10:27 PM Bug #38070 (Resolved): A PG repairing doesn't mean PG is damaged
- 09:00 PM Bug #37919 (Fix Under Review): osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- 08:38 PM Bug #38069 (New): upgrade:jewel-x-luminous with short_pg_log.yaml fails with assert(s <= can_roll...
- Run: http://pulpito.ceph.com/yuriw-2019-01-24_16:20:56-upgrade:jewel-x-luminous-distro-basic-smithi/
Jobs: '3501809'... - 03:17 PM Bug #38066 (New): "AdminSocketConfigObs::init: failed:" in upgrade:mimic-x-master
- Run: http://pulpito.ceph.com/teuthology-2019-01-25_02:30:02-upgrade:mimic-x-master-distro-basic-smithi/
Jobs: all
L... - 02:55 PM Bug #37269 (Resolved): Prioritize user specified scrubs
- 02:55 PM Backport #37342 (Resolved): mimic: Prioritize user specified scrubs
- 02:55 PM Bug #37507: osd_memory_target: failed assert when options mismatch
- 02:55 PM Backport #37698 (Resolved): mimic: osd_memory_target: failed assert when options mismatch
- 12:57 PM Bug #38064 (Duplicate): librados::OPERATION_FULL_TRY not completely implemented, test LibRadosAio...
- Test LibRadosAio.PoolQuotaPP hanged on
/a/sage-2019-01-28_03:48:46-rados-wip-sage2-testing-2019-01-27-1015-distro-ba... - 08:09 AM Bug #38062 (Resolved): proxy write misordering
- 1-pg-log-overrides/short_pg_log.yaml
the cache tier osd trimmed the event for the older op on the object, which di... - 07:34 AM Bug #38062 (Resolved): proxy write misordering
- out of order replies......
- 03:34 AM Bug #38057 (New): "ceph -s" hangs indefinitely when a machine running a monitor has failed storage.
- TL;DR; -- the bug is that "ceph -s" hangs indefinitely. It should report failure eventually.
I have a 3 node clu...
01/27/2019
- 08:44 PM Bug #37886 (Pending Backport): Adding back the IOPS line for client and recovery IO in cluster logs
- 03:38 AM Bug #38053 (Fix Under Review): Add hashinfo testing for dump command of ceph-objectstore-tool
- 03:37 AM Bug #38053: Add hashinfo testing for dump command of ceph-objectstore-tool
- https://github.com/ceph/ceph/pull/26158
01/26/2019
- 04:44 PM Bug #24531: Mimic MONs have slow/long running ops
- I've now encountered this on a total of 3 different clusters with 13.2.2 and 13.2.4
- 04:32 PM Bug #37886: Adding back the IOPS line for client and recovery IO in cluster logs
- 12:18 AM Bug #37886 (Pending Backport): Adding back the IOPS line for client and recovery IO in cluster logs
- Once it is merged in master I can backport it to mimic and luminous.
- 12:14 AM Bug #37886: Adding back the IOPS line for client and recovery IO in cluster logs
- Hi Neha,
As discussed I did some testing in luminous branch after adding this patch and changes looks great and wo... - 01:19 AM Bug #38053 (Resolved): Add hashinfo testing for dump command of ceph-objectstore-tool
Also, this test is broken in master, so fix that too.- 12:50 AM Bug #27985 (Pending Backport): force-backfill sets forced_recovery instead of forced_backfill in ...
01/25/2019
- 07:14 PM Documentation #38051 (Resolved): doc/rados/configuration: refresh osdmap section
- "osd map cache size" and "osd map message max" were reduced in commit
855955e ("osd: reduce size of osdmap cache, me... - 04:09 PM Bug #38027: osd/osd-backfill-space.sh fails
- http://pulpito.ceph.com/kchai-2019-01-25_08:53:00-rados-wip-kefu2-testing-2019-01-22-2130-distro-basic-smithi/3505875/
- 04:08 PM Backport #37342: mimic: Prioritize user specified scrubs
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25513
merged - 04:08 PM Backport #37698: mimic: osd_memory_target: failed assert when options mismatch
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25605
merged - 04:06 PM Backport #37814 (Resolved): mimic: workunits/rados/test_health_warnings.sh fails with <9 osds down
- 04:04 PM Backport #37814: mimic: workunits/rados/test_health_warnings.sh fails with <9 osds down
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25850
merged - 02:42 PM Bug #24531: Mimic MONs have slow/long running ops
- I see the same symptoms on a system running 13.2.2 - each monitor has a small number of slow ops, all initiated withi...
- 01:05 PM Backport #38046 (In Progress): luminous: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_...
- 10:39 AM Backport #38046 (Resolved): luminous: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log...
- https://github.com/ceph/ceph/pull/26148
- 12:59 PM Backport #38045 (In Progress): mimic: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log...
- 10:39 AM Backport #38045 (Resolved): mimic: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_en...
- https://github.com/ceph/ceph/pull/26147
- 03:51 AM Cleanup #38042 (Resolved): qa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests
- 12:56 AM Cleanup #38042 (Fix Under Review): qa/suites/rados/thrash: change crush_tunables to jewel in rado...
- 12:50 AM Cleanup #38042 (Resolved): qa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests
- 01:20 AM Bug #38041: Fix recovery and backfill priority handling
A PG in backfill_wait can be set to force-backfill state, but the reservation request has already been queued at a ...- 12:06 AM Bug #38041 (Resolved): Fix recovery and backfill priority handling
- 01:18 AM Cleanup #38025 (Pending Backport): qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_en...
01/24/2019
- 11:59 PM Bug #37393: mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
- This is still failing in mimic.
/a/yuriw-2019-01-16_22:57:44-rados-wip-yuri3-testing-2019-01-16-2038-mimic-distro-... - 11:56 PM Backport #37688: mimic: Command failed on smithi191 with status 1: '\n sudo yum -y install ceph-r...
- Seeing more of these on mimic.
Kefu, is this ready to be backported? - 09:05 PM Bug #38040: osd_map_message_max default is too high?
- Assigning Sage, as the author of commit 855955e58e63 ("osd: reduce size of osdmap cache, messages").
- 09:04 PM Bug #38040 (Resolved): osd_map_message_max default is too high?
- In a thread on ceph-users [1], three different users with fairly large clusters (~600 OSDs, ~3500 OSDs) reported runn...
- 12:31 PM Bug #38034 (Resolved): pg stuck in backfill_wait with plenty of disk space
- ...
- 03:49 AM Bug #37886 (Fix Under Review): Adding back the IOPS line for client and recovery IO in cluster logs
- 01:38 AM Bug #38012: osd bad crc cause the whole cluster stop accepting new request.
- Josh Durgin wrote:
> This is likely to be bad networking hardware - the CRC at the ceph level that is failing is des... - 01:15 AM Feature #38029 (Resolved): [RFE] If the nodeep-scrub/noscrub flags are set in pools instead of gl...
- [RFE] If the nodeep-scrub/noscrub flags are set in pools instead of global cluster. List the pool names in the ceph s...
- 01:11 AM Bug #38027: osd/osd-backfill-space.sh fails
This doesn't look like a big deal. The test expected backfilling to finish within 2 minutes. According to the log...- 12:23 AM Bug #38027 (Resolved): osd/osd-backfill-space.sh fails
- ...
- 12:24 AM Bug #21592: LibRadosCWriteOps.CmpExt got 0 instead of -4095-1
- /a/sage-2019-01-23_18:09:58-rados-wip-sage2-testing-2019-01-23-0826-distro-basic-smithi/3497934
01/23/2019
- 10:17 PM Bug #38012: osd bad crc cause the whole cluster stop accepting new request.
- This is likely to be bad networking hardware - the CRC at the ceph level that is failing is designed to detect exactl...
- 09:18 AM Bug #38012 (New): osd bad crc cause the whole cluster stop accepting new request.
- I have encounter this problem both on jewel cluster and luminous cluster.
The symptom is some request will be blocke... - 10:16 PM Bug #37975: assert failure in OSDService::shutdown()
- 10:15 PM Bug #37978 (Duplicate): osd killed by kernel for Segmentation fault
- 09:04 PM Bug #24531: Mimic MONs have slow/long running ops
- I am also seeing this on latest mimic (13.2.4). So far it seems like its cosmetic and has no impact....
- 08:49 PM Cleanup #38025 (Fix Under Review): qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_en...
- 08:45 PM Cleanup #38025 (Resolved): qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries
- We have noticed that a very short pg log helps catch more bugs, hence make osd_min_pg_log_entries=1 and osd_max_pg_lo...
- 08:01 PM Bug #36739: ENOENT in collection_move_rename on EC backfill target
- /a/nojha-2019-01-23_02:37:14-rados:thrash-erasure-code-master-distro-basic-smithi/3494110
- 07:58 PM Bug #38024 (Resolved): segv, heap corruption in ec encode_and_write
- ...
- 07:55 PM Bug #38023 (Closed): segv on FileJournal::prepare_entry in bufferlist
- ...
- 07:53 PM Bug #37509: require past_interval bounds mismatch due to osd oldest_map
- /a/nojha-2019-01-23_02:37:14-rados:thrash-erasure-code-master-distro-basic-smithi/3494085/
- 05:23 PM Bug #38011: [Mimic version]extra null list in json output of command: ceph osd crush tree --forma...
- Not a ceph-deploy issue but should fall under ceph
- 04:47 AM Bug #38011: [Mimic version]extra null list in json output of command: ceph osd crush tree --forma...
- Changcheng Liu wrote:
> Changcheng Liu wrote:
> > The extra null list should be removed
> > [{"id":-1,"name":"d... - 04:47 AM Bug #38011: [Mimic version]extra null list in json output of command: ceph osd crush tree --forma...
- Changcheng Liu wrote:
> The extra null list should be removed
> [{"id":-1,"name":"default","type":"root","type_i... - 04:46 AM Bug #38011: [Mimic version]extra null list in json output of command: ceph osd crush tree --forma...
- The extra null list should be removed
[{"id":-1,"name":"default","type":"root","type_id":10,"children":[]}]*[]* - 04:40 AM Bug #38011 (Closed): [Mimic version]extra null list in json output of command: ceph osd crush tre...
- When executing below command to get osd crush tree on Mimic version, it will output extra null list[] which makes jso...
- 11:44 AM Feature #36737: Allow multi instances of "make tests" on the same machine
- https://github.com/ceph/ceph/pull/26091
- 11:31 AM Bug #37966 (Resolved): cli: dump osd-fsid as part of osd find <id>
- 04:36 AM Bug #36498: failed to recover before timeout expired due to pg stuck in creating+peering
- Still seeing PG stuck in "creating+peering".
/a/nojha-2019-01-23_02:37:14-rados:thrash-erasure-code-master-distro-... - 02:47 AM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- Thanks a lot.
> disabling THP or setting max_ptes_none to 0
I will try this later and see if that helps. Since it ...
01/22/2019
- 05:07 PM Bug #37264: scrub warning check incorrectly uses mon scrub interval
- 05:06 PM Bug #19753 (Resolved): Deny reservation if expected backfill size would put us over backfill_full...
- 05:06 PM Bug #24801 (Resolved): PG num_bytes becomes huge
- 04:02 PM Backport #37984 (Resolved): mimic: cli: dump osd-fsid as part of osd find <id>
- 03:35 PM Backport #37984: mimic: cli: dump osd-fsid as part of osd find <id>
- https://github.com/ceph/ceph/pull/26035
- 04:02 PM Backport #37985 (Resolved): luminous: cli: dump osd-fsid as part of osd find <id>
- 03:35 PM Backport #37985: luminous: cli: dump osd-fsid as part of osd find <id>
- https://github.com/ceph/ceph/pull/26036
- 03:55 PM Backport #37993 (In Progress): luminous: ec pool lost data due to snap clone
- 03:50 PM Backport #37992 (In Progress): mimic: ec pool lost data due to snap clone
- 03:47 PM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- Hi,
Often times this kind of thing is related to transparent huge pages. There definitely seems to be different k... - 02:16 PM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- > ceph 12.2.1
Are you really running that version, 12.2.1 ? - 03:38 AM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- I am using bluestore, and my client is rbd with ec datapool.
The cluster is running on Centos 7.0.1406, tcmalloc ver... - 02:54 PM Backport #37995 (In Progress): luminous: Change osd_objectstore default to bluestore
- 02:52 PM Backport #37994 (In Progress): mimic: Change osd_objectstore default to bluestore
- 12:58 PM Backport #37903 (Resolved): luminous: osd: pg log hard limit can cause crash during upgrade
- 12:40 PM Bug #36515 (Resolved): config options: 'services' field is empty for many config options
- 09:38 AM Bug #38000 (Duplicate): The osd shutdown procedure accesses the memory that has been released
- https://tracker.ceph.com/issues/37975
- 03:54 AM Bug #38000: The osd shutdown procedure accesses the memory that has been released
- int OSD::shutdown()
{
store->umount();
delete store; // The cache is destroyed
store = 0;
...
... - 03:52 AM Bug #38000 (Duplicate): The osd shutdown procedure accesses the memory that has been released
- [Switching to thread 2 (Thread 0x7f7314cc8700 (LWP 32025))]
#0 0x00007f73395a842d in __lll_lock_wait () from /lib64... - 09:24 AM Bug #37871: Ceph cannot connect to any monitors if one of them has a DNS resolution problem
- In practical terms, what's the difference between not being able to connect because the host name cannot be resolved,...
- 03:32 AM Bug #37871: Ceph cannot connect to any monitors if one of them has a DNS resolution problem
- i think the unresolvable address(es) is more of a configuration issue. and we should not ignore this. it's quite diff...
01/21/2019
- 10:19 PM Bug #37875: osdmaps aren't being cleaned up automatically on healthy cluster
- Sounds like Dan's is behaving as expected, but if there's any more info about Bryan's let us know.
- 11:58 AM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- And what OS are you using?
- 11:56 AM Bug #37980: luminous: osd memery use very high,and missmatch between res and heap stats
- Are you using FileStore or BlueStore?
- 03:36 AM Bug #37980 (New): luminous: osd memery use very high,and missmatch between res and heap stats
- ceph 12.2.1
3 nodes, 30 osds per node
ec pool:4+2
After running for 2 months,we find some osds memery use very h... - 10:36 AM Backport #37904 (In Progress): mimic: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in Pr...
- 10:05 AM Backport #37905 (In Progress): luminous: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in...
- 09:16 AM Backport #37995 (Rejected): luminous: Change osd_objectstore default to bluestore
- https://github.com/ceph/ceph/pull/26076
- 09:16 AM Backport #37994 (Rejected): mimic: Change osd_objectstore default to bluestore
- https://github.com/ceph/ceph/pull/26075
- 09:15 AM Backport #37993 (Resolved): luminous: ec pool lost data due to snap clone
- https://github.com/ceph/ceph/pull/26078
- 09:15 AM Backport #37992 (Resolved): mimic: ec pool lost data due to snap clone
- https://github.com/ceph/ceph/pull/26077
- 09:14 AM Backport #37985 (Resolved): luminous: cli: dump osd-fsid as part of osd find <id>
- https://github.com/ceph/ceph/pull/26036
- 09:13 AM Backport #37984 (Resolved): mimic: cli: dump osd-fsid as part of osd find <id>
- 02:22 AM Bug #37978 (Duplicate): osd killed by kernel for Segmentation fault
- My env is:
[root@gz-ceph-52-204 ceph]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
[root@gz-ceph-...
01/20/2019
- 06:31 AM Bug #37975 (Fix Under Review): assert failure in OSDService::shutdown()
- 05:27 AM Bug #37975: assert failure in OSDService::shutdown()
- the return value was 22, as the mutex being acquired was destroyed already.
- 05:12 AM Bug #37975 (Resolved): assert failure in OSDService::shutdown()
- ...
- 02:38 AM Bug #37593 (Pending Backport): ec pool lost data due to snap clone
01/19/2019
- 04:23 PM Backport #37972 (In Progress): luminous: FreeBSD/Linux integration - monitor map with wrong sa_fa...
- 04:22 PM Backport #37972: luminous: FreeBSD/Linux integration - monitor map with wrong sa_family
- PR: https://github.com/ceph/ceph/pull/26042
- 04:06 PM Backport #37972: luminous: FreeBSD/Linux integration - monitor map with wrong sa_family
- Need to backport https://github.com/ceph/ceph/pull/17615/commits/9099ca599de5238cde917f1e1f933247392de03e
- 04:05 PM Backport #37972 (Resolved): luminous: FreeBSD/Linux integration - monitor map with wrong sa_family
- https://github.com/ceph/ceph/pull/26042
- 09:23 AM Backport #37438 (Resolved): luminous: crushtool: add --reclassify operation to convert legacy cru...
- 02:07 AM Bug #37969 (Can't reproduce): ENOENT on setattrs
- ...
01/18/2019
- 11:04 PM Bug #23145: OSD crashes during recovery of EC pg
- I've generated a log for this at https://www.dropbox.com/s/8zoos5hhvakcpc4/ceph-osd.3.log?dl=0
haven't been able t... - 10:43 PM Bug #37968 (Resolved): maybe_remove_pg_upmaps incorrectly cancels valid pending upmaps
- It appears that OSDMap::maybe_remove_pg_upmaps's sanity checks are overzealous. With some crush rules it is possible ...
- 09:56 PM Backport #37438: luminous: crushtool: add --reclassify operation to convert legacy crush maps to ...
- Mykola Golub wrote:
> https://github.com/ceph/ceph/pull/25307
merged - 08:38 PM Backport #37903: luminous: osd: pg log hard limit can cause crash during upgrade
- Neha Ojha wrote:
> https://github.com/ceph/ceph/pull/25949
merged - 07:22 PM Backport #37903: luminous: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/25949
- 01:11 PM Backport #37903 (Need More Info): luminous: osd: pg log hard limit can cause crash during upgrade
- Marking "Need More Info" just to make sure backporting team doesn't take it by accident.
- 07:44 PM Bug #37966 (Resolved): cli: dump osd-fsid as part of osd find <id>
- https://github.com/ceph/ceph/pull/26015
- 05:32 PM Bug #37965 (Can't reproduce): rados/upgrade test fails
- recent regression. looking at /a/sage-2019-01-18_06:11:36-rados-wip-sage-testing-2019-01-17-2111-distro-basic-smithi...
- 02:37 PM Bug #24676 (Pending Backport): FreeBSD/Linux integration - monitor map with wrong sa_family
- Richard, i don't think 9099ca5 was ever backported to luminous. if you want to get it fixed sooner in luminous. proba...
- 01:26 PM Bug #36515: config options: 'services' field is empty for many config options
- I think with https://github.com/ceph/ceph/pull/25456 the issue can be resolved. I'm not allowed to do it myself.
- 01:11 PM Backport #37902 (Need More Info): mimic: osd: pg log hard limit can cause crash during upgrade
- Marking "Need More Info" just to make sure backporting team doesn't take it by accident.
- 03:29 AM Bug #19753: Deny reservation if expected backfill size would put us over backfill_full_ratio
- 12:51 AM Bug #36494 (Pending Backport): Change osd_objectstore default to bluestore
01/17/2019
- 03:43 PM Bug #37910: segv during crc of incoming message front
- Putting on shelf in the sake of msgr V2.
Runs on wip-bug-37910 with **client** failures:
* http://pulpito.ceph.co... - 11:41 AM Bug #36741 (Resolved): debian: packaging need to reflect move of /etc/bash_completion.d/radosgw-a...
- 11:40 AM Backport #37274 (Resolved): luminous: debian: packaging need to reflect move of /etc/bash_complet...
01/16/2019
- 02:40 PM Bug #37910: segv during crc of incoming message front
- Hmm, interesting. The same thread 0x7f6ea2dad700 is handling two instances of AsyncConnection: 0x5615360ef000
and th... - 10:32 AM Backport #37806 (Resolved): luminous: OSD logs are not logging slow requests
- 10:15 AM Feature #37935 (Resolved): Add clear-data-digest command to objectstore tool
- There may be a situation where data digest in object info is
inconsistent with that computed from object data, then ... - 12:42 AM Bug #37930 (New): osd/PrimaryLogPG.cc: 11997: FAILED ceph_assert(object_c ontexts.empty())
- @2019-01-15T11:29:05.078 INFO:tasks.ceph.osd.1.smithi055.stderr:2019-01-15 11:29:05.069 7f35017d1700 -1 osd.1 pg_epoc...
01/15/2019
- 09:37 PM Bug #37910: segv during crc of incoming message front
- ...
- 08:34 PM Bug #37910: segv during crc of incoming message front
- ...
- 05:18 PM Bug #37919: osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- Looks like we are testing with leveldb here, not sure that matters for the purpose of this bug, but we could get rid ...
- 01:14 PM Bug #37919 (Resolved): osd/ECBackend.cc: 1547: FAILED ceph_assert(!(*m).is_missing(hoid))
- ...
- 03:52 PM Bug #36163 (Resolved): mon osdmap cash too small during upgrade to mimic
- 03:51 PM Backport #36506 (Resolved): luminous: mon osdmap cash too small during upgrade to mimic
- 03:35 PM Backport #37343 (Resolved): luminous: Prioritize user specified scrubs
- 03:34 PM Backport #37697 (Resolved): luminous: osd_memory_target: failed assert when options mismatch
- 02:40 PM Bug #37875: osdmaps aren't being cleaned up automatically on healthy cluster
- Hrm.. actually, after enabling debug_paxos=10 on the mon leader, I see that there's a hysteresis between 500 and 750:...
- 02:12 PM Bug #37875: osdmaps aren't being cleaned up automatically on healthy cluster
- I just updated a cluster from v12.2.8 to 12.2.10.
At the beginning we had:
"oldest_map": 281368,
"newes... - 01:56 PM Bug #22597 (Resolved): "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails in upgrade test
- 01:56 PM Backport #37288 (Resolved): mimic: "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails in u...
- 09:14 AM Bug #24531: Mimic MONs have slow/long running ops
- I've seen this on a 13.2.2 cluster after restarting OSDs
- 06:55 AM Backport #37904: mimic: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in ProtocolV1::repl...
- https://github.com/ceph/ceph/pull/25958
- 06:06 AM Documentation #24924 (Resolved): doc: typo in crush-map docs
- 04:07 AM Documentation #24924: doc: typo in crush-map docs
- Don't care one way or another. Go ahead if you want.
- 03:52 AM Documentation #24924: doc: typo in crush-map docs
- Hi Michael,
Thank you for reporting this typo. I opened a PR to correct. Is it ok if I
credit you for this repo... - 05:54 AM Backport #37905: luminous: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in ProtocolV1::r...
- https://github.com/ceph/ceph/pull/25956
01/14/2019
- 11:49 PM Bug #37915 (Can't reproduce): osd: Segmentation fault in OpRequest::_unregistered
- ...
- 03:06 PM Bug #37911 (Can't reproduce): osd dequeue misorder
- ...
- 01:25 PM Bug #37910 (Resolved): segv during crc of incoming message front
- ...
- 01:21 PM Feature #36474 (Resolved): Add support for osd_delete_sleep configuration value
- 01:21 PM Backport #36729 (Resolved): mimic: Add support for osd_delete_sleep configuration value
- 10:43 AM Backport #37905 (Resolved): luminous: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in Pr...
- https://github.com/ceph/ceph/pull/25956
- 10:43 AM Backport #37904 (Resolved): mimic: FAILED ceph_assert(can_write == WriteStatus::NOWRITE) in Proto...
- https://github.com/ceph/ceph/pull/25958
- 10:42 AM Backport #37903 (Resolved): luminous: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/25949
- 10:42 AM Backport #37902 (Resolved): mimic: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/26206
01/13/2019
- 03:46 AM Bug #37886: Adding back the IOPS line for client and recovery IO in cluster logs
- I did some search in the code and testing with vstart cluster and we need to bring back this commit.
https://github... - 03:37 AM Bug #37886 (Resolved): Adding back the IOPS line for client and recovery IO in cluster logs
- In luminous cluster logs, client and recovery IOPS log lines were removed it used to help a lot in RCA. These logs we...
- 01:50 AM Bug #37795 (Resolved): luminous: "'hello_world_cpp' failed" in rados
01/12/2019
- 10:06 PM Bug #37795: luminous: "'hello_world_cpp' failed" in rados
- Brad Hubbard wrote:
> https://github.com/ceph/ceph/pull/25829
merged - 04:26 PM Bug #37868 (Resolved): qa/standalone/mon/mon-handle-forward.sh failure
- 11:41 AM Bug #37868 (Fix Under Review): qa/standalone/mon/mon-handle-forward.sh failure
- https://github.com/ceph/ceph/pull/25902
- 11:11 AM Bug #21592: LibRadosCWriteOps.CmpExt got 0 instead of -4095-1
- /a/sage-2019-01-11_13:08:30-rados-wip-sage-testing-2019-01-10-1950-distro-basic-smithi/3447059
01/11/2019
- 06:34 PM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- Nathan, can you please help generate backport tracker tickets for this?
- 04:17 PM Bug #37875 (Duplicate): osdmaps aren't being cleaned up automatically on healthy cluster
- After doing an expansion from ~1,500 OSDs to ~1,900 OSDs on a Luminous 12.2.8 cluster using FileStore, I've noticed t...
- 01:40 PM Backport #37806: luminous: OSD logs are not logging slow requests
- https://github.com/ceph/ceph/pull/25824 merged
- 11:39 AM Bug #37871 (New): Ceph cannot connect to any monitors if one of them has a DNS resolution problem
- My ceph cluster is configured with this:...
01/10/2019
- 10:45 PM Bug #37868 (Resolved): qa/standalone/mon/mon-handle-forward.sh failure
- /a/sage-2019-01-10_15:03:47-rados-wip-sage-testing-2019-01-10-0709-distro-basic-smithi/3442655...
- 10:29 PM Bug #37807 (Resolved): osd: valgrind catches InvalidRead
- 09:34 PM Feature #36737: Allow multi instances of "make tests" on the same machine
- @Kefu FYI
- 06:03 PM Backport #37274: luminous: debian: packaging need to reflect move of /etc/bash_completion.d/rados...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24997
merged - 04:21 AM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/25816
https://github.com/ceph/ceph/pull/25887 - 12:47 AM Cleanup #37852 (New): qa/suites/rados/thrash: review coverage
01/09/2019
- 10:29 PM Bug #37786: test fails in mon/crush_ops.sh
- This does look weird to me as well; we should investigate. I can't offhand think of any way for a bug like that to im...
- 10:29 PM Bug #37777 (Closed): OSD dies on assert triggered by a spicific other OSD joining the cluster
- 10:25 PM Bug #37788 (Closed): ceph osd process run out of memory
- Unfortunately there are a number of known issues with tcmalloc and hugepages. I don't think I've seen it this bad bef...
- 10:21 PM Bug #37808: osd: osdmap cache weak_refs assert during shutdown
- Apparently these are popping up again so we should try and track them down, but note that since https://github.com/ce...
- 10:12 PM Bug #23879: test_mon_osdmap_prune.sh fails
- We aren't hitting this in recent rados runs anymore
- 10:05 PM Bug #37797 (Resolved): radosbench tests hit ENOSPC
- https://github.com/ceph/ceph/pull/25801
- 09:41 PM Bug #36686 (Pending Backport): osd: pg log hard limit can cause crash during upgrade
- 09:07 PM Bug #25182: Upmaps forgotten after restarting OSDs
- After upgrading to 13.2.4 this problem went away. I believe this was the change which made it happen:
https://git... - 03:21 PM Bug #37844 (New): OSD medium errors do not generate warning or error
- Hi,
I've been seeing inconsistent pgs for a few times past weeks... - 01:36 PM Bug #37671 (Resolved): race between split and pg create
- 12:46 PM Bug #37840 (New): FAILED assert(0 == "we got a bad state machine event") after upgrade from 13.2....
- Running a 3 node cluster, no issues on two of the hosts, but one of the hosts has osds crashing like this:
--- beg... - 05:59 AM Bug #37807: osd: valgrind catches InvalidRead
- 05:13 AM Backport #37821 (In Progress): mimic: ceph-objectstore-tool export from luminous, import to maste...
- 04:25 AM Backport #37833 (In Progress): luminous: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- 04:22 AM Backport #37832 (In Progress): mimic: FAILED assert(is_up(osd)) in OSDMap::get_inst(int)
- 04:14 AM Backport #37815 (In Progress): luminous: workunits/rados/test_health_warnings.sh fails with <9 os...
- 04:11 AM Backport #37814 (In Progress): mimic: workunits/rados/test_health_warnings.sh fails with <9 osds ...
Also available in: Atom