Activity
From 05/12/2020 to 06/10/2020
06/10/2020
- 09:30 PM Bug #45916 (Fix Under Review): cls_lock: unlimited shared lock created by libradosstriper api let...
- 09:25 PM Bug #43861 (Pending Backport): ceph_test_rados_watch_notify hang
- Let's remove these tests from the stable branches too.
- 09:02 AM Feature #41564 (In Progress): Issue health status warning if num_shards_repaired exceeds some thr...
- 12:25 AM Bug #44314 (Pending Backport): osd-backfill-stats.sh failing intermittently in TEST_backfill_size...
06/09/2020
- 09:34 PM Backport #45780 (Resolved): nautilus: rados/test_envlibrados_for_rocksdb.sh build failure (seen i...
- 02:58 PM Backport #45780: nautilus: rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35387
merged - 09:02 PM Bug #42716: Pool creation error message is hidden on FileStore-backed pools
- That wasn't the initial issue reported.
What happen if you run "ceph osd pool create foo2 2048" instead ? (assumin... - 07:38 PM Bug #42716 (Resolved): Pool creation error message is hidden on FileStore-backed pools
- closing this as already resolved....
- 02:41 PM Bug #36337: OSDs crash with failed assertion in PGLog::merge_log as logs do not overlap
- ...
- 02:41 PM Bug #45956 (New): verify takes forever to finish
- rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thra...
- 12:24 PM Bug #45661 (Resolved): valgrind issue: UninitValue in ProtocolV2
- In @master@ the PR #35407 has been closed in favor of https://github.com/ceph/ceph/pull/35186.
#35407 still might be... - 06:34 AM Bug #45948 (Duplicate): ceph_test_rados_delete_pools_parallel failed with error -2 on nautilus
- Oops, this is a dup of #43887
- 06:31 AM Bug #45948 (Duplicate): ceph_test_rados_delete_pools_parallel failed with error -2 on nautilus
- /a/yuriw-2020-06-08_16:06:08-rados-wip-yuri2-testing-2020-06-08-1458-nautilus-distro-basic-smithi/5129541...
- 06:06 AM Bug #45947: ceph_test_rados_watch_notify hang seen in nautilus
- Note https://tracker.ceph.com/issues/43861 removed this test from master because it was hanging.
- 06:02 AM Bug #45947: ceph_test_rados_watch_notify hang seen in nautilus
- This is very similar to what is seen in #45946 so they may be related.
- 06:01 AM Bug #45947 (New): ceph_test_rados_watch_notify hang seen in nautilus
- /a/yuriw-2020-06-08_16:06:08-rados-wip-yuri2-testing-2020-06-08-1458-nautilus-distro-basic-smithi/5129565...
- 05:32 AM Bug #45946 (New): ceph_test_rados_delete_pools_parallel hang seen in octopus
- /a/yuriw-2020-05-29_15:51:00-rados-wip-yuri-testing-2020-05-28-2238-octopus-distro-basic-smithi/5103106...
- 04:28 AM Bug #20960: ceph_test_rados: mismatched version (due to pg import/export)
- ...
- 12:05 AM Bug #44510: osd/osd-recovery-space.sh TEST_recovery_test_simple failure
- Seen again:
http://pulpito.ceph.com/dzafman-2020-06-08_11:45:40-rados-wip-zafman-testing-distro-basic-smithi/5130114
06/08/2020
- 11:51 PM Bug #43888: osd/osd-bench.sh 'tell osd.N bench' hang
- Saw this in at least 17 jobs:
http://pulpito.ceph.com/dzafman-2020-06-08_11:45:40-rados-wip-zafman-testing-distro-... - 11:39 PM Bug #45944 (Triaged): osd/osd-markdown.sh: TEST_osd_stop failed
- This appears to be a rare condition when 15 seconds sleep was not enough.
- 09:14 PM Bug #45944 (Triaged): osd/osd-markdown.sh: TEST_osd_stop failed
- ...
- 09:10 PM Bug #45318: Health check failed: 2/6 mons down, quorum b,a,c,e (MON_DOWN)" in cluster log running...
- rados/multimon/{clusters/21 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados suppor...
- 07:39 PM Bug #45943 (Fix Under Review): Ceph Monitor heartbeat grace period does not reset.
- 07:09 PM Bug #45943 (Resolved): Ceph Monitor heartbeat grace period does not reset.
- The heartbeat grace timer does not reset after cluster network is stable for multiple days.
Implement a mechanism to... - 06:31 PM Backport #45891 (In Progress): luminous: osd: pg stuck in waitactingchange when new acting set do...
- 06:22 PM Backport #45892 (In Progress): mimic: osd: pg stuck in waitactingchange when new acting set doesn...
- 12:51 PM Bug #45795 (Fix Under Review): PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_back...
- 07:01 AM Bug #45916: cls_lock: unlimited shared lock created by libradosstriper api let node crash
- add pr: https://github.com/ceph/ceph/pull/35467
- 06:50 AM Bug #45916 (Fix Under Review): cls_lock: unlimited shared lock created by libradosstriper api let...
- _Background: Ceph liminous are running on our production and a service uses libradosstriper api to access ceph._
W...
06/06/2020
- 08:45 AM Backport #45357 (Resolved): octopus: rados: Sharded OpWQ drops suicide_grace after waiting for work
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/34881
m... - 08:31 AM Backport #45884 (In Progress): octopus: osd-scrub-repair.sh: SyntaxError: invalid syntax
- 08:31 AM Backport #45882 (In Progress): octopus: Objecter: don't attempt to read from non-primary on EC pools
- 08:30 AM Backport #45779 (In Progress): octopus: rados/test_envlibrados_for_rocksdb.sh build failure (seen...
- 08:29 AM Backport #45775 (In Progress): octopus: build_incremental_map_msg missing incremental map while s...
- 08:28 AM Backport #45673 (In Progress): octopus: qa: powercycle: install task runs twice with double unwin...
- 12:53 AM Bug #44314 (In Progress): osd-backfill-stats.sh failing intermittently in TEST_backfill_sizeup_ou...
06/05/2020
- 10:52 PM Bug #44314: osd-backfill-stats.sh failing intermittently in TEST_backfill_sizeup_out() (degraded ...
It would be helpful to see the osd logs when this happens. We are expecting the following sequence to occur.
St...- 04:20 PM Bug #45721: CommandFailedError: Command failed (workunit test rados/test_python.sh) FAIL: test_ra...
- /a/yuriw-2020-06-04_18:03:48-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5117777
- 04:17 PM Bug #45424: api_watch_notify_pp: [ FAILED ] LibRadosWatchNotifyECPP.WatchNotify watch_notify_cx...
- /a/yuriw-2020-06-04_18:03:48-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5117783
- 04:01 PM Bug #20960: ceph_test_rados: mismatched version (due to pg import/export)
- /a/yuriw-2020-06-04_18:03:48-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5118028
- 03:58 PM Bug #44517: osd/osd-backfill-space.sh TEST_backfill_multi_partial: pgs didn't go active+clean
- ...
06/04/2020
- 09:15 PM Bug #45868: rados_api_tests: LibRadosWatchNotify.AioWatchNotify2 fails
- Similar...
- 09:06 PM Bug #45661 (Fix Under Review): valgrind issue: UninitValue in ProtocolV2
- https://github.com/ceph/ceph/pull/35407
- 10:07 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- Pin-pointed to a branch of @PrimaryLogPG::do_manifest_flush()@:...
- 08:36 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- ...
- 06:08 PM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- Ah, that makes sense. It should suffice to simply not populate_obc_watchers if replica.
- 05:42 PM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- After more digging, this doesn't appear to be related to notifies being sent to replicas.
The issue seems to be wi... - 12:48 PM Backport #45890 (In Progress): nautilus: osd: pg stuck in waitactingchange when new acting set do...
- 11:58 AM Backport #45890 (Resolved): nautilus: osd: pg stuck in waitactingchange when new acting set doesn...
- https://github.com/ceph/ceph/pull/35389
- 12:44 PM Backport #45883 (In Progress): nautilus: osd-scrub-repair.sh: SyntaxError: invalid syntax
- 11:55 AM Backport #45883 (Resolved): nautilus: osd-scrub-repair.sh: SyntaxError: invalid syntax
- https://github.com/ceph/ceph/pull/35388
- 12:44 PM Backport #45780 (In Progress): nautilus: rados/test_envlibrados_for_rocksdb.sh build failure (see...
- 12:43 PM Backport #45776 (In Progress): nautilus: build_incremental_map_msg missing incremental map while ...
- 11:59 AM Backport #45892 (Rejected): mimic: osd: pg stuck in waitactingchange when new acting set doesn't ...
- https://github.com/ceph/ceph/pull/35484
- 11:59 AM Backport #45891 (Rejected): luminous: osd: pg stuck in waitactingchange when new acting set doesn...
- https://github.com/ceph/ceph/pull/35485
- 11:55 AM Backport #45884 (Resolved): octopus: osd-scrub-repair.sh: SyntaxError: invalid syntax
- https://github.com/ceph/ceph/pull/35445
- 11:55 AM Backport #45882 (Resolved): octopus: Objecter: don't attempt to read from non-primary on EC pools
- https://github.com/ceph/ceph/pull/35444
- 07:16 AM Bug #45871 (New): Incorrect (0) number of slow requests in health check
- ceph version 14.2.9-899-gc02349c600 (c02349c60052aaa6c7bd0c2270c7f7be16fab632) nautilus (stable)
Our cluster shows... - 12:24 AM Bug #40117 (Duplicate): PG stuck in WaitActingChange
- Fixed in https://tracker.ceph.com/issues/41190
- 12:21 AM Bug #41190 (Pending Backport): osd: pg stuck in waitactingchange when new acting set doesn't change
- 12:20 AM Bug #41236 (Resolved): cosbench failures in rados/perf
- 12:18 AM Bug #41550 (Resolved): os/bluestore: fadvise_flag leak in generate_transaction
- 12:17 AM Bug #41677 (Resolved): Cephmon:fix mon crash
- Fixed as a part of https://tracker.ceph.com/issues/41680.
- 12:14 AM Bug #41913 (Resolved): With auto scaler operating stopping an OSD can lead to COT crashing instea...
- 12:08 AM Bug #45356 (Resolved): nautilus: rados/upgrade/mimic-x-singleton failures due to mon_client_direc...
06/03/2020
- 09:06 PM Bug #45733 (Pending Backport): osd-scrub-repair.sh: SyntaxError: invalid syntax
- 06:12 PM Bug #45733: osd-scrub-repair.sh: SyntaxError: invalid syntax
- https://github.com/ceph/ceph/pull/35279 merged
- 08:50 PM Backport #45357: octopus: rados: Sharded OpWQ drops suicide_grace after waiting for work
- Dan Hill wrote:
> https://github.com/ceph/ceph/pull/34881
merged - 08:34 PM Bug #45868 (Resolved): rados_api_tests: LibRadosWatchNotify.AioWatchNotify2 fails
- ...
- 08:30 PM Bug #45761: mon_thrasher: "Error ENXIO: mon unavailable" during sync_force command leads to "fail...
- /a/yuriw-2020-06-02_15:07:59-rados-wip-yuri7-testing-2020-06-01-2256-octopus-distro-basic-smithi/5113082 - octopus
- 04:44 AM Bug #45761: mon_thrasher: "Error ENXIO: mon unavailable" during sync_force command leads to "fail...
- Moving this since it appears to be a problem with the mon_thrasher (or the MONs or monclients)....
- 02:44 PM Bug #45793 (Pending Backport): Objecter: don't attempt to read from non-primary on EC pools
- 01:24 PM Backport #41533: mimic: Move bluefs alloc size initialization log message to log level 1
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/30219
m... - 12:59 PM Bug #45857 (New): crimson/alien_store: alienstore cannot open_collections
- setup: setting debug level 20 for bluestore, filestore and osd and using seastar with seastar_default_allocator + Rel...
- 01:50 AM Bug #9984: lttng_probe_unregister hangs on shutdown
- /a/yuriw-2020-05-30_02:18:17-rados-wip-yuri-master_5.29.20-distro-basic-smithi/5104372
Possibly an instance of thi...
06/02/2020
- 07:14 PM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- I see. Watch being a write and notify being a read has always tripped me, but I guess I looked at it from the side e...
- 03:28 PM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- Well, osd-side notifies are reads in that they don't result in mutation. I think lingerops in general probably shoul...
- 10:38 AM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- Samuel Just wrote:
> Did that fire on the replica? At a guess, the problem is that notifies are being sent to repli... - 02:07 AM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- It probably isn't https://tracker.ceph.com/issues/15391.
- 02:05 AM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- Did that fire on the replica? At a guess, the problem is that notifies are being sent to replicas, which would be wr...
- 07:08 PM Bug #45802 (Resolved): Health check failed: Reduced data availability: PG_AVAILABILITY
- 06:19 PM Bug #45802 (Fix Under Review): Health check failed: Reduced data availability: PG_AVAILABILITY
- 06:17 PM Bug #45802 (Triaged): Health check failed: Reduced data availability: PG_AVAILABILITY
- Same root cause as https://tracker.ceph.com/issues/45619.
http://pulpito.ceph.com/teuthology-2020-05-30_03:05:02... - 07:16 AM Bug #45809 (New): When out a osd, the `MAX AVAIL` doesn't change.
- Environment: Luminous 12.2.12
I have a question about the pool's `MAX AVAIL` of `ceph df`.
When i out a osd, th... - 06:00 AM Bug #45761: mon_thrasher: "Error ENXIO: mon unavailable" during sync_force command leads to "fail...
- /a/yuriw-2020-05-30_02:18:17-rados-wip-yuri-master_5.29.20-distro-basic-smithi/5104057
- 05:13 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- /a/yuriw-2020-05-30_02:18:17-rados-wip-yuri-master_5.29.20-distro-basic-smithi/5103952
/a/yuriw-2020-05-30_02:18:17-...
06/01/2020
- 03:21 PM Bug #45802 (Resolved): Health check failed: Reduced data availability: PG_AVAILABILITY
- multiple RGW tests are failing on different branches, with:...
- 12:13 AM Bug #45796 (New): Ceph mon's sporadically report slow ops
- We have recently upgraded our cluster to 14.2.9 from 10.2.6 and are in the process of a rolling rebuild of many of th...
05/31/2020
- 01:20 PM Bug #45795: PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().empty())
- Sam, could you please take a look?
- 01:19 PM Bug #45795 (Resolved): PrimaryLogPG.cc: 627: FAILED ceph_assert(!get_acting_recovery_backfill().e...
- I'm running into this assert while trying to exercise krbd with replica reads (particularly balanced reads):...
- 12:34 PM Bug #45793: Objecter: don't attempt to read from non-primary on EC pools
- Marking only for octopus, since replica reads are safe for general use only in octopus.
- 12:32 PM Bug #45793 (Fix Under Review): Objecter: don't attempt to read from non-primary on EC pools
- 12:25 PM Bug #45793 (Resolved): Objecter: don't attempt to read from non-primary on EC pools
05/29/2020
- 05:31 PM Backport #45781 (Rejected): mimic: rados/test_envlibrados_for_rocksdb.sh build failure (seen in n...
- 05:31 PM Backport #45780 (Resolved): nautilus: rados/test_envlibrados_for_rocksdb.sh build failure (seen i...
- https://github.com/ceph/ceph/pull/35387
- 05:31 PM Backport #45779 (Resolved): octopus: rados/test_envlibrados_for_rocksdb.sh build failure (seen in...
- https://github.com/ceph/ceph/pull/35443
- 05:30 PM Backport #45776 (Resolved): nautilus: build_incremental_map_msg missing incremental map while sna...
- https://github.com/ceph/ceph/pull/35386
- 05:30 PM Backport #45775 (Resolved): octopus: build_incremental_map_msg missing incremental map while snap...
- https://github.com/ceph/ceph/pull/35442
- 05:16 AM Bug #45761 (Need More Info): mon_thrasher: "Error ENXIO: mon unavailable" during sync_force comma...
- /a/yuriw-2020-05-28_02:23:45-rados-wip-yuri-master_5.27.20-distro-basic-smithi/5097794...
- 04:11 AM Bug #45619 (Resolved): Health check failed: Reduced data availability: PG_AVAILABILITY
- 03:58 AM Bug #45760 (Resolved): osd-scrub-snaps.sh: TEST_scrub_snaps failed
05/28/2020
- 10:48 PM Bug #45760 (Fix Under Review): osd-scrub-snaps.sh: TEST_scrub_snaps failed
- 09:12 PM Bug #45760 (Resolved): osd-scrub-snaps.sh: TEST_scrub_snaps failed
- ...
- 09:39 PM Bug #45660 (Resolved): osd-scrub-repair.sh:TEST_corrupt_scrub_replicated failed
- 12:42 AM Bug #45660 (Fix Under Review): osd-scrub-repair.sh:TEST_corrupt_scrub_replicated failed
- 08:57 PM Bug #45619 (Fix Under Review): Health check failed: Reduced data availability: PG_AVAILABILITY
- 01:52 PM Bug #41399 (Resolved): Move bluefs alloc size initialization log message to log level 1
- 01:52 PM Backport #41533 (Resolved): mimic: Move bluefs alloc size initialization log message to log level 1
- 07:17 AM Bug #45606 (Pending Backport): build_incremental_map_msg missing incremental map while snaptrim o...
- 06:38 AM Bug #44595: cache tiering: Error: oid 48 copy_from 493 returned error code -2
- ...
- 06:08 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- @/a/kchai-2020-05-27_23:43:53-rados-wip-kefu-testing-2020-05-27-2242-distro-basic-smithi/5097299/remote/*/log/valgrin...
- 02:10 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- /a/yuriw-2020-05-24_19:30:40-rados-wip-yuri-master_5.24.20-distro-basic-smithi/5088037
/a/yuriw-2020-05-24_19:30:40-...
05/27/2020
- 10:34 PM Bug #45733 (Fix Under Review): osd-scrub-repair.sh: SyntaxError: invalid syntax
- 10:29 PM Bug #45733 (Resolved): osd-scrub-repair.sh: SyntaxError: invalid syntax
- /a/yuriw-2020-05-23_15:15:01-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5085557...
- 09:21 PM Bug #45660: osd-scrub-repair.sh:TEST_corrupt_scrub_replicated failed
- ...
- 06:59 AM Bug #45660: osd-scrub-repair.sh:TEST_corrupt_scrub_replicated failed
- /a/yuriw-2020-05-23_15:15:01-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5085557
- 09:05 PM Bug #44715: common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_flight_list.back())->ops_in_...
- Lowering severity since we haven't seen it in two weeks.
- 08:37 PM Bug #45619 (Triaged): Health check failed: Reduced data availability: PG_AVAILABILITY
- http://pulpito.front.sepia.ceph.com/yuvalif-2020-05-19_14:52:46-rgw:verify-fix-amqp-urls-with-vhosts-distro-basic-smi...
- 06:35 PM Bug #45619: Health check failed: Reduced data availability: PG_AVAILABILITY
- Neha Ojha wrote:
> Seen in the rados suite: /a/nojha-2020-05-21_19:33:40-rados-wip-32601-distro-basic-smithi/5077159... - 01:58 PM Bug #44981 (Pending Backport): rados/test_envlibrados_for_rocksdb.sh build failure (seen in nauti...
- 08:22 AM Bug #45721 (Resolved): CommandFailedError: Command failed (workunit test rados/test_python.sh) FA...
- /a/yuriw-2020-05-24_19:30:40-rados-wip-yuri-master_5.24.20-distro-basic-smithi/5088170...
- 07:02 AM Bug #43888: osd/osd-bench.sh 'tell osd.N bench' hang
- /a/yuriw-2020-05-23_15:15:01-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5085549
- 02:20 AM Bug #43888: osd/osd-bench.sh 'tell osd.N bench' hang
- /a/yuriw-2020-05-22_19:55:53-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5083462
- 06:55 AM Bug #45661: valgrind issue: UninitValue in ProtocolV2
- /a/yuriw-2020-05-23_15:15:01-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5085545
/a/yuriw-2020-05-23_15:15:01-...
05/26/2020
- 03:11 PM Bug #45695: librados: significant memory consumption
- David Disseldorp wrote:
> I've tested with in-memory logging disabled via the client ceph.conf:
>
> [...]
>
> ... - 11:33 AM Bug #45706 (New): Memory usage in buffer_anon showing unbounded growth in osds on EC pool. (14.2.9)
- Hi,
Re these threads in the mailing list: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/DPBVNJQX... - 07:12 AM Bug #45588 (Resolved): test_envlibrados_for_rocksdb.sh fails on master
- 04:44 AM Bug #45702 (Fix Under Review): PGLog::read_log_and_missing: ceph_assert(miter == missing.get_item...
- /a/yuriw-2020-05-22_19:55:53-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5083350...
05/25/2020
- 05:01 PM Backport #45677 (In Progress): nautilus: rados/test_envlibrados_for_rocksdb.sh fails on Xenial (s...
- 04:58 PM Backport #45676 (In Progress): octopus: rados/test_envlibrados_for_rocksdb.sh fails on Xenial (se...
- 02:28 PM Bug #43825 (Resolved): osd stuck down
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 02:28 PM Bug #44062 (Resolved): LibRadosWatchNotify.WatchNotify failure
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 02:27 PM Bug #44439 (Resolved): osd/osd-scrub-repair.sh fails: scrub/osd-scrub-repair.sh:698: TEST_repair_...
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 02:27 PM Bug #44518 (Resolved): osd/osd-backfill-stats.sh TEST_backfill_out2: wait_for_clean timeout
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 02:27 PM Bug #44532 (Resolved): nautilus: FAILED ceph_assert(head.version == 0 || e.version.version > head...
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 02:26 PM Bug #45266 (Resolved): follower monitors can grow beyond memory target
- While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ...
- 11:51 AM Bug #45698 (New): PrioritizedQueue: messages in normal queue
- if(i->second.front().first < i->second.num_tokens())
{
//nenver go in, if cost equal to num_tockens(),which valu... - 11:09 AM Backport #44686 (Resolved): nautilus: osd/osd-backfill-stats.sh TEST_backfill_out2: wait_for_clea...
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/35047
m... - 11:09 AM Backport #45224 (Resolved): nautilus: LibRadosWatchNotify.WatchNotify failure
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/35049
m... - 11:09 AM Backport #44689 (Resolved): nautilus: osd/osd-scrub-repair.sh fails: scrub/osd-scrub-repair.sh:69...
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/35048
m... - 11:08 AM Backport #43919 (Resolved): nautilus: osd stuck down
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/35024
m... - 11:08 AM Backport #44841 (Resolved): nautilus: nautilus: FAILED ceph_assert(head.version == 0 || e.version...
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/34957
m... - 11:06 AM Backport #44490 (Resolved): nautilus: lz4 compressor corrupts data when buffers are unaligned
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/35004
m... - 11:06 AM Backport #45391 (Resolved): nautilus: follower monitors can grow beyond memory target
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/34916
m... - 11:06 AM Backport #45359 (Resolved): nautilus: rados: Sharded OpWQ drops suicide_grace after waiting for work
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/34882
m... - 10:55 AM Bug #45695: librados: significant memory consumption
- I've tested with in-memory logging disabled via the client ceph.conf:...
- 10:27 AM Bug #45695: librados: significant memory consumption
- I should have mentioned that my client ceph.conf is minimal, with only the _mon host_ and _keyring_ options set.
- 10:22 AM Bug #45695 (New): librados: significant memory consumption
- I did some valgrind massif heap profiling with the following simple librados (octopus 15.2.1) program:...
- 02:28 AM Bug #45690 (New): pg_interval_t::check_new_interval is overly generous about guessing when EC PGs...
- One EC PG stuck at peering+down forever, the problem occurs through the following steps:
Suppose the pg's acting set...
05/24/2020
- 10:09 PM Bug #44981: rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- Nathan Cutler wrote:
>
> New -> In Progress -> Fix Under Review -> Pending Backport
>
> This, I thought, was th... - 08:59 PM Bug #44981: rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- Brad Hubbard wrote:
> Sorry Nathan, Could you explain why you changed this from 'In Progress' to 'Fix Under Review'?... - 09:04 PM Backport #45677 (Resolved): nautilus: rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen...
- https://github.com/ceph/ceph/pull/35237
- 09:04 PM Backport #45676 (Resolved): octopus: rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen ...
- https://github.com/ceph/ceph/pull/35236
- 09:03 PM Backport #45673 (Resolved): octopus: qa: powercycle: install task runs twice with double unwind c...
- https://github.com/ceph/ceph/pull/35441
- 07:55 PM Bug #45606 (Fix Under Review): build_incremental_map_msg missing incremental map while snaptrim o...
- 04:00 PM Bug #22052: ceph-mon: possible Leak in OSDMap::build_simple_optioned
- ...
05/23/2020
- 09:56 PM Bug #45561 (Pending Backport): rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen in nau...
- 03:11 PM Bug #24531: Mimic MONs have slow/long running ops
- We had this issue yesterday. We had a broken mon cluster which I was able to repair by shutting down all mons, scalin...
05/22/2020
- 06:54 PM Backport #44686: nautilus: osd/osd-backfill-stats.sh TEST_backfill_out2: wait_for_clean timeout
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35047
merged - 06:47 PM Backport #45224: nautilus: LibRadosWatchNotify.WatchNotify failure
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35049
merged - 06:46 PM Backport #44689: nautilus: osd/osd-scrub-repair.sh fails: scrub/osd-scrub-repair.sh:698: TEST_rep...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35048
merged - 06:40 PM Backport #43919: nautilus: osd stuck down
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35024
merged - 06:39 PM Backport #44841: nautilus: nautilus: FAILED ceph_assert(head.version == 0 || e.version.version > ...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/34957
merged - 04:53 PM Bug #45661 (Resolved): valgrind issue: UninitValue in ProtocolV2
- ...
- 04:32 PM Bug #20960: ceph_test_rados: mismatched version (due to pg import/export)
- Has started appearing more frequently recently - /a/nojha-2020-05-21_19:33:40-rados-wip-32601-distro-basic-smithi/507...
- 04:30 PM Bug #45660 (Resolved): osd-scrub-repair.sh:TEST_corrupt_scrub_replicated failed
- ...
- 04:24 PM Bug #45647: "ceph --cluster ceph --log-early osd last-stat-seq osd.0" times out due to msgr-failu...
- /a/nojha-2020-05-21_19:33:40-rados-wip-32601-distro-basic-smithi/5076944/
- 03:36 AM Bug #45647 (New): "ceph --cluster ceph --log-early osd last-stat-seq osd.0" times out due to msgr...
- ...
- 02:40 PM Bug #45619 (New): Health check failed: Reduced data availability: PG_AVAILABILITY
- Seen in the rados suite: /a/nojha-2020-05-21_19:33:40-rados-wip-32601-distro-basic-smithi/5077159/
- 02:35 PM Bug #45619: Health check failed: Reduced data availability: PG_AVAILABILITY
- We've been seeing a lot of this in the rgw suite over the last month or two.
- 02:37 PM Bug #45298 (Resolved): cram: balancer/misplaced.t fails with 'Error EAGAIN: Some objects (0.00891...
- This was a result of d4fbaf7ea959fd945857abd327271a97fb1da631, which only applies to master.
- 04:41 AM Bug #45298 (Pending Backport): cram: balancer/misplaced.t fails with 'Error EAGAIN: Some objects ...
- 04:40 AM Feature #43324 (Resolved): Make zlib windowBits configurable for compression
- 04:30 AM Bug #45612 (Pending Backport): qa: powercycle: install task runs twice with double unwind causing...
- 04:03 AM Bug #44595: cache tiering: Error: oid 48 copy_from 493 returned error code -2
- ...
- 03:59 AM Bug #24613: luminous: rest/test.py fails with expected 200, got 400
- /a/nojha-2020-05-21_19:42:29-rados-wip-29089-luminous-distro-basic-smithi/5077334
05/21/2020
- 09:32 PM Bug #44981: rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- Sorry Nathan, Could you explain why you changed this from 'In Progress' to 'Fix Under Review'? The PR has been review...
- 04:43 PM Bug #44981 (Fix Under Review): rados/test_envlibrados_for_rocksdb.sh build failure (seen in nauti...
- 05:34 PM Bug #45614 (Resolved): qa/workunits/cephtool/test.sh failures due to dropping obsolete cache tier...
- 04:41 PM Bug #45614: qa/workunits/cephtool/test.sh failures due to dropping obsolete cache tiering options
- Backport will be handled via #45514
- 02:52 AM Bug #45619: Health check failed: Reduced data availability: PG_AVAILABILITY
- it's a new thing. also, before whitelist things, better off figure out why we should whitelist it.
05/20/2020
- 09:13 PM Bug #45606: build_incremental_map_msg missing incremental map while snaptrim or backfilling
- Nothing to worry about, this message should just be a dout instead.
- 09:08 PM Bug #45619 (Need More Info): Health check failed: Reduced data availability: PG_AVAILABILITY
- Is this something that has started appearing recently? If not, probably just needs whitelisting.
- 07:34 AM Bug #45619 (Resolved): Health check failed: Reduced data availability: PG_AVAILABILITY
- multiple RGW tests are failing on different branches, with:...
- 07:29 PM Bug #20960: ceph_test_rados: mismatched version (due to pg import/export)
- /a/nojha-2020-05-19_23:54:26-rados-wip-cephadm-test-distro-basic-smithi/5070712
- 03:20 PM Backport #44490: nautilus: lz4 compressor corrupts data when buffers are unaligned
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/35004
merged - 03:19 PM Backport #45391: nautilus: follower monitors can grow beyond memory target
- Sridhar Seshasayee wrote:
> https://github.com/ceph/ceph/pull/34916
merged - 03:19 PM Backport #45359: nautilus: rados: Sharded OpWQ drops suicide_grace after waiting for work
- Dan Hill wrote:
> https://github.com/ceph/ceph/pull/34882
merged - 10:42 AM Bug #45611: crimson: centos 8 vstart failure
- caught segfault at points
1. run with next option in gdb: ... - 10:39 AM Bug #45611: crimson: centos 8 vstart failure
- How to reproduce:
1. launch a centos 8 container and build vstart with -DWITH_SEASTAR=ON
2. start a vstart base... - 02:17 AM Bug #44981: rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- Thanks Nathan.
- 02:16 AM Bug #44981 (In Progress): rados/test_envlibrados_for_rocksdb.sh build failure (seen in nautilus)
- 12:34 AM Bug #45615 (Pending Backport): api_watch_notify_pp: LibRadosWatchNotifyPPTests/LibRadosWatchNotif...
- ...
- 12:26 AM Bug #45614: qa/workunits/cephtool/test.sh failures due to dropping obsolete cache tiering options
- /a/nojha-2020-05-19_00:53:41-rados-wip-revert-34894-distro-basic-smithi/5068016
- 12:24 AM Bug #45614 (Resolved): qa/workunits/cephtool/test.sh failures due to dropping obsolete cache tier...
- Caused by https://github.com/ceph/ceph/pull/35015
05/19/2020
- 10:30 PM Bug #45612 (Fix Under Review): qa: powercycle: install task runs twice with double unwind causing...
- 10:24 PM Bug #45612 (Resolved): qa: powercycle: install task runs twice with double unwind causing fatal e...
- Continuation of #45387. My fix was incomplete.
http://pulpito.ceph.com/teuthology-2020-04-25_03:09:02-powercycle-m... - 07:23 PM Bug #45611: crimson: centos 8 vstart failure
- caught some memory leaks using core dumps, but they seem to be related to asan/libc...
- 02:37 PM Bug #45611 (New): crimson: centos 8 vstart failure
- ...
- 11:44 AM Bug #45606 (Resolved): build_incremental_map_msg missing incremental map while snaptrim or backfi...
- Hello,
I'm not sure if this is an issue or not. On one Cluster I see the following Messages, most times when snapt... - 09:34 AM Backport #44370: nautilus: msg/async: the event center is blocked by rdma construct conection for...
- This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/34780
m... - 02:33 AM Backport #44370 (Resolved): nautilus: msg/async: the event center is blocked by rdma construct co...
- 02:54 AM Bug #45588: test_envlibrados_for_rocksdb.sh fails on master
- http://pulpito.ceph.com/kchai-2020-05-19_02:54:14-rados:singleton-wip-kefu2-testing-2020-05-13-1200-distro-basic-smithi/
05/18/2020
- 03:42 PM Bug #45588: test_envlibrados_for_rocksdb.sh fails on master
- https://github.com/facebook/rocksdb/pull/6855
- 03:41 PM Bug #45588 (Resolved): test_envlibrados_for_rocksdb.sh fails on master
- ...
- 02:44 PM Backport #44370: nautilus: msg/async: the event center is blocked by rdma construct conection for...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/34780
merged
05/15/2020
- 03:38 PM Backport #44413: nautilus: FTBFS on s390x in openSUSE Build Service due to presence of -O2 in RPM...
- c8af73e19ab02617411fe689ff1b98b8f4d096ca did not make v14.2.9, and it will be in v14.2.10.
- 11:24 AM Bug #45561 (In Progress): rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen in nautilus)
- 06:59 AM Bug #45561 (Fix Under Review): rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen in nau...
- 06:40 AM Bug #45561 (Resolved): rados/test_envlibrados_for_rocksdb.sh fails on Xenial (seen in nautilus)
- http://qa-proxy.ceph.com/teuthology/bhubbard-2020-05-13_06:50:26-rados-wip-nautilus-badone-testing-2-distro-basic-smi...
05/14/2020
- 12:13 AM Bug #44715 (Need More Info): common/TrackedOp.cc: 163: FAILED ceph_assert((sharded_in_flight_list...
- I am not able to reproduce this failure on octopus or on master:
http://pulpito.ceph.com/nojha-2020-05-13_17:20:4...
05/13/2020
- 09:09 PM Bug #45533 (Resolved): cls/queue: fix empty markers when listing entries
- 12:53 PM Bug #45533: cls/queue: fix empty markers when listing entries
- already fixed in: https://github.com/ceph/ceph/pull/34788
- 12:51 PM Bug #45533 (Resolved): cls/queue: fix empty markers when listing entries
- markers are sometimes empty when listing entries
- 09:02 PM Backport #44489 (In Progress): mimic: lz4 compressor corrupts data when buffers are unaligned
- 03:43 PM Backport #45224 (In Progress): nautilus: LibRadosWatchNotify.WatchNotify failure
- 03:42 PM Backport #44689 (In Progress): nautilus: osd/osd-scrub-repair.sh fails: scrub/osd-scrub-repair.sh...
- 03:40 PM Backport #44686 (In Progress): nautilus: osd/osd-backfill-stats.sh TEST_backfill_out2: wait_for_c...
- 02:49 AM Bug #44981 (Fix Under Review): rados/test_envlibrados_for_rocksdb.sh build failure (seen in nauti...
05/12/2020
- 06:07 PM Bug #45292: pg autoscaler merging issue
- Sorry for the delay. We are working to get a reservation on one of our internal labs so we can recreate the issue and...
- 03:02 PM Bug #37875: osdmaps aren't being cleaned up automatically on healthy cluster
- nautilus backport tracked by https://tracker.ceph.com/issues/45402
- 03:01 PM Bug #37875 (Duplicate): osdmaps aren't being cleaned up automatically on healthy cluster
- 02:30 PM Backport #43919 (In Progress): nautilus: osd stuck down
- 02:29 PM Backport #43919: nautilus: osd stuck down
- first attempted backport - https://github.com/ceph/ceph/pull/33156 - was closed
- 02:29 PM Backport #43919 (New): nautilus: osd stuck down
Also available in: Atom