Activity

09:15 PM Bug #51083: Raw space filling up faster than used space: I don't have any ideas from the logs. Moving this back to RADOS. I doubt it has anything to do with CephFS. Patrick Donnelly
11:56 AM Bug #51083: Raw space filling up faster than used space: Patrick Donnelly wrote:
> Scrub is unlikely to help.
I came to the same conclusion after reading the documentatio... Jan-Philipp Litza

06/17/2021

11:07 PM Bug #51083: Raw space filling up faster than used space: Jan-Philipp Litza wrote:
> Yesterday evening we finally managed to upgrade the MDS daemons as well, and that seems t... Patrick Donnelly
09:13 PM Bug #51083: Raw space filling up faster than used space: Patrick: do you understand how upgrading the MDS daemons helped in this case? There is nothing in the osd/bluestore s... Neha Ojha
09:03 PM Bug #51254: deep-scrub stat mismatch on last PG in pool: We definitely do not use cache tiering on any of our clusters. On the cluster above, we do use snapshots (via cephfs... Andras Pataki
08:48 PM Bug #51254: deep-scrub stat mismatch on last PG in pool: It seems like you are using cache tiering, and there has been similar bugs reported like this. I don't understand why... Neha Ojha
09:01 PM Bug #51234 (Fix Under Review): LibRadosService.StatusFormat failed, Expected: (0) != (retry), act...: Sage Weil
08:57 PM Bug #50842 (Need More Info): pacific: recovery does not complete because of rw_manager lock not ...: Neha Ojha
08:53 PM Backport #51269 (In Progress): octopus: rados/perf: cosbench workloads hang forever: Deepika Upadhyay
07:14 PM Backport #51269 (Resolved): octopus: rados/perf: cosbench workloads hang forever: https://github.com/ceph/ceph/pull/41922 Deepika Upadhyay
08:42 PM Bug #51074 (Pending Backport): standalone/osd-rep-recov-eio.sh: TEST_rep_read_unfound failed with...: marking Pending Backport, needs to be included with https://github.com/ceph/ceph/pull/41731 Neha Ojha
08:40 PM Bug #51168 (Need More Info): ceph-osd state machine crash during peering process: Can you please attach the osd log for this crash? Neha Ojha
08:07 PM Bug #51270 (Fix Under Review): mon: stretch mode clusters do not sanely set default crush rules: Greg Farnum
08:03 PM Bug #51270 (Pending Backport): mon: stretch mode clusters do not sanely set default crush rules: If you do not specify a crush rule when creating a pool, the OSDMonitor picks the default one for you out of the conf... Greg Farnum
07:12 PM Bug #49139 (Pending Backport): rados/perf: cosbench workloads hang forever: Deepika Upadhyay
02:32 PM Backport #51237: nautilus: rebuild-mondb hangs: Backport Bot wrote:
> https://github.com/ceph/ceph/pull/41874
merged Yuri Weinstein

06/16/2021

10:40 PM Bug #51254 (New): deep-scrub stat mismatch on last PG in pool: In the past few weeks, we got inconsistent PGs in deep-scrub a few times, always on the very last PG in the pool:
... Andras Pataki
07:25 PM Bug #38357: ClsLock.TestExclusiveEphemeralStealEphemeral failed: ... Deepika Upadhyay
07:22 PM Bug #45423: api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWrite: /ceph/teuthology-archive/yuriw-2021-06-14_19:20:57-rados-wip-yuri6-testing-2021-06-14-1106-octopus-distro-basic-smith... Deepika Upadhyay
06:48 PM Bug #50042: rados/test.sh: api_watch_notify failures: ... Deepika Upadhyay
02:49 PM Bug #51246 (New): error in open_pools_parallel: rados_write(0.obj) failed with error: -2: ... Deepika Upadhyay
01:22 PM Bug #50587: mon election storm following osd recreation: huge tcmalloc and ceph::msgr::v2::FrameA...: > Will this patch be released in 14.2.22?
yes the PR has been merged to the nautilus branch, so it will be in the ... Dan van der Ster
12:59 PM Bug #50587: mon election storm following osd recreation: huge tcmalloc and ceph::msgr::v2::FrameA...: We hit this bug yesterday in a nautilus 14.2.18 cluster.
All monitors went down and started crashing on restart.
... Rob Haverkamp
02:18 AM Backport #51237 (In Progress): nautilus: rebuild-mondb hangs: Kefu Chai
02:16 AM Backport #51237 (Resolved): nautilus: rebuild-mondb hangs: https://github.com/ceph/ceph/pull/41874 Backport Bot
02:13 AM Bug #38219 (Pending Backport): rebuild-mondb hangs: Kefu Chai

06/15/2021

08:05 PM Bug #50462: OSDs crash in osd/osd_types.cc: FAILED ceph_assert(clone_overlap.count(clone)): Just to note:
IMO ceph-bluestore-tool crash is caused by a bag in AvlAllocator and is a duplicate of https://tracker... Igor Fedotov
06:59 PM Backport #51215 (In Progress): pacific: Global Recovery Event never completes: Kamoltat (Junior) Sirivadhna
12:55 AM Backport #51215 (Resolved): pacific: Global Recovery Event never completes: Backport PR https://github.com/ceph/ceph/pull/41872 Backport Bot
06:50 PM Backport #50706: pacific: _delete_some additional unexpected onode list: Backport Bot wrote:
> https://github.com/ceph/ceph/pull/41680
merged Yuri Weinstein
06:47 PM Bug #50842: pacific: recovery does not complete because of rw_manager lock not being released: @Neha: I did, but am afraid they are lost, the test was from https://pulpito.ceph.com/ideepika-2021-05-17_10:16:28-ra... Deepika Upadhyay
06:31 PM Bug #50842: pacific: recovery does not complete because of rw_manager lock not being released: @Deepika, do you happen to have the logs saved somewhere? Neha Ojha
06:41 PM Bug #51234 (Pending Backport): LibRadosService.StatusFormat failed, Expected: (0) != (retry), act...: ... Neha Ojha
06:30 PM Bug #51076: "wait_for_recovery: failed before timeout expired" during thrashosd test with EC back...: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-... Neha Ojha
06:26 PM Bug #51076: "wait_for_recovery: failed before timeout expired" during thrashosd test with EC back...: Looks very similar... Neha Ojha
04:12 PM Backport #50750: octopus: max_misplaced was replaced by target_max_misplaced_ratio: Backport Bot wrote:
> https://github.com/ceph/ceph/pull/41624
merged Yuri Weinstein
12:20 PM Bug #51223 (New): statfs: a cluster with filestore and bluestore OSD's will report bytes_used == ...: Cluster migrated from Luminous mixed bluestore+filestore OSD's to Nautilus 14.2.21
After last filestore OSD purged f... Konstantin Shalygin
10:47 AM Bug #49677 (Resolved): debian ceph-common package post-inst clobbers ownership of cephadm log dirs: While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ... Loïc Dachary
10:47 AM Bug #49781 (Resolved): unittest_mempool.check_shard_select failed: While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ... Loïc Dachary
10:44 AM Bug #50501 (Resolved): osd/scheduler/mClockScheduler: Async reservers are not updated with the ov...: While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ... Loïc Dachary
10:44 AM Bug #50558 (Resolved): Data loss propagation after backfill: While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ... Loïc Dachary
10:42 AM Backport #50795 (Resolved): nautilus: mon: spawn loop after mon reinstalled: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41762
m... Loïc Dachary
10:41 AM Backport #50704 (Resolved): nautilus: _delete_some additional unexpected onode list: Loïc Dachary
10:36 AM Backport #50153 (Resolved): nautilus: Reproduce https://tracker.ceph.com/issues/48417: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41611
m... Loïc Dachary
10:36 AM Backport #49729 (Resolved): nautilus: debian ceph-common package post-inst clobbers ownership of ...: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/40698
m... Loïc Dachary
10:32 AM Backport #50988: nautilus: mon: slow ops due to osd_failure: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41519
m... Loïc Dachary
09:05 AM Backport #50406: pacific: mon: new monitors may direct MMonJoin to a peon instead of the leader: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41131
m... Loïc Dachary
09:04 AM Backport #50344: pacific: mon: stretch state is inconsistently-maintained on peons, preventing pr...: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41130
m... Loïc Dachary
09:04 AM Backport #50794 (Resolved): pacific: osd: FAILED ceph_assert(recovering.count(*i)) after non-prim...: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41320
m... Loïc Dachary
09:03 AM Backport #50702 (Resolved): pacific: Data loss propagation after backfill: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41236
m... Loïc Dachary
09:03 AM Backport #50606 (Resolved): pacific: osd/scheduler/mClockScheduler: Async reservers are not updat...: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/41125
m... Loïc Dachary
09:02 AM Backport #49992 (Resolved): pacific: unittest_mempool.check_shard_select failed: This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/40566
m... Loïc Dachary
12:54 AM Bug #49988 (Pending Backport): Global Recovery Event never completes: Neha Ojha

06/14/2021

03:24 PM Feature #51213 (Resolved): [ceph osd set noautoscale] Global on/off flag for PG autoscale feature: For now, we do not have a global flag, like `ceph osd set noout` for the pg autoscale feature. We have pool flags[1] ... Vikhyat Umrao
09:24 AM Bug #50346: OSD crash FAILED ceph_assert(!is_scrubbing()): A related crash, happened when I disabled scrubbing:
-1> 2021-06-14T11:17:15.373+0200 7fb9916f5700 -1 /home/... Andrej Filipcic

06/13/2021

03:34 PM Bug #51194: PG recovery_unfound after scrub repair failed on primary: To prevent the user IO from being blocked, we took this action:
1. First, we queried the unfound objects. osd.951 ... Dan van der Ster
01:36 PM Bug #51194 (New): PG recovery_unfound after scrub repair failed on primary: This comes from a mail I send to the ceph-users ML: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/3... Dan van der Ster
03:30 PM Backport #51195 (In Progress): pacific: [rfe] increase osd_max_write_op_reply_len default value t...: https://github.com/ceph/ceph/pull/53470 Backport Bot
03:28 PM Bug #51166 (Pending Backport): [rfe] increase osd_max_write_op_reply_len default value to 64 bytes: Kefu Chai

06/12/2021

12:38 AM Bug #49988 (Resolved): Global Recovery Event never completes: Kefu Chai

06/10/2021

08:35 PM Backport #51173 (Rejected): nautilus: regression in ceph daemonperf command output, osd columns a...: Backport Bot
08:35 PM Backport #51172 (Resolved): pacific: regression in ceph daemonperf command output, osd columns ar...: https://github.com/ceph/ceph/pull/44175 Backport Bot
08:35 PM Backport #51171 (Resolved): octopus: regression in ceph daemonperf command output, osd columns ar...: https://github.com/ceph/ceph/pull/44176 Backport Bot
08:32 PM Bug #51002 (Pending Backport): regression in ceph daemonperf command output, osd columns aren't v...: Igor Fedotov
06:48 PM Backport #50795: nautilus: mon: spawn loop after mon reinstalled: Dan van der Ster wrote:
> https://github.com/ceph/ceph/pull/41762
merged Yuri Weinstein
03:58 PM Bug #51168 (New): ceph-osd state machine crash during peering process: ... Yao Ning
02:22 PM Bug #51166 (Fix Under Review): [rfe] increase osd_max_write_op_reply_len default value to 64 bytes: Matt Benjamin
02:16 PM Bug #51166 (Pending Backport): [rfe] increase osd_max_write_op_reply_len default value to 64 bytes: As agreed in #ceph-devel, Sage, Josh, Neha concurring.
Matt Benjamin
01:53 PM Bug #51076: "wait_for_recovery: failed before timeout expired" during thrashosd test with EC back...: For the dead jobs, relevant logs have been uploaded to senta02 under /home/sseshasa/recovery_timeout.
Please let me ... Sridhar Seshasayee
10:25 AM Bug #50346: OSD crash FAILED ceph_assert(!is_scrubbing()): On a 60-node, 1500 HDD cluster, and 16.2.4 release, this issue become very frequent, especially when RBD writes excee... Andrej Filipcic

06/09/2021

01:32 PM Bug #51076: "wait_for_recovery: failed before timeout expired" during thrashosd test with EC back...: From the logs of 6161181 snapshot recovery is not able to proceed since a rwlock on the head version
(3:cb63772d:::... Sridhar Seshasayee
08:07 AM Bug #51076: "wait_for_recovery: failed before timeout expired" during thrashosd test with EC back...: Ran the same test repeatedly (5 times) on master by setting osd_op_queue to 'wpq' and 'mclock_scheduler' on different... Sridhar Seshasayee
10:39 AM Bug #51074: standalone/osd-rep-recov-eio.sh: TEST_rep_read_unfound failed with "Bad data after pr...: Kefu, yes I did read the update and your effort to find the commit(s) that caused the regression in the standalone te... Sridhar Seshasayee
10:25 AM Bug #51074: standalone/osd-rep-recov-eio.sh: TEST_rep_read_unfound failed with "Bad data after pr...: Sridhar, please read the https://tracker.ceph.com/issues/51074#note-3. that's my finding in the last 3 days. Kefu Chai
09:44 AM Bug #51074: standalone/osd-rep-recov-eio.sh: TEST_rep_read_unfound failed with "Bad data after pr...: Raised PR https://github.com/ceph/ceph/pull/41782 to address the test failure.
Please see latest update to https:/... Sridhar Seshasayee
09:05 AM Backport #51151 (Rejected): nautilus: When read failed, ret can not take as data len, in FillInVe...: Backport Bot
09:05 AM Backport #51150 (Resolved): pacific: When read failed, ret can not take as data len, in FillInVer...: https://github.com/ceph/ceph/pull/44173 Backport Bot
09:05 AM Backport #51149 (Resolved): octopus: When read failed, ret can not take as data len, in FillInVer...: https://github.com/ceph/ceph/pull/44174 Backport Bot
09:02 AM Bug #51115 (Pending Backport): When read failed, ret can not take as data len, in FillInVerifyExtent: Kefu Chai

06/08/2021

11:10 PM Bug #38219: rebuild-mondb hangs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-06-08_20:53:36-rados-wip-yuri-testing-2021-06-04-0753-nautilus-distro-... Deepika Upadhyay
06:39 PM Bug #38219: rebuild-mondb hangs: 2021-06-04T23:05:38.775 INFO:tasks.ceph.mon.a.smithi071.stderr:/build/ceph-14.2.21-305-gac8fcfa6/src/mon/OSDMonitor.c... Deepika Upadhyay
07:52 PM Backport #50797 (In Progress): pacific: mon: spawn loop after mon reinstalled: Neha Ojha
05:58 PM Backport #50795: nautilus: mon: spawn loop after mon reinstalled: https://github.com/ceph/ceph/pull/41762 Dan van der Ster
04:50 PM Bug #50681: memstore: apparent memory leak when removing objects: Sven Anderson wrote:
> Greg Farnum wrote:
> > How long did you wait to see if memory usage dropped? Did you look at... Greg Farnum
08:48 AM Bug #51074 (Triaged): standalone/osd-rep-recov-eio.sh: TEST_rep_read_unfound failed with "Bad dat...: Kefu Chai

06/07/2021

11:37 AM Backport #51117 (In Progress): pacific: osd: Run osd bench test to override default max osd capac...: Sridhar Seshasayee
10:25 AM Backport #51117 (Resolved): pacific: osd: Run osd bench test to override default max osd capacity...: https://github.com/ceph/ceph/pull/41731 Backport Bot
10:22 AM Fix #51116 (Resolved): osd: Run osd bench test to override default max osd capacity for mclock.: Sridhar Seshasayee
08:28 AM Bug #51115: When read failed, ret can not take as data len, in FillInVerifyExtent: https://github.com/ceph/ceph/pull/41727 yanqiang sun
08:12 AM Bug #51115 (Fix Under Review): When read failed, ret can not take as data len, in FillInVerifyExtent: Kefu Chai
07:42 AM Bug #51115 (Resolved): When read failed, ret can not take as data len, in FillInVerifyExtent: when read failed, such as return -EIO, FillInVerifyExtent take ret as data length. yanqiang sun
06:59 AM Bug #51083: Raw space filling up faster than used space: Yesterday evening we finally managed to upgrade the MDS daemons as well, and that seems to have stopped the space was... Jan-Philipp Litza

06/06/2021

11:22 AM Feature #51110 (New): invalidate crc in buffer::ptr::c_str(): h3. what:
*buffer::ptr* (or more precisely, *buffer::raw*) has the ability to cache CRC codes that are calculated ... Wenjun Huang

« Previous

Also available in: Atom

Project

General

Profile

Ceph » RADOS

Activity

Activity

07/05/2021

07/03/2021

07/02/2021

07/01/2021

06/30/2021

06/29/2021

06/28/2021

06/26/2021

06/25/2021

06/24/2021

06/23/2021

06/22/2021

06/21/2021

06/20/2021

06/19/2021

06/18/2021

06/17/2021

06/16/2021

06/15/2021

06/14/2021

06/13/2021

06/12/2021

06/10/2021

06/09/2021

06/08/2021

06/07/2021

06/06/2021