Activity

From 04/04/2018 to 05/03/2018

08:07 PM Bug #21977: null map from OSDService::get_map in advance_pg: From the latest logs, the peering thread id does not appear at all in the log until the crash.
I'm wondering if we... Josh Durgin
08:05 PM Bug #21977: null map from OSDService::get_map in advance_pg: Seen again here:
http://pulpito.ceph.com/yuriw-2018-04-20_20:02:29-upgrade:jewel-x-luminous-distro-basic-ovh/2420862/ Josh Durgin

04:06 PM Bug #23793: ceph-osd consumed 10+GB rss memory: Set osd_debug_op_order to false can fix this problem.
My ceph cluster is created through vstart.sh which set osd_deb... Honggang Yang
03:57 PM Bug #23816: disable bluestore cache caused a rocksdb error: https://github.com/ceph/ceph/pull/21583 Honggang Yang
03:53 PM Bug #23816 (Resolved): disable bluestore cache caused a rocksdb error: I disabled bluestore/rocksdb cache to estimate ceph-osd's memory consumption
by set bluestore_cache_size_ssd/bluesto... Honggang Yang
06:55 AM Bug #23145: OSD crashes during recovery of EC pg: `2018-03-09 08:29:09.170227 7f901e6b30 10 merge_log log((17348'18587,17348'18587], crt=17348'18585) from osd.6(2) int... Zengran Zhang

07:00 PM Backport #23772 (Resolved): luminous: ceph status shows wrong number of objects: https://github.com/ceph/ceph/pull/22680 Nathan Cutler
06:36 PM Bug #23769 (Resolved): osd/EC: slow/hung ops in multimds suite test: ... Patrick Donnelly
03:40 PM Feature #23364: Special scrub handling of hinfo_key errors: This pull request is another follow on:
https://github.com/ceph/ceph/pull/21450 David Zafman
11:41 AM Bug #20924: osd: leaked Session on osd.7: /a/sage-2018-04-17_04:17:03-rados-wip-sage3-testing-2018-04-16-2028-distro-basic-smithi/2404155
this time on osd.4... Sage Weil
07:37 AM Bug #23767: "ceph ping mon" doesn't work: so "ceph ping mon.<id>" will remind you mon.<id> doesn't existed. however, if you run "ceph ping mon.a", you can get ... cory gu
07:33 AM Bug #23767 (New): "ceph ping mon" doesn't work: if there is only mon_host= ip1, ip2...in the ceph.conf, then "ceph ping mon.<id>" doesn't work.
Root cause is in the... cory gu
06:14 AM Bug #23273: segmentation fault in PrimaryLogPG::recover_got(): Sorry for late reply, but it's hard to reproduce. we reproduce it once with... Yan Jun
02:09 AM Documentation #23765 (New): librbd hangs if permissions are incorrect: I've been building rust bindings for librbd against ceph jewel and luminous. I found out by accident that if a cephx... Chris Holcombe
12:14 AM Bug #23763 (Resolved): upgrade: bad pg num and stale health status in mixed lumnious/mimic cluster: This happened in a luminous-x/point-to-point run. Logs in teuthology:/home/yuriw/logs/2387999/
Versions at this po... Josh Durgin

06:11 AM Support #23719: Three nodes shutdown,only boot two of the nodes,many pgs down.(host failure-domai...: fix description: If pg1.1 have acting set [1,2,3], I power down osd.3 first,then power down osd.2. In case I boot osd... junwei liao
05:50 AM Support #23719 (New): Three nodes shutdown,only boot two of the nodes,many pgs down.(host failure...: The interval mechanism of PG will cause a problem in the process of cluster restart.If I have 3 nodes(host failure-do... junwei liao

10:40 PM Bug #23716: osd/ECUtil.cc: 25: FAILED assert(i->second.length() == total_data_size) (on upgrade f...: ... Sage Weil
10:21 PM Bug #23716 (Resolved): osd/ECUtil.cc: 25: FAILED assert(i->second.length() == total_data_size) (o...: ... Sage Weil
07:33 PM Bug #21931: osd: src/osd/ECBackend.cc: 2164: FAILED assert((offset + length) <= (range.first.get_...: Live multimds run: /ceph/teuthology-archive/pdonnell-2018-04-13_04:19:33-multimds-wip-pdonnell-testing-20180413.02283... Patrick Donnelly
07:30 PM Bug #21992: osd: src/common/interval_map.h: 161: FAILED assert(len > 0): /ceph/teuthology-archive/pdonnell-2018-04-13_04:19:33-multimds-wip-pdonnell-testing-20180413.022831-testing-basic-smi... Patrick Donnelly
06:28 PM Bug #23713: High MON cpu usage when cluster is changing: My guess is that this is the compat reencoding of the OSDMap for the pre-luminous clients.
Are you by chance makin... Sage Weil
06:10 PM Bug #23713 (Resolved): High MON cpu usage when cluster is changing: After upgrading to Luminous 12.2.4 (from Jewel 10.2.5), we consistently see high cpu usage when OSDMap changes , esp... Xiaoxi Chen
03:03 PM Bug #23228 (Closed): scrub mismatch on objects: The failure in comment (2) looks unrelated, but i twas a test branch. let's see if it happens again.
The original ... Sage Weil
01:51 PM Bug #23576: osd: active+clean+inconsistent pg will not scrub or repair: Is there any testing, logs, etc that will be helpful for tracking down the cause of this problem. I had a fairly bad... David Turner
08:20 AM Bug #23701: oi(object_info_t) size mismatch with real object size during old write arrives after ...: here is my pull request to fix this problem
https://github.com/ceph/ceph/pull/21408 Peng Xie
08:08 AM Bug #23701 (Fix Under Review): oi(object_info_t) size mismatch with real object size during old w...: currently, in our test environment (jewel : cephfs + cache tier + ec pool), we found several osd coredump
in the fol... Peng Xie
01:52 AM Backport #23654 (In Progress): luminous: Special scrub handling of hinfo_key errors: https://github.com/ceph/ceph/pull/21397 Kefu Chai

11:22 PM Bug #23662 (Fix Under Review): osd: regression causes SLOW_OPS warnings in multimds suite: https://github.com/ceph/teuthology/pull/1166 Patrick Donnelly
09:38 PM Bug #23662: osd: regression causes SLOW_OPS warnings in multimds suite: Looks like the obvious cause: https://github.com/ceph/ceph/pull/20660 Patrick Donnelly
07:56 PM Bug #23662 (Resolved): osd: regression causes SLOW_OPS warnings in multimds suite: See: [1], first instance of the problem at [0].
The last run which did not cause most multimds jobs to fail with S... Patrick Donnelly
11:20 PM Bug #23576: osd: active+clean+inconsistent pg will not scrub or repair: Any scrub that completes without errors will set num_scrub_errors in pg stats to 0. That will cause the inconsiste... David Zafman
10:51 PM Bug #23576: osd: active+clean+inconsistent pg will not scrub or repair: David, is there any way a missing object wouldn't be reported in list-inconsistent output? Josh Durgin
11:01 PM Bug #23492: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.sh: Let's see if this happens again now that sage's fast peering branch is merged. Josh Durgin
10:58 PM Backport #23485 (Resolved): luminous: scrub errors not cleared on replicas can cause inconsistent...: David Zafman
10:58 PM Bug #23585: osd: safe_timer segfault: Possibly the same as http://tracker.ceph.com/issues/23431 Josh Durgin
02:10 PM Bug #23585: osd: safe_timer segfault: Got segfault in safe_timer too. Got it just once so can not provide more info at the moment.
2018-04-03 05:53:07... Sergey Malinin
10:57 PM Bug #23564: OSD Segfaults: Is this on bluestore? there are a few reports of this occurring on bluestore including your other bug http://tracker.... Josh Durgin
10:44 PM Bug #23590: kstore: statfs: (95) Operation not supported: Josh Durgin
10:42 PM Bug #23601 (Resolved): Cannot see the value of parameter osd_op_threads via ceph --show-config: Josh Durgin
10:37 PM Bug #23614: local_reserver double-reservation of backfilled pg: This may be the same root cause as http://tracker.ceph.com/issues/23490 Josh Durgin
10:36 PM Bug #23565: Inactive PGs don't seem to cause HEALTH_ERR: Brad, can you take a look at this? I think it can be handled by the stuck pg code, that iirc already warns about pgs ... Josh Durgin
10:25 PM Bug #23664 (Resolved): cache-try-flush hits wrlock, busy loops: ... Sage Weil
10:12 PM Bug #23403 (Closed): Mon cannot join quorum: Thanks for letting us know. Brad Hubbard
01:15 PM Bug #23403: Mon cannot join quorum: After more investigation we discovered that one of the bonds on the machine was not behaving properly. We removed the... Gauvain Pocentek
11:28 AM Bug #23403: Mon cannot join quorum: Thanks for the investigation Brad.
The "fault, initiating reconnect" and "RESETSESSION" messages only appear when ... Gauvain Pocentek
07:57 PM Bug #23595: osd: recovery/backfill is extremely slow: @Greg Farnum: Ah, great that part is already handled!
What about my other questions though, like
> I think it i... Niklas Hambuechen
06:45 PM Bug #23595 (Duplicate): osd: recovery/backfill is extremely slow: https://tracker.ceph.com/issues/23141
Sorry you ran into this, it's a bug in BlueStore/BlueFS. The fix will be in ... Greg Farnum
07:49 PM Backport #23315: luminous: pool create cmd's expected_num_objects is not correctly interpreted: Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/20907
merged Yuri Weinstein
05:45 PM Feature #23660 (New): when scrub errors are due to disk read errors, ceph status can say "likely ...: If some of the scrub errors are due to disk read errors, we can also say in the status output "likely disk errors" an... Vasu Kulkarni
03:49 PM Bug #23487 (Pending Backport): There is no 'ceph osd pool get erasure allow_ec_overwrites' command: Sage Weil
03:39 PM Backport #23654 (Resolved): luminous: Special scrub handling of hinfo_key errors: https://github.com/ceph/ceph/pull/21397 David Zafman
03:09 PM Bug #23621 (Resolved): qa/standalone/mon/misc.sh fails on master: Kefu Chai
01:40 PM Backport #23316: jewel: pool create cmd's expected_num_objects is not correctly interpreted: -https://github.com/ceph/ceph/pull/21042-
but test/mon/osd-pool-create.sh failing, looking into it. Prashant D
05:00 AM Bug #23610 (Pending Backport): pg stuck in activating because of dropped pg-temp message: Kefu Chai
04:56 AM Bug #23648 (New): max-pg-per-osd.from-primary fails because of activating pg: the reason why we have activating pg when the number of pg is under the hard limit of max-pg-per-osd is that:
1. o... Kefu Chai
03:01 AM Bug #23647 (In Progress): thrash-eio test can prevent recovery: We are injecting random EIOs. However, in a recovery situation an EIO leads us to decide the object is missing in on... Sage Weil

10:24 PM Feature #23616 (New): osd: admin socket should help debug status at all times: Last week I was looking at an LRC OSD which was having trouble, and it wasn't clear why.
The cause ended up being ... Greg Farnum
10:18 PM Bug #22882 (Resolved): Objecter deadlocked on op budget while holding rwlock in ms_handle_reset(): Whoops, this merged way back then with a slightly different plan than discussed here (see PR discussion). Greg Farnum
09:59 PM Bug #22525: auth: ceph auth add does not sanity-check caps: https://github.com/ceph/ceph/pull/21311 Sage Weil
09:21 PM Feature #23045 (Resolved): mon: warn on slow ops in OpTracker: That PR got merged a while ago and we've been working through the slow ops warnings that turn up since. Seems to be a... Greg Farnum
08:59 PM Feature #21084 (Resolved): auth: add osd auth caps based on pool metadata: Patrick Donnelly
06:53 PM Bug #23614: local_reserver double-reservation of backfilled pg: Looking through the code I don't see where the reservation is supposed to be released. I see releases for
- the p... Sage Weil
06:52 PM Bug #23614 (Resolved): local_reserver double-reservation of backfilled pg: - pg gets reservations (incl local_reserver)
- pg backfills, finishes
- ...apparently enver releases the reservatio... Sage Weil
06:15 PM Bug #23365: CEPH device class not honored for erasure encoding.: A quote from Greg Farnum on the crash from another ticket:... Brian Woods
06:13 PM Bug #23365: CEPH device class not honored for erasure encoding.: I put 12.2.2, but that is incorrect. It is version ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) lu... Brian Woods
05:38 PM Bug #23365: CEPH device class not honored for erasure encoding.: What version are you running? How are your OSDs configured?
There was a bug with BlueStore SSDs being misreported ... Greg Farnum
05:36 PM Bug #23371: OSDs flaps when cluster network is made down: You tested this on a version prior to luminous and the behavior has *changed*?
This must be a result of some chang... Greg Farnum
05:24 PM Documentation #23613 (Closed): doc: add description of new fs-client auth profile: Patrick Donnelly
05:23 PM Documentation #23613 (Closed): doc: add description of new fs-client auth profile: On that page: http://docs.ceph.com/docs/master/rados/operations/user-management/#authorization-capabilities
https:... Patrick Donnelly
05:23 PM Documentation #23612 (New): doc: add description of new auth profiles: On that page: http://docs.ceph.com/docs/master/rados/operations/user-management/#authorization-capabilities
https:... Patrick Donnelly
05:18 PM Support #23455 (Resolved): osd: large number of inconsistent objects after recover or backfilling: fiemap is disabled by default precisely because there are a number of known bugs in the local filesystems across kern... Greg Farnum
05:07 PM Bug #23610 (Fix Under Review): pg stuck in activating because of dropped pg-temp message: https://github.com/ceph/ceph/pull/21310 Kefu Chai
05:02 PM Bug #23610 (Resolved): pg stuck in activating because of dropped pg-temp message: http://pulpito.ceph.com/yuriw-2018-04-05_22:33:03-rados-wip-yuri3-testing-2018-04-05-1940-luminous-distro-basic-smith... Kefu Chai
05:06 PM Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade ...: This is a dupe of...something. We can track it down later.
For now, note that the crash is happening with Hammer c... Greg Farnum
06:17 AM Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade ...: Hm hm hm Nathan Cutler
02:56 AM Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade ...: h3. rados bisect
Reproducer: ... Nathan Cutler
02:11 AM Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade ...: This problem was not happening so reproducibly before the current integration run, so one of the following PRs might ... Nathan Cutler
02:05 AM Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade ...: Set priority to Urgent because this prevents us from getting a clean rados run in jewel 10.2.11 integration testing. Nathan Cutler
02:04 AM Bug #23598 (Duplicate): hammer->jewel: ceph_test_rados crashes during radosbench task in jewel ra...: Test description: rados/upgrade/{hammer-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-hammer-install/hammer.ya... Nathan Cutler
04:39 PM Bug #23595: osd: recovery/backfill is extremely slow: *I have it figured out!*
The issue was "osd_recovery_sleep_hdd", which defaults to 0.1 seconds.
After setting
... Niklas Hambuechen
03:23 PM Bug #23595: osd: recovery/backfill is extremely slow: OK, if I only have the 6 large files in the cephfs AND set the options... Niklas Hambuechen
02:55 PM Bug #23595: osd: recovery/backfill is extremely slow: I have now tested with only the 6*1GB files, having deleted the 270k empty files from cephfs.
I continue to see ex... Niklas Hambuechen
12:30 PM Bug #23595: osd: recovery/backfill is extremely slow: You can find a core dump of the -O0 version created with GDB at http://nh2.me/ceph-issue-23595-osd-O0.core.xz Niklas Hambuechen
12:06 PM Bug #23595: osd: recovery/backfill is extremely slow: Attached are two GDB runs of a sender node.
In the release build there were many values "<optimized out>", so I re... Niklas Hambuechen
11:45 AM Bug #23595: osd: recovery/backfill is extremely slow: On https://forum.proxmox.com/threads/increase-ceph-recovery-speed.36728/ people reported the same number as me of 10 ... Niklas Hambuechen
10:43 AM Bug #23601 (Resolved): Cannot see the value of parameter osd_op_threads via ceph --show-config: I have set the parameter of "osd op threads" in configuration file
but I cannot see the value of parameter "osd op t... Cyril Chang
10:17 AM Bug #23403 (Need More Info): Mon cannot join quorum: Brad Hubbard
07:23 AM Bug #23578 (In Progress): large-omap-object-warnings test fails: https://github.com/ceph/ceph/pull/21295 Brad Hubbard
01:33 AM Bug #23578: large-omap-object-warnings test fails: We instruct the OSDs to scrub at around 16:15.... Brad Hubbard
04:31 AM Bug #23593 (Fix Under Review): RESTControllerTest.test_detail_route and RESTControllerTest.test_f...: Kefu Chai
02:08 AM Bug #22123: osd: objecter sends out of sync with pg epochs for proxied ops: Despite the jewel backport of this fix being merged, this problem has reappeared in jewel 10.2.11 integration testing... Nathan Cutler

07:55 PM Bug #23595: osd: recovery/backfill is extremely slow: For the record, I installed the following debugging packages for gdb stack traces:... Niklas Hambuechen
07:53 PM Bug #23595: osd: recovery/backfill is extremely slow: I have read https://www.spinics.net/lists/ceph-devel/msg38331.html which suggests that there is some throttling going... Niklas Hambuechen
06:17 PM Bug #23595 (Duplicate): osd: recovery/backfill is extremely slow: I made a Ceph 12.2.4 (luminous stable) cluster of 3 machines with 10-Gigabit networking on Ubuntu 16.04, using pretty... Niklas Hambuechen
05:40 PM Bug #23593: RESTControllerTest.test_detail_route and RESTControllerTest.test_fill fail: PR: https://github.com/ceph/ceph/pull/21290 Ricardo Dias
03:10 PM Bug #23593 (Resolved): RESTControllerTest.test_detail_route and RESTControllerTest.test_fill fail: ... Kefu Chai
04:31 PM Documentation #23594: auth: document what to do when locking client.admin out: I found one way to fix it on the mailing list:
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-January/01... Niklas Hambuechen
04:23 PM Documentation #23594 (New): auth: document what to do when locking client.admin out: I accidentally ran ... Niklas Hambuechen
11:06 AM Bug #23590: kstore: statfs: (95) Operation not supported: https://github.com/ceph/ceph/pull/21287 Honggang Yang
11:01 AM Bug #23590 (Fix Under Review): kstore: statfs: (95) Operation not supported: 2018-04-07 16:19:07.248 7fdec4675700 -1 osd.0 0 statfs() failed: (95) Operation not supported
2018-04-07 16:19:08.... Honggang Yang
08:50 AM Bug #23589 (New): jewel: KStore Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3: Test description: rados/objectstore/objectstore.yaml
Log excerpt:... Nathan Cutler
08:39 AM Bug #23588 (New): LibRadosAioEC.IsCompletePP test fails in jewel 10.2.11 integration testing: Test description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yam... Nathan Cutler
06:53 AM Bug #23511: forwarded osd_failure leak in mon: Greg, no. both tests below include the no_reply() fix.
see
- http://pulpito.ceph.com/kchai-2018-03-30_15:29:52-r... Kefu Chai
06:42 AM Bug #23585 (Duplicate): osd: safe_timer segfault: ... Alex Gorbachev

« Previous

Also available in: Atom

Ceph » RADOS

Activity

Project

General

Profile

Ceph » RADOS

Activity

Activity

05/03/2018

05/02/2018

05/01/2018

04/30/2018

04/29/2018

04/28/2018

04/27/2018

04/26/2018

04/25/2018

04/24/2018

04/23/2018

04/22/2018

04/21/2018

04/20/2018

04/19/2018

04/18/2018

04/17/2018

04/16/2018

04/15/2018

04/14/2018

04/13/2018

04/12/2018

04/11/2018

04/10/2018

04/09/2018

04/08/2018

04/07/2018

04/06/2018

04/05/2018

04/04/2018