Activity
From 10/29/2018 to 11/27/2018
11/27/2018
- 08:40 PM Backport #36321 (In Progress): luminous: Add support for osd_delete_sleep configuration value
- 08:39 PM Backport #36321: luminous: Add support for osd_delete_sleep configuration value
- h3. original description
[RFE] Introduce an option or flag to throttle the pg deletion process
https://bugzilla.r... - 07:45 PM Bug #36250: ceph-osd process crashing
- I believe this issue was due to a malfunctioning ceph-fuse client, although I don't have data to back that up as it w...
- 06:02 PM Fix #37410 (Duplicate): change default osd_objectstore to bluestore
- duplicate of #36494
- 05:53 PM Fix #37410 (Fix Under Review): change default osd_objectstore to bluestore
- https://github.com/ceph/ceph/pull/25288
- 05:38 PM Fix #37410 (Duplicate): change default osd_objectstore to bluestore
- This way, the mon and associated tools know what the default actually is on the cluster.
- 06:01 PM Bug #36494: Change osd_objectstore default to bluestore
- Can you set this for backport to mimic and luminous?
- 03:30 PM Backport #37341 (In Progress): luminous: doc: Add bluestore memory autotuning docs
- 03:26 PM Backport #37340 (In Progress): mimic: doc: Add bluestore memory autotuning docs
- 02:27 PM Bug #36525: osd-scrub-snaps.sh failure
- /a/kchai-2018-11-27_11:44:27-rados-wip-kefu2-testing-2018-11-27-1724-distro-basic-smithi/3285226/teuthology.log
- 11:45 AM Bug #37404 (Fix Under Review): OSD mkfs might assert when working agains bluestore disk that alre...
- https://github.com/ceph/ceph/pull/25281/files
- 11:04 AM Bug #37404 (In Progress): OSD mkfs might assert when working agains bluestore disk that already h...
- 11:01 AM Bug #37404 (Resolved): OSD mkfs might assert when working agains bluestore disk that already has ...
- One might face an assert on collection's release which happens
after store destroy. For now is observable in some qa...
11/26/2018
- 11:49 PM Bug #24612 (Resolved): FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor::prune_init()
- 11:49 PM Backport #35071 (Resolved): mimic: FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor::p...
- 08:56 PM Backport #35071: mimic: FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor::prune_init()
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24918
merged - 11:48 PM Bug #22544 (Resolved): objecter cannot resend split-dropped op when racing with con reset
- 11:48 PM Backport #35843 (Resolved): mimic: objecter cannot resend split-dropped op when racing with con r...
- 08:55 PM Backport #35843: mimic: objecter cannot resend split-dropped op when racing with con reset
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24970
merged - 11:48 PM Bug #36358 (Resolved): Interactive mode CLI prints no output since Mimic
- 11:47 PM Backport #36432 (Resolved): mimic: Interactive mode CLI prints no output since Mimic
- 08:54 PM Backport #36432: mimic: Interactive mode CLI prints no output since Mimic
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24971
merged - 11:47 PM Backport #36433 (Resolved): mimic: monstore tool rebuild does not generate creating_pgs
- 08:54 PM Backport #36433: mimic: monstore tool rebuild does not generate creating_pgs
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25016
merged - 11:46 PM Backport #36435 (Resolved): mimic: rados rm --force-full is blocked when cluster is in full status
- 08:53 PM Backport #36435: mimic: rados rm --force-full is blocked when cluster is in full status
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25017
merged - 11:45 PM Backport #36505 (Resolved): mimic: mon osdmap cash too small during upgrade to mimic
- 08:53 PM Backport #36505: mimic: mon osdmap cash too small during upgrade to mimic
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25019
merged - 11:44 PM Backport #36557 (Resolved): mimic: RBD client IOPS pool stats are incorrect (2x higher; includes ...
- 08:52 PM Backport #36557: mimic: RBD client IOPS pool stats are incorrect (2x higher; includes IO hints as...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25024
merged - 11:44 PM Backport #36637 (Resolved): mimic: osd: race condition opening heartbeat connection
- 08:51 PM Backport #36637: mimic: osd: race condition opening heartbeat connection
- Patrick Donnelly wrote:
> https://github.com/ceph/ceph/pull/25026
merged - 11:43 PM Backport #36647 (Resolved): mimic: librados api aio tests race condition
- 08:51 PM Backport #36647: mimic: librados api aio tests race condition
- Patrick Donnelly wrote:
> https://github.com/ceph/ceph/pull/25027
merged - 11:40 PM Backport #36658 (Resolved): mimic: Cache-tier forward mode hang in luminous (again)
- 08:48 PM Backport #36658: mimic: Cache-tier forward mode hang in luminous (again)
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25075
merged - 08:45 PM Bug #37393 (Resolved): mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml
- Run: http://pulpito.front.sepia.ceph.com/yuriw-2018-11-21_22:16:20-rados-wip-yuri5-testing-2018-11-21-1510-mimic-dist...
11/25/2018
- 09:56 AM Bug #37326: Daily inconsistent objects
- Anyone has any idea?
11/23/2018
- 04:52 PM Bug #22597: "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails in upgrade test
- The problematic chown was introduced in mimic, so backporting only that far back.
See https://github.com/ceph/ceph... - 02:34 AM Backport #37288 (In Progress): mimic: "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails i...
- https://github.com/ceph/ceph/pull/25227
11/22/2018
- 05:19 PM Backport #37273 (Resolved): mimic: debian: packaging need to reflect move of /etc/bash_completion...
- 04:46 PM Backport #37273: mimic: debian: packaging need to reflect move of /etc/bash_completion.d/radosgw-...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/25115
merged - 07:32 AM Bug #36767: OSD: unrecoverable heartbeat connections
- see also: https://tracker.ceph.com/issues/36175
11/21/2018
- 08:25 AM Backport #37340 (Need More Info): mimic: doc: Add bluestore memory autotuning docs
- 07:19 AM Bug #37326: Daily inconsistent objects
- It happens on different disks, even on different host nodes.
- 06:40 AM Bug #24676: FreeBSD/Linux integration - monitor map with wrong sa_family
- Hello,
Just tested this and received the same "NetHandler create_socket couldn't create socket (97) Address family...
11/20/2018
- 09:42 PM Bug #36725: luminous: Apparent Memory Leak in OSD
- Upgraded one OSD server to 12.2.9. Clean reboot. Generating hourly report on memory and mempools. Three examples a...
- 09:10 PM Backport #37340: mimic: doc: Add bluestore memory autotuning docs
- This is blocked by mimic version of https://github.com/ceph/ceph/pull/24065
- 07:54 PM Backport #37340 (Resolved): mimic: doc: Add bluestore memory autotuning docs
- https://github.com/ceph/ceph/pull/25283
- 07:54 PM Backport #37343 (Resolved): luminous: Prioritize user specified scrubs
- https://github.com/ceph/ceph/pull/25514
- 07:54 PM Backport #37342 (Resolved): mimic: Prioritize user specified scrubs
- https://github.com/ceph/ceph/pull/25513
- 07:54 PM Backport #37341 (Resolved): luminous: doc: Add bluestore memory autotuning docs
- https://github.com/ceph/ceph/pull/25284
- 11:01 AM Bug #37289: Issue with overfilled OSD for cache-tier pools
- Whithout cache tiering everything is good.
After reaching 95% utilization of OSD for my replicated pool (whithout...
11/19/2018
- 10:57 PM Bug #36667: OSD object_map sync returned error
- This might also indicate something screwe dup the file permissions or ownership in /var/lib/ceph/osd/ceph-10. maybe ...
- 10:56 PM Bug #36709 (Need More Info): OSD stuck while flushing rocksdb WAL
- I'm not sure know rocksdb is what's stuck.. can you dump 'ceph daemon osd.NNN ops' to see what state teh oeprations a...
- 10:54 PM Bug #37264: scrub warning check incorrectly uses mon scrub interval
- You should be able to get the pool info out of the monitor's OSDMap, if that was a question... :)
- 10:51 PM Bug #37289: Issue with overfilled OSD for cache-tier pools
- I think teh first question to answer is if this can be reproduced without cache tiering. It's not immediately clear ...
- 10:48 PM Bug #37326 (Need More Info): Daily inconsistent objects
- Is this happening on the same disk all the time, or the same node? If so, that suggests a piece of hardware (e.g. con...
- 10:31 AM Bug #37326 (Need More Info): Daily inconsistent objects
- We have many Ceph mimic 13.2.1 installed with a similar configuration on ubuntu, but on one of them we get inconsiste...
- 10:48 PM Bug #36304 (Can't reproduce): FAILED ceph_assert(p != pg_slots.end()) in OSDShard::register_and_w...
- I'm guessing this was fixed by 450f337d6fd048c8c95a0ec0dec0d97f5474922e
- 10:43 PM Bug #36598: osd: "bluestore(/var/lib/ceph/osd/ceph-6) ENOENT on clone suggests osd bug"
- Sage thinks this might also be #36739.
- 10:40 PM Bug #36686 (In Progress): osd: pg log hard limit can cause crash during upgrade
- 10:40 PM Bug #36725 (Need More Info): luminous: Apparent Memory Leak in OSD
- can you dump the mempools (ceph daemon osd.NNN dump_mempools) several times over the growht of the process so we can ...
- 07:15 PM Bug #37269 (Pending Backport): Prioritize user specified scrubs
- 04:47 PM Bug #37329 (Pending Backport): doc: Add bluestore memory autotuning docs
- 04:44 PM Bug #37329 (Resolved): doc: Add bluestore memory autotuning docs
- https://github.com/ceph/ceph/pull/25069
11/17/2018
- 03:45 AM Bug #37299 (New): ceph-disk: ceph osd start failed: Command '['/usr/bin/systemctl', 'disable', 'c...
- Please see the details at:
https://bugzilla.redhat.com/show_bug.cgi?id=1649208#c0
11/16/2018
- 12:47 PM Bug #37289 (New): Issue with overfilled OSD for cache-tier pools
- We have bad issue in our ceph cluster.
Centos 7.5 (3.10.0-862.3.2.el7.x86_64)
Luminous 12.2.5, bluestore OSDs, us... - 11:35 AM Backport #37288 (Resolved): mimic: "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails in u...
- https://github.com/ceph/ceph/pull/25227
- 10:34 AM Bug #16500 (Resolved): ceph_erasure_code_benchmark parameter checking error for LRC plugin
- 06:22 AM Bug #22597 (Pending Backport): "sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0'" fails in upgra...
- 04:53 AM Bug #36767 (Fix Under Review): OSD: unrecoverable heartbeat connections
- 02:53 AM Feature #23493: config: strip/escape single-quotes in values when setting them via conf file/assi...
- Joao,
Could you take a look at https://github.com/ceph/ceph/pull/20610 and see whether you consider it something t... - 01:59 AM Bug #37264: scrub warning check incorrectly uses mon scrub interval
The scrub warning also doesn't consider the pool specific scrub interval if specified. The scrub code gets the p...
11/15/2018
- 01:16 PM Bug #25146 (Resolved): "rocksdb: Corruption: Can't access /000000.sst" in upgrade:mimic-x:paralle...
- 11:36 AM Backport #37273 (In Progress): mimic: debian: packaging need to reflect move of /etc/bash_complet...
- 10:47 AM Backport #37273: mimic: debian: packaging need to reflect move of /etc/bash_completion.d/radosgw-...
- PR with this backport is https://github.com/ceph/ceph/pull/25115
- 09:44 AM Backport #37273 (Resolved): mimic: debian: packaging need to reflect move of /etc/bash_completion...
- https://github.com/ceph/ceph/pull/25115
- 10:36 AM Backport #37274 (In Progress): luminous: debian: packaging need to reflect move of /etc/bash_comp...
- 09:45 AM Backport #37274 (Resolved): luminous: debian: packaging need to reflect move of /etc/bash_complet...
- https://github.com/ceph/ceph/pull/24997
- 09:38 AM Bug #36725: luminous: Apparent Memory Leak in OSD
- raising priority since this might be a regression in 12.2.9
- 06:31 AM Bug #36741 (Pending Backport): debian: packaging need to reflect move of /etc/bash_completion.d/r...
- https://github.com/ceph/ceph/pull/24996
- 06:20 AM Bug #37269 (Resolved): Prioritize user specified scrubs
When scrubs start backing up, when a user asks for a scrub it doesn't get priority compared to overdue scrubs. The...- 06:14 AM Bug #37264 (Resolved): scrub warning check incorrectly uses mon scrub interval
When checking the mon_warn_not_scrubbed the mon_scrub_interval is used instead of osd_scrub_max_interval.
11/14/2018
- 08:01 PM Bug #36725: luminous: Apparent Memory Leak in OSD
- Note: Downgrading both OSD servers to v12.2.8 returned memory usage to normal.
- 11:43 AM Backport #36636: luminous: osd: race condition opening heartbeat connection
- std::lock_guard is a C++11 feature: https://en.cppreference.com/w/cpp/header/mutex
11/13/2018
- 02:23 PM Backport #36658 (In Progress): mimic: Cache-tier forward mode hang in luminous (again)
- 02:15 PM Backport #36657 (In Progress): luminous: Cache-tier forward mode hang in luminous (again)
- 11:57 AM Bug #36388: osd: "out of order op"
- This looks like the dup op entries were exceeded so the op was not detected as a dup. Perhaps we should increase the ...
- 04:55 AM Bug #25146: "rocksdb: Corruption: Can't access /000000.sst" in upgrade:mimic-x:parallel-master-di...
- https://github.com/ceph/ceph/pull/25070
11/12/2018
- 03:41 PM Bug #36767: OSD: unrecoverable heartbeat connections
- Pull request:
https://github.com/ceph/ceph/pull/25061 - 03:09 PM Bug #36767 (Fix Under Review): OSD: unrecoverable heartbeat connections
- There are several unrecoverable heartbeat connections according to logs.
They usually appears after problems/reprodu... - 07:05 AM Bug #36758 (Duplicate): aborts in rocksdb::TableFileName() in mimic-x upgrade test suite
- 05:26 AM Bug #36758: aborts in rocksdb::TableFileName() in mimic-x upgrade test suite
- i think it's a dup of #25146
- 02:57 AM Bug #16500 (Fix Under Review): ceph_erasure_code_benchmark parameter checking error for LRC plugin
- https://github.com/ceph/ceph/pull/25046
11/10/2018
- 10:01 PM Bug #36758: aborts in rocksdb::TableFileName() in mimic-x upgrade test suite
- marking it "urgent", as it can be consistently reproducible. and it renders the cluster unusable after upgrading from...
- 06:11 PM Bug #36758 (Duplicate): aborts in rocksdb::TableFileName() in mimic-x upgrade test suite
- ...
- 02:33 PM Backport #36636 (In Progress): luminous: osd: race condition opening heartbeat connection
- 11:46 AM Backport #36636 (Need More Info): luminous: osd: race condition opening heartbeat connection
- The master commit uses std::lock_guard, which is a C++17-ism, and this makes the backport non-trivial (?)
- 12:42 PM Subtask #36091 (Resolved): [rbd top] collect client perf stats when query is enabled
- *PR*: https://github.com/ceph/ceph/pull/24265
- 11:56 AM Backport #36646 (In Progress): luminous: librados api aio tests race condition
- 11:52 AM Backport #36647 (In Progress): mimic: librados api aio tests race condition
- 11:40 AM Backport #36637 (In Progress): mimic: osd: race condition opening heartbeat connection
- 11:38 AM Backport #36556 (In Progress): luminous: RBD client IOPS pool stats are incorrect (2x higher; inc...
- 11:37 AM Backport #36557 (In Progress): mimic: RBD client IOPS pool stats are incorrect (2x higher; includ...
- 10:19 AM Backport #36506 (In Progress): luminous: mon osdmap cash too small during upgrade to mimic
- 10:05 AM Backport #36505 (In Progress): mimic: mon osdmap cash too small during upgrade to mimic
- 09:59 AM Backport #36436 (In Progress): luminous: rados rm --force-full is blocked when cluster is in full...
- 09:54 AM Backport #36435 (In Progress): mimic: rados rm --force-full is blocked when cluster is in full st...
- 09:02 AM Backport #36433 (In Progress): mimic: monstore tool rebuild does not generate creating_pgs
11/09/2018
- 10:08 PM Bug #36667: OSD object_map sync returned error
- Check dmesg for hardware errors, this is leveldb/rocksdb returning an error writing to disk. You may want to ask the ...
- 10:05 PM Bug #36677 (Resolved): /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- 10:05 PM Bug #36732 (Fix Under Review): tools/rados: fix segmentation fault
- https://github.com/ceph/ceph/pull/24990
- 08:55 PM Bug #36610 (Resolved): filestore merge collection replay problem
- 08:54 PM Bug #36748 (New): ms_deliver_verify_authorizer no AuthAuthorizeHandler found for protocol 0
- ...
- 05:18 PM Bug #36746 (New): Ignore osd_find_best_info_ignore_history_les for erasure-coded PGs
The only case that osd_find_best_info_ignore_history_les would work for erasure coded pools is if an interval didn'...- 09:29 AM Bug #36741 (Resolved): debian: packaging need to reflect move of /etc/bash_completion.d/radosgw-a...
- Hi,
Between version 12.0.2 and 12.0.3, the file /etc/bash_completion.d/radosgw-admin moved from the radosgw packag...
11/08/2018
- 11:34 PM Bug #36739: ENOENT in collection_move_rename on EC backfill target
- we create a gen object normally, on a backfill target,...
- 10:25 PM Bug #36739: ENOENT in collection_move_rename on EC backfill target
- 10:24 PM Bug #36739 (Resolved): ENOENT in collection_move_rename on EC backfill target
- ...
- 09:13 PM Feature #36737: Allow multi instances of "make tests" on the same machine
- @Kefu pls take a look, IIRC you mentioned that this may not be a big effort.
- 09:12 PM Feature #36737 (Resolved): Allow multi instances of "make tests" on the same machine
- Currently it's only possible to run `...make; make tests -j8; ctest ...` on the same machine.
Please consider chan... - 10:02 AM Bug #36732 (Resolved): tools/rados: fix segmentation fault
- when connected to ceph cluster, if call exit(1) directly, will
cause the finisher thread segmentation fault as follo...
11/07/2018
- 11:37 PM Feature #24917: Gracefully deal with upgrades when bluestore skipping of data_digest becomes active
Josh, this code needs to be written. It needs a feature bit AND a mon flag that can only be set when all OSDs are ...- 10:07 PM Backport #36729 (Resolved): mimic: Add support for osd_delete_sleep configuration value
- https://github.com/ceph/ceph/pull/25507
- 10:06 PM Feature #36474 (Pending Backport): Add support for osd_delete_sleep configuration value
- 04:40 PM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- Tests added:
https://github.com/ceph/ceph/pull/24954
https://github.com/ceph/ceph/pull/24938 - 04:27 PM Bug #36725 (Closed): luminous: Apparent Memory Leak in OSD
- Since last update (late October), been experiencing apparent memory leak in OSD process on two ceph servers in small ...
- 11:44 AM Backport #36432 (In Progress): mimic: Interactive mode CLI prints no output since Mimic
- 11:42 AM Backport #35843 (In Progress): mimic: objecter cannot resend split-dropped op when racing with co...
11/06/2018
- 01:22 PM Bug #20798: LibRadosLockECPP.LockExclusiveDurPP gets EEXIST
- /a/sage-2018-11-05_22:04:25-rados-wip-sage3-testing-2018-11-05-1406-distro-basic-smithi/3227352
- 11:54 AM Support #36326: Huge traffic spike and assert(is_primary())
- Thanks for the answer! It looks like traffic spike was caused by another issue: ceph-mon's db grows up to 15GB and it...
- 10:07 AM Bug #36709 (Closed): OSD stuck while flushing rocksdb WAL
- Hi all,
We use:
ceph version 12.2.8 (ae699615bac534ea496ee965ac6192cb7e0e07c0) luminous (stable)
Clients work on:
... - 01:30 AM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- Quoting my reply to ceph-devel for reference:
"Nathan, I don't think we want to revert it for 13.2.2.
This is b...
11/05/2018
- 10:42 PM Bug #22902 (Resolved): src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")
- 10:32 PM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- So, the luminous revert was merged. Neha, will there be a mimic revert as well? Since the pg hard limit patches are p...
- 10:13 PM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- https://github.com/ceph/ceph/pull/24903 merged
- 10:28 PM Bug #36508 (Resolved): gperftools-libs-2.6.1-1 or newer required for binaries linked against corr...
- 10:28 PM Backport #36552 (Resolved): luminous: gperftools-libs-2.6.1-1 or newer required for binaries link...
- 10:10 PM Backport #36552: luminous: gperftools-libs-2.6.1-1 or newer required for binaries linked against ...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24706
merged - 10:25 PM Bug #34541 (Resolved): deep scrub cannot find the bitrot if the object is cached
- 10:25 PM Backport #35067 (Resolved): luminous: deep scrub cannot find the bitrot if the object is cached
- 10:08 PM Backport #35067: luminous: deep scrub cannot find the bitrot if the object is cached
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24802
merged - 10:18 PM Backport #36678 (Resolved): luminous: src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state...
- 05:20 PM Feature #24917: Gracefully deal with upgrades when bluestore skipping of data_digest becomes active
- Let's include this with any other feature bit addition.
- 01:30 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- > I suspect it shouldn't.
But it does exactly that.
> That's will only re-copy the data to the HEAD revision.
...
11/04/2018
- 06:55 PM Bug #36677 (Fix Under Review): /usr/include/rados/buffer.h:657:61: error: expected ',' before ')'...
- A fix is already available. See Sage's PR: https://github.com/ceph/ceph/pull/24835.
11/03/2018
- 11:27 PM Bug #24923 (Resolved): doc: http://docs.ceph.com/docs/mimic/rados/operations/pg-states/
- 11:27 PM Backport #25055 (Resolved): mimic: doc: http://docs.ceph.com/docs/mimic/rados/operations/pg-states/
- 11:26 PM Backport #35071 (In Progress): mimic: FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor...
- 04:42 AM Backport #23670 (In Progress): luminous: auth: ceph auth add does not sanity-check caps
- 04:24 AM Backport #23670 (New): luminous: auth: ceph auth add does not sanity-check caps
- Kefu did the jewel backport, so assigning this to him in hopes he'll pick it up.
- 04:00 AM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- -Also, is this bug reproducible in master and mimic as well? If not, the Backport field should probably be modified.....
- 03:58 AM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- Neha, 12.2.9 has already been cut, so we'll need to expedite 12.2.10 to push the revert out to users.
- 03:52 AM Backport #36678 (In Progress): luminous: src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad st...
11/02/2018
- 11:57 PM Bug #36686: osd: pg log hard limit can cause crash during upgrade
- The immediate fix is to revert this for luminous before 12.2.9: https://github.com/ceph/ceph/pull/24903
- 11:51 PM Bug #36686 (Resolved): osd: pg log hard limit can cause crash during upgrade
- During an upgrade from an earlier version, a primary running the new code will send a trim_to value to a replica that...
- 05:14 PM Bug #36677: /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- Ceph has already moved to C++17. The main question is: have we transitioned to C++17 also our public headers xor put ...
- 04:58 PM Bug #36677: /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- The no-message-taking-variant of *static_assert* has been introduced in C++17. The code is being compiled with *-std=...
- 04:55 PM Bug #36677 (In Progress): /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- 05:14 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- Back-and-forth question answering like this is probably better for the mailing list (the ticket is currently closed F...
- 04:57 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- since you've identified that this is an RBD workload, assigning it to that project so that RBD team notices it. HTH.
- 02:37 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- Oops. That's more than 2 questions. But anyway :)
- 02:36 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- OK, I looked into OSD datastore using ceph-objectstore-tool and I see that for almost every object there are two copi...
- 01:39 PM Bug #24835: osd daemon spontaneous segfault
- We do use some configuration set by "ceph config set" or "ceph config-key set":...
11/01/2018
- 11:46 PM Backport #36678 (Resolved): luminous: src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state...
- https://github.com/ceph/ceph/pull/24902
- 11:19 PM Bug #22902 (Pending Backport): src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machin...
- Based on similar failures seen in luminous: http://pulpito.ceph.com/yuriw-2018-10-31_22:45:22-rados-wip-yuri4-testing...
- 09:10 PM Bug #36677: /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- ...
- 09:06 PM Bug #36677 (Resolved): /usr/include/rados/buffer.h:657:61: error: expected ',' before ')' token
- ...
- 04:44 PM Bug #36289: Converting Filestore OSD from leveldb to rocksdb backend on CentOS
- Looking through the ceph/rocksdb repo I don't see how it's possible for rocksdb to be compiled without snappy support...
- 03:35 PM Bug #36289: Converting Filestore OSD from leveldb to rocksdb backend on CentOS
- This seems to be a problem where rocksdb on CentOS doesn't support snappy compression but the ceph-kvstore-tool is co...
- 06:14 AM Bug #36667 (New): OSD object_map sync returned error
- i deploy a cephfs and the used the vdbench tool to wirte data in cephfs mount point,after a while osd appears down.
...
10/31/2018
- 09:21 PM Bug #36411 (Closed): OSD crash starting recovery/backfill with EC pool
- It's my current belief that these objects were broken as a result of intentional metadata manipulation when some of t...
- 09:18 PM Bug #36572: ceph-in: --connect-timeout doesn't work while pinging mon
- New PR: https://github.com/ceph/ceph/pull/24733
- 09:17 PM Support #36584 (Closed): OSD Anomaly behaviour in ceph-reweight
- Are you running the command repeatedly? reweight-by-utilization does not provide a stable balance; it's really just a...
- 08:43 PM Bug #21496: doc: Manually editing a CRUSH map, Word 'type' missing.
- https://github.com/ceph/ceph/pull/24868
- 05:35 PM Feature #36661: osd: add sanity check on startup to compare osd memory target to available memory...
- - in OSD::handle_conf_change, we should sanity check this against current memory available on the system and refuse t...
- 04:59 PM Feature #36661 (New): osd: add sanity check on startup to compare osd memory target to available ...
- This is needed so that we do not fail due to osd_memomory_target being set too high compared to the amount of memory ...
- 11:42 AM Backport #36658 (Resolved): mimic: Cache-tier forward mode hang in luminous (again)
- https://github.com/ceph/ceph/pull/25075
- 11:42 AM Backport #36657 (Resolved): luminous: Cache-tier forward mode hang in luminous (again)
- https://github.com/ceph/ceph/pull/25074
10/30/2018
- 08:08 PM Bug #36345 (Resolved): librados C API aio read empty buffer
- 08:07 PM Bug #36406 (Pending Backport): Cache-tier forward mode hang in luminous (again)
- 05:16 PM Backport #36647 (Resolved): mimic: librados api aio tests race condition
- https://github.com/ceph/ceph/pull/25027
- 05:16 PM Backport #36646 (Resolved): luminous: librados api aio tests race condition
- https://github.com/ceph/ceph/pull/25028
- 05:14 PM Backport #36637 (Resolved): mimic: osd: race condition opening heartbeat connection
- https://github.com/ceph/ceph/pull/25026
- 05:14 PM Backport #36636 (Resolved): luminous: osd: race condition opening heartbeat connection
- https://github.com/ceph/ceph/pull/25035
- 04:06 PM Bug #36634 (New): LibRadosWatchNotify.WatchNotify2Timeout failure
- ...
- 03:33 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- Yes, I'm using EC with RBD and partial overwrites enabled. CephFS pools are only created recently for tests and do no...
- 01:05 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- How are you writing these objects? Most sites that used EC were using RGW, but I don't see all the pools that go wit...
- 10:31 AM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- In fact it doesn't seem that it will self-heal, and nobody seems to care about it in the mailing list by now...)
C... - 02:33 PM Bug #36631 (In Progress): potential deadlock in PG::_scan_snaps when repairing snap mapper
- If during a pg scrub a snap mapper error is detected in PG::_scan_snaps, on repair `ObjectStore::apply_transactions` ...
- 02:28 PM Backport #36630 (Resolved): luminous: potential deadlock in PG::_scan_snaps when repairing snap m...
- If during a pg scrub a snap mapper error is detected in PG::_scan_snaps, on repair `ObjectStore::apply_transactions` ...
- 02:00 PM Bug #36629 (New): osd:the new file was stored in cache pool which mode was none
- ceph version:13.2.1
kernel client 4.17
I created the cache data pool as ceph's instructions:
(1) ceph osd tier add... - 01:41 AM Bug #36620: osd:the vim will be hanged when I saved the file
- the client: 4.17 kernel client
- 01:36 AM Bug #36620 (New): osd:the vim will be hanged when I saved the file
- ceph version: 13.2.1
situtation: the data pool tiered by a cache data pool and the cache tier pool's mode was read...
10/29/2018
- 10:33 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- Thanks for the response, I wrote to the mailing list ceph-users (is it the correct place?) :)
- 08:37 PM Support #36614 (Closed): Cluster uses substantially more space after rebalance (erasure codes)
- The mailing list is a better place to resolve this. My guess is data hasn't been cleaned up from its old locations ye...
- 12:13 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- How to heal it? If I don't heal it I'll need to purge the whole cluster? O_o...
- 12:12 PM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- ceph df output:...
- 11:11 AM Support #36614: Cluster uses substantially more space after rebalance (erasure codes)
- Proofs from our prometheus monitoring. Two graphs from yesterday: one with number of objects in cluster and other wit...
- 10:17 AM Support #36614 (Closed): Cluster uses substantially more space after rebalance (erasure codes)
- Hi
After I recreated one OSD + increased pg count of my erasure-coded (2+1) pool (which was way too low, only 100 ... - 10:21 PM Bug #36525: osd-scrub-snaps.sh failure
Looking at the log another scrub has made the number of "_scan_snaps start" in the log from 2 to 4. It results in ...- 01:06 AM Bug #36525: osd-scrub-snaps.sh failure
- /a/sage-2018-10-28_14:12:19-rados-master-distro-basic-smithi/3196520
another instance on current master - 09:48 PM Bug #23827 (Resolved): osd sends op_reply out of order
- 09:47 PM Backport #25010 (Resolved): mimic: osd sends op_reply out of order
- 08:47 PM Backport #25010: mimic: osd sends op_reply out of order
- https://github.com/ceph/ceph/pull/23136 has merged, can we resolve this issue?
- 09:43 PM Bug #25154 (Resolved): librados application's symbol could conflict with the libceph-common
- 09:42 PM Backport #26839 (Resolved): mimic: librados application's symbol could conflict with the libceph-...
- 08:21 PM Backport #26839: mimic: librados application's symbol could conflict with the libceph-common
- Patrick Donnelly wrote:
> https://github.com/ceph/ceph/pull/24708
merged - 09:40 PM Bug #35969 (Resolved): "symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm" on centos 7.4
- 09:39 PM Backport #36553 (Resolved): mimic: gperftools-libs-2.6.1-1 or newer required for binaries linked ...
- 08:16 PM Backport #36553: mimic: gperftools-libs-2.6.1-1 or newer required for binaries linked against cor...
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24260
merged - 09:39 PM Backport #36132 (Resolved): mimic: "symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm" on ...
- 08:16 PM Backport #36132: mimic: "symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm" on centos 7.4
- Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/24260
merged - 08:47 PM Bug #23387: Building Ceph on armhf fails due to out-of-memory
- The above changes is not entirely correct. This section needs to be ommited:...
- 08:13 PM Bug #23387: Building Ceph on armhf fails due to out-of-memory
- Hello!
I've used the instruction created by Daniel Glasser and with some small code adjustments in a few files I w... - 04:17 PM Bug #36610 (Fix Under Review): filestore merge collection replay problem
- https://github.com/ceph/ceph/pull/24806
- 03:51 PM Bug #36610: filestore merge collection replay problem
- the osd is stopped during the merge operation:...
- 03:46 PM Bug #36182 (Resolved): osd: hung op "osd.3 22 get_health_metrics reporting 2 slow ops, oldest is ...
- 02:59 PM Bug #36473 (Resolved): hung osd_repop, bluestore committed but failed to trigger repop_commit
- this is presumably https://github.com/ceph/ceph/pull/24761
- 02:58 PM Bug #36548 (Resolved): qa/standalone/osd/osd-rep-recov-eio.sh
- 01:34 PM Bug #20798: LibRadosLockECPP.LockExclusiveDurPP gets EEXIST
- /a/sage-2018-10-29_01:11:58-rados-wip-sage-testing-2018-10-28-0943-distro-basic-smithi/3197984
- 01:10 AM Bug #36408 (Resolved): [cache tier] failed guarded write + promotion results in "success" op result
Also available in: Atom