Project

General

Profile

Activity

From 03/01/2018 to 03/30/2018

03/30/2018

05:18 AM Bug #22424 (Resolved): balancer should warn about missing requirements
Nathan Cutler
02:27 AM Backport #22983 (Resolved): luminous: balancer should warn about missing requirements
xie xingguo

03/29/2018

01:22 PM Backport #22983: luminous: balancer should warn about missing requirements
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/20359
merged
Yuri Weinstein
08:18 AM Bug #23083 (Resolved): ceph-mgr fails to start after a system reboot on Ubuntu 16.04
Nathan Cutler
08:17 AM Backport #23101 (Resolved): luminous: ceph-mgr fails to start after a system reboot on Ubuntu 16.04
Nathan Cutler

03/28/2018

10:28 PM Backport #23101: luminous: ceph-mgr fails to start after a system reboot on Ubuntu 16.04
Nathan Cutler wrote:
> https://github.com/ceph/ceph/pull/20604
merged
Yuri Weinstein
04:25 PM Bug #23219 (Resolved): Update mgr/restful documentation
Nathan Cutler
04:25 PM Backport #23230 (Resolved): luminous: Update mgr/restful documentation
Nathan Cutler
03:35 PM Bug #23482: ceph-mgr --help stopped working (regression in master)
If my regression hypothesis is correct, the "culprit" would be one or more of these:... Nathan Cutler
02:39 PM Bug #23482 (Resolved): ceph-mgr --help stopped working (regression in master)
(Opening with increased priority since this appears to be a regression)
With a recent master, ceph-mgr --help stop...
Nathan Cutler
08:31 AM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
So I set the logging of the active manager to:
- debug_mgr: 20
- debug_auth: 5
After running for over 24 hours...
Wido den Hollander
02:31 AM Backport #23416 (Need More Info): luminous: mgr sends early beacon with no modules reported
File: src/mgr/PyModule.h needs to be added to luminous to backport this PR. Prashant D

03/27/2018

10:32 AM Dashboard Bug #23406 (Resolved): Attempt to set dashboard login credentials causes ceph-mgr to crash in Pyt...
Ricardo Dias
07:18 AM Backport #23224 (In Progress): luminous: mgr log spamming about down osds
https://github.com/ceph/ceph/pull/21053 Prashant D

03/26/2018

02:21 PM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
Raising priority since this affects a stable release (Luminous) Nathan Cutler
01:52 PM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
I just found another mgr in this cluster which is "dead", but is eating 300% CPU on 3 cores:... Wido den Hollander
01:39 PM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
I found another mgr in a cluster which has been down now for 4 days.... Wido den Hollander
11:05 AM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
I don't think the TZ is related. In all the cases I've seen the Mon and Mgr were running on the same node.
I saw i...
Wido den Hollander
10:50 AM Bug #23460: mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too...
This one's still a mystery to me :-/
Only initial thought is that we're in daylight-savings territory right now, c...
John Spray
08:46 AM Bug #23460 (Resolved): mgr deadlock: _check_auth_rotating possible clock skew, rotating keys expi...
On Luminous v12.2.2 and v12.2.4 clusters running either CentOS or Ubuntu I've seen many Manager going offline with th... Wido den Hollander
10:00 AM Dashboard Bug #23404 (Resolved): dashboard module does not work in Python 3-only environment
Ricardo Dias
08:40 AM Bug #22226 (Rejected): ceph zabbix plugin sends incorrect motinoring info to zabbix server
I this one still active? Otherwise we can close it I think.
Setting it to Rejected for now as I think it is resolv...
Wido den Hollander

03/25/2018

03:19 PM Bug #23205: Blocked requests no longer show details
We continue to get these with no clues as to where to go next:
2018-03-25 06:58:39.693404 7fa0e9ee2700 0 log_chan...
Alex Gorbachev

03/23/2018

02:03 AM Dashboard Bug #23326 (Resolved): mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot ...
Fixed by: https://github.com/ceph/ceph/pull/20986 Patrick Donnelly

03/22/2018

12:21 PM Dashboard Bug #23404 (Fix Under Review): dashboard module does not work in Python 3-only environment
PR: https://github.com/ceph/ceph/pull/21006 Ricardo Dias
08:30 AM Dashboard Bug #23404 (In Progress): dashboard module does not work in Python 3-only environment
Ricardo Dias
12:13 PM Dashboard Bug #23406 (Fix Under Review): Attempt to set dashboard login credentials causes ceph-mgr to cras...
PR: https://github.com/ceph/ceph/pull/21005 Ricardo Dias
12:12 PM Dashboard Bug #23406: Attempt to set dashboard login credentials causes ceph-mgr to crash in Python 3-only ...
The current `handle_pyerror` function implementation relies in the `traceback.format_exception_only` python function ... Ricardo Dias
08:31 AM Dashboard Bug #23406 (In Progress): Attempt to set dashboard login credentials causes ceph-mgr to crash in ...
Ricardo Dias

03/20/2018

11:20 AM Backport #23409 (In Progress): luminous: mgr: fix MSG_MGR_MAP handling
https://github.com/ceph/ceph/pull/20973 Prashant D

03/19/2018

08:29 PM Bug #23418 (Won't Fix): doc: Dashboard account creation
The documentation at http://docs.ceph.com/docs/master/mgr/dashboard/ refers the the following incorrect command:
...
Marc Schöchlin
04:43 PM Backport #23416 (Rejected): luminous: mgr sends early beacon with no modules reported
Nathan Cutler
04:42 PM Backport #23409 (Resolved): luminous: mgr: fix MSG_MGR_MAP handling
https://github.com/ceph/ceph/pull/20973 Nathan Cutler
03:18 PM Dashboard Bug #23406 (Resolved): Attempt to set dashboard login credentials causes ceph-mgr to crash in Pyt...
Environment: SLE-15 (Python 3-only)
Currently running mimic_dev2 but the issue is presumed to be present on master...
Nathan Cutler
02:25 PM Dashboard Bug #23404 (Resolved): dashboard module does not work in Python 3-only environment
Environment: SLE-15 (Python 3-only)
After enabling the dashboard, it is impossible to run "ceph mgr dashboard set-...
Nathan Cutler
01:14 AM Feature #23400 (New): ceph-mgr should raise health alert if OSDs are up but no data being received
John Spray

03/17/2018

10:46 AM Feature #23397 (Resolved): Central "mgr self-test <module>" command
Currently, many of the manager modules implement a "self-test" command (validate that they can fetch and parse their ... John Spray
10:38 AM Bug #21598 (Fix Under Review): Users can do "config-key set" while mgr runs, but it doesn't see s...
This issue will be fixed implicitly when we merge https://github.com/ceph/ceph/pull/20458 John Spray
10:27 AM Feature #21682 (Resolved): mgr should raise health alert when a module throws exceptions (post-load)
John Spray
10:27 AM Dashboard Feature #22522 (Resolved): dashboard: configuration setting browser
John Spray

03/16/2018

02:37 PM Dashboard Bug #23389 (Resolved): dashboard: OSD throughput sparkline graphic appears to show running total
... I suspect it's meant to show historical rates instead. Jason Dillaman
11:14 AM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
A vstart flag is easier to implement, and we should fix this problem that way in the short term. But we can also thin... Ricardo Dias
03:26 AM Bug #23368 (Pending Backport): mgr: fix MSG_MGR_MAP handling
Kefu Chai
03:24 AM Bug #23378 (Resolved): Test failure: test_perf_counters_mgr_get (tasks.mgr.dashboard_v2.test_perf...
https://github.com/ceph/ceph/pull/20916 Kefu Chai

03/15/2018

07:15 PM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
John Spray wrote:
> I think what's going on here is that the dashboard used to fail silently (it always required rbd...
Patrick Donnelly
12:40 PM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
I think what's going on here is that the dashboard used to fail silently (it always required rbd bindings), because v... John Spray
11:14 AM Bug #23378 (Resolved): Test failure: test_perf_counters_mgr_get (tasks.mgr.dashboard_v2.test_perf...
... Kefu Chai
06:21 AM Bug #22918 (Pending Backport): mgr sends early beacon with no modules reported
Kefu Chai
02:10 AM Bug #23368 (Fix Under Review): mgr: fix MSG_MGR_MAP handling
Kefu Chai
01:26 AM Bug #23368 (Resolved): mgr: fix MSG_MGR_MAP handling
ceph config show mgr.x doesn't work
root cause is mgr daemon's mgrc has no chance
to process MSG_MGR_MAP in the mgr...
cory gu

03/14/2018

05:38 AM Backport #23313 (In Progress): luminous: mgr: prometheus: internal server error while new OSDs ar...
https://github.com/ceph/ceph/pull/20891 Prashant D

03/13/2018

04:28 PM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
I understand the temptation but I'd prefer this be fixed. Taking out RBD compilation has significantly improved compi... Patrick Donnelly
04:21 PM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
It's not impossible to put some special handling in the dashboard for disabling rbd functionality when it's missing, ... John Spray
07:19 AM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
Patrick Donnelly wrote:
> Omitting RBD must be why. Is there a way I can avoid this mgr error without building RBD...
Ricardo Dias
12:48 AM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
Oh, I'm building with:... Patrick Donnelly
12:16 AM Dashboard Bug #23326: mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot handle comm...
Generally that's what you'd get from an incomplete build, perhaps doing the `cephfs_testing` vstart target rather tha... John Spray
04:28 PM Bug #23300 (Duplicate): ceph-mgr returns internal error
This was fixed in master recently and is being backported to luminous here: https://github.com/ceph/ceph/pull/20642
John Spray
01:55 PM Bug #23330: mon command "mgr metadata $name" has inconsistent argument naming
Marking for backport because will want to take it along with fix for 23286 when that's done John Spray
01:55 PM Bug #23330 (Fix Under Review): mon command "mgr metadata $name" has inconsistent argument naming
This command was using `id` where all the other metadata commands were using `who`, so anyone passing `who` is gettin... John Spray
11:33 AM Bug #23330 (Resolved): mon command "mgr metadata $name" has inconsistent argument naming
When running the following mon command:... Jan Fajerski
11:34 AM Bug #23286: mgr: ActivePyModules::list_servers_python() returns mgr with empty hostname
I have a potential fix (updating the metadata in DaemonServer::got_mgr_map()) but encountered http://tracker.ceph.com... Jan Fajerski
12:47 AM Bug #23276: balancer/osd: segfault in calc_pg_upmaps
Hi Dan,
There is already a pending backport for Luminous, see https://github.com/ceph/ceph/pull/20840
xie xingguo

03/12/2018

11:43 PM Dashboard Bug #23326 (Resolved): mgr: Error EIO: Module 'dashboard_v2' has experienced an error and cannot ...
When running vstart against master:... Patrick Donnelly
04:08 PM Bug #23276: balancer/osd: segfault in calc_pg_upmaps
Great thanks.
Could someone please add the backports tag for l?
Dan van der Ster
06:42 AM Bug #23276: balancer/osd: segfault in calc_pg_upmaps
https://github.com/ceph/ceph/pull/20655 xie xingguo
09:14 AM Backport #23313 (Resolved): luminous: mgr: prometheus: internal server error while new OSDs are b...
https://github.com/ceph/ceph/pull/21492 Nathan Cutler
03:10 AM Bug #23205: Blocked requests no longer show details
in other words, what is the best way to troubleshoot the following situation:
2018-03-11 22:00:00.000132 mon.roc-v...
Alex Gorbachev

03/11/2018

07:26 PM Bug #23300: ceph-mgr returns internal error
Found it! We had several osds without a device class attached, because we did not want to use them at the moment.
Ad...
Nico Schottelius
07:20 PM Bug #23300: ceph-mgr returns internal error
Fun fact: it used to run fine until we were introducing new crush rules and changing the crush rule for a pool:
<p...
Nico Schottelius
07:17 PM Bug #23300 (Duplicate): ceph-mgr returns internal error
Hello,
after some weeks of running a new ceph cluster, we get the following answer from the mgr:
black3.place6:...
Nico Schottelius
05:10 AM Bug #23205: Blocked requests no longer show details
the one big advantage of having the few slowest OSDs listed in ceph.log or MON log was the ability to go back to trou... Alex Gorbachev

03/09/2018

01:59 PM Bug #23286: mgr: ActivePyModules::list_servers_python() returns mgr with empty hostname
This bug is a result of how we populate the mgrs into DaemonState from DaemonServer::got_mgr_map without ever reading... John Spray
01:08 PM Bug #23286: mgr: ActivePyModules::list_servers_python() returns mgr with empty hostname
- Hello, what's your ceph version? -
Oh, sorry. it's "ceph version 13.0.1-2832 .. mimic (dev)".
Chang Liu
10:50 AM Bug #23286 (Resolved): mgr: ActivePyModules::list_servers_python() returns mgr with empty hostname
When running a vstart.sh cluster on master at cf52fc5a, @list_servers_python()@ returns:... Sebastian Wagner
02:23 AM Dashboard Bug #23265 (Resolved): FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterC...
Kefu Chai

03/08/2018

04:24 PM Bug #23276 (Resolved): balancer/osd: segfault in calc_pg_upmaps
This cluster has upmapped pgs and the upmap balancer enabled. We moved two hosts from root=default to a new root=drai... Dan van der Ster
03:50 PM Bug #23136 (In Progress): mgr: disable then enable of Restful plugin does not work without restar...
https://github.com/ceph/ceph/pull/20365 is aiming to address this. but it needs some more polish. Kefu Chai
03:16 PM Bug #23166 (Pending Backport): mgr: prometheus: internal server error while new OSDs are being ad...
Kefu Chai
09:25 AM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
> I don't understand how the issue is fixed just by removing unused code,
The test failure was fixed by removing u...
Sebastian Wagner
07:51 AM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
Sebastian Wagner wrote:
> ... the second one was a rename of "developer" to "dev", which was fixed by removing unuse...
Ricardo Dias

03/07/2018

05:58 PM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
Turned out, this issue consists of two parts: the first one is described above, and the second one was a rename of "d... Sebastian Wagner
04:26 PM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
Some details. Before my commit,
1. @md_config_t::config_options@ expected @Option::dump@ to start an @object@ sec...
Sebastian Wagner
03:40 PM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
Here is the PR: https://github.com/ceph/ceph/pull/20782 Sebastian Wagner
03:08 PM Dashboard Bug #23265: FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterConfiguratio...
To reproduce this bug:... Ricardo Dias
03:04 PM Dashboard Bug #23265 (Resolved): FAIL: test_get (tasks.mgr.dashboard_v2.test_cluster_configuration.ClusterC...
The dashboard_v2 API tests became broken in master (sha-1 480f078c147792c6bb234ee0c0b5ab9dbead5b05)... Ricardo Dias
03:56 PM Dashboard Bug #22098 (Resolved): dashboard failing to load audit log
John Spray
03:55 PM Feature #21502 (Resolved): Enable mgr modules to report their "runnability"
John Spray
03:54 PM Bug #22717 (Resolved): mgr: prometheus: impossible to join labels from metadata metrics.
John Spray
11:21 AM Feature #23042 (Fix Under Review): Prompt ServiceMap removal on clean shutdown
https://github.com/ceph/ceph/pull/20761 John Spray
02:39 AM Bug #23205: Blocked requests no longer show details
https://tracker.ceph.com/issues/23236 was rejected.
"ceph health" should list the osds with slow ops and dump_bloc...
Brad Hubbard

03/06/2018

03:56 PM Bug #23249 (Resolved): ceph osd safe-to-destroy crashes the mgr
On luminous v12.2.4:... Dan van der Ster
02:46 PM Bug #23205: Blocked requests no longer show details
Got it, thanks John. I think you are exactly right that most people either see problems coming from one or a couple O... Alex Gorbachev
09:56 AM Bug #23205: Blocked requests no longer show details
The "last X slow requests" was emitted separately by each OSD, so even though it was a summary, you'd still get a flo... John Spray
12:33 AM Bug #23205: Blocked requests no longer show details
John, thank you for answering. Was it not the reason for "last X slow requests" in case this helps at least lead in ... Alex Gorbachev

03/05/2018

09:51 PM Backport #23230 (In Progress): luminous: Update mgr/restful documentation
Nathan Cutler
09:49 PM Backport #23230 (Resolved): luminous: Update mgr/restful documentation
https://github.com/ceph/ceph/pull/20725 Nathan Cutler
09:38 PM Bug #23219 (Pending Backport): Update mgr/restful documentation
Boris Ranto
01:49 PM Bug #23219: Update mgr/restful documentation
https://github.com/ceph/ceph/pull/20717 Nathan Cutler
01:45 PM Bug #23219 (Resolved): Update mgr/restful documentation
Boris Ranto
09:10 PM Bug #22457 (In Progress): ceph-mgr dashboard has dependency on python-jinja2
Reopening - there is a problem with the original master fix (it introduced the python-jinja2 as a build dependency in... Nathan Cutler
05:15 PM Bug #23205: Blocked requests no longer show details
The details are there, but they're no longer dumped to the cluster log by default. The reason is that if you have 10... John Spray
05:02 PM Backport #23224 (Resolved): luminous: mgr log spamming about down osds
https://github.com/ceph/ceph/pull/21053 Nathan Cutler
10:47 AM Backport #23175 (In Progress): luminous: mgr not reporting when ports conflict
https://github.com/ceph/ceph/pull/20712 Prashant D
09:57 AM Bug #23017 (Pending Backport): mgr log spamming about down osds
Kefu Chai

03/04/2018

06:24 PM Bug #23205: Blocked requests no longer show details
Further note: the slowdowns and blocked requests start occurring after about 2TB of I/O on the image. Specifically, ... Alex Gorbachev
04:00 AM Bug #23205: Blocked requests no longer show details
Further info: these occur when journaling, object-map and exclusive-lock are set on the rbd image Alex Gorbachev

03/02/2018

11:15 PM Bug #23205 (Resolved): Blocked requests no longer show details
In Luminous, blocked requests no longer show details, i.e. OSD id or hints as to the operation:
018-03-01 21:52:54...
Alex Gorbachev
03:43 AM Bug #23118 (Resolved): Balancer maps PG to OSDs on the same host with upmap
Nathan Cutler
01:56 AM Bug #23118: Balancer maps PG to OSDs on the same host with upmap
https://github.com/ceph/ceph/pull/20653 xie xingguo
 

Also available in: Atom