Ceph : Issueshttps://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2020-10-01T10:03:53ZCeph
Redmine Dashboard - Bug #47714 (New): mgr/dashboard: Implement an expert settinghttps://tracker.ceph.com/issues/477142020-10-01T10:03:53ZStephan Müller
<p>To simplify forms implement an expert setting that if disabled hides not mandatory fields in the forms as the first step.</p>
<p>How should it look like?<br />Maybe on the top right an expert slider on the form and on the top panel. As it will have impact on what a users sees in the future.</p>
<p>Tip before getting to deep into the implementation please ask in the stand up if that's path we want to go down.</p> Dashboard - Bug #46757 (New): mgr/dashboard: Only show identify action if inventory device can blinkhttps://tracker.ceph.com/issues/467572020-07-29T14:03:08ZStephan Müller
<p>If a device can't be blink but is manged by cephadm the action "Identify" will be shown in the inventory page. The problem is that the command doesn't throw an error if it fails on the dashboard. I observed the following error through running `ceph -W cephadm` in parallel to the execution.</p>
<pre>
2020-07-29T08:53:30.649950-0500 mgr.x [ERR] executing blink(([DeviceLightLoc(host='osd0', dev='/dev/vdb', path='/dev/vdb')],)) failed.
Traceback (most recent call last):
File "/ceph/src/pybind/mgr/cephadm/utils.py", line 67, in do_work
return f(*arg)
File "/ceph/src/pybind/mgr/cephadm/module.py", line 1591, in blink
raise OrchestratorError(
orchestrator._interface.OrchestratorError: Unable to affect ident light for osd0:/dev/vdb. Command: lsmcli local-disk-ident-led-on --path /dev/vdb
2020-07-29T08:53:30.653157-0500 mgr.x [ERR] _Promise failed
Traceback (most recent call last):
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 292, in _finalize
next_result = self._on_complete(self._value)
File "/ceph/src/pybind/mgr/cephadm/module.py", line 102, in <lambda>
return CephadmCompletion(on_complete=lambda _: f(*args, **kwargs))
File "/ceph/src/pybind/mgr/cephadm/module.py", line 1599, in blink_device_light
return blink(locs)
File "/ceph/src/pybind/mgr/cephadm/utils.py", line 73, in forall_hosts_wrapper
return CephadmOrchestrator.instance._worker_pool.map(do_work, vals)
File "/usr/lib64/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib64/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
File "/usr/lib64/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/lib64/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/ceph/src/pybind/mgr/cephadm/utils.py", line 67, in do_work
return f(*arg)
File "/ceph/src/pybind/mgr/cephadm/module.py", line 1591, in blink
raise OrchestratorError(
orchestrator._interface.OrchestratorError: Unable to affect ident light for osd0:/dev/vdb. Command: lsmcli local-disk-ident-led-on --path /dev/vdb
</pre> Orchestrator - Tasks #46551 (Resolved): cephadm: Add better a better hint how to add a hosthttps://tracker.ceph.com/issues/465512020-07-15T14:14:32ZStephan Müller
<p>Currently:</p>
<pre>
master:~ # ceph orch host add mgr0 192.168.121.230
Error ENOENT: Failed to connect to mgr0 (192.168.121.230).
Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to run:
> ceph cephadm get-ssh-config > ssh_config
> ceph config-key get mgr/cephadm/ssh_identity_key > key
> ssh -F ssh_config -i key root@mgr0
</pre>
<p>What actually needs to be done:<br /><pre>
master:~ # ceph config-key get mgr/cephadm/ssh_identity_pub > key.pub
master:~ # ssh-copy-id -i "key.pub" root@mgr0
</pre></p>
<p>What the message should look like in the end:<br /><pre>
master:~ # ceph orch host add mgr0 192.168.121.230
Error ENOENT: Failed to connect to mgr0 (192.168.121.230).
Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to add the SSH key to the host:
> ceph config-key get mgr/cephadm/ssh_identity_pub > ~/cephadm_ssh_key.pub
> ssh-copy-id -i ~/cephadm_ssh_key.pub root@mgr0
you may want to check that everything works, before rerunning the command:
> ceph cephadm get-ssh-config > ssh_config
> ceph config-key get mgr/cephadm/ssh_identity_key > ~/cephadm_ssh_key
> ssh -F ssh_config -i ~/cephadm_ssh_key root@mgr0
</pre></p> Orchestrator - Support #46547 (Resolved): cephadm: Exception adding host via FQDN if host was alr...https://tracker.ceph.com/issues/465472020-07-15T12:17:02ZStephan Müller
<p>To reproduce you need nodes that have a subdomain (not like in current Vagrantfile). I used sesdev to find this issue.</p>
<pre>
master:~ # ceph orch host add node1.pacific.test
Error ENOENT: New host node1.pacific.test (node1.pacific.test) failed check: [
'INFO:cephadm:podman|docker (/usr/bin/podman) is present',
'INFO:cephadm:systemctl is present', 'INFO:cephadm:lvcreate is present',
'INFO:cephadm:Unit chronyd.service is enabled and running',
'INFO:cephadm:Hostname "node1.pacific.test" matches what is expected.',
'ERROR: hostname "node1" does not match expected hostname "node1.pacific.test"'
]
</pre>
<p>With `ceph -W cephadm` one observes</p>
<pre>
2020-07-15T13:24:21.159126+0200 mgr.node1.zybwkb [ERR] _Promise failed
Traceback (most recent call last):
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 277, in _finalize
next_result = self._on_complete(self._value)
File "/usr/share/ceph/mgr/cephadm/module.py", line 132, in <lambda>
return CephadmCompletion(on_complete=lambda _: f(*args, **kwargs))
File "/usr/share/ceph/mgr/cephadm/module.py", line 1098, in add_host
return self._add_host(spec)
File "/usr/share/ceph/mgr/cephadm/module.py", line 1087, in _add_host
spec.hostname, spec.addr, err))
orchestrator._interface.OrchestratorError: New host node1.pacific.test (node1.pacific.test) failed check: ['INFO:cephadm:podman|docker (/usr/bin/podman) is present', 'INFO:cephadm:systemctl is present', 'INFO:cephadm:lvcreate is present', 'INFO:cephadm:Unit chronyd.service is enabled and running', 'INFO:cephadm:Hostname "node1.pacific.test" matches what is expected.', 'ERROR: hostname "node1" does not match expected hostname "node1.pacific.test"']
</pre> Orchestrator - Documentation #46377 (Resolved): cephadm: Missing 'service_id' in last example in ...https://tracker.ceph.com/issues/463772020-07-06T15:33:59ZStephan Müller
<p>Missing 'service_id' in last example in orchestrator#service-specification. Example can be found right above <a class="external" href="https://docs.ceph.com/docs/master/mgr/orchestrator/#placement-specification">https://docs.ceph.com/docs/master/mgr/orchestrator/#placement-specification</a> and it should look like specified in <a class="external" href="https://docs.ceph.com/docs/master/cephadm/drivegroups/#osd-service-specification">https://docs.ceph.com/docs/master/cephadm/drivegroups/#osd-service-specification</a> .</p> Orchestrator - Tasks #46376 (Resolved): cephadm: Make vagrant usage more comfortablehttps://tracker.ceph.com/issues/463762020-07-06T15:28:51ZStephan Müller
<p>Currently you can only use a big scale factor using the vagrant setup. You can have x * (mgr, mon, osd with 2 disks). It would be nicer to use the same constants as vstart is using to select how many mgr, mons and osds one likes to have. I would go further and add a disks constant two.</p>
<p>This would make the creation a lot more flexible. Another thing that is missing is an script to easily snapshot the created vm's and recreate them</p> Orchestrator - Bug #45724 (Resolved): check-host should not fail using fqdn or not that hardhttps://tracker.ceph.com/issues/457242020-05-27T09:26:51ZStephan Müller
<p>I would suggest either identify that it's an FQDN or answer "Host not found. Use 'ceph orch host ls' to see all managed hosts."</p>
<pre>
# ceph cephadm check-host node3.ses7.com
Error EINVAL: Traceback (most recent call last):
File "/usr/share/ceph/mgr/mgr_module.py", line 1153, in _handle_command
return self.handle_command(inbuf, cmd)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 110, in handle_command
return dispatch[cmd['prefix']].call(self, cmd, inbuf)
File "/usr/share/ceph/mgr/mgr_module.py", line 308, in call
return self.func(mgr, **kwargs)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 72, in <lambda>
wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 63, in wrapper
return func(*args, **kwargs)
File "/usr/share/ceph/mgr/cephadm/module.py", line 1482, in check_host
error_ok=True, no_fsid=True)
File "/usr/share/ceph/mgr/cephadm/module.py", line 1569, in _run_cephadm
conn, connr = self._get_connection(addr)
File "/usr/share/ceph/mgr/cephadm/module.py", line 1521, in _get_connection
n = self.ssh_user + '@' + host
TypeError: must be str, not NoneType
</pre> Dashboard - Feature #45306 (New): mgr/dashboard: asynchronous back-end: Use HTTP2 or websocketshttps://tracker.ceph.com/issues/453062020-04-28T13:11:52ZStephan Müller
<p>In order to determine what we want to use in future. I will compare both HTTP2 and websockets.</p>
<p>First a bunch of information.</p>
<p>Currently we use the protocol HTTP1.1, which only allows one request per connection.</p>
<p>With HTTP2 and websockets it is possible to allow an unlimited amount of request per connection.</p>
<p>What does one request per connection mean? For example a client asks the server for a file, this will open a connection telling the server GET me something, the server will respond and close the connection. As our dashboard does not only consist of one file, a lot of connections are made. To meet the demand of any modern site of so many connections all modern browsers will do 8 connections simultaneously. On every connection also the same header is send.</p>
<p>What does unlimited amount of requests per connection mean? For example a client asks for a (whole) website. The client sends the first request like in HTTP1.1, the server responds with an HTTP1.1 Upgrade header, client and server negotiate which protocol to use (handshake). A connection is established and left open for requests. The client sends requests for multiple files while the server already responds with the files. This maxes out the established connection, as both participants can send at the same time (for example a video chat). As the connection is left open the server can PUSH data to the client even if he had not explicitly asked for (removes polling). To save data, only the headers during the handshake are send, they will not be send multiple times.</p>
<p>Whats the difference between HTTP2 (released as standard 2015) and websockets (released as standard 2011)?<br />Both only need one connection. Websockets can run insecure using port 80 and both can run secure using port 443. Websockets use a different URL prefix <strong>ws://</strong> for insecure connections or <strong>wss://</strong> for secure ones, HTTP2 uses only <strong>https://</strong> as prefix. If HTTP2 is used data will automatically be compressed and the handshake is easier to implement than with websockets.</p>
<p>Sure HTTP2 is the better one as the protocol is much newer, but can we use it with cherrypy?<br />Currently I only found a <a href="https://docs.cherrypy.org/en/latest/advanced.html#websocket-support" class="external">plugin</a> for cherrypy to allow websockets.<br />I've not found one for HTTP2 yet but I'm still collecting information.</p> Dashboard - Bug #44753 (New): mgr/dashboard: Secure the Alertmanger receiver endpointhttps://tracker.ceph.com/issues/447532020-03-25T14:10:42ZStephan Müller
<p>Currently it is possible send push notification unauthenticated to the dashboard and the push notifications are not verified if they actually are coming from an Alertmanager instance.</p>
<p>To see whats configurable see <a class="external" href="https://prometheus.io/docs/alerting/configuration/#http_config">https://prometheus.io/docs/alerting/configuration/#http_config</a></p>
<p>Removing the endpoint is not a solution to be considered as ceph orchestrator is configuring every Alertmanager instance to talk to the receiver of the dashboard.</p>
<p>The receiver is at the moment the only part that can handle multiple Altermanger instances.</p> Dashboard - Feature #44621 (Pending Backport): mgr/dashboard: Automatic preselection of failure d...https://tracker.ceph.com/issues/446212020-03-16T12:12:19ZStephan Müller
<p>Use the automatic preselection of the crush rule creation form inside the erasure code profile form to prevent wrong configured ec profiles which can't be used in the end.</p> Dashboard - Bug #44617 (New): mgr/dashboard: Some notifications are not shown in the notification...https://tracker.ceph.com/issues/446172020-03-16T10:02:31ZStephan Müller
<p>Some notifications are only popping up, but are not stored in the notifications table.<br />Observed the behavior for the erasure code profile creation and deletion as well for the crush rule creation and deletion.<br />It's possible that this affects also other pages.</p> Dashboard - Bug #44224 (New): mgr/dashboard: Timeouts for rbd.py callshttps://tracker.ceph.com/issues/442242020-02-20T10:10:30ZStephan Müller
<p>As the corner cases are not implemented in many rbd.py methods, they can fail without a response on a specific pool (mostly bad pools).</p>
<p>If this is implemented remove the workaround that was implemented to fix <a class="issue tracker-1 status-3 priority-4 priority-default closed" title="Bug: mgr/dashboard: Dashboard breaks on the selection of a bad pool (Resolved)" href="https://tracker.ceph.com/issues/43765">#43765</a>.</p>
<p>For details what known issue exists see <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: pybind/rbd: config_list hangs if given an pool with a bad pg state (Rejected)" href="https://tracker.ceph.com/issues/43771">#43771</a>.</p>
<p>For details about the discussion that was made look at the PR that fixed <a class="issue tracker-1 status-3 priority-4 priority-default closed" title="Bug: mgr/dashboard: Dashboard breaks on the selection of a bad pool (Resolved)" href="https://tracker.ceph.com/issues/43765">#43765</a>.</p>
<p>Make sure that <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: pybind/rbd: config_list hangs if given an pool with a bad pg state (Rejected)" href="https://tracker.ceph.com/issues/43771">#43771</a> is still not addressed before starting with this issue.</p>
<p>For details how this was implemented in openATTIC look <a href="https://bitbucket.org/openattic/openattic/pull-requests/682/add-librados-command-name-to-external/diff" class="external">here</a></p> Dashboard - Feature #43930 (New): mgr/dashboard: Make user creation with password change on logon...https://tracker.ceph.com/issues/439302020-01-31T11:37:12ZStephan Müller
<p>After <a class="issue tracker-2 status-5 priority-6 priority-high2 closed child" title="Feature: mgr/dashboard: Enforce password change upon first login (Closed)" href="https://tracker.ceph.com/issues/24655">#24655</a> has been resolved, I can create a new user without a password an check "User must change password at next logon".</p>
<p>But I can't click the logon button without typing a password, but the user has no password as it should be set on next logon.</p>
<p>There are 3 ways to solve this:<br />1. Enable logon without a password<br />2. Enable passwords without password rule if the user has to change his PW anyway on his first logon according to the rules.<br />3. Button to automatically generate a password that complies with the rules and copy into to the clipboard.</p> Dashboard - Feature #43351 (New): mgr/dashboard: [RFC] Actions assistanthttps://tracker.ceph.com/issues/433512019-12-17T10:34:10ZStephan Müller
<p>Not sure if this is needed for all pages but it could help users.</p>
<p>I just looked at the OSD page which is crowed by actions.</p>
<p>There are two ways to implement it,<br />as an modal that triggers the action modal,<br />or as an modal that describes what to do in order to trigger the action.</p>
<p>The second approach could be implemented globally and not only for a specific page as it could search through every page actions that are available.</p>
<p>How the modal should look like?<br />It should be pretty straight forward like an FAQ search.<br />If you open it you will see a big input field to type in words that describe what you want to do.</p>
<p>The string will be used to calculate a score for each available description. Than the highest ranked actions (3, 5 or 10?) will be shown, sorted by rank.</p>
<p>The action will shown as accordion showing the description of it if expanded and the button to take the action or the help text to get to the page and action.</p>
<p>As said in the beginning, I'm not sure if we need this.</p> Dashboard - Documentation #42350 (New): mgr/dashboard: The reason to testhttps://tracker.ceph.com/issues/423502019-10-17T12:33:33ZStephan Müller
<p>Give an interesting introduction into our tests.</p>
<p>This task should be split in the following subtasks</p>
<p>1. Describe the benefits of the different tests.</p>
<p>2. Describe consideration that are to be made to decide where I use the one or the other.</p>
<p>3. Show the benefits of different testing strategies (Test first / TDD)</p>
<p>4. Add some best practice links and showcase good tests in our code base.</p>
<p>The benefit of this should be that everybody should be capable of writing tests with the different test scopes in mind.</p>