Ceph : Issueshttps://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2022-03-16T10:59:51ZCeph
Redmine RADOS - Feature #54580 (New): common/options: add FLAG_SECURE to Ceph optionshttps://tracker.ceph.com/issues/545802022-03-16T10:59:51ZErnesto Puerta
<a name="Context"></a>
<h3 >Context<a href="#Context" class="wiki-anchor">¶</a></h3>
<p>It has been reported by several users that <code>ceph config dump</code> and <code>ceph config-key dump</code> may expose sensitive information (Grafana admin password, RGW API keys, RBD mirror secrets, etc.). These users would prefer to have a dump that omits or redacts the secrets from that output (in a similar way as password inputs mask the remembered passwords).</p>
<pre>
# ceph config-key dump
...
"config-history/xx/+mgr/mgr/dashboard/RGW_API_ACCESS_KEY": "12345679ABCDEF",
...
</pre>
<a name="Implementation-proposal"></a>
<h3 >Implementation proposal<a href="#Implementation-proposal" class="wiki-anchor">¶</a></h3>
<p>Ceph options support <a href="https://github.com/ceph/ceph/blob/cac29ca70a76ddf7a04ee12aaf7ed2012cd12f22/src/common/options.h#L119-L127" class="external">flags</a>, so a new flag (e.g.: <code>FLAG_SECRET</code>) could be used to hint that a given option contains sensitive information and it therefore shouldn't be reproduced by default.</p>
<p>In order to still allow users to perform a complete dump of settings, a new command option should be specified (e.g.: @ceph config-key dump --include-secrets).</p>
<p><strong>Before:</strong><br /><pre>
# ceph config-key dump
...
"config-history/xx/+mgr/mgr/dashboard/RGW_API_ACCESS_KEY": "*****************",
...
</pre></p>
<p><strong>After:</strong><br /><pre>
# ceph config-key dump --secrets
...
"config-history/xx/+mgr/mgr/dashboard/RGW_API_ACCESS_KEY": "12345679ABCDEF",
...
</pre></p>
<h3>References
<ul>
<li><a class="external" href="https://tracker.ceph.com/issues/37503">https://tracker.ceph.com/issues/37503</a></li>
<li><a href="https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/" class="external">Kubernetes secrets</a></li>
</ul></h3> Dashboard - Cleanup #50963 (Resolved): mgr/dashboard: stop polling when page is not visiblehttps://tracker.ceph.com/issues/509632021-05-24T18:47:23ZErnesto Puerta
<p>Modern browsers provide support to detect whether a page is visible by the user or not (<a class="external" href="https://www.w3.org/TR/page-visibility/">https://www.w3.org/TR/page-visibility/</a>).</p>
<p>We should rely on that to only poll the back-end when the browser tab is visible. That way we'd heavily reduce the load generated from the front-end. That might have an impact, for example, if a user has many Dashboard tabs opened, which would multiply the number of HTTP requests to the back-end.</p> Dashboard - Bug #47857 (Won't Fix): mgr/dashboard: sensitive information stored in cleartexthttps://tracker.ceph.com/issues/478572020-10-14T11:06:05ZErnesto Puerta
<p>Description</p>
<p>The application stores sensitive information (i.e. usernames, passwords and access tokens) cleartext inside<br />a RocksDB database. An attacker with read access to the database files could compromise the application<br />and other systems.</p>
<p>Exploitation</p>
<p>The testing team identified a RocksDB instance in use by the Ceph Monitor daemon that contains sensitive<br />data, such as S3 access and secret keys, usernames and passwords.<br />It was possible to easily obtain the data either by searching for ASCII strings in the database files or by<br />using RocksDB command line tool to dump the database.</p>
<p>Recommendation</p>
<p>Either encrypt whole RocksDB or perform application-level encryption.</p>
<p>Caveat</p>
Application level encryption still requires an encryption key to saved somewhere, which simply shifts the problem to where to securely store this key:
<ul>
<li>Key-Value Store... un-encrypted. Same as original issue.</li>
<li>Hardware Security Module (HSM), like a FIPS-140</li>
<li>Remote key server (e.g.: Vault)</li>
</ul> Dashboard - Bug #47356 (Resolved): mgr/dashboard: some nfs-ganesha endpoints are not in correct s...https://tracker.ceph.com/issues/473562020-09-08T08:11:20ZKiefer Chang
<p>The <a href="https://github.com/ceph/ceph/blob/056d8776ce1133928f4473a2f63e4537cecec5f2/src/pybind/mgr/dashboard/controllers/nfsganesha.py#L271-L311" class="external">endpoints</a> in the <code>/ui-api/ganesha-nfs</code> and <code>/api/ganesha-nfs/daemons</code> paths need to be scoped.</p> Dashboard - Cleanup #47341 (Resolved): mgr/dashboard: securing CherryPyhttps://tracker.ceph.com/issues/473412020-09-07T16:44:51ZErnesto Puerta
<p>Ensuring we follow, as much as possible, <a href="https://docs.cherrypy.org/en/3.3.0/progguide/security.html" class="external">Cherrypy security guidelines</a></p>
<ul>
<li>Transmitting data:
<ul>
<li>Use Secure Cookies</li>
</ul>
</li>
<li>Rendering pages:
<ul>
<li>Set HttpOnly cookies</li>
<li>Set XFrame options</li>
<li>Enable XSS Protection</li>
<li>Set the Content Security Policy</li>
</ul></li>
</ul> Dashboard - Feature #45372 (New): mgr/dashboard: monitoring/grafana: any user can run any query o...https://tracker.ceph.com/issues/453722020-05-04T11:34:20ZPatrick Seidensal
<p>Any user with the viewer role can send any query to the Prometheus data source through Grafana.</p>
<p>According to Prometheus' security model, neither the data which could be accessed nor the access to Prometheus itself is considered an issue, though malicious actions might be possible and enhanced security might be a requirement in some cases.</p>
<blockquote>
<p>It is presumed that untrusted users have access to the Prometheus HTTP endpoint and logs.<br />-- <a class="external" href="https://prometheus.io/docs/operating/security/#prometheus">https://prometheus.io/docs/operating/security/#prometheus</a></p>
</blockquote>
<p>By default though, this access is limited to read-only operations (provided the admin API isn't enabled, which isn't the case in our default deployment). Measures to mitigate against Denial of Service attacks are built in Prometheus itself, though it can not be guaranteed to prevent all attacks. Secrets are not exposed.</p>
<p>Grafana enables users of its enterprise version to control which queries can be send to Prometheus, the community version does not include such options.</p>
<p>-- <a class="external" href="https://grafana.com/docs/grafana/latest/installation/security/#limit-viewer-query-permissions">https://grafana.com/docs/grafana/latest/installation/security/#limit-viewer-query-permissions</a></p>
<p>A possible solution would be to implement a reverse proxy in Ceph Dashboard, which will be capable of filtering queries before they are relayed to Grafana. This solution should not replace the current integration with Grafana, where Grafana is enabled to allow anonymous access, but offered as additional option.</p>
<p>By implementing such a proxy, the benefit of being able to access Grafana directly and outside of Ceph Dashboard will be lost, as Grafana will need to be configured to run behind a reverse proxy.</p>
<p>By implementing this solution, we will need to specify which operations can be performed by a Ceph Dashboard user (by its group). This involves continues maintenance.</p>
<p>Please also note that Prometheus' HTTP API in our current deployments is not protected from being accessed and that restrictions to access Prometheus' API should probably be resolved before this issue needs to be fixed.</p> Dashboard - Bug #44591 (Resolved): CVE-2020-27839: mgr/dashboard: The ceph dashboard is vulnerabl...https://tracker.ceph.com/issues/445912020-03-13T07:49:08ZAnonymous
<p>To authenticate the user with the ceph dashboard the user can exchange a username and password for a jwt token. This token will be stored inside the users browser. On every request, the ceph dashboard attaches this token to the 'Authorization' header to get access to the api.<br />The problem is how we store this token. Currently, we just save it inside the local storage of the browser which makes it vulnerable to XSS attacks. If any of our npm dependencies is compromised, an attacker could steal the token of a user and use it. I also tried to inject malicious code into the translation files, we host on transifex. I did not manage to attack the dashboard with that approach, because the angular i18n compiler rejected all script tags, but that does not mean that it is not possible. <br />Keeping the npm packages up to date is very important, but doesn't fix the problem either. Not anyone, who is running a ceph cluster, is going to update it on a regular basis (every week or two).</p>
Currently I see three solutions:
<ul>
<li>Critical actions should always ask for user credentials<br />What are critical actions? From my point of view, at least all user management actions. Changing the role of a user or en- and disabling an account of another user should not be possible without entering the password of the current user.</li>
</ul>
<ul>
<li>2FA<br />Using two factor authentication will mitigate attacks. At least the attacker cannot request a token with just the username and password. The XSS problem would still exist.</li>
</ul>
<ul>
<li>Double submitted cookies<br />Storing the jwt token in a httpOnly cookie prevents XSS attacks, but opens the door for csrf attacks. With sending two coockies, one containing the jwt header and payload (httpOnly disabled), and one containing the signature, could be a solution. JavaScript can not read the content of httpOnly cookies, but the content of cookies without httpOnly. So, our frontend could read the header.payload and sends it with every request, inside a meta tag. The backend could then reconstruct the token.</li>
</ul>
<p>Please see: <a class="external" href="https://medium.com/lightrail/getting-token-authentication-right-in-a-stateless-single-page-application-57d0c6474e3">https://medium.com/lightrail/getting-token-authentication-right-in-a-stateless-single-page-application-57d0c6474e3</a> for more details</p> Dashboard - Bug #44237 (Resolved): mgr/dashboard: security: some system roles allow accessing sen...https://tracker.ceph.com/issues/442372020-02-21T12:11:04ZErnesto Puerta
<p>Some system roles (<code>pool-manager</code>, <code>cephfs-manager</code>, <code>ganesha-manager</code> etc) have the <code>configOpt</code> read permissions enabled, which allows to read all cluster config options and manager module config options. The latter includes RGW keys or Grafana user/admin, plus any sensitive information used by existing or new modules. As dashboard cannot control what new information is exposed by these modules, the suggestion is to remove that read permission from all system roles except the specific management ones (<code>adminstrator</code> and <code>cluster-manager</code>).</p>
The reason why configOpts was added to those roles is that at some point they require access to some cluster configuration settings:
<ul>
<li><b><code>pool-manager</code></b>: checks <code>/api/cluster_conf/osd_pool_default_pg_autoscale_mode</code>. This parameter could/should also be exposed via <code>/api/pools/_info</code>, which already returns other ceph config params (e.g.: <code>bluestore_compression_algorithm</code>).</li>
<li><b><code>ganesha-manager</code>, <code>cephfs-manager</code>, <code>rgw-manager</code></b>: I couldn't find any direct dependency with cluster config options.</li>
</ul>
<p>different case is the <code>read-only</code> role. While it initially makes sense to allow <code>configOpt</code> read permission, dashboard administrator might guess that <code>read-only</code> perfectly fits for a <code>guest</code>/low-privileged user. On the contrary, a <code>read-only</code> user has access to the same sensitive data as mentioned above.</p>
Suggested next steps:
<ul>
<li>Discuss and agree on <code>read-only</code> user with/without access to <code>configOpts</code>. This could improve by splitting it into 2: <code>administrator-read-only</code> and <code>guest</code> (without the read permission on sensitive data). As I'm against adding more roles, I'd simply leave the low-privilege <code>guest</code> one.</li>
<li>Make <code>pool-form</code> get <code>osd_pool_default_pg_autoscale_mode</code> from <code>/pool/_info</code>.</li>
<li>Remove <code>configOpt</code> read perm (and test) in all other roles.</li>
</ul> Dashboard - Bug #43607 (Resolved): mgr/dashboard: fix improper URL checkinghttps://tracker.ceph.com/issues/436072020-01-15T12:59:37ZErnesto Puerta
<p>From <a class="external" href="https://github.com/rook/rook/issues/4635">https://github.com/rook/rook/issues/4635</a></p>
<p>Only release 14.2.5 and above show this behaviour (including master) introduced in <a class="external" href="https://github.com/ceph/ceph/pull/30694">https://github.com/ceph/ceph/pull/30694</a>.</p>
<p>Assigned CVE-2020-1699</p>
<p><a href="https://cwe.mitre.org/data/definitions/22.html" class="external">CWE-22</a></p> Dashboard - Bug #43262 (New): mgr/dashboard: security: upgrade serialize-javascripthttps://tracker.ceph.com/issues/432622019-12-11T16:38:46ZErnesto Puerta
<p><a href="https://github.com/advisories/GHSA-h9rv-jmmf-4pgx" class="external">GHSA-h9rv-jmmf-4pgx</a>. Seems it might have no impact on dashboard, as it's a dependency of webpack, and hence used only at build time.</p> Dashboard - Documentation #42165 (New): mgr/dashboard: Document new password requirements in the ...https://tracker.ceph.com/issues/421652019-10-02T13:45:17ZLenz Grimmer
<p>Starting with Ceph Octopus, the Dashboard supports enforcing some minimum password complexity rules. These limitations should be documented, so users are aware when creating new user accounts. In particular, it should be noted that a password:</p>
<ul>
<li>Must contain at least 8 characters</li>
<li>Cannot contain username</li>
<li>Cannot contain any keyword used in Ceph, e.g. "osd", "host", "dashboard", "pool", "block", "nfs", "ceph", "monitors", "gateway", "logs", "crush", "maps" </li>
<li>Cannot contain any repetitive characters e.g. "aaa" </li>
<li>Cannot contain any sequencial characters e.g. "abc" </li>
<li>Must consist of characters from the following groups:
<ul>
<li>alphabetic a-z, A-Z</li>
<li>numbers 0-9</li>
<li>special chars: !"#$%& \'()*+,-./:;<=>?@[]^_`|~</li>
<li>any other characters (signs)</li>
</ul></li>
</ul> Dashboard - Bug #41990 (Resolved): mgr/dashboard: hide Python tracebacks in response errorshttps://tracker.ceph.com/issues/419902019-09-23T09:57:58ZErnesto Puerta
<p>Currently all errors handled by Cherrypy tools result in Python traceback including sensitive context information (Python version, file locations, packages, etc).</p>
<p>This does not only pose a security risk, but also pollutes logs with traceback lines, which makes it really hard to find where an unexpected traceback happened.</p>
<p>This could be easily fixed by setting Cherrypy environment to <a href="https://docs.cherrypy.org/en/latest/config.html#environments" class="external"><code>production</code></a></p> Dashboard - Bug #41320 (Resolved): mgr/dashboard: passwords and other sensitive information is wr...https://tracker.ceph.com/issues/413202019-08-16T17:56:29ZErnesto Puerta
<p>Currently dashboard is storing in plain text logs the following sentitive information:</p>
<p>- <del>Dashboard user names, passwords and roles.</del> -> handled in <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: Audit log: mgr module passwords set on CLI written as plaintext in log files (Resolved)" href="https://tracker.ceph.com/issues/37503">#37503</a></p>
<blockquote>
<p><del>log_channel(audit) log [DBG] : from='client.4126 -' entity='client.admin' cmd=[{"username": "admin", "rolename": "administrator", "prefix": "dashboard ac-user-create", "password": "admin"}]</del></p>
</blockquote>
<p>- RGW API keys:</p>
<blockquote>
<p>cmd=[{"prefix": "dashboard set-rgw-api-access-key", "target": ["mgr", ""], "value": "<real_key>"}]<br />cmd=[{"prefix": "dashboard set-rgw-api-secret-key", "target": ["mgr", ""], "value": "<real_key>"}]:</p>
</blockquote>
<p>- JWT tokens:</p>
<blockquote>
<p>"JWT Token: <real_token>"</p>
</blockquote>
<p>This information should be redacted from the logs. While access to logs could be limited to privileged users, this is considered insecure (even with hashed passwords).</p> Dashboard - Feature #40329 (Closed): mgr/dashboard: It should be possible to set an expiration da...https://tracker.ceph.com/issues/403292019-06-13T12:29:48ZTiago Melo
<p>As as admin I should be able to set a TTL for the password to expire.<br />It should be a cluster wide configuration.<br />p.e.: every user should change his password every 3 months.</p>
<p>Further questions:</p>
<ul>
<li>admin password expiry: Should it be possible to set an expiry date for the admin password as well? Or only if there is at least another admin account? If it should not be possible to set expiry date prevent the user from doing so.</li>
<li>disabled users password expiry: Should it be possible to set/have an expiry date for disabled users?</li>
<li>'ac_user_create_cmd' requires timestamp as 'pwd_expiry_date': The function (ac_user_create_cmd) to create a user on the command line requires a timestamp as 'pwd_expiry_date' at the moment. Do we want to keep it or change the behavior here?</li>
<li>recalculate password expiry date: issue <a class="external" href="https://tracker.ceph.com/issues/40329">https://tracker.ceph.com/issues/40329</a> introduces a default expiry span (USER_PWD_DEFAULT_EXPIRY_SPAN) for the user passwords and adds a password expiry date field (pwd_expiry_date) to the User class. If the administrator edits the USER_PWD_DEFAULT_EXPIRY_SPAN variable the password expiry dates need to be re-calculated.</li>
<li>update password expiry date (which is set manually): If the 'USER_PWD_DEFAULT_EXPIRY_SPAN' is set and the user changes the password, it's easy to update the expiry date to the next date. But what happens if 'USER_PWD_DEFAULT_EXPIRY_SPAN' is not set and the password expiry date was entered manually?</li>
</ul> Dashboard - Feature #40248 (Closed): mgr/dashboard: As a user, I want to change my passwordhttps://tracker.ceph.com/issues/402482019-06-10T21:41:00ZRicardo Marquesrimarques@suse.com
<p>Ceph Dashboard must have a page where the logged in user can change his own password.</p>