Strong AuthN and AuthZ for CephFS¶

Summary¶

This is a proposal to address two security features:

1. to add a kerberos plugin which can be used in place of cephx, marrying kerberos authentication with ceph capabilities;
2. to address how POSIX ACLs may be both implemented and enforced by all of the client, MDS, and OSD.

Owners¶

• Andrea Ieri, Andreas Peters, Dan van der Ster, Arne Wiebalck (all at CERN)

Interested Parties¶

• Yan, Zheng (Intel)
• Greg Farnum (Inktank)
• Name

Current Status¶

Conceptual phase.

Detailed Description¶

The current security model of Ceph (cephx) authenticates users in similar way as Kerberos does: a client host authenticates to a monitor and acquires a session key which is subsequently used to communicate with the Ceph servers (mon, mds, osd).
This model is not well suited for shared environments like interactive clusters or personal desktop machines/laptops:

• the model relies on the fact that client machines can be trusted;
• the model authenticates the client machine (not the individual user, which would be needed for file system authorization checks like mode bits or ACLs);
• the model does not scale well operationally (key distribution, key management);
• the model cannot be integrated with existing authentication systems, like an Active Directory Domain Controller.

The first proposal is to kerberize Ceph, which can be used in place of cephx:

• users, not client hosts, should acquire per user service tickets for a Ceph service from a central KDC infrastructure outside Ceph;
• users should use the service ticket / session key pair to authenticate to Ceph;
• after having proven his identity, a user’s uid can now be used for any authorization (on the pool or for file system access), using either cephx-like capabilities and/or mode bits or POSIX ACLs.

Due to the similarity between cephx and kerberos, this should be a relatively simple change. Each kerberos user would still need to be known by Ceph (e.g. ceph auth create …) but Ceph would not store a secret for each user. For example, a kerb user could be:

[client.krb.user1@REALM]
key = !krb/user1@REALM
caps mds = "allow" 
caps mon = "allow r" 
caps osd = "allow x, allow rwx pool=data, allow rwx pool=metadata" 

Or another approach would be to assign a default capability for all users in a kerberos realm, e.g.:

[client.krb.*@REALM]
key = !krb/*@REALM
caps mds = "allow" 
caps mon = "allow r" 
caps osd = "allow x, allow rwx pool=data, allow rwx pool=metadata" 

The second aspect of this proposal is for the enforcement of POSIX ACLs by the MDS and OSDs. One security implication of the current CephFS implementation is that while users having the osd rwx pool=data capability may be blocked by ACLs enforcement on their kernel, these users may still bypass the MDS and modify any object in the data pool directly. This may lead to intentional or accidental data corruption, and hence, this mechanism needs adaptation to work with untrusted clients.
Since clients are not trusted, it is not useful to rely on the authorization checking on the client side. This is however currently done for the user/group/world mode bits and is also planned for more sophisticated ACL schemes.
To overcome this, the MDS could hand out time-limited signed leases for extra Ceph capabilities for a given user, pool and object name: as these leases are signed by the MDS with the service key, the client has no way to manipulate these while passing them on to the OSD. The current capabilities in Ceph seem sufficient for POSIX ACLs, so integrating the check of these additional capabilities should be relatively easy to integrate. The basic idea is that we would change each kerberos user’s (or realm default) base capabilities to exclude full rwx access to the data pools, e.g.

[client.krb.*@REALM]
key = !krb/*@REALM
caps mds = "allow" 
caps mon = "allow r" 
caps osd = "allow x" 

And then in the signed lease, the MDS would generate an extra capability which would be temporarily appended to the base capabilities for the duration of a transaction, e.g. add

caps osd = "allow rwx pool=data object_prefix=<CephFS object name>" 

This temporary additional capability would be sent and used only by the relevant OSDs, not appended to the main auth list on the MONs.
In Sage’s thesis, this concept is already mentioned (chapter 3.2.1: “File I/O capabilities”), but it is unclear how far this has gone in the actual CephFS code (at a quick glance it seems it is not there).

Questions:

• Can the shared secret (cephx client keys) be replaced by Kerberos session keys issued for a “Ceph Service” from a centralized standard KDC?
• Can an ACL implementation do the checks server-side to account for untrusted clients?
• Do capabilities need to stay for CephFS pools? If so, how do they interact with ACLs?
• What would be the expected performance impact of a kerberized CephFS?
• Should/could an implementation offer “security flavors” with varied encryption, like “none”, “only MDS”, “only header”, “full” to let users find their compromise between security and performance?

Work items¶

Coding tasks¶

1. Task 1
2. Task 2
3. Task 3

Build / release tasks¶

1. Task 1
2. Task 2
3. Task 3

Documentation tasks¶

1. Task 1
2. Task 2
3. Task 3

Deprecation tasks¶

1. Task 1
2. Task 2
3. Task 3