Ceph

QA & document this feature -- all the necessary code might already be there.

From the mailing list:

2011/8/16 Maciej Ga?kiewicz <maciejgalkiewicz@ragnarson.com>:
> I have a storage cluster based on glusterfs. Each client have its own
> volume and his is not able to mount any other. Is it possible to
> implement such separation with ceph? I have read about authentication
> and mounting subdirectories. I am not sure if I can configure
> different login/pass for selected directories stored in ceph. All
> clients are running on Xen's domU (virtual machine). Maybe there is
> some other way to achieve this?

This seems to be possible with Ceph as it is now, but it is definitely
not the normal setup. As in, test carefully and understand you're
going off the beaten path.

To avoid confusion with the word "client", I'll call these
mutually-untrusting domains "customers". Each customer can have
multiple clients.

It builds up something like this:

1. Each customer naturally needs separate keys (really, every client
should have a separate key).
2. Because clients talk to OSDs directly, you need to have different
trust domains use different pools (otherwise they can just read/write
each other's raw objects); use "cauthtool --cap osd 'allow rwx
pool=something'" on the client/customer key to specify who can write
where.
3. You probably want the root of your ceph filesystem stored in
pool=data, but give most clients just read-only access to it.
4. Use "cephfs /your/ceph/mountpoint/customerA set_location --pool
customerA" to tell the MDS what pool subtrees of your ceph filesystem
are stored in.
5. Tell clients to mount their part of the filesystem directly, use
your-ceph-mon-here:/customerA as the mount device.

Disclaimer: not tested, not security audited, customers can still DoS
each other etc nasty things, your mileage may vary, if you break it we
will probably help you fix at least one of the halves, warning this
sign has sharp edges.