Ceph

h3. Hadoop over Ceph RGW status update

*Summary*

The goal is to write a Hadoop Compatible Filesystem(RGWFS) to allow Hadoop run over RGW. We also want to add load balancing feature to allow this scale to some rack rachitecture

*Owners*

Yuan Zhou(Intel)

Jian Zhang(Intel)

*Interested Parties*

If you are interested in contributing to this blueprint, or want to be a "speaker" during the Summit session, list your name here.

Name (Affiliation)

Name (Affiliation)

Name

*Current Status*

In Infernalis we proposed this BP(Hadoop over Ceph Radosgw with SSD cache). During the last several months, we've got some progress.

* RGWFS

Thanks to SwiftFS, RGWFS is able to reuse lots of code. Currently the general code path is done. We're able to read/write with Hadoop command line tool through RGWFS, which talks to the backend Rados cluster.

* RGW-Proxy

We have implented a simple WSGI server that can give out the nearest RGW instance by looking through the internal data mapping in the Rados cluster. By giving the object name, RGW-Proxy would query in the cluster to check the mapping of data(ceph osd map obj_name), and then give out corresponding RGW instance

*Detailed Description*

There're a few things we're working on. 

* Make RGWFS work with multiple RGW instance

* Performance testing

*Work items*

This section should contain a list of work tasks created by this blueprint.  Please include engineering tasks as well as related build/release and documentation work.  If this blueprint requires cleanup of deprecated features, please list those tasks as well.

*Coding tasks*

Task 1

Task 2

Task 3

*Build / release tasks*

Task 1

Task 2

Task 3

*Documentation tasks*

Task 1

Task 2

Task 3

*Deprecation tasks*

Task 1

Task 2

Task 3