Ceph

h1. Create and Maintain S3 feature list for compatibility tracking

h3. Summary

Create a detailled page, listing each feature of S3, with the link to the S3 documentation, it's implementation status in Ceph, and links to the code and the tests of this features, would give a better visibility to developpers what's missing, but also to users what's working.

h3. Owners

* Name (Affiliation)

* Name (Affiliation)

* Name

h3. Interested Parties

* Christophe Courtaut <christophe.courtaut@gmail.com>

* Loic Dachary <loic@dachary.org>

* Yehuda Sadeh (Inktank)

* Danny Al-Gaaf

* Name

h3. Current Status

As of now, the documentation page about S3 Api ( http://eu.ceph.com/docs/next/radosgw/s3/ ) is not accurate, and only contains a very brief list of feature, and if it is supported or not.

h3. Detailed Description

The main idea is to maintain a list of S3 Features, with links to the amazon documentation, the status of the ceph implementation, links to the code and the tests that implements it.

A draft page live here :  https://github.com/kri5/ceph/blob/wip-s3-compliance-doc/doc/radosgw/s3_compliance.rst

It is divided in multiple parts, which one reflects the Amazon S3 API documentation : http://docs.aws.amazon.com/AmazonS3/...I/APIRest.html . To track the evolution of the API, a static mirror of the web pages is updated on a regular basis and published as soon as a diff shows. A custom script is likely to be needed to get rid of the page cosmetic elements. Based on the diff, someone would have to copy paste the differences so that they can be acted upon later on.

Each part of the documentation is represented by a table, in which we will have the following columns :

p((. - Header (For HTTP Header documentation only)

p((. - Type (For HTTP Request only) -> GET / POST / PUT / DELETE / HEAD / OPTIONS

p((. - Operation (For HTTP Request only)

p((. - Supported -> Yes / No / Partial / N/A

p((. - Code Links -> Links to the implementation of the feature

p((. - Tracker Links -> Links to the http://tracker.ceph.com/ issue matching the implementation

p((. - Tests Links -> Links to the tests related to the feature

For each Operation ( for instance "List Multipart Uploads":http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html ) the list of parameters should also be listed with a checkbox showing if it is supported or not. A link to a relevant part of the code helps check the actual implementation when it's there and helps implement the missing parameters that should probably be implemented in the same area of the code base.

"Bucket restrictions and limitations":http://docs.aws.amazon.com/AmazonS3/2006-03-01/dev/BucketRestrictions.html and other transversal rules / behavior should be listed and matched against code such as the one stating that a bucket name can "only start with a number, letter and underscore":https://github.com/ceph/ceph/blob/master/src/rgw/rgw_rest_s3.cc#L1906 and highlight that underscore is not allowed by S3 to start a bucket name.

In order to help with the maintenance, the permalink to the table entry matching the S3 feature would have to be inserted as a comment in the code. For instance

  @// https://github.com/kri5/ceph/blob/wi...ons-on-buckets@

Would allow the developer to run the following to quickly find the entry point as well as the history:

@git log --stat --patch -S 's3_compliance.rst#operations-on-buckets'@

A use case of this list could be:

* a casual developer sees the list

* (s)he sees response element KeyMarker of List multipart bucket is missing. Clicks on the link describing it, in the table.

* it looks achievable and (s)he clicks on the link to the area where it should probably be implemented. Gives it a try and submit a pull request.

As mentionned earlier, this is a draft, and the format might evolve. 

Tip: A Very useful plugin for vim to edit rst tables : https://github.com/nvie/vim-rst-tables

h3. Work items

h3. Coding tasks

# script to track the S3 api changes

# Add tests to s3-tests for unsupported features (https://github.com/ceph/s3-tests)

h3. Documentation tasksEdit section

# Create a page which list all S3 Feature

# Check S3 feature implementation status in RGW.

# For each task, link to the implementation code, and the tests.

# Replace  http://eu.ceph.com/docs/next/radosgw/s3/ with the new document

# mark the entry points of each feature in the code