Ceph

h1. Rgw - Snapshots

h3. Summary

*NOTE: The approach on this blueprint is outdated and needs to be revisited for Hammer development.*

RADOS supports snapshots, but RGW doesn't make this visible.  When using RADOS snapshots to implement RGW snapshots, it is very time consuming to manually reconstruct a RGW object from RADOS objects.  It should be easy to create/remove snapshots via the radosgw-admin tool and the REST API.  It should be easy to retrieve a list of snapshots.  It should be easy to retrieve an object from a particular snapshot.

h3. Owners

* Craig Lewis (Central Desktop) <clewis@centraldesktop.com> 

h3. Interested Parties

h3. Current Status

Not currently accepted.  I am planning to do this work myself, but I'm willing to work with anybody that is interested.

I have hacked up a demo that uses a hardcoded snapshot name.  It works remarkably well.  I believe that rados provides all the necessary functionality, and it's just a matter of letting radosgw use snapshots.

h3. Detailed Description

h3. Use Case

These snapshots are intended to be RGW wide, not per bucket.  They are intended to be controlled by the RGW administrator, not the user.  The user should be able to make use of them.

The primary goal is to implement RGW snapshots for backup purposes.  Mainly to provide insurance against end-user or administrative error.  Coupled with Federation, this will provide offsite backups.  Since snapshots are "online", this is not protection against malicious intent.  Malicious intent could be managed by administrator partitioning (ie, no one user has system access to all zones).

h3. Code changes

My demo hack:

<pre>

01| int RGWRados::open_bucket_pool_ctx(const string& bucket_name, const string& pool, librados::IoCtx&  io_ctx)

02| {

03|   int r = rados->ioctx_create(pool.c_str(), io_ctx);

04| -  if (r != -ENOENT)

05| +  if (r != -ENOENT) {

06| +    rados_snap_t snapid;

07| +    r = rados->ioctx_snap_lookup( "snap3", &snapid, io_ctx);

08| +    if (r < 0)

09| +      return r;

10| +

11| +    rados->ioctx_snap_set_read( snapid, io_ctx);

12| +

13|     return r;

14| +  }

</pre>

That's the heart of the change.  As a bonus, this change already prevents all write operations, and only allows read operations.  The rest of the work is parsing the GET parameters or HTTP headers, and passing the values around.

The changes to radosgw-admin and the Admin REST API are relatively straight forward.  These changes are slightly complicated by making sure that it works correctly with regions and zones.  These changes aren't technically required.  They're syntactic sugar to make managing the snapshots easier.  Everything can already be accomplished using the rados command line tool. 

h3. Federation

I am not planning on making snapshots replicate with radosgw-admin.  For my use case, I would prefer to have different snapshot policies on the master and slave zones.  It's a nice bonus that avoiding replication makes the change simpler. 

h3. Additional Tools

As part of this project, I intend to port a ZFS snapshot management tool to manage RGW snapshots.  I have not decided which tool yet.  The ZFS tool I currently use (zfs-snapshot-mgmt?) is written in Ruby.  I would prefer to fork a tool that is written in python, for consistency.  If I can't find a suitable python project, I'll fork zfs-snapshot-mgmt rather than implement from scratch.

It would be nice if this tool can manage both rados and radosgw snapshots, but I'm going to focus on radosgw.

h3. Things to discuss

h4. Naming Conventions

When extending radosgw-admin, I'm planning to add @radosgw-admin (lssnap|mksnap|rmsnap)@ commands.  I would prefer to use a sub-command, ie @radosgw-admin snap (ls|mk|rm)@, but I think it's more important to be consistent with the existing rados commands.

I'm open to discussion on this.

h4. Extending the S3 API

I would like some input on how best to add optional snapshot support to the S3 API.  I'm unsure of the best way to add Ceph only features.

I was thinking of using an X-Snapshot header.  It's a bit cumbersome though, and would require the user to write code to use this feature.  I think I prefer a URL parameter, eg http://bucket.ceph.com/object?snapshot=snapshot.  This would be easier to use, and  should work with s3cmd out of the box.

Mostly I'm looking for advice on the best way to add Ceph only extensions to the S3 API.

h3. Work items

h4. Coding tasks

# REST API, GET with snapshot argument

# REST API, GET (/|bucket|object) returning a list of snapshots

# REST API, verify all modifying operations fail with snapshot argument

# @radosgw-admin lssnap@

# @radosgw-admin mksnap@

# @radosgw-admin rmsnap@  

# Investigate implementation of @radosgw-admin rollback <object>@.  I suspect that implementing this will require more code that I want to write at this point.  Verify or implement.

# Add optional --snap argument support to radosgw-admin,  Add support in reading sub-commands, like user show, bucket list, region show, etc.  Make sure writing sub-commands fail with an error when called with --snap.

# Admin REST API, List Snapshots

# Admin REST API, Create Snapshot

# Admin REST API, Remove Snapshot

# Optional: Admin REST API, Rollback Snapshot <object>

# Find suitable ZFS snapshot manager, and port to manage RGW snapshots

h4. Documentation tasks

# Update radosgw-admin documentation for snapshots, snapshot create/remove

# Update Admin REST API for List/Create/Remote snapshot

# Update radosgw GET documentation to include the optional snapshot parameter.

h4. Deprecation tasks

None that I'm aware of.