rgw: make bucket removal "atomic"
Right now, bucket removal consists of two steps:
1) Remove the bucket object (making sure the bucket index doesn't list any objects),
2) remove the bucket from the user index.
If the daemon doing the removal fails between those two steps, nothing cleans it up appropriately and the tools don't really handle it right. The amount of damage it can cause is pretty limited and it's easy to clean up, but it's not a scenario we want to run into or have to deal with. We should instead do deletion in a way that subsequent readers can clean up. eg:
1) mark bucket as to-delete in user index
2) delete bucket index object
3) remove bucket from user index object
Then if it fails partway through, subsequent readers of the user bucket index will see it marked that way and can re-do the deletion steps.