Ceph

h1. Ceph Vagrant Setup

[Reposted from: "Sebastien Han":http://www.sebastien-han.fr/blog/2013/04/22/play-with-ceph-vagrant-box/]

{{toc}}

Materials to start playing with Ceph. This Vagrant box contains a all-in-one Ceph installation.

h3. I. SETUP

First "Download":https://www.vagrantup.com/downloads-archive.html and "Install":http://docs.vagrantup.com/v2/installation/index.html Vagrant.

Download the Ceph box: "here":https://www.dropbox.com/s/hn28qgjn59nud6h/ceph-all-in-one.box. This box contains one virtual machine:

* Ceph VM contains 2 OSDs (1 disk each), 1 MDS, 1 MON, 1 RGW. A modified CRUSH Map, it simply represents a full datacenter and applies a replica per OSD

* VagrantFile for both VM client and ceph

* Other include files

Download an extra VM for the client "here":https://dl.dropboxusercontent.com/u/1537815/precise64.box, note that Debian and Red Hat based system work perfectly, thus it’s up to you:

* Client: just an Ubuntu installation

Initialize the Ceph box:

<pre>

$ wget https://www.dropbox.com/s/hn28qgjn59nud6h/ceph-all-in-one.box

...

...

$ vagrant box add big-ceph ceph-all-in-one.box

[vagrant] Downloading with Vagrant::Downloaders::File...

[vagrant] Copying box to temporary location...

[vagrant] Extracting box...

[vagrant] Verifying box...

[vagrant] Cleaning up downloaded box...

</pre>

Initialize the Client box:

<pre>

$ wget http://dl.dropbox.com/u/1537815/precise64.box

$ vagrant box add ubuntu-12.04.1 precise64.box

[vagrant] Downloading with Vagrant::Downloaders::File...

[vagrant] Copying box to temporary location...

[vagrant] Extracting box...

[vagrant] Verifying box...

[vagrant] Cleaning up downloaded box...

</pre>

Check your boxes:

<pre>

$ vagrant box list

ceph-all-in-one

ubuntu-12.04.1

</pre>

Import all the files from the box:

<pre> 

$ mkdir setup

$ cp /Users/leseb/.vagrant.d/boxes/ceph-all-in-one/include/* setup/

$ mv setup/_Vagrantfile Vagrantfile

</pre>

In order to make the setup easy, I assume that your working directory is $HOME/ceph. At the end, your tree directory looks like this:

<pre>

.

??? Vagrantfile

??? ceph-all-in-one.box

??? precise64.box

??? setup

    ??? ceph.conf

    ??? ceph.sh

    ??? keyring

</pre>

h3. II. START IT!

Check the state of your virtual machines:

<pre>

$ vagrant status

Current VM states:

client                   poweroff

ceph                     poweroff

This environment represents multiple VMs. The VMs are all listed

above with their current state. For more information about a specific

VM, run `vagrant status NAME`.

</pre>

Eventually run them:

<pre>

$ vagrant up ceph && vagrant up client

...

...

</pre>

The next time, you’ll run the client, run it this way to don’t re-provision the machine:

<pre>

$ vagrant up --no-provision client

</pre>

Eventually SSH on your client:

<pre>

$ vagrant ssh client

...

vagrant@ceph:~$ sudo ceph -s   

   health HEALTH_OK

   monmap e3: 1 mons at {1=192.168.251.100:6790/0}, election epoch 1, quorum 0 1

   osdmap e179: 2 osds: 2 up, 2 in

   pgmap v724: 96 pgs: 96 active+clean; 9199 bytes data, 2071 MB used, 17906 MB / 19978 MB avail; 232B/s wr, 0op/s

   mdsmap e54: 1/1/1 up {0=0=up:active}

vagrant@ceph:~$ sudo ceph osd tree

# id  weight  type name up/down reweight

-1  2 root default

-4  2   datacenter dc

-5  2     room laroom

-6  2       row larow

-3  2         rack lerack

-2  2           host ceph

0 1             osd.0 up  1

1 1             osd.1 up  1

</pre>

h3. III. BONUS UPGRADE TO CUTTLEFISH

It’s fairly easy to upgrade the box to last stable version Cuttlefish. For this simply edit <pre>/etc/apt/sources.list.d/ceph.list/ceph.list</pre> with the following:

<pre>deb http://ceph.com/debian-cuttlefish/ precise main</pre>

Then run:

<pre>

$ sudo apt-get update && apt-get install ceph

$ sudo service ceph restart

$ sudo ceph -v

ceph version 0.61 (237f3f1e8d8c3b85666529860285dcdffdeda4c5)

</pre>

*Note*: if for some reasons you get a status were only 1/2 OSDs are up, just restart the mon. This should do the trick :-).