Project

General

Profile

Create a Scalable and Resilient Object Gateway with Ceph and VirtualBox » History » Version 2

Jessica Mack, 06/22/2015 11:06 PM

1 1 Jessica Mack
h1. Create a Scalable and Resilient Object Gateway with Ceph and VirtualBox
2
3
{{toc}}
4
5
h3. Introducing the Ceph Object Gateway
6
7
Ceph is a highly reliable distributed storage system, with self-healing and self-managing characteristics. One of its unique characteristics is its unified storage interface, which supports object storage, block device storage and file system storage all in the same Ceph cluster. Of course, it's also open source, so you can freely download and experiment with it at your leisure.
8
The Ceph Object Gateway provides a way to host scalable data storage "buckets", similar to those provided by Amazon Simple Storage Service (Amazon S3) and OpenStack Swift. These objects are accessible via a REST API, making them ideal for cloud-based applications, big data storage and processing, and many other use cases. And because the underlying cluster infrastructure is managed by Ceph, fault-tolerance and scalability are guaranteed.
9
Setting up a Ceph object gateway can be a little complex, especially if you're unfamiliar with how scalable object storage works. That's where this tutorial comes in. Over the next few pages, I'll walk you through the process of setting up a Ceph-based object gateway and adding data to it. We'll set up the cluster using VirtualBox, so you'll get a chance to see Ceph's object storage features in action in a "real" environment where you have total control, but which doesn't cost you anything to run or scale out with new nodes.
10
Sounds good? Keep reading.
11
12
h3. Assumptions and Requirements
13
14
For this tutorial, I'll be using VirtualBox, which provides an easy way to set up independent virtual servers, with CentOS as the operating system for the virtual servers. VirtualBox is available for Windows, Linux, Macintosh, and Solaris hosts. I'll make the following assumptions:
15
You have a working knowledge of CentOS, VirtualBox and VirtualBox networking.
16
You have downloaded and installed the latest version of VirtualBox.
17
You have either already configured 5 virtual CentOS servers, or you have downloaded an ISO installation image for the latest version of CentOS (CentOS 7.0 at the time of writing). These servers must be using kernel version 3.10 or later
18
You're familiar with installing software using the yum, the CentOS package manager.
19
You’re familiar with SSH-based authentication.
20
You're familiar with object storage in the cloud.
21
In case you’re not familiar with the above topics, look in the “Read More” section at the end of this tutorial, which has links to relevant guides.
22
To set up a Ceph storage cluster with VirtualBox, here are the steps you'll follow:
23
Create cluster nodes
24
Install the Ceph deployment toolkit
25
Configure authentication between cluster nodes
26
Configure and activate a cluster monitor
27
Prepare and activate OSDs
28
Verify cluster health
29
Test the cluster
30
Install the Ceph object gateway
31
Configure the Ceph object gateway
32
Start working with buckets and objects
33
The next sections will walk you through these steps in detail.
34
35
h3. Step 1: Create Cluster Nodes
36
37
If you already have 5 virtual CentOS servers configured and talking to each other, you can skip this step. If not, you must first create the virtual servers that will make up your Ceph cluster. To do this:
38
Launch VirtualBox and use the Machine -> New menu to create a new virtual server.
39
40 2 Jessica Mack
!image1.jpg!
41
42 1 Jessica Mack
Keeping in mind that you will need 5 virtual servers running simultaneously, calculate the available RAM on the host system and set the server memory accordingly.
43
44 2 Jessica Mack
!image2.jpg!
45
46 1 Jessica Mack
Add a virtual hard drive of at least 10 GB.
47
48 2 Jessica Mack
!image3.jpg!
49
50 1 Jessica Mack
Ensure that you have an IDE controller with a virtual CD/DVD drive (to enable CentOS installation) and at least two network adapters, one NAT (to enable download of required software) and one bridged adapter or internal network adapter (for internal communication between the cluster nodes).
51
Once the server basics are defined, install CentOS to the server using the ISO installation image. Ensure that your kernel version is at least 3.10 or later.
52
Once the installation process is complete, log in to the server and configure the second network interface with a static IP address, by editing the appropriate template file in the /etc/sysconfig/network-scripts/ directory. Here's a sample of what the interface configuration might look like:
53
HWADDR=08:00:27:AE:14:41
54
TYPE=Ethernet
55
BOOTPROTO=static
56
DEFROUTE=yes
57
PEERDNS=yes
58
PEERROUTES=yes
59
IPV4_FAILURE_FATAL=no
60
IPV6INIT=yes
61
IPV6_AUTOCONF=yes
62
IPV6_DEFROUTE=yes
63
IPV6_PEERDNS=yes
64
IPV6_PEERROUTES=yes
65
IPV6_FAILURE_FATAL=no
66
NAME=enp0s8
67
UUID=5fc74119-1ab2-4c0c-9aa1-284fd484e6c6
68
ONBOOT=no
69
IPADDR=192.168.1.25
70
NETMASK=255.255.255.0
71
GATEWAY=192.168.1.1
72
DNS1=192.168.1.1
73
DNS2=8.8.8.8
74
Should any of the above steps be unfamiliar to you, refer to the VirtualBox manual, especially the VirtualBox networking guide, and to the networking section of the CentOS deployment guide.
75
Repeat this process until you have 5 virtual servers. Of these, identify one as the cluster administration node and assign it the hostname admin-node. The remaining servers may be identified with hostnames such as node1, node2, and so on. Here's an example of what the final cluster might look like (note that you should obviously modify the IP addresses to match your local network settings).
76
 
77
Server host name	IP address	
78
Purpose
79
admin-node	192.168.1.25	Administration node for cluster
80
node1	192.168.1.26	Monitor
81
node2	192.168.1.27	OSD daemon
82
node3	192.168.1.28	OSD daemon
83
node4	192.168.1.29	
84
Object gateway host / PHP client
85
Before proceeding to the next step, ensure that all the servers are accessible by pinging them using their host names. If you don't have a local DNS server, add the host names and IP addresses to each server's /etc/hosts file to ease network access.
86
87
h3. Step 2: Install the Ceph Deployment Toolkit
88
89
The next step is to install the Ceph deployment toolkit on the administration node. This toolkit will help install Ceph on the nodes in the cluster, as well as prepare and activate the cluster.
90
Log in to the administration node as the root user.
91
Add the package to the yum repository by creating a new file at /etc/yum.repos.d/ceph.repo with the following content:
92
[ceph-noarch]
93
name=Ceph noarch packages
94
baseurl=http://ceph.com/rpm-firefly/el7/noarch
95
enabled=1
96
gpgcheck=1
97
type=rpm-md
98
gpgkey=https://ceph.com/git/?p=ceph.git;a=b...ys/release.asc
99
Update the repository.
100
shell> yum update
101
Install the Ceph deployment toolkit.
102
shell> yum install ceph-deploy
103
 
104 2 Jessica Mack
!image4.jpg!
105 1 Jessica Mack
106
h3. Step 3: Configure Authentication between Cluster Nodes
107
108
Now, you need to create a ceph user on each server in the cluster, including the administration node. This user account will handle performing cluster-related operations on each node. Perform the following steps on each of the 5 virtual servers:
109
Log in as the root user.
110
Create a ceph user account.
111
shell> useradd ceph
112
shell> passwd ceph
113
Give the ceph user account root privileges with sudo.
114
shell> echo "ceph ALL = (root) NOPASSWD:ALL" | tee /etc/sudoers.d/ceph
115
shell> chmod 0440 /etc/sudoers.d/ceph
116
Disable 'requiretty' for the ceph user.
117
shell> sudo visudo
118
In the resulting file, locate the line containing
119
Defaults requiretty
120
and change it to read
121
Defaults:ceph !requiretty
122
Now, set up passphraseless SSH between the nodes:
123
Log in to the administration node as the ceph user.
124
Generate an SSH key for the administration node.
125
shell> ssh-keygen
126 2 Jessica Mack
127
!image52.jpg!
128
129 1 Jessica Mack
Copy the generated public key to the ceph user account of all the nodes in the cluster.
130
shell> ssh-copy-id ceph@node1
131
shell> ssh-copy-id ceph@node2
132
shell> ssh-copy-id ceph@node3
133
shell> ssh-copy-id ceph@node4
134
shell> ssh-copy-id ceph@admin-node
135
image6.jpg
136
Test that the ceph user on the administration node can log in to any other node as ceph using SSH and without providing a password.
137
shell> ssh ceph@node1
138
image7.jpg
139
Modify the administration node's SSH configuration file so that it can easily log in to each node as the ceph user. Create the /home/ceph/.ssh/config file with the following lines:
140
Host node1
141
  Hostname node1
142
  User ceph
143
Host node2
144
  Hostname node2
145
  User ceph
146
Host node3
147
  Hostname node3
148
  User ceph
149
Host node4
150
  Hostname node4
151
  User ceph
152
Host admin-node
153
  Hostname admin-node
154
  User ceph
155
Change the permissions of the /home/ceph/.ssh/config file.
156
shell> chmod 0400 ~/.ssh/config
157
Test that the ceph user on the administration node can log in to any other node using SSH and without providing a password or username.
158
shell> ssh node1
159
image8.jpg
160
 
161
Finally, create a directory on the administration node to store cluster information, such as configuration files and keyrings.
162
shell> mkdir my-cluster
163
shell> cd my-cluster
164
You're now ready to begin preparing and activating the cluster!
165
166
h3. Step 4: Configure and Activate a Cluster Monitor
167
168
A Ceph storage cluster consists of two types of daemons:
169
Monitors maintain copies of the cluster map
170
Object Storage Daemons (OSD) store data as objects on storage nodes
171
Apart from this, other actors in a Ceph storage cluster include metadata servers and clients such as Ceph block devices, Ceph object gateways or Ceph filesystems. Read more about Ceph’s architecture.
172
All the commands in this and subsequent sections are to be run when logged in as the ceph user on the administration node, from the my-cluster/ directory. Ensure that you are directly logged in as ceph and are not using root with su - ceph.
173
A minimal system will have at least one monitor and two OSD daemons for data replication.
174
Begin by setting up a Ceph monitor on node1 with the Ceph deployment toolkit.
175
shell> ceph-deploy new node1
176
This will define the name of the initial monitor node and create a default Ceph configuration file and monitor keyring in the current directory.
177
image9.jpg
178
 
179
Change the number of replicas in the Ceph configuration file at /home/ceph/my-cluster/ceph.conf from 3 to 2 so that Ceph can achieve a stable state with just two OSDs. Add the following line in the [global] section:
180
osd pool default size = 2
181
osd pool default min size = 2
182
In the same file, set the OSD journal size. A good general setting is 10 GB; however, since this is a simulation, you can use a smaller amount such as 4 GB. Add the following line in the [global] section:
183
osd journal size = 4000
184
In the same file, set the default number of placement groups for a pool. Since we’ll have less than 5 OSDs, 128 placement groups per pool should suffice. Add the following line in the [global] section:
185
osd pool default pg num = 128
186
Install Ceph on each node in the cluster, including the administration node.
187
shell> ceph-deploy install admin-node node1 node2 node3 node4
188
The Ceph deployment toolkit will now go to work installing Ceph on each node. Here's an example of what you will see during the installation process.
189
image10.jpg
190
Create the Ceph monitor on node1 and gather the initial keys.
191
shell> ceph-deploy mon create-initial node1
192
image11.jpg
193
194
h3. Step 5: Prepare and Activate OSDs
195
196
The next set is to prepare and activate Ceph OSDs. We'll need a minimum of 2 OSDs, and these should be set up on node2 and node3, as it's not recommended to mix monitors and OSD daemons on the same host. To begin, set up an OSD on node2 as follows:
197
Log into node2 as the ceph user.
198
shell> ssh node2
199
Create a directory for the OSD daemon.
200
shell> sudo mkdir /var/local/osd
201
Log out of node2. Then, from the administrative node, prepare and activate the OSD.
202
shell> ceph-deploy osd prepare node2:/var/local/osd
203
image12.jpg
204
shell> ceph-deploy osd activate node2:/var/local/osd
205
image13.jpg
206
Repeat the above steps for node3.
207
At this point, the OSD daemons have been created and the storage cluster is ready.
208
209
h3. Step 6: Verify Cluster Health
210
211
Copy the configuration file and admin keyring from the administration node to all the nodes in the cluster.
212
shell> ceph-deploy admin admin-node node1 node2 node3 node4
213
image14.jpg
214
Log in to each node as the ceph user and change the permissions of the admin keyring.
215
shell> ssh node1
216
shell> sudo chmod +r /etc/ceph/ceph.client.admin.keyring
217
You should now be able to check cluster health from any node in the cluster with the ceph status command. Ideally, you want to see the status active + clean, as that indicates the cluster is operating normally.
218
shell> ceph status
219
image15.jpg
220
221
h3. Step 7: Test the Cluster
222
223
You can now perform a simple test to see the distributed Ceph storage cluster in action, by writing a file on one node and retrieving it on another:
224
Log in to node1 as the ceph user.
225
shell> ssh node1
226
Create a new file with some dummy data.
227
shell> echo "Hello world" > /tmp/hello.txt
228
Data is stored in Ceph within storage pools, which are logical groups in which to organize your data. By default, a Ceph storage cluster has 3 pools - data, metadata and rbd - and it's also possible to create your own custom pools. In this case, copy the file to the data pool with the rados put command and assign it a name.
229
shell> rados put hello-object /tmp/hello.txt --pool data
230
To verify that the Ceph storage cluster stored the object:
231
Log in to node2 as the ceph user.
232
Check that the file exists in the cluster's data storage pool with the rados ls command.
233
shell> rados ls --pool data
234
Copy the file out of the storage cluster to a local directory with the rados get command and verify its contents
235
shell> rados get hello-object /tmp/hello.txt --pool data
236
shell> cat hello.txt
237
image16.jpg
238
239
h3. Step 8: Install the Ceph Object Gateway
240
241
Now that the cluster is operating, it’s time to do something with it. First, you must install and configure an Apache Web server with FastCGI on node4, as described below.
242
Log into node4 as the ceph user.
243
shell> ssh node4
244
Install Apache and FastCGI from the Ceph repositories. To do this, you need to first install the yum priorities plugin, then add the repositories to your yum repository list.
245
shell> sudo yum install yum-plugin-priorities
246
Edit the /etc/yum/pluginconf.d/priorities.conf file and ensure it looks like this:
247
[main]
248
enabled = 1
249
Create a file at /etc/yum.repos.d/ceph-apache.repo and fill it with the following content:
250
[apache2-ceph-noarch]
251
name=Apache noarch packages for Ceph
252
baseurl=http://gitbuilder.ceph.com/apache2-r...sic/ref/master
253
enabled=1
254
priority=2
255
gpgcheck=1
256
type=rpm-md
257
gpgkey=https://ceph.com/git/?p=ceph.git;a=b.../autobuild.asc
258
[apache2-ceph-source]
259
name=Apache source packages for Ceph
260
baseurl=http://gitbuilder.ceph.com/apache2-r...sic/ref/master
261
enabled=0
262
priority=2
263
gpgcheck=1
264
type=rpm-md
265
gpgkey=https://ceph.com/git/?p=ceph.git;a=b.../autobuild.asc
266
Create a file at /etc/yum.repos.d/ceph-fastcgi.repo and fill it with the following content:
267
[fastcgi-ceph-basearch]
268
name=FastCGI basearch packages for Ceph
269
baseurl=http://gitbuilder.ceph.com/mod_fastc...sic/ref/master
270
enabled=1
271
priority=2
272
gpgcheck=1
273
type=rpm-md
274
gpgkey=https://ceph.com/git/?p=ceph.git;a=b.../autobuild.asc
275
[fastcgi-ceph-noarch]
276
name=FastCGI noarch packages for Ceph
277
baseurl=http://gitbuilder.ceph.com/mod_fastc...sic/ref/master
278
enabled=1
279
priority=2
280
gpgcheck=1
281
type=rpm-md
282
gpgkey=https://ceph.com/git/?p=ceph.git;a=b.../autobuild.asc
283
[fastcgi-ceph-source]
284
name=FastCGI source packages for Ceph
285
baseurl=http://gitbuilder.ceph.com/mod_fastc...sic/ref/master
286
enabled=0
287
priority=2
288
gpgcheck=1
289
type=rpm-md
290
gpgkey=https://ceph.com/git/?p=ceph.git;a=b.../autobuild.asc
291
Update the repository and install Apache and FastCGI.
292
shell> sudo yum update
293
shell> sudo yum install httpd mod_fastcgi
294
Edit the /etc/httpd/conf/httpd.conf file and modify the ServerName directive to reflect the server's host name. Uncomment the line if needed.
295
ServerName node4
296
Review the files in the /etc/httpd/conf.modules.d/* directory to ensure that Apache's URL rewriting and FastCGI modules are enabled. You should find the following entries in the files:
297
LoadModule rewrite_module modules/mod_rewrite.so
298
LoadModule fastcgi_module modules/mod_fastcgi.so
299
In case these entries don't exist, add them to the end of the /etc/httpd/conf/httpd.conf file.
300
Restart Apache.
301
shell> sudo service httpd restart
302
Amazon S3 lets you refer to buckets using subdomains, such as http://mybucket.s3.amazonaws.com. You can also accomplish this with Ceph, but you must first install a local DNS server like dnsmasq and add support for wildcard subdomains. Follow these steps:
303
Log into node4 as the ceph user.
304
shell> ssh node4
305
Install dnsmasq.
306
shell> yum install dnsmasq
307
Edit the dnsmasq configuration file at /etc/dnsmasq.conf and add the following line to the end of the file:
308
address=/.node4/192.168.1.29
309
Save the file and restart dnsmasq.
310
shell> sudo service dnsmasq restart
311
If necessary, update the /etc/resolv.conf file on the client host so that it knows about the new DNS server.
312
nameserver 192.168.1.29
313
You should now be able to successfully ping any subdomain of *.node4, such as mybucket.node4 or example.node4, as shown in the image below.
314
 
315
image17.png
316
 
317
TIP: If you're not able to configure wildcard subdomains, you can also simply decide a list of subdomains you wish to use and then add them as static entries to the client system's /etc/hosts file. Ensure that the entries resolve to the node4 virtual host.
318
The final step is to install radosgw on node4:
319
shell> ssh node4
320
shell> sudo yum install ceph-radosgw
321
At this point, you have a Web server running with the Ceph object gateway and FastCGI support, and subdomains that resolve to the object gateway host.
322
323
h3. Step 9: Configure the Ceph Object Gateway
324
325
The next step is to configure the Ceph Object Gateway daemon. Follow these steps:
326
Log into the administration node as the ceph user.
327
shell> ssh admin-node
328
Create a keyring for the gateway.
329
shell> sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
330
shell> sudo chmod +r /etc/ceph/ceph.client.radosgw.keyring
331
Generate a user name and key to use when accessing the gateway. For this example, the user name is client.radosgw.gateway.
332
shell> sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n  client.radosgw.gateway --gen-key
333
Add read and write capabilities to the new key:
334
shell> sudo ceph-authtool -n  client.radosgw.gateway --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
335
Add the new key to the storage cluster and distribute it to the object gateway node.
336
shell> sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.radosgw.gateway -i /etc/ceph/ceph.client.radosgw.keyring
337
shell> sudo scp /etc/ceph/ceph.client.radosgw.keyring  ceph@node4:/home/ceph
338
shell> ssh node4
339
shell> sudo mv ceph.client.radosgw.keyring /etc/ceph/ceph.client.radosgw.keyring
340
shell> exit
341
This process should also have created a number of storage pools for the gateway. You can verify this by running the following command and verifying that the output includes various .rgw pools.
342
shell> rados lspools
343
image18.png
344
Change to your cluster configuration directory.
345
shell> cd ~/my-cluster
346
Edit the Ceph configuration file at ~/my-cluster/ceph/ceph.conf and add a new [client.radosgw.gateway] section to it, as below:
347
[client.radosgw.gateway]
348
host = node4
349
keyring = /etc/ceph/ceph.client.radosgw.keyring
350
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
351
log file = /var/log/radosgw/client.radosgw.gateway.log
352
rgw dns name = node4
353
rgw print continue = false
354
Transmit the new Ceph configuration file to all the other nodes in the cluster.
355
shell> ceph-deploy config push admin-node node1 node2 node3 node4
356
Log into node4 as the ceph user.
357
shell> ssh node4
358
Add a Ceph object gateway script, by creating a file at /var/www/html/s3gw.fcgi with the following content:
359
#!/bin/sh
360
exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
361
Change the permissions of the script to make it executable.
362
shell> sudo chmod +x /var/www/html/s3gw.fcgi
363
Create a data directory for the radosgw daemon.
364
shell> sudo mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway
365
Add a gateway configuration file, by creating a file at /etc/httpd/conf.d/rgw.conf and filling it with the following content:
366
FastCgiExternalServer /var/www/html/s3gw.fcgi -socket /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
367
<VirtualHost *:80>
368
    ServerName node4
369
    ServerAlias *.node4
370
    ServerAdmin admin@localhost
371
    DocumentRoot /var/www/html
372
    RewriteEngine On
373
    RewriteRule  ^/(.*) /s3gw.fcgi?%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
374
    <IfModule mod_fastcgi.c>
375
       <Directory /var/www/html>
376
            Options +ExecCGI
377
            AllowOverride All
378
            SetHandler fastcgi-script
379
            Order allow,deny
380
            Allow from all
381
            AuthBasicAuthoritative Off
382
        </Directory>
383
    </IfModule>
384
    AllowEncodedSlashes On
385
    ErrorLog /var/log/httpd/error.log
386
    CustomLog /var/log/httpd/access.log combined
387
    ServerSignature Off
388
</VirtualHost>
389
<VirtualHost *:443>
390
    ServerName node4
391
    ServerAlias *.node4
392
    ServerAdmin admin@localhost
393
    DocumentRoot /var/www/html
394
    RewriteEngine On
395
    RewriteRule  ^/(.*) /s3gw.fcgi?%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
396
    <IfModule mod_fastcgi.c>
397
       <Directory /var/www/html>
398
            Options +ExecCGI
399
            AllowOverride All
400
            SetHandler fastcgi-script
401
            Order allow,deny
402
            Allow from all
403
            AuthBasicAuthoritative Off
404
        </Directory>
405
    </IfModule>
406
    AllowEncodedSlashes On
407
    ErrorLog /var/log/httpd/error.log
408
    CustomLog /var/log/httpd/access.log combined
409
    ServerSignature Off
410
  SSLEngine on
411
  SSLCertificateFile /etc/apache2/ssl/apache.crt
412
  SSLCertificateKeyFile /etc/apache2/ssl/apache.key
413
  SetEnv SERVER_PORT_SECURE 443
414
</VirtualHost>
415
Edit the /etc/httpd/conf.d/fastcgi.conf file and ensure that the line referencing the FastCgiWrapper looks like this:
416
FastCgiWrapper off
417
Restart the Apache server, followed by the radosgw daemon.
418
shell> sudo service httpd restart
419
shell> sudo /etc/init.d/ceph-radosgw restart
420
You can quickly test that the object gateway is running by sending an HTTP GET request to the Web server, as shown below:
421
image19.png
422
At this point, your Ceph object gateway is running and you can begin using it.
423
424
h3. Step 10: Start Working with Buckets and Objects
425
426
Before you can begin using the Ceph object gateway, you must create a user account.
427
Log in to node4 as the ceph user.
428
shell> ssh admin-node
429
Create a new user account using the radosgw-admin command. In this example, the user is named 'john'.
430
shell> radosgw-admin user create --uid=john --display-name="Example User"
431
Here's an example of what you should see. Note the access key and secret key in the output, as you will need this to access the object gateway from another client.
432
image20.png
433
You can also verify that the user was created with the following command:
434
shell> radosgw-admin user info --uid=john
435
While you can interact with the object gateway directly over HTTP, by sending authenticated GET, PUT and DELETE requests to the gateway endpoints, an easier way is with Amazon's AWS SDK. This SDK includes classes and constructs to help you work with buckets and objects in Amazon S3. Since the Ceph object gateway is S3-compatible, you can use the same SDK to interact with it as well.
436
The AWS SDK is available for multiple programming languages. In the examples that follow, I'll use the AWS SDK for PHP, but you will find code examples for other languages as well on the AWS developer website.
437
Log in to node4 (which will now also double as the client node) as the root user and install PHP and related tools.
438
shell> sudo yum install php curl php-curl
439
Create a working directory for your PHP files. Download Composer, the PHP dependency manager, into this directory.
440
shell> cd /tmp
441
shell> mkdir ceph
442
shell> cd ceph
443
shell> curl -sS https://getcomposer.org/installer | php
444
Create a composer.json file in the working directory and fill it with the following content:
445
{
446
    "require": {
447
        "aws/aws-sdk-php": "2.*"
448
    }
449
}
450
Download the AWS SDK for PHP and related dependencies using Composer:
451
shell> cd /tmp/ceph
452
shell> php composer.phar install
453
You can now begin interacting with your object gateway using PHP. For example, here's a simple PHP script to create a new bucket:
454
<?php
455
// create-bucket.php
456
// autoload files
457
require 'vendor/autoload.php';
458
use Aws\S3\S3Client;
459
// instantiate S3 client
460
$s3 = S3Client::factory(array(
461
        'key' => 'YOUR_ACCESS_KEY',
462
        'secret' => 'YOUR_SECRET_KEY',
463
        'endpoint' => 'http://node4'
464
));
465
// create bucket
466
try {
467
  $s3->createBucket(array('Bucket' => 'mybucket'));
468
  echo "Bucket created \n";
469
} catch (Aws\S3\Exception\S3Exception $e) {
470
  echo "Request failed: $e";
471
}
472
This script begins by initializing the Composer auto-loader and an instance of the S3Client object. The object is provided with the access key and secret for the user created earlier, and a custom endpoint points to the object gateway Web server.
473
The S3Client object provides a number of methods to create and manage buckets and objects. One of these is the createBucket() method, which accepts a bucket name and generates the necessary PUT request to create a new bucket in the object gateway.
474
You can run this script at the console as follows:
475
shell> php create-bucket.php
476
Here's an example of what the output might look like:
477
image21.png
478
You can also create a bucket and then add a file to it as an object, using the client object's upload() method. Here's an example:
479
<?php
480
// create-bucket-object.php
481
// autoload files
482
require 'vendor/autoload.php';
483
use Aws\S3\S3Client;
484
// instantiate S3 client
485
$s3 = S3Client::factory(array(
486
        'key' => 'YOUR_ACCESS_KEY',
487
        'secret' => 'YOUR_SECRET_KEY',
488
        'endpoint' => 'http://node4'
489
));
490
// create bucket and upload file to it
491
try {
492
  $s3->createBucket(array('Bucket' => 'myotherbucket'));
493
  $s3->upload('myotherbucket', 'test.tgz', file_get_contents('/tmp/test.tgz'), 'public-read');
494
  echo 'Bucket and object created';     
495
} catch (Aws\S3\Exception\S3Exception $e) {
496
  echo "Request failed: $e";
497
}
498
Of course, you can also list all the buckets and objects available to the authenticated user with the listBuckets() and listObjects() methods:
499
<?php
500
// list-bucket-contents.php
501
// autoload files
502
require 'vendor/autoload.php';
503
use Aws\S3\S3Client;
504
// instantiate S3 client
505
$s3 = S3Client::factory(array(
506
        'key' => 'YOUR_ACCESS_KEY',
507
        'secret' => 'YOUR_SECRET_KEY',
508
        'endpoint' => 'http://node4'
509
));
510
// create bucket and upload file to it
511
try {
512
  $bucketsColl = $s3->listBuckets();
513
  foreach ($bucketsColl['Buckets'] as $bucket) {
514
    echo $bucket['Name'] . "\n";
515
    $objColl = $s3->listObjects(array('Bucket' => $bucket['Name']));
516
     if ($objColl['Contents']) {
517
        foreach ($objColl['Contents'] as $obj) {
518
          echo '- ' . $obj['Key'] . "\n";
519
        }
520
     }
521
  }
522
} catch (Aws\S3\Exception\S3Exception $e) {
523
  echo "Request failed: $e";
524
}
525
Here's an example of what the output might look like:
526
image22.png
527
Of course, you can do a lot more with the AWS SDK for PHP. Refer to the reference documentation for a complete list of methods and example code.
528
529
h3. Conclusion
530
531
As this tutorial has illustrated, Ceph makes it easy to set up a standards-compliant object gateway for your applications or users, with all the benefits of a resilient, infinitely scalable underlying storage cluster.
532
The simple object gateway you created here with VirtualBox is just the tip of the iceberg: you can transition your object gateway to the cloud and run it in federated mode across regions and zones for even greater flexibility, and because the Ceph object gateway is also Swift-compliant, you can maximize compatibility for OpenStack users without any changes to your existing infrastructure. And of course, you can also use the underlying object storage cluster for fault-tolerant Ceph block devices or the POSIX-compliant CephFS filesystem.
533
The bottom line: Ceph's unique architecture gives you improved performance and flexibility without any loss in reliability and security. And it's open source, so you can experiment with it, improve it and use it without worrying about vendor lock-in. You can't get any better than that!
534
535
h3. Read More
536
537
"Introduction to Ceph":http://ceph.com/docs/master/start/intro/
538
"Ceph Architecture":http://ceph.com/docs/master/architecture/
539
"Getting Started With Ceph":http://www.inktank.com/resource/getting-started-with-ceph-miroslav-klivansky/
540
"Introduction to Ceph & OpenStack":http://www.inktank.com/resource/introduction-to-ceph-openstack-miroslav-klivansky/    
541
"Managing A Distributed Storage System At Scale":http://www.inktank.com/resource/managing-a-distributed-storage-system-at-scale-sage-weil/
542
"Scaling Storage With Ceph":http://www.inktank.com/resource/scaling-storage-with-ceph-ross-turk/
543
"Ceph API Documentation":http://ceph.com/docs/master/api/