Documentation #42315

Improve rados command usage, man page and turorial

Added by David Zafman over 4 years ago. Updated about 4 years ago.

David Zafman
Target version:
% Done:


Affected Versions:
Pull request ID:

Related issues

Duplicated by RADOS - Bug #42518: rados man page is badly out of sync with actual usage Duplicate 10/29/2019


#1 Updated by Josh Durgin over 4 years ago

  • Duplicated by Bug #42518: rados man page is badly out of sync with actual usage added

#2 Updated by Brad Hubbard over 4 years ago

  • Tags set to low-hanging-fruit

#3 Updated by Zac Dover about 4 years ago

[zdover@192-168-1-112 ~]$ rados -h
usage: rados [options] [commands]
lspools list pools
cppool <pool-name> <dest-pool> copy content of a pool
purge <pool-name> --yes-i-really-really-mean-it
remove all objects from pool <pool-name> without removing it
df show per-pool and total usage
ls list objects in pool

lssnap list snaps
mksnap <snap-name> create snap <snap-name>
rmsnap <snap-name> remove snap <snap-name>

get <obj-name> <outfile> fetch object
put <obj-name> <infile> [--offset offset]
write object with start offset (default:0)
append <obj-name> <infile> append object
truncate <obj-name> length truncate object
create <obj-name> create object
rm <obj-name> ...[--force-full] [force no matter full or not]remove object(s)
cp <obj-name> [target-obj] copy object
listxattr <obj-name>
getxattr <obj-name> attr
setxattr <obj-name> attr val
rmxattr <obj-name> attr
stat <obj-name> stat the named object
stat2 <obj-name> stat2 the named object (with high precision time)
touch <obj-name> [timestamp] change the named object modification time
mapext <obj-name>
rollback <obj-name> <snap-name> roll back object to snap <snap-name>

listsnaps &lt;obj-name&gt;             list the snapshots of this object
bench &lt;seconds&gt; write|seq|rand [-t concurrent_operations] [--no-cleanup] [--run-name run_name] [--no-hints] [--reuse-bench]
default is 16 concurrent IOs and 4 MB ops
default is to clean up after write benchmark
default run-name is 'benchmark_last_metadata'
cleanup [--run-name run_name] [--prefix prefix]
clean up a previous benchmark operation
default run-name is 'benchmark_last_metadata'
load-gen [options] generate load on the cluster
listomapkeys &lt;obj-name&gt; list the keys in the object map
listomapvals &lt;obj-name&gt; list the keys and vals in the object map
getomapval &lt;obj-name&gt; &lt;key&gt; [file] show the value for the specified key
in the object's object map
setomapval &lt;obj-name&gt; &lt;key&gt; &lt;val&gt;
rmomapkey &lt;obj-name&gt; &lt;key&gt;
clearomap &lt;obj-name&gt; [obj-name2 obj-name3...] clear all the omap keys for the specified objects
getomapheader &lt;obj-name&gt; [file]
setomapheader &lt;obj-name&gt; &lt;val&gt;
watch &lt;obj-name&gt; add watcher on this object
notify &lt;obj-name&gt; &lt;message&gt; notify watcher of this object with message
listwatchers &lt;obj-name&gt; list the watchers of this object
set-alloc-hint &lt;obj-name&gt; &lt;expected-object-size&gt; &lt;expected-write-size&gt;
set allocation hint for an object
set-redirect &lt;object A&gt; --target-pool &lt;caspool&gt; &lt;target object A&gt; [--with-reference]
set redirect target
set-chunk &lt;object A&gt; &lt;offset&gt; &lt;length&gt; --target-pool &lt;caspool&gt; &lt;target object A&gt; &lt;taget-offset&gt; [--with-reference]
convert an object to chunked object
tier-promote &lt;obj-name&gt; promote the object to the base tier
unset-manifest &lt;obj-name&gt; unset redirect or chunked object

export [filename]
Serialize pool contents to a file or standard out.
import [--dry-run] [--no-overwrite] < filename | - >
Load pool contents from a file or standard in

lock list <obj-name>
List all advisory locks on an object
lock get <obj-name> <lock-name>
Try to acquire a lock
lock break <obj-name> <lock-name> <locker-name>
Try to break a lock acquired by another client
lock info <obj-name> <lock-name>
Show lock information
--lock-tag Lock tag, all locks operation should use
the same tag
--lock-cookie Locker cookie
--lock-description Description of lock
--lock-duration Lock duration (in seconds)
--lock-type Lock type (shared, exclusive)

list-inconsistent-pg <pool> list inconsistent PGs in given pool
list-inconsistent-obj <pgid> list inconsistent objects in given pg
list-inconsistent-snapset <pgid> list inconsistent snapsets in the given pg

CACHE POOLS: (for testing/development only)
cache-flush <obj-name> flush cache pool object (blocking)
cache-try-flush <obj-name> flush cache pool object (non-blocking)
cache-evict <obj-name> evict cache pool object
cache-flush-evict-all flush+evict all objects
cache-try-flush-evict-all try-flush+evict all objects

--object_locator object_locator
set object_locator for operation
-p pool
select given pool by name
select target pool by name
-f [--format plain|json|json-pretty]
--format=[--format plain|json|json-pretty]
-b op_size
set the block size for put/get ops and for write benchmarking
-o object_size
set the object size for put/get ops and for write benchmarking
set the max number of objects for write benchmarking
-s name
--snap name
select given snap name for (read) IO
create the pool or directory that was specified
-N namespace
specify the namespace to use for the object
Use with ls to list objects in all namespaces
Put in CEPH_ARGS environment variable to make this the default
Use with ls to list objects in default namespace
Takes precedence over --all in case --all is in environment
Use with cp to specify the locator of the new object
Use with cp to specify the namespace of the new object
Use radostriper interface rather than pure rados
Available for stat, get, put, truncate, rm, ls and
all xattr related operations

-t N
Set number of concurrent I/O operations
prefix output with date/time
do not verify contents of read objects
write contents to the objects
write contents to the omap
write contents to the extended attributes

--num-objects total number of objects
--min-object-size min object size
--max-object-size max object size
--min-op-len min io size of operations
--max-op-len max io size of operations
--max-ops max number of operations
--max-backlog max backlog size
--read-percent percent of operations that are read
--target-throughput target throughput (in bytes)
--run-length total time (in seconds)
--offset-align at what boundary to align random op offsetsCACHE POOLS OPTIONS:
--with-clones include clones when doing flush or evict
--omap-key-file file read the omap key from a file
[zdover@192-168-1-112 ~]$

#4 Updated by Zac Dover about 4 years ago


rados - rados object storage utility

rados [ options ] [ command ]

rados is a utility for interacting with a Ceph object storage cluster
(RADOS), part of the Ceph distributed storage system.

-p pool, --pool pool
Interact with the given pool. Required by most commands.

-s snap, --snap snap
Read from the given pool snapshot. Valid for all pool-specific
read operations.
-i infile
will specify an input file to be passed along as a payload with
the command to the monitor cluster. This is only used for spe‐
cific monitor commands.
-o outfile
will write any payload returned by the monitor cluster with its
reply to outfile. Only specific monitor commands (e.g. osd
getmap) return a payload.
-c ceph.conf, --conf=ceph.conf
Use ceph.conf configuration file instead of the default
/etc/ceph/ceph.conf to determine monitor addresses during
-m monaddress[:port]
Connect to specified monitor (instead of looking through
-b block_size
Set the block size for put/get/append ops and for write bench‐
Uses the striping API of rados rather than the default one.
Available for stat, stat2, get, put, append, truncate, rm, ls
and all xattr related operation

List object pools

df     Show  utilization  statistics,  including disk usage (bytes) and
object counts, over the entire system and broken down by pool.
list-inconsistent-pg pool
List inconsistent PGs in given pool.
list-inconsistent-obj pgid
List inconsistent objects in given PG.
list-inconsistent-snapset pgid
List inconsistent snapsets in given PG.

get name outfile
Read object name from the cluster and write it to outfile.

put name infile [--offset offset]
Write object name with start offset (default:0) to the cluster
with contents from infile. Warning: The put command creates a
single RADOS object, sized just as large as your input file.
Unless your objects are of reasonable and consistent sizes, that
is probably not what you want -- consider using RGW/S3, CephFS,
or RBD instead.
append name infile
Append object name to the cluster with contents from infile.
rm name
Remove object name.
listwatchers name
List the watchers of object name.
ls outfile
List objects in given pool and write to outfile.
lssnap List snapshots for given pool.
clonedata srcname dstname --object-locator key
Clone object byte data from srcname to dstname. Both objects
must be stored with the locator key key (usually either srcname
or dstname). Object attributes and omap keys are not copied or
mksnap foo
Create pool snapshot named foo.
rmsnap foo
Remove pool snapshot named foo.
bench seconds mode [ -b objsize ] [ -t threads ]
Benchmark for seconds. The mode can be write, seq, or rand. seq
and rand are read benchmarks, either sequential or random.
Before running one of the reading benchmarks, run a write bench‐
mark with the --no-cleanup option. The default object size is 4
MB, and the default number of simulated threads (parallel
writes) is 16. The --run-name &lt;label&gt; option is useful for
benchmarking a workload test from multiple clients. The &lt;label&gt;
is an arbitrary object name. It is "benchmark_last_metadata" by
default, and is used as the underlying object name for "read"
and "write" ops. Note: -b objsize option is valid only in write
mode. Note: write and seq must be run on the same host other‐
wise the objects created by write will have names that will fail
cleanup [ --run-name run_name ] [ --prefix prefix ]
Clean up a previous benchmark operation. Note: the default
run-name is "benchmark_last_metadata"
listxattr name
List all extended attributes of an object.
getxattr name attr
Dump the extended attribute value of attr of an object.
setxattr name attr value
Set the value of attr in the extended attributes of an object.
rmxattr name attr
Remove attr from the extended attributes of an object.
stat name
Get stat (ie. mtime, size) of given object
stat2 name
Get stat (similar to stat, but with high precision time) of
given object
listomapkeys name
List all the keys stored in the object map of object name.
listomapvals name
List all key/value pairs stored in the object map of object
name. The values are dumped in hexadecimal.
getomapval [ --omap-key-file file ] name key [ out-file ]
Dump the hexadecimal value of key in the object map of object
name. If the optional out-file argument is not provided, the
value will be written to standard output.
setomapval [ --omap-key-file file ] name key [ value ]
Set the value of key in the object map of object name. If the
optional value argument is not provided, the value will be read
from standard input.
rmomapkey [ --omap-key-file file ] name key
Remove key from the object map of object name.
getomapheader name
Dump the hexadecimal value of the object map header of object
setomapheader name value
Set the value of the object map header of object name.
export filename
Serialize pool contents to a file or standard output.n"
import [--dry-run] [--no-overwrite] < filename | - >
Load pool contents from a file or standard input

To view cluster utilization:

rados df
To get a list object in pool foo sent to stdout:
rados -p foo ls -
To write an object:
rados -p foo put myobject blah.txt
To create a snapshot:
rados -p foo mksnap mysnap
To delete the object:
rados -p foo rm myobject
To read a previously snapshotted version of an object:
rados -p foo -s mysnap get myobject blah.txt.old
To list inconsistent objects in PG 0.6:
rados list-inconsistent-obj 0.6 --format=json-pretty

rados is part of Ceph, a massively scalable, open-source, distributed
storage system. Please refer to the Ceph documentation at for more information.


2010-2014, Inktank Storage, Inc. and contributors. Licensed under Cre‐
ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)

dev Sep 17, 2019 RADOS

Also available in: Atom PDF