Ceph

Alfredo Deza wrote:

In a few parts of the documentation it is requested to make changes to "all nodes in the cluster".

could you be more specific ?

- what a user might want in this case? for example, the node name? in the case of OSD nodes, it could be host, or the osd.{osd-id}? the format of the output?
- when he/she need the list? how he/she will use this list?

they are basically the same question: a user story. so i can figure out what the feature really wants.

thanks.

There are a few places in the docs, I was specifically following the federated config docs for RGW.

For example in this section (as well as in a couple of others) http://ceph.com/docs/master/radosgw/federated-config/#restart-services
it would mention things like:

use the following on each Ceph Node:

so we'd like to have a command which is able to list all nodes in the term of "http://ceph.com/docs/master/glossary/#term-ceph-node"

Any single machine or server in a Ceph System.

so it could be

$ ceph node list # list all nodes
osd host1 host2 host3
mon host4 host5 host6
mds host10 host11 host12

$ ceph node list --osd # list all OSD nodes
osd host1 host2 host3

hi Alfredo, the sample usage and output looks like following, if no objection, i will post the PR for review.

$ ./ceph node ls
mon: 127.0.0.1:6789 127.0.0.1:6790 127.0.0.1:6791
osd: 127.0.0.1:6800 127.0.0.1:6804 127.0.0.1:6815
mds: 127.0.0.1:6819 127.0.0.1:6821 127.0.0.1:6820
[kchai@rex003 src]$ ./ceph node ls all
mon: 127.0.0.1:6789 127.0.0.1:6790 127.0.0.1:6791
osd: 127.0.0.1:6800 127.0.0.1:6804 127.0.0.1:6815
mds: 127.0.0.1:6819 127.0.0.1:6821 127.0.0.1:6820
[kchai@rex003 src]$ ./ceph node ls osd
osd: 127.0.0.1:6800 127.0.0.1:6804 127.0.0.1:6815

loïc wrote

ceph-deploy enforces constraints that do not match the constraints of ceph, which may make things easier or more difficult, depending. For instance, the mon name must be a hostname for ceph-deploy (although it's possible to write ceph-deploy create mon hostname:moname, but it's not consistently supported accross the command lines and won't do what you expect). So, if you're to list the nodes for mons (and not for osds), this constraints makes it trivial for ceph-deploy to figure out the hostname: it's the same as the mon name. For osds, ceph does not care at all about the host name and there may be situations where a perfectly valid osd IP/port will not mean anything to ceph-deploy.

so different commands are expecting different representation of node address.

alfredo deza wrote:

default to hostnames unless --ipaddr is passed in? If some node doesn't have a hostname (or is not known) then you can fallback

talked with joao, seems we don't store the hostname anywhere in the monmap, osdmap or the mdsmap. but we can get_hostname on each of the mon.

joao wrote:

one alternative though. extend 'mon_status' to supply the mon's hostname. have each mon outputting the mon's hostname somewhere in mon_status. it will still require a check on an individual monitor basis, but it's better than not knowing it I suppose.

i am not sure this is the way we should go.

for the osd,

$ ./ceph osd tree
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
ID WEIGHT  TYPE NAME     UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 3.00000 root default
-2 3.00000     host gen8
 0 1.00000         osd.0      up  1.00000          1.00000
 1 1.00000         osd.1      up  1.00000          1.00000
 2 1.00000         osd.2      up  1.00000          1.00000

we should print the two level parent of device:

default gen8 osd.0
default gen8 osd.1
default gen8 osd.2

for the mds:

$ ./ceph mds dump
dumped mdsmap epoch 10
epoch   10
flags   0
created 2015-03-18 17:35:30.947077
modified        2015-03-18 17:35:39.878665
tableserver     0
root    0
session_timeout 60
session_autoclose       300
max_file_size   1099511627776
last_failure    0
last_failure_osd_epoch  0
compat  compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4
=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
max_mds 3
in      0,1,2
up      {0=4122,1=4120,2=4112}
failed
stopped
data_pools      1
metadata_pool   2
inline_data     disabled
4122:   127.0.0.1:6812/59748 'a' mds.0.1 up:active seq 3
4120:   127.0.0.1:6814/60103 'c' mds.1.1 up:active seq 2
4112:   127.0.0.1:6813/59932 'b' mds.2.1 up:active seq 3

we should print

a
c
b

for the mon,

dumped monmap epoch 1
epoch 1
fsid d75a486d-44de-47b5-885b-711ce03914a6
last_changed 2015-03-18 17:35:00.500817
created 2015-03-18 17:35:00.500817
0: 127.0.0.1:6789/0 mon.a
1: 127.0.0.1:6790/0 mon.b
2: 127.0.0.1:6791/0 mon.c

we should print

mon.a
mon.b
mon.c

following is the sample output of the newly added "node ls" command:

$ ./ceph node ls
mon a
mon b
mon c
osd rex003 osd.0
osd rex003 osd.1
osd rex003 osd.2
mds a
mds c
mds b
[kchai@rex003 src]$ ./ceph node ls mon
mon a
mon b
mon c
[kchai@rex003 src]$ ./ceph node ls osd
osd rex003 osd.0
osd rex003 osd.1
osd rex003 osd.2
[kchai@rex003 src]$ ./ceph node ls mds
mds a
mds c
mds b

will implement "mds metadata" and "mon medata" following the model of "osd metadata" first. and add "node ls" later on. just sent mail to the mailing list for comments.