|
|
|
[dpivonka@localhost kcli_plans]$ ssh fedora@192.168.122.246
|
|
fedora@192.168.122.246's password:
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ rsync dpivonka@192.168.0.115:/home/dpivonka/Documents/ceph/src/cephadm/cephadm .
|
|
The authenticity of host '192.168.0.115 (192.168.0.115)' can't be established.
|
|
ECDSA key fingerprint is SHA256:J3NnR0OIbse1AehwtLkORKLoOSHS9NhDMyv5aRWgsLY.
|
|
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
|
|
Warning: Permanently added '192.168.0.115' (ECDSA) to the list of known hosts.
|
|
dpivonka@192.168.0.115's password:
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ cat /etc/hosts
|
|
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
|
|
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
|
|
192.168.122.246 node00
|
|
192.168.122.167 node01
|
|
192.168.122.30 node02
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ ./cephadm --image quay.io/daniel_pivonka/ceph:latest bootstrap --mon-ip 192.168.122.246 --ssh-user fedora
|
|
ERROR: cephadm should be run as root
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ sudo ./cephadm --image quay.io/daniel_pivonka/ceph:latest bootstrap --mon-ip 192.168.122.246 --ssh-user fedora
|
|
Creating directory /etc/ceph for ceph.conf
|
|
Verifying podman|docker is present...
|
|
Verifying lvm2 is present...
|
|
Verifying time synchronization is in place...
|
|
Unit chronyd.service is enabled and running
|
|
Repeating the final host check...
|
|
podman|docker (/usr/bin/podman) is present
|
|
systemctl is present
|
|
lvcreate is present
|
|
Unit chronyd.service is enabled and running
|
|
Host looks OK
|
|
Cluster fsid: bfe58fc0-9c71-11eb-893e-52540031b82d
|
|
Verifying IP 192.168.122.246 port 3300 ...
|
|
Verifying IP 192.168.122.246 port 6789 ...
|
|
Mon IP 192.168.122.246 is in CIDR network 192.168.122.0/24
|
|
- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network
|
|
Pulling container image quay.io/daniel_pivonka/ceph:latest...
|
|
Ceph version: ceph version 17.0.0-2956-g9d7a42e7 (9d7a42e709558e51e7b83fc404c5ee4223c061ea) quincy (dev)
|
|
Extracting ceph user uid/gid from container image...
|
|
Creating initial keys...
|
|
Creating initial monmap...
|
|
Creating mon...
|
|
Waiting for mon to start...
|
|
Waiting for mon...
|
|
mon is available
|
|
Assimilating anything we can from ceph.conf...
|
|
Generating new minimal ceph.conf...
|
|
Restarting the monitor...
|
|
Setting mon public_network to 192.168.122.0/24
|
|
Wrote config to /etc/ceph/ceph.conf
|
|
Wrote keyring to /etc/ceph/ceph.client.admin.keyring
|
|
Creating mgr...
|
|
Verifying port 9283 ...
|
|
Waiting for mgr to start...
|
|
Waiting for mgr...
|
|
mgr not available, waiting (1/15)...
|
|
mgr not available, waiting (2/15)...
|
|
mgr not available, waiting (3/15)...
|
|
mgr is available
|
|
Enabling cephadm module...
|
|
Waiting for the mgr to restart...
|
|
Waiting for mgr epoch 5...
|
|
mgr epoch 5 is available
|
|
Setting orchestrator backend to cephadm...
|
|
Generating ssh key...
|
|
Wrote public SSH key to /etc/ceph/ceph.pub
|
|
Adding key to fedora@localhost authorized_keys...
|
|
Adding host node00...
|
|
Deploying mon service with default placement...
|
|
Deploying mgr service with default placement...
|
|
Deploying crash service with default placement...
|
|
Enabling mgr prometheus module...
|
|
Deploying prometheus service with default placement...
|
|
Deploying grafana service with default placement...
|
|
Deploying node-exporter service with default placement...
|
|
Deploying alertmanager service with default placement...
|
|
Enabling the dashboard module...
|
|
Waiting for the mgr to restart...
|
|
Waiting for mgr epoch 13...
|
|
mgr epoch 13 is available
|
|
Generating a dashboard self-signed certificate...
|
|
Creating initial admin user...
|
|
Fetching dashboard port number...
|
|
Ceph Dashboard is now available at:
|
|
|
|
URL: https://node00:8443/
|
|
User: admin
|
|
Password: 79mmczlpgc
|
|
|
|
You can access the Ceph CLI with:
|
|
|
|
sudo ./cephadm shell --fsid bfe58fc0-9c71-11eb-893e-52540031b82d -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring
|
|
|
|
Please consider enabling telemetry to help improve Ceph:
|
|
|
|
ceph telemetry on
|
|
|
|
For more information see:
|
|
|
|
https://docs.ceph.com/docs/master/mgr/telemetry/
|
|
|
|
Bootstrap complete.
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ sudo ./cephadm shell
|
|
Inferring fsid bfe58fc0-9c71-11eb-893e-52540031b82d
|
|
Inferring config /var/lib/ceph/bfe58fc0-9c71-11eb-893e-52540031b82d/mon.node00/config
|
|
Using recent ceph image quay.io/daniel_pivonka/ceph@sha256:74ad88107efd38f1d336bfb7b6fc48358ef566f72049664b027a573817ef08cb
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]# ceph orch host ls
|
|
HOST ADDR LABELS STATUS
|
|
node00 192.168.122.246
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]# exit
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ ssh-copy-id -f -i /etc/ceph/ceph.pub fedora@node01
|
|
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/etc/ceph/ceph.pub"
|
|
The authenticity of host 'node01 (192.168.122.167)' can't be established.
|
|
ECDSA key fingerprint is SHA256:BVQaq5n6JczginbY4npcTWYBLHq28aUdj2cu4aSnPCg.
|
|
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
|
|
fedora@node01's password:
|
|
|
|
Number of key(s) added: 1
|
|
|
|
Now try logging into the machine, with: "ssh 'fedora@node01'"
|
|
and check to make sure that only the key(s) you wanted were added.
|
|
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ sudo ./cephadm shell
|
|
Inferring fsid bfe58fc0-9c71-11eb-893e-52540031b82d
|
|
Inferring config /var/lib/ceph/bfe58fc0-9c71-11eb-893e-52540031b82d/mon.node00/config
|
|
Using recent ceph image quay.io/daniel_pivonka/ceph@sha256:74ad88107efd38f1d336bfb7b6fc48358ef566f72049664b027a573817ef08cb
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]# ceph orch host add node01
|
|
Error EINVAL: Failed to connect to node01 (node01).
|
|
Please make sure that the host is reachable and accepts connections using the cephadm SSH key
|
|
|
|
To add the cephadm SSH key to the host:
|
|
> ceph cephadm get-pub-key > ~/ceph.pub
|
|
> ssh-copy-id -f -i ~/ceph.pub fedora@node01
|
|
|
|
To check that the host is reachable:
|
|
> ceph cephadm get-ssh-config > ssh_config
|
|
> ceph config-key get mgr/cephadm/ssh_identity_key > ~/cephadm_private_key
|
|
> chmod 0600 ~/cephadm_private_key
|
|
> ssh -F ssh_config -i ~/cephadm_private_key fedora@node01
|
|
[ceph: root@node00 /]# ceph cephadm get-ssh-config > ssh_config
|
|
[ceph: root@node00 /]# ceph config-key get mgr/cephadm/ssh_identity_key > ~/cephadm_private_key
|
|
[ceph: root@node00 /]# chmod 0600 ~/cephadm_private_key
|
|
[ceph: root@node00 /]# ssh -F ssh_config -i ~/cephadm_private_key fedora@node01
|
|
Warning: Permanently added 'node01,192.168.122.167' (ECDSA) to the list of known hosts.
|
|
[fedora@node01 ~]$
|
|
[fedora@node01 ~]$
|
|
[fedora@node01 ~]$ # WHY DID THIS WORK BUT THE HOST COULD NOT BE ADDED TO THE CLUSTER
|
|
[fedora@node01 ~]$
|
|
[fedora@node01 ~]$
|
|
[fedora@node01 ~]$ logout
|
|
Connection to node01 closed.
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]# cat /etc/hosts
|
|
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
|
|
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
|
|
192.168.122.246 node00
|
|
192.168.122.167 node01
|
|
192.168.122.30 node02
|
|
[ceph: root@node00 /]#
|
|
[ceph: root@node00 /]# # /ETC/HOSTS IS IN THE SHELL CONTIANER
|
|
[ceph: root@node00 /]# exit
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ sudo podman ps
|
|
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
|
1f32d184f3e1 docker.io/prom/alertmanager:v0.20.0 --web.listen-addr... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-alertmanager.node00
|
|
fead1b7a10ad docker.io/ceph/ceph-grafana:6.7.4 /bin/bash About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-grafana.node00
|
|
e97603b3e87b docker.io/prom/prometheus:v2.18.1 --config.file=/et... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-prometheus.node00
|
|
cf4a39893f10 docker.io/prom/node-exporter:v0.18.1 --no-collector.ti... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-node-exporter.node00
|
|
c74d0cbafe7d quay.io/daniel_pivonka/ceph:latest -n client.crash.n... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-crash.node00
|
|
9124c0eb4296 quay.io/daniel_pivonka/ceph:latest -n mgr.node00.fkv... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-mgr.node00.fkveyp
|
|
b01d1ba02f18 quay.io/daniel_pivonka/ceph:latest -n mon.node00 -f ... About an hour ago Up About an hour ago ceph-bfe58fc0-9c71-11eb-893e-52540031b82d-mon.node00
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$
|
|
[fedora@node00 ~]$ sudo podman exec -it 9124c0eb4296 bash
|
|
[root@node00 /]#
|
|
[root@node00 /]#
|
|
[root@node00 /]# cat /etc/hosts
|
|
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
|
|
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
|
|
[root@node00 /]#
|
|
[root@node00 /]#
|
|
[root@node00 /]# # /ETC/HOSTS IS NOT IN THE MGR CONTIANER
|