Project

General

Profile

Actions
Copy link

Bug #45990

open

Add MDS Daemon with ceph orch

Added by Mario Ohnewald almost 4 years ago. Updated almost 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v15.2.1, Ceph - v15.2.2, Ceph - v15.2.3
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I am trying to get cephfs up & running:

root@ceph01:~# cephadm shell --fsid 5436dd5d-83d4-4dc8-a93b-60ab5db145df
INFO:cephadm:Using recent ceph image ceph/ceph:v15

root@ceph01:/# ceph orch ps --daemon-type mds
No daemons reported

root@ceph01:/# ceph fs volume create cephfs
root@ceph01:/# # 5mins wait

root@ceph01:/# ceph orch ps --daemon-type mds
No daemons reported

root@ceph01:/# ceph -s
cluster:
id: 5436dd5d-83d4-4dc8-a93b-60ab5db145df
health: HEALTH_ERR
1 filesystem is offline
1 filesystem is online with fewer MDS than max_mds

services:
    mon: 3 daemons, quorum ceph01,ceph02,ceph03 (age 2h)
    mgr: ceph03(active, since 7d), standbys: ceph02, ceph01
    mds: cephfs:0
    osd: 55 osds: 55 up (since 2h), 54 in (since 4d)
data:
    pools:   7 pools, 2177 pgs
    objects: 9.29M objects, 33 TiB
    usage:   100 TiB used, 34 TiB / 133 TiB avail
    pgs:     2174 active+clean
             3    active+clean+scrubbing+deep
io:
    client:   17 KiB/s rd, 91 MiB/s wr, 4 op/s rd, 23 op/s wr

The man page from: https://docs.ceph.com/docs/master/cephadm/install/ says:
One or more MDS daemons is required to use the CephFS file system. These are created automatically if the newer ceph fs volume interface is used to create a new file system.

More info:

root@ceph01:/# ceph versions {
"mon": {
"ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable)": 3
},
"mgr": {
"ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable)": 3
},
"osd": {
"ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable)": 55
},
"mds": {},
"overall": {
"ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable)": 61
}
}

root@ceph01:/# ceph orch ps
NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID
mgr.ceph01 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 274a48a62794
mgr.ceph02 ceph01 stopped 5d ago - <unknown> <unknown> <unknown> <unknown>
mgr.ceph02 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 11f01d55c80b
mgr.ceph03 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 745b6e4b87b8
mgr.cph02 ceph02 stopped 5d ago - <unknown> <unknown> <unknown> <unknown>
mon.ceph01 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f b12e80504654
mon.ceph02 ceph01 stopped 5d ago - <unknown> <unknown> <unknown> <unknown>
mon.ceph02 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 54b96ade3efd
mon.ceph03 ceph01 stopped 5d ago - <unknown> <unknown> <unknown> <unknown>
mon.ceph03 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f d019ae354a56
osd.0 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 43563b61772e
osd.1 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 5dc101bc2aef
osd.10 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f e123baa2bfa7
osd.11 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f c8299e462552
osd.12 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 831e02aa83c6
osd.13 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 6a8f0d10d9bc
osd.14 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f f76587a1ff28
osd.15 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 559590c57a4a
osd.16 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 11a22c249238
osd.17 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 62cb6f5763dc
osd.18 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 9e4d49c40220
osd.19 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 434ba7ecd322
osd.2 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f bb2f83bbee85
osd.20 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 9d6a97293aac
osd.21 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f ca1bc2c0bebf
osd.22 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 40d174919684
osd.23 ceph02 running (5d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 89d9918766a0
osd.24 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 9177f9b01512
osd.25 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 65cb739a25c4
osd.26 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 38cd0977525b
osd.27 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f dc11f321484c
osd.28 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 0a31b062fa0d
osd.29 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 92760be1218d
osd.3 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f ec677319f8da
osd.30 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 497fff57eb2c
osd.31 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 08521ba36cbf
osd.32 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f e791e0b81671
osd.33 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 214f08ed8601
osd.34 ceph04 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f ba1b9d7c853f
osd.35 ceph04 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f a224141a043a
osd.36 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 65678f209b15
osd.37 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 482d5d29764e
osd.38 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f b49e7bbc725c
osd.39 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f df13f747541b
osd.4 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 8c38dab352ac
osd.40 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 934e1a7e5100
osd.41 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f c2a6f540aae8
osd.42 ceph06 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f b11d651e43ff
osd.43 ceph06 running (13d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 94a2704b2d9f
osd.44 ceph04 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f e61cf28deb67
osd.45 ceph04 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f b1dc57108902
osd.46 ceph05 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 674f28a37a77
osd.47 ceph05 running (2w) 5d ago 5w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f ffab448fbd88
osd.48 ceph04 running (2w) 5d ago 3w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 89f6e54c88e6
osd.49 ceph05 running (2w) 5d ago 3w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 12fd6a326f11
osd.5 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 7ad984b4f900
osd.50 ceph05 running (2w) 5d ago 3w 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 7f397a636913
osd.51 ceph04 running (10d) 5d ago 10d 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 63ce61bb3b85
osd.52 ceph04 running (10d) 5d ago 10d 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 52e69a9e949f
osd.53 ceph05 running (10d) 5d ago 10d 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 8784f36db594
osd.54 ceph05 running (10d) 5d ago 10d 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 6f56f8b673ab
osd.6 ceph03 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 1e224a96da06
osd.7 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 1cfe28d6f8d1
osd.8 ceph01 running (8d) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 91e9573f87b2
osd.9 ceph02 running (2w) 5d ago - 15.2.1 docker.io/ceph/ceph:v15 bc83a388465f 8afea3c593ef

root@ceph01:~# dpkg -l | grep ceph
ii ceph-base 15.2.3-1~bpo10+1 amd64 common ceph daemon libraries and management tools
ii ceph-common 15.2.3-1~bpo10+1 amd64 common utilities to mount and interact with a ceph storage cluster
ii ceph-deploy 2.0.1 all Ceph-deploy is an easy to use configuration tool
ii ceph-fuse 15.2.3-1~bpo10+1 amd64 FUSE-based client for the Ceph distributed file system
ii ceph-grafana-dashboards 15.2.3-1~bpo10+1 all grafana dashboards for the ceph dashboard
ii ceph-mds 15.2.3-1~bpo10+1 amd64 metadata server for the ceph distributed file system
ii ceph-mgr 15.2.3-1~bpo10+1 amd64 manager for the ceph distributed storage system
ii ceph-mgr-cephadm 15.2.3-1~bpo10+1 all cephadm orchestrator module for ceph-mgr
ii ceph-mgr-dashboard 15.2.3-1~bpo10+1 all dashboard module for ceph-mgr
ii ceph-mgr-diskprediction-cloud 15.2.3-1~bpo10+1 all diskprediction-cloud module for ceph-mgr
ii ceph-mgr-diskprediction-local 15.2.3-1~bpo10+1 all diskprediction-local module for ceph-mgr
ii ceph-mgr-k8sevents 15.2.3-1~bpo10+1 all kubernetes events module for ceph-mgr
ii ceph-mgr-modules-core 15.2.3-1~bpo10+1 all ceph manager modules which are always enabled
ii ceph-mon 15.2.3-1~bpo10+1 amd64 monitor server for the ceph storage system
ii ceph-osd 15.2.3-1~bpo10+1 amd64 OSD server for the ceph storage system
ii cephadm 15.2.3-1~bpo10+1 amd64 cephadm utility to bootstrap ceph daemons with systemd and containers
ii libcephfs1 10.2.11-2 amd64 Ceph distributed file system client library
ii libcephfs2 15.2.3-1~bpo10+1 amd64 Ceph distributed file system client library
ii python-ceph-argparse 14.2.8-1 all Python 2 utility libraries for Ceph CLI
ii python3-ceph-argparse 15.2.3-1~bpo10+1 all Python 3 utility libraries for Ceph CLI
ii python3-ceph-common 15.2.3-1~bpo10+1 all Python 3 utility libraries for Ceph
ii python3-cephfs 15.2.3-1~bpo10+1 amd64 Python 3 libraries for the Ceph libcephfs library

Maybe its related to: https://tracker.ceph.com/issues/45834

Actions
Copy link
 #1

Updated by Sage Weil almost 3 years ago

  • Project changed from Ceph to CephFS
Actions
Copy link

Also available in: Atom PDF