Project

General

Profile

Tasks #10587

split ceph-mon, ceph-osd, and ceph-mds in the Debian and Red Hat packages

Added by Ken Dreyer almost 4 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
01/20/2015
Due date:
% Done:

0%

Tags:
Reviewed:
Affected Versions:
Pull request ID:

Description

ceph-mon and ceph-osd should be their own subpackages.

ceph-mon should contain:

/usr/bin/ceph-mon
/usr/lib/python2.7/site-packages/ceph_rest_api.py
/usr/lib/python2.7/site-packages/ceph_rest_api.pyc
/usr/lib/python2.7/site-packages/ceph_rest_api.pyo
/usr/share/man/man8/ceph-mon.8.gz
/var/lib/ceph/mon

ceph-osd should contain:

/usr/bin/ceph-clsinfo
/usr/bin/ceph-osd
/usr/lib/udev/rules.d/60-ceph-partuuid-workaround.rules
/usr/lib/udev/rules.d/95-ceph-osd.rules
/usr/lib64/rados-classes
/usr/lib64/rados-classes/libcls_hello.so
/usr/lib64/rados-classes/libcls_kvs.so
/usr/lib64/rados-classes/libcls_lock.so
/usr/lib64/rados-classes/libcls_log.so
/usr/lib64/rados-classes/libcls_rbd.so
/usr/lib64/rados-classes/libcls_refcount.so
/usr/lib64/rados-classes/libcls_replica_log.so
/usr/lib64/rados-classes/libcls_rgw.so
/usr/lib64/rados-classes/libcls_statelog.so
/usr/lib64/rados-classes/libcls_user.so
/usr/lib64/rados-classes/libcls_user.so.1
/usr/lib64/rados-classes/libcls_user.so.1.0.0
/usr/lib64/rados-classes/libcls_version.so
/usr/sbin/ceph-disk
/usr/sbin/ceph-disk-activate
/usr/sbin/ceph-disk-prepare
/usr/share/man/man8/ceph-clsinfo.8.gz
/usr/share/man/man8/ceph-osd.8.gz
/var/lib/ceph/osd

Associated revisions

Revision 8a050928 (diff)
Added by Ken Dreyer almost 4 years ago

debian: move /var/lib/ceph/mds to ceph-mds package

On Debian, ceph-mds was split out into its own package in
9d6013e0db47b258cbcde4c692554a2764812099, but the /var/lib/ceph/mds
directory was not moved along with the rest of the mds pieces.

The /var/lib/ceph/mds directory is only necessary if a user has
installed ceph-mds. Move it to the ceph-mds subpackage.

http://tracker.ceph.com/issues/10587 Refs: #10587

Signed-off-by: Ken Dreyer <>

Revision 7dd9af0a (diff)
Added by Ken Dreyer almost 3 years ago

init-ceph.in: skip ceph-disk if it is not present

With the ceph-mon vs ceph-osd split packaging, users are expected to
have the ceph-mon package installed and not ceph-osd (and vice versa).
However, the init script (/etc/init.d/ceph) has a call to `ceph-disk`,
which may not be present on the machine.

Given that our packaging is not yet split upstream, this bug does not
manifest itself currently, because both ceph-mon and ceph-disk are
currently in the same "ceph" package. Once we split the packaging,
though, this will become an issue.

http://tracker.ceph.com/issues/10587 Refs: #10587

Signed-off-by: Ken Dreyer <>

Revision 0cbe3dea (diff)
Added by Ali Maredia almost 3 years ago

debian/rpm: split mon/osd/mds server packages

Split up the "ceph" package into four new packages:

1. ceph-mon
2. ceph-osd
3. ceph-mds (already done on Debian)
4. ceph-base (files shared among multiple servers)

and then:

5. Make "ceph" into a metapackage that depends on both -mon and -osd
(and -mds, for RPMs).

To describe the outcome of this change another way:

For RPMs:
- "ceph-{mon,osd,mds}" Require: ceph-base.
- "ceph" will become a metapackage that Requires: ceph-{mon,osd,mds}.
For DEBs:
- "ceph-{mon,osd,mds}" will Depends: ceph-base.
- "ceph" will become a metapackage that Depends: ceph-{mon,osd}.
- "ceph" will continue to Recommends: ceph-mds

New users should "yum install ceph-mon" or "yum install ceph-osd" (or
"apt-get install ceph-mon", etc) in order to install the exact daemons
that they need.

http://tracker.ceph.com/issues/10587 Fixes: #10587

Signed-off-by: Ken Dreyer <>

History

#1 Updated by Ken Dreyer almost 4 years ago

In addition to the ceph-mon file list above, today I found that the ceph-rest-api command-line utility should also move to ceph-mon:

/usr/bin/ceph-rest-api

#2 Updated by Ken Dreyer almost 4 years ago

  • Subject changed from split ceph-mon and ceph-osd in the Debian and Red Hat packages to split ceph-mon, ceph-osd, and ceph-mds in the Debian and Red Hat packages

Additionally, ceph-mds should be its own subpackage.

#3 Updated by Ken Dreyer almost 4 years ago

Looks like ceph-mds is already split out into a separate package in Debian. So we just need to split that one in the RPM as well.

#4 Updated by Ken Dreyer almost 4 years ago

  • Status changed from New to In Progress

I was working on this today, and found two smaller things that I'd like to land in master before the bigger PR gets proposed:

https://github.com/ceph/ceph/pull/3992 - ceph_perf_objectstore is missing from the packaging
https://github.com/ceph/ceph/pull/3997 - /var/lib/ceph/mds is still in "ceph"

#5 Updated by Ken Dreyer almost 4 years ago

Current work is on the wip-10587-split-servers branch in GitHub.

Sage made the suggestion that we move the common files into a "ceph-server-common" package, and then make "ceph" an empty metapackage that installs "all of Ceph".

#6 Updated by Ken Dreyer almost 4 years ago

From #11376, the ceph-objectstore-tool utility should also be in ceph-osd.

/usr/bin/ceph-objectstore-tool

#7 Updated by Ken Dreyer almost 4 years ago

Ken Dreyer wrote:

Sage made the suggestion that we move the common files into a "ceph-server-common" package, and then make "ceph" an empty metapackage that installs "all of Ceph".

After further discussion with Sage, we'll punt a "ceph-server-common" package, and simply attack the problem of splitting up the mon/osd/mds packages.

#8 Updated by Ken Dreyer over 3 years ago

At the moment I'm stuck trying to find a way to make 1) upgrades via apt/yum work seamlessly and 2) avoid having the "ceph" package depend on ceph-mon/ceph-osd/ceph-mds . I think we can use RPM's Obsoletes to do this (untested), and I'm unsure of the equivalent for Debian packages.

#9 Updated by Ken Dreyer over 3 years ago

I think I've got upgrades working now on the latest version of https://github.com/ceph/ceph/tree/wip-10587-split-servers.

More testing needed.

#10 Updated by Ken Dreyer over 3 years ago

After discussion on ceph-devel, I'm going to go with the plan to re-structure ceph into a meta-package, and have "ceph-base" be the common package that all servers will share.

#11 Updated by Ken Dreyer over 3 years ago

To clarify the work that remains to be done here:

  1. The "wip-10587-split-servers" branch needs to be rebased onto the tip of master (to account for any ceph.spec.in and debian/* changes that have occurred since I last got a chance to look at it)
  2. Verify that the debian/* stuff is correct - at the moment I don't fully remember whether all the debian/* changes are complete or whether there are further things to be done there
  3. Run some Teuthology upgrade tests with that branch, going from hammer -> that branch (to be sure that we've got all the upgrade paths covered)

#12 Updated by Chengshun Xia almost 3 years ago

Hi,Ken
Has the split work done for YUM, today when i try to deploy hammer ceph on rhel6, i notice the following error

[root@c596n19 ceph_deploy]# ceph-deploy install c596n20 c596n22 c596n23 c596n24
....

[c596n20][DEBUG ] : jenkins local-rhels6.5-x86_64-HighAvailability
[c596n20][DEBUG ] : local-rhels6.5-x86_64-LoadBalancer
[c596n20][DEBUG ] : local-rhels6.5-x86_64-ResilientStorage
[c596n20][DEBUG ] : local-rhels6.5-x86_64-ScalableFileSystem
[c596n20][DEBUG ] : local-rhels6.5-x86_64-Server local-rhels6.5-x86_64-x86_64
[c596n20][DEBUG ] : mirror.centos.org_centos_6_os_x86_64_
[c596n20][DEBUG ] : mirrors.kernel.org_fedora-epel_6Server_x86_64_ rhel-source
[c596n20][DEBUG ] : rhel-source-beta
[c596n20][DEBUG ] Cleaning up Everything
[c596n20][INFO ] Running command: yum -y install ceph-osd ceph-mds ceph-mon ceph-radosgw
[c596n20][DEBUG ] Loaded plugins: product-id, security, subscription-manager
[c596n20][WARNIN] This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
[c596n20][WARNIN] Repository ibm-open-platform.ibm.com_repos_Ambari_rhel_6_x86_64_2.1.x_GA_2.1_ is listed more than once in the configuration
[c596n20][DEBUG ] Setting up Install Process
[c596n20][DEBUG ] No package ceph-osd available.
[c596n20][DEBUG ] No package ceph-mds available.
[c596n20][DEBUG ] No package ceph-mon available.
[c596n20][DEBUG ] Package 1:ceph-radosgw-0.94.5-0.el6.x86_64 already installed and latest version
[c596n20][DEBUG ] Nothing to do
[c596n20][INFO ] Running command: ceph --version
[c596n20][DEBUG ] ceph version 0.94.5 (9764da52395923e0b32908d83a9f7304401fee43)
.....

and the repo file is following

[root@c596n20 yum.repos.d]# cat ceph.repo
[ceph]
name=Ceph packages for $basearch
baseurl=http://download.ceph.com/rpm-hammer/el6/$basearch
enabled=1
priority=2
gpgcheck=0
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc

[ceph-noarch]
name=Ceph noarch packages
baseurl=http://download.ceph.com/rpm-hammer/el6/noarch
enabled=1
priority=2
gpgcheck=0
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc

[ceph-source]
name=Ceph source packages
baseurl=http://download.ceph.com/rpm-hammer/el6/SRPMS
enabled=0
priority=2
gpgcheck=0
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
[root@c596n20 yum.repos.d]#

os release :
[root@c596n20 yum.repos.d]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)
[root@c596n20 yum.repos.d]#

Does that mean i need to back to giant release ?

Thanks
Chengshun

#13 Updated by Ken Dreyer almost 3 years ago

  • Status changed from In Progress to Resolved

the final patch was merged to master in https://github.com/ceph/ceph/pull/7746

#14 Updated by Ken Dreyer almost 3 years ago

Chengshun I don't think ceph-deploy will easily deploy upstream Ceph on RHEL (vs CentOS). You might want to open a separate ceph-deploy package for RHEL support.

Also available in: Atom PDF