Ceph » devops

(fingers were too fast...)

Colocating rados gateway and osd on the same host, ceph-common and ceph-base are installed as dependencies.

ceph-common installed /etc/logrotate.d/ceph-common, ceph-base contains /etc/logrotate.d/ceph.logrotate. Both target "/var/log/ceph/*.log", which breaks logrotate since the same files are targeted by multiple entries.

Proposed solution(s):

- provide specific logrotate entries with specific packages (e.g. /var/log/ceph/ceph-osd* in ceph-osd package)
- move log rotation to own package

Ceph version: 10.2.7-1xenial
OS: Ubuntu 16.04.2

revival of http://tracker.ceph.com/issues/15569 ?

kchai@circle:~$ dpkg -c ceph-common_10.2.7-239-g966f222-1xenial_amd64.deb | grep logrotate
kchai@circle:~$ dpkg -c ceph-base_10.2.7-239-g966f222-1xenial_amd64.deb | grep logrotate
drwxr-xr-x root/root         0 2017-05-17 09:05 ./etc/logrotate.d/
-rw-r--r-- root/root       228 2017-05-17 09:05 ./etc/logrotate.d/ceph.logrotate

why do you think ceph-common installed /etc/logrotate.d/ceph-common? is it possible that it was installed by previous installation? and from where/how exactly did you download these package?

The files are reported as belonging to the ceph-base and ceph-common packages:

root@ceph-storage-02:/var/lib/dpkg/info# fgrep logrotate *.list | grep ceph
ceph-base.list:/etc/logrotate.d
ceph-base.list:/etc/logrotate.d/ceph.logrotate
ceph-common.list:/etc/logrotate.d/ceph-common

I use the european ceph mirror:

1. This file is managed by Puppet. DO NOT EDIT.
2. ceph
   deb http://eu.ceph.com/debian-jewel/ xenial main

The host was built from scratch, so there was no previous installation of other ceph packages (neither other version nor other origin). Installation was done using some custom puppet module based on puppet's apt source management and its package management.

kchai@circle:/tmp/19938$ wget -q http://eu.ceph.com/debian-jewel/pool/main/c/ceph/ceph-base_10.2.7-1xenial_amd64.deb
kchai@circle:/tmp/19938$ wget -q http://eu.ceph.com/debian-jewel/pool/main/c/ceph/ceph-common_10.2.7-1xenial_amd64.deb
kchai@circle:/tmp/19938$ dpkg -c ceph-common_10.2.7-1xenial_amd64.deb | grep logrotate
kchai@circle:/tmp/19938$ dpkg -c ceph-base_10.2.7-1xenial_amd64.deb | grep logrotate
drwxr-xr-x root/root         0 2017-04-10 13:23 ./etc/logrotate.d/
-rw-r--r-- root/root       228 2017-04-10 13:23 ./etc/logrotate.d/ceph.logrotate
kchai@circle:/tmp/19938$ sha1sum ceph-common_10.2.7-1xenial_amd64.deb
593fecf7efa426f72bef5b2622b7ba8e34ce4500  ceph-common_10.2.7-1xenial_amd64.deb
kchai@circle:/tmp/19938$ sha1sum ceph-base_10.2.7-1xenial_amd64.deb
ab4b90153d02aaefcfd8b8bfcded8f7df6ccfed7  ceph-base_10.2.7-1xenial_amd64.deb

i am confused. Burkhard, could you post the output of

dpkg -s ceph-common

my wild guess is that you are using a downstream package.

And I am confused, too.

1. dpkg -s ceph-common
   Package: ceph-common
   Status: install ok installed
   Priority: optional
   Section: admin
   Installed-Size: 61313
   Maintainer: Ceph Maintainers <ceph-maintainers@lists.ceph.com>
   Architecture: amd64
   Source: ceph
   Version: 10.2.7-1xenial
   Replaces: ceph (<< 10), ceph-client-tools, ceph-test (<< 9.0.3-1646), librbd1 (<< 0.92-1238), python-ceph (<< 0.92-1223)
   Depends: librbd1 (= 10.2.7-1xenial), init-system-helpers (>= 1.18~), libbabeltrace-ctf1 (>= 1.2.1), libbabeltrace1 (>= 1.2.1), libblkid1 (>= 2.17.2), libboost-iostreams1.58.0, libboost-program-options1.58.0, libboost-regex1.58.0, libboost-system1.58.0, libboost-thread1.58.0, libc6 (>= 2.16), libgcc1 (>= 1:3.0), libgoogle-perftools4, libkeyutils1 (>= 1.4), libnspr4 (>= 2:4.9-2~) | libnspr4-0d (>= 1.8.0.10), libnss3 (>= 2:3.13.4-2~) | libnss3-1d (>= 3.12.9~beta2), librados2, libradosstriper1, librgw2, libstdc++6 (>= 5.2), libudev1 (>= 183), python-rados (= 10.2.7-1xenial), python-cephfs (= 10.2.7-1xenial), python-rbd (= 10.2.7-1xenial), python-requests
   Suggests: ceph-base (= 10.2.7-1xenial), ceph-mds (= 10.2.7-1xenial)
   Breaks: ceph (<< 10), ceph-test (<< 9.0.3-1646), librbd1 (<< 0.92-1238), python-ceph (<< 0.92-1223)
   Conflicts: ceph-client-tools
   Conffiles:
   /etc/bash_completion.d/rados c9fd29e1a807f545e4514f3b1ede7225
   /etc/bash_completion.d/rbd e4fe86afa78419de017f79bbae5760ee
   /etc/ceph/rbdmap 638e8968005eb2b959919f0c5e31e9a4
   /etc/default/ceph 5645b70243d88f951df86f0bdd97dbb0
   /etc/init.d/rbdmap 93e42ffcf65d32ab29dbb5fa6d3caed3
   /etc/init/rbdmap.conf ceceeb4e61bfef80dcc6fc1cbc608aa5
   /etc/logrotate.d/ceph-common 0b02320d237751a26ed35b495495c6e2 obsolete
   Description: common utilities to mount and interact with a ceph storage cluster
   Ceph is a massively scalable, open-source, distributed
   storage system that runs on commodity hardware and delivers object,
   block and file system storage. This is a collection
   of common tools that allow one to interact with and administer a Ceph cluster.
   Homepage: http://ceph.com/

It seems that during host provisioning the distribution package was installed, which contains the rogue logrotate file. The ceph provided packages are fine. The logrotate file is even recognized as being obsolete, but not automatically removed during package upgrade. This is only done if the former package is purged.

I'm not sure what's the right way to handle this (and whether this problem happens often). According to https://raphaelhertzog.com/2010/10/07/the-right-way-to-remove-an-obsolete-conffile-in-a-debian-package/ , a package might be able to remove obsolete conffiles in package postprocessing scripts, but the information is from 2010.

https://github.com/ceph/ceph/pull/15514