Bug #58920
openlogrotate - delaycompress and duplicate entry errors
0%
Description
we get couple of logrotate errors and warning regarding open file during logrotate and duplicate entry error
actualy v16.2.10, containerized deployment, combination with ceph-common packages on the same host (for directly available ceph commands for management)
in ceph-common package (`/etc/logrotate.d/ceph-common`), there would be best to add glob `/var/log/ceph/ceph-*.log` instead of `/var/log/ceph/*.log`. this eliminates duplicate error, because of glob `/var/log/ceph/cephadm.log` from /etc/logrotate.d/cephadm (managed by containerized mgr)
adding `delaycompress` (could be added to all logrotate configurations without issues) eliminates errors caused by rotating file that is still open - this is usualy because of running ceph-disk logging to `/var/log/ceph/@fsid@/ceph-volume.log` and active connection from mgr to ceph node and logging to `/var/log/cephadm.log`
if i get some acknowledgement, i could try to prepare fix/pull request to fix these issues.
Updated by Jan Horacek about 1 year ago
sample errror messages:
```
error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/@fsid@/ceph-volume.log.1
```
```
logrotate244801: error: cephadm:2 duplicate log entry for /var/log/ceph/cephadm.log
logrotate244801: error: found error in file cephadm, skipping
```
there is duplicate for this issue: https://tracker.ceph.com/issues/57293
Updated by Manuel Lausch 2 months ago
I opened a PR vor this issue
https://github.com/ceph/ceph/pull/55662
Updated by Jan Horacek 2 months ago
Manuel Lausch wrote:
I opened a PR vor this issue
https://github.com/ceph/ceph/pull/55662
thanks Manuel for taking this.
there is the second issue that could be fixed with glob `/var/log/ceph/ceph-*.log` instead of `/var/log/ceph/*.log`
and couple of days ago i noticed duplicat sighup to daemons. this is another things caused by the postrotate script doing killall to daemons and if this fails, it tries to do another try with pkill and regex. sadly if not all daemons are present, the first part sends the signal to existing ones and the second part then sends it again (noticed in containerized ceph deployment) ... i didn't report this yet and i'm not sure about the way to fix this.
Updated by Manuel Lausch 2 months ago
I think the pattern ceph-*.log is not enough.
there are not onloy ceph-<daemon>.log files but also the ceph.log and the ceph.audit.log
it could be possible to define this three patterns in the general ceph logrotate config. But if there will appear more/further logfiles this could get out of mind.
I am not sure whats the best way to solve this.
Updated by Jan Horacek 2 months ago
i could not comment installation without containers at the moment.
in case of containerized installation, ceph.audit.log and ceph.log are in /var/log/ceph/$FSID/ dir. directly in /var/log/ceph i have only cephadm.log
i'm not sure now if non-containerized installaction (mon/osd/... daemons) depends on logrotate from ceph-common package.