Project

General

Profile

Feature #11588

teuthology: set up log rotate for MDS logs

Added by Greg Farnum over 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Testing
Target version:
-
Start date:
05/11/2015
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
Labels (FS):
Pull request ID:

Description

There might be some stuff around log rotation already in teuthology. If not, it shouldn't be hard to add handling.

We'd like to set this up so that we can rotate logs very frequently and compress them on rotation, so as to avoid the disk space issues we've been seeing in typica.

History

#1 Updated by Greg Farnum over 4 years ago

  • Status changed from New to Verified
  • Assignee set to Greg Farnum

We explicitly disable logrotate in ceph-qa-suite/tasks/ceph.py, because of #5451.

I think maybe the answer is to keep that, create a ceph logrotate script that rotates based on log size, and then manually invoke that regularly in a greenlet as part of the log task? :/

#2 Updated by John Spray over 4 years ago

Perhaps disabling rotation for #5451 was just a quick fix, would it work make teuthology allow rotation, but stop all the relevant daemons before it tries to copy the files back? It seems a bit layers-on-layers to have teuthology do its own log rotation

#3 Updated by Greg Farnum over 4 years ago

Yes, we could also stop the rotation after the tests execute and before archiving — but that doesn't actually solve our problem. We want to logrotate on a rapid basis (many times per test), but by default logrotate is invoked daily via crontab. We could perhaps change the crontab for it back and forth, but we'd still need a custom logrotate.d/mds.conf and maintaining the changes seems even more unpleasant than simply launching it ourselves with the appropriate config file (witness the troubles that have been had with the ssh_keys task).

#4 Updated by Greg Farnum about 4 years ago

  • Status changed from Verified to Resolved

Also available in: Atom PDF