teuthology: set up log rotate for MDS logs
There might be some stuff around log rotation already in teuthology. If not, it shouldn't be hard to add handling.
We'd like to set this up so that we can rotate logs very frequently and compress them on rotation, so as to avoid the disk space issues we've been seeing in typica.
#1 Updated by Greg Farnum over 4 years ago
- Status changed from New to Verified
- Assignee set to Greg Farnum
We explicitly disable logrotate in ceph-qa-suite/tasks/ceph.py, because of #5451.
I think maybe the answer is to keep that, create a ceph logrotate script that rotates based on log size, and then manually invoke that regularly in a greenlet as part of the log task? :/
#3 Updated by Greg Farnum over 4 years ago
Yes, we could also stop the rotation after the tests execute and before archiving — but that doesn't actually solve our problem. We want to logrotate on a rapid basis (many times per test), but by default logrotate is invoked daily via crontab. We could perhaps change the crontab for it back and forth, but we'd still need a custom logrotate.d/mds.conf and maintaining the changes seems even more unpleasant than simply launching it ourselves with the appropriate config file (witness the troubles that have been had with the ssh_keys task).