Feature #11588
closedteuthology: set up log rotate for MDS logs
0%
Description
There might be some stuff around log rotation already in teuthology. If not, it shouldn't be hard to add handling.
We'd like to set this up so that we can rotate logs very frequently and compress them on rotation, so as to avoid the disk space issues we've been seeing in typica.
Updated by Greg Farnum almost 9 years ago
- Status changed from New to 12
- Assignee set to Greg Farnum
We explicitly disable logrotate in ceph-qa-suite/tasks/ceph.py, because of #5451.
I think maybe the answer is to keep that, create a ceph logrotate script that rotates based on log size, and then manually invoke that regularly in a greenlet as part of the log task? :/
Updated by John Spray almost 9 years ago
Perhaps disabling rotation for #5451 was just a quick fix, would it work make teuthology allow rotation, but stop all the relevant daemons before it tries to copy the files back? It seems a bit layers-on-layers to have teuthology do its own log rotation
Updated by Greg Farnum almost 9 years ago
Yes, we could also stop the rotation after the tests execute and before archiving — but that doesn't actually solve our problem. We want to logrotate on a rapid basis (many times per test), but by default logrotate is invoked daily via crontab. We could perhaps change the crontab for it back and forth, but we'd still need a custom logrotate.d/mds.conf and maintaining the changes seems even more unpleasant than simply launching it ourselves with the appropriate config file (witness the troubles that have been had with the ssh_keys task).
Updated by Greg Farnum almost 9 years ago
- Status changed from 12 to Resolved