Project

General

Profile

Bug #250

mon: delete old states to avoid filling disk

Added by Sage Weil almost 11 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Monitor
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

We trim the pgmap (which gets the most activity), but not the others: logs, mdsmap, osdmap, etc. Unfortunately the others are trickier. mds may be safe, but for the osdmap the clients/osds currently needs a full history from where they are to present in order to behave. Need to be careful about throwing out old states. Maybe a min age would help?

History

#1 Updated by Sage Weil almost 11 years ago

  • Target version set to 12

#2 Updated by Wido den Hollander about 10 years ago

I keep running into this, my monitors keep running out of inodes which causes them to stall.

Right now I have cron's like:

@daily find /var/lib/ceph/mon.logger/logm -type f -ctime +7 -delete

pgmap already gets trimmed, but the logm doesn't.

I was thinking about a cron which runs every day and ships with the Ceph packages, would that be OK? Or do you prefer to implement this in the monitor itself?

#3 Updated by Greg Farnum about 10 years ago

We should probably just do this in the monitor. The logm is never read so it's safe to delete whenever, unlike the other maps we're concerned about.

#4 Updated by Sage Weil about 10 years ago

log trimming fixed by 4c1cb28cb6cbb3d23a1736d02f6af524481c886a

osdmap, mdsmap, monmap still need trimming, but that's a whole 'nother thing.

#5 Updated by Sage Weil almost 10 years ago

  • Target version deleted (12)

#6 Updated by Sage Weil over 9 years ago

  • Target version set to v0.38
  • translation missing: en.field_position deleted (569)
  • translation missing: en.field_position set to 958

#7 Updated by Sage Weil over 9 years ago

  • Status changed from New to Resolved
  • translation missing: en.field_position deleted (958)
  • translation missing: en.field_position set to 958

#8 Updated by Sage Weil over 9 years ago

  • translation missing: en.field_position deleted (958)
  • translation missing: en.field_position set to 958

Also available in: Atom PDF