Bug #12416
The cthulhu should have the thead clear the database and log.
0%
Description
Now the calamari always write data to the postgresql and write log to /var/log/calamari
Maybe after a long time ,there are a lot of no use data in postgressql and a lot of no use log in /var/log/calamari.
The cthulhu should have a thread to delete the no use data in postgresql and no use log in /var/log/calamari.
It can be configed in the /etc/calamari/cthulhu.conf how long data or log we keep.
History
#1 Updated by Dan Mick over 8 years ago
Log file rotation should be set up and configured with logrotate. I don't think the postgres data grows without bound. Do you have any evidence that it's growing?
#2 Updated by ceph zte over 8 years ago
The cthulhu_event in my machine is now have so many data.
And the cthulhu_event table will keep on growing.
I think we should have a thead to clear the table , maybe some day the table is full.
calamari=# select count(*) from cthulhu_event; -----
| count |-----
| 37059 |-----
1 row in set
5 Calamari server started
5 Added server song with 3 OSDs, 1 monitor service bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Server song is late reporting in, last report at 2015-07-03 20:10:30.719684+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
4 Server song regained contact bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:12:39.820499+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
5 Calamari server started
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
5 Added server song with 3 OSDs, 1 monitor service bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:16:32.149754+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
4 Server song regained contact bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:18:40.387047+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
4 Server song regained contact bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:20:51.821198+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
4 Server song regained contact bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:23:01.438075+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
4 Server song regained contact bc253614-d8aa-48ab-8b75-996e152d1222 song
4 Cluster 'ceph' regained contact bc253614-d8aa-48ab-8b75-996e152d1222
3 Server song is late reporting in, last report at 2015-07-03 20:25:13.126374+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
5 Calamari server started
5 Added server song with 3 OSDs, 1 monitor service bc253614-d8aa-48ab-8b75-996e152d1222 song
3 Server song is late reporting in, last report at 2015-07-03 20:55:35.872227+00:00 bc253614-d8aa-48ab-8b75-996e152d1222 song
5 Calamari server started
3 Cluster 'ceph' is late reporting in bc253614-d8aa-48ab-8b75-996e152d1222
5 Added server song with 3 OSDs, 1 monitor service bc253614-d8aa-48ab-8b75-996e152d1222 song
#3 Updated by Christina Meno over 8 years ago
Thanks for the report!
#4 Updated by ceph zte over 8 years ago
I have write the clear log and evevt table code.
class ClearlHistory(gevent.greenlet.Greenlet):
def init(self):
super(ClearlHistory, self).__init__()
self._complete = gevent.event.Event()
def stop(self):
self._complete.set()
def clearlog(self):
logfile = os.listdir("/var/log/calamari/")
from datetime import date
for elem in logfile:
m = re.search(r"gz", elem)
if m:
history = elem[-11:-3]
histtime = date(int(history[0:4]), int(history[4:6]), int(history[6:8]))
diff = (date.today() - histtime).days
if int(diff) > 14:
file = "rm /var/log/calamari/" + elem
os.system(file)
def cleardatabase(self):
if sqlalchemy is not None:
try:
# Prepare persistence
engine = create_engine(config.get('cthulhu', 'db_path'))
Session.configure(bind=engine)
except sqlalchemy.exc.ArgumentError as e:
log.error("Database error: %s" % e)
raise
try:
from calamari_common.db.event import Event
from dateutil import tz
import datetime
session = Session()
now = datetime.datetime.utcnow().replace(tzinfo=tz.tzutc())
now = now + datetime.timedelta(days=-90)
session.query(Event).filter(Event.when<now).delete()
object_types = [row[0] for row in session.query(SyncObject.sync_type).distinct()]
for sync in object_types:
session.query(SyncObject).filter_by(sync_type=sync).filter(SyncObject.when<now).delete()
session.commit()
except:
session.rollback()
def Run(self):
self.clearlog()
self.cleardatabase()