Bug #3773
closedmds crashed at LogEvent::decode
0%
Description
ceph version: 0.56.1 (e4a541624df62ef353e754391cbbb707f54b16f7)
I had a cluster [burnupi06, burnupi07, burnupi08] running on ceph version 0.56 and upgraded them one by one to v0.56.1 while pumping IO from the client.
mds crashed when it was upgraded and restarted.
0> 2013-01-09 11:35:36.314638 7fa65d038700 -1 mds/LogEvent.cc: In function 'static LogEvent* LogEvent::decode(ceph::bufferlist&)' thread 7fa65d038700 time 2013-01-09 11:35:34.715866
mds/LogEvent.cc: 95: FAILED assert(p.end())
ceph version 0.56.1 (e4a541624df62ef353e754391cbbb707f54b16f7)
1: (LogEvent::decode(ceph::buffer::list&)+0x9ff) [0x6b308f]
2: (MDLog::_replay_thread()+0x2d8) [0x69e678]
3: (MDLog::ReplayThread::entry()+0xd) [0x4c7c5d]
4: (()+0x7e9a) [0x7fa664f2ae9a]
5: (clone()+0x6d) [0x7fa663dcbcbd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
ubuntu@burnupi06:~$ sudo cat /etc/ceph/ceph.conf
[global]
auth client required = cephx
auth cluster required = cephx
auth service required = cephx
debug ms = 1
[osd]
osd journal size = 1000
filestore xattr use omap = true
debug osd = 20
[osd.1]
host = burnupi06
[osd.2]
host = burnupi06
[osd.3]
host = burnupi07
[osd.4]
host = burnupi07
[osd.5]
host = burnupi08
[osd.6]
host = burnupi08
[mon.a]
host = burnupi06
mon addr = 10.214.133.8:6789
[mon.b]
host = burnupi07
mon addr = 10.214.134.38:6789
[mon.c]
host = burnupi08
mon addr = 10.214.134.36:6789
[mds.a]
host = burnupi08
[client.radosgw.gateway]
host = burnupi06
keyring = /etc/ceph/keyring.radosgw.gateway
rgw socket path = /tmp/radosgw.sock
log file = /var/log/ceph/radosgw.log
leaving the cluster state as it is.