Ceph

The wiki procedure at http://ceph.newdream.net/wiki/Monitor_cluster_expansion does not work. It crashes with:

10.05.10_21:44:11.214934 7f1f4697f710 mon3(starting) e2 initializing monmap
10.05.10_21:44:11.214963 7f1f4697f710 mon3(starting).paxos(monmap recovering lc 0) init
10.05.10_21:44:11.214976 7f1f4697f710 mon3(starting).monmap v2 create_pending monmap epoch 3
10.05.10_21:44:11.214988 7f1f4697f710 mon3(starting).monmap v2 create_initial was fed epoch 2
10.05.10_21:44:11.214995 7f1f4697f710 mon3(starting).monmap v2 encode_pending epoch 2
mon/MonmapMonitor.cc: In function 'virtual void MonmapMonitor::encode_pending(ceph::bufferlist&)':
mon/MonmapMonitor.cc:77: FAILED assert(mon->monmap->epoch + 1 == pending_map.epoch || pending_map.epoch == 1)
 1: (MonmapMonitor::encode_pending(ceph::buffer::list&)+0x9a) [0x60f83e]
 2: (Monitor::mkfs(ceph::buffer::list&)+0x463) [0x59c1bd]
 3: (main()+0x40c) [0x595f88]
 4: (__libc_start_main()+0xfd) [0x7f1f456d6abd]
 5: (std::ios_base::Init::~Init()+0x49) [0x5959d9]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
mon/MonmapMonitor.cc: In function 'virtual void MonmapMonitor::encode_pending(ceph::bufferlist&)':
mon/MonmapMonitor.cc:77: FAILED assert(mon->monmap->epoch + 1 == pending_map.epoch || pending_map.epoch == 1)
 1: (MonmapMonitor::encode_pending(ceph::buffer::list&)+0x9a) [0x60f83e]
 2: (Monitor::mkfs(ceph::buffer::list&)+0x463) [0x59c1bd]
 3: (main()+0x40c) [0x595f88]
 4: (__libc_start_main()+0xfd) [0x7f1f456d6abd]
 5: (std::ios_base::Init::~Init()+0x49) [0x5959d9]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Reported by Paul Chiang:

Date: Tue, 11 May 2010 09:53:05 +0800
From: Paul Chiang <paul_chiang@tcloudcomputing.com>
To: ceph-devel@vger.kernel.org
Subject: mkmonfs error

Hello,

I'm getting an error with mkmonfs while trying to expand the monitor
cluster as per the instructions on the wiki.

The error seems to originate from line 951~970 of Monitor.cc

Suppose we start with map epoch 3:
1. create_pending() increments the epoch to 4
2. create_initial() then sets it back to 3
3. encode_pending() finds 3+1 != 3, throws an error and dies

Not sure if this is a bug or there's something wrong with my proceedure.

Thanks in advance,
Paul Chiang

P.S. I'm using the unstable branch