Bug #4747
closedUpgrade monitors from argonaut->bobtail->next fails w/"Existing store has not been converted to 0.52 format"
0%
Description
Testing upgrade from Argonaut to Bobtail to Next (cuttlefish). I am using Argonaut and bobtail distros on gitbuilder. The initial cluster creation is done via mkcephfs. on the final upgrade from bobtail 0.56.4 to Next I get the following output from sudo service ceph -a restart:
=== mon.a ===
=== mon.a ===
Stopping Ceph mon.a on burnupi57...kill 53032...done
=== mon.a ===
Starting Ceph mon.a on burnupi57...
starting mon.a rank 2 at 10.214.136.16:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid dc9f30aa-9c2b-4311-9206-94a6a2de25ac
=== mon.b ===
=== mon.b ===
Stopping Ceph mon.b on burnupi63...kill 23393...done
=== mon.b ===
Starting Ceph mon.b on burnupi63...
starting mon.b rank 0 at 10.214.136.4:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid dc9f30aa-9c2b-4311-9206-94a6a2de25ac
=== mon.c ===
=== mon.c ===
Stopping Ceph mon.c on burnupi59...kill 18264...done
=== mon.c ===
Starting Ceph mon.c on burnupi59...
Invalid argument: /var/lib/ceph/mon/ceph-c/store.db: does not exist (create_if_missing is false)
mon/Monitor.cc: In function 'int Monitor::StoreConverter::needs_conversion()' thread 7f2032224780 time 2013-04-18 11:02:12.444351
mon/Monitor.cc: 4147: FAILED assert(0 "Existing store has not been converted to 0.52 format")
ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
2: (main()+0x7a1) [0x48a671]
3: (__libc_start_main()+0xed) [0x7f203054976d]
4: /usr/bin/ceph-mon() [0x48dc2d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2013-04-18 11:02:12.444761 7f2032224780 -1 mon/Monitor.cc: In function 'int Monitor::StoreConverter::needs_conversion()' thread 7f2032224780 time 2013-04-18 11:02:12.444351
mon/Monitor.cc: 4147: FAILED assert(0 "Existing store has not been converted to 0.52 format")
ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
2: (main()+0x7a1) [0x48a671]
3: (__libc_start_main()+0xed) [0x7f203054976d]
4: /usr/bin/ceph-mon() [0x48dc2d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.0> 2013-04-18 11:02:12.444761 7f2032224780 -1 mon/Monitor.cc: In function 'int Monitor::StoreConverter::needs_conversion()' thread 7f2032224780 time 2013-04-18 11:02:12.444351
mon/Monitor.cc: 4147: FAILED assert(0 "Existing store has not been converted to 0.52 format")ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
2: (main()+0x7a1) [0x48a671]
3: (__libc_start_main()+0xed) [0x7f203054976d]
4: /usr/bin/ceph-mon() [0x48dc2d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.- Caught signal (Aborted)
in thread 7f2032224780
ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: /usr/bin/ceph-mon() [0x590eba]
2: (()+0xfcb0) [0x7f2031e06cb0]
3: (gsignal()+0x35) [0x7f203055e425]
4: (abort()+0x17b) [0x7f2030561b8b]
5: (_gnu_cxx::_verbose_terminate_handler()+0x11d) [0x7f2030eb069d]
6: (()+0xb5846) [0x7f2030eae846]
7: (()+0xb5873) [0x7f2030eae873]
8: (()+0xb596e) [0x7f2030eae96e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649a9f]
10: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
11: (main()+0x7a1) [0x48a671]
12: (__libc_start_main()+0xed) [0x7f203054976d]
13: /usr/bin/ceph-mon() [0x48dc2d]
2013-04-18 11:02:12.445418 7f2032224780 -1 Caught signal (Aborted) *
in thread 7f2032224780
ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: /usr/bin/ceph-mon() [0x590eba]
2: (()+0xfcb0) [0x7f2031e06cb0]
3: (gsignal()+0x35) [0x7f203055e425]
4: (abort()+0x17b) [0x7f2030561b8b]
5: (_gnu_cxx::_verbose_terminate_handler()+0x11d) [0x7f2030eb069d]
6: (()+0xb5846) [0x7f2030eae846]
7: (()+0xb5873) [0x7f2030eae873]
8: (()+0xb596e) [0x7f2030eae96e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649a9f]
10: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
11: (main()+0x7a1) [0x48a671]
12: (__libc_start_main()+0xed) [0x7f203054976d]
13: /usr/bin/ceph-mon() [0x48dc2d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.0> 2013-04-18 11:02:12.445418 7f2032224780 -1 ** Caught signal (Aborted) *
in thread 7f2032224780ceph version 0.60-525-g7e4f80b (7e4f80b12e86d0da9cedc1569c63d78cd27bb8ed)
1: /usr/bin/ceph-mon() [0x590eba]
2: (()+0xfcb0) [0x7f2031e06cb0]
3: (gsignal()+0x35) [0x7f203055e425]
4: (abort()+0x17b) [0x7f2030561b8b]
5: (_gnu_cxx::_verbose_terminate_handler()+0x11d) [0x7f2030eb069d]
6: (()+0xb5846) [0x7f2030eae846]
7: (()+0xb5873) [0x7f2030eae873]
8: (()+0xb596e) [0x7f2030eae96e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649a9f]
10: (Monitor::StoreConverter::needs_conversion()+0x77e) [0x4a41ee]
11: (main()+0x7a1) [0x48a671]
12: (__libc_start_main()+0xed) [0x7f203054976d]
13: /usr/bin/ceph-mon() [0x48dc2d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.bash: line 1: 19264 Aborted (core dumped) /usr/bin/ceph-mon -i c --pid-file /var/run/ceph/mon.c.pid -c /tmp/ceph.conf.4af6179d7498b7b9da2f9c74dea5c1d1
failed: 'ssh burnupi59 ulimit -n 8192; /usr/bin/ceph-mon -i c --pid-file /var/run/ceph/mon.c.pid -c /tmp/ceph.conf.4af6179d7498b7b9da2f9c74dea5c1d1 '
= mds.a ===
=== mds.a ===
Stopping Ceph mds.a on burnupi57...kill 53621...done
=== mds.a ===
Starting Ceph mds.a on burnupi57...
starting mds.a at :/0
=== osd.0 ===
=== osd.0 ===
Stopping Ceph osd.0 on burnupi57...kill 53849...done
=== osd.0 ===
Starting Ceph osd.0 on burnupi57...
starting osd.0 at :/0 osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal
=== osd.1 ===
=== osd.1 ===
Stopping Ceph osd.1 on burnupi59...kill 18533...done
=== osd.1 ===
Starting Ceph osd.1 on burnupi59...
starting osd.1 at :/0 osd_data /var/lib/ceph/osd/ceph-1 /var/lib/ceph/osd/ceph-1/journal
=== osd.2 ===
=== osd.2 ===
Stopping Ceph osd.2 on burnupi63...kill 23636...done
=== osd.2 ===
Starting Ceph osd.2 on burnupi63...
starting osd.2 at :/0 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
ceph.conf:
[global]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx[osd]
osd journal size = 1000
filestore xattr use omap = true#osd mkfs type = {fs-type}
#osd mkfs options {fs-type} = {mkfs options} # default for xfs is "-f"
#osd mount options {fs-type} = {mount options} # default mount option is "rw,noatime"- For example, for ext4, the mount option might look like this:
#osd mkfs options ext4 = user_xattr,rw,noatime[mon.a]
host = burnupi57
mon addr = 10.214.136.16:6789
[mon.b]host = burnupi63
mon addr = 10.214.136.4:6789
[mon.c]host = burnupi59
mon addr = 10.214.136.12:6789[osd.0]
host = burnupi57
#devs = {path-to-device}
[osd.1]
host = burnupi59
#devs = {path-to-device}
[osd.2]
host = burnupi63
#devs = {path-to-device}
[mds.a]
host = burnupi57
[client.radosgw.gateway]
host = burnupi57
keyring = /etc/ceph/keyring.radosgw.gateway
rgw socket path = /tmp/radosgw.sock
log file = /var/log/ceph/radosgw.log
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024
rgw usage max shards = 32
rgw usage max user shards = 1