Bug #10430
closedosd/osd_types.cc: 456: FAILED assert(m_seed < old_pg_num)
0%
Description
wip-9780-9781 (5a812b2eaa28c05937f615511e9f57b8c8c8e467)
dzafman-2014-12-22_19:17:47-rados:thrash-wip-9780-9781-testing-basic-multi/673097
2014-12-22T22:10:09.606 INFO:teuthology.orchestra.run.plana08:Running: 'sudo ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --log-file=/var/log/ceph/objectstore_tool.\\$pid.log --op import --file /home/ubuntu/cephtest/data/exp.1.2a.2'
2014-12-22T22:10:10.194 INFO:teuthology.orchestra.run.plana08.stdout:Importing pgid 1.2a
2014-12-22T22:10:10.194 INFO:teuthology.orchestra.run.plana08.stdout:Write c3c2202a/benchmark_data_plana08_28611_object1134/head//1
2014-12-22T22:10:10.264 INFO:teuthology.orchestra.run.plana08.stdout:Write df72302a/benchmark_data_plana08_28611_object1073/head//1
2014-12-22T22:10:10.321 INFO:teuthology.orchestra.run.plana08.stdout:Write 7ab0b62a/benchmark_data_plana08_28611_object1273/head//1
2014-12-22T22:10:10.403 INFO:teuthology.orchestra.run.plana08.stdout:Write a311d82a/benchmark_data_plana08_28611_object1003/head//1
2014-12-22T22:10:10.460 INFO:teuthology.orchestra.run.plana08.stdout:Write f9431a2a/benchmark_data_plana08_28611_object1274/head//1
2014-12-22T22:10:10.516 INFO:teuthology.orchestra.run.plana08.stdout:Write 776aac2a/benchmark_data_plana08_28611_object869/head//1
2014-12-22T22:10:10.574 INFO:teuthology.orchestra.run.plana08.stdout:Write 33bdc26a/benchmark_data_plana08_28611_object379/head//1
2014-12-22T22:10:10.634 INFO:teuthology.orchestra.run.plana08.stdout:Write 230936a/benchmark_data_plana08_28611_object1099/head//1
2014-12-22T22:10:10.723 INFO:teuthology.orchestra.run.plana08.stdout:Write c1a1a46a/benchmark_data_plana08_28611_object592/head//1
2014-12-22T22:10:10.781 INFO:teuthology.orchestra.run.plana08.stdout:Write b8efe56a/benchmark_data_plana08_28611_object1144/head//1
2014-12-22T22:10:10.838 INFO:teuthology.orchestra.run.plana08.stdout:Write d49ed86a/benchmark_data_plana08_28611_object1059/head//1
2014-12-22T22:10:10.951 INFO:teuthology.orchestra.run.plana08.stdout:Write 2e48a6a/benchmark_data_plana08_28611_object533/head//1
2014-12-22T22:10:11.090 INFO:teuthology.orchestra.run.plana08.stdout:Write 771dcc6a/benchmark_data_plana08_28611_object924/head//1
2014-12-22T22:10:11.171 INFO:teuthology.orchestra.run.plana08.stdout:Write debcd0aa/benchmark_data_plana08_28611_object264/head//1
2014-12-22T22:10:11.665 INFO:teuthology.orchestra.run.plana08.stdout:Write a16ad6aa/benchmark_data_plana08_28611_object689/head//1
2014-12-22T22:10:11.779 INFO:teuthology.orchestra.run.plana08.stdout:Write bbbcd6aa/benchmark_data_plana08_28611_object840/head//1
2014-12-22T22:10:11.891 INFO:teuthology.orchestra.run.plana08.stdout:Write 985d5aaa/benchmark_data_plana08_28611_object201/head//1
2014-12-22T22:10:11.978 INFO:teuthology.orchestra.run.plana08.stdout:Write f497aaa/benchmark_data_plana08_28611_object688/head//1
2014-12-22T22:10:12.075 INFO:teuthology.orchestra.run.plana08.stdout:Write de0d00ea/benchmark_data_plana08_28611_object783/head//1
2014-12-22T22:10:12.134 INFO:teuthology.orchestra.run.plana08.stdout:Write bafac0ea/benchmark_data_plana08_28611_object1288/head//1
2014-12-22T22:10:12.192 INFO:teuthology.orchestra.run.plana08.stdout:Write ad6d51ea/benchmark_data_plana08_28611_object716/head//1
2014-12-22T22:10:12.249 INFO:teuthology.orchestra.run.plana08.stdout:Write 3381c2ea/benchmark_data_plana08_28611_object1030/head//1
2014-12-22T22:10:12.305 INFO:teuthology.orchestra.run.plana08.stdout:Write 4719b3ea/benchmark_data_plana08_28611_object333/head//1
2014-12-22T22:10:12.363 INFO:teuthology.orchestra.run.plana08.stdout:Write 1390f3ea/benchmark_data_plana08_28611_object666/head//1
2014-12-22T22:10:12.418 INFO:teuthology.orchestra.run.plana08.stdout:Write c658e6ea/benchmark_data_plana08_28611_object40/head//1
2014-12-22T22:10:12.477 INFO:teuthology.orchestra.run.plana08.stdout:Write 6b1169ea/benchmark_data_plana08_28611_object355/head//1
2014-12-22T22:10:12.532 INFO:teuthology.orchestra.run.plana08.stdout:Write 7c44dea/benchmark_data_plana08_28611_object583/head//1
2014-12-22T22:10:13.124 INFO:teuthology.orchestra.run.plana08.stdout:done, clearing removal flag flag
2014-12-22T22:10:13.190 INFO:teuthology.orchestra.run.plana08.stdout:Import successful
ceph-osd.2.log
2014-12-22 22:10:31.143835 7f6a11ddd900 0 osd.2 61 load_pgs opened 46 pgs
2014-12-22 22:10:31.143837 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.1b(unlocked)] _calc_past_interval_range: already have past intervals back to 23
2014-12-22 22:10:31.143843 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.1a(unlocked)] _calc_past_interval_range start epoch 61 >= end epoch 60, nothing to do
2014-12-22 22:10:31.143847 7f6a11ddd900 10 osd.2 pg_epoch: 49 pg[1.1a(unlocked)] _calc_past_interval_range: already have past intervals back to 29
2014-12-22 22:10:31.143851 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.1f(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143855 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.1e(unlocked)] _calc_past_interval_range: already have past intervals back to 42
2014-12-22 22:10:31.143859 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.1f(unlocked)] _calc_past_interval_range: already have past intervals back to 61
2014-12-22 22:10:31.143863 7f6a11ddd900 10 osd.2 pg_epoch: 48 pg[0.20(unlocked)] _calc_past_interval_range start epoch 48 >= end epoch 47, nothing to do
2014-12-22 22:10:31.143867 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.20(unlocked)] _calc_past_interval_range: already have past intervals back to 50
2014-12-22 22:10:31.143871 7f6a11ddd900 10 osd.2 pg_epoch: 36 pg[0.21(unlocked)] _calc_past_interval_range start epoch 36 >= end epoch 34, nothing to do
2014-12-22 22:10:31.143875 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.23(unlocked)] _calc_past_interval_range: already have past intervals back to 57
2014-12-22 22:10:31.143879 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.22(unlocked)] _calc_past_interval_range start epoch 57 >= end epoch 41, nothing to do
2014-12-22 22:10:31.143883 7f6a11ddd900 10 osd.2 pg_epoch: 48 pg[0.23(unlocked)] _calc_past_interval_range: already have past intervals back to 48
2014-12-22 22:10:31.143886 7f6a11ddd900 10 osd.2 pg_epoch: 59 pg[1.25(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143890 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.27(unlocked)] _calc_past_interval_range: already have past intervals back to 45
2014-12-22 22:10:31.143894 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.26(unlocked)] _calc_past_interval_range: already have past intervals back to 33
2014-12-22 22:10:31.143898 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.2b(unlocked)] _calc_past_interval_range: already have past intervals back to 22
2014-12-22 22:10:31.143902 7f6a11ddd900 10 osd.2 61 1.2a needs 31-61
2014-12-22 22:10:31.143905 7f6a11ddd900 10 osd.2 pg_epoch: 51 pg[0.2b(unlocked)] _calc_past_interval_range: already have past intervals back to 50
2014-12-22 22:10:31.143909 7f6a11ddd900 10 osd.2 pg_epoch: 59 pg[1.2d(unlocked)] _calc_past_interval_range: already have past intervals back to 37
2014-12-22 22:10:31.143912 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.2c(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143916 7f6a11ddd900 10 osd.2 pg_epoch: 55 pg[0.2f(unlocked)] _calc_past_interval_range start epoch 54 >= end epoch 52, nothing to do
2014-12-22 22:10:31.143920 7f6a11ddd900 10 osd.2 pg_epoch: 50 pg[0.1(unlocked)] _calc_past_interval_range start epoch 50 >= end epoch 49, nothing to do
2014-12-22 22:10:31.143924 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.0(unlocked)] _calc_past_interval_range: already have past intervals back to 46
2014-12-22 22:10:31.143932 7f6a11ddd900 10 osd.2 pg_epoch: 59 pg[0.3(unlocked)] _calc_past_interval_range: already have past intervals back to 59
2014-12-22 22:10:31.143935 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.5(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143939 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.7(unlocked)] _calc_past_interval_range: already have past intervals back to 45
2014-12-22 22:10:31.143943 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.6(unlocked)] _calc_past_interval_range: already have past intervals back to 33
2014-12-22 22:10:31.143947 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.37(unlocked)] _calc_past_interval_range: already have past intervals back to 61
2014-12-22 22:10:31.143951 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.38(unlocked)] _calc_past_interval_range start epoch 61 >= end epoch 60, nothing to do
2014-12-22 22:10:31.143955 7f6a11ddd900 10 osd.2 pg_epoch: 45 pg[0.9(unlocked)] _calc_past_interval_range start epoch 45 >= end epoch 44, nothing to do
2014-12-22 22:10:31.143959 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.b(unlocked)] _calc_past_interval_range: already have past intervals back to 22
2014-12-22 22:10:31.143965 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.3a(unlocked)] _calc_past_interval_range start epoch 61 >= end epoch 60, nothing to do
2014-12-22 22:10:31.143969 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.a(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143973 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.d(unlocked)] _calc_past_interval_range: already have past intervals back to 37
2014-12-22 22:10:31.143977 7f6a11ddd900 10 osd.2 pg_epoch: 56 pg[1.c(unlocked)] _calc_past_interval_range: already have past intervals back to 31
2014-12-22 22:10:31.143981 7f6a11ddd900 10 osd.2 pg_epoch: 49 pg[1.e(unlocked)] _calc_past_interval_range: already have past intervals back to 36
2014-12-22 22:10:31.143985 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[0.f(unlocked)] _calc_past_interval_range start epoch 53 >= end epoch 52, nothing to do
2014-12-22 22:10:31.143989 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.3f(unlocked)] _calc_past_interval_range: already have past intervals back to 61
2014-12-22 22:10:31.143992 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.10(unlocked)] _calc_past_interval_range: already have past intervals back to 61
2014-12-22 22:10:31.143996 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.14(unlocked)] _calc_past_interval_range: already have past intervals back to 19
2014-12-22 22:10:31.144000 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.17(unlocked)] _calc_past_interval_range: already have past intervals back to 33
2014-12-22 22:10:31.144004 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.16(unlocked)] _calc_past_interval_range: already have past intervals back to 39
2014-12-22 22:10:31.144008 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.17(unlocked)] _calc_past_interval_range: already have past intervals back to 61
2014-12-22 22:10:31.144011 7f6a11ddd900 10 osd.2 pg_epoch: 54 pg[1.19(unlocked)] _calc_past_interval_range: already have past intervals back to 14
2014-12-22 22:10:31.144015 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[0.18(unlocked)] _calc_past_interval_range start epoch 61 >= end epoch 60, nothing to do
2014-12-22 22:10:31.144019 7f6a11ddd900 10 osd.2 pg_epoch: 61 pg[1.18(unlocked)] _calc_past_interval_range: already have past intervals back to 51
2014-12-22 22:10:31.144023 7f6a11ddd900 1 osd.2 61 build_past_intervals_parallel over 31-61
2014-12-22 22:10:31.144025 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 31
2014-12-22 22:10:31.144029 7f6a11ddd900 20 osd.2 0 get_map 31 - loading and decoding 0x4d79e00
2014-12-22 22:10:31.144034 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641719cc/osdmap.31/0//-1 0~0
2014-12-22 22:10:31.144095 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641719cc/osdmap.31/0//-1 0~5424/5424
2014-12-22 22:10:31.144101 7f6a11ddd900 10 osd.2 0 add_map_bl 31 5424 bytes
2014-12-22 22:10:31.144195 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 31 pg 1.2a first map, acting [2,5] up [2,5], same_interval_since = 31
2014-12-22 22:10:31.144202 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 32
2014-12-22 22:10:31.144205 7f6a11ddd900 20 osd.2 0 get_map 32 - loading and decoding 0x4d79c00
2014-12-22 22:10:31.144210 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/64171e9c/osdmap.32/0//-1 0~0
2014-12-22 22:10:31.144261 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/64171e9c/osdmap.32/0//-1 0~5424/5424
2014-12-22 22:10:31.144266 7f6a11ddd900 10 osd.2 0 add_map_bl 32 5424 bytes
2014-12-22 22:10:31.144354 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 33
2014-12-22 22:10:31.144359 7f6a11ddd900 20 osd.2 0 get_map 33 - loading and decoding 0x4d79a00
2014-12-22 22:10:31.144364 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/64171e2c/osdmap.33/0//-1 0~0
2014-12-22 22:10:31.144410 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/64171e2c/osdmap.33/0//-1 0~5424/5424
2014-12-22 22:10:31.144415 7f6a11ddd900 10 osd.2 0 add_map_bl 33 5424 bytes
2014-12-22 22:10:31.144501 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 34
2014-12-22 22:10:31.144505 7f6a11ddd900 20 osd.2 0 get_map 34 - loading and decoding 0x4d79800
2014-12-22 22:10:31.144510 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/64171ffc/osdmap.34/0//-1 0~0
2014-12-22 22:10:31.144558 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/64171ffc/osdmap.34/0//-1 0~5424/5424
2014-12-22 22:10:31.144564 7f6a11ddd900 10 osd.2 0 add_map_bl 34 5424 bytes
2014-12-22 22:10:31.144649 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 35
2014-12-22 22:10:31.144654 7f6a11ddd900 20 osd.2 0 get_map 35 - loading and decoding 0x4d79600
2014-12-22 22:10:31.144658 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/64171c8c/osdmap.35/0//-1 0~0
2014-12-22 22:10:31.144705 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/64171c8c/osdmap.35/0//-1 0~5685/5685
2014-12-22 22:10:31.144710 7f6a11ddd900 10 osd.2 0 add_map_bl 35 5685 bytes
2014-12-22 22:10:31.144808 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 36
2014-12-22 22:10:31.144822 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 37
2014-12-22 22:10:31.144827 7f6a11ddd900 20 osd.2 0 get_map 37 - loading and decoding 0x511f000
2014-12-22 22:10:31.144832 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/64171dec/osdmap.37/0//-1 0~0
2014-12-22 22:10:31.144881 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/64171dec/osdmap.37/0//-1 0~5685/5685
2014-12-22 22:10:31.144887 7f6a11ddd900 10 osd.2 0 add_map_bl 37 5685 bytes
2014-12-22 22:10:31.144986 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 38
2014-12-22 22:10:31.144991 7f6a11ddd900 20 osd.2 0 get_map 38 - loading and decoding 0x511ee00
2014-12-22 22:10:31.144996 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641712bc/osdmap.38/0//-1 0~0
2014-12-22 22:10:31.145042 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641712bc/osdmap.38/0//-1 0~5598/5598
2014-12-22 22:10:31.145048 7f6a11ddd900 10 osd.2 0 add_map_bl 38 5598 bytes
2014-12-22 22:10:31.145148 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 39
2014-12-22 22:10:31.145153 7f6a11ddd900 20 osd.2 0 get_map 39 - loading and decoding 0x511ec00
2014-12-22 22:10:31.145158 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417124c/osdmap.39/0//-1 0~0
2014-12-22 22:10:31.145204 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417124c/osdmap.39/0//-1 0~5598/5598
2014-12-22 22:10:31.145210 7f6a11ddd900 10 osd.2 0 add_map_bl 39 5598 bytes
2014-12-22 22:10:31.145302 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 40
2014-12-22 22:10:31.145307 7f6a11ddd900 20 osd.2 0 get_map 40 - loading and decoding 0x511ea00
2014-12-22 22:10:31.145311 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641710ac/osdmap.40/0//-1 0~0
2014-12-22 22:10:31.145359 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641710ac/osdmap.40/0//-1 0~5598/5598
2014-12-22 22:10:31.145372 7f6a11ddd900 10 osd.2 0 add_map_bl 40 5598 bytes
2014-12-22 22:10:31.145465 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 41
2014-12-22 22:10:31.145474 7f6a11ddd900 20 osd.2 0 get_map 41 - loading and decoding 0x511e800
2014-12-22 22:10:31.145481 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417107c/osdmap.41/0//-1 0~0
2014-12-22 22:10:31.145530 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417107c/osdmap.41/0//-1 0~5743/5743
2014-12-22 22:10:31.145540 7f6a11ddd900 10 osd.2 0 add_map_bl 41 5743 bytes
2014-12-22 22:10:31.145639 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 42
2014-12-22 22:10:31.145648 7f6a11ddd900 20 osd.2 0 get_map 42 - loading and decoding 0x511e600
2014-12-22 22:10:31.145654 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417110c/osdmap.42/0//-1 0~0
2014-12-22 22:10:31.145702 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417110c/osdmap.42/0//-1 0~5772/5772
2014-12-22 22:10:31.145712 7f6a11ddd900 10 osd.2 0 add_map_bl 42 5772 bytes
2014-12-22 22:10:31.145810 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 43
2014-12-22 22:10:31.145819 7f6a11ddd900 20 osd.2 0 get_map 43 - loading and decoding 0x511e400
2014-12-22 22:10:31.145824 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641716dc/osdmap.43/0//-1 0~0
2014-12-22 22:10:31.145870 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641716dc/osdmap.43/0//-1 0~5772/5772
2014-12-22 22:10:31.145880 7f6a11ddd900 10 osd.2 0 add_map_bl 43 5772 bytes
2014-12-22 22:10:31.145980 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 44
2014-12-22 22:10:31.145988 7f6a11ddd900 20 osd.2 0 get_map 44 - loading and decoding 0x511e200
2014-12-22 22:10:31.145994 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417166c/osdmap.44/0//-1 0~0
2014-12-22 22:10:31.146040 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417166c/osdmap.44/0//-1 0~5772/5772
2014-12-22 22:10:31.146050 7f6a11ddd900 10 osd.2 0 add_map_bl 44 5772 bytes
2014-12-22 22:10:31.146145 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 45
2014-12-22 22:10:31.146164 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 46
2014-12-22 22:10:31.146167 7f6a11ddd900 20 osd.2 0 get_map 46 - loading and decoding 0x511e000
2014-12-22 22:10:31.146172 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641714cc/osdmap.46/0//-1 0~0
2014-12-22 22:10:31.146221 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641714cc/osdmap.46/0//-1 0~5772/5772
2014-12-22 22:10:31.146231 7f6a11ddd900 10 osd.2 0 add_map_bl 46 5772 bytes
2014-12-22 22:10:31.146324 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 47
2014-12-22 22:10:31.146333 7f6a11ddd900 20 osd.2 0 get_map 47 - loading and decoding 0x5120400
2014-12-22 22:10:31.146339 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417159c/osdmap.47/0//-1 0~0
2014-12-22 22:10:31.146387 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417159c/osdmap.47/0//-1 0~5076/5076
2014-12-22 22:10:31.146397 7f6a11ddd900 10 osd.2 0 add_map_bl 47 5076 bytes
2014-12-22 22:10:31.146519 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 48
2014-12-22 22:10:31.146537 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 49
2014-12-22 22:10:31.146556 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 49 pg 1.2a generate_past_intervals interval(31-48 up [2,5](2) acting [2,5](2)): not rw, up_thru 47 up_from 3 last_epoch_clean 31
generate_past_intervals interval(31-48 up [2,5](2) acting [2,5](2) maybe_went_rw) : primary up 3-47 includes interval
2014-12-22 22:10:31.146560 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 50
2014-12-22 22:10:31.146571 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 51
2014-12-22 22:10:31.146581 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 52
2014-12-22 22:10:31.146584 7f6a11ddd900 20 osd.2 0 get_map 52 - loading and decoding 0x5120200
2014-12-22 22:10:31.146589 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/641729bc/osdmap.52/0//-1 0~0
2014-12-22 22:10:31.146637 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/641729bc/osdmap.52/0//-1 0~5076/5076
2014-12-22 22:10:31.146642 7f6a11ddd900 10 osd.2 0 add_map_bl 52 5076 bytes
2014-12-22 22:10:31.146737 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 53
2014-12-22 22:10:31.146747 7f6a11ddd900 20 osd.2 0 get_map 53 - loading and decoding 0x5120000
2014-12-22 22:10:31.146753 7f6a11ddd900 15 filestore(/var/lib/ceph/osd/ceph-2) read meta/6417294c/osdmap.53/0//-1 0~0
2014-12-22 22:10:31.146805 7f6a11ddd900 10 filestore(/var/lib/ceph/osd/ceph-2) FileStore::read meta/6417294c/osdmap.53/0//-1 0~5076/5076
2014-12-22 22:10:31.146814 7f6a11ddd900 10 osd.2 0 add_map_bl 53 5076 bytes
2014-12-22 22:10:31.146904 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 54
2014-12-22 22:10:31.146921 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 55
2014-12-22 22:10:31.146931 7f6a11ddd900 10 osd.2 61 build_past_intervals_parallel epoch 56
2014-12-22 22:10:31.151226 7f6a11ddd900 -1 osd/osd_types.cc: In function 'bool pg_t::is_split(unsigned int, unsigned int, std::set<pg_t>*) const' thread 7f6a11ddd900 time 2014-12-22 22:10:31.146942
osd/osd_types.cc: 456: FAILED assert(m_seed < old_pg_num)
ceph version 0.90-667-g9c8fc7f (9c8fc7f2a505f6e2f94995068bd3307f9392ae3a)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xb9a79b]
2: ceph-osd() [0x78be11]
3: (pg_interval_t::is_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, unsigned int, unsigned int, pg_t)+0xcd) [0x78beed]
4: (pg_interval_t::is_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, std::tr1::shared_ptr<OSDMap const>, std::tr1::shared_ptr<OSDMap const>, pg_t)+0x1a5) [0x78c0a5]
5: (pg_interval_t::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::tr1::shared_ptr<OSDMap const>, std::tr1::shared_ptr<OSDMap const>, pg_t, std::map<unsigned int, pg_interval_t, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > >, std::ostream)+0x101) [0x793891]
6: (OSD::build_past_intervals_parallel()+0xa5e) [0x69e98e]
7: (OSD::load_pgs()+0x1c23) [0x6b62c3]
8: (OSD::init()+0x71f) [0x6b78bf]
9: (main()+0x2805) [0x644795]
10: (__libc_start_main()+0xf5) [0x7f6a0ef22ec5]
11: ceph-osd() [0x65d477]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.Updated by Josh Durgin over 9 years ago
possibly a similar problem in the objecter: #10399
Updated by David Zafman over 9 years ago
- Status changed from 7 to Fix Under Review
Updated by David Zafman over 9 years ago
- Status changed from Fix Under Review to Resolved
Updated by Loïc Dachary about 9 years ago
- Status changed from Resolved to Pending Backport
- Backport set to giant
Updated by Loïc Dachary about 9 years ago
- Severity changed from 3 - minor to 2 - major
Updated by Loïc Dachary about 9 years ago
- Backport changed from giant to firefly
giant is end of life
Updated by Loïc Dachary about 9 years ago
91b2aca osd: Get pgid ancestor from last_map when building past intervals (in firefly),
Updated by Loïc Dachary about 9 years ago
- Status changed from Pending Backport to Resolved
Updated by Yuri Weinstein about 9 years ago
- Status changed from Resolved to New
Still see in
Run: http://pulpito.ceph.com/teuthology-2015-03-28_17:05:02-upgrade:giant-x-hammer-distro-basic-multi/
Job: ['825911']
Logs: http://pulpito.ceph.com/teuthology-2015-03-28_17:05:02-upgrade:giant-x-hammer-distro-basic-multi/825911/
Assertion: osd/osd_types.cc: 411: FAILED assert(m_seed < old_pg_num) ceph version 0.87.1-101-g90b37d9 (90b37d9bdcc044e26f978632cd68f19ece82d19a) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0x7f05097f0e0f] 2: (()+0x310232) [0x7f05098b5232] 3: (pg_interval_t::is_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, unsigned int, unsigned int, pg_t)+0xe4) [0x7f05098b5324] 4: (Objecter::_calc_target(Objecter::op_target_t*, bool)+0x965) [0x7f050bc2c035] 5: (Objecter::_recalc_linger_op_target(Objecter::LingerOp*, RWLock::Context&)+0x59) [0x7f050bc3acc9] 6: (Objecter::_scan_requests(Objecter::OSDSession*, bool, bool, std::map<unsigned long, Objecter::Op*, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, Objecter::Op*> > >&, std::list<Objecter::LingerOp*, std::allocator<Objecter::LingerOp*> >&, std::map<unsigned long, Objecter::CommandOp*, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, Objecter::CommandOp*> > >&)+0x1cf) [0x7f050bc3b0df] 7: (Objecter::handle_osd_map(MOSDMap*)+0x5f6) [0x7f050bc418a6] 8: (Objecter::ms_dispatch(Message*)+0x257) [0x7f050bc47dd7] 9: (Messenger::ms_deliver_dispatch(Message*)+0x77) [0x7f05099021d7] 10: (DispatchQueue::entry()+0x44a) [0x7f05098ff37a] 11: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f0509975ccd] 12: (()+0x7e9a) [0x7f050938fe9a] 13: (clone()+0x6d) [0x7f05089a13fd]
Updated by Loïc Dachary about 9 years ago
- giant backport https://github.com/ceph/ceph/pull/4214
Updated by Loïc Dachary about 9 years ago
- Backport changed from firefly to firefly,giant
Updated by Loïc Dachary about 9 years ago
- Status changed from New to Pending Backport
Updated by Loïc Dachary about 9 years ago
The backport seems straightforward: it is basically the same in Firefly and reading it I can't see how it could cause problems in giant.
Updated by David Zafman about 9 years ago
- Status changed from Pending Backport to Resolved