Project

General

Profile

Bug #2646

mon:update_from_paxos: error parsing incremental update: buffer::end_of_buffer

Added by Yehuda Sadeh almost 12 years ago. Updated over 11 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Monitor
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

   -7> 2012-06-22 00:36:45.912018 7f0cbe00a700  1 -- [2607:f298:4:2243::5754]:6789/0 >> :/0 pipe(0x5147500 sd=17 pgs=0 cs=0 l=0).accept sd=17
    -6> 2012-06-22 00:39:08.677789 7f0c9d3d1700  1 -- [2607:f298:4:2243::5754]:6789/0 --> mon.1 [2607:f298:4:2243::5753]:6789/0 -- mon_probe(slurp 31b8be2f-ac05-4e56-96b7-e702df166e29 name peon5754 machine_name pgmap 3116382-3116545) v1 -- ?+0 0x15a13900
    -5> 2012-06-22 00:39:09.044030 7f0c9d3d1700  1 -- [2607:f298:4:2243::5754]:6789/0 <== mon.1 [2607:f298:4:2243::5753]:6789/0 3092 ==== mon_probe(data 31b8be2f-ac05-4e56-96b7-e702df166e29 name peon5753 machine_name pgmap 3116591-3117092) v1 ==== 274258+0+0 (3632809312 0 0) 0x15a13900 con 0x9066c80
    -4> 2012-06-22 00:39:42.918877 7f0cbe00a700  1 -- [2607:f298:4:2243::5754]:6789/0 >> :/0 pipe(0x5e8f500 sd=18 pgs=0 cs=0 l=0).accept sd=18
    -3> 2012-06-22 00:39:44.523942 7f0c9d3d1700  1 -- [2607:f298:4:2243::5754]:6789/0 --> mon.1 [2607:f298:4:2243::5753]:6789/0 -- mon_probe(slurp 31b8be2f-ac05-4e56-96b7-e702df166e29 name peon5754 machine_name pgmap 3116591-3116632) v1 -- ?+0 0x15a13900
    -2> 2012-06-22 00:39:44.534313 7f0c9d3d1700  1 -- [2607:f298:4:2243::5754]:6789/0 <== osd.0 [2607:f298:4:2243::6236]:6800/3992 1 ==== auth(proto 0 26 bytes epoch 3) v1 ==== 56+0+0 (452516168 0 0) 0x62fa200 con 0x54c0b40
    -1> 2012-06-22 00:39:44.812095 7f0c9cbd0700  0 mon.peon5754@2(slurping).pg v3116545 update_from_paxos: error parsing incremental update: buffer::end_of_buffer
     0> 2012-06-22 00:39:44.875370 7f0c9cbd0700 -1 mon/PGMonitor.cc: In function 'virtual void PGMonitor::update_from_paxos()' thread 7f0c9cbd0700 time 2012-06-22 00:39:44.832068
mon/PGMonitor.cc: 181: FAILED assert(0 == "update_from_paxos: error parsing incremental update")

 ceph version 0.46-95-g66b2bb3 (commit:66b2bb339ed59198dfd622279c3671f7304329c1)
 1: (PGMonitor::update_from_paxos()+0xdd4) [0x4ef274]
 2: (PGMonitor::tick()+0x56) [0x4e7896]
 3: (Monitor::tick()+0x5d) [0x481c7d]
 4: (SafeTimer::timer_thread()+0x34a) [0x5cdb2a]
 5: (SafeTimerThread::entry()+0xd) [0x5ce6fd]
 6: (()+0x7e9a) [0x7f0cbdbe6e9a]
 7: (clone()+0x6d) [0x7f0cbc39f4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- end dump of recent events ---
2012-06-22 00:39:50.365160 7f0c9cbd0700 -1 *** Caught signal (Aborted) **
 in thread 7f0c9cbd0700

 ceph version 0.46-95-g66b2bb3 (commit:66b2bb339ed59198dfd622279c3671f7304329c1)
 1: /usr/bin/ceph-mon() [0x61cd4a]
 2: (()+0xfcb0) [0x7f0cbdbeecb0]
 3: (gsignal()+0x35) [0x7f0cbc2e3445]
 4: (abort()+0x17b) [0x7f0cbc2e6bab]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f0cbcc3169d]
 6: (()+0xb5846) [0x7f0cbcc2f846]
 7: (()+0xb5873) [0x7f0cbcc2f873]
 8: (()+0xb596e) [0x7f0cbcc2f96e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x282) [0x5d5ed2]
 10: (PGMonitor::update_from_paxos()+0xdd4) [0x4ef274]
 11: (PGMonitor::tick()+0x56) [0x4e7896]
 12: (Monitor::tick()+0x5d) [0x481c7d]
 13: (SafeTimer::timer_thread()+0x34a) [0x5cdb2a]
 14: (SafeTimerThread::entry()+0xd) [0x5ce6fd]
 15: (()+0x7e9a) [0x7f0cbdbe6e9a]
 16: (clone()+0x6d) [0x7f0cbc39f4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2012-06-22 00:39:50.365160 7f0c9cbd0700 -1 *** Caught signal (Aborted) **
 in thread 7f0c9cbd0700

 ceph version 0.46-95-g66b2bb3 (commit:66b2bb339ed59198dfd622279c3671f7304329c1)
 1: /usr/bin/ceph-mon() [0x61cd4a]
 2: (()+0xfcb0) [0x7f0cbdbeecb0]
 3: (gsignal()+0x35) [0x7f0cbc2e3445]
 4: (abort()+0x17b) [0x7f0cbc2e6bab]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f0cbcc3169d]
 6: (()+0xb5846) [0x7f0cbcc2f846]
 7: (()+0xb5873) [0x7f0cbcc2f873]
 8: (()+0xb596e) [0x7f0cbcc2f96e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x282) [0x5d5ed2]
 10: (PGMonitor::update_from_paxos()+0xdd4) [0x4ef274]
 11: (PGMonitor::tick()+0x56) [0x4e7896]
 12: (Monitor::tick()+0x5d) [0x481c7d]
 13: (SafeTimer::timer_thread()+0x34a) [0x5cdb2a]
 14: (SafeTimerThread::entry()+0xd) [0x5ce6fd]
 15: (()+0x7e9a) [0x7f0cbdbe6e9a]
 16: (clone()+0x6d) [0x7f0cbc39f4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- end dump of recent events ---

Related issues

Duplicates Ceph - Feature #2651: mon: race calling tick() when doing slurping Rejected 06/25/2012

History

#1 Updated by Sage Weil over 11 years ago

  • Status changed from New to Resolved

#2 Updated by Greg Farnum over 11 years ago

840ae244499496d543d634713bdee7c7884ce527

The tick happened at the same time as slurping, which meant the disk was inconsistent and the pg_map it was trying to read didn't exist on the disk.

Also available in: Atom PDF