Project

General

Profile

Bug #1493

cmon: nice error message on undecodable (osdmap, monmap) input

Added by Anonymous almost 10 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Monitor
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

from a mkcephfs -a run:

pushing everything to sepia24
 ** WARNING: Ceph is still under development.  Any feedback can be directed  **
 **          at ceph-devel@vger.kernel.org or http://ceph.newdream.net/.     **
2011-09-02 14:54:27.972594 7f96b948d720 AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/mon.a.asok': error 2: No such file or directory
2011-09-02 14:54:27.972612 7f96b948d720 AdminSocketConfigObs: failed to start AdminSocket
terminate called after throwing an instance of 'ceph::buffer::end_of_buffer'
  what():  buffer::end_of_buffer
*** Caught signal (Aborted) **
 in thread 0x7f96b948d720
 ceph version 0.34 (commit:2f039eeeb745622b866d80feda7afa055e15f6d6)
 1: /usr/bin/cmon() [0x519a94]
 2: (()+0xfb40) [0x7f96b906cb40]
 3: (gsignal()+0x35) [0x7f96b7a45ba5]
 4: (abort()+0x180) [0x7f96b7a496b0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f96b82e96bd]
 6: (()+0xb9906) [0x7f96b82e7906]
 7: (()+0xb9933) [0x7f96b82e7933]
 8: (()+0xb9a3e) [0x7f96b82e7a3e]
 9: (ceph::buffer::list::iterator::copy(unsigned int, char*)+0x13e) [0x50995e]
 10: (CrushWrapper::decode(ceph::buffer::list::iterator&)+0x44) [0x4a4b44]
 11: (OSDMap::decode(ceph::buffer::list&)+0x798) [0x4a5b48]
 12: (OSDMonitor::create_initial(ceph::buffer::list&)+0x18b) [0x48a1ab]
 13: (Monitor::mkfs(ceph::buffer::list&)+0x9ce) [0x46544e]
 14: (main()+0x53d) [0x4541bd]
 15: (__libc_start_main()+0xfe) [0x7f96b7a30d8e]
 16: /usr/bin/cmon() [0x4532a9]
Aborted (core dumped)
failed: 'ssh root@sepia24 mkcephfs -d /tmp/mkfs.ceph.2673 --init-daemon mon.a'
root@sepia24:/root# gdb /usr/bin/cmon core.cmon.2011-09-02 
GNU gdb (GDB) 7.2-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying" 
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/cmon...Reading symbols from /usr/lib/debug/usr/bin/cmon...done.
done.
[New Thread 4130]
[New Thread 4131]

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libcrypto++.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libcrypto++.so.8
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /usr/lib/libtcmalloc.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libtcmalloc.so.0
Reading symbols from /usr/lib/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib/libunwind.so.7...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libunwind.so.7
Core was generated by `/usr/bin/cmon -c /tmp/mkfs.ceph.2673/conf --mkfs -i a --monmap /tmp/mkfs.ceph.2'.
Program terminated with signal 6, Aborted.
#0  0x00007f96b906ca0b in raise () from /lib/libpthread.so.0
(gdb) bt
#0  0x00007f96b906ca0b in raise () from /lib/libpthread.so.0
#1  0x0000000000519703 in reraise_fatal (signum=4130) at global/signal_handler.cc:59
#2  0x0000000000519ccc in handle_fatal_signal (signum=<value optimized out>)
    at global/signal_handler.cc:106
#3  <signal handler called>
#4  0x00007f96b7a45ba5 in raise () from /lib/libc.so.6
#5  0x00007f96b7a496b0 in abort () from /lib/libc.so.6
#6  0x00007f96b82e96bd in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6
#7  0x00007f96b82e7906 in ?? () from /usr/lib/libstdc++.so.6
#8  0x00007f96b82e7933 in std::terminate() () from /usr/lib/libstdc++.so.6
#9  0x00007f96b82e7a3e in __cxa_throw () from /usr/lib/libstdc++.so.6
#10 0x000000000050995e in copy_out (this=0x7fff50b6cf90, len=3, dest=0x7fff50b6ce5d "\177")
    at ./include/buffer.h:185
#11 ceph::buffer::list::iterator::copy (this=0x7fff50b6cf90, len=3, dest=0x7fff50b6ce5d "\177")
    at common/buffer.cc:492
#12 0x00000000004a4b44 in CrushWrapper::decode(ceph::buffer::list::iterator&) ()
---Type <return> to continue, or q <return> to quit---
#13 0x00000000004a5b48 in OSDMap::decode(ceph::buffer::list&) ()
#14 0x000000000048a1ab in OSDMonitor::create_initial (this=0x2728800, bl=...)
    at mon/OSDMonitor.cc:62

#15 0x000000000046544e in Monitor::mkfs (this=0x7fff50b6e540, osdmapbl=<value optimized out>)
    at mon/Monitor.cc:1082
#16 0x00000000004541bd in main (argc=<value optimized out>, argv=<value optimized out>)
    at cmon.cc:120
(gdb) 

reproduces nicely when i run

root@sepia24:/root# /usr/bin/cmon -c bork/conf --mkfs -i a --monmap bork/monmap --osdmap bork/osdmap -k bork/keyring.mon

Attaching tarball of bork.

bork.tgz (1.11 KB) Anonymous, 09/02/2011 03:05 PM

Associated revisions

Revision 7eff37be (diff)
Added by Sage Weil over 9 years ago

mon: validate osmdap input

And clean up some error return paths while we're here.

Fixes: #1493
Signed-off-by: Sage Weil <>

History

#1 Updated by Anonymous almost 10 years ago

Diagnosis from Sage: version mismatch

Host running mkcephfs -a:

$ ./ceph --version
ceph version 0.34-326-gcc1cb8d (commit:cc1cb8da7e4b4e5e0dec64638976c8dd442807fc)

Target cluster:

$ ceph --version
ceph version 0.34 (commit:2f039eeeb745622b866d80feda7afa055e15f6d6)

#2 Updated by Sage Weil almost 10 years ago

  • Subject changed from cmon --mkfs throws ceph::buffer::end_of_buffer in CrushWrapper::decode to cmon: nice error message on undecodable (osdmap, monmap) input
  • Category changed from 10 to Monitor
  • Target version set to v0.36

#3 Updated by Sage Weil almost 10 years ago

  • translation missing: en.field_position set to 1
  • translation missing: en.field_position changed from 1 to 891

#4 Updated by Sage Weil almost 10 years ago

  • translation missing: en.field_story_points set to 2
  • translation missing: en.field_position deleted (899)
  • translation missing: en.field_position set to 898

#5 Updated by Sage Weil almost 10 years ago

  • Target version changed from v0.36 to v0.37
  • translation missing: en.field_position deleted (899)
  • translation missing: en.field_position set to 10

#6 Updated by Sage Weil almost 10 years ago

  • Target version deleted (v0.37)

#7 Updated by Sage Weil over 9 years ago

  • Status changed from New to Resolved
  • Assignee set to Sage Weil
  • Target version set to v0.42

Also available in: Atom PDF