Project

General

Profile

Backport #44868

Updated by Nathan Cutler about 4 years ago

https://github.com/ceph/ceph/pull/34326 My environment: 
 [root@gz-ceph-52-205 ceph]# cat /etc/redhat-release 
 CentOS Linux release 7.2.1511 (Core) 
 [root@gz-ceph-52-205 ceph]# ceph -v 
 ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable) 

 i built a monitor system by prometheus,now it seem ceph-mgr's prometheus module got troubles: 

 2018-07-18 05:56:20.006383 7fdc4d3a6700 -1 *** Caught signal (Segmentation fault) ** 
  in thread 7fdc4d3a6700 thread_name:prometheus 

  ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable) 
  1: (()+0x3f0c81) [0x7fdc6f014c81] 
  2: (()+0xf100) [0x7fdc6c7d3100] 
  3: (ceph::buffer::ptr::ptr(ceph::buffer::ptr const&)+0x17) [0x7fdc6f016777] 
  4: (std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::operator=(std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> > const&)+0xe9) [0x7fdc6eef3a49] 
  5: (ceph::buffer::list::operator=(ceph::buffer::list const&)+0x16) [0x7fdc6eef3af6] 
  6: (ActivePyModules::get_python(std::string const&)+0x12d1) [0x7fdc6eef0411] 
  7: (()+0x2ea7cd) [0x7fdc6ef0e7cd] 
  8: (PyEval_EvalFrameEx()+0x4594) [0x7fdc6e71aaa4] 
  9: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  10: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  11: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  12: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  13: (()+0x6f05d) [0x7fdc6e6a905d] 
  14: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  15: (PyEval_EvalFrameEx()+0xde7) [0x7fdc6e7172f7] 
  16: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  17: (()+0x6f05d) [0x7fdc6e6a905d] 
  18: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  19: (()+0x590a5) [0x7fdc6e6930a5] 
  20: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  21: (()+0xa13f7) [0x7fdc6e6db3f7] 
  22: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  23: (PyEval_EvalFrameEx()+0xde7) [0x7fdc6e7172f7] 
  24: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  25: (()+0x6ef68) [0x7fdc6e6a8f68] 
  26: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  27: (()+0x590a5) [0x7fdc6e6930a5] 
  28: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  29: (()+0x55448) [0x7fdc6e68f448] 
  30: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  31: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  32: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  33: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  34: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  35: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  36: (()+0x6ef68) [0x7fdc6e6a8f68] 
  37: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  38: (()+0x590a5) [0x7fdc6e6930a5] 
  39: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  40: (()+0xa1057) [0x7fdc6e6db057] 
  41: (()+0x9fd6f) [0x7fdc6e6d9d6f] 
  42: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  43: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  44: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  45: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  46: (()+0x6f05d) [0x7fdc6e6a905d] 
  47: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  48: (()+0x590a5) [0x7fdc6e6930a5] 
  49: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  50: (()+0xa13f7) [0x7fdc6e6db3f7] 
  51: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  52: (PyEval_EvalFrameEx()+0xde7) [0x7fdc6e7172f7] 
  53: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  54: (PyEval_EvalFrameEx()+0x425f) [0x7fdc6e71a76f] 
  55: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  56: (()+0x6ef68) [0x7fdc6e6a8f68] 
  57: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  58: (()+0x590a5) [0x7fdc6e6930a5] 
  59: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  60: (()+0xa1057) [0x7fdc6e6db057] 
  61: (()+0x9fd6f) [0x7fdc6e6d9d6f] 
  62: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  63: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  64: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  65: (()+0x6ef68) [0x7fdc6e6a8f68] 
  66: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  67: (()+0x590a5) [0x7fdc6e6930a5] 
  68: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  69: (()+0xa13f7) [0x7fdc6e6db3f7] 
  70: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  71: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  72: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  73: (()+0x6ef68) [0x7fdc6e6a8f68] 
  74: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  75: (()+0x590a5) [0x7fdc6e6930a5] 
  76: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  77: (()+0xa13f7) [0x7fdc6e6db3f7] 
  78: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  79: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  80: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  81: (()+0x6ef68) [0x7fdc6e6a8f68] 
  82: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  83: (()+0x590a5) [0x7fdc6e6930a5] 
  84: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  85: (()+0xa13f7) [0x7fdc6e6db3f7] 
  86: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  87: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  88: (PyEval_EvalCodeEx()+0x7ed) [0x7fdc6e71c0bd] 
  89: (()+0x6ef68) [0x7fdc6e6a8f68] 
  90: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  91: (()+0x590a5) [0x7fdc6e6930a5] 
  92: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  93: (()+0xa13f7) [0x7fdc6e6db3f7] 
  94: (PyObject_Call()+0x43) [0x7fdc6e6840b3] 
  95: (PyEval_EvalFrameEx()+0x1d4c) [0x7fdc6e71825c] 
  96: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  97: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  98: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  99: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  100: (PyEval_EvalFrameEx()+0x4350) [0x7fdc6e71a860] 
  NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 

 Thanks to god,this cluster is not my online-environment,but it still scare me -.-

Back