Project

General

Profile

Bug #40724

mgr: recursive lock in vstart

Added by Patrick Donnelly 4 months ago. Updated 4 months ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
-
Category:
ceph-mgr
Target version:
Start date:
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

vstart stuck at:

$ env MDS=1 ../src/vstart.sh -d -b -l -n
** going verbose **
rm -f core*
ip 127.0.0.1
port 40331

NOTE: hostname resolves to loopback; remote hosts will not be able to
  connect.  either adjust /etc/hosts, or edit this script to use your
  machine's real IP.

/home/pdonnell/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mon. /home/pdonnell/ceph/build/keyring --cap mon 'allow *'
creating /home/pdonnell/ceph/build/keyring
/home/pdonnell/ceph/build/bin/ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /home/pdonnell/ceph/build/keyring
/home/pdonnell/ceph/build/bin/ceph-authtool --gen-key --name=client.fs --cap mon 'allow r' --cap osd 'allow rw tag cephfs data=*' --cap mds 'allow rwp' /home/pdonnell/ceph/build/keyring
/home/pdonnell/ceph/build/bin/ceph-authtool --gen-key --name=client.rgw --cap mon 'allow rw' --cap osd 'allow rwx' --cap mgr 'allow rw' /home/pdonnell/ceph/build/keyring
/home/pdonnell/ceph/build/bin/monmaptool --create --clobber --addv a [v2:127.0.0.1:40331,v1:127.0.0.1:40332] --addv b [v2:127.0.0.1:40333,v1:127.0.0.1:40334] --addv c [v2:127.0.0.1:40335,v1:127.0.0.1:40336] --print /tmp/ceph_monmap.5728
/home/pdonnell/ceph/build/bin/monmaptool: monmap file /tmp/ceph_monmap.5728
/home/pdonnell/ceph/build/bin/monmaptool: generated fsid f8d12fe4-485f-4784-8edf-add19850b4c6
epoch 0
fsid f8d12fe4-485f-4784-8edf-add19850b4c6
last_changed 2019-07-10T16:29:13.152410-0400
created 2019-07-10T16:29:13.152410-0400
min_mon_release 0 (unknown)
0: [v2:127.0.0.1:40331/0,v1:127.0.0.1:40332/0] mon.a
1: [v2:127.0.0.1:40333/0,v1:127.0.0.1:40334/0] mon.b
2: [v2:127.0.0.1:40335/0,v1:127.0.0.1:40336/0] mon.c
/home/pdonnell/ceph/build/bin/monmaptool: writing epoch 0 to /tmp/ceph_monmap.5728 (3 monitors)
rm -rf -- /home/pdonnell/ceph/build/dev/mon.a
mkdir -p /home/pdonnell/ceph/build/dev/mon.a
/home/pdonnell/ceph/build/bin/ceph-mon --mkfs -c /home/pdonnell/ceph/build/ceph.conf -i a --monmap=/tmp/ceph_monmap.5728 --keyring=/home/pdonnell/ceph/build/keyring
rm -rf -- /home/pdonnell/ceph/build/dev/mon.b
mkdir -p /home/pdonnell/ceph/build/dev/mon.b
/home/pdonnell/ceph/build/bin/ceph-mon --mkfs -c /home/pdonnell/ceph/build/ceph.conf -i b --monmap=/tmp/ceph_monmap.5728 --keyring=/home/pdonnell/ceph/build/keyring
rm -rf -- /home/pdonnell/ceph/build/dev/mon.c
mkdir -p /home/pdonnell/ceph/build/dev/mon.c
/home/pdonnell/ceph/build/bin/ceph-mon --mkfs -c /home/pdonnell/ceph/build/ceph.conf -i c --monmap=/tmp/ceph_monmap.5728 --keyring=/home/pdonnell/ceph/build/keyring
rm -- /tmp/ceph_monmap.5728
/home/pdonnell/ceph/build/bin/ceph-mon -i a -c /home/pdonnell/ceph/build/ceph.conf
/home/pdonnell/ceph/build/bin/ceph-mon -i b -c /home/pdonnell/ceph/build/ceph.conf
/home/pdonnell/ceph/build/bin/ceph-mon -i c -c /home/pdonnell/ceph/build/ceph.conf
Populating config ...
Setting debug configs ...
creating /home/pdonnell/ceph/build/dev/mgr.x/keyring
/home/pdonnell/ceph/build/bin/ceph -c /home/pdonnell/ceph/build/ceph.conf -k /home/pdonnell/ceph/build/keyring -i /home/pdonnell/ceph/build/dev/mgr.x/keyring auth add mgr.x mon 'allow profile mgr' mds 'allow *' osd 'allow *'
added key for mgr.x
/home/pdonnell/ceph/build/bin/ceph -c /home/pdonnell/ceph/build/ceph.conf -k /home/pdonnell/ceph/build/keyring config set mgr mgr/dashboard/x/ssl_server_port 41331 --force
/home/pdonnell/ceph/build/bin/ceph -c /home/pdonnell/ceph/build/ceph.conf -k /home/pdonnell/ceph/build/keyring config set mgr mgr/restful/x/server_port 42331 --force
Starting mgr.x
/home/pdonnell/ceph/build/bin/ceph-mgr -i x -c /home/pdonnell/ceph/build/ceph.conf
/home/pdonnell/ceph/build/bin/ceph -c /home/pdonnell/ceph/build/ceph.conf -k /home/pdonnell/ceph/build/keyring tell mgr dashboard ac-user-create admin admin administrator

   -5> 2019-07-10T16:29:24.554-0400 7f285a4e0d00  1 mgr[py] Loading python module 'ansible'
    -4> 2019-07-10T16:29:24.554-0400 7f285a4e0d00  1 lockdep using id 40
    -3> 2019-07-10T16:29:24.562-0400 7f285a4e0d00 10 mgr[py] Computed sys.path '/home/pdonnell/ceph/src/pybind:/home/pdonnell/ceph/build/lib/cython_modules/lib.2::/usr/lib/python2.7/:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/home/pdonnell/ceph/src/pybind/mgr:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages'
    -2> 2019-07-10T16:29:24.718-0400 7f285a4e0d00  0
recursive lock of  (5)
 ceph version v15.0.0-2597-gac64007 (ac640077da858a521d28036f7d3e7ff1889c347f) octopus (dev)
 1: (ceph::mutex_debug_detail::mutex_debugging_base::_will_lock(bool)+0x4a) [0x7f28511edc3c]
 2: (ceph::shared_mutex_debug::lock()+0x3a) [0x7f285128d16e]
 3: (()+0x1582532) [0x7f285112a532]
 4: (CRYPTO_mem_ctrl()+0x39) [0x7f284da7c6e9]
 5: (()+0x444ee) [0x7f28418ac4ee]
 6: (SSL_COMP_get_compression_methods()+0x9) [0x7f28418ae5f9]
 7: (SSL_library_init()+0x203) [0x7f28418b3d83]
 8: (init_ssl()+0xa2) [0x7f2841ad9c22]
 9: (_PyImport_LoadDynamicModule()+0x97) [0x7f284f773ab7]
 10: (()+0x15a8d0) [0x7f284f7748d0]
 11: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 12: (()+0xb88f8) [0x7f284f6d28f8]
 13: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 14: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 15: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 16: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 17: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 18: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 19: (()+0x15915f) [0x7f284f77315f]
 20: (()+0x15a8d0) [0x7f284f7748d0]
 21: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 22: (()+0xb88f8) [0x7f284f6d28f8]
 23: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 24: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 25: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 26: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 27: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 28: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 29: (()+0x15915f) [0x7f284f77315f]
 30: (()+0x15a8d0) [0x7f284f7748d0]
 31: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 32: (()+0xb88f8) [0x7f284f6d28f8]
 33: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 34: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 35: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 36: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 37: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 38: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 39: (()+0x15915f) [0x7f284f77315f]
 40: (()+0x15a102) [0x7f284f774102]
 41: (()+0x15a8d0) [0x7f284f7748d0]
 42: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 43: (()+0xb88f8) [0x7f284f6d28f8]
 44: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 45: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 46: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 47: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 48: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 49: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 50: (()+0x15915f) [0x7f284f77315f]
 51: (()+0x15a102) [0x7f284f774102]
 52: (()+0x15a8d0) [0x7f284f7748d0]
 53: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 54: (()+0xb88f8) [0x7f284f6d28f8]
 55: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 56: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 57: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 58: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 59: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 60: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 61: (()+0x15915f) [0x7f284f77315f]
 62: (()+0x15a102) [0x7f284f774102]
 63: (()+0x15a8d0) [0x7f284f7748d0]
 64: (PyImport_ImportModuleLevel()+0x62e) [0x7f284f775f7e]
 65: (()+0xb88f8) [0x7f284f6d28f8]
 66: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 67: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 68: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 69: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 70: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 71: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 72: (()+0x15915f) [0x7f284f77315f]
 73: (()+0x15a8d0) [0x7f284f7748d0]
 74: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 75: (()+0xb88f8) [0x7f284f6d28f8]
 76: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 77: (PyEval_CallObjectWithKeywords()+0x47) [0x7f284f801547]
 78: (PyEval_EvalFrameEx()+0x38c6) [0x7f284f6c7826]
 79: (PyEval_EvalCodeEx()+0x85c) [0x7f284f80211c]
 80: (PyEval_EvalCode()+0x19) [0x7f284f6c3de9]
 81: (PyImport_ExecCodeModuleEx()+0x94) [0x7f284f772784]
 82: (()+0x15915f) [0x7f284f77315f]
 83: (()+0x15a102) [0x7f284f774102]
 84: (()+0x15a8d0) [0x7f284f7748d0]
 85: (PyImport_ImportModuleLevel()+0x1da) [0x7f284f775b2a]
 86: (()+0xb88f8) [0x7f284f6d28f8]
 87: (PyObject_Call()+0x43) [0x7f284f72b2b3]
 88: (PyObject_CallFunction()+0xcf) [0x7f284f72c69f]
 89: (PyImport_Import()+0x116) [0x7f284f776536]

With 9aa1eeb37a9c344e34fa513498230cc8765a45b0.


Related issues

Duplicates mgr - Bug #40698: mgr: openSSL SSL_library_init() failed to initialize Resolved 07/09/2019

History

#1 Updated by Sebastian Wagner 4 months ago

  • Duplicates Bug #40698: mgr: openSSL SSL_library_init() failed to initialize added

#2 Updated by Sebastian Wagner 4 months ago

  • Status changed from New to Duplicate

Also available in: Atom PDF