Actions
Bug #24077
closedtest_pool_create_fail (tasks.mgr.dashboard.test_pool.PoolTest) fails
% Done:
0%
Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2018-05-10T04:37:48.278 INFO:tasks.mgr.dashboard.helper:request POST to https://mira078.front.sepia.ceph.com:7789/api/auth
2018-05-10T04:37:48.585 INFO:tasks.mgr.dashboard.helper:request POST to https://mira078.front.sepia.ceph.com:7789/api/pool/
2018-05-10T04:37:48.586 INFO:tasks.ceph.mgr.x.mira078.stdout:::ffff:172.21.0.51 - - [10/May/2018:04:37:48] "POST /api/auth HTTP/1.1" 201 21 "" "python-requests/2.12.5"
2018-05-10T04:37:48.597 INFO:tasks.ceph.mgr.x.mira078.stderr:[10/May/2018:04:37:48] HTTP
2018-05-10T04:37:48.597 INFO:tasks.ceph.mgr.x.mira078.stderr:Request Headers:
2018-05-10T04:37:48.597 INFO:tasks.ceph.mgr.x.mira078.stderr: Content-Length: 79
2018-05-10T04:37:48.597 INFO:tasks.ceph.mgr.x.mira078.stderr: COOKIE: session_id=dcd6e8ff5833aefb2f7cdf2b75181269f4d2d1e7
2018-05-10T04:37:48.597 INFO:tasks.ceph.mgr.x.mira078.stderr: HOST: mira078.front.sepia.ceph.com:7789
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: CONNECTION: keep-alive
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: Remote-Addr: ::ffff:172.21.0.51
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: ACCEPT: */*
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: USER-AGENT: python-requests/2.12.5
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: Content-Type: application/json
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr: ACCEPT-ENCODING: gzip, deflate
2018-05-10T04:37:48.598 INFO:tasks.ceph.mgr.x.mira078.stderr:[10/May/2018:04:37:48] HTTP Traceback (most recent call last):
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: response.body = self.handler()
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: self.body = self.oldhandler(*args, **kwargs)
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/python2.7/dist-packages/cherrypy/_cptools.py", line 231, in wrap
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: return self.newhandler(innerfunc, *args, **kwargs)
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/ceph/mgr/dashboard/services/exception.py", line 94, in dashboard_exception_handler
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: return json.dumps(serialize_dashboard_exception(e)).encode('utf-8')
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/ceph/mgr/dashboard/services/exception.py", line 73, in serialize_dashboard_exception
2018-05-10T04:37:48.599 INFO:tasks.ceph.mgr.x.mira078.stderr: out['code'] = e.code
2018-05-10T04:37:48.600 INFO:tasks.ceph.mgr.x.mira078.stderr: File "/usr/lib/ceph/mgr/dashboard/exceptions.py", line 45, in code
2018-05-10T04:37:48.600 INFO:tasks.ceph.mgr.x.mira078.stderr: return str(abs(self.errno))
2018-05-10T04:37:48.600 INFO:tasks.ceph.mgr.x.mira078.stderr:TypeError: bad operand type for abs(): 'NoneType'
2018-05-10T04:37:48.600 INFO:tasks.ceph.mgr.x.mira078.stderr:
2018-05-10T04:37:48.606 INFO:tasks.cephfs_test_runner:test_pool_create_fail (tasks.mgr.dashboard.test_pool.PoolTest) ... FAIL
see /a/kchai-2018-05-09_15:14:14-rados-wip-kefu-testing-2018-05-09-2109-distro-basic-mira/2509998
i suspect it's related to https://github.com/ceph/ceph/pull/21066
Updated by Josh Durgin almost 6 years ago
Looks to have caused a monitor crash:
2018-05-10T04:37:39.127 INFO:tasks.mgr.dashboard.helper:request POST to https://mira078.front.sepia.ceph.com:7789/api/auth 2018-05-10T04:37:39.438 INFO:tasks.mgr.dashboard.helper:request POST to https://mira078.front.sepia.ceph.com:7789/api/pool/ 2018-05-10T04:37:39.440 INFO:tasks.ceph.mgr.x.mira078.stdout:::ffff:172.21.0.51 - - [10/May/2018:04:37:39] "POST /api/auth HTTP/1.1" 201 21 "" "python-requests/2.12.5" 2018-05-10T04:37:43.483 INFO:tasks.ceph.mon.b.mira053.stderr:2018-05-10 04:37:43.477 7f3b27514700 -1 bad boost::get: key val is not type std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr:2018-05-10 04:37:43.481 7f3b27514700 -1 ceph version 13.0.2-2283-gf16a158 (f16a15847fc04a93cacb395c0e34fadc67c985c8) mimic (dev) 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 1: (bool cmd_getval<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(CephContext*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)+0xe3) [0x56040b924f13] 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 2: (OSDMonitor::prepare_command_pool_set(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&)+0x2e4) [0x56040ba746e4] 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 3: (OSDMonitor::prepare_command_impl(boost::intrusive_ptr<MonOpRequest>, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&)+0x1dc8e) [0x56040ba9488e] 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 4: (OSDMonitor::prepare_command(boost::intrusive_ptr<MonOpRequest>)+0x252) [0x56040ba9a742] 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 5: (OSDMonitor::prepare_update(boost::intrusive_ptr<MonOpRequest>)+0x170) [0x56040ba9aaa0] 2018-05-10T04:37:43.487 INFO:tasks.ceph.mon.b.mira053.stderr: 6: (PaxosService::dispatch(boost::intrusive_ptr<MonOpRequest>)+0x996) [0x56040ba2bb26] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 7: (Monitor::handle_command(boost::intrusive_ptr<MonOpRequest>)+0x1a12) [0x56040b8f6352] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 8: (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0x39d) [0x56040b8fb57d] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 9: (Monitor::_ms_dispatch(Message*)+0x7a2) [0x56040b8fcc22] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 10: (Monitor::handle_forward(boost::intrusive_ptr<MonOpRequest>)+0x7a7) [0x56040b8fdef7] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 11: (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0xe71) [0x56040b8fc051] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 12: (Monitor::_ms_dispatch(Message*)+0x7a2) [0x56040b8fcc22] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 13: (Monitor::ms_dispatch(Message*)+0x23) [0x56040b9257c3] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 14: (DispatchQueue::entry()+0xb92) [0x7f3b32aadc72] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 15: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f3b32b4becd] 2018-05-10T04:37:43.488 INFO:tasks.ceph.mon.b.mira053.stderr: 16: (()+0x76ba) [0x7f3b323336ba] 2018-05-10T04:37:43.489 INFO:tasks.ceph.mon.b.mira053.stderr: 17: (clone()+0x6d) [0x7f3b3169141d]
Updated by Kefu Chai almost 6 years ago
- Assignee changed from Ricardo Dias to Kefu Chai
Updated by Kefu Chai almost 6 years ago
- Category set to Tests
- Status changed from New to Fix Under Review
Josh, it's not a mon crash. mon was just not happy with this command, please see handle_bad_get() in cmd_getval(CephContext *cct, const cmdmap_t& cmdmap, const std::string& k, T& val)
Updated by Kefu Chai almost 6 years ago
- Status changed from Fix Under Review to Resolved
Actions