Project

General

Profile

Actions

Bug #20383

closed

restful: ValueError: No JSON object could be decoded

Added by Sage Weil almost 7 years ago. Updated almost 7 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2017-06-22T13:29:15.688 INFO:tasks.workunit.client.a.smithi007.stderr:+ port=9999
2017-06-22T13:29:15.688 INFO:tasks.workunit.client.a.smithi007.stderr:+ url=https://127.0.0.1:9999
2017-06-22T13:29:15.688 INFO:tasks.workunit.client.a.smithi007.stderr:+ echo 'prefix mgr/restful/x url https://127.0.0.1:9999'
2017-06-22T13:29:15.688 INFO:tasks.workunit.client.a.smithi007.stderr:+ /home/ubuntu/cephtest/clone.client.a/qa/workunits/rest/test_mgr_rest_api.py https://127.0.0.1:9999 7af0fbd8-fb1c-483b-a99c-298ab96c96a4
2017-06-22T13:29:15.688 INFO:tasks.workunit.client.a.smithi007.stdout:prefix mgr/restful/x url https://127.0.0.1:9999
2017-06-22T13:29:16.096 INFO:tasks.ceph.mgr.x.smithi007.stderr:127.0.0.1 - - [22/Jun/2017 13:29:16] "POST /pool?wait=yes HTTP/1.1" 200 -
2017-06-22T13:29:16.111 INFO:tasks.ceph.mgr.x.smithi007.stderr:127.0.0.1 - - [22/Jun/2017 13:29:16] "GET /pool HTTP/1.1" 200 -
2017-06-22T13:29:16.155 INFO:tasks.ceph.mgr.x.smithi007.stderr:127.0.0.1 - - [22/Jun/2017 13:29:16] "GET /mon HTTP/1.1" 500 -
2017-06-22T13:29:16.160 INFO:tasks.ceph.mgr.x.smithi007.stderr:Error on request:
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:Traceback (most recent call last):
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 177, in run_wsgi
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:    execute(self.server.app)
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 165, in execute
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:    application_iter = app(environ, start_response)
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/pecan/middleware/recursive.py", line 56, in __call__
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:    return self.application(environ, start_response)
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/pecan/core.py", line 570, in __call__
2017-06-22T13:29:16.161 INFO:tasks.ceph.mgr.x.smithi007.stderr:    self.handle_request(req, resp)
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/pecan/core.py", line 508, in handle_request
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:    result = controller(*args, **kwargs)
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/ceph/mgr/restful/decorators.py", line 33, in decorated
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:    return f(*args, **kwargs)
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/ceph/mgr/restful/api/mon.py", line 39, in get
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:    return module.instance.get_mons()
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/ceph/mgr/restful/module.py", line 487, in get_mons
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:    mon_status = json.loads(self.get('mon_status')['json'])
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2017-06-22T13:29:16.162 INFO:tasks.ceph.mgr.x.smithi007.stderr:    return _default_decoder.decode(s)
2017-06-22T13:29:16.163 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2017-06-22T13:29:16.163 INFO:tasks.ceph.mgr.x.smithi007.stderr:    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2017-06-22T13:29:16.163 INFO:tasks.ceph.mgr.x.smithi007.stderr:  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2017-06-22T13:29:16.168 INFO:tasks.ceph.mgr.x.smithi007.stderr:    raise ValueError("No JSON object could be decoded")
2017-06-22T13:29:16.168 INFO:tasks.ceph.mgr.x.smithi007.stderr:ValueError: No JSON object could be decoded
2017-06-22T13:29:16.422 INFO:tasks.workunit.client.a.smithi007.stdout:{
2017-06-22T13:29:16.423 INFO:tasks.workunit.client.a.smithi007.stdout:    "failed": [],
2017-06-22T13:29:16.423 INFO:tasks.workunit.client.a.smithi007.stdout:    "finished": [
2017-06-22T13:29:16.423 INFO:tasks.workunit.client.a.smithi007.stdout:        {
2017-06-22T13:29:16.423 INFO:tasks.workunit.client.a.smithi007.stdout:            "command": "osd pool create pg_num=128 pool=supertestfriends",
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:            "outb": "",
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:            "outs": "pool 'supertestfriends' created" 
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:        }
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    ],
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "has_failed": false,
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "id": "139959302967824",
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "is_finished": true,
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "is_waiting": false,
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "running": [],
2017-06-22T13:29:16.424 INFO:tasks.workunit.client.a.smithi007.stdout:    "state": "success",
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stdout:    "waiting": []
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stdout:}
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:Traceback (most recent call last):
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:  File "/home/ubuntu/cephtest/clone.client.a/qa/workunits/rest/test_mgr_rest_api.py", line 40, in <module>
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:    firstmon = request.json()[0]['name']
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:    return json.loads(self.text, **kwargs)
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2017-06-22T13:29:16.425 INFO:tasks.workunit.client.a.smithi007.stderr:    return _default_decoder.decode(s)
2017-06-22T13:29:16.426 INFO:tasks.workunit.client.a.smithi007.stderr:  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2017-06-22T13:29:16.426 INFO:tasks.workunit.client.a.smithi007.stderr:    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2017-06-22T13:29:16.426 INFO:tasks.workunit.client.a.smithi007.stderr:  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2017-06-22T13:29:16.426 INFO:tasks.workunit.client.a.smithi007.stderr:    raise ValueError("No JSON object could be decoded")
2017-06-22T13:29:16.426 INFO:tasks.workunit.client.a.smithi007.stderr:ValueError: No JSON object could be decoded
2017-06-22T13:29:16.436 INFO:tasks.workunit:Stopping ['rest/test-restful.sh'] on client.a...

/a/sage-2017-06-22_13:19:56-rados-master-distro-basic-smithi/1316768

Previous test at /a/sage-2017-06-22_01:17:23-rados-wip-sage-testing-distro-basic-smithi/1313186 passed, not sure what is different.

Actions #1

Updated by Sage Weil almost 7 years ago

/a/sage-2017-06-27_05:44:05-rados-wip-sage-testing-distro-basic-smithi/1331647

Actions #3

Updated by Joao Eduardo Luis almost 7 years ago

  • Assignee set to Ricardo Dias
Actions #4

Updated by Boris Ranto almost 7 years ago

The PR for the python part of the fix (we should still try to investigate why ceph-mgr is returning an object that is not json decodable):

https://github.com/ceph/ceph/pull/15977

Actions #5

Updated by John Spray almost 7 years ago

  • Status changed from 12 to In Progress
  • Assignee changed from Ricardo Dias to John Spray

Currently there's a brief window at startup where ceph-mgr has loaded the python modules but hasn't received the mon_status and health information from the mons yet. We should just block during startup until all the info is available.

Actions #6

Updated by John Spray almost 7 years ago

  • Status changed from In Progress to Fix Under Review
Actions #7

Updated by Sage Weil almost 7 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF