Project

General

Profile

Actions

Bug #37999

open

500 - Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request.

Added by kevin huang over 5 years ago. Updated about 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Component - RBD
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
Yes
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Go to dashboard, to Block, to Images
500 - Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.

The error log from ceph-mgr

2019-01-22 04:03:32.673 7efd9d33f700  0 mgr[dashboard] Error while calling fn=<function _rbd_pool_list at 0x7efda5f9e0c8> ex=[errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a
Traceback (most recent call last):
  File "/usr/lib64/ceph/mgr/dashboard/tools.py", line 120, in run
    val = self.fn(*self.args, **self.kwargs)
  File "/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py", line 237, in _rbd_pool_list
    stat = self._rbd_image(ioctx, pool_name, name)
  File "/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py", line 155, in _rbd_image
    stat['id'] = img.id()
  File "rbd.pyx", line 1970, in rbd.Image.id (/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.4/rpm/el7/BUILD/ceph-13.2.4/build/src/pybind/rbd/pyrex/rbd.c:16379)
InvalidArgument: [errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a
2019-01-22 04:03:32.676 7efda0e06700  0 mgr[dashboard] [100.98.60.91:49367] [GET] [500] [0.020s] [sre] [1.7K] /api/block/image
2019-01-22 04:03:32.676 7efda0e06700  0 mgr[dashboard] ['{"status": "500 Internal Server Error", "version": "3.2.2", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "traceback": "Traceback (most recent call last):\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\\", line 656, in respond\\n    response.body = self.handler()\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\\", line 188, in __call__\\n    self.body = self.oldhandler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\\", line 34, in __call__\\n    return self.callable(*self.args, **self.kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 169, in wrapper\\n    return meth(self, *vpath, **kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 498, in _collection\\n    return self._rest_request(False, *vpath, **params)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 534, in _rest_request\\n    return method(*vpath, **params)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 569, in inner\\n    ret = func(*args, **kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py\\", line 258, in list\\n    return self._rbd_list(pool_name)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py\\", line 253, in _rbd_list\\n    status, value = self._rbd_pool_list(pool)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/tools.py\\", line 212, in wrapper\\n    return rvc.run(fn, args, kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/tools.py\\", line 194, in run\\n    raise self.exception\\nInvalidArgument: [errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a\\n"}']
the reason for this error (also affects nautilus) are rbd image format 1 images
rbd image 'kubernetes-dynamic-pvc-01f001a8-18cd-11e9-80b9-fa163ecf25c6':
        size 5 GiB in 1280 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        block_name_prefix: rb.0.6482d76.238e1f29
        format: 1

rbd image 'kubernetes-dynamic-pvc-fb6c9756-41c0-11e9-bbf7-fa163ecf25c6':
        size 10 GiB in 2560 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: a1e4c26b8b4567
        block_name_prefix: rbd_data.a1e4c26b8b4567
        format: 2
        features: layering
        op_features: 
        flags: 
        create_timestamp: Fri Mar  8 17:41:12 2019
Actions #1

Updated by Brad Hubbard over 5 years ago

  • Project changed from Ceph to mgr
Actions #2

Updated by Lenz Grimmer over 5 years ago

  • Category set to 139
Actions #3

Updated by Lenz Grimmer over 5 years ago

Thank you for reporting this. Can you still reproduce this error? We're wondering if the metadata of the "kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a" somehow got corrupted? Was the image deleted? Do you see this in the list of RBDs in the CLI? What does the output of "rbd info" reveal about this image?

Actions #4

Updated by Lenz Grimmer over 5 years ago

  • Status changed from New to Need More Info
Actions #5

Updated by Lenz Grimmer about 5 years ago

  • Status changed from Need More Info to Can't reproduce

No feedback was provided after 3 months. Resolving this one as "Can't reproduce". Feel free to reopen it, if it still occurs and you can provide additional details.

Actions #6

Updated by Alfredo Deza over 4 years ago

  • Description updated (diff)
Actions #7

Updated by Alfredo Deza over 4 years ago

  • Status changed from Can't reproduce to New
  • Target version deleted (v13.2.4)
Actions #8

Updated by Alfredo Deza over 4 years ago

Re-opening per IRC user jenglisch. Had to remove the tag to re-open it.

Actions #9

Updated by Robert Sander about 4 years ago

I also see this error:

Failed to executeRBD
[errno 22] error getting id for image vm-101-disk-1

in a Nautilus 14.2.7 dashboard with vm-101-disk-1 being an RBD v1 image.

Actions #10

Updated by Robert Sander about 4 years ago

rbd info shows:

rbd image 'vm-101-disk-1':
        size 233 GiB in 59603 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        block_name_prefix: rb.0.33bef48.238e1f29
        format: 1
Actions #11

Updated by Alex Litvak almost 4 years ago

Same problem in Nautilus 14.2.10
0 mgr[dashboard] Error while calling fn=<function _rbd_pool_list at 0x7f794c0e15f0> ex=[errno 22] error getting id for image one-37

rbd image 'one-37':
size 2.0 TiB in 512000 objects
order 22 (4 MiB objects)
snapshot_count: 0
block_name_prefix: rb.0.476aea.238e1f29
format: 1

Actions #12

Updated by Ernesto Puerta about 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 139 to Component - RBD
Actions

Also available in: Atom PDF