Project

General

Profile

Bug #37999

500 - Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request.

Added by kevin huang over 2 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Component - RBD
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
Yes
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Go to dashboard, to Block, to Images
500 - Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.

The error log from ceph-mgr

2019-01-22 04:03:32.673 7efd9d33f700  0 mgr[dashboard] Error while calling fn=<function _rbd_pool_list at 0x7efda5f9e0c8> ex=[errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a
Traceback (most recent call last):
  File "/usr/lib64/ceph/mgr/dashboard/tools.py", line 120, in run
    val = self.fn(*self.args, **self.kwargs)
  File "/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py", line 237, in _rbd_pool_list
    stat = self._rbd_image(ioctx, pool_name, name)
  File "/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py", line 155, in _rbd_image
    stat['id'] = img.id()
  File "rbd.pyx", line 1970, in rbd.Image.id (/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.4/rpm/el7/BUILD/ceph-13.2.4/build/src/pybind/rbd/pyrex/rbd.c:16379)
InvalidArgument: [errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a
2019-01-22 04:03:32.676 7efda0e06700  0 mgr[dashboard] [100.98.60.91:49367] [GET] [500] [0.020s] [sre] [1.7K] /api/block/image
2019-01-22 04:03:32.676 7efda0e06700  0 mgr[dashboard] ['{"status": "500 Internal Server Error", "version": "3.2.2", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "traceback": "Traceback (most recent call last):\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\\", line 656, in respond\\n    response.body = self.handler()\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\\", line 188, in __call__\\n    self.body = self.oldhandler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\\", line 34, in __call__\\n    return self.callable(*self.args, **self.kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 169, in wrapper\\n    return meth(self, *vpath, **kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 498, in _collection\\n    return self._rest_request(False, *vpath, **params)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 534, in _rest_request\\n    return method(*vpath, **params)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/__init__.py\\", line 569, in inner\\n    ret = func(*args, **kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py\\", line 258, in list\\n    return self._rbd_list(pool_name)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/controllers/rbd.py\\", line 253, in _rbd_list\\n    status, value = self._rbd_pool_list(pool)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/tools.py\\", line 212, in wrapper\\n    return rvc.run(fn, args, kwargs)\\n  File \\"/usr/lib64/ceph/mgr/dashboard/tools.py\\", line 194, in run\\n    raise self.exception\\nInvalidArgument: [errno 22] error getting id for image kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a\\n"}']
the reason for this error (also affects nautilus) are rbd image format 1 images
rbd image 'kubernetes-dynamic-pvc-01f001a8-18cd-11e9-80b9-fa163ecf25c6':
        size 5 GiB in 1280 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        block_name_prefix: rb.0.6482d76.238e1f29
        format: 1

rbd image 'kubernetes-dynamic-pvc-fb6c9756-41c0-11e9-bbf7-fa163ecf25c6':
        size 10 GiB in 2560 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: a1e4c26b8b4567
        block_name_prefix: rbd_data.a1e4c26b8b4567
        format: 2
        features: layering
        op_features: 
        flags: 
        create_timestamp: Fri Mar  8 17:41:12 2019

History

#1 Updated by Brad Hubbard over 2 years ago

  • Project changed from Ceph to mgr

#2 Updated by Lenz Grimmer over 2 years ago

  • Category set to 139

#3 Updated by Lenz Grimmer over 2 years ago

Thank you for reporting this. Can you still reproduce this error? We're wondering if the metadata of the "kubernetes-dynamic-pvc-08cd2beb-1b2e-11e9-a91f-005056b04a8a" somehow got corrupted? Was the image deleted? Do you see this in the list of RBDs in the CLI? What does the output of "rbd info" reveal about this image?

#4 Updated by Lenz Grimmer over 2 years ago

  • Status changed from New to Need More Info

#5 Updated by Lenz Grimmer over 2 years ago

  • Status changed from Need More Info to Can't reproduce

No feedback was provided after 3 months. Resolving this one as "Can't reproduce". Feel free to reopen it, if it still occurs and you can provide additional details.

#6 Updated by Alfredo Deza about 2 years ago

  • Description updated (diff)

#7 Updated by Alfredo Deza about 2 years ago

  • Status changed from Can't reproduce to New
  • Target version deleted (v13.2.4)

#8 Updated by Alfredo Deza about 2 years ago

Re-opening per IRC user jenglisch. Had to remove the tag to re-open it.

#9 Updated by Robert Sander over 1 year ago

I also see this error:

Failed to executeRBD
[errno 22] error getting id for image vm-101-disk-1

in a Nautilus 14.2.7 dashboard with vm-101-disk-1 being an RBD v1 image.

#10 Updated by Robert Sander over 1 year ago

rbd info shows:

rbd image 'vm-101-disk-1':
        size 233 GiB in 59603 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        block_name_prefix: rb.0.33bef48.238e1f29
        format: 1

#11 Updated by Alex Litvak about 1 year ago

Same problem in Nautilus 14.2.10
0 mgr[dashboard] Error while calling fn=<function _rbd_pool_list at 0x7f794c0e15f0> ex=[errno 22] error getting id for image one-37

rbd image 'one-37':
size 2.0 TiB in 512000 objects
order 22 (4 MiB objects)
snapshot_count: 0
block_name_prefix: rb.0.476aea.238e1f29
format: 1

#12 Updated by Ernesto Puerta 5 months ago

  • Project changed from mgr to Dashboard
  • Category changed from 139 to Component - RBD

Also available in: Atom PDF