Project

General

Profile

Bug #17800

ceph_volume_client.py : Error: Can't handle arrays of non-strings

Added by Thomas Bechtold 12 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Manila
Target version:
-
Start date:
11/04/2016
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
jewel
Component(FS):
VolumeClient
Needs Doc:
No

Description

When using Ceph (python-cephfs-10.2.3+git.1475228057.755cf99 , SLE12SP2) together with OpenStack Manila and trying to allow share access, the manila-share service fails with:

2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server [req-ea32cd1e-ceb8-4c04-a237-ad0df3a3885f 7804d3df369a40d3a8d60acc1b970f91 f648a37bd19b4a9c9b32a28f3a2d2452 - - -] Exception during message handling
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 163, in wrapped
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     return f(self, *args, **kwargs)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/utils.py", line 617, in wrapper
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     return func(self, *args, **kwargs)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 2572, in allow_access
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     share_server=share_server
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 60, in update_access_rules
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     share_server=share_server,
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 53, in _update_access_rules_locked
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     return self._update_access_rules(*args, **kwargs)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 112, in _update_access_rules
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     share_server=share_server
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 288, in update_access
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     access_key = self._allow_access(context, share, rule)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 237, in _allow_access
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     tenant_id=share['project_id'])
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 974, in authorize
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     key = self._authorize_volume(volume_path, auth_id, readonly)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1010, in _authorize_volume
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     key = self._authorize_ceph(volume_path, auth_id, readonly)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1087, in _authorize_ceph
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     'mon', cap['caps'].get('mon')]
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1278, in _rados_command
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server     raise rados.Error(outs)
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server Error: Can't handle arrays of non-strings
2016-11-04 13:29:03.514 16317 ERROR oslo_messaging.rpc.server 

Printing the sent json with some debug messages added to the code:

{'format': 'json', 'entity': 'client.alice', 'caps': ['mds', ',allow rw path=/volumes/_nogroup/dba70486-fc31-4
f83-a7dd-e391a00d759c', 'osd', ',allow rw pool=cephfs_data namespace=fsvolumens_dba70486-fc31-4f83-a7dd-e391a00d759c', 'mon', None]}

So the problem is that the mon caps are None. The problematic code is: https://github.com/ceph/ceph/blob/master/src/pybind/ceph_volume_client.py#L1087


Related issues

Copied to fs - Backport #18026: jewel: ceph_volume_client.py : Error: Can't handle arrays of non-strings Resolved

History

#1 Updated by Greg Farnum 12 months ago

  • Project changed from Ceph to fs
  • Category set to Manila
  • Assignee set to Ramana Raja
  • Source changed from other to Community (dev)

#2 Updated by Ramana Raja 12 months ago

  • Status changed from New to In Progress
  • Release jewel added
  • Component(FS) VolumeClient added

#3 Updated by Ramana Raja 11 months ago

  • Status changed from In Progress to Need Review

#4 Updated by John Spray 11 months ago

Thomas: what was the series of operations running up to this? To hit this code path it seems like the user you were authorizing would have to have already existed, but with no mon auth caps?

#5 Updated by John Spray 11 months ago

  • Status changed from Need Review to Pending Backport
  • Backport set to jewel

#6 Updated by Nathan Cutler 11 months ago

  • Copied to Backport #18026: jewel: ceph_volume_client.py : Error: Can't handle arrays of non-strings added

#7 Updated by John Spray 9 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF