Bug #43762: pybind/mgr/volumes: create fails with TypeError - CephFS - Ceph

Actions

Copy link

Bug #43762

closed

pybind/mgr/volumes: create fails with TypeError

Added by Victoria Martinez de la Cruz over 4 years ago. Updated over 3 years ago.

Status:

Closed

Priority:

High

Assignee:

Jos Collin

Category:

Target version:

% Done:

Source:

Community (dev)

Tags:

Backport:

octopus,nautilus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v14.2.6

ceph-qa-suite:

Component(FS):

mgr/volumes

Labels (FS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

stack@bionicbeaver:~/devstack$ sudo ceph --version
ceph version 14.2.6-268-g2404588 (24045889bfe0c51d31753f37ee53dff33ae14b13) nautilus (stable)
stack@bionicbeaver:~$ sudo ceph fs subvolumegroup create manila-volume manila-group
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 923, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/volumes/module.py", line 230, in handle_command
    return handler(inbuf, cmd)
  File "/usr/share/ceph/mgr/volumes/module.py", line 253, in _cmd_fs_subvolumegroup_create
    gid=cmd.get('gid', None), mode=cmd.get('mode', '755'))
  File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 356, in conn_wrapper
    fs_h = self.connection_pool.get_fs_handle(fs_name)
  File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 145, in get_fs_handle
    raise VolumeException(-e.args[0], e.args[1])
TypeError: bad operand type for unary -: 'str'

< ceph mgr logs >

root@bionicbeaver:~# sudo journalctl -f --unit ceph-mgr@x.service
-- Logs begin at Thu 2020-01-09 14:41:46 UTC. --
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:     return self.handle_command(inbuf, cmd)
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/module.py", line 230, in handle_command
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:     return handler(inbuf, cmd)
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/module.py", line 253, in _cmd_fs_subvolumegroup_create
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:     gid=cmd.get('gid', None), mode=cmd.get('mode', '755'))
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 356, in conn_wrapper
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:     fs_h = self.connection_pool.get_fs_handle(fs_name)
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 145, in get_fs_handle
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]:     raise VolumeException(-e.args[0], e.args[1])
Jan 20 14:44:50 bionicbeaver ceph-mgr[50888]: TypeError: bad operand type for unary -: 'str'

Jan 20 17:20:23 bionicbeaver ceph-mgr[50888]: 2020-01-20 17:20:23.522 7fae12c97700 -1 client.0 error registering admin socket command: (17) File exists

Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]: 2020-01-20 17:25:23.562 7fae12c97700 -1 mgr handle_command module 'volumes' command handler threw exception: bad operand type for unary -: 'str'
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]: 2020-01-20 17:25:23.566 7fae12c97700 -1 mgr.server reply reply (22) Invalid argument Traceback (most recent call last):
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/mgr_module.py", line 923, in _handle_command
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:     return self.handle_command(inbuf, cmd)
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/module.py", line 230, in handle_command
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:     return handler(inbuf, cmd)
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/module.py", line 253, in _cmd_fs_subvolumegroup_create
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:     gid=cmd.get('gid', None), mode=cmd.get('mode', '755'))
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 356, in conn_wrapper
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:     fs_h = self.connection_pool.get_fs_handle(fs_name)
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:   File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 145, in get_fs_handle
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]:     raise VolumeException(-e.args[0], e.args[1])
Jan 20 17:25:23 bionicbeaver ceph-mgr[50888]: TypeError: bad operand type for unary -: 'str'

Checked the ceph-mon and the ceph-osd, nothing relevant there

Should I check logs for some other service?

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Patrick Donnelly over 4 years ago

Subject changed from ceph fs subvolumegroup create fails with typeerror to pybind/mgr/volumes: create fails with TypeError
Description updated (diff)
Status changed from New to Triaged
Assignee set to Jos Collin
Priority changed from Normal to High
Target version set to v15.0.0
Source set to Community (dev)
Backport set to nautilus
Component(FS) mgr/volumes added

Actions

Copy link

Updated by Jos Collin over 4 years ago

Status changed from Triaged to Need More Info

This needs more information, as Victoria is checking if this happens because of their configuration/python version.

Actions

Copy link

Updated by Victoria Martinez de la Cruz about 4 years ago

Adding more context to this

This happened after creating a second volume. I had to enable creation of multi filesystems support (experimental) for this.

Precisely,

Error EINVAL: Creation of multiple filesystems is disabled. To enable this experimental feature, use 'ceph fs flag set enable_multiple true'
stack@bionicbeaver:~/devstack$ sudo ceph fs flag set enable_multiple true
Warning! This feature is experimental.It may cause problems up to and including data loss.Consult the documentation at ceph.com, and if unsure, do not proceed.Add --yes-i-really-mean-it if you are certain.

Then I could create the filesystem and reach the error mentioned.

I still need to continue debugging this, next step will be to check the interpreter (as Jos mentioned)

Actions

Copy link

Updated by Patrick Donnelly about 4 years ago

Target version deleted (~~v15.0.0~~)
Backport changed from nautilus to octopus,nautilus

Actions

Copy link

Updated by Jos Collin almost 4 years ago

Victoria Martinez de la Cruz wrote:

Adding more context to this

This happened after creating a second volume. I had to enable creation of multi filesystems support (experimental) for this.

Precisely,

Error EINVAL: Creation of multiple filesystems is disabled. To enable this experimental feature, use 'ceph fs flag set enable_multiple true'
stack@bionicbeaver:~/devstack$ sudo ceph fs flag set enable_multiple true
Warning! This feature is experimental.It may cause problems up to and including data loss.Consult the documentation at ceph.com, and if unsure, do not proceed.Add --yes-i-really-mean-it if you are certain.

Then I could create the filesystem and reach the error mentioned.

I still need to continue debugging this, next step will be to check the interpreter (as Jos mentioned)

Victoria, any update on this?

Actions

Copy link

Updated by Victoria Martinez de la Cruz over 3 years ago

Jos Collin wrote:

Victoria Martinez de la Cruz wrote:

Adding more context to this

This happened after creating a second volume. I had to enable creation of multi filesystems support (experimental) for this.

Precisely,

Error EINVAL: Creation of multiple filesystems is disabled. To enable this experimental feature, use 'ceph fs flag set enable_multiple true'
stack@bionicbeaver:~/devstack$ sudo ceph fs flag set enable_multiple true
Warning! This feature is experimental.It may cause problems up to and including data loss.Consult the documentation at ceph.com, and if unsure, do not proceed.Add --yes-i-really-mean-it if you are certain.

Then I could create the filesystem and reach the error mentioned.

I still need to continue debugging this, next step will be to check the interpreter (as Jos mentioned)

Victoria, any update on this?

Jos, I'll try to reproduce again in stable/nautilus and see if we hit the same issue. When I hit with this, we were using the shaman builds, so it might be due to an unstable build. Thanks!

Actions

Copy link