Bug #47798
Updated by Patrick Donnelly over 3 years ago
A stack trace when facing ETIMEDOUT errno during subvolume operations is presented below, <pre> 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/fs/operations/volume.py", line 158, in get_fs_handle 2020-10-05T18:36:21.739384142Z conn.connect() 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/fs/operations/volume.py", line 85, in connect 2020-10-05T18:36:21.739384142Z self.fs.mount(filesystem_name=self.fs_name.encode('utf-8')) 2020-10-05T18:36:21.739384142Z File "cephfs.pyx", line 694, in cephfs.LibCephFS.mount 2020-10-05T18:36:21.739384142Z cephfs.Error: error calling ceph_mount: Connection timed out [Errno 110] 2020-10-05T18:36:21.739384142Z 2020-10-05T18:36:21.739384142Z During handling of the above exception, another exception occurred: 2020-10-05T18:36:21.739384142Z 2020-10-05T18:36:21.739384142Z Traceback (most recent call last): 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/mgr_module.py", line 974, in _handle_command 2020-10-05T18:36:21.739384142Z return self.handle_command(inbuf, cmd) 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/module.py", line 270, in handle_command 2020-10-05T18:36:21.739384142Z return handler(inbuf, cmd) 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/module.py", line 338, in _cmd_fs_subvolume_getpath 2020-10-05T18:36:21.739384142Z group_name=cmd.get('group_name', None)) 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/fs/volume.py", line 213, in subvolume_getpath 2020-10-05T18:36:21.739384142Z with open_volume(self, volname) as fs_handle: 2020-10-05T18:36:21.739384142Z File "/lib64/python3.6/contextlib.py", line 81, in __enter__ 2020-10-05T18:36:21.739384142Z return next(self.gen) 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/fs/operations/volume.py", line 283, in open_volume 2020-10-05T18:36:21.739384142Z fs_handle = vc.connection_pool.get_fs_handle(volname) 2020-10-05T18:36:21.739384142Z File "/usr/share/ceph/mgr/volumes/fs/operations/volume.py", line 164, in get_fs_handle 2020-10-05T18:36:21.739384142Z raise VolumeException(-e.args[0], e.args[1]) 2020-10-05T18:36:21.739384142Z TypeError: bad operand type for unary -: 'str' </pre> The traceback, rather than a graceful error seems to stem from not handling ETIMEDOUT errno (110), and requires a fix along the lines of https://github.com/ceph/ceph/pull/36180 The issue was seen on a Nautilus build as such, unsure if octopus or master/Pacific have the same issue.