Ceph » CephFS

The job was halted with following log messages -

2023-10-27T04:56:30.074 INFO:tasks.workunit.client.0.smithi173.stderr:2023-10-27T04:56:30.071+0000 7fe309fda700  1 -- 172.21.15.173:0/3256449946 --> [v2:172.21.15.173:6808/84007,v1:172.21.15.173:6809/84007] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sub_0", "snap_name": "snap_0", "target_sub_name": "clone_10", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2e4000c70 con 0x7fe2f00317b0
2023-10-27T04:56:30.633 INFO:tasks.ceph.mon.a.smithi173.stderr:2023-10-27T04:56:30.630+0000 7fa3bef7e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL)
2023-10-27T04:56:31.058 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-10-27T04:56:31.055+0000 7f6b4316d700 -1 mgr.server reply reply (22) Invalid argument exception in subvolume metadata: No space left on device
2023-10-27T04:56:31.059 INFO:tasks.workunit.client.0.smithi173.stderr:2023-10-27T04:56:31.056+0000 7fe2eb7fe700  1 -- 172.21.15.173:0/3256449946 <== mgr.4101 v2:172.21.15.173:6808/84007 1 ==== mgr_command_reply(tid 0: -22 exception in subvolume metadata: No space left on device) v1 ==== 64+0+0 (secure 0 0 0) 0x7fe2e4000c70 con 0x7fe2f00317b0
2023-10-27T04:56:31.060 INFO:tasks.workunit.client.0.smithi173.stderr:Error EINVAL: exception in subvolume metadata: No space left on device
2023-10-27T04:56:31.063 DEBUG:teuthology.orchestra.run:got remote process result: 22
2023-10-27T04:56:31.063 INFO:tasks.workunit:Stopping ['fs/full/subvolume_clone.sh'] on client.0...
2023-10-27T04:56:31.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0
2023-10-27T04:56:31.364 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):

This error is similar to ones reported on ticket #62501 and ticket #63132 but the traceback reported on these ticket wasn't anywhere in this job directory.

Failed job - yuriw-2023-10-24_00:02:48-fs-wip-yuri7-testing-2023-10-23-1230-pacific-distro-default-smithi/7434991/teuthology.log