Actions
Bug #9178
closedsamba: ENOTEMPTY on "rm -rf"
Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2014-08-18T01:23:14.854 INFO:teuthology.orchestra.run.plana83:Running: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp' 2014-08-18T01:23:19.626 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p0/d1/d12/d57/d65': Directory not empty 2014-08-18T01:23:22.085 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p0/d1/d8c/db0/de0/d11b': Directory not empty 2014-08-18T01:23:25.481 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p2/dc/d45': Directory not empty 2014-08-18T01:23:30.356 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p3/da/d2e/d3a': Directory not empty 2014-08-18T01:23:37.616 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p4/d0/dc/d11/d94/d101': Directory not empty 2014-08-18T01:23:38.884 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p5/d7/d37/d28': Directory not empty 2014-08-18T01:23:43.076 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p5/d7/da0/d11c': Directory not empty 2014-08-18T01:23:51.992 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p7/d1/dc': Directory not empty 2014-08-18T01:23:53.705 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p8/d0/d44': Directory not empty 2014-08-18T01:23:54.804 INFO:tasks.workunit.client.1.plana83.stderr:rm: cannot remove `/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress-plana835050/p8/d0/dd/d15/d35/d61/d8b/d102/d4c/d50/df2': Directory not empty 2014-08-18T01:24:04.190 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2014-08-18T01:24:04.190 INFO:teuthology.orchestra.run.plana83:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list /home/ubuntu/cephtest/workunit.client.1' 2014-08-18T01:24:04.207 ERROR:teuthology.parallel:Exception in parallel execution Traceback (most recent call last): File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__ for result in self: File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next resurrect_traceback(result) File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback return func(*args, **kwargs) File "/var/lib/teuthworker/src/ceph-qa-suite_next/tasks/workunit.py", line 363, in _run_tests args=['sudo', 'rm', '-rf', '--', scratch_tmp], File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 114, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 401, in run r.wait() File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 102, in wait exitstatus=status, node=self.hostname) CommandFailedError: Command failed on plana83 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp'
So far this is new and just on Samba; we'll see if it pops up elsewhere.
Updated by Zheng Yan over 9 years ago
The strange thing is that the MDS never reply ENOTEMPTY.
Updated by Greg Farnum over 9 years ago
It might not need to if there's a client bug somewhere. (Or some other issue?)
Updated by Zheng Yan over 9 years ago
- Status changed from New to 7
2014-09-08 02:25:36.393142 7fe28a9d2740 10 client.4119 readdir_r_cb 1000000036a.head(ref=3 cap_refs={} open={} mode=40755 size=0/0 mtime=2014-09-08 02:24:24.131352 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7fe28cc00220 0x7fe25818c510) offset 0 frag * fragpos 0 at_end=0 2014-09-08 02:25:36.393156 7fe28a9d2740 15 client.4119 including . 2014-09-08 02:25:36.393158 7fe28a9d2740 10 client.4119 fill_dirent '.' -> 1000000036a type 4 w/ next_off 1 2014-09-08 02:25:36.393160 7fe28a9d2740 10 client.4119 fill_stat on 1000000036a snap/devhead mode 040755 mtime 2014-09-08 02:24:24.131352 ctime 2014-09-08 02:24:24.131352 2014-09-08 02:25:36.393167 7fe28a9d2740 10 client.4119 readdir_r_cb 1000000036a.head(ref=3 cap_refs={} open={} mode=40755 size=0/0 mtime=2014-09-08 02:24:24.131352 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7fe28cc00220 0x7fe25818c510) offset 1 frag * fragpos 1 at_end=0 2014-09-08 02:25:36.393178 7fe28a9d2740 15 client.4119 including .. 2014-09-08 02:25:36.393180 7fe28a9d2740 10 client.4119 fill_dirent '..' -> 1000000036a type 4 w/ next_off 2 2014-09-08 02:25:36.393182 7fe28a9d2740 10 client.4119 fill_stat on 1000000036a snap/devhead mode 040755 mtime 2014-09-08 02:24:24.131352 ctime 2014-09-08 02:24:24.131352 2014-09-08 02:25:36.393187 7fe28a9d2740 10 client.4119 readdir_r_cb 1000000036a.head(ref=3 cap_refs={} open={} mode=40755 size=0/0 mtime=2014-09-08 02:24:24.131352 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7fe28cc00220 0x7fe25818c510) offset 2 frag * fragpos 2 at_end=0 2014-09-08 02:25:36.393198 7fe28a9d2740 10 client.4119 offset 2 at_cache_name snapid head complete 1 issued pAsLsXsFsx 2014-09-08 02:25:36.393204 7fe28a9d2740 10 client.4119 _readdir_cache_cb 0x7fe28cc55cc0 on 1000000036a at_cache_name offset 2 2014-09-08 02:25:36.393206 7fe28a9d2740 15 client.4119 skipping null 'd87' 2014-09-08 02:25:36.393208 7fe28a9d2740 10 client.4119 fill_stat on 1000000044d snap/devhead mode 040755 mtime 2014-09-08 02:25:29.206007 ctime 2014-09-08 02:25:29.206007 2014-09-08 02:25:36.393214 7fe28a9d2740 10 client.4119 fill_dirent 'da9' -> 1000000044d type 4 w/ next_off 3 2014-09-08 02:25:36.393216 7fe28a9d2740 15 client.4119 de da9 off 5 = 1 2014-09-08 02:25:36.393219 7fe28a9d2740 3 client.4119 telldir(0x7fe28cc55cc0) = 6
"de da9 off 5 = 1" is wrong, off should be 2
https://github.com/ceph/ceph/commit/2249179a8d6a6feec5992a1b9bb71f49fe910a5c
Updated by Greg Farnum over 9 years ago
/a/teuthology-2014-09-08_23:14:02-samba-master-testing-basic-multi/474551/
Updated by Zheng Yan over 9 years ago
the fix https://github.com/ceph/ceph/pull/2431 hasn't been merged yet
Actions