[testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)
When testing with my postgres changes:
I've observed failures with the kernel testing branch. The set of PRs notably introduce periodic snapshots to fs:workload which seems to catch this behavior:
2022-09-22T14:04:02.650 INFO:tasks.workunit.client.0.smithi124.stdout: write failed on handle 10009 (Resource temporarily unavailable) 2022-09-22T14:04:02.650 INFO:tasks.workunit.client.0.smithi124.stdout:Child failed with status 1
I've not gone through all of these events but the failures seem to share: (a) testing branch; (b) snapshots turned on.
#1 Updated by Patrick Donnelly 2 months ago
#3 Updated by Xiubo Li about 2 months ago
Another failure is :
2022-09-26T20:03:01.601 INFO:tasks.workunit.client.0.smithi066.stdout:Wrote -1 instead of 4096 bytes. 2022-09-26T20:03:01.601 INFO:tasks.workunit.client.0.smithi066.stdout:Probably out of disk space 2022-09-26T20:03:01.602 INFO:tasks.workunit.client.0.smithi066.stderr:write: Resource temporarily unavailable 2022-09-26T20:03:01.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-26T20:03:01.640 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0...
This is kclient, and I didn't see any error in kernel logs. And also checked the mds, osd logs, didn't find nothing!
This should be a known issue: https://tracker.ceph.com/issues/51410
Could you reproduce this ?
#5 Updated by Xiubo Li about 1 month ago
- Status changed from In Progress to Need More Info
Today I spent more than half day to read the mds, osd side logs, but still couldn't find any suspect logs. Usually if we could some logs from kernel, it will be easy to know what exactly happen as the one in a previous tracker https://tracker.ceph.com/issues/54461.
I tried but couldn't reproduce it locally! Let's wait new ones to see whether could we get any useful info then.