Bug #58934
closed
Bug #54460: snaptest-multiple-capsnaps.sh test failure
Bug #59343: qa: fs/snaps/snaptest-multiple-capsnaps.sh failed
snaptest-git-ceph.sh failure with ceph-fuse
Added by Venky Shankar about 1 year ago.
Updated 11 months ago.
Category:
Correctness/Safety
Component(FS):
MDS, ceph-fuse
Description
https://pulpito.ceph.com/vshankar-2023-03-07_05:15:12-fs-wip-vshankar-testing-20230307.030510-testing-default-smithi/7195921/
Seems similar to https://tracker.ceph.com/issues/55332 which involved the kclient (async unlink/create bug), but this is with ceph-fuse.
2023-03-07T09:13:26.893 INFO:tasks.ceph.osd.7.smithi042.stderr:2023-03-07T09:13:26.891+0000 7f3b8e81e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph
-i 7 (PID: 71679) UID: 0
2023-03-07T09:13:26.917 INFO:tasks.workunit.client.0.smithi031.stdout:diff --git a/src/mds/CInode.h b/src/mds/CInode.h
2023-03-07T09:13:26.917 INFO:tasks.workunit.client.0.smithi031.stdout:deleted file mode 100644
2023-03-07T09:13:26.917 INFO:tasks.workunit.client.0.smithi031.stdout:index 8181be69c66..00000000000
2023-03-07T09:13:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:--- a/src/mds/CInode.h
2023-03-07T09:13:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:+++ /dev/null
`git diff` is seeing a deleted file (in one of the snapshots) which should have been present?
- Status changed from New to Duplicate
- Parent task set to #59343
Xiubo Li wrote:
This should be the same issue with https://tracker.ceph.com/issues/59343.
Which means this can happen with kclient runs.
I'm running into this here: https://pulpito.ceph.com/vshankar-2023-05-17_11:52:03-fs-wip-vshankar-testing-20230509.090020-6-testing-default-smithi/7276651/
But the failure is a bit different
2023-05-17T12:36:49.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2023-05-17T12:36:49.246 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2023-05-17T12:36:54.701 INFO:tasks.workunit.client.0.smithi092.stderr:Updating files: 7% (881/11784)^MUpdating files: 8% (943/11784)^MUpdating files: 9% (1061/11784)^MUpdating files: 10% (1179/11784)^MUpdating files: 11% (1297/11784)^MUpdating files: 12% (1415/11784)^MUpdating files: 13% (1532/11784)^MUpdating files: 14% (1650/11784)^MUpdating files: 15% (1768/11784)^MUpdating files: 16% (1886/11784)^MUpdating files: 17% (2004/11784)^MUpdating files: 18% (2122/11784)^MUpdating files: 19% (2239/11784)^MUpdating files:
20% (2357/11784)^MUpdating files: 20% (2401/11784)^MUpdating files: 21% (2475/11784)^MUpdating files: 22% (2593/11784)^MUpdating files: 23% (2711/11784)^MUpdating files: 24% (2829/11784)^MUpdating files: 25% (2946/11784)^MUpdating files: 26% (3064/11784)^MUpdating files: 26% (3123/11784)^MUpdating files: 27% (3182/11784)^MUpdating files: 28% (3300/11784)^MUpdating files: 29% (3418/11784)^MUpdating files: 30% (3536/11784)^MUpdating files: 31% (3654/11784)^MUpdating files: 32% (3771/11784)^MUpdating files: 33% (3889/11784)^MUpdating files: 33% (3909/11784)^MUpdating files: 34% (4007/11784)^MUpdating files: 35% (4125/11784)^MUpdating files: 36% (4243/11784)^MUpdating files: 37% (4361/11784)^MUpdating files: 38% (4478/11784)^MUpdating files: 39% (4596/11784)^MUpdating files: 39% (4692/11784)^MUpdating files: 40% (4714/11784)^MUpdating files: 41% (4832/11784)^MUpdating files: 42% (4950/11784)^MUpdating files: 43% (5068/11784)^MUpdating files: 44% (5185/11784)^MUpdating files: 45% (5303/11784)^MUpdating files: 46% (5421/11784)^MUpdating files: 46% (5516/11784)^MUpdating files: 46% (5524/11784)^MUpdating files: 47% (5539/11784)^MUpdating files: 47% (5579/11784)^MUpdating files: 48% (5657/11784)^MUpdating files: 49% (5775/11784)^MUpdating files: 50% (5892/11784)^MUpdating files: 51% (6010/11784)^MUpdating files: 52% (6128/11784)^MUpdating files: 53% (6246/11784)^MUpdating files: 54% (6364/11784)^MUpdating files: 55% (6482/11784)^MUpdating files: 55% (6527/11784)^MUpdating files: 56% (6600/11784)^MUpdating files: 57% (6717/11784)^MUpdating files: 58% (6835/11784)^MUpdating files: 59% (6953/11784)^MUpdating files: 60% (7071/11784)^MUpdating files: 61% (7189/11784)^MUpdating files: 62% (7307/11784)^MUpdating files: 62% (7346/11784)^MUpdating files: 63% (7424/11784)^MUpdating files: 64% (7542/11784)^MUpdating files:
65% (7660/11784)^MUpdating files: 66% (7778/11784)^MUpdating files: 67% (7896/11784)^MUpdating files: 68% (8014/11784)^MUpdating files: 69% (8131/11784)^MUpdating files: 69% (8143/11784)^MUpdating files: 70% (8249/11784)^MUpdating files: 71% (8367/11784)^MUpdating files: 72% (8485/11784)^MUpdating files: 73% (8603/11784)^MUpdating files: 74% (8721/11784)^MUpdating files: 75% (8838/11784)^MUpdating files: 75% (8906/11784)^MUpdating files: 76% (8956/11784)^MUpdating files: 77% (9074/11784)^MUpdating files: 78% (9192/11784)^MUpdating files: 79% (9310/11784)^MUpdating files: 80% (9428/11784)^MUpdating files: 81% (9546/11784)^MUpdating files: 81% (9617/11784)^MUpdating files: 82% (9663/11784)^MUpdating files: 83% (9781/11784)^MUpdating files: 84% (9899/11784)^MUpdating files: 84% (9924/11784)^MUpdating files: 85% (10017/11784)^MUpdating files: 86% (10135/11784)^MUpdating files: 86% (10144/11784)^MUpdating files: 87% (10253/11784)^MUpdating files: 88% (10370/11784)^MUpdating files: 89% (10488/11784)^MUpdating files: 89% (10598/11784)^MUpdating files: 90% (10606/11784)^MUpdating files: 91% (10724/11784)^MUpdating files: 91% (10731/11784)^MUpdating files: 92% (10842/11784)^MUpdating files: 93% (10960/11784)^MUpdating files: 94% (11077/11784)^MUpdating files: 95% (11195/11784)^MUpdating files: 96% (11313/11784)^MUpdating files: 97% (11431/11784)^MUpdating files: 98% (11549/11784)^MUpdating files: 98% (11577/11784)^MUpdating files: 99% (11667/11784)^MUpdating files: 100% (11784/11784)^MUpdating files: 100% (11784/11784), done.
2023-05-17T12:36:54.725 DEBUG:teuthology.orchestra.run:got remote process result: 128
2023-05-17T12:36:54.726 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0...
rather than an error that shows up with `git diff`.
should we move to cloning from github.com instead of git.ceph.com for this specific test ?
git.ceph.com network connectivity is unreliable as compared to github.com
all sorts of errors crop up in the test since the input side itself isn't stable
we'd end up debugging 'git' itself instead of 'ceph', and it would be such a waste
Milind Changire wrote:
should we move to cloning from github.com instead of git.ceph.com for this specific test ?
git.ceph.com network connectivity is unreliable as compared to github.com
all sorts of errors crop up in the test since the input side itself isn't stable
we'd end up debugging 'git' itself instead of 'ceph', and it would be such a waste
This issue isn't a repo clone issue or anything related to flakey endpoints, isn't it?
Also available in: Atom
PDF