Project

General

Profile

Bug #48811

qa: fs/snaps/snaptest-realm-split.sh hang

Added by Patrick Donnelly 14 days ago. Updated 14 days ago.

Status:
Closed
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature:

Description

2021-01-08T06:01:25.181 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-realm-split.sh...
2021-01-08T06:01:25.181 DEBUG:teuthology.orchestra.run.smithi143:workunit test fs/snaps/snaptest-realm-split.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=07b169f75941f1cb7a6c54bf9635b099f49a6647 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-realm-split.sh
2021-01-08T06:01:25.204 INFO:tasks.workunit.client.0.smithi143.stderr:+ set -e
2021-01-08T06:01:25.205 INFO:tasks.workunit.client.0.smithi143.stderr:+ mkdir -p 1/a
2021-01-08T06:01:25.208 INFO:tasks.workunit.client.0.smithi143.stderr:+ exec
2021-01-08T06:01:25.209 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo -n a
2021-01-08T06:01:25.209 INFO:tasks.workunit.client.0.smithi143.stderr:+ mkdir 1/.snap/s1
2021-01-08T06:01:25.226 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo -n b
2021-01-08T06:01:26.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2021-01-08T06:01:26.576+0000 7f5173508700 -1 osd.0 pg_epoch: 37 pg[4.3( v 37'6390 (0'0,37'6390] local-lis/les=20/21 n=2 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) [0,5] r=0 lpr=20 luod=37'6389 crt=37'6390 lcod 36'6387 mlcod 36'6387 active+clean+snaptrim trimq=[22c~c] ps=[224~8]] removing snap head
2021-01-08T06:01:32.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2021-01-08T06:01:32.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2021-01-08T06:01:33.135 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c
2021-01-08T06:01:33.135 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c

From: /ceph/teuthology-archive/pdonnell-2021-01-08_05:07:01-fs-wip-pdonnell-testing-20210107.220115-distro-basic-smithi/5764498/teuthology.log

Second one without thrashing: /ceph/teuthology-archive/pdonnell-2021-01-08_05:07:01-fs-wip-pdonnell-testing-20210107.220115-distro-basic-smithi/5764494/teuthology.log

History

#1 Updated by Patrick Donnelly 14 days ago

  • Subject changed from qa: fs/snaps/snaptest-realm-split.sh during mon thrashing to qa: fs/snaps/snaptest-realm-split.sh hang
  • Description updated (diff)

#2 Updated by Patrick Donnelly 14 days ago

Probably related group of failures:

Timeout 3h running clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh
3 jobs: ['5764700', '5764630', '5764635']
suites intersection: ['conf/{client', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}']
suites union: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1-mds-1-client-coloc', 'clusters/1a3s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/snaps/{begin', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount/fuse', 'msgr-failures/none', 'objectstore-ec/bluestore-comp', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'tasks/workunit/snaps}', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Timeout 3h running clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh
1 jobs: ['5764568']
suites: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount/fuse', 'msgr-failures/none', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']

#3 Updated by Patrick Donnelly 14 days ago

  • Status changed from New to Closed

Also available in: Atom PDF