Project

General

Profile

Actions

Bug #55815

closed

RERUN is broken does not schedule correct number of jobs

Added by Yuri Weinstein almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Category:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

Variables:

SHA1=18d575f5af7790222ee9d36af1d4518581525769
CEPH_BRANCH=wip-yuri-testing-2022-05-31-1642-quincy
CEPH_QA_MAIL="ceph-qa@ceph.io" 
CEPH_REPO=https://github.com/ceph/ceph-ci.git
SUITE_REPO=https://github.com/ceph/ceph-ci.git
LIMIT=10000
DISTRO=distro
TEUTH=master
MACHINE_NAME=smithi
PRIO=71

Run:
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-01_02:28:14-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi/

Rerun command line:

RERUN=yuriw-2022-06-01_02:28:14-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi
teuthology-suite -v -c $CEPH_BRANCH -m $MACHINE_NAME -r $RERUN --suite-repo $CEPH_REPO --ceph-repo $CEPH_REPO -p $PRIO -R fail,dead,running,waiting --force-priority -k $DISTRO

Output:

teuthology-suite -v -c $CEPH_BRANCH -m $MACHINE_NAME -r $RERUN --suite-repo $CEPH_REPO --ceph-repo $CEPH_REPO -p $PRIO -R fail,dead,running,waiting --force-priority -k $DISTRO
2022-06-01 14:00:31,155.155 INFO:teuthology.suite:Using random seed=7374
2022-06-01 14:00:31,156.156 INFO:teuthology.suite.run:kernel sha1: distro
2022-06-01 14:00:32,547.547 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci wip-yuri-testing-2022-05-31-1642-quincy -> 18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:32,547.547 INFO:teuthology.suite.run:ceph sha1: 18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:32,547.547 DEBUG:teuthology.suite.util:Defaults for machine_type smithi distro centos: arch=x86_64, release=centos/7, pkg_type=rpm
2022-06-01 14:00:32,548.548 INFO:teuthology.suite.util:container build centos/8, checking for build_complete
2022-06-01 14:00:32,548.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:32,890.890 DEBUG:teuthology.packaging:looking for centos/8 x86_64 default
2022-06-01 14:00:32,890.890 DEBUG:teuthology.packaging:build: centos/8 arm64 default
2022-06-01 14:00:32,891.891 DEBUG:teuthology.packaging:build: centos/8 x86_64 crimson
2022-06-01 14:00:32,891.891 DEBUG:teuthology.packaging:build: centos/8 x86_64 default
2022-06-01 14:00:32,891.891 INFO:teuthology.suite.run:ceph version: 17.2.0-382.g18d575f5
2022-06-01 14:00:33,136.136 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci.git wip-yuri-testing-2022-05-31-1642-quincy -> 18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:33,379.379 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci.git wip-yuri-testing-2022-05-31-1642-quincy -> 18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:33,380.380 INFO:teuthology.suite.run:ceph-ci branch: wip-yuri-testing-2022-05-31-1642-quincy 18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:00:33,381.381 DEBUG:teuthology.repo_utils:Setting repo remote to https://github.com/ceph/ceph-ci.git
2022-06-01 14:00:33,388.388 INFO:teuthology.repo_utils:Fetching wip-yuri-testing-2022-05-31-1642-quincy from origin
2022-06-01 14:00:33,983.983 INFO:teuthology.repo_utils:Resetting repo at /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy to origin/wip-yuri-testing-2022-05-31-1642-quincy
2022-06-01 14:00:34,076.076 DEBUG:teuthology.suite.run:Check file /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy/qa/.teuthology_branch exists
2022-06-01 14:00:34,076.076 DEBUG:teuthology.suite.run:Found teuthology branch config file /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy/qa/.teuthology_branch
2022-06-01 14:00:34,077.077 DEBUG:teuthology.suite.run:The teuthology branch is overridden with master
2022-06-01 14:00:34,298.298 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/teuthology master -> 1b30281f276b97a8186594b7f92fe1f728418ada
2022-06-01 14:00:34,298.298 INFO:teuthology.suite.run:teuthology branch: master 1b30281f276b97a8186594b7f92fe1f728418ada
2022-06-01 14:00:34,313.313 DEBUG:teuthology.suite.run:Suite rados in /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy/qa/suites/rados
2022-06-01 14:00:34,313.313 DEBUG:teuthology.suite.run:subset = None
2022-06-01 14:00:34,313.313 DEBUG:teuthology.suite.run:no_nested_subset = False
2022-06-01 14:07:45,177.177 INFO:teuthology.suite.run:Suite rados in /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy/qa/suites/rados generated 1591241 jobs (not yet filtered)
2022-06-01 14:07:50,550.550 DEBUG:teuthology.suite.util:Defaults for machine_type smithi distro centos: arch=x86_64, release=centos/7, pkg_type=rpm
2022-06-01 14:07:50,551.551 INFO:teuthology.suite.util:container build centos/8, checking for build_complete
2022-06-01 14:07:50,551.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:07:50,891.891 DEBUG:teuthology.packaging:looking for centos/8 x86_64 default
2022-06-01 14:07:50,891.891 DEBUG:teuthology.packaging:build: centos/8 arm64 default
2022-06-01 14:07:50,892.892 DEBUG:teuthology.packaging:build: centos/8 x86_64 crimson
2022-06-01 14:07:50,892.892 DEBUG:teuthology.packaging:build: centos/8 x86_64 default
2022-06-01 14:08:01,180.180 DEBUG:teuthology.suite.util:Defaults for machine_type smithi distro ubuntu: arch=x86_64, release=ubuntu/16.04, pkg_type=deb
2022-06-01 14:08:01,180.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:08:04,242.242 DEBUG:teuthology.suite.util:Defaults for machine_type smithi distro centos: arch=x86_64, release=centos/7, pkg_type=rpm
2022-06-01 14:08:04,243.243 INFO:teuthology.suite.util:container build centos/8, checking for build_complete
2022-06-01 14:08:04,243.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=18d575f5af7790222ee9d36af1d4518581525769
2022-06-01 14:08:04,692.692 DEBUG:teuthology.packaging:looking for centos/8 x86_64 default
2022-06-01 14:08:04,693.693 DEBUG:teuthology.packaging:build: centos/8 arm64 default
2022-06-01 14:08:04,693.693 DEBUG:teuthology.packaging:build: centos/8 x86_64 crimson
2022-06-01 14:08:04,693.693 DEBUG:teuthology.packaging:build: centos/8 x86_64 default
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858526
2022-06-01 14:08:18,583.583 INFO:teuthology.suite.run:Scheduling rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/sync workloads/pool-create-delete}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858528
2022-06-01 14:08:20,069.069 INFO:teuthology.suite.run:Scheduling rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.2}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858530
2022-06-01 14:08:21,536.536 INFO:teuthology.suite.run:Scheduling rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/master}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858532
2022-06-01 14:08:23,119.119 INFO:teuthology.suite.run:Scheduling rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/1.7.2}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858534
2022-06-01 14:08:24,600.600 INFO:teuthology.suite.run:Scheduling rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858536
2022-06-01 14:08:26,090.090 INFO:teuthology.suite.run:Scheduling rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858538
2022-06-01 14:08:27,560.560 INFO:teuthology.suite.run:Scheduling rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858540
2022-06-01 14:08:29,065.065 INFO:teuthology.suite.run:Suite rados in /home/yuriw/src/github.com_ceph_ceph-c_wip-yuri-testing-2022-05-31-1642-quincy/qa/suites/rados scheduled 7 jobs.
2022-06-01 14:08:29,065.065 INFO:teuthology.suite.run:1591234/1591241 jobs were filtered out.
Job scheduled with name yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi and ID 6858545
2022-06-01 14:08:35,962.962 INFO:teuthology.suite.run:Test results viewable at http://pulpito.front.sepia.ceph.com:80/yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi/

Result:

http://pulpito.front.sepia.ceph.com:80/yuriw-2022-06-01_14:00:31-rados-wip-yuri-testing-2022-05-31-1642-quincy-distro-default-smithi/

Expected: 15 jobs, got 7

Actions

Also available in: Atom PDF