Actions
Documentation #54366
closeddocument: github unreachable kills dispatcher
% Done:
0%
Tags:
Backport:
Reviewed:
Affected Versions:
Description
I'm not even sure anything should be done about this but wanted to document it just in case.
Github was unreachable this morning and the smith dispatcher died because of it.
2022-02-22T09:10:05.744 INFO:teuthology.dispatcher:Creating job's archive dir /home/teuthworker/archive/benhanokh-2022-02-22_08:19:10-rados-safe_shutdown_v2_FF6-distro-basic-smithi/6700733 2022-02-22T09:10:05.767 INFO:teuthology.dispatcher:Job supervisor PID: 15603 2022-02-22T09:10:05.780 INFO:teuthology.dispatcher:Reserved job 6700734 2022-02-22T09:10:05.780 INFO:teuthology.dispatcher:Config is: description: null email: null first_in_suite: false last_in_suite: true machine_type: smithi name: benhanokh-2022-02-22_08:19:10-rados-safe_shutdown_v2_FF6-distro-basic-smithi owner: scheduled_benhanokh@teuthology priority: 50 results_timeout: '43200' tube: smithi verbose: true 2022-02-22T09:12:15.581 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-dispatcher", line 33, in <module> sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/dispatcher/__init__.py", line 125, in main archive_dir, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/worker.py", line 154, in prep_job teuthology_sha1 = ls_remote(repo_url, teuthology_branch) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/repo_utils.py", line 66, in ls_remote cmd, shell=True).split() File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'git ls-remote https://github.com/ceph/teuthology master' returned non-zero exit status 128.
I'm actually happy this happened so we didn't end up with hours worth of dead jobs.
Updated by Kamoltat (Junior) Sirivadhna about 2 years ago
- Subject changed from github unreachable kills dispatcher to document: github unreachable kills dispatcher
- Assignee set to Kamoltat (Junior) Sirivadhna
Updated by Kamoltat (Junior) Sirivadhna about 2 years ago
- Tracker changed from Bug to Documentation
- Status changed from New to In Progress
Updated by Kamoltat (Junior) Sirivadhna about 2 years ago
- Status changed from In Progress to Resolved
merged
Actions