Project

General

Profile

Actions

Documentation #54366

closed

document: github unreachable kills dispatcher

Added by David Galloway about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Category:
-
% Done:

0%

Tags:
Backport:
Reviewed:
Affected Versions:

Description

I'm not even sure anything should be done about this but wanted to document it just in case.

Github was unreachable this morning and the smith dispatcher died because of it.

2022-02-22T09:10:05.744 INFO:teuthology.dispatcher:Creating job's archive dir /home/teuthworker/archive/benhanokh-2022-02-22_08:19:10-rados-safe_shutdown_v2_FF6-distro-basic-smithi/6700733
2022-02-22T09:10:05.767 INFO:teuthology.dispatcher:Job supervisor PID: 15603
2022-02-22T09:10:05.780 INFO:teuthology.dispatcher:Reserved job 6700734
2022-02-22T09:10:05.780 INFO:teuthology.dispatcher:Config is: description: null
email: null
first_in_suite: false
last_in_suite: true
machine_type: smithi
name: benhanokh-2022-02-22_08:19:10-rados-safe_shutdown_v2_FF6-distro-basic-smithi
owner: scheduled_benhanokh@teuthology
priority: 50
results_timeout: '43200'
tube: smithi
verbose: true

2022-02-22T09:12:15.581 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/dispatcher/__init__.py", line 125, in main
    archive_dir,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/worker.py", line 154, in prep_job
    teuthology_sha1 = ls_remote(repo_url, teuthology_branch)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/repo_utils.py", line 66, in ls_remote
    cmd, shell=True).split()
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'git ls-remote https://github.com/ceph/teuthology master' returned non-zero exit status 128.

I'm actually happy this happened so we didn't end up with hours worth of dead jobs.

Actions #1

Updated by Kamoltat (Junior) Sirivadhna about 2 years ago

  • Subject changed from github unreachable kills dispatcher to document: github unreachable kills dispatcher
  • Assignee set to Kamoltat (Junior) Sirivadhna
Actions #2

Updated by Kamoltat (Junior) Sirivadhna about 2 years ago

  • Tracker changed from Bug to Documentation
  • Status changed from New to In Progress
Actions #3

Updated by Kamoltat (Junior) Sirivadhna about 2 years ago

  • Status changed from In Progress to Resolved

merged

Actions

Also available in: Atom PDF