Project

General

Profile

Cleanup #6271

In scripts, separate arg parsing from actual functionality and stop chaining scripts

Added by Zack Cerza over 10 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
% Done:

0%

Tags:
Backport:
Reviewed:
Affected Versions:

Description

Here is just one example that I ran into while scratching my head trying to figure out how to discover a given suite-run's archive directory from within a job.

The teuthology-worker script is mapped to teuthology.queue.worker(). That function does a whole lot, then calls run(), which calls the teuthology script. Observe.

queue.py:
95 def worker():
96 parser = argparse.ArgumentParser(description="""
97 Grab jobs from a beanstalk queue and run the teuthology tests they
98 describe. One job is run at a time.
99 """)
100 parser.add_argument(
101 '-v', '--verbose',
102 action='store_true', default=None,
103 help='be more verbose',
104 )
[...]
194 log.info('Running job %d', job.jid)
195 run_job(job_config, archive_path, teuth_bin_path)
196 job.delete()
197
198
199 def run_job(job_config, archive_path, teuth_bin_path):
200 arg = [
201 os.path.join(teuth_bin_path, 'teuthology'),
202 ]
203
204 if job_config['verbose']:
205 arg.append('-v')
206
207 arg.extend([
208 '--lock',
209 '--block',
210 '--owner', job_config['owner'],
211 '--archive', archive_path,
212 '--name', job_config['name'],
213 ])
214 if job_config['description'] is not None:
215 arg.extend(['--description', job_config['description']])

This makes my head hurt. I can't for the life of me figure out why we're using Python to chain together Python scripts rather than simply calling functions. Aside from the fact that it all needs to be refactored. But at this point this design is really getting in the way of things.

History

#1 Updated by Ian Colle over 10 years ago

  • Assignee set to Zack Cerza

#2 Updated by Josh Durgin over 10 years ago

There are a couple reasons teuthology-worker runs teuthology as a script. Mainly it's about using a separate process for actual test runs so the workers aren't affected by errors in the teuthology tasks, and global state like the logging library doesn't need any special handling. This also lets you kill a test (e.g. if it hangs) without affecting the worker process, which continues to process more jobs from the queue.

#3 Updated by Zack Cerza over 10 years ago

  • Priority changed from High to Normal

#4 Updated by Ian Colle over 10 years ago

  • Target version changed from v0.70 to v0.71

#5 Updated by Ian Colle over 10 years ago

  • Status changed from New to Resolved

Also available in: Atom PDF