Feature #7773
closedpaddles: mirror teuthology's beanstalkd queue
0%
Updated by Ian Colle about 10 years ago
- Target version changed from sprint2 to sprint3
Updated by Ian Colle about 10 years ago
- Target version changed from sprint3 to sprint4
Updated by Ian Colle about 10 years ago
- Translation missing: en.field_story_points set to 1.0
Updated by Zack Cerza about 10 years ago
Just a note: This ticket is only intended to represent implementing a sort of mirrored queue inside paddles. Fully replacing beanstalkd as teuthology's queue is a larger undertaking.
Updated by Zack Cerza about 10 years ago
- Subject changed from use paddles for the queue to paddles: mirror teuthology's beanstalkd queue
Plan for mirroring:
- [x] paddles: Add 'queued' to possible Run.status and Job.status values
- [x] paddles: Add Job.started column
- [x] paddles: Add Run.started column
- [x] paddles: Create Alembic migration for DB schema changes
- [x] paddles: Add unit tests for the above
- [x] pulpito: Add views for queued runs
- [x] pulpito: Use Job.started (when available) to calculate job runtime instead of Job.posted
- [x] teuthology: Add some sort of method to query paddles for queued jobs in a given run
- [x] teuthology: Make teuthology-kill delete jobs from paddles
- [x] teuthology: Make teuthology-queue delete jobs from paddles
- [x] teuthology: Make teuthology-schedule delete jobs from paddles
- [x] teuthology: Make teuthology-schedule use teuthology.report to queue jobs
- [ ] This ticket is resolved
- [ ] paddles: Implement a way to pause the queue
- [ ] teuthology: Pause the queue before removing jobs from it
- [ ] Or: add some sort of a 'reserved' state between 'queued' and 'running' so that multiple jobs aren't picked up accidentally
- [ ] paddles: Come up with a scheme for generating job_ids
- ...
- [ ]
Profit!Ditch beanstalkd
Updated by Zack Cerza about 10 years ago
PR for paddles work so far:
https://github.com/ceph/paddles/pull/39
Merged here: https://github.com/ceph/paddles/commit/3c2f97059ba6802c34b2e08dffdd3a8d46bc138e
Updated by Zack Cerza about 10 years ago
Made some more paddles commits to improve transitions into the running state, and also added the alembic migration which I had forgotten to add to the PR before. And added more tests.
Started working on the pulpito changes, and in the process decided to deploy the paddles work. I ran the DB migrations and am manually populating the Job and Run started values so that I don't have to deal with edge cases in pulpito.
Updated by Zack Cerza about 10 years ago
- Translation missing: en.field_story_points changed from 1.0 to 10.0
Updated by Zack Cerza about 10 years ago
Updated the plan again with several more completed tasks. In the process I've discovered that teuthology-schedule has a method for deleting queued jobs. The way it's implemented won't work with the way I have job deletion with teuthology-queue working. I'll have to think about how to do that tomorrow.
Updated by Ian Colle about 10 years ago
- Target version changed from sprint4 to sprint5
Updated by Zack Cerza about 10 years ago
Got teuthology-schedule job deletion working.
Updated by Zack Cerza about 10 years ago
- Status changed from In Progress to Fix Under Review
Updated by Zack Cerza about 10 years ago
- Status changed from Fix Under Review to Resolved