Project

General

Profile

Actions

Feature #9255

closed

lock: add os constraints

Added by Sage Weil over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Development
Tags:
Backport:
Reviewed:
Affected Versions:

Description

add lock contraints:

- os-type
- os-version
- arch

we should make the strings match what teuthology-lock and downburst accept:

$ downburst list
centos:          ['6.3', '6.4', '6.5', '7.0']
centos_minimal:          ['6.4', '6.5']
debian:          ['6.0', '7.0']
fedora:          ['17', '18', '19', '20']
opensuse:        ['12.2']
rhel:            ['6.3', '6.4', '6.5', '7.0', '7beta']
rhel_minimal:            ['6.4', '6.5']
sles:            ['11-sp2']
ubuntu:          ['8.04(hardy)', '9.10(karmic)', '10.04(lucid)', '10.10(maverick)', '11.04(natty)', '11.10(oneiric)', '12.04(precise)', '12.10(quantal)', '13.04(raring)', '13.10(saucy)', '14.04(trusty)', 'utopic(utopic)']

note for those ubuntu ones either the numeric or string names are valid.

teuthology-updatekeys should refresh these fields from the target machine as it is already run after we reimage a machine. we coudl rename the command too, tho i have no strong opinion there.

ideally, each option could either be a string literal or a list of acceptable values.

Also, it would be nice to be able to blacklist, e.g.:

os-version-not: [precise, 6.5]
Actions #1

Updated by Sage Weil over 9 years ago

  • Description updated (diff)
Actions #2

Updated by Dan Mick over 9 years ago

This would have the awesome side effect of being able to tell from the lockdb what distro/version one expects to find on a machine.

Actions #3

Updated by Zack Cerza over 9 years ago

I'm wondering where might be a sane place to put this in our job yamls.

Unrelated, do you think using a regex to match would be a good idea? I haven't thought it through; it just occured to me.

Actions #4

Updated by Sage Weil over 9 years ago

Zack Cerza wrote:

I'm wondering where might be a sane place to put this in our job yamls.

Unrelated, do you think using a regex to match would be a good idea? I haven't thought it through; it just occured to me.

regex might be handy, although I'm not sure that we need it yet.

right now when we force install of an os/version on vps, we do this:

maetl:supported 05:12 PM $ pwd
/home/sage/src/ceph-qa-suite/distros/supported
maetl:supported 05:12 PM $ cat ubuntu_14.04.yaml
os_type: ubuntu
os_version: "14.04"

can we reuse this same syntax? that would let us run the same suite (like, the upgrades) on bare metal as on vps, and each job would end up selecting machines that have the right os installed.

Actions #5

Updated by Zack Cerza over 9 years ago

  • Target version set to sprint13
Actions #6

Updated by Zack Cerza over 9 years ago

Looks like my work in #8544 is going to be useful after all. I can use it to populate paddles with node information.

Edit: I'll elaborate:

Remote.distro.name will look like 'ubuntu' or 'centos'
Remote.distro.release will look like '12.04' or '6.5'

I should add a Remote.arch - should I get that from uname -i or uname -p?

I am thinking that adding a new subtask to teuthology.task.internal, executed after lock_machines(), could push values back to paddles.

It seems to me that name, and release and arch would be enough information to store in paddles. We already store arch, and name as distro - we just need to make sure they get updated. I can add a distro_version field easily.

Edit2: I'm not actually sure if anything uses the distro value in the locks db. In that case I could rename it to os_type and we could use os_version - so that when we get Ceph running on iPads we have one less thing to worry about ;)

Actions #8

Updated by Zack Cerza over 9 years ago

Inventory work for paddles:
https://github.com/ceph/paddles/pull/45

Actions #9

Updated by Zack Cerza over 9 years ago

Paddles work is merged.

Lots of auto-population added - details in #7400.

Actions #10

Updated by Zack Cerza over 9 years ago

Looks like we already have teuthology-suite checking for exclude_arch and exclude_os_type. Adding exclude_os_version would probably be easy.

Of course, making it do something for non-vps is not trivial.

Um, so, here's a question: do we have any non-ubuntu machines in the lab that I can actually test this against?

Actions #11

Updated by Loïc Dachary over 9 years ago

There are a few rhel7 recently installed

Actions #12

Updated by Zack Cerza over 9 years ago

Ah, I found Sandon's email from when he re-imaged a few. They are:

plana31
plana37
plana40
plana54
plana91

Unfortunately they are all locked at the moment.

Actions #13

Updated by Zack Cerza over 9 years ago

  • Target version changed from sprint13 to sprint14
Actions #14

Updated by Zack Cerza over 9 years ago

  • Status changed from New to In Progress
Actions #15

Updated by Zack Cerza over 9 years ago

Whoa-boy. Turns out the handling of os_type and os_version in teuthology is very tangled.

Actions #16

Updated by Zack Cerza over 9 years ago

paddles can now lock machines for you based on os_type and os_version
https://github.com/ceph/paddles/commit/756bb69af45fcd50768aba9ecd263907fde827da

Actions #17

Updated by Zack Cerza over 9 years ago

aaand, RHEL6 doesn't have /etc/os-release. I guess I'm doing a smidge more parser work.

Actions #18

Updated by Ian Colle over 9 years ago

  • Target version changed from sprint14 to sprint15
Actions #19

Updated by Zack Cerza over 9 years ago

Merged os_type and os_version filtering.

Actions #20

Updated by Zack Cerza over 9 years ago

  • Status changed from In Progress to Resolved

I'm going to close out this ticket and open a few others, since each of these potential features are complex in their own way. What we have now is the ability to:

  • Lock machines matching a given os_type and/or os_version
  • Run jobs with machines matching a given os_type and/or os_version
What we may want is:
  • Lock/run jobs against machines matching a given arch (#9633)
  • Run jobs against a set of machines, with each one individually matching a different os_type and/or os_version (My interpretation of #9317)

If we have the above, I'm not convinced adding negative constraints is worth it.

Actions #21

Updated by Loïc Dachary over 9 years ago

The erasure code use case is to create a yaml including a task that runs a rados task using an erasure coded pool using the isa plugin. This yaml file should be used by all upgrade suites, to demonstrate that the erasure coded pool using the isa plugin can indeed be used after the upgrade. Such a test is relevant for all intel architectures that run an operating system for which the plugin was compiled.

For reference the isa plugin rados task

Actions #22

Updated by Zack Cerza over 9 years ago

Negative constraints: #9637

Actions

Also available in: Atom PDF