Project

General

Profile

Feature #23208

Updated by Yuri Weinstein about 6 years ago

Here is a chat from #sepia IRC for memory of what we discussed, IRC, we need to flash out details. 


 (12:21:29 PM) yuriw: I just realized that ever since we started using fog in sepia we have been under-testing other distros, b/c most of our suites don't ask for specific distro and we used to have percentages of diff OS/versions imaged on bare-metals before fog.    So the questions now is what do we do ?    schedule suites with "--distro XX --distro-version YY" - this will significantly increase amount of jobs or add add OS/versions to yaml configs with the same effect.    Other option is to add a feature to teuthology to be able to specify mix of OS/versions to be run... this may allow us to use resources more retionally.     well I'm just thinking aloud...    Any takers for other options? 
 (12:21:52 PM) yuriw: sage joshd zackc dgalloway ^ 
 (12:24:23 PM) yuriw: rationally * 
 (01:20:41 PM) yuriw:    joshd1 ^ 
 (01:53:32 PM) zackc: yuriw: last time we talked about this, i suggested adding distros to suites where they're missing 
 (01:54:19 PM) dmick: perhaps pseudorandomly sampled 
 (01:54:58 PM) yuriw:    zackc: I am not sure its practical, say you have 300+ rados jobs, adding distros will make them x4 !! 
 (01:55:14 PM) yuriw: what's "pseudorandomly sampled" ? 
 (01:56:29 PM) zackc: how would adding distros multiply? 
 (01:56:49 PM) zackc: i mean, sure if you add *multiple* distros i could see that 
 (01:58:05 PM) yuriw: we had a pool of smithi divided by dostro 30% each , adding distro will make load x time number of ditros  
 (01:58:14 PM) yuriw: distro* 
 (01:58:35 PM) gregsfortytwo: well we certainly want to run everything against more than one distro 
 (01:58:59 PM) dmick: that's what I meant, yeah; assuming you want to run the same amount of testing, but on a mix rather than a default distro 
 (01:59:38 PM) gregsfortytwo: I mean, if we add it as another dimension on the yaml fragment matrix that would get the mix 
 (01:59:55 PM) gregsfortytwo: be pretty annoying to just double it again though 
 (02:00:04 PM) yuriw: exactly ^  
 (02:00:33 PM) yuriw: or a new option in t-suite  
 (02:00:36 PM) joshd1: what's the issue with adding a separate facet for distro, and multiplying all the --subset denominators by number of distros? 
 (02:00:59 PM) yuriw: example ? 
 (02:02:12 PM) joshd1: e.g. if we have x/300 now, add a distro fragment with 4 distros to the suite, and run x/1200 instead 
 (02:03:00 PM) gregsfortytwo: means mucking around to make the crontab even more ridiculous to get the jobs down, and everybody who scheduled stuff manually has to adjust 
 (02:03:16 PM) yuriw: in other words, we use distro matrix in all suite (as oppose to none now) and use subset for all suites during scheduling  
 (02:03:38 PM) gregsfortytwo: do we have the ability to tell teuthology to fix a particular facet right now? 
 (02:03:57 PM) joshd1: sure, remove the other yaml files 
 (02:03:58 PM) yuriw: we could use --filter I'd say yes 
 (02:04:05 PM) vasu: yuriw:    just add —distro and —distro version in cronjob and rotate it based on supported distros, like 1,3,5 for centos/ 2,6 for ubuntu 
 (02:04:29 PM) vasu: one thing we want is run smoke for all distros which will catch package related issues  
 (02:04:34 PM) vasu: and it runs fast 
 (02:05:08 PM) vasu: for others we can just run subsets and rotate based on the days its runs 
 (02:05:43 PM) yuriw: as ^ "means mucking around to make the crontab even more ridiculous" 
 (02:06:25 PM) vasu: nothing ridiculous i see,    only kernel jobs are highly dependent on distros 
 (02:06:40 PM) vasu: running smoke on all distro's is more than sufficient to find package issues on daily basis 
 (02:07:24 PM) yuriw: smoke is a small and fast suite, nothing to compare vs rados and rbd  
 (02:08:20 PM) zackc: one feature i could see maybe being helpful here, is if we had another type of facet that would, instead of producing jobs containing each combination, would create only one job but with a *random* choice 
 (02:08:26 PM) zackc: not sure how helpful that would be, though 
 (02:08:51 PM) gregsfortytwo: yep, that's pretty much what we had before and would be nice 
 (02:09:04 PM) yuriw: yes but why *random* choice ? 
 (02:09:18 PM) yuriw: instead of *desired* or *fixed* 
 (02:10:05 PM) yuriw: say you asked for 3 distro, and you will get 1/3 off total amount of jobs ? 
 (02:11:09 PM) dmick: the idea would be "you want 1 job, but want it occasionally run on each of 3 distros" 
 (02:11:47 PM) vasu: zackc: are you saying add random choice instead of default ubuntu all the time? 
 (02:12:07 PM) yuriw: "you want 1 job" is applicable if you do manual scheduling, I am more worried about scheduled  
 (02:12:34 PM) dmick: of course I meant for a particular job, and no, I mean scheduled 
 (02:12:40 PM) dmick: never mind.    others will explain better. 
 (02:12:53 PM) zackc: to explain my idea a bit more: 
 (02:13:14 PM) zackc: a % inside a directory tells us: "create one job using each of the facets in here" 
 (02:13:26 PM) zackc: so 3 distros => 3x the jobs 
 (02:14:27 PM) zackc: the idea would be to perhaps use a, i dunno, $, to tell us "create one job, selecting a random facet in here" 
 (02:14:47 PM) vasu: +1 for that ^^ 
 (02:15:16 PM) zackc: this would let you get some sort of distro coverage without increasing job load, and also without changing default behavior 
 (02:16:06 PM) vasu: yes makes sense 
 (02:17:47 PM) yuriw: well, maybe its Friday, but I did not get how it won't    increase the total of jobs  
 (02:17:49 PM) yuriw: :( 
 (02:21:47 PM) dmick: "choose one of three vs choose three" 
 (02:22:54 PM) yuriw: if the result is always the same then I see  
 (02:45:20 PM) gregsfortytwo: no, it will select randomly on each invocation of the scheduler 
 (02:45:29 PM) gregsfortytwo: the point is to not always be the same 
 (02:45:39 PM) gregsfortytwo: but also not multiply the number of suites in a default run 
 (02:45:59 PM) gregsfortytwo: or require affirmative action from whoever's doing the scheduling to make sure we get a cycle 
 (02:56:12 PM) zackc: if you want the same thing every time, you have that right now

Back