teuthology-suite should check presence/translate to sha1 based on the actual request
teuthology-suite does several things based on the 'default distro/version', including at least:
1) translating branch/tag to sha1
2) checking for presence of a requested branch/tag/sha1
This leads to confusing situations where, say, the centos gb thinks 'testing' is sha1a, but the ubuntu gb thinks it's sha1b, and so an otherwise-proper job fails or installs the wrong version.
There's no reason the gbs should have to be in sync.
#4 Updated by Andrew Schoen over 4 years ago
Pasting a conversation here from #sepia. There is concern that testing with a mix of sha1s per distro would not be ideal.
[10:51:29] <jcsp> ah -- so teuthology needs to do it by going straight to github to learn about what sha1s exist, and then poll the builders to learn which sha1s are built, and then make a decision [10:52:36] <andrewschoen> jcsp: can you explain more why "having them (potentially) consist of a mix of versions is kind of scary"? [10:52:40] <jcsp> er [10:52:46] <jcsp> let me run my imagination wild for a second :-) [10:53:04] kefu (~email@example.com) left IRC (Max SendQ exceeded) [10:53:08] <jcsp> so when we introduce bugs, I'll get a mixture of buggy and non buggy nodes [10:53:13] <yuriw> *reality is wild !* [10:53:18] <jcsp> and I'll look at the buggy node doing something strange [10:53:31] kefu (~firstname.lastname@example.org) joined the channel [10:53:32] <jcsp> and then read the non-buggy node's log and eliminiate some particular failure mode based on its behaviour [10:53:40] <jcsp> unless I *realise* they're running different versions [10:53:55] <jcsp> in which case I have to look at every failure individually rather than ever being able to make generalisations [10:53:57] <jcsp> also [10:54:13] <jcsp> from a practical POV, when I read the code from a given test run, I go look at one config.yaml to see what sha1 it was [10:54:16] <jcsp> and pull up the code [10:54:24] <jcsp> if they're running different versions, I'm flipping back and forth potentially [10:54:45] <jcsp> aaaalso [10:54:53] <jcsp> I might add client vs. server changes in separate commits [10:54:57] <jcsp> and one might depend on the other [10:55:03] <jcsp> it's kind of endless really! [10:55:24] <jcsp> having mixtures of proper, released versions is something that we explicitly test, and we make sure it works inasmuch as we support it [10:55:35] <jcsp> but mixing around commits on master, that's just asking for trouble [10:55:59] <jcsp> *rant over*