Feature #65398
openallow images from private repos in teuthology test/ceph orch/cephadm
0%
Description
It appears as though the cephadm teuthology task supports private registries (those that require username/password login), but it doesn't seem to work. This may be because cephadm itself does not support pulls with credentials, or just the task config, or both, but it would be nice to be able to add something like
overrides:
cephadm:
image: '<my private image:tag>'
containers:
image: ''
registry-login:
url: 'https://quay.ceph.io/<private_repo>'
username: <user>
password: <password>
and have the job distribute credentials around for the pull commands.
This would be useful for testing in general, but also specifically for testing prerelease containers (stuff we build for release and want to vet before making public).
Updated by John Mulligan 23 days ago
In theory it should work. The code in the task translates the yaml paramaters to cli parameters for bootstrap. Here's a copy-paste of the block:
if config.get('registry-login'):
registry = config['registry-login']
cmd += [
"--registry-url", registry['url'],
"--registry-username", registry['username'],
"--registry-password", registry['password'],
]
Could you share, or examine, logs from your run to see if bootstrap got passed these arguments? It might help tell if the problem you hit lies with the task code or the cephadm code.
Updated by Dan Mick 23 days ago
the command that was failing was cephadm.py:pull_image, which invokes sudo cephadm --image <name> pull. I'm not 100% certain that everything in the configuration was correct so I'll test more carefully; one thing that I continue to be tripped up by is "naming the wrong tag or repo" looks exactly the same as "not enough permission" (despite a years-old bug against podman and quay) so it's possible I had multiple failures. Also, the job I'm using needs to allocate three hosts, and that can take a long time in our current lab, so cycle time on tests is longer than I'd like; I could probably simplify the test and/or semi-permanently lock some targets.
Updated by John Mulligan 23 days ago
Thanks, I see that the pull_image function doesn't honor those settings currently. I have some other somewhat related stuff in this general area so I'll take a look at this in addition to my wants.
Updated by Dan Mick 23 days ago
it's possible that the intent was to preface the call to pull_image with something that logs into the repo on the remote hosts, in which case the unadorned pull should work. I don't know the flow there. It's complicated by the fact that each remote needs to authenticate, and there isn't an "authenticate for the lifetime of this pull command" mechanism I'm aware of. If you know all that, that's fine.
Updated by John Mulligan 9 days ago
Sorry the last two weeks have been much busier than usual and this slipped my mind. I discussed this with Adam King and we thought that it may be possible to simply do the docker/podman login before the initial pull command would work but the discussion phase is where things stopped. I'm willing to look at this a little more today. But I need a private repo to test with, so if you trust me enough ping me over chat with a repo I can use and I promise to spend at least an hour on it today :-)