-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cloud: add new Uploader interface and implement for AWS #1185
base: main
Are you sure you want to change the base?
Conversation
This commit switches bib to the new uploader interface in images. This requires osbuild/images#1185
This commit adds a new `upload` command that can be used to upload a raw image to the cloud. Currently only AWS is supported but as images adds more clouds to the uploader interfac we can trivially expand more. Note that this needs osbuild/images#1185
This commit adds a new `upload` command that can be used to upload a raw image to the cloud. Currently only AWS is supported but as images adds more clouds to the uploader interfac we can trivially expand more. Note that this needs osbuild/images#1185
This commit adds a new `cloud.Uploader` interface that combines the upload and register into a single operation. The rational is that with that we avoid leaking resource if e.g. the upload works but the registration fails.
It's used in our testing and was meant to be the first of a handful of similar commands for different clouds. We should still do that. |
I would prefer the opposite. Having cloud uploaders as part of image definitions might seem strange, but we discussed it a few times in the past and came to the agreement that it makes sense because integrating image definitions with a client for their target platform's API makes for a more complete story. One argument that often comes up goes "an AMI isn't an AMI until it's uploaded and registered on AWS EC2". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the end goal here and what does this PR do?
The answer I'm assuming is that it defines an interface for uploading stuff to cloud platforms and begins with one implementation based on the existing AWS code.
But I think it doesn't go far enough and isn't providing much value right now (though, the testing is nice). I'd like to see this common interface happen, but I think it makes more sense to look at the other cloud providers and consider what it might look like when we add those to the mix. UploadAndRegister()
sounds generic enough but I don't know if it makes sense for Azure, or GCP, or Oracle (it probably does). It's not set in stone, we can always change later, so maybe we should go with this and start iterating with the other platforms, but if we're going to start working on cloud clients seriously, I think we should review what we already have and potentially tear it down and build it back up in a way that works for what we're trying to do.
The boot-aws
command was written specifically for our testing, so the run
, setup
, and teardown
subcommands were written to serve development and testing and those requirements in turn informed some of the choices in the client library (though parts of it were already there from the split from osbuild-composer). For example, I never considered adding a Check()
function like we have now because it was never targeted at any user that might need that function; the keys were set in the CI setup and never looked at again and when I use it myself for troubleshooting, I have my own set that works (and have probably way more permissions than they need but let's not think about that right now). I'm all in favour of making a cloud.Uploader
interface for all our uploaders, but I also want to do it right, especially if it's going to serve osbuild-composer, ib-cli, and bib.
Now all that said, I'm fine to merge this as is.
This commit adds a new
cloud.Uploader
interface that combines the upload and register into a single operation. The rational is that with that we avoid leaking resource if e.g. the upload works but the registration fails(c.f. #1145). This would replace most of the existing bib uplodaer code so that it is shareable.
This would looks like this:
After that I would like to look into the following: