Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ELK images for 9.0.0-beta1 #18475

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jmlrt
Copy link
Contributor

@jmlrt jmlrt commented Feb 18, 2025

No description provided.

@yosifkit
Copy link
Member

From the failing CI:

 Invalid 'FROM' + 'Architectures' combinations detected:

 - elasticsearch:9.0.0-beta1 (FROM docker.elastic.co/ubi9/ubi-minimal:latest) -- completely unsupported base!
 - logstash:9.0.0-beta1 (FROM docker.elastic.co/ubi8/ubi-minimal:latest) [amd64, arm64v8]
 
Read https://github.com/docker-library/official-images#multiple-architectures for more details.

In this case, these are referencing images outside Docker Official Images, which is not allowed in Official Images. There are a few Redhat ubi images allowed as exceptions listed in https://github.com/docker-library/official-images/tree/master/.external-pins/redhat, e.g. redhat/ubi9-minimal:latest.

No official images can be derived from, or depend on, non-official images

- https://github.com/docker-library/official-images/blob/5b59b4e1f2123d308af6681cc08d75a89057589f/README.md#repeatability


The Diff comment is failing because the Logstash 9.0.0-beta1 Dockerfile is referencing files that are not available in the build context:

cp: cannot stat 'tar/logstash_9.0.0-beta1/env2yaml/env2yaml.go': No such file or directory

@tianon
Copy link
Member

tianon commented Feb 18, 2025

Also, if we're touching these lines anyways, can we please update them with things like set -e so they fail properly if/when things go sideways, like we've suggested prior?

#18008 (comment)

#17158 (comment)

#16747 (comment) 👀 (which would help make sure it fails sooner, such that if it fails on our official build infrastructure that might use aggressive build cache that the cache is "busted" in the appropriate place and the rebuild has a chance at success)

To elaborate, it's failing on RUN tar -zxf /tmp/elasticsearch.tar.gz --strip-components=1 (in the Elasticsearch case), meaning the RUN curl line won't be retried on a rebuild attempt on our official infrastructure, so it will just fail indefinitely if we were to have a similar issue with the download there.

Friendly reminder: #16698 (comment) (set -e/set -ux and avoiding arch; curl -fS would be good too, but likely fail in other ways if curl were to fail)

This is good for now, but please do revisit elastic/logstash#16063 (comment) (adding at the very least set -e to all semi-colon-using RUN lines, if not the full set -eux I suggest there).

(We should also apply the same "avoiding arch in favor of querying the image userspace environment" logic to these lines...)

@jmlrt
Copy link
Contributor Author

jmlrt commented Feb 24, 2025

@jsvd can you take a look 🙏🏻

@jsvd
Copy link

jsvd commented Feb 24, 2025

Opened elastic/logstash#17147 to track the missing files on the Logstash image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants