Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Github] Add Agent Container Image #123486

Merged
merged 1 commit into from
Jan 20, 2025

Conversation

boomanaiden154
Copy link
Contributor

This patch adds an agent container image on top of the normal CI container image. They are the exact same except that the agent container image also contains Github Runner binaries. I've split it into a separate container as only one user of these images (the new premerge) needs this binary installed, and it bloats the container image size significantly (900MB->1.3GB or so).

This patch adds an agent container image on top of the normal CI container
image. They are the exact same except that the agent container image also
contains Github Runner binaries. I've split it into a separate container as
only one user of these images (the new premerge) needs this binary installed,
and it bloats the container image size significantly (900MB->1.3GB or so).
@llvmbot
Copy link
Member

llvmbot commented Jan 18, 2025

@llvm/pr-subscribers-github-workflow

Author: Aiden Grossman (boomanaiden154)

Changes

This patch adds an agent container image on top of the normal CI container image. They are the exact same except that the agent container image also contains Github Runner binaries. I've split it into a separate container as only one user of these images (the new premerge) needs this binary installed, and it bloats the container image size significantly (900MB->1.3GB or so).


Full diff: https://github.com/llvm/llvm-project/pull/123486.diff

2 Files Affected:

  • (modified) .github/workflows/build-ci-container.yml (+9-2)
  • (modified) .github/workflows/containers/github-action-ci/Dockerfile (+12-1)
diff --git a/.github/workflows/build-ci-container.yml b/.github/workflows/build-ci-container.yml
index 4fa0713b381ceb..50e7a0bcc679af 100644
--- a/.github/workflows/build-ci-container.yml
+++ b/.github/workflows/build-ci-container.yml
@@ -36,19 +36,22 @@ jobs:
           tag=`date +%s`
           container_name="ghcr.io/$GITHUB_REPOSITORY_OWNER/ci-ubuntu-22.04"
           echo "container-name=$container_name" >> $GITHUB_OUTPUT
+          echo "container-name-agent=$container_name-agent" >> $GITHUB_OUTPUT
           echo "container-name-tag=$container_name:$tag" >> $GITHUB_OUTPUT
+          echo "container-name-agent-tag=$container_name-agent:$tag" >> $GITHUB_OUTPUT
           echo "container-filename=$(echo $container_name:$tag  | sed -e 's/\//-/g' -e 's/:/-/g').tar" >> $GITHUB_OUTPUT
       - name: Build container
         working-directory: ./.github/workflows/containers/github-action-ci/
         run: |
-          podman build -t ${{ steps.vars.outputs.container-name-tag }} .
+          podman build --target ci-container -t ${{ steps.vars.outputs.container-name-tag }} .
+          podman build --target ci-container-agent -t ${{ steps.vars.outputs.container-name-agent-tag }} .
 
       # Save the container so we have it in case the push fails.  This also
       # allows us to separate the push step into a different job so we can
       # maintain minimal permissions while building the container.
       - name: Save container image
         run: |
-          podman save  ${{ steps.vars.outputs.container-name-tag }} >  ${{ steps.vars.outputs.container-filename }}
+          podman save  ${{ steps.vars.outputs.container-name-tag }} ${{ steps.vars.outputs.container-name-agent-tag }} >  ${{ steps.vars.outputs.container-filename }}
 
       - name: Upload container image
         uses: actions/upload-artifact@v4
@@ -86,3 +89,7 @@ jobs:
           podman login -u ${{ github.actor }} -p $GITHUB_TOKEN ghcr.io
           podman push ${{ needs.build-ci-container.outputs.container-name-tag }}
           podman push ${{ needs.build-ci-container.outputs.container-name }}:latest
+
+          podman tag ${{ needs.build-ci-container.outpus.container-name-agent-tag }} ${{ needs.build-ci-container.outputs.container-name-agent }}:latest
+          podman push ${{ needs.build-ci-container.outputs.container-name-agent-tag }}
+          podman push ${{ needs.build-ci-container.outputs.container-name-agent }}:latest
diff --git a/.github/workflows/containers/github-action-ci/Dockerfile b/.github/workflows/containers/github-action-ci/Dockerfile
index 3757e603f8a102..01bfd7a53db903 100644
--- a/.github/workflows/containers/github-action-ci/Dockerfile
+++ b/.github/workflows/containers/github-action-ci/Dockerfile
@@ -38,7 +38,7 @@ RUN cmake -B ./build -G Ninja ./llvm \
 
 RUN ninja -C ./build stage2-clang-bolt stage2-install-distribution && ninja -C ./build install-distribution
 
-FROM base
+FROM base as ci-container
     
 COPY --from=stage1-toolchain $LLVM_SYSROOT $LLVM_SYSROOT
 
@@ -91,4 +91,15 @@ RUN adduser gha sudo
 RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
 
 USER gha
+WORKDIR /home/gha
+
+FROM ci-container as ci-container-agent
+
+ENV GITHUB_RUNNER_VERSION=2.321.0
+
+RUN mkdir actions-runner && \
+    cd actions-runner && \
+    curl -O -L https://github.com/actions/runner/releases/download/v$GITHUB_RUNNER_VERSION/actions-runner-linux-x64-$GITHUB_RUNNER_VERSION.tar.gz && \
+    tar xzf ./actions-runner-linux-x64-$GITHUB_RUNNER_VERSION.tar.gz && \
+    rm ./actions-runner-linux-x64-$GITHUB_RUNNER_VERSION.tar.gz
 

Copy link
Collaborator

@tstellar tstellar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@boomanaiden154 boomanaiden154 merged commit d35d7f4 into main Jan 20, 2025
10 checks passed
@boomanaiden154 boomanaiden154 deleted the users/boomanaiden154/add-agent-linux-container branch January 20, 2025 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants