Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Dockerfile and script to build base docker image for hermetic builds #2830

Open
wants to merge 1 commit into
base: develop-upstream
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions tensorflow/tools/tf_sig_build_dockerfiles/Dockerfile.hermetic.ub22
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
################################################################################
FROM ubuntu:22.04
################################################################################

# Install build dependencies
COPY setup.packages.sh setup.packages.sh
COPY builder.packages.txt builder.packages.txt
RUN /setup.packages.sh /builder.packages.txt

# Install additional requirements
RUN wget -qO - https://apt.llvm.org/llvm-snapshot.gpg.key | tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
RUN echo "deb [arch=amd64 trusted=yes] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-18 main" | tee /etc/apt/sources.list.d/llvm.list
RUN apt-get update --allow-insecure-repositories
COPY setup.packages.sh /setup.packages.sh
COPY devel.packages.hermetic.txt /devel.packages.hermetic.txt
RUN /setup.packages.sh /devel.packages.hermetic.txt

# Install various tools.
# - bats: bash unit testing framework
# - bazelisk: always use the correct bazel version
# - buildifier: clean bazel build deps
# - buildozer: clean bazel build deps
# - gcloud SDK: communicate with Google Cloud Platform (GCP) for RBE, CI
RUN git clone --branch v1.7.0 https://github.com/bats-core/bats-core.git && bats-core/install.sh /usr/local && rm -rf bats-core
RUN wget https://github.com/bazelbuild/bazelisk/releases/download/v1.11.0/bazelisk-linux-amd64 -O /usr/local/bin/bazel && chmod +x /usr/local/bin/bazel
RUN wget https://github.com/bazelbuild/buildtools/releases/download/3.5.0/buildifier -O /usr/local/bin/buildifier && chmod +x /usr/local/bin/buildifier
RUN wget https://github.com/bazelbuild/buildtools/releases/download/3.5.0/buildozer -O /usr/local/bin/buildozer && chmod +x /usr/local/bin/buildozer

# All lines past this point are reset when $CACHEBUSTER is set. We need this
# for Python specifically because we install some nightly packages which are
# likely to change daily.
ARG CACHEBUSTER=0
RUN echo $CACHEBUSTER

# Setup Python environment. PYTHON_VERSION is e.g. "python3.8"
ARG PYTHON_VERSION
COPY setup.build-python_ubuntu.sh /setup.build-python_ubuntu.sh
COPY devel.requirements.txt /devel.requirements.txt
RUN /setup.build-python_ubuntu.sh $PYTHON_VERSION devel.requirements.txt

ARG TF_WHEEL_URL
RUN if [ -n "${TF_WHEEL_URL}" ]; then pip install "${TF_WHEEL_URL}"; fi
ARG DWLD_TF_SRC_CMD
RUN if [ -n "${DWLD_TF_SRC_CMD}" ]; then eval "${DWLD_TF_SRC_CMD}"; fi

# Setup build and environment
COPY devel.usertools /usertools
COPY devel.bashrc /root/.bashrc

# Don't use the bazel cache when a new docker image is created.
RUN echo build --action_env=DOCKER_CACHEBUSTER=$(date +%s%N)$RANDOM >> /etc/bazel.bazelrc
RUN echo build --host_action_env=DOCKER_HOST_CACHEBUSTER=$(date +%s%N)$RANDOM >> /etc/bazel.bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Other build-related tools
apt-transport-https
ca-certificates
colordiff
llvm-18
clang-18
curl
ffmpeg
git
gpg-agent
jq
less
libcurl3-dev
libcurl4-openssl-dev
libfreetype6-dev
libhdf5-serial-dev
libssl-dev
libtool
libzmq3-dev
lld-18
moreutils
openjdk-11-jdk
openjdk-11-jre-headless
patchelf
pkg-config
python3-dev
python3-setuptools
python3-pip
rsync
software-properties-common
sudo
swig
unzip
vim
wget
zip
zlib1g-dev