Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set Runtime as artifact dependency #985

Closed
wants to merge 48 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0ed578b
Set Runtime as artifact dependency
gabrielbosio Dec 23, 2024
a771f3f
embed library
FrancoGiachetta Dec 30, 2024
d0ef6bb
embed library
FrancoGiachetta Dec 30, 2024
e8f5e76
link runtime correctly
FrancoGiachetta Jan 2, 2025
4ec3cce
format
FrancoGiachetta Jan 2, 2025
75922d8
Merge branch 'main' into runtime_artifact_dependency
FrancoGiachetta Jan 2, 2025
1d47835
add bindeps by default in unstable mode
FrancoGiachetta Jan 2, 2025
20da7fa
set toolchain to nightly
FrancoGiachetta Jan 2, 2025
94bedb5
clippy
FrancoGiachetta Jan 2, 2025
a97c7c6
clippy
FrancoGiachetta Jan 2, 2025
6e6bee9
fix block execution
FrancoGiachetta Jan 2, 2025
2de94fa
stable
FrancoGiachetta Jan 2, 2025
127ea2c
cargo nightly
FrancoGiachetta Jan 2, 2025
8d3d68b
cargo nightly
FrancoGiachetta Jan 2, 2025
2f76a24
cargo nightly
FrancoGiachetta Jan 2, 2025
6d533b9
fix blocks workflow
FrancoGiachetta Jan 2, 2025
1352c80
update replay version in blocks workflow
FrancoGiachetta Jan 2, 2025
4628c9e
remove runtime building in some workflows
FrancoGiachetta Jan 2, 2025
39eb9ba
try fix benches
FrancoGiachetta Jan 2, 2025
a643519
remove unwanted files
FrancoGiachetta Jan 2, 2025
cd490ae
try cargo +nightly
FrancoGiachetta Jan 3, 2025
ccc20d5
Revert "try cargo +nightly"
FrancoGiachetta Jan 3, 2025
36a0a92
Merge branch 'main' into runtime_artifact_dependency
FrancoGiachetta Jan 3, 2025
6c2198e
fix head v base bench worklfow
FrancoGiachetta Jan 3, 2025
fb96813
Merge branch 'runtime_artifact_dependency' of github.com:lambdaclass/…
FrancoGiachetta Jan 3, 2025
1d2c2bf
remove cache checks in head v base workflow
FrancoGiachetta Jan 3, 2025
cf9ff56
fix head v base workflow
FrancoGiachetta Jan 3, 2025
d5fe403
fix head v base workflow
FrancoGiachetta Jan 3, 2025
5dbebcf
fix head v base workflow and toolchain stable
FrancoGiachetta Jan 3, 2025
8a24aca
install both stable and nightly rust
FrancoGiachetta Jan 3, 2025
886c1ae
install both stable and nightly rust
FrancoGiachetta Jan 3, 2025
fcea95a
install both stable and nightly rust
FrancoGiachetta Jan 3, 2025
5d06630
install only nightly rust
FrancoGiachetta Jan 3, 2025
4fa7714
fix head v base workflow
FrancoGiachetta Jan 3, 2025
9460dce
fix blocks workflow
FrancoGiachetta Jan 3, 2025
3fabe85
fix blocks workflow
FrancoGiachetta Jan 3, 2025
5a39d0e
fix blocks workflow
FrancoGiachetta Jan 3, 2025
ea8e804
fix workflows + docker with nightly
FrancoGiachetta Jan 3, 2025
8d74368
fix workflows
FrancoGiachetta Jan 3, 2025
8b860ec
try fix docker
FrancoGiachetta Jan 3, 2025
7675cdb
fix head v base workflow
FrancoGiachetta Jan 3, 2025
98aad74
fix head v base workflow
FrancoGiachetta Jan 3, 2025
3195e11
fix head v base workflow
FrancoGiachetta Jan 3, 2025
f4a6aaa
Merge branch 'main' into runtime_artifact_dependency
FrancoGiachetta Jan 3, 2025
4f34d99
merge main
FrancoGiachetta Jan 8, 2025
0e35af5
Cargo.lock
FrancoGiachetta Jan 8, 2025
71b2eb5
Merge branch 'main' into runtime_artifact_dependency
FrancoGiachetta Jan 8, 2025
3f10099
fix runtime compilation
FrancoGiachetta Jan 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[unstable]
bindeps = true
154 changes: 89 additions & 65 deletions .github/workflows/bench-hyperfine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
df -h
- uses: dtolnay/rust-toolchain@1.83.0
- uses: dtolnay/rust-toolchain@nightly
with:
components: clippy
- uses: Swatinem/rust-cache@v2
Expand Down Expand Up @@ -83,34 +83,21 @@ jobs:
body-path: bench-hyperfine.md
edit-mode: replace

build-binaries:
run-bench-head:
env:
MLIR_SYS_190_PREFIX: /usr/lib/llvm-19/
LLVM_SYS_191_PREFIX: /usr/lib/llvm-19/
TABLEGEN_190_PREFIX: /usr/lib/llvm-19/
RUST_LOG: cairo_native=debug,cairo_native_test=debug
strategy:
matrix:
branch: [ base, head ]
name: Build Binary for ${{ matrix.branch }}
name: Run benches for HEAD
runs-on: ubuntu-24.04
steps:
- name: Cache binary
uses: actions/cache@v3
id: cache-binary
with:
path: bin/cairo-native-run-${{ matrix.branch }}
key: binary-${{ github.event.pull_request[matrix.branch].sha }}

- name: Cache runtime-library
uses: actions/cache@v3
id: cache-library
- name: Checkout HEAD commit
uses: actions/checkout@v4
with:
path: lib/libcairo-native-runtime-${{ matrix.branch }}.a
key: library-${{ github.event.pull_request[matrix.branch].sha }}
ref: ${{ github.event.pull_request.head.ref }}

- name: check and free hdd space left
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' }}
run: |
echo "Listing 20 largest packages"
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 20
Expand All @@ -129,99 +116,136 @@ jobs:
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
df -h
- name: Install Rust
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
uses: dtolnay/[email protected]

- name: Install Rust Nightly
uses: dtolnay/rust-toolchain@nightly

- name: add llvm deb repository
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
uses: myci-actions/add-deb-repo@11
with:
repo: deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-19 main
repo-name: llvm-repo
keys-asc: https://apt.llvm.org/llvm-snapshot.gpg.key

- name: Install LLVM
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
run: sudo apt-get update && sudo apt-get install llvm-19 llvm-19-dev llvm-19-runtime clang-19 clang-tools-19 lld-19 libpolly-19-dev libmlir-19-dev mlir-19-tools

- name: Checkout base commit
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
uses: actions/checkout@v4
- name: Install Hyperfine
uses: taiki-e/install-action@v2
with:
ref: ${{ github.event.pull_request[matrix.branch].sha }}

- name: Fetch Rust cache
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
uses: Swatinem/rust-cache@v2
tool: [email protected]

- name: Build binary
if: ${{ steps.cache-binary.outputs.cache-hit != 'true' || steps.cache-library.outputs.cache-hit != 'true' }}
- name: Build Cairo Native
run: |
make build
mkdir lib bin
cp target/release/cairo-native-run bin/cairo-native-run-${{ matrix.branch }}
cp target/release/libcairo_native_runtime.a lib/libcairo-native-runtime-${{ matrix.branch }}.a

run-bench:
- name: Run Program
run: |
make deps
mkdir bench-results-head
cp programs/benches/*.cairo bench-results-head
for f in bench-results-head/*.cairo; do
hyperfine \
--show-output \
--export-markdown "${f%.*}.md" \
--export-json "${f%.*}.json" \
--warmup 3 \
-n "head $(basename $f) (JIT)" "./target/release/cairo-native-run --run-mode=jit -s $f --opt-level 3 --available-gas 18446744073709551615" \
-n "head $(basename $f) (AOT)" "./target/release/cairo-native-run --run-mode=aot -s $f --opt-level 3 --available-gas 18446744073709551615" \
>> /dev/stderr
done

- name: Upload benches
uses: actions/upload-artifact@v4
with:
name: results-head
path: bench-results-head

run-bench-base:
env:
MLIR_SYS_190_PREFIX: /usr/lib/llvm-19/
LLVM_SYS_191_PREFIX: /usr/lib/llvm-19/
TABLEGEN_190_PREFIX: /usr/lib/llvm-19/
RUST_LOG: cairo_native=debug,cairo_native_test=debug
strategy:
matrix:
branch: [ base, head ]
name: Run benches for ${{ matrix.branch }}
name: Run benches for BASE
runs-on: ubuntu-24.04
needs: [ build-binaries ]
steps:
- name: Checkout base commit
- name: Checkout BASE commit
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request[matrix.branch].sha }}
ref: ${{ github.event.pull_request.base.ref }}

- name: check and free hdd space left
run: |
echo "Listing 20 largest packages"
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 20
df -h
sudo apt-get update
sudo apt-get remove -y '^llvm-.*'
sudo apt-get remove -y 'php.*'
sudo apt-get remove -y '^dotnet-.*'
sudo apt-get remove -y '^temurin-.*'
sudo apt-get remove -y azure-cli microsoft-edge-stable google-chrome-stable firefox mono-devel
sudo apt-get autoremove -y
sudo apt-get clean
df -h
echo "Removing large directories"
# deleting 15GB
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
df -h

- name: Install Rust Stable
uses: dtolnay/[email protected]

- name: Install Rust Nightly
uses: dtolnay/rust-toolchain@nightly

- name: add llvm deb repository
uses: myci-actions/add-deb-repo@11
with:
repo: deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-19 main
repo-name: llvm-repo
keys-asc: https://apt.llvm.org/llvm-snapshot.gpg.key

- name: Install LLVM
run: sudo apt-get update && sudo apt-get install llvm-19 llvm-19-dev llvm-19-runtime clang-19 clang-tools-19 lld-19 libpolly-19-dev libmlir-19-dev mlir-19-tools

- name: Install Hyperfine
uses: taiki-e/install-action@v2
with:
tool: [email protected]

- name: Fetch ${{ matrix.branch }} binary
uses: actions/cache/restore@v3
with:
path: bin/cairo-native-run-${{ matrix.branch }}
key: binary-${{ github.event.pull_request[matrix.branch].sha }}

- name: Fetch ${{ matrix.branch }} runtime-library
uses: actions/cache/restore@v3
with:
path: lib/libcairo-native-runtime-${{ matrix.branch }}.a
key: library-${{ github.event.pull_request[matrix.branch].sha }}
- name: Build Cairo Native
run: |
make build

- name: Run Program
run: |
make deps
mkdir bench-results-${{ matrix.branch }}
cp programs/benches/*.cairo bench-results-${{ matrix.branch }}
for f in bench-results-${{ matrix.branch }}/*.cairo; do
mkdir bench-results-base
cp programs/benches/*.cairo bench-results-base
for f in bench-results-base/*.cairo; do
hyperfine \
--show-output \
--export-markdown "${f%.*}.md" \
--export-json "${f%.*}.json" \
--warmup 3 \
-n "${{matrix.branch}} $(basename $f) (JIT)" "./bin/cairo-native-run-${{ matrix.branch }} --run-mode=jit -s $f --opt-level 3 --available-gas 18446744073709551615" \
-n "${{matrix.branch}} $(basename $f) (AOT)" "CAIRO_NATIVE_RUNTIME_LIBRARY=lib/libcairo-native-runtime-${{ matrix.branch }}.a ./bin/cairo-native-run-${{ matrix.branch }} --run-mode=aot -s $f --opt-level 3 --available-gas 18446744073709551615" \
-n "base $(basename $f) (JIT)" "./target/release/cairo-native-run --run-mode=jit -s $f --opt-level 3 --available-gas 18446744073709551615" \
-n "base $(basename $f) (AOT)" "./target/release/cairo-native-run --run-mode=aot -s $f --opt-level 3 --available-gas 18446744073709551615" \
>> /dev/stderr
done

- name: Upload benches ${{matrix.branch}}
- name: Upload benches
uses: actions/upload-artifact@v4
with:
name: results-${{matrix.branch}}
path: bench-results-${{matrix.branch}}
name: results-base
path: bench-results-base

show-outputs:
name: Print Bench Results
runs-on: ubuntu-24.04
needs: [ run-bench ]
needs: [ run-bench-head, run-bench-base ]
steps:
- name: Fetch Base Result
uses: actions/download-artifact@v4
Expand Down
26 changes: 12 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@ jobs:
TABLEGEN_190_PREFIX: /usr/lib/llvm-19/
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@1.83.0
- uses: dtolnay/rust-toolchain@nightly
with:
components: clippy, rustfmt

- uses: Swatinem/rust-cache@v2
- name: add llvm deb repository
uses: myci-actions/add-deb-repo@11
Expand All @@ -41,10 +42,10 @@ jobs:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@1.83.0
- uses: dtolnay/rust-toolchain@nightly
with:
components: rustfmt
- run: cargo fmt --all -- --check
- run: cargo +nightly fmt --all -- --check
- name: Find Comment
continue-on-error: true
if: success() || failure()
Expand Down Expand Up @@ -114,6 +115,7 @@ jobs:
RUST_LOG: cairo_native=debug,cairo_native_test=debug
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- name: check and free hdd space left
run: |
echo "Listing 20 largest packages"
Expand All @@ -133,8 +135,6 @@ jobs:
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
df -h
- name: Setup rust env
uses: dtolnay/[email protected]
- name: Retreive cached dependecies
uses: Swatinem/rust-cache@v2
- name: add llvm deb repository
Expand Down Expand Up @@ -166,7 +166,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Rustup toolchain install
uses: dtolnay/rust-toolchain@1.83.0
uses: dtolnay/rust-toolchain@nightly
with:
components: clippy
- name: Rust `$PATH` workaround.
Expand Down Expand Up @@ -205,6 +205,7 @@ jobs:
RUST_LOG: cairo_native=debug
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- name: check and free hdd space left
run: |
echo "Listing 20 largest packages"
Expand All @@ -224,8 +225,6 @@ jobs:
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
df -h
- name: Setup rust env
uses: dtolnay/[email protected]
- name: Retreive cached dependecies
uses: Swatinem/rust-cache@v2
- name: add llvm deb repository
Expand All @@ -246,15 +245,15 @@ jobs:
scarb-version: "2.9.2"
- name: Install deps
run: make deps
- name: Build runtime and alexandria
run: make runtime-ci && make check-llvm && make needs-cairo2 && make build-alexandria
- name: Build alexandria
run: make check-llvm && make needs-cairo2 && make build-alexandria

- name: Run tests and generate coverage partition ${{ matrix.partition }}
run: cargo llvm-cov nextest --verbose --features=scarb --workspace --lcov --output-path ${{ matrix.output }} --partition count:${{ matrix.partition }}/4
run: cargo +nightly llvm-cov nextest --verbose --features=scarb --workspace --lcov --output-path ${{ matrix.output }} --partition count:${{ matrix.partition }}/4

- name: test and generate coverage corelib
if: ${{ matrix.partition == '1' }}
run: cargo llvm-cov nextest --verbose --features=scarb --lcov --output-path lcov-test.info run --bin cairo-native-test -- corelib
run: cargo +nightly llvm-cov nextest --verbose --features=scarb --lcov --output-path lcov-test.info run --bin cairo-native-test -- corelib

- name: save coverage data with corelib
if: ${{ matrix.partition == '1' }}
Expand All @@ -277,8 +276,7 @@ jobs:
runs-on: ubuntu-24.04
needs: [coverage]
steps:
- name: Setup rust env
uses: dtolnay/[email protected]
- uses: dtolnay/rust-toolchain@nightly
- name: Retreive cached dependencies
uses: Swatinem/rust-cache@v2
- name: Install testing tools
Expand Down
10 changes: 2 additions & 8 deletions .github/workflows/daily.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ jobs:
with:
repository: lambdaclass/starknet-replay
path: starknet-replay

# We need native if we want to run with cairo native main
- name: Checkout Native
uses: actions/checkout@v4
Expand All @@ -93,20 +94,13 @@ jobs:
# Install dependencies
- uses: ./cairo_native/.github/actions/install-linux-deps
- name: Setup rust env
uses: dtolnay/rust-toolchain@1.83.0
uses: dtolnay/rust-toolchain@nightly
- name: Retreive cached dependecies
uses: Swatinem/rust-cache@v2
with:
workspaces: "starknet-replay -> target"
cache-directories: "cairo_native"

- name: Build Cairo Native Runtime Library
shell: bash
run: |
cd ../cairo_native
make runtime
echo "CAIRO_NATIVE_RUNTIME_LIBRARY=$(pwd)/libcairo_native_runtime.a" > $GITHUB_ENV

- name: Patch replay dependencies
run: |
# Updates sequencer dependency to local path
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/local/lib/android
- name: Setup rust env
uses: dtolnay/rust-toolchain@1.83.0
uses: dtolnay/rust-toolchain@nightly
- name: Retreive cached dependecies
uses: Swatinem/rust-cache@v2
- name: add llvm deb repository
Expand Down Expand Up @@ -60,7 +60,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Rustup toolchain install
uses: dtolnay/rust-toolchain@1.83.0
uses: dtolnay/rust-toolchain@nightly
with:
components: clippy
- name: Rust `$PATH` workaround.
Expand Down
Loading
Loading