Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[build] Migrate to uv #1803

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions .github/workflows/code-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,17 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python

- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version: "3.10"
cache: 'pip'
cache-dependency-path: |
**/pyproject.toml
**/constraints.txt
**/requirements-dev.txt
- uses: pre-commit/[email protected]
python-version-file: ".python-version"

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: "Run pre-commit"
uses: pre-commit/[email protected]
100 changes: 25 additions & 75 deletions .github/workflows/daily-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,100 +5,50 @@ on:
- cron: '0 4 * * *'
workflow_dispatch:

## COMMENTED OUT: only for testing CI action changes
# pull_request:
# branches:
# - main
## END
# COMMENTED OUT: only for testing CI action changes
pull_request:
branches:
- main
# END

jobs:
daily-ci:
strategy:
matrix:
dependencies-factor: ["lowest-direct", "highest"]
module-factor: ["cartesian", "eve", "next", "storage"]
os: ["ubuntu-latest"] #, "macos-latest"]
python-version: ["3.10", "3.11"]
tox-module-factor: ["cartesian", "eve", "next", "storage"]
os: ["ubuntu-latest"]
requirements-file: ["requirements-dev.txt", "min-requirements-test.txt", "min-extra-requirements-test.txt"]
fail-fast: false

runs-on: ${{ matrix.os }}
steps:

- uses: actions/checkout@v4

- name: Install C++ libraries
if: ${{ matrix.os == 'macos-latest' }}
shell: bash
run: |
brew install boost
run: brew install boost

- name: Install C++ libraries
if: ${{ matrix.os == 'ubuntu-latest' }}
shell: bash
run: |
sudo apt install libboost-dev
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
run: sudo apt install libboost-dev

- name: Install uv and set the python version
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: |
**/pyproject.toml
**/constraints.txt
**/requirements-dev.txt
- name: Install tox
run: |
python -m pip install -c ./constraints.txt pip setuptools wheel tox
python -m pip list
- name: Update requirements
run: |
pyversion=${{ matrix.python-version }}
pyversion_no_dot=${pyversion//./}
tox run -e requirements-py${pyversion_no_dot}
# TODO(egparedes): add notification for dependencies updates
# - name: Check for updated requirements
# id: update-requirements
# continue-on-error: true
# if: ${{ matrix.python-version == '3.8' && matrix.tox-module-factor == 'cartesian' }}
# shell: bash
# run: |
# if diff -q constraints.txt CURRENT-constraints.txt; then
# echo "REQS_DIFF=''" >> $GITHUB_OUTPUT
# else
# diff --changed-group-format='%<' --unchanged-group-format='' constraints.txt CURRENT-constraints.txt | tr '\n' ' ' > constraints.txt.diff
# echo "REQS_DIFF='$(cat constraints.txt.diff)'" >> $GITHUB_OUTPUT
# fi
# echo "REQS_DIFF_TEST="FOOOOOOOO" >> $GITHUB_OUTPUT
# - name: Notify updated requirements (if any)
# if: ${{ steps.update-requirements.outputs.REQS_DIFF }}
# env:
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
# uses: slackapi/[email protected]
# with:
# channel-id: ${{ vars.SLACK_BOT_CHANNEL }}
# payload: |
# {
# "text": "TEXT",
# "blocks": [
# {
# "type": "section",
# "text": {
# "type": "plain_text",
# "text": "@channel: AA/${{ steps.update-requirements.outputs.REQS_DIFF }}/BB/ ${{ steps.update-requirements.outputs.REQS_DIFF_TEST }} /CC"
# }
# },
# {
# "type": "section",
# "text": {
# "type": "mrkdwn",
# "text": "@channel: AA/${{ steps.update-requirements.outputs.REQS_DIFF }}/BB/ ${{ steps.update-requirements.outputs.REQS_DIFF_TEST }} /CC"
# }
# }
# ]
# }
- name: Run tests

- name: Run CPU tests for '${{ matrix.module-factor }}' with '${{ matrix.dependencies-factor }}' resolution strategy
env:
NUM_PROCESSES: auto
ENV_REQUIREMENTS_FILE: ${{ matrix.requirements-file }}
run: |
tox run --skip-missing-interpreters -m test-${{ matrix.tox-module-factor }}-cpu
UV_RESOLUTION: ${{ matrix.dependencies-factor }}
run: uv run nox -s '${{ matrix.module-factor }}_tests-${{ matrix.python-version }}' -t 'cpu'

- name: Notify slack
if: ${{ failure() }}
env:
Expand All @@ -108,13 +58,13 @@ jobs:
channel-id: ${{ vars.SLACK_BOT_CHANNEL }}
payload: |
{
"text": "${{ github.workflow }}: `test-${{ matrix.tox-module-factor }}-cpu (python${{ matrix.python-version }})`>: *Failed tests!*",
"text": "${{ github.workflow }}: `Daily CI: '**${{ matrix.module-factor }} (CPU)** for **Python-${{ matrix.python-version }}**>: *Failed tests!*",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "<https://github.com/GridTools/gt4py/actions/runs/${{ github.run_id }}|${{ github.workflow }}: `test-${{ matrix.tox-module-factor }}-cpu (python${{ matrix.python-version }})`>: *Failed tests!*"
"text": "<https://github.com/GridTools/gt4py/actions/runs/${{ github.run_id }}|${{ github.workflow }}: **${{ matrix.module-factor }} (CPU)** for **Python-${{ matrix.python-version }}**>: *Failed tests!*"
}
}
]
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-cartesian-fallback.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ on:

jobs:
test-cartesian:
runs-on: ubuntu-latest
strategy:
matrix:
codegen-factor: [internal, dace]
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11"]
tox-factor: [internal, dace]

runs-on: ${{ matrix.os }}
steps:
- run: 'echo "No build required"'
41 changes: 21 additions & 20 deletions .github/workflows/test-cartesian.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,35 +20,36 @@ concurrency:

jobs:
test-cartesian:
runs-on: ubuntu-latest
strategy:
matrix:
codegen-factor: [internal, dace]
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11"]
tox-factor: [internal, dace]
fail-fast: false

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4

- name: Install C++ libraries
if: ${{ matrix.os == 'macos-latest' }}
shell: bash
run: |
sudo apt install libboost-dev
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
run: brew install boost

- name: Install C++ libraries
if: ${{ matrix.os == 'ubuntu-latest' }}
shell: bash
run: sudo apt install libboost-dev

- name: Install uv and set the python version
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: |
**/pyproject.toml
**/constraints.txt
**/requirements-dev.txt
- name: Install python dependencies
run: |
python -m pip install -c ./constraints.txt pip setuptools wheel
python -m pip install -r ./requirements-dev.txt
- name: Test with tox

- name: Run CPU 'cartesian' tests with nox
env:
NUM_PROCESSES: auto
shell: bash
run: |
pyversion=${{ matrix.python-version }}
pyversion_no_dot=${pyversion//./}
tox run -e cartesian-py${pyversion_no_dot}-${{ matrix.tox-factor }}-cpu
run: uv run nox -s 'cartesian_tests-${{ matrix.python-version }}(${{ matrix.codegen-factor }}, cpu)'
10 changes: 6 additions & 4 deletions .github/workflows/test-eve-fallback.yml
Original file line number Diff line number Diff line change
@@ -1,24 +1,26 @@
name: "Fallback: Test Eve"

on:
push:
branches:
- main
pull_request:
branches:
- main
paths-ignore: # Inverse of corresponding workflow
- "src/gt4py/eve/**"
- "tests/eve_tests/**"
- "workflows/**"
- "*.cfg"
- "*.ini"
- ".github/workflows/**"
- "*.lock"
- "*.toml"
- "*.yml"

jobs:
test-eve:
strategy:
matrix:
python-version: ["3.10", "3.11"]
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11"]

runs-on: ${{ matrix.os }}
steps:
Expand Down
51 changes: 11 additions & 40 deletions .github/workflows/test-eve.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,61 +10,32 @@ on:
paths: # Run when gt4py.eve files (or package settings) are changed
- "src/gt4py/eve/**"
- "tests/eve_tests/**"
- "workflows/**"
- "*.cfg"
- "*.ini"
- ".github/workflows/**"
- "*.lock"
- "*.toml"
- "*.yml"

jobs:
test-eve:
strategy:
matrix:
python-version: ["3.10", "3.11"]
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11"]
fail-fast: false

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5

- name: Install uv and set the python version
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: |
**/pyproject.toml
**/constraints.txt
**/requirements-dev.txt
- name: Install python dependencies
run: |
python -m pip install -c ./constraints.txt pip setuptools wheel
python -m pip install -r ./requirements-dev.txt
- name: Run tox tests

- name: Run 'eve' tests with nox
env:
NUM_PROCESSES: auto
shell: bash
run: |
pyversion=${{ matrix.python-version }}
pyversion_no_dot=${pyversion//./}
tox run -e eve-py${pyversion_no_dot}
# mv coverage.json coverage-py${{ matrix.python-version }}-${{ matrix.os }}.json
# - name: Upload coverage.json artifact
# uses: actions/upload-artifact@v3
# with:
# name: coverage-py${{ matrix.python-version }}-${{ matrix.os }}
# path: coverage-py${{ matrix.python-version }}-${{ matrix.os }}.json
# - name: Gather info
# run: |
# echo ${{ github.ref_type }} >> info.txt
# echo ${{ github.ref }} >> info.txt
# echo ${{ github.sha }} >> info.txt
# echo ${{ github.event.number }} >> info.txt
# echo ${{ github.event.pull_request.head.ref }} >> info.txt
# echo ${{ github.event.pull_request.head.sha }} >> info.txt
# echo ${{ github.run_id }} >> info.txt
# - name: Upload info artifact
# uses: actions/upload-artifact@v3
# with:
# name: info-py${{ matrix.python-version }}-${{ matrix.os }}
# path: info.txt
run: uv run nox -s eve_tests-${{ matrix.python-version }}
5 changes: 3 additions & 2 deletions .github/workflows/test-next-fallback.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ jobs:
test-next:
strategy:
matrix:
python-version: ["3.10", "3.11"]
tox-factor: ["nomesh", "atlas"]
codegen-factor: [internal, dace]
mesh-factor: [nomesh, atlas]
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11"]

runs-on: ${{ matrix.os }}
steps:
Expand Down
Loading
Loading