Skip to content

Commit

Permalink
Merge pull request #9 from scipy-conference/feat/add-pixi-project-sup…
Browse files Browse the repository at this point in the history
…port

feat: Add pixi support for multi platform lock files

* Add `pixi` support so that the entire project can be run with the same
  dependencies across the linux-64, osx-64, osx-arm64 platforms with the
  same lock file (which installs in seconds).
* Add `pixi` support to the CI.
  • Loading branch information
matthewfeickert authored Mar 29, 2024
2 parents ec4cda7 + 16542f6 commit d0232ef
Show file tree
Hide file tree
Showing 9 changed files with 6,430 additions and 1,279 deletions.
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML
18 changes: 6 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,15 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
- name: Set up pixi
uses: prefix-dev/setup-[email protected]
with:
python-version: "3.12"

- name: Install dependencies
run: |
python -m pip install uv
python -m uv pip install --system --upgrade pip wheel
uv pip install --system pytest coverage
uv pip install --system --strict --requirement requirements.lock
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}

- name: List installed Python packages
run: python -m pip list
run: pixi list

- name: Test with pytest and coverage
run: |
coverage run --module pytest tests/
pixi run --environment test test
21 changes: 9 additions & 12 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,19 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
- name: Set up pixi
uses: prefix-dev/setup-[email protected]
with:
python-version: "3.x"

- name: Install dependencies
run: |
python -m pip install uv
python -m uv pip install --system --upgrade pip wheel
python -m uv pip install --system ruff black
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}

- name: List installed Python packages
run: python -m pip list
run: pixi list

- name: Lint with ruff
run: ruff check .
run: |
pixi run --environment lint ruff
- name: Check formatting with black
run: black . --check --verbose
run: |
pixi run --environment lint black
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ venv/
*ipynb_checkpoints
.DS_Store
*.ipynb
# pixi environments
.pixi
37 changes: 27 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,34 +23,51 @@ Locally, you can open it in VS Code with the Dev Containers extension installed.

## Without devcontainer

If you can't or don't want to use the devcontainer, then you should first create a virtual environment:
If you can't or don't want to use the devcontainer, then use [`pixi`](https://pixi.sh/) to control the application.
If you don't have `pixi` installed yet, follow the 1-liner [install command](https://pixi.sh/latest/#installation) for the Rust binary for your operating system.

Then to install the full environment from the multi platform lock file simply just run

```
pixi install
```

To execute a specific task defined in the [task runner section](https://pixi.sh/latest/advanced/advanced_tasks/) just run

```
python3 -m venv .venv
source .venv/bin/activate
pixi run <task name>
```

Then with the virtual environment activated, install the dev tools and pre-commit hooks:
So for example, to run all the tests run

```
python3 -m pip install -r requirements-dev.txt
pre-commit install
pixi run --environment test test
```

To install the requirements for the application from the `uv` generated lock file, run:
or to lint

```
uv pip install --strict --requirement requirements.lock
pixi run --environment lint lint
```

If you would like to have interactive shell access (like a classic virtual environment) run

```
pixi shell
```

and you will be dropped into a new shell with the environment activated.

## Updating the lock file

To regenerate the lock file from the high level `requirements.txt` run
To regenerate the lock file from the project `pixi.toml` run

```
uv pip compile requirements.txt --generate-hashes --output-file requirements.lock
rm pixi.lock && pixi install
```

This will be very fast!

## Adding code and tests

This repository starts with a very simple `main.py` and a test for it at `tests/main_test.py`.
Expand Down
6,339 changes: 6,339 additions & 0 deletions pixi.lock

Large diffs are not rendered by default.

44 changes: 44 additions & 0 deletions pixi.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
[project]
name = "assign-reviews"
version = "0.1.0"
description = "Workflow to assign reviewers for the SciPy Conference"
authors = [
"Guen Prawiroatmodjo <[email protected]>",
"Matthew Feickert <[email protected]>"
]
channels = ["conda-forge"]
platforms = ["linux-64", "osx-64", "osx-arm64"]

[tasks]

[dependencies]
python = "3.12.*"
scipy = ">=1.10.0"
pandas = ">=2.2.0"
notebook = ">=7.1.2"
jupytext = ">=1.10.0"

[pypi-dependencies]
duckdb = { version = ">=0.10.0" }
duckdb-engine = { version = ">=0.11.2" }

[feature.test.tasks]
test = "coverage run --module pytest tests/"

[feature.test.dependencies]
pytest = ">=8.0.0"
coverage = ">=7.0.0"

[feature.lint.tasks]
lint = "pre-commit run --all-files"
ruff = "ruff check ."
black = "black . --check --verbose"

[feature.lint.dependencies]
pre-commit = ">=3.7.0"
ruff = ">=0.3.4"
black = ">=24.3.0"

[environments]
test = ["test"]
lint = ["lint"]
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ line-length = 120
target-version = ["py311"]

[tool.pytest.ini_options]
addopts = "-ra --cov"
addopts = "-ra"
testpaths = ["tests"]
pythonpath = ['.']

Expand Down
Loading

0 comments on commit d0232ef

Please sign in to comment.