Add release GitHub actions (#336)

* add __version__

* add ccds.__version__

* add make dist

* add release.yml

* update python version

* add publish to pypi

* use currently released version by default

* Update README and docs

* find the index of the checkout branch parameter in cookiecutter by using its parameter name rather than a fixed index

* update readme with accurate version info

* add releasing instructions, fix version string for default tag download

* Apply suggestions from code review

Documentation changes

Co-authored-by: Robert Gibboni <[email protected]>

* add deprecation warning if running <2.0.1 that the user is always pulling the latest template version

* lint

* Update hooks/pre_prompt.py

Co-authored-by: Robert Gibboni <[email protected]>

* update warning string [skip ci]

---------

Co-authored-by: Chris Kucharczyk <[email protected]>
Co-authored-by: Chris Kucharczyk <[email protected]>
Co-authored-by: Robert Gibboni <[email protected]>

.github/workflows/release.yml

@@ -0,0 +1,52 @@
+name: release
+
+on:
+  release:
+    types:
+      - published
+
+jobs:
+  build:
+    name: Build and publish new release
+    runs-on: "ubuntu-latest"
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r dev-requirements.txt
+
+      - name: Check that versions match
+        id: version
+        run: |
+          echo "Release tag: [${{ github.event.release.tag_name }}]"
+          PACKAGE_VERSION=$(python -c "import ccds; print(ccds.__version__)")
+          echo "Package version: [$PACKAGE_VERSION]"
+          [ ${{ github.event.release.tag_name }} == "v$PACKAGE_VERSION" ] || { exit 1; }
+          echo "::set-output name=major_minor_version::v${PACKAGE_VERSION%.*}"
+
+      - name: Build package
+        run: |
+          make dist
+          
+      - name: Publish to Test PyPI
+        uses: pypa/[email protected]
+        with:
+          user: ${{ secrets.PYPI_TEST_USERNAME }}
+          password: ${{ secrets.PYPI_TEST_PASSWORD }}
+          repository_url: https://test.pypi.org/legacy/
+          skip_existing: true
+
+      - name: Publish to Production PyPI
+        uses: pypa/[email protected]
+        with:
+          user: ${{ secrets.PYPI_PROD_USERNAME }}
+          password: ${{ secrets.PYPI_PROD_PASSWORD }}
+          skip_existing: false

HISTORY.md

@@ -3,6 +3,7 @@
 ## UNRELEASED
 
 - Fixes issue with scaffold code that import of config did not work. Adds testing of imports to test suite. (Issue [#370](https://github.com/drivendataorg/cookiecutter-data-science/issues/370))
+- Create automated release mechanism (Issue [#317](https://github.com/drivendataorg/cookiecutter-data-science/issues/317)) and pin template version to installed release (Issue [#389](https://github.com/drivendataorg/cookiecutter-data-science/issues/389))
 
 ## v2.0.0 (2024-05-22)
 

Makefile

@@ -34,6 +34,31 @@ lint:
 	isort --check --profile black ccds hooks tests docs/scripts
 	black --check ccds hooks tests docs/scripts
 
+clean: clean-build clean-pyc clean-test ## remove all build, test, coverage and Python artifacts
+
+clean-build: ## remove build artifacts
+	rm -fr build/
+	rm -fr dist/
+	rm -fr .eggs/
+	find . -name '*.egg-info' -exec rm -fr {} +
+	find . -name '*.egg' -exec rm -f {} +
+
+clean-pyc: ## remove Python file artifacts
+	find . -name '*.pyc' -exec rm -f {} +
+	find . -name '*.pyo' -exec rm -f {} +
+	find . -name '*~' -exec rm -f {} +
+	find . -name '__pycache__' -exec rm -fr {} +
+
+clean-test: ## remove test and coverage artifacts
+	rm -fr .tox/
+	rm -f .coverage
+	rm -fr htmlcov/
+	rm -fr .pytest_cache
+
+dist: clean ## builds source and wheel package
+	python -m build
+	ls -l dist
+
 
 ###     DOCS
 

README.md

@@ -82,6 +82,14 @@ The directory structure of your new project will look something like this (depen
     └── plots.py                <- Code to create visualizations   
 ```
 
+## Using unreleased changes
+
+By default, `ccds` will use the _project template_ version that corresponds to the _installed `ccds` package_ version (e.g., if you have installed `ccds` v2.0.1, you'll use the v2.0.1 version of the project template by default). To use a specific version of the project template, use the `-c/--checkout` flag to provide the branch (or tag or commit hash) of the version you'd like to use. For example to use the project template from the `master` branch:
+
+```bash
+ccds -c master
+```
+
 ## Using v1
 
 If you want to use the old v1 project template, you need to have either the cookiecutter-data-science package or cookiecutter package installed. Then, use either command-line program with the `-c v1` option:

RELEASING.md

@@ -0,0 +1,11 @@
+# Information for releases and versioning of ccds
+
+## Background
+
+The release of [ccds v2](https://drivendata.co/blog/ccds-v2) introduced the `ccds` utility and the concept of versioning to cookiecutter data science. Prior to this release, cookiecutter-data-science only provided a project template, which the generic [cookiecutter](https://github.com/cookiecutter/cookiecutter) utility could use to instantiate a project. Branches and forks could be used in the usual way to get different versions of the template.
+
+To give the utility and the template a bit more stability, PR [#336](https://github.com/drivendataorg/cookiecutter-data-science/pull/336) created automated release mechanics for publishing new releases and, by default, pinned the template used by the `ccds` utility to the installed version. 
+
+## Issuing a new release
+
+`ccds` uses [semantic versioning](https://semver.org/). When issuing a new release, **ensure that your release version tag has the format `vMAJOR.MINOR.PATCH`. The `v` prefix is important because the utility will look for the tag with that name to download by default.

ccds/__init__.py

@@ -0,0 +1,3 @@
+from ccds.version import __version__
+
+__version__

ccds/__main__.py

@@ -23,6 +23,8 @@
 from cookiecutter import cli
 from cookiecutter import main as api_main  # noqa: F401 referenced by tests
 
+from ccds.version import __version__
+
 
 def default_ccds_main(f):
     """Set the default for the cookiecutter template argument to the CCDS template."""
@@ -31,6 +33,11 @@ def _main(*args, **kwargs):
         f.params[1].default = (
             "https://github.com/drivendataorg/cookiecutter-data-science"
         )
+        # Find the "checkout" option in the cookiecutter cli (currently the fifth)
+        # Per #389, set this to the currently released version by default
+        param_names = [p.name for p in f.params]
+        checkout_index = param_names.index("checkout")
+        f.params[checkout_index].default = f"v{__version__}"
         return f(*args, **kwargs)
 
     return _main

ccds/version.py

@@ -0,0 +1,9 @@
+import sys
+
+if sys.version_info[:2] >= (3, 8):
+    import importlib.metadata as importlib_metadata
+else:
+    import importlib_metadata
+
+
+__version__ = importlib_metadata.version("cookiecutter-data-science")

docs/docs/all-options.md

@@ -3,4 +3,12 @@
 CCDS provides a number of choices that you can use to customize your project. The defaults work well for many projects, but lots of tooling choices are supported. Here are the options for tools that you can use:
 
 
-<!-- configuration-table.py output -->
+<!-- configuration-table.py output -->
+
+## Checking out other branches / using unreleased changes to the template
+
+By default, `ccds` will download the most recently _released_ version of the template. If there are any _unreleased_ changes to the template (or changes in a separate branch) that you want to incorporate, you can do so by checking out whatever branch you'd like to use (checkout `master` for the latest changes):
+
+```bash
+ccds -c master
+```

hooks/pre_prompt.py

@@ -0,0 +1,13 @@
+import warnings
+
+from ccds.version import __version__ as version
+
+if __name__ == "__main__":
+    if version < "2.0.1":
+        warnings.warn(
+            "You're currently using a CCDS version that always applies the "
+            "newest template. For more stable behavior, upgrade to "
+            "CCDS version 2.0.1 or later with your package manager. "
+            "For example, with pip, run: pip install -U cookiecutter-data-science.",
+            DeprecationWarning,
+        )