Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an installation script #29

Closed
wants to merge 47 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
813165b
Make a deb
velovix May 4, 2021
c260e54
Add build_deb to now workflow
velovix May 4, 2021
8cee561
Install poetry, but for real
velovix May 4, 2021
7d2465c
Fix a dash
velovix May 4, 2021
f0b57b3
Do things
velovix May 4, 2021
5a5dddf
More thangs
velovix May 4, 2021
b3e28c4
Oh yeah we playin it
velovix May 4, 2021
0dc6577
Add missing sudo
velovix May 4, 2021
9ba1e14
I really bit the big one this time
velovix May 4, 2021
6f2d732
Use single quotes
velovix May 4, 2021
f003d91
Try to store the deb as a build artifact
velovix May 4, 2021
4db2ea7
Fix naming
velovix May 4, 2021
c2c8f55
Rename to fix artifacts
velovix May 4, 2021
c43cdc3
Fix paths
velovix May 4, 2021
a484204
Bake system package-friendly defaults in
velovix May 4, 2021
48181be
Don't use CircleCI's Python image for deb builds
velovix May 4, 2021
a76163b
Load defaults from a file
velovix May 4, 2021
2c5ccdd
Fix dumb mistakes
velovix May 4, 2021
d2b098b
Convert default
velovix May 4, 2021
03e98c3
Add distutils, which Docker Compose seems to need
velovix May 4, 2021
afe9fab
Use distutils.util.strtobool
velovix May 4, 2021
72ee69a
Use cimg/python's included Poetry version
velovix May 4, 2021
5d41542
Add type annotations
velovix May 4, 2021
2114786
Formatting
velovix May 4, 2021
ded8237
Convince MyPy regarding nullability of a var
velovix May 4, 2021
980b878
Handle situation where config provides a bool
velovix May 4, 2021
0ff8b4c
Restore ability to auto-install curl and rsync
velovix May 5, 2021
1bd04ed
Add deb test to CI
velovix May 7, 2021
bc3e763
Add requirement
velovix May 7, 2021
3a77d4b
Move jobs around
velovix May 7, 2021
8c4bcae
Add requires
velovix May 7, 2021
26cb89a
Try building with 20.04
velovix May 7, 2021
f5dc014
Make deb building a matrix
velovix May 9, 2021
09bbdae
Use executors
velovix May 9, 2021
59f8520
Move up executors
velovix May 9, 2021
ef8c5a7
Fix matrix syntax
velovix May 9, 2021
b90ce3b
Fix deb naming
velovix May 9, 2021
8b700f5
Try with env vars
velovix May 9, 2021
65df3b3
Try out some of that clever stuff
velovix May 9, 2021
b362da8
Fix naming mistake
velovix May 9, 2021
9039d5d
Include defaults.yaml in Pip version
velovix May 9, 2021
666b8b2
Use the right name when setting data path env var
velovix May 9, 2021
97b29c8
Formatting
velovix May 9, 2021
80354c6
Make the structure of the install script
velovix May 10, 2021
b641d16
Add cool exclamation marks
velovix May 10, 2021
383b1ca
Minor comment adjustment
velovix May 10, 2021
a186cf2
Rely on the system's Compose for unsupported oses
velovix May 22, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 91 additions & 18 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,36 @@
version: 2.1

parameters:
poetry_version:
poetry-version:
type: string
default: "1.1.4"


executors:
focal:
docker:
- image: cimg/base:stable-20.04
bionic:
docker:
- image: cimg/base:stable-18.04


workflows:
lint:
jobs:
- lint
deb:
jobs:
- build-deb:
matrix:
parameters:
os: [bionic, focal]
- test-installation-deb:
requires:
- build-deb
deploy:
jobs:
- upload_to_pypi:
- upload-to-pypi:
filters:
# Ignore any commit on any branch by default
branches:
Expand All @@ -21,60 +40,114 @@ workflows:
only: /^v.+\..+\..+/
tests:
jobs:
- test_installation
- test-installation-source


jobs:
lint:
docker:
- image: circleci/python:3.6
- image: cimg/python:3.6
steps:
- checkout
- run: "curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
| python3 - --version << pipeline.parameters.poetry_version >>"
- run: poetry install --no-root
- run: poetry run isort --check .
- run: poetry run black --check .
- run: poetry run mypy -p "brainframe.cli"
upload_to_pypi:

build-deb:
parameters:
os:
type: executor
executor: << parameters.os >>
steps:
- run: sudo apt-get update
- run: sudo apt-get install -y software-properties-common dpkg-dev devscripts equivs
- run: sudo add-apt-repository ppa:jyrki-pulliainen/dh-virtualenv
- run: sudo apt-get install -y dh-virtualenv
- install-poetry
- checkout
- run: cp -r package/debian .
- run: cp package/setup.py .
- run: cp package/system_package_defaults.yaml brainframe/cli/defaults.yaml
- run: sudo mk-build-deps --install debian/control
- run: poetry export --output requirements.txt
- run: dpkg-buildpackage --unsigned-source --unsigned-changes --build=binary
- run: mkdir dist
- run: |
export CODENAME=$(lsb_release --codename --short)
mv ../brainframe-cli*.deb dist/brainframe-cli-${CODENAME}.deb
- store_artifacts:
path: dist
- persist_to_workspace:
root: .
paths:
- dist/*

test-installation-deb:
machine:
image: ubuntu-2004:202010-01
steps:
- run: sudo apt-get update
- attach_workspace:
at: /tmp/workspace
- run: sudo apt-get install /tmp/workspace/dist/brainframe-cli-focal.deb
- run: sudo brainframe install --noninteractive
- run: sudo brainframe compose up -d
- assert-core-container-running
- assert-core-responds-to-http
- run: sudo brainframe compose down

upload-to-pypi:
docker:
- image: circleci/python:3.6
- image: cimg/python:3.6
environment:
POETRY_HTTP_BASIC_PYPI_USERNAME: __token__

steps:
- checkout
- run: "curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
| python3 - --version << pipeline.parameters.poetry_version >>"
- run: poetry build
- run: POETRY_HTTP_BASIC_PYPI_PASSWORD=${PYPI_PASSWORD} poetry publish
test_installation:

test-installation-source:
machine:
image: ubuntu-2004:202010-01
steps:
- run: sudo apt-get update
- run: sudo apt-get install python3 python3-dev curl git -y
- checkout
- run: "curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
| python3 - --version << pipeline.parameters.poetry_version >>"
- run: echo "export PATH=$PATH:$HOME/.poetry/bin" >> $BASH_ENV
- install-poetry
- run: sudo $(which poetry) install
- run: sudo $(which poetry) run brainframe install --noninteractive
- run: sudo $(which poetry) run brainframe compose up -d
- assert-core-container-running
- assert-core-responds-to-http
- run: sudo $(which poetry) run brainframe compose down

commands:
install-poetry:
steps:
- run: sudo apt-get update && sudo apt-get install -y python3-pip
- run: >
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
| python3 - --version << pipeline.parameters.poetry-version >>
# Add to the PATH
- run: echo 'export PATH=$HOME/.poetry/bin:$PATH' >> $BASH_ENV
assert-core-container-running:
steps:
- run:
name: Check BrainFrame containers are running
name: Check BrainFrame core container is running
command: |
until docker container inspect -f {{.State.Running}} $(docker ps -q -f name="brainframe_core") >/dev/null 2>/dev/null; do
sleep 0.1;
done;
echo "BrainFrame core container is running."
no_output_timeout: 1m
assert-core-responds-to-http:
steps:
- run:
name: Check BrainFrame core service is up
name: Check BrainFrame core responds to HTTP requests
command: |
until curl -f http://localhost/api/version >/dev/null 2>/dev/null; do
sleep 0.1;
done;
echo "BrainFrame core service is up."
no_output_timeout: 1m
- run: sudo $(which poetry) run brainframe compose down
6 changes: 3 additions & 3 deletions brainframe/cli/commands/backup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@

import i18n
from brainframe.cli import (
config,
dependencies,
docker_compose,
env_vars,
os_utils,
print_utils,
)
Expand All @@ -19,8 +19,8 @@

@command("backup")
def backup():
install_path = env_vars.install_path.get()
data_path = env_vars.data_path.get()
install_path = config.install_path.value
data_path = config.data_path.value

args = _parse_args(data_path)

Expand Down
5 changes: 2 additions & 3 deletions brainframe/cli/commands/compose.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
import sys

import i18n
from brainframe.cli import docker_compose, env_vars, os_utils, print_utils
from brainframe.cli import config, docker_compose

from .utils import command


@command("compose")
def compose():
install_path = env_vars.install_path.get()
install_path = config.install_path.value
docker_compose.assert_installed(install_path)
docker_compose.run(install_path, sys.argv[2:])
10 changes: 5 additions & 5 deletions brainframe/cli/commands/info.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from argparse import ArgumentParser

import i18n
from brainframe.cli import docker_compose, env_vars, print_utils
from brainframe.cli import config, docker_compose, print_utils

from .utils import command, subcommand_parse_args

Expand All @@ -10,15 +10,15 @@
def info():
args = _parse_args()

docker_compose.assert_installed(env_vars.install_path.get())
docker_compose.assert_installed(config.install_path.value)

server_version = docker_compose.check_existing_version(
env_vars.install_path.get()
config.install_path.value
)

fields = {
"install_path": env_vars.install_path.get(),
"data_path": env_vars.data_path.get(),
"install_path": config.install_path.value,
"data_path": config.data_path.value,
"server_version": server_version,
}

Expand Down
35 changes: 13 additions & 22 deletions brainframe/cli/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

import i18n
from brainframe.cli import (
config,
dependencies,
docker_compose,
env_vars,
os_utils,
print_utils,
)
Expand All @@ -32,9 +32,6 @@ def install():
# Check all dependencies
dependencies.curl.ensure(args.noninteractive, args.install_curl)
dependencies.docker.ensure(args.noninteractive, args.install_docker)
dependencies.docker_compose.ensure(
args.noninteractive, args.install_docker_compose
)

_, _, download_version = docker_compose.check_download_version()
print_utils.translate("install.install-version", version=download_version)
Expand All @@ -55,8 +52,8 @@ def install():
# Ask the user if they want to specify special paths for installation
print_utils.translate(
"install.default-paths",
default_install_path=env_vars.install_path.default,
default_data_path=env_vars.data_path.default,
default_install_path=config.install_path.default,
default_data_path=config.data_path.default,
)
use_default_paths = print_utils.ask_yes_no(
"install.ask-use-default-paths",
Expand All @@ -66,22 +63,21 @@ def install():
if args.noninteractive:
install_path = args.install_path
elif use_default_paths:
install_path = env_vars.install_path.default
install_path = config.install_path.default
else:
install_path = print_utils.ask_path(
"install.ask-brainframe-install-path",
env_vars.install_path.default,
"install.ask-brainframe-install-path", config.install_path.default,
)
install_path.mkdir(exist_ok=True, parents=True)

# Set up the data path
if args.noninteractive:
data_path = args.data_path
elif use_default_paths:
data_path = env_vars.data_path.default
data_path = config.data_path.default
else:
data_path = print_utils.ask_path(
"install.ask-data-path", env_vars.data_path.default
"install.ask-data-path", config.data_path.default
)
data_path.mkdir(exist_ok=True, parents=True)

Expand Down Expand Up @@ -120,15 +116,15 @@ def install():
# Recommend to the user to add their custom paths to environment variables
# so that future invocations of the program will know where to look.
if (
install_path != env_vars.install_path.default
or data_path != env_vars.data_path.default
install_path != config.install_path.default
or data_path != config.data_path.default
):
print()
print_utils.translate("install.set-custom-directory-env-vars")
print(
f"\n"
f'export {env_vars.install_path.name}="{install_path}"\n'
f'export {env_vars.data_path.name}="{data_path}"\n'
f'export {config.install_path.name}="{install_path}"\n'
f'export {config.data_path.name}="{data_path}"\n'
)


Expand All @@ -146,25 +142,20 @@ def _parse_args():
parser.add_argument(
"--install-path",
type=Path,
default=env_vars.install_path.default,
default=config.install_path.default,
help=i18n.t("install.install-path-help"),
)
parser.add_argument(
"--data-path",
type=Path,
default=env_vars.data_path.default,
default=config.data_path.default,
help=i18n.t("install.data-path-help"),
)
parser.add_argument(
"--install-docker",
action="store_true",
help=i18n.t("install.install-docker-help"),
)
parser.add_argument(
"--install-docker-compose",
action="store_true",
help=i18n.t("install.install-docker-compose-help"),
)
parser.add_argument(
"--install-curl",
action="store_true",
Expand Down
4 changes: 2 additions & 2 deletions brainframe/cli/commands/update.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from argparse import ArgumentParser

import i18n
from brainframe.cli import docker_compose, env_vars, print_utils
from brainframe.cli import config, docker_compose, print_utils
from packaging import version

from .utils import command, subcommand_parse_args
Expand All @@ -11,7 +11,7 @@
def update():
args = _parse_args()

install_path = env_vars.install_path.get()
install_path = config.install_path.value

docker_compose.assert_installed(install_path)

Expand Down
Loading