Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/environment overrides #3

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Feat/environment overrides #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c5f8af4
Merge pull request #1 from originalsosa/feat/directory-structure
originalsosa Sep 7, 2024
a70f319
Environment Overrides and README updates
Sep 7, 2024
a21e5af
Updated name of environment file
Sep 7, 2024
434664d
Merge branch 'mhendrey:main' into feat/environment-overrides
originalsosa Sep 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,4 +164,4 @@ cython_debug/
*.tar.gz
.task
models
task.env
*.env
22 changes: 21 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Running tasks is orchestrated by using [Taskfile.dev](https://taskfile.dev/)

This document provides instructions on how to run tasks defined in the `Taskfile.yml`.

Create a task.env at the root of project to define enviroment overrides.
Create a `.env` at the root of project to define enviroment overrides.

## Tasks Overview

Expand Down Expand Up @@ -142,3 +142,23 @@ task triton-start
# Tail logs of running containr
docker logs -f $(docker ps -q --filter "name=triton-inference-server")
```

### `Overriding Defaults`

At Root of project create file **.env**

### Environment Variables for `.env`

The following docker compose and Task variables can be overridden in the `local.env` file:

| Variable Name | Description | Default Value |
|-------------------|------------------------------------------------------------------|---------------------------------|
| `TRITON_INFERENCE_SERVER_IMAGE` | TRITON BASE IMAGE | `nvcr.io/nvidia/tritonserver` |
| `TRITON_INFERENCE_SERVER_TAG` | TRITON BASE IMAGE VERSION | `24.07-py3` |
| `MODEL_REPOSITORY_SOURCE` | SOURCE MODEL REPO DIR | `./model-repository` |
| `MODEL_REPOSITORY_TARGET` | TARGET MODEL REPO DIR | `/model-repository` |
| `HF_HUB_CACHE_SOURCE` | SOURCE HUGGING FACE CACHE DIR | `./models` |
| `HF_HUB_CACHE_TARGET` | TARGET HUGGING FACE CACHE DIR | `/models` |
| `COMPOSE_CONDA_PACK_BASE_IMAGE` | COMPOSE CONDA PACK BASE IMAGE | `continuumio/miniconda3:latest` |

**Note:** Format in `.env` file is as following `VAR`=`VALUE`.
2 changes: 1 addition & 1 deletion Taskfile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
version: '3'

dontenv: ['task.env']
dontenv: ['.env']

vars:
COMPUTE_TYPE: '{{default "gpu" .COMPUTE_TYPE}}'
Expand Down