Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Moving environment variables of docker-compose.yml to .dockercompose_env file #1055

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .dockercompose_env
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this could improve the self-hosting experience by consolidating many of the user-configuration options into one file!

One potential issue I see is that it increases complexity of setup, because it requires a two-file install vs. one file. I would also rename the file to something like .khoj_docker_compose_env for specificity.

Another issue is that some environment variable names would be repurposed between containers implicitly, and it may not immediately be clear how. For example, the POSTGRES_* related variables.

This could be mitigated if we wrap the overall setup process within a shell script that can guide the user through setup via CLI, so they don't have to worry about the internals. A prerequisite to setup could be that Docker Desktop be installed.

Open to thoughts. Thanks for creating the PR. If we do accept this change, the Docker setup instructions would need to be updated.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A shell script isn’t necessary since the setup is straightforward with Docker Compose. Renaming the file to .khoj_docker_compose_env is a great idea as it makes the configuration clear for new users.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another issue is that some environment variable names would be repurposed between containers implicitly, and it may not immediately be clear how. For example, the POSTGRES_* related variables.

Since there’s no scaling involved, repurposing environment variables seems over-engineered in my opinion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment variables are already being repurposed, but the single compose file makes it opaque which containers are using which variables.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you have plans on revamping this do let me know , happy to support khoj .

Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
## DATABASE

POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres

## SEARCH

SEARXNG_BASE_URL=http://localhost:8080/

## SERVER

POSTGRES_DB=postgres
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_HOST=database
POSTGRES_PORT=5432
KHOJ_DJANGO_SECRET_KEY=secret
KHOJ_DEBUG=False
[email protected]
KHOJ_ADMIN_PASSWORD=password
# Default URL of Terrarium, the Python sandbox used by Khoj to run code. Its container is specified above
KHOJ_TERRARIUM_URL=http://sandbox:8080
# Default URL of SearxNG, the default web search engine used by Khoj. Its container is specified above
KHOJ_SEARXNG_URL=http://search:8080
# Uncomment line below to use with Ollama running on your local machine at localhost:11434.
# Change URL to use with other OpenAI API compatible providers like VLLM, LMStudio etc.
# OPENAI_API_BASE=http://host.docker.internal:11434/v1/
#
# Uncomment appropriate lines below to use chat models by OpenAI, Anthropic, Google.
# Ensure you set your provider specific API keys.
# ---
# OPENAI_API_KEY=your_openai_api_key
# GEMINI_API_KEY=your_gemini_api_key
# ANTHROPIC_API_KEY=your_anthropic_api_key
#
# Uncomment appropriate lines below to enable web results with Khoj
# Ensure you set your provider specific API keys.
# ---
# Free, Slower API. Does both web search and webpage read. Get API key from https://jina.ai/
# JINA_API_KEY=your_jina_api_key
# Paid, Fast API. Only does web search. Get API key from https://serper.dev/
# SERPER_DEV_API_KEY=your_serper_dev_api_key
# Paid, Fast, Open API. Only does webpage read. Get API key from https://firecrawl.dev/
# FIRECRAWL_API_KEY=your_firecrawl_api_key
# Paid, Fast, Higher Read Success API. Only does webpage read. Get API key from https://olostep.com/
# OLOSTEP_API_KEY=your_olostep_api_key
#
# Uncomment the necessary lines below to make your instance publicly accessible.
# Replace the KHOJ_DOMAIN with either your domain or IP address (no http/https prefix).
# Proceed with caution, especially if you are using anonymous mode.
# ---
# KHOJ_NO_HTTPS=True
# KHOJ_DOMAIN=192.168.0.104
# KHOJ_DOMAIN=khoj.example.com
# Uncomment the line below to disable telemetry.
# Telemetry helps us prioritize feature development and understand how people are using Khoj
# Read more at https://docs.khoj.dev/miscellaneous/telemetry
# KHOJ_TELEMETRY_DISABLE=True
# Comment out this line when you're using the official ghcr.io/khoj-ai/khoj-cloud:latest prod image.
61 changes: 6 additions & 55 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@ services:
database:
image: ankane/pgvector
restart: always
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
env_file:
- .dockercompose_env
networks:
- default
volumes:
Expand All @@ -27,8 +25,8 @@ services:
- default
volumes:
- khoj_search:/etc/searxng
environment:
- SEARXNG_BASE_URL=http://localhost:8080/
env_file:
- .dockercompose_env
server:
depends_on:
database:
Expand All @@ -55,55 +53,8 @@ services:
- khoj_models:/root/.cache/torch/sentence_transformers
- khoj_models:/root/.cache/huggingface
# Use 0.0.0.0 to explicitly set the host ip for the service on the container. https://pythonspeed.com/articles/docker-connection-refused/
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST=database
- POSTGRES_PORT=5432
- KHOJ_DJANGO_SECRET_KEY=secret
- KHOJ_DEBUG=False
- [email protected]
- KHOJ_ADMIN_PASSWORD=password
# Default URL of Terrarium, the Python sandbox used by Khoj to run code. Its container is specified above
- KHOJ_TERRARIUM_URL=http://sandbox:8080
# Default URL of SearxNG, the default web search engine used by Khoj. Its container is specified above
- KHOJ_SEARXNG_URL=http://search:8080
# Uncomment line below to use with Ollama running on your local machine at localhost:11434.
# Change URL to use with other OpenAI API compatible providers like VLLM, LMStudio etc.
# - OPENAI_API_BASE=http://host.docker.internal:11434/v1/
#
# Uncomment appropriate lines below to use chat models by OpenAI, Anthropic, Google.
# Ensure you set your provider specific API keys.
# ---
# - OPENAI_API_KEY=your_openai_api_key
# - GEMINI_API_KEY=your_gemini_api_key
# - ANTHROPIC_API_KEY=your_anthropic_api_key
#
# Uncomment appropriate lines below to enable web results with Khoj
# Ensure you set your provider specific API keys.
# ---
# Free, Slower API. Does both web search and webpage read. Get API key from https://jina.ai/
# - JINA_API_KEY=your_jina_api_key
# Paid, Fast API. Only does web search. Get API key from https://serper.dev/
# - SERPER_DEV_API_KEY=your_serper_dev_api_key
# Paid, Fast, Open API. Only does webpage read. Get API key from https://firecrawl.dev/
# - FIRECRAWL_API_KEY=your_firecrawl_api_key
# Paid, Fast, Higher Read Success API. Only does webpage read. Get API key from https://olostep.com/
# - OLOSTEP_API_KEY=your_olostep_api_key
#
# Uncomment the necessary lines below to make your instance publicly accessible.
# Replace the KHOJ_DOMAIN with either your domain or IP address (no http/https prefix).
# Proceed with caution, especially if you are using anonymous mode.
# ---
# - KHOJ_NO_HTTPS=True
# - KHOJ_DOMAIN=192.168.0.104
# - KHOJ_DOMAIN=khoj.example.com
# Uncomment the line below to disable telemetry.
# Telemetry helps us prioritize feature development and understand how people are using Khoj
# Read more at https://docs.khoj.dev/miscellaneous/telemetry
# - KHOJ_TELEMETRY_DISABLE=True
# Comment out this line when you're using the official ghcr.io/khoj-ai/khoj-cloud:latest prod image.
env_file:
- .dockercompose_env
command: --host="0.0.0.0" --port=42110 -vv --anonymous-mode --non-interactive

volumes:
Expand Down