Skip to content

Tags: nubison/nubison-model

Tags

v0.0.6

Verified

This commit was signed with the committer’s verified signature.
kyuwoo-choi KyuWoo Choi
chore: remove useless file link created by test

v0.0.5

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat: add context support for parallel inference (#12)

* feat: enhance model loading with context support

- Updated `load_model` method in `UserModel` and `NubisonModel` to accept a `ModelContext` argument, allowing for better handling of worker-specific information during model loading.
- Introduced `ModelContext` type definition to encapsulate worker index and total number of workers for GPU initialization in parallel setups.
- Adjusted related code in service and tests to accommodate the new context parameter.
- Updated documentation in `README.md` to reflect changes in the `load_model` method and its parameters.

* feat: update Dockerfile to support parallel inference

- Added Open Container Initiative (OCI) labels for better image description and source tracking.
- Introduced a new environment variable `NUM_WORKERS` with a default value of 4 to configure the number of workers for the application.

* refactor: reduce default worker count for improved resource management

- Updated the Dockerfile to change the `NUM_WORKERS` environment variable from 4 to 2, optimizing resource allocation.
- Adjusted the default number of workers in `Service.py` from 4 to 1 to align with the new Docker configuration, enhancing performance and efficiency during model loading.

v0.0.4

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat: uv instead of conda in inference container (#11)

v0.0.3

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fix: test_client fails due to lmodel loading wrong way (#10)

v0.0.2

Verified

This commit was signed with the committer’s verified signature.
kyuwoo-choi KyuWoo Choi
doc: update example for better nubison integration

v0.0.1

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat: add test client for easy test against server (#4)