v0.4.2

Bump `web-rwkv` to v0.7.6 (#107)

v0.4.1

Add a `top_p` filter to mirostat sampler.

v0.4.0

What's Changed

• Port v0.7 by @cryscan in #100
• Update index.zip by @cuijinsen in #103
• Update index.zip by @cuijinsen in #104

Full Changelog: v0.3.25...v0.4.0

v0.3.25

What's Changed

• Bump web-rwkv to v0.6.36 by @josStorer in #95

• Fixes #93

Full Changelog: v0.3.24...v0.3.25

v0.3.24

What's Changed?

🎈1. Support Save & Load “Pre-quantized Model” with Extension .perfab #83

You can use /api/models/save (post model_path to this api after loading the model with quantization and LoRA merged to save the prefab), and you will get a smaller model file.
And then you can load this file using out favourite /api/models/load. Note that when loading a prefab, parameters like lora and quant are ignored.

👑2. Support BNF #92

Now you could specify bnf_schema in /api/oai/chat/completions.
This will force the model to generate contents that fit the schema (e.g., in JSON format).

🧨 Fixed Many Minor Issues

🥱🥱🥱

Changed list

• docs: add longzou as a contributor for code, and security by @allcontributors in #77
• Auth by @cryscan in #78
• All contributors/add cryscan by @cryscan in #79
• Small fix. by @cryscan in #80
• Bump web-rwkv to v0.6.33 by @cryscan in #81
• Remove axum api. by @cryscan in #82
• Allow loading prefab. by @cryscan in #83
• Do not check for existence when building path. by @cryscan in #85
• Fix typo in main.rs by @cryscan in #86
• Update index.zip by @cuijinsen in #87
• Update index.zip by @cuijinsen in #88
• Fix the null finish reason representation. by @cryscan in #91
• Bnf constraint by @cryscan in #92

New Contributors

• @cuijinsen made their first contribution in #87

Full Changelog: v0.3.23...v0.3.24

v0.3.23

What's Changed?

🎈1. Support TLS and http2/http3

You can enable TLS by turning tls = true in assets/configs/Config.toml.
Thanks to salvo!

👑2. Support openAPI and Swagger-UI

Now you can access https://127.0.0.1:65530/swagger-ui/ to use Swagger UI
Thanks to salvo again!

🔔3. Fix something for Loading Lora

Hmm...

ps. 📢📢📢 Something BIG on the way! (⓿_⓿)

v0.3.22

Bump `web-rwkv` to v0.6.25

v0.3.21

Default `penalty_decay` to 0.996.

v0.3.20

Fix token sampling parallelism.

v0.3.19

Update `web-rwkv` to v0.6.21