Skip to content

0.0.3

Compare
Choose a tag to compare
@michaelfeil michaelfeil released this 30 Oct 13:58
· 877 commits to main since this release
8116680

What's Changed

  • add Flash-Attention+ optimum-BetterTransformers by @michaelfeil in #20
  • Improve real-time / sleep strategy, async await for queues and result futures - reducing latency a bit by @michaelfeil in #12
  • add better FIFO queueing strategy - your requests now have a upper bound how long they queue by @michaelfeil in #19

Docs:

Full Changelog: 0.0.2rc0...0.0.3