Skip to content

Tags: scaleapi/llm-engine

Tags

v0.0.0beta34

Toggle v0.0.0beta34's commit message
[Client] Add guided_grammar and other missing fields (#532)

Add guided_grammar to the client, + add some missing fields to some codepaths

v0.0.0beta33

Toggle v0.0.0beta33's commit message
Image cache and balloon on H100s, also temporarily stop people from u…

…sing A100 (#523)

* Cache H100

* Stop people from using A100

* no cover

* no cover

* update client version

v0.0.0beta32

Toggle v0.0.0beta32's commit message
Fix package name mapping (#514)

v0.0.0beta28

Toggle v0.0.0beta28's commit message
Guided decoding (#476)

* Guided decoding

* endpoints

* fix

* update client

* unit tests

* fix test

* coverage

* coverage

* fix

* try to bump coverage

* more tests!

* lint

v0.0.0beta27

Toggle v0.0.0beta27's commit message
[Client] Add num_prompt_tokens to the client's CompletionOutputs (#467)

* add prompt token, untested

* comment

* remove stop_str stuff, it doesn't do anything with the public api, and it breaks on certain frameworks when hosted locally

v0.0.0beta26

Toggle v0.0.0beta26's commit message
Change back batch infer GPU util and add tool completion client chang…

…es (#465)

* Change back batch infer gpu util

* Add client changes

* fixes

* bump

v0.0.0beta25

Toggle v0.0.0beta25's commit message
[Client] Add Auth headers to the python async routes (#434)

* add Auth headers to the async routes

* bump version

v0.0.0beta22

Toggle v0.0.0beta22's commit message
Minor fixes for batch inference (#426)

* Fix file not found

* progress fix

* add tests

* bump

* typing

v0.0.0beta20

Toggle v0.0.0beta20's commit message
pypi version bump (#391)

v0.0.0beta19

Toggle v0.0.0beta19's commit message
publish 0.0.0b19 (#322)