dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-21 11:04:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

Bump vLLM version + more options when loading models in vLLM (#1782)

Browse source 
* Bump vLLM version to 0.3.2

* Add vLLM model loading options

* Remove transformers-exllama

* Fix install exllama
This commit is contained in:
Ludovic Leroux 2024-03-01 16:48:53 -05:00 • committed by GitHub
parent 1c312685aa
commit 939411300a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
28 changed files with 736 additions and 641 deletions
Download patch file Download diff file Expand all files Collapse all files

6
backend/python/common-env/transformers/transformers.yml
View file

@ -69,7 +69,7 @@ dependencies:
- requests==2.31.0
- rouge==1.0.1
- s3transfer==0.7.0
- safetensors==0.3.3
- safetensors>=0.4.1
- scipy==1.11.3
- six==1.16.0
- sympy==1.12
@ -101,7 +101,7 @@ dependencies:
- sudachipy
- sudachidict_core
- vocos
- vllm==0.2.7
- transformers>=4.36.0 # Required for Mixtral.
- vllm==0.3.2
- transformers>=4.38.0 # Required for Gemma.
- xformers==0.0.23.post1
prefix: /opt/conda/envs/transformers