dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 18:45:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

Bump vLLM version + more options when loading models in vLLM (#1782)

Browse source 
* Bump vLLM version to 0.3.2

* Add vLLM model loading options

* Remove transformers-exllama

* Fix install exllama
This commit is contained in:
Ludovic Leroux 2024-03-01 16:48:53 -05:00 • committed by GitHub
parent 1c312685aa
commit 939411300a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
28 changed files with 736 additions and 641 deletions
Download patch file Download diff file Expand all files Collapse all files

18
backend/python/exllama/install.sh
View file

@ -1,14 +1,22 @@
#!/bin/bash
set -ex
##
## A bash script installs the required dependencies of VALL-E-X and prepares the environment
export PATH=$PATH:/opt/conda/bin
# Activate conda environment
source activate transformers
# Check if environment exist
conda_env_exists(){
! conda list --name "${@}" >/dev/null 2>/dev/null
}
echo $CONDA_PREFIX
if conda_env_exists "exllama" ; then
echo "Creating virtual environment..."
conda env create --name exllama --file $1
echo "Virtual environment created."
else
echo "Virtual environment already exists."
fi
source activate exllama
git clone https://github.com/turboderp/exllama $CONDA_PREFIX/exllama && pushd $CONDA_PREFIX/exllama && pip install -r requirements.txt && popd