dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 18:45:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat: migrate python backends from conda to uv (#2215)

Browse source 
* feat: migrate diffusers backend from conda to uv

  - replace conda with UV for diffusers install (prototype for all
    extras backends)
  - add ability to build docker with one/some/all extras backends
    instead of all or nothing

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate autogtpq bark coqui from conda to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: convert exllama over to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate exllama2 to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate mamba to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate parler to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate petals to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: fix tests

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate rerankers to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate sentencetransformers to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: install uv for tests-linux

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: make sure file exists before installing on intel images

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate transformers backend to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate transformers-musicgen to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate vall-e-x to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: migrate vllm to uv

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: add uv install to the rest of test-extra.yml

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: adjust file perms on all install/run/test scripts

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: add missing acclerate dependencies

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: add some more missing dependencies to python backends

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: parler tests venv py dir fix

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: correct filename for transformers-musicgen tests

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: adjust the pwd for valle tests

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: cleanup and optimization work for uv migration

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: add setuptools to requirements-install for mamba

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: more size optimization work

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: make installs and tests more consistent, cleanup some deps

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: cleanup

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: mamba backend is cublas only

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: uncomment lines in makefile

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
This commit is contained in:
cryptk 2024-05-10 08:08:08 -05:00 • committed by GitHub
parent e6768097f4
commit 28a421cb1d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
101 changed files with 988 additions and 740 deletions
Download patch file Download diff file Expand all files Collapse all files

9
backend/python/mamba/Makefile
View file

@ -1,7 +1,6 @@
.PHONY: mamba
mamba: protogen
$(MAKE) -C ../common-env/transformers
bash install.sh
bash install.sh
.PHONY: run
run: protogen
@ -23,4 +22,8 @@ protogen-clean:
$(RM) backend_pb2_grpc.py backend_pb2.py
backend_pb2_grpc.py backend_pb2.py:
python3 -m grpc_tools.protoc -I../.. --python_out=. --grpc_python_out=. backend.proto
python3 -m grpc_tools.protoc -I../.. --python_out=. --grpc_python_out=. backend.proto
.PHONY: clean
clean: protogen-clean
$(RM) -r venv

33
backend/python/mamba/install.sh
View file

@ -1,21 +1,38 @@
#!/bin/bash
set -e
##
## A bash script installs the required dependencies of VALL-E-X and prepares the environment
set -ex
if [ "$BUILD_TYPE" != "cublas" ]; then
echo "[mamba] Attention!!! nvcc is required - skipping installation"
exit 0
fi
export PATH=$PATH:/opt/conda/bin
BUILD_ISOLATION_FLAG=""
# Activate conda environment
source activate transformers
MY_DIR="$(dirname -- "${BASH_SOURCE[0]}")"
echo $CONDA_PREFIX
uv venv ${MY_DIR}/venv
source ${MY_DIR}/venv/bin/activate
pip install causal-conv1d==1.0.0 mamba-ssm==1.0.1
if [ -f "requirements-install.txt" ]; then
# If we have a requirements-install.txt, it means that a package does not properly declare it's build time
# dependencies per PEP-517, so we have to set up the proper build environment ourselves, and then install
# the package without build isolation
BUILD_ISOLATION_FLAG="--no-build-isolation"
uv pip install --requirement ${MY_DIR}/requirements-install.txt
fi
uv pip install ${BUILD_ISOLATION_FLAG} --requirement ${MY_DIR}/requirements.txt
if [ -f "requirements-${BUILD_TYPE}.txt" ]; then
uv pip install ${BUILD_ISOLATION_FLAG} --requirement ${MY_DIR}/requirements-${BUILD_TYPE}.txt
fi
if [ -d "/opt/intel" ]; then
# Intel GPU: If the directory exists, we assume we are using the Intel image
# https://github.com/intel/intel-extension-for-pytorch/issues/538
if [ -f "requirements-intel.txt" ]; then
uv pip install ${BUILD_ISOLATION_FLAG} --index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ --requirement ${MY_DIR}/requirements-intel.txt
fi
fi
if [ "$PIP_CACHE_PURGE" = true ] ; then
pip cache purge

7
backend/python/mamba/requirements-install.txt Normal file
View file

@ -0,0 +1,7 @@
# mabma does not specify it's build dependencies per PEP517, so we need to disable build isolation
# this also means that we need to install the basic build dependencies into the venv ourselves
# https://github.com/Dao-AILab/causal-conv1d/issues/24
packaging
setuptools
wheel
torch==2.2.0

6
backend/python/mamba/requirements.txt Normal file
View file

@ -0,0 +1,6 @@
causal-conv1d==1.2.0.post2
mamba-ssm==1.2.0.post1
grpcio==1.63.0
protobuf
certifi
transformers

12
backend/python/mamba/run.sh
View file

@ -1,14 +1,10 @@
#!/bin/bash
##
## A bash script wrapper that runs the diffusers server with conda
## A bash script wrapper that runs the GRPC server
export PATH=$PATH:/opt/conda/bin
MY_DIR="$(dirname -- "${BASH_SOURCE[0]}")"
# Activate conda environment
source activate transformers
source $MY_DIR/venv/bin/activate
# get the directory where the bash script is located
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
python $DIR/backend_mamba.py $@
python $MY_DIR/backend_mamba.py $@

2
backend/python/mamba/test_backend_mamba.py → backend/python/mamba/test.py
View file

@ -20,7 +20,7 @@ class TestBackendServicer(unittest.TestCase):
This class contains methods to test the startup and shutdown of the gRPC service.
"""
def setUp(self):
self.service = subprocess.Popen(["python", "backend_vllm.py", "--addr", "localhost:50051"])
self.service = subprocess.Popen(["python", "backend_mamba.py", "--addr", "localhost:50051"])
time.sleep(10)
def tearDown(self) -> None:

17
backend/python/mamba/test.sh Normal file → Executable file
View file

@ -1,11 +1,16 @@
#!/bin/bash
##
## A bash script wrapper that runs the transformers server with conda
## A bash script wrapper that runs python unittests
# Activate conda environment
source activate transformers
MY_DIR="$(dirname -- "${BASH_SOURCE[0]}")"
# get the directory where the bash script is located
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source $MY_DIR/venv/bin/activate
python -m unittest $DIR/test_backend_mamba.py
if [ -f "${MY_DIR}/test.py" ]; then
pushd ${MY_DIR}
python -m unittest test.py
popd
else
echo "ERROR: No tests defined for backend!"
exit 1
fi