fix/docs: Python backend dependencies (#1360)

* Update docs for new requirements.txt path

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>

* Fix typo (.PONY -> .PHONY) in python backend makefiles

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>

---------

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>
This commit is contained in:
B4ckslash 2023-11-30 17:46:55 +01:00 committed by GitHub
parent 9b98be160a
commit 2d64d8b444
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 30 additions and 27 deletions

View file

@ -1,4 +1,4 @@
.PONY: autogptq .PHONY: autogptq
autogptq: autogptq:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name autogptq --file autogptq.yml @conda env create --name autogptq --file autogptq.yml

View file

@ -1,4 +1,4 @@
.PONY: ttsbark .PHONY: ttsbark
ttsbark: ttsbark:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name ttsbark --file ttsbark.yml @conda env create --name ttsbark --file ttsbark.yml

View file

@ -1,10 +1,10 @@
.PONY: diffusers .PHONY: diffusers
diffusers: diffusers:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name diffusers --file diffusers.yml @conda env create --name diffusers --file diffusers.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running diffusers..." @echo "Running diffusers..."
bash run.sh bash run.sh

View file

@ -1,10 +1,10 @@
.PONY: exllama .PHONY: exllama
exllama: exllama:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name exllama --file exllama.yml @conda env create --name exllama --file exllama.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running exllama..." @echo "Running exllama..."
bash run.sh bash run.sh

View file

@ -1,10 +1,10 @@
.PONY: petals .PHONY: petals
petals: petals:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name petals --file petals.yml @conda env create --name petals --file petals.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running petals..." @echo "Running petals..."
bash run.sh bash run.sh

View file

@ -1,17 +1,17 @@
.PONY: sentencetransformers .PHONY: sentencetransformers
sentencetransformers: sentencetransformers:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name sentencetransformers --file sentencetransformers.yml @conda env create --name sentencetransformers --file sentencetransformers.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running sentencetransformers..." @echo "Running sentencetransformers..."
bash run.sh bash run.sh
@echo "sentencetransformers run." @echo "sentencetransformers run."
# It is not working well by using command line. It only6 works with IDE like VSCode. # It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test .PHONY: test
test: test:
@echo "Testing sentencetransformers..." @echo "Testing sentencetransformers..."
bash test.sh bash test.sh

View file

@ -1,17 +1,17 @@
.PONY: transformers .PHONY: transformers
transformers: transformers:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name transformers --file transformers.yml @conda env create --name transformers --file transformers.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running transformers..." @echo "Running transformers..."
bash run.sh bash run.sh
@echo "transformers run." @echo "transformers run."
# It is not working well by using command line. It only6 works with IDE like VSCode. # It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test .PHONY: test
test: test:
@echo "Testing transformers..." @echo "Testing transformers..."
bash test.sh bash test.sh

View file

@ -1,10 +1,10 @@
.PONY: ttsvalle .PHONY: ttsvalle
ttsvalle: ttsvalle:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name ttsvalle --file ttsvalle.yml @conda env create --name ttsvalle --file ttsvalle.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running ttsvalle..." @echo "Running ttsvalle..."
bash run.sh bash run.sh

View file

@ -1,10 +1,10 @@
.PONY: vllm .PHONY: vllm
vllm: vllm:
@echo "Creating virtual environment..." @echo "Creating virtual environment..."
@conda env create --name vllm --file vllm.yml @conda env create --name vllm --file vllm.yml
@echo "Virtual environment created." @echo "Virtual environment created."
.PONY: run .PHONY: run
run: run:
@echo "Running vllm..." @echo "Running vllm..."
bash run.sh bash run.sh

View file

@ -75,7 +75,10 @@ The `sentencetransformers` backend uses Python [sentence-transformers](https://g
{{% notice note %}} {{% notice note %}}
- The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use. If you are running `LocalAI` manually you must install the python dependencies (`pip install -r /path/to/LocalAI/extra/requirements`) and specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable ( `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"` ) . - The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use.
- If you are running `LocalAI` manually you must install the python dependencies (`make prepare-extra-conda-environments`). This requires `conda` to be installed.
- For local execution, you also have to specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable.
- Example: `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"`
- The `sentencetransformers` backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the `bert` backend or `llama.cpp`. - The `sentencetransformers` backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the `bert` backend or `llama.cpp`.
- No models are required to be downloaded before using the `sentencetransformers` backend. The models will be downloaded automatically the first time the API is used. - No models are required to be downloaded before using the `sentencetransformers` backend. The models will be downloaded automatically the first time the API is used.