dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-25 04:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

docs: add distributed inferencing docs

Browse source 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2024-05-25 16:11:59 +02:00
parent 785c54e7b0
commit bb3ec56de3
5 changed files with 109 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/content/docs/advanced/advanced-usage.md
View file

@ -370,6 +370,8 @@ there are additional environment variables available that modify the behavior of
| `GO_TAGS` | | Go tags. Available: `stablediffusion` |
| `HUGGINGFACEHUB_API_TOKEN` | | Special token for interacting with HuggingFace Inference API, required only when using the `langchain-huggingface` backend |
| `EXTRA_BACKENDS` | | A space separated list of backends to prepare. For example `EXTRA_BACKENDS="backend/python/diffusers backend/python/transformers"` prepares the conda environment on start |
| `DISABLE_AUTODETECT` | `false` | Disable autodetect of CPU flagset on start |
| `LLAMACPP_GRPC_SERVERS` | | A list of llama.cpp workers to distribute the workload. For example `LLAMACPP_GRPC_SERVERS="address1:port,address2:port"` |
Here is how to configure these variables: