dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat(alias): alias llama to llama-cpp, update docs (#1448)

Browse source 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto 2023-12-16 12:22:45 -05:00 • committed by GitHub
parent 1c286c3c2f
commit 3d83128f16
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 15 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/content/model-compatibility/_index.en.md
View file

@ -50,6 +50,8 @@ Besides llama based models, LocalAI is compatible also with other architectures.
| `diffusers` | SD,... | no | Image generation | no | no | N/A |
| `vall-e-x` | Vall-E | no | Audio generation and Voice cloning | no | no | CPU/CUDA |
| `vllm` | Various GPTs and quantization formats | yes | GPT | no | no | CPU/CUDA |
| `exllama2` | GPTQ | yes | GPT only | no | no | N/A |
| `transformers-musicgen` | | no | Audio generation | no | no | N/A |
Note: any backend name listed above can be used in the `backend` field of the model configuration file (See [the advanced section]({{%relref "advanced" %}})).

6
docs/content/model-compatibility/llama-cpp.md
View file

@ -9,7 +9,7 @@ weight = 1
{{% notice note %}}
The `ggml` file format has been deprecated. If you are using `ggml` models and you are configuring your model with a YAML file, specify, use the `llama-stable` backend instead. If you are relying in automatic detection of the model, you should be fine. For `gguf` models, use the `llama` backend.
The `ggml` file format has been deprecated. If you are using `ggml` models and you are configuring your model with a YAML file, specify, use the `llama-ggml` backend instead. If you are relying in automatic detection of the model, you should be fine. For `gguf` models, use the `llama` backend. The go backend is deprecated as well but still available as `go-llama`. The go backend supports still features not available in the mainline: speculative sampling and embeddings.
{{% /notice %}}
@ -65,11 +65,11 @@ parameters:
In the example above we specify `llama` as the backend to restrict loading `gguf` models only.
For instance, to use the `llama-stable` backend for `ggml` models:
For instance, to use the `llama-ggml` backend for `ggml` models:
```yaml
name: llama
backend: llama-stable
backend: llama-ggml
parameters:
# Relative to the models path
model: file.ggml.bin