dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat(tts): respect YAMLs config file, add sycl docs/examples (#1692)

Browse source 
* feat(refactor): refactor config and input reading

* feat(tts): read config file for TTS

* examples(kubernetes): Add simple deployment example

* examples(kubernetes): Add simple deployment for intel arc

* docs(sycl): add sycl example

* feat(tts): do not always pick a first model

* fixups to run vall-e-x on container

* Correctly resolve backend
This commit is contained in:
Ettore Di Giacinto 2024-02-10 21:37:03 +01:00 • committed by GitHub
parent 081bd07fd1
commit 53dbe36f32
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 297 additions and 98 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/content/docs/features/GPU-acceleration.md
View file

@ -112,14 +112,24 @@ llama_init_from_file: kv self size = 512.00 MB
## Intel acceleration (sycl)
#### Requirements
### Requirements
Requirement: [Intel oneAPI Base Toolkit](https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html)
If building from source, you need to install [Intel oneAPI Base Toolkit](https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html) and have the Intel drivers available in the system.
### Container images
To use SYCL, use the images with the `sycl-f16` or `sycl-f32` tag, for example `{{< version >}}-sycl-f32-core`, `{{< version >}}-sycl-f16-ffmpeg-core`, ...
The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=tags).
#### Example
To run LocalAI with Docker and sycl starting `phi-2`, you can use the following command as an example:
```bash
docker run -e DEBUG=true --privileged -ti -v $PWD/models:/build/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-sycl-f32-ffmpeg-core phi-2
```
### Notes
In addition to the commands to run LocalAI normally, you need to specify `--device /dev/dri` to docker, for example:
@ -128,3 +138,4 @@ In addition to the commands to run LocalAI normally, you need to specify `--devi
docker run --rm -ti --device /dev/dri -p 8080:8080 -e DEBUG=true -e MODELS_PATH=/models -e THREADS=1 -v $PWD/models:/models quay.io/go-skynet/local-ai:{{< version >}}-sycl-f16-ffmpeg-core
```
Note also that sycl does have a known issue to hang with `mmap: true`. You have to disable it in the model configuration if explicitly enabled.