dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-21 02:55:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

Issue-1720: Updated Build on mac documentations (#2171)

Browse source 
updated build on macs documentation

Signed-off-by: LeonSijiaLu <leonsijialu1@gmail.com>
This commit is contained in:
Sijia Lu 2024-04-28 13:38:02 -04:00 • committed by GitHub
parent a78cd67737
commit 26e1892521
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 17 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

24
docs/content/docs/getting-started/build.md
View file

@ -154,11 +154,11 @@ cd LocalAI
# build the binary # build the binary
make build make build
# Download gpt4all-j to models/ # Download phi-2 to models/
wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j wget https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q2_K.gguf -O models/phi-2.Q2_K
# Use a template from the examples # Use a template from the examples
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/ cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/phi-2.Q2_K.tmpl
# Run LocalAI # Run LocalAI
./local-ai --models-path=./models/ --debug=true ./local-ai --models-path=./models/ --debug=true
@ -167,7 +167,7 @@ cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/
curl http://localhost:8080/v1/models curl http://localhost:8080/v1/models
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "ggml-gpt4all-j", "model": "phi-2.Q2_K",
"messages": [{"role": "user", "content": "How are you?"}], "messages": [{"role": "user", "content": "How are you?"}],
"temperature": 0.9 "temperature": 0.9
}' }'
@ -175,9 +175,19 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso
#### Troublshooting mac #### Troublshooting mac
If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store. 1. If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256). 2. After the installation of Xcode, if you receive a xcrun error `'xcrun: error: unable to find utility "metal", not a developer tool or in PATH'`. You might have installed the Xcode command line tools before installing Xcode, the former one is pointing to an incomplete SDK.
If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
```
# print /Library/Developer/CommandLineTools, if command line tools were installed in advance
xcode-select --print-path
# point to a complete SDK
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
```
3. If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
4. If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
``` ```
# reinstall build dependencies # reinstall build dependencies