dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-06-26 20:55:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 22

using " marks in the config.yaml file

Browse source 
Signed-off-by: TheDropZone <brandonbeiler@gmail.com>
This commit is contained in:
TheDropZone 2025-02-17 16:25:24 -05:00
parent f0f2c87553
commit fade34394c
No known key found for this signature in database
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
gallery/vllm.yaml
View file

@ -16,8 +16,8 @@ config_file: |
use_tokenizer_template: true
# Uncomment to specify a quantization method (optional)
# quantization: "awq"
# Uncomment to set dtype, choices are: 'auto', 'half', 'float16', 'bfloat16', 'float', 'float32'. awq on vLLM does not support bfloat16
# dtype: 'float16'
# Uncomment to set dtype, choices are: "auto", "half", "float16", "bfloat16", "float", "float32". awq on vLLM does not support bfloat16
# dtype: "float16"
# Uncomment to limit the GPU memory utilization (vLLM default is 0.9 for 90%)
# gpu_memory_utilization: 0.5
# Uncomment to trust remote code from huggingface