3 commits

Author	SHA1	Message	Date
Brandon Beiler	6a6e1a0ea9	feat(vllm): Additional vLLM config options (Disable logging, dtype, and Per-Prompt media limits) (#4855) ... * Adding the following vLLM config options: disable_log_status, dtype, limit_mm_per_prompt Signed-off-by: TheDropZone <brandonbeiler@gmail.com> * using " marks in the config.yaml file Signed-off-by: TheDropZone <brandonbeiler@gmail.com> * adding in missing colon Signed-off-by: TheDropZone <brandonbeiler@gmail.com> --------- Signed-off-by: TheDropZone <brandonbeiler@gmail.com>	2025-02-18 19:27:58 +01:00
Ettore Di Giacinto	84d6e5a987	chore(model-gallery): add more quants for popular models (#3365) ... * models(gallery): add higher quants for some llama and hermes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * models(gallery): vllm: specify a reasonable max_tokens Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-24 00:29:24 +02:00
Ettore Di Giacinto	a913fd310d	models(gallery): add hermes-3-llama-3.1(8B,70B,405B) with vLLM (#3360) ... models(gallery): add hermes-3-llama-3.1 with vLLM it adds 8b, 70b and 405b to the gallery Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-23 09:24:34 +02:00