dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-06-01 16:34:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 15

models(gallery): add hermes-3-llama-3.1(8B,70B,405B) with vLLM (#3360)

Browse source 
models(gallery): add hermes-3-llama-3.1 with vLLM

it adds 8b, 70b and 405b to the gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2024-08-23 09:24:34 +02:00 • committed by GitHub
parent fbaae8528d
commit a913fd310d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 152 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

32
gallery/index.yaml
View file

@ -4752,6 +4752,38 @@
- filename: Hermes-3-Llama-3.1-70B.Q4_K_M.gguf
sha256: 955c2f42caade4278f3c9dbffa32bb74572652b20e49e5340e782de3585bbe3f
uri: huggingface://NousResearch/Hermes-3-Llama-3.1-70B-GGUF/Hermes-3-Llama-3.1-70B.Q4_K_M.gguf
- &hermes-vllm
url: "github:mudler/LocalAI/gallery/hermes-vllm.yaml@master"
name: "hermes-3-llama-3.1-8b:vllm"
icon: https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/vG6j5WxHX09yj32vgjJlI.jpeg
tags:
- llm
- vllm
- gpu
- function-calling
license: llama-3
urls:
- https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B
description: |
Hermes 3 is a generalist language model with many improvements over Hermes 2, including advanced agentic capabilities, much better roleplaying, reasoning, multi-turn conversation, long context coherence, and improvements across the board. It is designed to focus on aligning LLMs to the user, with powerful steering capabilities and control given to the end user. The model uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue. It also supports function calling and structured output capabilities, generalist assistant capabilities, and improved code generation skills.
overrides:
parameters:
model: NousResearch/Hermes-3-Llama-3.1-8B
- !!merge <<: *hermes-vllm
name: "hermes-3-llama-3.1-70b:vllm"
urls:
- https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-70B
overrides:
parameters:
model: NousResearch/Hermes-3-Llama-3.1-70B
- !!merge <<: *hermes-vllm
name: "hermes-3-llama-3.1-405b:vllm"
icon: https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/-kj_KflXsdpcZoTQsvx7W.jpeg
urls:
- https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-405B
overrides:
parameters:
model: NousResearch/Hermes-3-Llama-3.1-405B
- !!merge <<: *hermes-2-pro-mistral
name: "biomistral-7b"
description: |