mirror of
https://github.com/mudler/LocalAI.git
synced 2025-06-20 09:44:59 +00:00
8b169f1dac
Also adds embeddings and llava models Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
81 lines
3.4 KiB
YAML
81 lines
3.4 KiB
YAML
name: "hermes-2-pro-mistral"
|
|
icon: https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ggO2sBDJ8Bhc6w-zwTx5j.png
|
|
license: apache-2.0
|
|
|
|
description: |
|
|
Hermes 2 Pro is an upgraded, retrained version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house.
|
|
|
|
This new version of Hermes maintains its excellent general task and conversation capabilities - but also excels at Function Calling, JSON Structured Outputs, and has improved on several other metrics as well, scoring a 90% on our function calling evaluation built in partnership with Fireworks.AI, and an 81% on our structured JSON Output evaluation.
|
|
|
|
Hermes Pro takes advantage of a special system prompt and multi-turn function calling structure with a new chatml role in order to make function calling reliable and easy to parse. Learn more about prompting below.
|
|
|
|
This work was a collaboration between Nous Research, @interstellarninja, and Fireworks.AI
|
|
|
|
Learn more about the function calling on our github repo here: https://github.com/NousResearch/Hermes-Function-Calling/tree/main
|
|
|
|
urls:
|
|
- https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
|
|
|
|
tags:
|
|
- llm
|
|
- gguf
|
|
- gpu
|
|
- cpu
|
|
|
|
config_file: |
|
|
mmap: true
|
|
parameters:
|
|
model: Hermes-2-Pro-Mistral-7B.Q6_K.gguf
|
|
|
|
template:
|
|
chat_message: |
|
|
<|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "tool"}}tool{{else if eq .RoleName "user"}}user{{end}}
|
|
{{- if .FunctionCall }}
|
|
<tool_call>
|
|
{{- else if eq .RoleName "tool" }}
|
|
<tool_response>
|
|
{{- end }}
|
|
{{- if .Content}}
|
|
{{.Content }}
|
|
{{- end }}
|
|
{{- if .FunctionCall}}
|
|
{{toJson .FunctionCall}}
|
|
{{- end }}
|
|
{{- if .FunctionCall }}
|
|
</tool_call>
|
|
{{- else if eq .RoleName "tool" }}
|
|
</tool_response>
|
|
{{- end }}
|
|
<|im_end|>
|
|
# https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling
|
|
function: |
|
|
<|im_start|>system
|
|
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools:
|
|
<tools>
|
|
{{range .Functions}}
|
|
{'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }}
|
|
{{end}}
|
|
</tools>
|
|
Use the following pydantic model json schema for each tool call you will make:
|
|
{'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}
|
|
For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
|
|
<tool_call>
|
|
{'arguments': <args-dict>, 'name': <function-name>}
|
|
</tool_call>
|
|
<|im_end|>
|
|
{{.Input -}}
|
|
<|im_start|>assistant
|
|
<tool_call>
|
|
chat: |
|
|
{{.Input -}}
|
|
<|im_start|>assistant
|
|
completion: |
|
|
{{.Input}}
|
|
context_size: 4096
|
|
f16: true
|
|
stopwords:
|
|
- <|im_end|>
|
|
- <dummy32000>
|
|
- "\n</tool_call>"
|
|
- "\n\n\n"
|
|
|