mirror of
https://github.com/mudler/LocalAI.git
synced 2025-05-31 07:54:59 +00:00
feat(functions/aio): all-in-one images, function template enhancements (#1862)
* feat(startup): allow to specify models from local files * feat(aio): add Dockerfile, make targets, aio profiles * feat(template): add Function and LastMessage * add hermes2-pro-mistral * update hermes2 definition * feat(template): add sprig * feat(template): expose FunctionCall * feat(aio): switch llm for text
This commit is contained in:
parent
eeaf8c7ccd
commit
e533dcf506
20 changed files with 462 additions and 2 deletions
22
aio/cpu/text-to-text.yaml
Normal file
22
aio/cpu/text-to-text.yaml
Normal file
|
@ -0,0 +1,22 @@
|
|||
name: gpt-3.5-turbo
|
||||
context_size: 2048
|
||||
f16: true
|
||||
gpu_layers: 90
|
||||
mmap: true
|
||||
trimsuffix:
|
||||
- "\n"
|
||||
parameters:
|
||||
model: huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
|
||||
|
||||
template:
|
||||
chat: &template |-
|
||||
Instruct: {{.Input}}
|
||||
Output:
|
||||
completion: *template
|
||||
|
||||
usage: |
|
||||
To use this model, interact with the API (in another terminal) with curl for instance:
|
||||
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
|
||||
"model": "phi-2",
|
||||
"messages": [{"role": "user", "content": "How are you doing?", "temperature": 0.1}]
|
||||
}'
|
Loading…
Add table
Add a link
Reference in a new issue