dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-28 22:44:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 12

fix(grammar): respect JSONmode and grammar from user input (#1935)

Browse source 
* fix(grammar): Fix JSON mode and custom grammar

* tests(aio): add jsonmode test

* tests(aio): add functioncall test

* fix(aio): use hermes-2-pro-mistral as llm for CPU profile

* add phi-2-orange
This commit is contained in:
Ettore Di Giacinto 2024-03-31 13:04:09 +02:00 • committed by GitHub
parent 784657a652
commit 35290e146b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 139 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

2
core/http/endpoints/openai/chat.go
View file

@ -185,6 +185,8 @@ func ChatEndpoint(cl *config.BackendConfigLoader, ml *model.ModelLoader, startup
input.Grammar = grammar.JSONBNF
}
config.Grammar = input.Grammar
// process functions if we have any defined or if we have a function call string
if len(input.Functions) > 0 && config.ShouldUseFunctions() {
log.Debug().Msgf("Response needs to process functions")