Ettore Di Giacinto
c89271b2e4
feat(llama.cpp): add distributed llama.cpp inferencing ( #2324 )
...
* feat(llama.cpp): support distributed llama.cpp
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat: let tweak how chat messages are merged together
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Makefile: register to ALL_GRPC_BACKENDS
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactoring, allow disable auto-detection of backends
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* minor fixups
Signed-off-by: mudler <mudler@localai.io>
* feat: add cmd to start rpc-server from llama.cpp
Signed-off-by: mudler <mudler@localai.io>
* ci: add ccache
Signed-off-by: mudler <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: mudler <mudler@localai.io>
2024-05-15 01:17:02 +02:00
Ettore Di Giacinto
84e2407afa
feat(functions): allow to set JSON matcher ( #2319 )
...
Signed-off-by: mudler <mudler@localai.io>
2024-05-14 09:39:20 +02:00
Ettore Di Giacinto
efa32a2677
feat(grammar): support models with specific construct ( #2291 )
...
When enabling grammar with functions, it might be useful to
allow more flexibility to support models that are fine-tuned against returning
function calls of the form of { "name": "function_name", "arguments" {...} }
rather then { "function": "function_name", "arguments": {..} }.
This might call out to a more generic approach later on, but for the moment being we can easily support both
as we have just to specific different types.
If needed we can expand on this later on
Signed-off-by: mudler <mudler@localai.io>
2024-05-12 01:13:22 +02:00
Ettore Di Giacinto
bbea62b907
feat(functions): support models with no grammar, add tests ( #2068 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-04-18 22:43:12 +02:00
Ettore Di Giacinto
af9e5a2d05
Revert #1963 ( #2056 )
...
* Revert "fix(fncall): fix regression introduced in #1963 (#2048 )"
This reverts commit 6b06d4e0af#2038 )"
This reverts commit fdec8a9d00#2035 )"
This reverts commit e843d7df0e#1963 )"
This reverts commit eed5706994Fixes : #1920
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-04-17 23:33:49 +02:00
Dave
eed5706994
refactor: backend/service split, channel-based llm flow ( #1963 )
...
Refactor: channel based llm flow and services split
---------
Signed-off-by: Dave Lee <dave@gray101.com>
2024-04-13 09:45:34 +02:00
Ludovic Leroux
12c0d9443e
feat: use tokenizer.apply_chat_template() in vLLM ( #1990 )
...
Use tokenizer.apply_chat_template() in vLLM
Signed-off-by: Ludovic LEROUX <ludovic@inpher.io>
2024-04-11 19:20:22 +02:00
cryptk
b85dad0286
feat: first pass at improving logging ( #1956 )
...
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
2024-04-04 09:24:22 +02:00
Ettore Di Giacinto
35290e146b
fix(grammar): respect JSONmode and grammar from user input ( #1935 )
...
* fix(grammar): Fix JSON mode and custom grammar
* tests(aio): add jsonmode test
* tests(aio): add functioncall test
* fix(aio): use hermes-2-pro-mistral as llm for CPU profile
* add phi-2-orange
2024-03-31 13:04:09 +02:00
Ettore Di Giacinto
957f428fd5
fix(tools): correctly render tools response in templates ( #1932 )
...
* fix(tools): allow to correctly display both Functions and Tools
* models(hermes-2-pro): correctly display function results
2024-03-30 19:02:07 +01:00
Ettore Di Giacinto
123a5a2e16
feat(swagger): Add swagger API doc ( #1926 )
...
* makefile(build): add minimal and api build target
* feat(swagger): Add swagger
2024-03-29 22:29:33 +01:00
Ettore Di Giacinto
e533dcf506
feat(functions/aio): all-in-one images, function template enhancements ( #1862 )
...
* feat(startup): allow to specify models from local files
* feat(aio): add Dockerfile, make targets, aio profiles
* feat(template): add Function and LastMessage
* add hermes2-pro-mistral
* update hermes2 definition
* feat(template): add sprig
* feat(template): expose FunctionCall
* feat(aio): switch llm for text
2024-03-21 01:12:20 +01:00
Dave
1c312685aa
refactor: move remaining api packages to core ( #1731 )
...
* core 1
* api/openai/files fix
* core 2 - core/config
* move over core api.go and tests to the start of core/http
* move over localai specific endpoints to core/http, begin the service/endpoint split there
* refactor big chunk on the plane
* refactor chunk 2 on plane, next step: port and modify changes to request.go
* easy fixes for request.go, major changes not done yet
* lintfix
* json tag lintfix?
* gitignore and .keep files
* strange fix attempt: rename the config dir?
2024-03-01 16:19:53 +01:00
Ettore Di Giacinto
db926896bd
Revert "[Refactor]: Core/API Split" ( #1550 )
...
Revert "[Refactor]: Core/API Split (#1506 )"
This reverts commit ab7b4d5ee9
2024-01-05 18:04:46 +01:00
Dave
ab7b4d5ee9
[Refactor]: Core/API Split ( #1506 )
...
Refactors api folder to core, creates firm split between backend code and api frontend.
2024-01-05 15:34:56 +01:00
