dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-06-17 08:15:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 23

feat: Add backend gallery (#5607)

Browse source 
* feat: Add backend gallery

This PR add support to manage backends as similar to models. There is
now available a backend gallery which can be used to install and remove
extra backends.
The backend gallery can be configured similarly as a model gallery, and
API calls allows to install and remove new backends in runtime, and as
well during the startup phase of LocalAI.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add backends docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* wip: Backend Dockerfile for python backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat: drop extras images, build python backends separately

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixup on all backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Tweaks

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Drop old backends leftovers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixup CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Move dockerfile upper

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fix proto

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Feature dropped for consistency - we prefer model galleries

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add missing packages in the build image

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* exllama is ponly available on cublas

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* pin torch on chatterbox

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups to index

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Debug CI

* Install accellerators deps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add target arch

* Add cuda minor version

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use self-hosted runners

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ci: use quay for test images

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups for vllm and chatterbox

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Small fixups on CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chatterbox is only available for nvidia

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Simplify CI builds

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Adapt test, use qwen3

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(model gallery): add jina-reranker-v1-tiny-en-gguf

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gguf-parser): recover from potential panics that can happen while reading ggufs with gguf-parser

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use reranker from llama.cpp in AIO images

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Limit concurrent jobs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto 2025-06-15 14:56:52 +02:00 • committed by GitHub
parent a7a6020328
commit 2d64269763
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
114 changed files with 3996 additions and 1382 deletions
Download patch file Download diff file Expand all files Collapse all files

6
swagger/docs.go
View file

@ -246,7 +246,7 @@ const docTemplate = `{
"schema": {
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/gallery.GalleryOpStatus"
"$ref": "#/definitions/services.GalleryOpStatus"
}
}
}
@ -260,7 +260,7 @@ const docTemplate = `{
"200": {
"description": "Response",
"schema": {
"$ref": "#/definitions/gallery.GalleryOpStatus"
"$ref": "#/definitions/services.GalleryOpStatus"
}
}
}
@ -987,7 +987,7 @@ const docTemplate = `{
}
}
},
"gallery.GalleryOpStatus": {
"services.GalleryOpStatus": {
"type": "object",
"properties": {
"deletion": {

6
swagger/swagger.json
View file

@ -239,7 +239,7 @@
"schema": {
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/gallery.GalleryOpStatus"
"$ref": "#/definitions/services.GalleryOpStatus"
}
}
}
@ -253,7 +253,7 @@
"200": {
"description": "Response",
"schema": {
"$ref": "#/definitions/gallery.GalleryOpStatus"
"$ref": "#/definitions/services.GalleryOpStatus"
}
}
}
@ -980,7 +980,7 @@
}
}
},
"gallery.GalleryOpStatus": {
"services.GalleryOpStatus": {
"type": "object",
"properties": {
"deletion": {

6
swagger/swagger.yaml
View file

@ -100,7 +100,7 @@ definitions:
type: string
type: array
type: object
gallery.GalleryOpStatus:
services.GalleryOpStatus:
properties:
deletion:
description: Deletion is true if the operation is a deletion
@ -891,7 +891,7 @@ paths:
description: Response
schema:
additionalProperties:
$ref: '#/definitions/gallery.GalleryOpStatus'
$ref: '#/definitions/services.GalleryOpStatus'
type: object
summary: Returns all the jobs status progress
/models/jobs/{uuid}:
@ -900,7 +900,7 @@ paths:
"200":
description: Response
schema:
$ref: '#/definitions/gallery.GalleryOpStatus'
$ref: '#/definitions/services.GalleryOpStatus'
summary: Returns the job status
/system:
get: