mirror of
https://github.com/mudler/LocalAI.git
synced 2025-06-17 08:15:00 +00:00
feat: Add backend gallery (#5607)
* feat: Add backend gallery This PR add support to manage backends as similar to models. There is now available a backend gallery which can be used to install and remove extra backends. The backend gallery can be configured similarly as a model gallery, and API calls allows to install and remove new backends in runtime, and as well during the startup phase of LocalAI. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add backends docs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * wip: Backend Dockerfile for python backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat: drop extras images, build python backends separately Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixup on all backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tweaks Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop old backends leftovers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fixup CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move dockerfile upper Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix proto Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Feature dropped for consistency - we prefer model galleries Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add missing packages in the build image Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * exllama is ponly available on cublas Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * pin torch on chatterbox Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fixups to index Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Debug CI * Install accellerators deps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add target arch * Add cuda minor version Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use self-hosted runners Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci: use quay for test images Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups for vllm and chatterbox Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small fixups on CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chatterbox is only available for nvidia Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Simplify CI builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt test, use qwen3 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(model gallery): add jina-reranker-v1-tiny-en-gguf Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gguf-parser): recover from potential panics that can happen while reading ggufs with gguf-parser Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use reranker from llama.cpp in AIO images Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Limit concurrent jobs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto
GitHub
parent
a7a6020328
commit
2d64269763
114 changed files with 3996 additions and 1382 deletions
148
core/http/views/backends.html
Normal file
148
core/http/views/backends.html
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
{{template "views/partials/head" .}}
|
||||
|
||||
<body class="bg-gradient-to-br from-gray-900 to-gray-950 text-gray-200">
|
||||
<div class="flex flex-col min-h-screen">
|
||||
|
||||
{{template "views/partials/navbar" .}}
|
||||
{{ $numBackendsPerPage := 21 }}
|
||||
<div class="container mx-auto px-4 py-8 flex-grow">
|
||||
|
||||
<!-- Hero Header -->
|
||||
<div class="bg-gradient-to-r from-indigo-900/30 to-purple-900/30 rounded-2xl shadow-xl p-6 mb-8">
|
||||
<div class="max-w-4xl mx-auto text-center">
|
||||
<h1 class="text-3xl md:text-4xl font-bold text-white mb-3">
|
||||
<span class="bg-clip-text text-transparent bg-gradient-to-r from-indigo-400 to-purple-400">
|
||||
Backend Management
|
||||
</span>
|
||||
</h1>
|
||||
<p class="text-lg text-gray-300 mb-2">
|
||||
<span class="font-semibold text-indigo-300">{{.AvailableBackends}}</span> backends available
|
||||
<a href="https://localai.io/backends/" target="_blank" class="ml-2 text-blue-400 hover:text-blue-300 transition">
|
||||
<i class="fas fa-circle-info"></i>
|
||||
</a>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{{template "views/partials/inprogress" .}}
|
||||
|
||||
<!-- Search and Filter Section -->
|
||||
<div class="bg-gray-800/70 rounded-xl p-6 mb-8 shadow-lg border border-gray-700/50">
|
||||
<!-- Search Input -->
|
||||
<div class="relative mb-6">
|
||||
<div class="absolute inset-y-0 start-0 flex items-center ps-3 pointer-events-none">
|
||||
<i class="fas fa-search text-gray-400"></i>
|
||||
</div>
|
||||
<input class="form-control block w-full pl-10 px-4 py-3 text-base font-normal text-gray-300 bg-gray-900/80 bg-clip-padding border border-gray-700/70 rounded-lg transition ease-in-out focus:text-gray-200 focus:bg-gray-900 focus:border-blue-500 focus:ring-1 focus:ring-blue-500/50 focus:outline-none"
|
||||
type="search"
|
||||
name="search"
|
||||
placeholder="Search backends by name, description or type..."
|
||||
hx-post="browse/search/backends"
|
||||
hx-trigger="input changed delay:500ms, search"
|
||||
hx-target="#search-results"
|
||||
oninput="hidePagination()"
|
||||
onchange="hidePagination()"
|
||||
onsearch="hidePagination()"
|
||||
hx-indicator=".htmx-indicator">
|
||||
<span class="htmx-indicator absolute right-3 top-3">
|
||||
<svg class="animate-spin h-5 w-5 text-blue-500" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
|
||||
<circle class="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4"></circle>
|
||||
<path class="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
|
||||
</svg>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<!-- Filter by Type -->
|
||||
<div class="mb-4">
|
||||
<h3 class="text-gray-200 font-medium mb-3">Filter by type:</h3>
|
||||
<div class="flex flex-wrap gap-2">
|
||||
<button hx-post="browse/search/backends"
|
||||
class="inline-flex items-center rounded-full px-4 py-2 text-sm font-medium bg-indigo-900/60 text-indigo-200 border border-indigo-700/50 hover:bg-indigo-800 transition duration-200 ease-in-out"
|
||||
hx-target="#search-results"
|
||||
hx-vals='{"search": "llm"}'
|
||||
onclick="hidePagination()"
|
||||
hx-indicator=".htmx-indicator">
|
||||
<i class="fas fa-brain mr-2"></i>LLM
|
||||
</button>
|
||||
<button hx-post="browse/search/backends"
|
||||
class="inline-flex items-center rounded-full px-4 py-2 text-sm font-medium bg-purple-900/60 text-purple-200 border border-purple-700/50 hover:bg-purple-800 transition duration-200 ease-in-out"
|
||||
hx-target="#search-results"
|
||||
hx-vals='{"search": "diffusion"}'
|
||||
onclick="hidePagination()"
|
||||
hx-indicator=".htmx-indicator">
|
||||
<i class="fas fa-image mr-2"></i>Diffusion
|
||||
</button>
|
||||
<button hx-post="browse/search/backends"
|
||||
class="inline-flex items-center rounded-full px-4 py-2 text-sm font-medium bg-blue-900/60 text-blue-200 border border-blue-700/50 hover:bg-blue-800 transition duration-200 ease-in-out"
|
||||
hx-target="#search-results"
|
||||
hx-vals='{"search": "tts"}'
|
||||
onclick="hidePagination()"
|
||||
hx-indicator=".htmx-indicator">
|
||||
<i class="fas fa-microphone mr-2"></i>TTS
|
||||
</button>
|
||||
<button hx-post="browse/search/backends"
|
||||
class="inline-flex items-center rounded-full px-4 py-2 text-sm font-medium bg-green-900/60 text-green-200 border border-green-700/50 hover:bg-green-800 transition duration-200 ease-in-out"
|
||||
hx-target="#search-results"
|
||||
hx-vals='{"search": "whisper"}'
|
||||
onclick="hidePagination()"
|
||||
hx-indicator=".htmx-indicator">
|
||||
<i class="fas fa-headphones mr-2"></i>Whisper
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Results Section -->
|
||||
<div id="search-results" class="transition-all duration-300">
|
||||
{{.Backends}}
|
||||
</div>
|
||||
|
||||
<!-- Pagination -->
|
||||
{{ if gt .AvailableBackends $numBackendsPerPage }}
|
||||
<div id="paginate" class="flex justify-center mt-8">
|
||||
<div class="flex items-center gap-4">
|
||||
{{ if .PrevPage }}
|
||||
<button onclick="window.location.href='browse/backends?page={{.PrevPage}}'"
|
||||
class="flex items-center justify-center h-10 w-10 bg-gray-800/80 text-gray-300 hover:bg-indigo-900/70 hover:text-white rounded-lg shadow transition duration-300 ease-in-out">
|
||||
<i class="fas fa-chevron-left"></i>
|
||||
</button>
|
||||
{{ end }}
|
||||
<div class="text-gray-400 text-sm">
|
||||
Page <span class="text-white font-medium">{{.CurrentPage}}</span> of <span class="text-white font-medium">{{.TotalPages}}</span>
|
||||
</div>
|
||||
{{ if .NextPage }}
|
||||
<button onclick="window.location.href='browse/backends?page={{.NextPage}}'"
|
||||
class="flex items-center justify-center h-10 w-10 bg-gray-800/80 text-gray-300 hover:bg-indigo-900/70 hover:text-white rounded-lg shadow transition duration-300 ease-in-out">
|
||||
<i class="fas fa-chevron-right"></i>
|
||||
</button>
|
||||
{{ end }}
|
||||
</div>
|
||||
</div>
|
||||
{{ end }}
|
||||
|
||||
</div>
|
||||
{{template "views/partials/footer" .}}
|
||||
</div>
|
||||
|
||||
<script>
|
||||
function hidePagination() {
|
||||
const paginateDiv = document.getElementById('paginate');
|
||||
if (paginateDiv) {
|
||||
paginateDiv.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
// Listen for the htmx:afterSwap event to handle cases when the search results are updated
|
||||
document.body.addEventListener('htmx:afterSwap', function(event) {
|
||||
if (event.detail.target.id === 'search-results') {
|
||||
hidePagination();
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
</script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -25,6 +25,9 @@
|
|||
<a href="browse/" class="text-gray-300 hover:text-white px-3 py-2 rounded-lg transition duration-300 ease-in-out hover:bg-blue-900/30 flex items-center">
|
||||
<i class="fas fa-brain text-blue-400 mr-2"></i>Models
|
||||
</a>
|
||||
<a href="browse/backends" class="text-gray-300 hover:text-white px-3 py-2 rounded-lg transition duration-300 ease-in-out hover:bg-blue-900/30 flex items-center">
|
||||
<i class="fas fa-server text-blue-400 mr-2"></i>Backends
|
||||
</a>
|
||||
<a href="chat/" class="text-gray-300 hover:text-white px-3 py-2 rounded-lg transition duration-300 ease-in-out hover:bg-blue-900/30 flex items-center">
|
||||
<i class="fa-solid fa-comments text-blue-400 mr-2"></i>Chat
|
||||
</a>
|
||||
|
|
@ -57,6 +60,9 @@
|
|||
<a href="browse/" class="block text-gray-300 hover:text-white hover:bg-blue-900/30 px-3 py-2 rounded-lg transition duration-300 ease-in-out flex items-center">
|
||||
<i class="fas fa-brain text-blue-400 mr-3 w-5 text-center"></i>Models
|
||||
</a>
|
||||
<a href="browse/backends" class="block text-gray-300 hover:text-white hover:bg-blue-900/30 px-3 py-2 rounded-lg transition duration-300 ease-in-out flex items-center">
|
||||
<i class="fas fa-server text-blue-400 mr-3 w-5 text-center"></i>Backends
|
||||
</a>
|
||||
<a href="chat/" class="block text-gray-300 hover:text-white hover:bg-blue-900/30 px-3 py-2 rounded-lg transition duration-300 ease-in-out flex items-center">
|
||||
<i class="fa-solid fa-comments text-blue-400 mr-3 w-5 text-center"></i>Chat
|
||||
</a>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue