285 commits

Author	SHA1	Message	Date
Ettore Di Giacinto	1cc4525f15	fix: adapt test to error changes ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-30 17:43:59 +02:00
Ettore Di Giacinto	45c58752e5	feat(ui): add audio upload button in chat view (#5526) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-30 16:47:31 +02:00
Ettore Di Giacinto	88de2ea01a	feat(llama.cpp): add support for audio input (#5466) ... * feat(llama.cpp): add support for audio input Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-26 16:06:03 +02:00
Richard Palethorpe	bf6426aef2	feat: Realtime API support reboot (#5392) ... * feat(realtime): Initial Realtime API implementation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore: go mod tidy Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat: Implement transcription only mode for realtime API Reduce the scope of the real time API for the initial realease and make transcription only mode functional. Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(build): Build backends on a separate layer to speed up core only changes Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Richard Palethorpe <io@richiejp.com> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-25 22:25:05 +02:00
Ettore Di Giacinto	159388cce8	chore: memoize detected GPUs (#5385) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-18 08:55:44 +02:00
Ettore Di Giacinto	6d5bde860b	feat(llama.cpp): upgrade and use libmtmd (#5379) ... * WIP * wip * wip * Make it compile * Update json.hpp * this shouldn't be private for now * Add logs * Reset auto detected template Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Re-enable grammars * This seems to be broken - 360a9c98e1 (diff-a18a8e64e12a01167d8e98fc)[…]cccf0d4eed09d76d879L2998-L3207 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Placeholder * Simplify image loading * use completion type * disable streaming Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * correctly return timings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove some debug logging * Adapt tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Keep header * embedding: do not use oai type Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Sync from server.cpp * Use utils and json directly from llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Sync with upstream Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: copy json.hpp from the correct location Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: add httplib * sync llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Embeddiongs: set OAICOMPAT_TYPE_EMBEDDING Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat: sync with server.cpp by including it Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * make it darwin-compatible Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-17 16:02:53 +02:00
omahs	0f365ac204	fix: typos (#5376) ... Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>	2025-05-16 12:45:48 +02:00
Ettore Di Giacinto	21bdfe5fa4	fix: use rice when embedding large binaries (#5309) ... * fix(embed): use go-rice for large backend assets Golang embed FS has a hard limit that we might exceed when providing many binary alternatives. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * simplify golang deps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): switch to testcontainers and print logs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tests): do not build a test binary Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * small fixup Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-04 16:42:42 +02:00
Ettore Di Giacinto	72111c597d	fix(gpu): do not assume gpu being returned has node and mem (#5310) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-03 19:00:24 +02:00
Ettore Di Giacinto	5c6cd50ed6	feat(llama.cpp): estimate vram usage (#5299) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-02 17:40:26 +02:00
Ettore Di Giacinto	2c9279a542	feat(video-gen): add endpoint for video generation (#5247) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-04-26 18:05:01 +02:00
Ettore Di Giacinto	9628860c0e	feat(llama.cpp/clip): inject gpu options if we detect GPUs (#5243) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-04-26 00:04:47 +02:00
Ettore Di Giacinto	8abecb4a18	chore: bump grpc limits to 50MB (#5212) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-04-19 08:53:24 +02:00
Ettore Di Giacinto	2c425e9c69	feat(loader): enhance single active backend by treating as singleton (#5107) ... feat(loader): enhance single active backend by treating at singleton Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-04-01 20:58:11 +02:00
Ettore Di Giacinto	05f7004487	fix: race during stop of active backends (#5106) ... * chore: drop double call to stop all backends, refactors Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: do lock when cycling to models to delete Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-04-01 00:01:10 +02:00
Ettore Di Giacinto	67f7bffd18	chore(deps): update llama.cpp and sync with upstream changes (#4950) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-03-06 00:40:58 +01:00
Ettore Di Giacinto	6b46c52789	feat(ui): complete design overhaul (#4942) ... This PR changes entirely the UI look and feeling. It updates all sections and makes it also mobile-ready. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-03-05 08:27:03 +01:00
Bas Hulsken	bbf30d416d	fix: change initialization order of llama-cpp-avx512 to go before avx2 variant (#4837) ... changed to initialization order of the avx512 version of llama.cpp, now tries before avx2 Signed-off-by: Bas Hulsken <bhulsken@hotmail.com>	2025-02-17 09:32:21 +01:00
Dave	3cddf24747	feat: Centralized Request Processing middleware (#3847) ... * squash past, centralize request middleware PR Signed-off-by: Dave Lee <dave@gray101.com> * migrate bruno request files to examples repo Signed-off-by: Dave Lee <dave@gray101.com> * fix Signed-off-by: Dave Lee <dave@gray101.com> * Update tests/e2e-aio/e2e_test.go Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> --------- Signed-off-by: Dave Lee <dave@gray101.com> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2025-02-10 12:06:16 +01:00
Ettore Di Giacinto	7f90ff7aec	chore(llama-ggml): drop deprecated backend (#4775) ... The GGML format is now dead, since in the next version of LocalAI we already bring many breaking compatibility changes, taking the occasion also to drop ggml support (pre-gguf). Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-02-06 18:36:23 +01:00
Ettore Di Giacinto	1d6afbd65d	feat(llama.cpp): Add support to grammar triggers (#4733) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-02-02 13:25:03 +01:00
Ettore Di Giacinto	72e52c4f6a	chore: drop embedded models (#4715) ... Since the remote gallery was introduced this is now completely superseded by it. In order to keep the code clean and remove redudant parts let's simplify the usage. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-30 00:03:01 +01:00
Maximilian Kenfenheuer	b4b67e00bd	refactor: function argument parsing using named regex (#4708) ... Signed-off-by: Maximilian Kenfenheuer <maximilian.kenfenheuer@ksol.it>	2025-01-28 22:58:02 +01:00
Maximilian Kenfenheuer	539e94db73	feat: function argument parsing using named regex (#4700) ... Signed-off-by: Maximilian Kenfenheuer <maximilian.kenfenheuer@ksol.it>	2025-01-27 15:53:05 +00:00
Ettore Di Giacinto	8282414583	chore(downloader): support hf.co and hf:// URIs (#4677) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-24 08:27:22 +01:00
Ettore Di Giacinto	5177837ab0	chore: detect and enable avx512 builds (#4675) ... chore(avx512): add support Fixes https://github.com/mudler/LocalAI/issues/4662 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-24 08:26:44 +01:00
Ettore Di Giacinto	f9e368b7c4	chore(refactor): group cpu cap detection (#4674) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-23 16:35:44 +01:00
Ettore Di Giacinto	89429a439b	feat(transformers): add support to Mamba (#4669) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-23 09:30:47 +01:00
Peter Cover	a05737c7e4	chore: fix some function names in comment (#4665) ... Signed-off-by: petercover <raowanxiang@outlook.com>	2025-01-22 19:35:53 +01:00
Ettore Di Giacinto	e15d29aba2	chore(stablediffusion-ncn): drop in favor of ggml implementation (#4652) ... * chore(stablediffusion-ncn): drop in favor of ggml implementation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ci): drop stablediffusion build Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): add Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): try to fixup current tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Try to fix tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tests improvements Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): use quality to specify step Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): switch to sd-1.5 also increase prep time for downloading models Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-22 19:34:16 +01:00
Gianluca Boiano	032a33de49	chore: remove deprecated tinydream backend (#4631) ... Signed-off-by: Gianluca Boiano <morf3089@gmail.com>	2025-01-18 18:35:30 +01:00
Ettore Di Giacinto	1e9bf19c8d	feat(transformers): merge sentencetransformers backend (#4624) ... * merge sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias to silently redirect sentencetransformers to transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias also for transformers-musicgen Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop from makefile Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move tests from sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove tests from CI (part of transformers) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Do not always try to load the tokenizer Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix typo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tiny adjustments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-18 18:30:30 +01:00
Ettore Di Giacinto	a761e01944	chore: alias transformers-musicgen to transformers (#4623) ... chore: alias transformers-muscigen to transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-17 18:16:17 +01:00
Ettore Di Giacinto	acb2eb23c8	feat(tts): Add Kokoro backend (#4616) ... * feat(kokoro): Add new TTS backend Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add kokoro to images Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Support combined voices Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Ignore pt and onnx Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add plbert and istfnet Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-16 22:23:09 +01:00
Saarthak Verma	6765b17acd	feat(dowloader): resume partial downloads (#4537) ... * feat(resume downloads): add basic tests Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * test(resume downloads): implement file download tc Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * test(resume downloads): add resume partial download test Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * feat(resume downloads): implement resumable downloads for interrupted transfers - Adds support for resuming partially downloaded files - Uses HTTP Range header to continue from last byte position - Maintains download progress across interruptions - Preserves partial downloads with .partial extension - Validates SHA256 checksum after completion Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * fix(resume downloads): incorrect download percent on front end Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * feat(resume download): add range header check tc Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> * feat(resume download): implement range header check Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com> --------- Signed-off-by: Saarthak Verma <saarthakverma739@gmail.com>	2025-01-09 09:22:52 +01:00
mintyleaf	2bc4b56a79	feat: stream tokens usage (#4415) ... * Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame * Fix 'hang' on empty message from the start Seems like that empty message marker trick was unnecessary --------- Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-12-18 09:48:50 +01:00
Ettore Di Giacinto	f943c4b803	Revert "feat: include tokens usage for streamed output" (#4336) ... Revert "feat: include tokens usage for streamed output (#4282)" This reverts commit 0d6c3a7d57.	2024-12-08 17:53:36 +01:00
Ettore Di Giacinto	cea5a0ea42	feat(template): read jinja templates from gguf files (#4332) ... * Read jinja templates as fallback Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move templating out of model loader Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Test TemplateMessages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Set role and content from transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tests: be more flexible Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * More jinja Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small refactoring and adaptations Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-08 13:50:33 +01:00
mintyleaf	0d6c3a7d57	feat: include tokens usage for streamed output (#4282) ... Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-28 14:47:56 +01:00
Ettore Di Giacinto	3c3050f68e	feat(backends): Drop bert.cpp (#4272) ... * feat(backends): Drop bert.cpp use llama.cpp 3.2 as a drop-in replacement for bert.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): make test more robust Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-27 16:34:28 +01:00
Ettore Di Giacinto	2b62260b6d	feat(models): use rwkv from llama.cpp (#4264) ... feat(rwkv): use rwkv from llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-26 14:22:55 +01:00
Ettore Di Giacinto	b1ea9318e6	feat(silero): add Silero-vad backend (#4204) ... * feat(vad): add silero-vad backend (WIP) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vad): add API endpoint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vad): correctly place the onnxruntime libs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(vad): hook silero-vad to binary and container builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gRPC): register VAD Server Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): consume ONNX_OS consistently Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): handle macOS Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-20 14:48:40 +01:00
Ettore Di Giacinto	1770b92fb6	chore(api): return values from schema (#4153) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-14 14:12:29 +01:00
Ikko Eltociear Ashimine	9688f516e0	chore: update jobresult_test.go (#4124) ... recieve -> receive Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>	2024-11-12 08:52:18 +01:00
Ettore Di Giacinto	6daef00d30	chore(refactor): drop unnecessary code in loader (#4096) ... * chore: simplify passing options to ModelOptions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(refactor): do not expose internal backend Loader Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-08 21:54:25 +01:00
Ettore Di Giacinto	c4af769d4f	chore: hide raw safetensors files (#4085) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-06 12:04:39 +01:00
Arnaud A	65c3df392c	feat(tts): Implement naive response_format for tts endpoint (#4035) ... Signed-off-by: n-Arno <arnaud.alcabas@gmail.com>	2024-11-02 19:13:35 +00:00
Ettore Di Giacinto	61c964dce7	fix(grpc): pass by modelpath (#4023) ... Instead of trying to derive it from the model file. In backends that specify HF url this results in a fragile logic. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-31 12:12:22 +01:00
Ettore Di Giacinto	b1243453f4	chore(tests): fix examples url ... Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-10-30 10:57:21 +01:00
Ettore Di Giacinto	ccc7cb0287	feat(templates): use a single template for multimodals messages (#3892) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-22 09:34:05 +02:00