dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat(transformers): Use downloaded model for Transformers backend if it already exists. (#3777)

Browse source 
* signing commit

Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com>

* Update transformers backend to check for existing model directory

Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com>

---------

Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com>
This commit is contained in:
Josh Bennett 2024-10-10 04:42:59 -04:00 • committed by GitHub
parent ea8675d473
commit 85a3cc8d8f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
backend/python/transformers/backend.py
View file

@ -72,8 +72,13 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
Returns:
A Result object that contains the result of the LoadModel operation.
"""
model_name = request.Model
# Check to see if the Model exists in the filesystem already.
if os.path.exists(request.ModelFile):
model_name = request.ModelFile
compute = torch.float16
if request.F16Memory == True:
compute=torch.bfloat16