dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-21 19:15:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

feat: 🐍 add mamba support (#1589)

Browse source 
feat(mamba): Initial import

This is a first iteration of the mamba backend, loosely based on
mamba-chat(https://github.com/havenhq/mamba-chat).
This commit is contained in:
Ettore Di Giacinto 2024-01-19 23:42:50 +01:00 • committed by GitHub
parent 52c9a7f45d
commit 9e653d6abe
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 762 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

8
backend/python/coqui/coqui_server.py
View file

@ -33,7 +33,13 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
def LoadModel(self, request, context):
# Get device
device = "cuda" if request.CUDA else "cpu"
# device = "cuda" if request.CUDA else "cpu"
if torch.cuda.is_available():
print("CUDA is available", file=sys.stderr)
device = "cuda"
else:
print("CUDA is not available", file=sys.stderr)
device = "cpu"
if not torch.cuda.is_available() and request.CUDA:
return backend_pb2.Result(success=False, message="CUDA is not available")