dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

chore(defaults): enlarge defaults, drop gpu layers which is infered (#5308)

Browse source 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2025-05-03 18:44:51 +02:00 • committed by GitHub
parent 1fc6d469ac
commit b2f9fc870b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 1 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

2
core/backend/options.go
View file

@ -99,7 +99,7 @@ func grpcModelOpts(c config.BackendConfig) *pb.ModelOptions {
mmap = *c.MMap
}
ctxSize := 1024
ctxSize := 4096
if c.ContextSize != nil {
ctxSize = *c.ContextSize
}