dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat: add ngqa and RMSNormEps parameters (#860)

Browse source 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2023-08-03 00:51:08 +02:00 • committed by GitHub
parent 953e30814a
commit 5ca21ee398
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 138 additions and 104 deletions
Download patch file Download diff file Expand all files Collapse all files

2
api/backend/options.go
View file

@ -18,6 +18,8 @@ func gRPCModelOpts(c config.Config) *pb.ModelOptions {
ContextSize: int32(c.ContextSize),
Seed: int32(c.Seed),
NBatch: int32(b),
NGQA: c.NGQA,
RMSNormEps: c.RMSNormEps,
F16Memory: c.F16,
MLock: c.MMlock,
RopeFreqBase: c.RopeFreqBase,