dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 10:35:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

Add tensor_parallel_size setting to vllm setting items (#2085)

Browse source 
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>
This commit is contained in:
Taikono-Himazin 2024-04-20 23:37:02 +09:00 • committed by GitHub
parent b319ed58b0
commit 03adc1f60d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
core/backend/options.go
View file

@ -74,6 +74,7 @@ func gRPCModelOpts(c config.BackendConfig) *pb.ModelOptions {
EnforceEager: c.EnforceEager,
SwapSpace: int32(c.SwapSpace),
MaxModelLen: int32(c.MaxModelLen),
TensorParallelSize: int32(c.TensorParallelSize),
MMProj: c.MMProj,
YarnExtFactor: c.YarnExtFactor,
YarnAttnFactor: c.YarnAttnFactor,