dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-31 07:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 34

docs(mixtral): add mixtral example (#1449)

Browse source
This commit is contained in:
Ettore Di Giacinto 2023-12-16 11:44:43 -05:00 • committed by GitHub
parent 2f7beb6744
commit 1c286c3c2f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 35 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
examples/configurations/mixtral/mixtral.yaml Executable file
View file

@ -0,0 +1,16 @@
context_size: 512
f16: true
threads: 11
gpu_layers: 90
name: mixtral
mmap: true
parameters:
model: mixtral-8x7b-instruct-v0.1.Q2_K.gguf
temperature: 0.2
top_k: 40
top_p: 0.95
batch: 512
tfz: 1.0
template:
chat: mixtral-chat
completion: mixtral