feat(models): use rwkv from llama.cpp (#4264)

feat(rwkv): use rwkv from llama.cpp

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2024-11-26 14:22:55 +01:00 committed by GitHub
parent 03800ccceb
commit 2b62260b6d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 18 additions and 177 deletions

View file

@ -1,18 +1,23 @@
name: rwkv_test
parameters:
model: rwkv
model: huggingface://bartowski/rwkv-6-world-7b-GGUF/rwkv-6-world-7b-Q4_K_M.gguf
top_k: 80
temperature: 0.9
max_tokens: 100
max_tokens: 4098
top_p: 0.8
context_size: 1024
backend: "rwkv"
cutwords:
- "Bob:.*"
context_size: 4098
roles:
user: "Bob:"
system: "Alice:"
assistant: "Alice:"
user: "User: "
system: "System: "
assistant: "Assistant: "
stopwords:
- 'Assistant:'
template:
completion: rwkv_completion
chat: rwkv_chat
chat: |
{{.Input}}
Assistant:
completion: |
{{.Input}}

View file

@ -1,13 +0,0 @@
The following is a verbose detailed conversation between Bob and a woman, Alice. Alice is intelligent, friendly and likeable. Alice is likely to agree with Bob.
Bob: Hello Alice, how are you doing?
Alice: Hi Bob! Thanks, I'm fine. What about you?
Bob: I am very good! It's nice to see you. Would you mind me chatting with you for a while?
Alice: Not at all! I'm listening.
{{.Input}}
Alice:

View file

@ -1 +0,0 @@
Complete the following sentence: {{.Input}}