This commit is contained in:
Paul Gauthier 2024-11-23 18:39:20 -08:00
parent bb78e2f57f
commit 757eac0579
2 changed files with 24 additions and 0 deletions

View file

@ -158,4 +158,27 @@
date: 2024-11-23
versions: 0.64.2.dev
seconds_per_case: 35.7
total_cost: 0.0000
- dirname: 2024-11-24-02-23-32--deepinfra-qwen-diff
test_cases: 133
model: Deepinfra (BF16)
edit_format: diff
commit_hash: bb78e2f
pass_rate_1: 58.6
pass_rate_2: 72.2
percent_cases_well_formed: 94.7
error_outputs: 15
num_malformed_responses: 13
num_with_malformed_responses: 7
user_asks: 3
lazy_comments: 0
syntax_errors: 0
indentation_errors: 0
exhausted_context_windows: 2
test_timeouts: 3
command: aider --model deepinfra/Qwen/Qwen2.5-Coder-32B-Instruct
date: 2024-11-24
versions: 0.64.2.dev
seconds_per_case: 17.5
total_cost: 0.0000

View file

@ -32,6 +32,7 @@ served both locally and from cloud providers.
- [4bit and 8bit quants for mlx](https://t.co/cwX3DYX35D).
- The results from [OpenRouter's mix of providers](https://openrouter.ai/qwen/qwen-2.5-coder-32b-instruct/providers) which serve the model with different levels of quantization.
- Ollama locally serving different quantizations from the [Ollama model library](https://ollama.com/library/qwen2.5-coder:32b-instruct-q4_K_M).
- Other API providers.
The best version of the model rivals GPT-4o, while the worst performer
is more like GPT-4 level.