dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-01 10:14:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

docs: Update Qwen 2.5 Coder 32B Instruct model comparison with Hyperbolic labs API

Browse source
This commit is contained in:
Paul Gauthier 2024-11-22 07:16:01 -08:00 • committed by Paul Gauthier (aider)
parent f9ef161991
commit 23095ada85
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
aider/website/_posts/2024-11-21-quantization.md
View file

@ -24,10 +24,11 @@ and local model servers like Ollama.
{% include quant-chart.js %}
</script>
The graph above compares 3 different versions of the Qwen 2.5 Coder 32B Instruct model,
The graph above compares 4 different versions of the Qwen 2.5 Coder 32B Instruct model,
served both locally and from cloud providers.
- The [HuggingFace weights](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct) served via [glhf.chat](https://glhf.chat).
- Hyperbolic labs API for [qwen2-5-coder-32b-instruct](https://app.hyperbolic.xyz/models/qwen2-5-coder-32b-instruct), which has BF16 quantization.
- The results from [OpenRouter's mix of providers](https://openrouter.ai/qwen/qwen-2.5-coder-32b-instruct/providers) which serve the model with different levels of quantization.
- Ollama locally serving [qwen2.5-coder:32b-instruct-q4_K_M)](https://ollama.com/library/qwen2.5-coder:32b-instruct-q4_K_M), which has `Q4_K_M` quantization.