dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-28 00:05:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Added 3.5 sonnet

Browse source
This commit is contained in:
Paul Gauthier 2024-06-20 08:26:35 -07:00
parent 068609e4ef
commit 090e0cdcfe
3 changed files with 68 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

22
website/docs/leaderboards/index.md
View file

@ -16,22 +16,16 @@ While [aider can connect to almost any LLM](/docs/llms.html),
it works best with models that score well on the benchmarks.
## DeepSeek Coder V2 beats GPT-4o, Opus
## Claude 3.5 Sonnet takes the top spot
The new
[DeepSeek Coder V2](https://aider.chat/docs/llms/deepseek.html)
model is now atop aider's code editing leaderboard!
It's worth noting that DeepSeek Coder V2 is only capable of using aider's "whole" edit format.
This means it returns a modified full copy of each file when it makes changes.
Most other strong models are able to use aider's "diff" editing format,
which allows them to return diffs of edits -- saving time and token costs.
Models which use the "whole" edit format can only edit files
which fit within their output token limits.
These output limits are often as low as 4k tokens, even for models
with very large context windows.
Claude 3.5 Sonnet is now the top ranked model on aider's code editing leaderboard.
DeepSeek Coder V2 previously took the #1 spot, only 4 days ago.
Sonnet ranked #1 when using the "whole" editing format,
but it also scored very well with
aider's "diff" editing format.
This format allows it to return code changes as diffs -- saving time and token costs,
and making it practical to work with larger source files.
## Code editing leaderboard