dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-31 17:55:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

copy

Browse source
This commit is contained in:
Paul Gauthier 2024-07-01 17:01:37 -03:00
parent 5e8458f123
commit 8c8b29c3cb
1 changed files with 0 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

14
website/docs/leaderboards/index.md
View file

@ -16,20 +16,6 @@ While [aider can connect to almost any LLM](/docs/llms.html),
it works best with models that score well on the benchmarks.
## Claude 3.5 Sonnet takes the top spot
Claude 3.5 Sonnet is now the top ranked model on aider's code editing leaderboard.
DeepSeek Coder V2 only spent 4 days in the top spot.
The new Sonnet came in 3rd on aider's refactoring leaderboard, behind GPT-4o and Opus.
Sonnet ranked #1 when using the "whole" editing format,
but it also scored very well with
aider's "diff" editing format.
This format allows it to return code changes as diffs -- saving time and token costs,
and making it practical to work with larger source files.
As such, aider uses "diff" by default with this new Sonnet model.
## Code editing leaderboard
[Aider's code editing benchmark](/docs/benchmarks.html#the-benchmark) asks the LLM to edit python source files to complete 133 small coding exercises. This benchmark measures the LLM's coding ability, but also whether it can consistently emit code edits in the format specified in the system prompt.