From cf0aff8c402cc7a7ebfd94a90e596621f378b584 Mon Sep 17 00:00:00 2001 From: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Date: Thu, 6 Mar 2025 19:43:16 -0800 Subject: [PATCH 1/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 404759b50..41411c6d2 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Aider lets you pair program with LLMs, to edit code in your local git repository. Start a new project or work with an existing code base. -Aider works best with Claude 3.5 Sonnet, DeepSeek R1 & Chat V3, OpenAI o1, o3-mini & GPT-4o. Aider can [connect to almost any LLM, including local models](https://aider.chat/docs/llms.html). +Aider works best with Claude 3.7 Sonnet, DeepSeek R1 & Chat V3, OpenAI o1, o3-mini & GPT-4o. Aider can [connect to almost any LLM, including local models](https://aider.chat/docs/llms.html).

From 65309854ac87f5843ae1d81a72d9d18d1feca04c Mon Sep 17 00:00:00 2001 From: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Date: Thu, 6 Mar 2025 19:43:46 -0800 Subject: [PATCH 2/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 41411c6d2..ad723860d 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ Pair program with AI. - [Add images to the chat](https://aider.chat/docs/usage/images-urls.html) (GPT-4o, Claude 3.5 Sonnet, etc). - [Add URLs to the chat](https://aider.chat/docs/usage/images-urls.html) and aider will read their content. - [Code with your voice](https://aider.chat/docs/usage/voice.html). -- Aider works best with Claude 3.5 Sonnet, DeepSeek V3, o1 & GPT-4o and can [connect to almost any LLM](https://aider.chat/docs/llms.html). +- Aider works best with Claude 3.7 Sonnet, DeepSeek V3, o1 & GPT-4o and can [connect to almost any LLM](https://aider.chat/docs/llms.html). ## Top tier performance From f111ab48fb05be0c27cc7d00fac40daaae5e3166 Mon Sep 17 00:00:00 2001 From: Paul Gauthier Date: Fri, 7 Mar 2025 09:26:32 -0800 Subject: [PATCH 3/4] chore: Update polyglot leaderboard data with new model test results --- aider/website/_data/polyglot_leaderboard.yml | 32 ++++++++++++++++++-- aider/website/docs/leaderboards/index.md | 2 +- 2 files changed, 31 insertions(+), 3 deletions(-) diff --git a/aider/website/_data/polyglot_leaderboard.yml b/aider/website/_data/polyglot_leaderboard.yml index a2fdf5516..715f94cdb 100644 --- a/aider/website/_data/polyglot_leaderboard.yml +++ b/aider/website/_data/polyglot_leaderboard.yml @@ -677,7 +677,7 @@ - dirname: 2025-03-06-17-40-24--qwq32b-diff-temp-topp-ex-sys-remind-user-for-real test_cases: 225 - model: qwq-32b + model: QwQ-32B edit_format: diff commit_hash: 51d118f-dirty pass_rate_1: 8.0 @@ -699,4 +699,32 @@ date: 2025-03-06 versions: 0.75.3.dev seconds_per_case: 228.6 - total_cost: 0.0000 \ No newline at end of file + total_cost: 0.0000 + +- dirname: 2025-03-07-15-11-27--qwq32b-arch-temp-topp-again + test_cases: 225 + model: QwQ-32B + Qwen 2.5 Coder Instruct + edit_format: architect + commit_hash: 52162a5 + editor_model: fireworks_ai/accounts/fireworks/models/qwen2p5-coder-32b-instruct + editor_edit_format: editor-diff + pass_rate_1: 9.8 + pass_rate_2: 26.2 + pass_num_1: 22 + pass_num_2: 59 + percent_cases_well_formed: 100.0 + error_outputs: 122 + num_malformed_responses: 0 + num_with_malformed_responses: 0 + user_asks: 489 + lazy_comments: 8 + syntax_errors: 0 + indentation_errors: 0 + exhausted_context_windows: 1 + test_timeouts: 2 + total_tests: 225 + command: aider --model fireworks_ai/accounts/fireworks/models/qwq-32b --architect + date: 2025-03-07 + versions: 0.75.3.dev + seconds_per_case: 137.4 + total_cost: 0 \ No newline at end of file diff --git a/aider/website/docs/leaderboards/index.md b/aider/website/docs/leaderboards/index.md index 652b91149..cb2c4ef3b 100644 --- a/aider/website/docs/leaderboards/index.md +++ b/aider/website/docs/leaderboards/index.md @@ -71,7 +71,7 @@ The model also has to successfully apply all its changes to the source file with - - - - + + + + + +
Model NameTotal TokensPercent
anthropic/claude-3-7-sonnet-20250219309,63748.8%
openrouter/REDACTED259,57040.9%
fireworks_ai/accounts/fireworks/models/deepseek-v340,2276.3%
o3-mini25,5464.0%
anthropic/claude-3-7-sonnet-20250219219,95860.6%
openrouter/REDACTED110,36430.4%
o3-mini25,6437.1%
deepseek/deepseek-reasoner4,4321.2%
fireworks_ai/accounts/fireworks/models/deepseek-r12,7650.8%
claude-3-7-sonnet-20250219930.0%
{: .note :} diff --git a/aider/website/docs/leaderboards/index.md b/aider/website/docs/leaderboards/index.md index cb2c4ef3b..cd142e261 100644 --- a/aider/website/docs/leaderboards/index.md +++ b/aider/website/docs/leaderboards/index.md @@ -116,6 +116,6 @@ mod_dates = [get_last_modified_date(file) for file in files] latest_mod_date = max(mod_dates) cog.out(f"{latest_mod_date.strftime('%B %d, %Y.')}") ]]]--> -March 06, 2025. +March 07, 2025.

diff --git a/aider/website/index.md b/aider/website/index.md index 30a245114..f63179cd5 100644 --- a/aider/website/index.md +++ b/aider/website/index.md @@ -33,7 +33,7 @@ cog.out(text) Aider lets you pair program with LLMs, to edit code in your local git repository. Start a new project or work with an existing code base. -Aider works best with Claude 3.5 Sonnet, DeepSeek R1 & Chat V3, OpenAI o1, o3-mini & GPT-4o. Aider can [connect to almost any LLM, including local models](https://aider.chat/docs/llms.html). +Aider works best with Claude 3.7 Sonnet, DeepSeek R1 & Chat V3, OpenAI o1, o3-mini & GPT-4o. Aider can [connect to almost any LLM, including local models](https://aider.chat/docs/llms.html).