dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-02 18:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

feat: Add benchmark data table

Browse source
This commit is contained in:
Paul Gauthier (aider) 2024-09-26 07:58:26 -07:00
parent 3602dc4819
commit f21c35c42a
1 changed files with 15 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

15
aider/website/_posts/2024-09-26-senior-junior.md
View file

@ -11,4 +11,19 @@ nav_exclude: true
# Separating code reasoning and editing
Here's a table containing the benchmark data for different model configurations:
| Model | Junior Model | Junior Edit Format | Pass Rate 2 (%) | Seconds per Case | Total Cost ($) |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
| claude-3.5-sonnet | claude-3.5-sonnet | junior-diff | 80.5 | 25.1 | 4.95 |
| o1-mini | gpt-4o | junior-diff | 70.7 | 23.7 | 9.32 |
| gpt-4o | gpt-4o | junior-diff | 75.2 | 18.2 | 6.09 |
| o1-preview | gpt-4o | junior-diff | 80.5 | 42.3 | 39.38 |
| o1-preview | claude-3.5-sonnet | junior-diff | 82.7 | 44.9 | 37.62 |
| o1-mini | deepseek | junior-diff | 69.2 | 52.2 | 5.79 |
| o1-preview | deepseek | junior-diff | 80.5 | 73.2 | 35.79 |
| o1-preview | deepseek | junior-whole | 85.0 | 67.4 | 35.32 |
This table provides a comparison of different model configurations, showing their performance in terms of pass rate, processing time, and cost.