dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-03 19:24:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat: Update benchmark data table in senior-junior blog post

Browse source
This commit is contained in:
Paul Gauthier 2024-09-26 08:07:36 -07:00 • committed by Paul Gauthier (aider)
parent 1676653ffb
commit 46f8b6dfe4
1 changed files with 7 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

17
aider/website/_posts/2024-09-26-senior-junior.md
View file

@ -13,19 +13,16 @@ nav_exclude: true
Here's a table containing the benchmark data for different model configurations:
| Senior | Junior | Junior Edit Format | Pass Rate 2 (%) | Seconds per Case | Total Cost ($) |
| Senior | Junior | Junior Edit Format | Pass Rate (%) | Average Time (sec) | Total Cost ($) |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
| claude-3.5-sonnet | claude-3.5-sonnet | junior-diff | 80.5 | 25.1 | 4.95 |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
| gpt-4o | gpt-4o | junior-diff | 75.2 | 18.2 | 6.09 |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
| o1-mini | deepseek | junior-diff | 69.2 | 52.2 | 5.79 |
| o1-mini | gpt-4o | junior-diff | 70.7 | 23.7 | 9.32 |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
| o1-preview | claude-3.5-sonnet | junior-diff | 82.7 | 44.9 | 37.62 |
| o1-preview | deepseek | junior-diff | 80.5 | 73.2 | 35.79 |
| o1-preview | deepseek | junior-whole | 85.0 | 67.4 | 35.32 |
| o1-preview | claude-3.5-sonnet | junior-diff | 82.7 | 44.9 | 37.62 |
| o1-preview | gpt-4o | junior-diff | 80.5 | 42.3 | 39.38 |
| o1-preview | deepseek | junior-diff | 80.5 | 73.2 | 35.79 |
| claude-3.5-sonnet | claude-3.5-sonnet | junior-diff | 80.5 | 25.1 | 4.95 |
| gpt-4o | gpt-4o | junior-diff | 75.2 | 18.2 | 6.09 |
| o1-mini | gpt-4o | junior-diff | 70.7 | 23.7 | 9.32 |
| o1-mini | deepseek | junior-diff | 69.2 | 52.2 | 5.79 |
|-------|--------------|---------------------|-----------------|-------------------|----------------|
This table provides a comparison of different model configurations, showing their performance in terms of pass rate, processing time, and cost.