dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-21 04:44:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

improved over time plot

Browse source
This commit is contained in:
Paul Gauthier 2024-11-20 20:16:25 -08:00
parent c797af020a
commit 8302e9d0dd
4 changed files with 540 additions and 976 deletions
Download patch file Download diff file Expand all files Collapse all files

6
aider/website/_data/edit_leaderboard.yml
View file

@ -1,7 +1,7 @@
- dirname: 2024-05-01-20-05-59--direct-opus-filenames-outside-fence
test_cases: 133
model: claude-3-opus-20240229
released: 2024-02-29
_released: 2024-02-29
edit_format: diff
commit_hash: f4b1797-dirty, f4b1797
pass_rate_1: 53.4
@ -623,7 +623,7 @@
commit_hash: d31eef3-dirty
pass_rate_1: 40.6
pass_rate_2: 55.6
released: 2024-07-18
_released: 2024-07-18
percent_cases_well_formed: 100.0
error_outputs: 1
num_malformed_responses: 0
@ -691,7 +691,7 @@
- dirname: 2024-07-24-06-30-29--llama-405b-whole
test_cases: 133
model: llama-3.1-405b-instruct (whole)
released: 2024-07-23
_released: 2024-07-23
edit_format: whole
commit_hash: a362dea-dirty
pass_rate_1: 48.9

BIN
aider/website/assets/models-over-time.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 155 KiB

After

Width:  |  Height:  |  Size: 99 KiB

Before After
Before After

1496
aider/website/assets/models-over-time.svg
View file

File diff suppressed because it is too large Load diff
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 81 KiB

After

Width:  |  Height:  |  Size: 57 KiB

Before After
Before After

8
benchmark/over_time.py
View file

@ -5,7 +5,7 @@ from matplotlib import rc
from aider.dump import dump # noqa: 401
LABEL_FONT_SIZE = 12 # Font size for scatter plot dot labels
LABEL_FONT_SIZE = 16 # Font size for scatter plot dot labels
def get_legend_label(model):
@ -128,7 +128,7 @@ def plot_over_time(yaml_file):
color_to_first_point = {}
color_to_label = {}
for date, rate, color, model in zip(dates, pass_rates, colors, models):
for date, rate, color, model in sorted(zip(dates, pass_rates, colors, models)):
if color not in color_to_first_point:
color_to_first_point[color] = (date, rate)
color_to_label[color] = get_legend_label(model)
@ -202,7 +202,7 @@ def plot_over_time(yaml_file):
ax.annotate(
color_to_label["brown"],
(date, rate),
xytext=(10, -5),
xytext=(10, -10),
textcoords="offset points",
color="brown",
alpha=0.8,
@ -247,7 +247,7 @@ def plot_over_time(yaml_file):
ax.annotate(
color_to_label["cyan"],
(date, rate),
xytext=(10, 5),
xytext=(10, -10),
textcoords="offset points",
color="cyan",
alpha=0.8,