dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-05 04:05:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

copy

Browse source
This commit is contained in:
Paul Gauthier 2024-06-03 11:21:12 -07:00
parent b184ab9977
commit 58bcfc3f6e
2 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
_posts/2024-05-22-swe-bench-lite.md
View file

@ -414,6 +414,7 @@ The "aider agent" internally makes multiple "attempts" at solving the problem,
but it picks and returns one single candidate solution. but it picks and returns one single candidate solution.
Only that one candidate solution is evaluated with the acceptance tests Only that one candidate solution is evaluated with the acceptance tests
and contributes to the benchmark score. and contributes to the benchmark score.
This is contrast to a pass@N result for N>1, where N attempts are made This is contrast to a pass@N result for N>1, where N attempts are made
and all N solutions are evaluated by the acceptance tests. and all N solutions are evaluated by the acceptance tests.
If *any* of the N solution pass, that counts as a pass@N success. If *any* of the N solution pass, that counts as a pass@N success.

1
_posts/2024-06-02-main-swe-bench.md
View file

@ -238,6 +238,7 @@ The "aider agent" internally makes multiple "attempts" at solving the problem,
but it picks and returns one single candidate solution. but it picks and returns one single candidate solution.
Only that one candidate solution is evaluated with the acceptance tests Only that one candidate solution is evaluated with the acceptance tests
and contributes to the benchmark score. and contributes to the benchmark score.
This is contrast to a pass@N result for N>1, where N attempts are made This is contrast to a pass@N result for N>1, where N attempts are made
and all N solutions are evaluated by the acceptance tests. and all N solutions are evaluated by the acceptance tests.
If *any* of the N solution pass, that counts as a pass@N success. If *any* of the N solution pass, that counts as a pass@N success.