dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-29 08:44:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

copy

Browse source
This commit is contained in:
Paul Gauthier 2024-06-03 12:21:09 -07:00
parent 5d08c69ba0
commit 8a95ce80ae
2 changed files with 0 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

5
_posts/2024-05-22-swe-bench-lite.md
View file

@ -446,8 +446,3 @@ which is not comparable
to the unhinted aider results being reported here.
[OpenDevin reported hinted results](https://x.com/gneubig/status/1791498953709752405)
without noting that hints were used.
The [official SWE Bench Lite leaderboard](https://www.swebench.com)
only accepts pass@1 results that do not use `hints_text`.

5
_posts/2024-06-02-main-swe-bench.md
View file

@ -261,8 +261,3 @@ Table 2 of their
[paper](https://arxiv.org/pdf/2404.05427v2)
reports an `ACR-avg` result of 10.59% which is an average pass@1 result.
The results presented here for aider are all pass@1, as
the [official SWE Bench leaderboard](https://www.swebench.com)
only accepts pass@1 results.