From 0f842652578ed0c34d70cac172ba460bd57b14d0 Mon Sep 17 00:00:00 2001 From: Paul Gauthier Date: Fri, 24 May 2024 15:21:57 -0700 Subject: [PATCH] added refs --- _posts/2024-05-22-swe-bench-lite.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/_posts/2024-05-22-swe-bench-lite.md b/_posts/2024-05-22-swe-bench-lite.md index 228149c75..922fd82d4 100644 --- a/_posts/2024-05-22-swe-bench-lite.md +++ b/_posts/2024-05-22-swe-bench-lite.md @@ -395,3 +395,14 @@ Also thanks to Albert Örwall who has making it faster, easier, and more reliable to run the acceptance tests. +## References + +Below are the references for the SWE-Bench Lite results +displayed in the graph at the top of this page. + +- 22.3% AutoCodeRover https://github.com/nus-apr/auto-code-rover +- 21.0% OpenDevin https://github.com/OpenDevin/OpenDevin +- 20.3% Amazon Q Developer Agent (v20240430-dev) https://www.swebench.com +- 18.0% SWE-Agent + GPT-4 https://www.swebench.com +- 11.7% SWE-Agent + Opus https://www.swebench.com +