From 5e9bf8993d33669c00f0d811273d6f1a488f31fa Mon Sep 17 00:00:00 2001 From: Paul Gauthier Date: Fri, 31 May 2024 09:28:42 -0700 Subject: [PATCH] copy --- _posts/2024-05-22-swe-bench-lite.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_posts/2024-05-22-swe-bench-lite.md b/_posts/2024-05-22-swe-bench-lite.md index dac61a3d4..120119ad5 100644 --- a/_posts/2024-05-22-swe-bench-lite.md +++ b/_posts/2024-05-22-swe-bench-lite.md @@ -415,12 +415,12 @@ displayed in the graph at the beginning of this article. Note, the graph was corrected on 5/30/24 as follows. -The graph now contains AutoCodeRover's pass@1 results. -Previously it displayed the pass@3 results, which are +The graph now contains AutoCodeRover's average pass@1 results. +Previously it displayed pass@3 results, which are not comparable to the pass@1 results for aider being reported here. The [AutoCodeRover GitHub page](https://github.com/nus-apr/auto-code-rover) -features the pass@3 results +features pass@3 results without being clearly labeled. The graph now contains the best OpenDevin results obtained without using