added draft article

This commit is contained in:
Paul Gauthier 2024-05-31 09:33:31 -07:00
parent a168daf5fc
commit 0120d434ff
5 changed files with 2433 additions and 2 deletions

7
benchmark/swe-bench.txt Normal file
View file

@ -0,0 +1,7 @@
18.8% Aider|GPT-4o|& Opus|(570)
17.0% Aider|GPT-4o|(570)
13.9% Devin|(570)
13.8% Amazon Q|Developer|Agent|(2294)
12.5% SWE-|Agent|+ GPT-4|(2294)
10.6% AutoCode|Rover|(2294)
10.5% SWE-|Agent|+ Opus|(2294)