This commit is contained in:
Paul Gauthier 2024-05-23 08:04:37 -07:00
parent 2a881faac8
commit 2f3baf7cdd
4 changed files with 51 additions and 51 deletions

View file

@ -23,21 +23,21 @@ The best result reported elsewhere seems to be
Aider achieved this result mainly through its focus on static code analysis,
reliable LLM code editing,
and pragmatic workflows for interactive pair programming with AI.
Aider intentionally has quite limited and narrow "agentic behavior":
it doesn't require a highly detailed upfront "spec" from the user,
use RAG or vector search, farm out sub-problems to an army of LLMs,
allow the LLM to use tools,
or perform web searches,
etc.
Aider intentionally has quite limited and narrow "agentic behavior"
to avoid long delays, high token costs
and the need for users to repeatedly code review incorrect solutions.
It's also worth noting that aider currently does not use
RAG, vector search, tools or give the LLM access to execute code
or run web searches.
Aider is first and foremost a tool for engineers to get real work done in
real code bases through a pair programming chat style interface.
When a user asks aider for a change, they see the edits performed in real-time,
and aider may also then offer additional
help like fixing lint or test errors.
In normal use, the user is in full interactive control.
Aider is first and foremost an interactive tool for engineers to get real work done in
real code bases using a chat interface.
Aider provides a pair programming experience where users can ask for a change
and see the edits performed in real-time.
Aider can also offer additional help like fixing lint or test errors,
but the user is always in full interactive control.
This lets them quickly steer misunderstandings back on course and
avoid wasted time, code reviews and token costs.
avoid wasting time and token costs.
## Benchmark methodology

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 36 KiB

Before After
Before After

View file

@ -6,7 +6,7 @@
<rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<cc:Work>
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
<dc:date>2024-05-23T07:38:15.931243</dc:date>
<dc:date>2024-05-23T07:52:54.138893</dc:date>
<dc:format>image/svg+xml</dc:format>
<dc:creator>
<cc:Agent>
@ -41,12 +41,12 @@ z
<g id="xtick_1">
<g id="line2d_1">
<defs>
<path id="m13d95e4709" d="M 0 0
<path id="mae9723abe5" d="M 0 0
L 0 3.5
" style="stroke: #000000; stroke-width: 0.8"/>
</defs>
<g>
<use xlink:href="#m13d95e4709" x="130.142981" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="130.142981" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_1">
@ -453,7 +453,7 @@ z
<g id="xtick_2">
<g id="line2d_2">
<g>
<use xlink:href="#m13d95e4709" x="213.207821" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="213.207821" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_2">
@ -479,7 +479,7 @@ z
<g id="xtick_3">
<g id="line2d_3">
<g>
<use xlink:href="#m13d95e4709" x="296.27266" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="296.27266" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_3">
@ -601,7 +601,7 @@ z
<g id="xtick_4">
<g id="line2d_4">
<g>
<use xlink:href="#m13d95e4709" x="379.3375" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="379.3375" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_4">
@ -674,7 +674,7 @@ z
<g id="xtick_5">
<g id="line2d_5">
<g>
<use xlink:href="#m13d95e4709" x="462.40234" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="462.40234" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_5">
@ -886,7 +886,7 @@ z
<g id="xtick_6">
<g id="line2d_6">
<g>
<use xlink:href="#m13d95e4709" x="545.467179" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="545.467179" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_6">
@ -1007,7 +1007,7 @@ z
<g id="xtick_7">
<g id="line2d_7">
<g>
<use xlink:href="#m13d95e4709" x="628.532019" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#mae9723abe5" x="628.532019" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_7">
@ -1043,16 +1043,16 @@ z
<g id="line2d_8">
<path d="M 68.675 273.70025
L 690 273.70025
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_9">
<defs>
<path id="mb0b2eca59c" d="M 0 0
<path id="m81053b1b2f" d="M 0 0
L -3.5 0
" style="stroke: #000000; stroke-width: 0.8"/>
</defs>
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="273.70025" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_8">
@ -1089,11 +1089,11 @@ z
<g id="line2d_10">
<path d="M 68.675 235.200207
L 690 235.200207
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_11">
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="235.200207" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="235.200207" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_9">
@ -1135,11 +1135,11 @@ z
<g id="line2d_12">
<path d="M 68.675 196.700164
L 690 196.700164
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_13">
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="196.700164" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="196.700164" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_10">
@ -1167,11 +1167,11 @@ z
<g id="line2d_14">
<path d="M 68.675 158.200121
L 690 158.200121
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_15">
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="158.200121" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="158.200121" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_11">
@ -1186,11 +1186,11 @@ L 690 158.200121
<g id="line2d_16">
<path d="M 68.675 119.700078
L 690 119.700078
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_17">
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="119.700078" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="119.700078" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_12">
@ -1232,11 +1232,11 @@ z
<g id="line2d_18">
<path d="M 68.675 81.200034
L 690 81.200034
" clip-path="url(#p535a156c8f)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
" clip-path="url(#pdd89e16c85)" style="fill: none; stroke: #b0b0b0; stroke-width: 0.2; stroke-linecap: square"/>
</g>
<g id="line2d_19">
<g>
<use xlink:href="#mb0b2eca59c" x="68.675" y="81.200034" style="stroke: #000000; stroke-width: 0.8"/>
<use xlink:href="#m81053b1b2f" x="68.675" y="81.200034" style="stroke: #000000; stroke-width: 0.8"/>
</g>
</g>
<g id="text_13">
@ -1411,7 +1411,7 @@ L 163.368917 273.70025
L 163.368917 71.190023
L 96.917045 71.190023
z
" clip-path="url(#p535a156c8f)" style="fill: #b3e6a8; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3e6a8; opacity: 0.75"/>
</g>
<g id="patch_8">
<path d="M 179.981885 273.70025
@ -1419,7 +1419,7 @@ L 246.433757 273.70025
L 246.433757 81.200034
L 179.981885 81.200034
z
" clip-path="url(#p535a156c8f)" style="fill: #b3e6a8; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3e6a8; opacity: 0.75"/>
</g>
<g id="patch_9">
<path d="M 263.046725 273.70025
@ -1427,7 +1427,7 @@ L 329.498596 273.70025
L 329.498596 101.990058
L 263.046725 101.990058
z
" clip-path="url(#p535a156c8f)" style="fill: #b3d1e6; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3d1e6; opacity: 0.75"/>
</g>
<g id="patch_10">
<path d="M 346.111564 273.70025
@ -1435,7 +1435,7 @@ L 412.563436 273.70025
L 412.563436 112.000069
L 346.111564 112.000069
z
" clip-path="url(#p535a156c8f)" style="fill: #b3d1e6; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3d1e6; opacity: 0.75"/>
</g>
<g id="patch_11">
<path d="M 429.176404 273.70025
@ -1443,7 +1443,7 @@ L 495.628275 273.70025
L 495.628275 117.390075
L 429.176404 117.390075
z
" clip-path="url(#p535a156c8f)" style="fill: #b3d1e6; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3d1e6; opacity: 0.75"/>
</g>
<g id="patch_12">
<path d="M 512.241243 273.70025
@ -1451,7 +1451,7 @@ L 578.693115 273.70025
L 578.693115 135.100095
L 512.241243 135.100095
z
" clip-path="url(#p535a156c8f)" style="fill: #b3d1e6; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3d1e6; opacity: 0.75"/>
</g>
<g id="patch_13">
<path d="M 595.306083 273.70025
@ -1459,11 +1459,11 @@ L 661.757955 273.70025
L 661.757955 183.610149
L 595.306083 183.610149
z
" clip-path="url(#p535a156c8f)" style="fill: #b3d1e6; opacity: 0.75"/>
" clip-path="url(#pdd89e16c85)" style="fill: #b3d1e6; opacity: 0.75"/>
</g>
<g id="text_15">
<!-- 26.3% -->
<g style="fill: #555555" transform="translate(110.295794 92.782849) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(110.295794 90.857847) scale(0.14 -0.14)">
<defs>
<path id="Helvetica-36" d="M 1872 4494
Q 2622 4494 2917 4105
@ -1546,7 +1546,7 @@ z
</g>
<g id="text_16">
<!-- 25.0% -->
<g style="fill: #555555" transform="translate(193.360633 102.79286) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(193.360633 100.867858) scale(0.14 -0.14)">
<use xlink:href="#Helvetica-32"/>
<use xlink:href="#Helvetica-35" x="55.615234"/>
<use xlink:href="#Helvetica-2e" x="111.230469"/>
@ -1556,7 +1556,7 @@ z
</g>
<g id="text_17">
<!-- 22.3% -->
<g style="fill: #555555" transform="translate(276.425473 123.582883) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(276.425473 121.657881) scale(0.14 -0.14)">
<use xlink:href="#Helvetica-32"/>
<use xlink:href="#Helvetica-32" x="55.615234"/>
<use xlink:href="#Helvetica-2e" x="111.230469"/>
@ -1566,7 +1566,7 @@ z
</g>
<g id="text_18">
<!-- 21.0% -->
<g style="fill: #555555" transform="translate(359.490313 133.592894) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(359.490313 131.667892) scale(0.14 -0.14)">
<use xlink:href="#Helvetica-32"/>
<use xlink:href="#Helvetica-31" x="55.615234"/>
<use xlink:href="#Helvetica-2e" x="111.230469"/>
@ -1576,7 +1576,7 @@ z
</g>
<g id="text_19">
<!-- 20.3% -->
<g style="fill: #555555" transform="translate(442.555152 138.9829) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(442.555152 137.057898) scale(0.14 -0.14)">
<use xlink:href="#Helvetica-32"/>
<use xlink:href="#Helvetica-30" x="55.615234"/>
<use xlink:href="#Helvetica-2e" x="111.230469"/>
@ -1586,7 +1586,7 @@ z
</g>
<g id="text_20">
<!-- 18.0% -->
<g style="fill: #555555" transform="translate(525.619992 156.69292) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(525.619992 154.767918) scale(0.14 -0.14)">
<defs>
<path id="Helvetica-38" d="M 1741 2600
Q 2113 2600 2322 2808
@ -1637,7 +1637,7 @@ z
</g>
<g id="text_21">
<!-- 11.7% -->
<g style="fill: #555555" transform="translate(608.684831 205.202975) scale(0.14 -0.14)">
<g style="fill: #555555" transform="translate(608.684831 203.277972) scale(0.14 -0.14)">
<defs>
<path id="Helvetica-37" d="M 3347 4400
L 3347 3909
@ -1745,7 +1745,7 @@ z
</g>
</g>
<defs>
<clipPath id="p535a156c8f">
<clipPath id="pdd89e16c85">
<rect x="68.675" y="50.4" width="621.325" height="223.30025"/>
</clipPath>
</defs>

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 43 KiB

Before After
Before After

View file

@ -38,7 +38,7 @@ def plot_swe_bench_lite(data_file):
yval = bar.get_height()
ax.text(
bar.get_x() + bar.get_width() / 2,
yval - 1.5,
yval - 1.25,
f"{yval}%",
ha="center",
va="top",