dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-31 01:35:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

move into website/

Browse source
This commit is contained in:
Paul Gauthier 2024-06-05 14:28:39 -07:00
parent 5a4d38418d
commit 56519361e2
103 changed files with 5 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

41
website/docs/benchmarks-0125.md Normal file
View file

@ -0,0 +1,41 @@
---
title: The January GPT-4 Turbo is lazier than the last version
excerpt: The new `gpt-4-0125-preview` model is quantiatively lazier at coding than previous GPT-4 versions, according to a new "laziness" benchmark.
highlight_image: /assets/benchmarks-0125.jpg
nav_exclude: true
---
# The January GPT-4 Turbo is lazier than the last version
[![benchmark results](/assets/benchmarks-0125.svg)](https://aider.chat/assets/benchmarks-0125.svg)
[OpenAI just released a new version of GPT-4 Turbo](https://openai.com/blog/new-embedding-models-and-api-updates).
This new model is intended to reduce the "laziness" that has been widely observed with the previous `gpt-4-1106-preview` model:
> Today, we are releasing an updated GPT-4 Turbo preview model, gpt-4-0125-preview. This model completes tasks like code generation more thoroughly than the previous preview model and is intended to reduce cases of “laziness” where the model doesn’t complete a task.
With that in mind, I've been benchmarking the new model using
aider's existing
[lazy coding benchmark](https://aider.chat/docs/unified-diffs.html).
## Benchmark results
Overall,
the new `gpt-4-0125-preview` model seems lazier
than the November `gpt-4-1106-preview` model:
- It gets worse benchmark scores when using the [unified diffs](https://aider.chat/docs/unified-diffs.html) code editing format.
- Using aider's older [SEARCH/REPLACE block](https://github.com/paul-gauthier/aider/blob/9033be74bf74ae70459013e54b2ae6a97c47c2e6/aider/coders/editblock_prompts.py#L75-L80) editing format, the new January model outperforms the older November model. But it still performs worse than both models using unified diffs.
## Related reports
This is one in a series of reports
that use the aider benchmarking suite to assess and compare the code
editing capabilities of OpenAI's GPT models.
You can review the other reports
for additional information:
- [GPT code editing benchmarks](https://aider.chat/docs/benchmarks.html) evaluates the March and June versions of GPT-3.5 and GPT-4.
- [Code editing benchmarks for OpenAI's "1106" models](https://aider.chat/docs/benchmarks-1106.html).
- [Aider's lazy coding benchmark](https://aider.chat/docs/unified-diffs.html).

88
website/docs/benchmarks-1106.md Normal file
View file

@ -0,0 +1,88 @@
---
title: Code editing benchmarks for OpenAI's "1106" models
excerpt: A quantitative comparison of the code editing capabilities of the new GPT-3.5 and GPT-4 versions that were released in Nov 2023.
highlight_image: /assets/benchmarks-1106.jpg
nav_exclude: true
---
# Code editing benchmarks for OpenAI's "1106" models
[![benchmark results](/assets/benchmarks-1106.svg)](https://aider.chat/assets/benchmarks-1106.svg)
[![benchmark results](/assets/benchmarks-speed-1106.svg)](https://aider.chat/assets/benchmarks-speed-1106.svg)
[OpenAI just released new versions of GPT-3.5 and GPT-4](https://openai.com/blog/new-models-and-developer-products-announced-at-devday),
and there's a lot
of interest about their ability to code compared to the previous versions.
With that in mind, I've been benchmarking the new models.
[Aider](https://github.com/paul-gauthier/aider)
is an open source command line chat tool that lets you work with GPT to edit
code in your local git repo.
To do this, aider needs to be able to reliably recognize when GPT wants to edit
your source code,
determine which files it wants to modify
and accurately apply the changes it's trying to make.
Doing a good job on this "code editing" task requires a good LLM, good prompting and
a good tool driving the interactions with the LLM.
Aider relies on a
[code editing benchmark](https://aider.chat/docs/benchmarks.html)
to quantitatively evaluate
performance
whenever one of these things changes.
For example,
whenever I change aider's prompting or the backend which drives LLM conversations,
I run the benchmark to make sure these changes produce improvements (not regressions).
The benchmark uses aider to try and complete
[133 Exercism Python coding exercises](https://github.com/exercism/python).
For each exercise, Exercism provides a starting python file with stubs for the needed functions,
a natural language description of the problem to solve
and a test suite to evaluate whether the coder has correctly solved the problem.
The benchmark gives aider two tries to complete the task:
1. On the first try, aider gives GPT the stub code file to edit and the natural language instructions that describe the problem. This reflects how you code with aider. You add your source code files to the chat and ask for changes, which are automatically applied.
2. If the test suite fails after the first try, aider gives GPT the test error output and asks it to fix the code. Aider supports this sort of interaction using a command like `/run pytest` to run and share pytest results in the chat with GPT. You can `/run` whatever tests/linters/etc make sense for your language/framework/situation.
## Benchmark results
### gpt-4-1106-preview
For now, I have only benchmarked the GPT-4 models using the `diff` edit method.
This is the edit format that aider uses by default with gpt-4.
- The new `gpt-4-1106-preview` model seems **2-2.5X faster** than the June GPT-4 model.
- **It seems better at producing correct code on the first try**. It gets
53% of the coding exercises correct, without needing to see errors from the test suite. Previous models only get 46-47% of the exercises correct on the first try.
- The new model seems to perform similar
(~65%) to the old models (63-64%) after their second chance to correct bugs by reviewing test suite error output.
### gpt-3.5-turbo-1106
I benchmarked the GPT-3.5 models with both the `whole` and `diff` edit format.
None of the gpt-3.5 models seem able to effectively use the `diff` edit format, including the newest November (1106) model.
The comments below only focus on comparing the `whole` edit format results:
- The new `gpt-3.5-turbo-1106` model is completing the benchmark **3-4X faster** than the earlier GPT-3.5 models.
- The success rate after the first try of 42% is comparable to the previous June (0613) model. The new November and previous June models are both worse than the original March (0301) model's 50% result on the first try.
- The new model's 56% success rate after the second try seems comparable to the original March model, and somewhat better than the June model's 50% score.
## Related reports
This is one in a series of reports
that use the aider benchmarking suite to assess and compare the code
editing capabilities of OpenAI's GPT models.
You can review the other reports
for additional information:
- [GPT code editing benchmarks](https://aider.chat/docs/benchmarks.html) evaluates the March and June versions of GPT-3.5 and GPT-4.
- [Code editing speed benchmarks for OpenAI's "1106" models](https://aider.chat/2023/11/06/benchmarks-speed-1106.html) compares the performance of the new GPT models.
## Updates
Last updated 11/14/23.
OpenAI has relaxed rate limits so these results are no longer considered preliminary.

55
website/docs/benchmarks-speed-1106.md Normal file
View file

@ -0,0 +1,55 @@
---
title: Speed benchmarks of GPT-4 Turbo and gpt-3.5-turbo-1106
excerpt: This report provides a detailed comparison of the speed of GPT-4 Turbo and gpt-3.5-turbo-1106 models based on the aider benchmarking suite.
canonical_url: https://aider.chat/2023/11/06/benchmarks-speed-1106.html
highlight_image: /assets/benchmarks-speed-1106.jpg
nav_exclude: true
---
# Speed benchmarks of GPT-4 Turbo and gpt-3.5-turbo-1106
<p class="post-date">{{ page.date | date: "%b %-d, %Y" }}</p>
[![benchmark results](/assets/benchmarks-speed-1106.svg)](https://aider.chat/assets/benchmarks-speed-1106.svg)
[OpenAI just released new versions of GPT-3.5 and GPT-4](https://openai.com/blog/new-models-and-developer-products-announced-at-devday),
and there's a lot
of interest about their capabilities and performance.
With that in mind, I've been benchmarking the new models.
[Aider](https://github.com/paul-gauthier/aider)
is an open source command line chat tool that lets you work with GPT to edit
code in your local git repo.
Aider relies on a
[code editing benchmark](https://aider.chat/docs/benchmarks.html)
to quantitatively evaluate
performance.
This is the latest in a series of reports
that use the aider benchmarking suite to assess and compare the code
editing capabilities of OpenAI's GPT models. You can review previous
reports to get more background on aider's benchmark suite:
- [GPT code editing benchmarks](https://aider.chat/docs/benchmarks.html) evaluates the March and June versions of GPT-3.5 and GPT-4.
- [Code editing skill benchmarks for OpenAI's "1106" models](https://aider.chat/docs/benchmarks-1106.html) compares the olders models to the November (1106) models.
## Speed
This report compares the **speed** of the various GPT models.
Aider's benchmark measures the response time of the OpenAI chat completion
endpoint each time it asks GPT to solve a programming exercise in the benchmark
suite. These results measure only the time spent waiting for OpenAI to
respond to the prompt.
So they are measuring
how fast these models can
generate responses which primarily consist of source code.
Some observations:
- **GPT-3.5 got 6-11x faster.** The `gpt-3.5-turbo-1106` model is 6-11x faster than the June (0613) version which has been the default `gpt-3.5-turbo` model.
- **GPT-4 Turbo is 2-2.5x faster.** The new `gpt-4-1106-preview` model is 2-2.5x faster than the June (0613) version which has been the default `gpt-4` model.
- The old March (0301) version of GPT-3.5 is actually faster than the June (0613) version. This was a surprising discovery.
## Updates
Last updated 11/14/23.
OpenAI has relaxed rate limits so these results are no longer considered preliminary.

364
website/docs/benchmarks.md Normal file
View file

@ -0,0 +1,364 @@
---
title: GPT code editing benchmarks
excerpt: Benchmarking GPT-3.5 and GPT-4 code editing skill using a new code editing benchmark suite based on the Exercism python exercises.
highlight_image: /assets/benchmarks.jpg
nav_exclude: true
---
# GPT code editing benchmarks
[![benchmark results](/assets/benchmarks.svg)](https://aider.chat/assets/benchmarks.svg)
Aider is an open source command line chat tool that lets you work with GPT to edit
code in your local git repo.
To do this, aider needs to be able to reliably recognize when GPT wants to edit local files,
determine which files it wants to modify and what changes to save.
Such automated
code editing hinges on using the system prompt
to tell GPT how to structure code edits in its responses.
Aider currently asks GPT to use simple text based "edit formats", but
[OpenAI's new function calling
API](https://openai.com/blog/function-calling-and-other-api-updates)
looks like a promising way to create more structured edit formats.
After implementing a couple of function based edit formats,
I wanted
to measure the potential benefits
of switching aider to use them by default.
With this in mind, I developed a
benchmark based on the [Exercism
python](https://github.com/exercism/python) coding exercises.
This
benchmark evaluates how effectively aider and GPT can translate a
natural language coding request into executable code saved into
files that pass unit tests.
It provides an end-to-end evaluation of not just
GPT's coding ability, but also its capacity to *edit existing code*
and *format those code edits* so that aider can save the
edits to the local source files.
I ran the benchmark
on all the ChatGPT models (except `gpt-4-32k`), using a variety of edit formats.
The results were interesting:
- **Plain text edit formats worked best.** Asking GPT to return an updated copy of the whole file in a standard markdown fenced code block proved to be the most reliable and effective edit format across all GPT-3.5 and GPT-4 models. The results for this `whole` edit format are shown in solid blue in the graph.
- **Function calls performed worse.** Using the new functions API for edits performed worse than the above whole file method, for all the models. GPT-3.5 especially produced inferior code and frequently mangled this output format. This was surprising, as the functions API was introduced to enhance the reliability of structured outputs. The results for these `...-func` edit methods are shown as patterned bars in the graph (both green and blue).
- **The new June GPT-3.5 models did a bit worse than the old June model.** The performance of the new June (`0613`) versions of GPT-3.5 appears to be a bit worse than the February (`0301`) version. This is visible if you look at the "first attempt" markers on the first three solid blue bars and also by comparing the first three solid green `diff` bars.
- **GPT-4 does better than GPT-3.5,** as expected.
The quantitative benchmark results agree with my intuitions
about prompting GPT for complex tasks like coding. It's beneficial to
minimize the "cognitive overhead" of formatting the response, allowing
GPT to concentrate on the coding task at hand.
As a thought experiment, imagine a slack conversation with a junior developer where
you ask them to write the code to add some new feature to your app.
They're going to type the response back to you by hand in the chat.
Should they type out the
code and wrap it in a normal markdown code block?
Or should they type up a properly escaped and
syntactically correct json data structure
that contains the text of the new code?
Using more complex output formats with GPT seems to cause two issues:
- It makes GPT write worse code. Keeping the output format simple seems to allow GPT to devote more attention to the actual coding task.
- It reduces GPT's adherence to the output format, making it more challenging for tools like aider to accurately identify and apply the edits GPT is attempting to make.
I was expecting to start using function call based edits in aider for both GPT-3.5 and GPT-4.
But given these benchmark results, I won't be adopting the functions API
at this time.
I will certainly plan to benchmark functions again with future versions of the models.
More details on the benchmark, edit formats and results are discussed below.
## The benchmark
The benchmark uses
[133 practice exercises from the Exercism python repository](https://github.com/exercism/python/tree/main/exercises/practice).
These
exercises were designed to help individuals learn Python and hone
their coding skills.
Each exercise includes:
- [Instructions](https://github.com/exercism/python/blob/main/exercises/practice/anagram/.docs/instructions.md), provided in markdown files.
- [Stub python code](https://github.com/exercism/python/blob/main/exercises/practice/anagram/anagram.py) in an *implementation file*, specifying the functions or classes that need to be implemented.
- [Unit tests](https://github.com/exercism/python/blob/main/exercises/practice/anagram/anagram_test.py) in a separate python file.
The goal is for GPT to read the instructions, implement the provided function/class skeletons
and pass all the unit tests. The benchmark measures what percentage of
the 133 exercises are completed successfully, causing all the associated unit tests to pass.
To start each exercise, aider sends GPT
the initial contents of the implementation file,
the Exercism instructions
and a final instruction:
```
Use the above instructions to modify the supplied files: <implementation file>
Keep and implement the existing function or class stubs, they will be called from unit tests.
Only use standard python libraries, don't suggest installing any packages.
```
Aider updates the implementation file based on GPT's reply and runs
the unit tests. If all tests pass, the exercise is considered
complete. If some tests fail, aider sends GPT a second message with
the test error output. It only sends the first 50 lines of test errors
to try and avoid exceeding the context window of the smaller models. Aider
also includes this final instruction:
```
See the testing errors above.
The tests are correct.
Fix the code in <implementation file> to resolve the errors.
```
Requiring GPT to fix its first implementation in response to test failures
is another way in which this benchmark stresses code editing skill.
This second chance is also important because it
gives GPT the opportunity to adjust if the
instructions were imprecise with respect to the
specific requirements of the unit tests.
Many of the exercises have multiple paragraphs of instructions,
and most human coders would likely fail some tests on their
first try.
The bars in the graph show the percent of exercises that were completed by
each model and edit format combination. The full bar height represents
the final outcome following both coding attempts.
Each bar also has a horizontal mark that shows
the intermediate performance after the first coding attempt,
without the benefit of the second try that includes the test error output.
It's worth noting that GPT never gets to see the source code of the
unit tests during the benchmark. It only sees the error output from
failed tests. Of course, all of this code was probably part of its
original training data!
In summary, passing an exercise means GPT was able to:
- Write the required code (possibly after reviewing test error output),
- Correctly package all of the code edits into the edit format so that aider can process and save it to the implementation file.
Conversely, failing an exercise only requires a breakdown in one of
those steps. In practice, GPT fails at different steps in different
exercises. Sometimes it simply writes the wrong code. Other times, it
fails to format the code edits in a way that conforms to the edit
format, resulting in the code not being saved correctly.
It's worth keeping in mind that changing the edit format often affects
both aspects of GPT's performance.
Complex edit formats often lead GPT to write worse code *and* make it less
successful at formatting the edits correctly.
## Edit formats
I benchmarked 4 different edit formats, described below.
Each description includes a sample response that GPT might provide to a user who
requests:
"Change the print from hello to goodbye."
### whole
The
[whole](https://github.com/paul-gauthier/aider/blob/main/aider/coders/wholefile_prompts.py)
format asks GPT to return an updated copy of the entire file, including any changes.
The file should be
formatted with normal markdown triple-backtick fences, inlined with the rest of its response text.
This format is very similar to how ChatGPT returns code snippets during normal chats, except with the addition of a filename right before the opening triple-backticks.
````
Here is the updated copy of your file demo.py:
demo.py
```python
def main():
print("goodbye")
```
````
### diff
The [diff](https://github.com/paul-gauthier/aider/blob/main/aider/coders/editblock_prompts.py)
format also asks GPT to return edits as part of the normal response text,
in a simple diff format.
Each edit is a fenced code block that
specifies the filename and a chunk of ORIGINAL and UPDATED code.
GPT provides some original lines from the file and then a new updated set of lines.
````
Here are the changes you requested to demo.py:
```python
demo.py
<<<<<<< ORIGINAL
print("hello")
=======
print("goodbye")
>>>>>>> UPDATED
```
````
### whole-func
The [whole-func](https://github.com/paul-gauthier/aider/blob/main/aider/coders/wholefile_func_coder.py)
format requests updated copies of whole files to be returned using the function call API.
```
{
"explanation": "Changed hello to goodbye.",
"files": [
{
"path": "demo.py",
"content": "def main():\n print(\"goodbye\")\n"
}
}
```
### diff-func
The
[diff-func](https://github.com/paul-gauthier/aider/blob/main/aider/coders/editblock_func_coder.py)
format requests a list of
original/updated style edits to be returned using the function call API.
```
{
"explanation": "Changed hello to goodbye.",
"edits": [
{
"path": "demo.py",
"original_lines": [
" print(\"hello\")"
],
"updated_lines": [
" print(\"goodbye\")"
],
}
]
}
```
## GPT-3.5's performance
### The `0613` models seem worse?
The GPT-3.5 benchmark results have me fairly convinced that the new
`gpt-3.5-turbo-0613` and `gpt-3.5-16k-0613` models
are a bit worse at code editing than
the older `gpt-3.5-turbo-0301` model.
This is visible in the "first attempt"
portion of each result, before GPT gets a second chance to edit the code.
Look at the horizontal white line in the middle of the first three blue bars.
Performance with the `whole` edit format was 46% for the
February model and only 39% for the June models.
But also note how much the solid green `diff` bars
degrade between the February and June GPT-3.5 models.
They drop from 30% down to about 19%.
I saw other signs of this degraded performance
in earlier versions of the
benchmark as well.
### Pathological use of `diff`
When GPT-3.5 is able to correctly generate the `diff` edit format,
it often uses it in a pathological manner. It places the *entire*
original source file in the ORIGINAL block and the entire updated file
in the UPDATED block. This is strictly worse than just using the
`whole` edit format, as GPT is sending two full copies of the file.
### Hallucinated function calls
When GPT-3.5 uses the functions API
it is prone to ignoring the JSON Schema that specifies valid functions.
It often returns a completely novel and semantically
invalid `function_call` fragment with `"name": "python"`.
The `arguments` attribute is supposed to be a set of key/value pairs
with the arguments to the function specified in the `name` field.
Instead, GPT-3.5 frequently just stuffs an entire python
file into that field.
```
"function_call": {
"name": "python",
"arguments": "def main():\n print(\"hello\")\n"
},
```
It seems like it might be getting confused by fine-tuning that was
done for the ChatGPT code interpreter plugin?
## Randomness
The benchmark attempts to be deterministic, always sending identical
requests for each exercise on repeated runs.
As part of this effort,
when sending test error output to GPT,
it removes the wall-clock timing information that
is normally included by the `unittest` module.
The benchmark harness also logs SHA hashes of
all the OpenAI API requests and replies.
This makes it possible to
detect randomness or nondeterminism
in the benchmarking process.
It turns out that the OpenAI chat APIs are not deterministic, even at
`temperature=0`. The same identical request will produce multiple
distinct responses, usually less than 5-10 variations. This suggests
that OpenAI may be load balancing their API across a number of
slightly different instances of the model?
For certain exercises, some of these variable responses pass the unit tests while
other variants do not. Results for exercises like this, which are
"on the bubble",
are therefore a bit random, depending on which variant OpenAI returns.
Given that, it would be ideal to run all 133 exercises many times for each
model/edit-format combination and report an average performance.
This would average away the effect of the API variance.
It would also significantly increase the cost of this sort of benchmarking.
So I didn't do that.
Benchmarking against 133 exercises already provides some robustness, since
we are measuring the performance across many exercises.
But to get a sense of how much the API variance impacts the benchmark outcomes,
I ran all 133 exercises 10 times each
against `gpt-3.5-turbo-0613` with the `whole` edit format.
You'll see one set of error bars in the graph, which show
the range of results from those 10 runs.
The OpenAI API randomness doesn't seem to
cause a large variance in the overall benchmark results.
## Conclusions
Based on these benchmark results, aider will continue to use
the `whole` edit format for GPT-3.5, and `diff` for GPT-4.
GPT-4 gets comparable results with the `whole` and `diff` edit formats,
but using `whole` significantly increases costs and latency compared to `diff`.
The latency of streaming back the entire updated copy of each edited file
is a real challenge with the `whole` format.
The GPT-3.5 models are quite responsive, and can
stream back entire files at reasonable speed.
Aider displays a progress bar and
live diffs of the files as they stream in,
which helps pass the time.
The GPT-4 models are much slower, and waiting for even small files
to be completely "retyped" on each request is probably unacceptable.

1
website/docs/browser.md Symbolic link
View file

@ -0,0 +1 @@
../_posts/2024-05-02-browser.md

53
website/docs/commands.md Normal file
View file

@ -0,0 +1,53 @@
---
nav_order: 50
---
# In-chat commands
## In-chat commands
Aider supports commands from within the chat, which all start with `/`.
Here are some of the most useful in-chat commands:
- `/help`: Show help about all commands
- `/exit`: Exit the application
## Context management
- `/add <file>`: Add matching files to the chat session using glob patterns
- `/drop <file>`: Remove matching files from the chat session
- `/web <url>`: Add the contents of a webpage to the chat
- `/clear`: Clear the chat history
- `/ls`: List all known files and those included in the chat session
- `/tokens`: Report on the number of tokens used by the current chat context
## Git
- `/undo`: Undo the last git commit if it was done by aider
- `/diff`: Display the diff of the last aider commit
- `/commit <message>`: Commit edits to the repo made outside the chat (commit message optional)
- `/git <command>`: Run a git command
## Other
- `/run <command>`: Run a shell command and optionally add the output to the chat, or use `!<command>`.
- `/test <command>`: Like run, but automatically adds the output to the chat on non-zero exit codes.
- `/voice`: Speak to aider to [request code changes with your voice](https://aider.chat/docs/voice.html).
# Keybindings
The interactive prompt is built with [prompt-toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) which provides a lot of Emacs and Vi-style keyboard. Some emacs bindings you may find useful are
- `Ctrl-A` : Move cursor to the start of the line.
- `Ctrl-B` : Move cursor back one character.
- `Ctrl-D` : Delete the character under the cursor.
- `Ctrl-E` : Move cursor to the end of the line.
- `Ctrl-F` : Move cursor forward one character.
- `Ctrl-K` : Delete from the cursor to the end of the line.
- `Ctrl-L` : Clear the screen.
- `Ctrl-N` : Move down to the next history entry.
- `Ctrl-P` : Move up to the previous history entry.
- `Ctrl-R` : Reverse search in command history.
Note: aider currently exits vi normal mode after a single command, (maybe something to do with the esc keybinding?).
Feel free to investigate and make a PR if you would like to see it fully supported.

85
website/docs/conventions.md Normal file
View file

@ -0,0 +1,85 @@
---
nav_order: 100
---
# Specifying coding conventions
Sometimes you want GPT to be aware of certain coding guidelines,
like whether to provide type hints, which libraries or packages
to prefer, etc.
The easiest way to do that with aider is to simply create
a small markdown file and include it in the chat.
For example, say we want our python code to:
```
- Prefer httpx over requests for making http requests.
- Use types everywhere possible.
```
We would simply create a file like `CONVENTIONS.md` with those lines
and then we can add it to the aider chat, along with the file(s)
that we want to edit:
<div class="chat-transcript" markdown="1">
> $ aider CONVENTIONS.md useragent.py
>
> Aider v0.24.2-dev
> Added CONVENTIONS.md to the chat.
> Added useragent.py to the chat.
#### write a function that uses httpbin to check the user agent on the request. return the ua.
To fulfill your request, I will:
1. Add an import for `httpx`.
2. Write a function named `check_user_agent` that makes a GET request to `httpbin.org/user-agent` and returns the user agent string.
Here is the diff for the `useragent.py` file:
```diff
--- useragent.py
+++ useragent.py
@@ ... @@
+import httpx
+
+def check_user_agent() -> str:
+ response = httpx.get('https://httpbin.org/user-agent')
+ return response.json()['user-agent']
```
This function uses `httpx` to make a GET request to `httpbin.org/user-agent`, which returns a JSON response containing the user agent of the HTTP client. The function then extracts the user agent string from the JSON response and returns it.
> Applied edit to useragent.py
</div>
GPT correctly used `httpx` and provided type hints.
See below for the transcript of this same
coding task conducted *without*
using `CONVENTIONS.md`.
In this case, GPT used `requests` and skipped types,
which is perhaps more typical in small python scripts.
<div class="chat-transcript" markdown="1">
> $ aider useragent.py
>
> Aider v0.24.2-dev
> Added useragent.py to the chat.
#### write a function that uses httpbin to check the user agent on the request. return the ua.
```diff
--- /dev/null
+++ useragent.py
@@ ... @@
+import requests
+
+def check_user_agent():
+ response = requests.get('https://httpbin.org/user-agent')
+ user_agent = response.json().get('user-agent')
+ return user_agent
```
> Applied edit to useragent.py
</div>

246
website/docs/ctags.md Normal file
View file

@ -0,0 +1,246 @@
---
title: Improving GPT-4's codebase understanding with ctags
excerpt: Using ctags to build a "repository map" to increase GPT-4's ability to understand a large code base.
highlight_image: /assets/robot-flowchart.png
nav_exclude: true
---
# Improving GPT-4's codebase understanding with ctags
![robot flowchat](/assets/robot-flowchart.png)
## Updated
Aider no longer uses ctags to build a repo map.
Please see the newer article about
[using tree-sitter to build a better repo map](https://aider.chat/docs/repomap.html).
-------
GPT-4 is extremely useful for "self-contained" coding tasks,
like generating brand new code or modifying a pure function
that has no dependencies.
But it's difficult to use GPT-4 to modify or extend
a large, complex pre-existing codebase.
To modify such code, GPT needs to understand the dependencies and APIs
which interconnect its subsystems.
Somehow we need to provide this "code context" to GPT
when we ask it to accomplish a coding task. Specifically, we need to:
- Help GPT understand the overall codebase, so that it
can decifer the meaning of code with complex dependencies and generate
new code that respects and utilizes existing abstractions.
- Convey all of this "code context" to GPT in an
efficient manner that fits within the 8k-token context window.
To address these issues, `aider` now
sends GPT a **concise map of your whole git repository**
that includes
all declared variables and functions with call signatures.
This *repo map* is built automatically using `ctags`, which
extracts symbol definitions from source files. Historically,
ctags were generated and indexed by IDEs and editors to
help humans search and navigate large codebases.
Instead, we're going to use ctags to help GPT better comprehend, navigate
and edit code in larger repos.
To get a sense of how effective this can be, this
[chat transcript](https://aider.chat/examples/add-test.html)
shows GPT-4 creating a black box test case, **without being given
access to the source code of the function being tested or any of the
other code in the repo.**
Using only the meta-data in the repo map, GPT is able to figure out how to
call the method to be tested, as well as how to instantiate multiple
class objects that are required to prepare for the test.
To code with GPT-4 using the techniques discussed here:
- Install [aider](https://aider.chat/docs/install.html).
- Install [universal ctags](https://aider.chat/docs/install.html#install-universal-ctags-optional).
- Run `aider` inside your repo, and it should say "Repo-map: universal-ctags using 1024 tokens".
## The problem: code context
GPT-4 is great at "self contained" coding tasks, like writing or
modifying a pure function with no external dependencies.
GPT can easily handle requests like "write a
Fibonacci function" or "rewrite the loop using list
comprehensions", because they require no context beyond the code
being discussed.
Most real code is not pure and self-contained, it is intertwined with
and depends on code from many different files in a repo.
If you ask GPT to "switch all the print statements in class Foo to
use the BarLog logging system", it needs to see the code in the Foo class
with the prints, and it also needs to understand the project's BarLog
subsystem.
A simple solution is to **send the entire codebase** to GPT along with
each change request. Now GPT has all the context! But this won't work
for even moderately
sized repos, because they won't fit into the 8k-token context window.
A better approach is to be selective,
and **hand pick which files to send**.
For the example above, you could send the file that
contains the Foo class
and the file that contains the BarLog logging subsystem.
This works pretty well, and is supported by `aider` -- you
can manually specify which files to "add to the chat" you are having with GPT.
But it's not ideal to have to manually identify the right
set of files to add to the chat.
And sending whole files is a bulky way to send code context,
wasting the precious 8k context window.
GPT doesn't need to see the entire implementation of BarLog,
it just needs to understand it well enough to use it.
You may quickly run out of context window if you
send many files worth of code just to convey context.
## Using a repo map to provide context
The latest version of `aider` sends a **repo map** to GPT along with
each change request. The map contains a list of all the files in the
repo, along with the symbols which are defined in each file. Callables
like functions and methods also include their signatures.
Here's a
sample of the map of the aider repo, just showing the maps of
[main.py](https://github.com/paul-gauthier/aider/blob/main/aider/main.py)
and
[io.py](https://github.com/paul-gauthier/aider/blob/main/aider/io.py)
:
```
aider/
...
main.py:
function
main (args=None, input=None, output=None)
variable
status
...
io.py:
class
FileContentCompleter
InputOutput
FileContentCompleter
member
__init__ (self, fnames, commands)
get_completions (self, document, complete_event)
InputOutput
member
__init__ (self, pretty, yes, input_history_file=None, chat_history_file=None, input=None, output=None)
ai_output (self, content)
append_chat_history (self, text, linebreak=False, blockquote=False)
confirm_ask (self, question, default="y")
get_input (self, fnames, commands)
prompt_ask (self, question, default=None)
tool (self, *messages, log_only=False)
tool_error (self, message)
...
```
Mapping out the repo like this provides some benefits:
- GPT can see variables, classes, methods and function signatures from everywhere in the repo. This alone may give it enough context to solve many tasks. For example, it can probably figure out how to use the API exported from a module just based on the details shown in the map.
- If it needs to see more code, GPT can use the map to figure out by itself which files it needs to look at. GPT will then ask to see these specific files, and `aider` will automatically add them to the chat context (with user approval).
Of course, for large repositories even just the map might be too large
for the context window. However, this mapping approach opens up the
ability to collaborate with GPT-4 on larger codebases than previous
methods. It also reduces the need to manually curate which files to
add to the chat context, empowering GPT to autonomously identify
relevant files for the task at hand.
## Using ctags to make the map
Under the hood, `aider` uses
[universal ctags](https://github.com/universal-ctags/ctags)
to build the
map. Universal ctags can scan source code written in many
languages, and extract data about all the symbols defined in each
file.
Historically, ctags were generated and indexed by IDEs or code editors
to make it easier for a human to search and navigate a
codebase, find the implementation of functions, etc.
Instead, we're going to use ctags to help GPT navigate and understand the codebase.
Here is the type of output you get when you run ctags on source code. Specifically,
this is the
`ctags --fields=+S --output-format=json` output for the `main.py` file mapped above:
```json
{
"_type": "tag",
"name": "main",
"path": "aider/main.py",
"pattern": "/^def main(args=None, input=None, output=None):$/",
"kind": "function",
"signature": "(args=None, input=None, output=None)"
}
{
"_type": "tag",
"name": "status",
"path": "aider/main.py",
"pattern": "/^ status = main()$/",
"kind": "variable"
}
```
The repo map is built using this type of `ctags` data,
but formatted into the space
efficient hierarchical tree format shown earlier.
This is a format that GPT can easily understand
and which conveys the map data using a
minimal number of tokens.
## Example chat transcript
This
[chat transcript](https://aider.chat/examples/add-test.html)
shows GPT-4 creating a black box test case, **without being given
access to the source code of the function being tested or any of the
other code in the repo.** Instead, GPT is operating solely off
the repo map.
Using only the meta-data in the map, GPT is able to figure out how to call the method to be tested, as well as how to instantiate multiple class objects that are required to prepare for the test.
GPT makes one reasonable mistake writing the first version of the test, but is
able to quickly fix the issue after being shown the `pytest` error output.
## Future work
Just as "send the whole codebase to GPT with every request"
is not an efficient solution to this problem,
there are probably better approaches than
"send the whole repo map with every request".
Sending an appropriate subset of the repo map would help `aider` work
better with even larger repositories which have large maps.
Some possible approaches to reducing the amount of map data are:
- Distill the global map, to prioritize important symbols and discard "internal" or otherwise less globally relevant identifiers. Possibly enlist `gpt-3.5-turbo` to perform this distillation in a flexible and language agnostic way.
- Provide a mechanism for GPT to start with a distilled subset of the global map, and let it ask to see more detail about subtrees or keywords that it feels are relevant to the current coding task.
- Attempt to analyize the natural language coding task given by the user and predict which subset of the repo map is relevant. Possibly by analysis of prior coding chats within the specific repo. Work on certain files or types of features may require certain somewhat predictable context from elsewhere in the repo. Vector and keyword search against the chat history, repo map or codebase may help here.
One key goal is to prefer solutions which are language agnostic or
which can be easily deployed against most popular code languages.
The `ctags` solution has this benefit, since it comes pre-built
with support for most popular languages.
I suspect that Language Server Protocol might be an even
better tool than `ctags` for this problem.
But it is more cumbersome to deploy for a broad
array of languages.
Users would need to stand up an LSP server for their
specific language(s) of interest.
## Try it out
To use this experimental repo map feature:
- Install [aider](https://aider.chat/docs/install.html).
- Install [universal ctags](https://aider.chat/docs/install.html#install-universal-ctags-optional).
- Run `aider` inside your repo, and it should say "Repo-map: universal-ctags using 1024 tokens".

35
website/docs/docker.md Normal file
View file

@ -0,0 +1,35 @@
---
nav_order: 100
---
# Run aider with docker
You can run aider via docker without doing any local installation, like this:
```
docker pull paulgauthier/aider
docker run -it --volume $(pwd):/app paulgauthier/aider --openai-api-key $OPENAI_API_KEY [...other aider args...]
```
You should run the above commands from the root of your git repo,
since the `--volume` arg maps your current directory into the
docker container.
Given that, you need to be in the root of your git repo for aider to be able to
see the repo and all its files.
You should be sure your that
git repo config contains your user name and email, since the
docker container won't have your global git config.
Run these commands while in your git repo, before
you do the `docker run` command:
```
git config user.email "you@example.com"
git config user.name "Your Name"
```
## Limitations
- When you use the in-chat `/run` command, it will be running shell commands *inside the docker container*. So those commands won't be running in your local environment, which may make it tricky to `/run` tests, etc for your project.
- The `/voice` command won't work unless you can figure out how to give the docker container access to your host audio device. The container has libportaudio2 installed, so it should work if you can do that.

312
website/docs/faq.md Normal file
View file

@ -0,0 +1,312 @@
---
nav_order: 60
---
# Frequently asked questions
{: .no_toc }
- TOC
{:toc}
## How does aider use git?
Aider works best with code that is part of a git repo.
Aider is tightly integrated with git, which makes it easy to:
- Use git to undo any aider changes that you don't like
- Go back in the git history to review the changes that aider made to your code
- Manage a series of aider's changes on a git branch
Aider specifically uses git in these ways:
- It asks to create a git repo if you launch it in a directory without one.
- Whenever aider edits a file, it commits those changes with a descriptive commit message. This makes it easy to undo or review aider's changes.
- Aider takes special care before editing files that already have uncommitted changes (dirty files). Aider will first commit any preexisting changes with a descriptive commit message. This keeps your edits separate from aider's edits, and makes sure you never lose your work if aider makes an inappropriate change.
Aider also allows you to use in-chat commands to `/diff` or `/undo` the last change.
To do more complex management of your git history, you cat use raw `git` commands,
either by using `/git` within the chat, or with standard git tools outside of aider.
While it is not recommended, you can disable aider's use of git in a few ways:
- `--no-auto-commits` will stop aider from git committing each of its changes.
- `--no-dirty-commits` will stop aider from committing dirty files before applying its edits.
- `--no-git` will completely stop aider from using git on your files. You should ensure you are keeping sensible backups of the files you are working with.
## Can I run aider in Google Colab?
User [imabutahersiddik](https://github.com/imabutahersiddik)
has provided this
[Colab notebook](https://colab.research.google.com/drive/1J9XynhrCqekPL5PR6olHP6eE--rnnjS9?usp=sharing).
## How can I run aider locally from source code?
To run the project locally, follow these steps:
```
# Clone the repository:
git clone git@github.com:paul-gauthier/aider.git
# Navigate to the project directory:
cd aider
# Install the dependencies listed in the `requirements.txt` file:
pip install -r requirements.txt
# Run the local version of Aider:
python -m aider.main
```
## Can I script aider?
You can script aider via the command line or python.
## Command line
Aider takes a `--message` argument, where you can give it a natural language instruction.
It will do that one thing, apply the edits to the files and then exit.
So you could do:
```bash
aider --message "make a script that prints hello" hello.js
```
Or you can write simple shell scripts to apply the same instruction to many files:
```bash
for FILE in *.py ; do
aider --message "add descriptive docstrings to all the functions" $FILE
done
```
User `aider --help` to see all the command line options, but these are useful for scripting:
```
--stream, --no-stream
Enable/disable streaming responses (default: True) [env var:
AIDER_STREAM]
--message COMMAND, --msg COMMAND, -m COMMAND
Specify a single message to send GPT, process reply then exit
(disables chat mode) [env var: AIDER_MESSAGE]
--message-file MESSAGE_FILE, -f MESSAGE_FILE
Specify a file containing the message to send GPT, process reply,
then exit (disables chat mode) [env var: AIDER_MESSAGE_FILE]
--yes Always say yes to every confirmation [env var: AIDER_YES]
--auto-commits, --no-auto-commits
Enable/disable auto commit of GPT changes (default: True) [env var:
AIDER_AUTO_COMMITS]
--dirty-commits, --no-dirty-commits
Enable/disable commits when repo is found dirty (default: True) [env
var: AIDER_DIRTY_COMMITS]
--dry-run, --no-dry-run
Perform a dry run without modifying files (default: False) [env var:
AIDER_DRY_RUN]
--commit Commit all pending changes with a suitable commit message, then exit
[env var: AIDER_COMMIT]
```
## Python
You can also script aider from python:
```python
from aider.coders import Coder
from aider.models import Model
# This is a list of files to add to the chat
fnames = ["greeting.py"]
model = Model("gpt-4-turbo", weak_model="gpt-3.5-turbo")
# Create a coder object
coder = Coder.create(main_model=model, fnames=fnames)
# This will execute one instruction on those files and then return
coder.run("make a script that prints hello world")
# Send another instruction
coder.run("make it say goodbye")
```
See the
[Coder.create() and Coder.__init__() methods](https://github.com/paul-gauthier/aider/blob/main/aider/coders/base_coder.py)
for all the supported arguments.
It can also be helpful to set the equivalend of `--yes` by doing this:
```
from aider.io import InputOutput
io = InputOutput(yes=True)
# ...
coder = Coder.create(client=client, fnames=fnames, io=io)
```
## What code languages does aider support?
Aider supports pretty much all the popular coding languages.
This is partly because top LLMs are fluent in most mainstream languages,
and familiar with popular libraries, packages and frameworks.
In fact, coding with aider is sometimes the most magical
when you're working in a language that you
are less familiar with.
the LLM often knows the language better than you,
and can generate all the boilerplate to get to the heart of your
problem.
The LLM will often solve your problem in an elegant way
using a library or package that you weren't even aware of.
Aider uses tree-sitter to do code analysis and help
the LLM navigate larger code bases by producing
a [repository map](https://aider.chat/docs/repomap.html).
Aider can currently produce repository maps for most mainstream languages, listed below.
But aider should work quite well for other languages, even without repo map support.
- C
- C#
- C++
- Emacs Lisp
- Elixir
- Elm
- Go
- Java
- Javascript
- OCaml
- PHP
- Python
- QL
- Ruby
- Rust
- Typescript
## How to use pipx to avoid python package conflicts?
If you are using aider to work on a python project, sometimes your project will require
specific versions of python packages which conflict with the versions that aider
requires.
If this happens, the `pip install` command may return errors like these:
```
aider-chat 0.23.0 requires somepackage==X.Y.Z, but you have somepackage U.W.V which is incompatible.
```
You can avoid this problem by installing aider using `pipx`,
which will install it globally on your system
within its own python environment.
This way you can use aider to work on any python project,
even if that project has conflicting dependencies.
Install [pipx](https://pipx.pypa.io/stable/) then just do:
```
pipx install aider-chat
```
## Aider isn't editing my files?
Sometimes the LLM will reply with some code changes that don't get applied to your local files.
In these cases, aider might say something like "Failed to apply edit to *filename*".
This usually happens because the LLM is not specifying the edits
to make in the format that aider expects.
GPT-3.5 is especially prone to disobeying the system prompt instructions in this manner, but it also happens with stronger models.
Aider makes every effort to get the LLM
to conform, and works hard to deal with
replies that are "almost" correctly formatted.
If Aider detects an improperly formatted reply, it gives
the LLM feedback to try again.
Also, before each release new versions of aider are
[benchmarked](https://aider.chat/docs/benchmarks.html).
This helps prevent regressions in the code editing
performance of an LLM that could have been inadvertantly
introduced.
But sometimes the LLM just won't cooperate.
In these cases, here are some things you might try:
- Use `/drop` to remove files from the chat session which aren't needed for the task at hand. This will reduce distractions and may help GPT produce properly formatted edits.
- Use `/clear` to remove the conversation history, again to help GPT focus.
- Try the a different LLM.
## How can I add ALL the files to the chat?
People regularly ask about how to add **many or all of their repo's files** to the chat.
This is probably not a good idea and will likely do more harm than good.
The best approach is think about which files need to be changed to accomplish
the task you are working on. Just add those files to the chat.
Usually when people want to add "all the files" it's because they think it
will give the LLM helpful context about the overall code base.
Aider will automatically give the LLM a bunch of additional context about
the rest of your git repo.
It does this by analyzing your entire codebase in light of the
current chat to build a compact
[repository map](https://aider.chat/2023/10/22/repomap.html).
Adding a bunch of files that are mostly irrelevant to the
task at hand will often distract or confuse the LLM.
The LLM will give worse coding results, and sometimese even fail to correctly edit files.
Addings extra files will also increase the token costs on your OpenAI invoice.
Again, it's usually best to just add the files to the chat that will need to be modified.
If you still wish to add lots of files to the chat, you can:
- Use a wildcard when you launch aider: `aider src/*.py`
- Use a wildcard with the in-chat `/add` command: `/add src/*.py`
- Give the `/add` command a directory name and it will recurisvely add every file under that dir: `/add src`
## Can I specify guidelines or conventions?
Sometimes you want the LLM to be aware of certain coding guidelines,
like whether to provide type hints, which libraries or packages
to prefer, etc.
Just put any extra instructions in a file
like `CONVENTIONS.md` and then add it to the chat.
For more details, see this documentation on
[using a conventions file with aider](https://aider.chat/docs/conventions.html).
## Can I change the system prompts that aider uses?
Aider is set up to support different system prompts and edit formats
in a modular way. If you look in the `aider/coders` subdirectory, you'll
see there's a base coder with base prompts, and then there are
a number of
different specific coder implementations.
If you're thinking about experimenting with system prompts
this document about
[benchmarking GPT-3.5 and GPT-4 on code editing](https://aider.chat/docs/benchmarks.html)
might be useful background.
While it's not well documented how to add new coder subsystems, you may be able
to modify an existing implementation or use it as a template to add another.
To get started, try looking at and modifying these files.
The wholefile coder is currently used by GPT-3.5 by default. You can manually select it with `--edit-format whole`.
- wholefile_coder.py
- wholefile_prompts.py
The editblock coder is currently used by GPT-4o by default. You can manually select it with `--edit-format diff`.
- editblock_coder.py
- editblock_prompts.py
The universal diff coder is currently used by GPT-4 Turbo by default. You can manually select it with `--edit-format udiff`.
- udiff_coder.py
- udiff_prompts.py
When experimenting with coder backends, it helps to run aider with `--verbose --no-pretty` so you can see
all the raw information being sent to/from the LLM in the conversation.
You can also refer to the
[instructions for installing a development version of aider](https://aider.chat/docs/install.html#install-development-versions-of-aider-optional).

186
website/docs/install.md Normal file
View file

@ -0,0 +1,186 @@
---
title: Installation
nav_order: 20
---
# Installing aider
{: .no_toc }
- TOC
{:toc}
## Quick start
```
$ pip install aider-chat
# To work with GPT-4o
$ export OPENAI_API_KEY=your-key-goes-here
$ aider
# To work with Claude 3 Opus:
$ export ANTHROPIC_API_KEY=your-key-goes-here
$ aider --opus
```
Or read on for more detailed installation instructions.
## Install git
Make sure you have git installed.
Here are
[instructions for installing git in various environments](https://github.com/git-guides/install-git).
## Get your API key
To work with OpenAI's GPT 3.5 or GPT-4 models you need a paid
[OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key).
Note that this is different than being a "ChatGPT Plus" subscriber.
To work with Anthropic's models like Claude 3 Opus you need a paid
[Anthropic API key](https://docs.anthropic.com/claude/reference/getting-started-with-the-api).
## Windows install
```
# Install aider
py -m pip install aider-chat
# To work with GPT-4o:
$ aider --openai-api-key sk-xxx...
# To work with Claude 3 Opus:
$ aider --anthropic-api-key sk-xxx... --opus
```
## Mac/Linux install
```
# Install aider
python -m pip install aider-chat
# To work with GPT-4o:
$ aider --openai-api-key sk-xxx...
# To work with Claude 3 Opus:
$ aider --anthropic-api-key sk-xxx... --opus
```
## Working with other LLMs
Aider works well with GPT-4o, Claude 3 Opus, GPT-3.5 and supports [connecting to almost any LLM](https://aider.chat/docs/llms.html).
## You are done!
See the [usage instructions](https://aider.chat/docs/usage.html) to start coding with aider.
---
# Optional steps
The steps below are completely optional.
- [Store your API key](#store-your-api-key-optional)
- [Enable Playwright](#enable-playwright-optional) to enhance the `/web <url>` command.
- [Enable voice coding](#enable-voice-coding-optional)
- [Add aider to your editor](#add-aider-to-your-editor-optional)
- [Install development versions of aider](#install-development-versions-of-aider-optional)
## Store your api key (optional)
{: .no_toc }
You can place your api key in an environment variable:
* `export OPENAI_API_KEY=sk-...` on Linux or Mac
* `setx OPENAI_API_KEY sk-...` in Windows PowerShell
Or you can create a `.aider.conf.yml` file in your home directory.
Put a line in it like this to specify your api key:
```
openai-api-key: sk-...
```
## Enable Playwright (optional)
{: .no_toc }
Aider supports adding web pages to the chat with the `/web <url>` command.
When you add a url to the chat, aider fetches the page and scrapes its
content.
By default, aider uses the `httpx` library to scrape web pages, but this only
works on a subset of web pages.
Some sites explicitly block requests from tools like httpx.
Others rely heavily on javascript to render the page content,
which isn't possible using only httpx.
Aider works best with all web pages if you install
Playwright's chromium browser and its dependencies:
```
playwright install --with-deps chromium
```
See the
[Playwright for Python documentation](https://playwright.dev/python/docs/browsers#install-system-dependencies)
for additional information.
## Enable voice coding (optional)
{: .no_toc }
Aider supports [coding with your voice](https://aider.chat/docs/voice.html)
using the in-chat `/voice` command.
Aider uses the [PortAudio](http://www.portaudio.com) library to
capture audio.
Installing PortAudio is completely optional, but can usually be accomplished like this:
- For Windows, there is no need to install PortAudio.
- For Mac, do `brew install portaudio`
- For Linux, do `sudo apt-get install libportaudio2`
## Add aider to your editor (optional)
{: .no_toc }
Other projects have integrated aider into some IDE/editors.
It's not clear if they are tracking the latest
versions of aider,
so it may be best to just run the latest
aider in a terminal alongside your editor.
### NeoVim
{: .no_toc }
[joshuavial](https://github.com/joshuavial) provided a NeoVim plugin for aider:
[https://github.com/joshuavial/aider.nvim](https://github.com/joshuavial/aider.nvim)
### VS Code
{: .no_toc }
joshuavial also confirmed that aider works inside a VS Code terminal window.
Aider detects if it is running inside VSCode and turns off pretty/color output,
since the VSCode terminal doesn't seem to support it well.
[MattFlower](https://github.com/MattFlower) provided a VSCode plugin for aider:
[https://marketplace.visualstudio.com/items?itemName=MattFlower.aider](https://marketplace.visualstudio.com/items?itemName=MattFlower.aider)
### Other editors
{: .no_toc }
If you are interested in creating an aider plugin for your favorite editor,
please let me know by opening a
[GitHub issue](https://github.com/paul-gauthier/aider/issues).
## Install development versions of aider (optional)
{: .no_toc }
If you want to install the very latest development version of aider:
* GitHub hosts the latest version, which is under active development:
* `python -m pip install git+https://github.com/paul-gauthier/aider.git`
* If you've git cloned the aider repository already, you can install "live" from your local copy. This is mostly useful if you are developing aider and want your current modifications to take effect immediately.
* `python -m pip install -e .`

224
website/docs/leaderboards/index.md Normal file
View file

@ -0,0 +1,224 @@
---
highlight_image: /assets/leaderboard.jpg
nav_order: 900
---
# Aider LLM Leaderboards
Aider works best with LLMs which are good at *editing* code, not just good at writing
code.
To evaluate an LLM's editing skill, aider uses a pair of benchmarks that
assess a model's ability to consistently follow the system prompt
to successfully edit code.
The leaderboards below report the results from a number of popular LLMs.
While [aider can connect to almost any LLM](/docs/llms.html),
it works best with models that score well on the benchmarks.
## GPT-4o takes the #1 & #2 spots
GPT-4o tops the aider LLM code editing leaderboard at 72.9%, versus 68.4% for Opus. GPT-4o takes second on aider's refactoring leaderboard with 62.9%, versus Opus at 72.3%.
GPT-4o did much better than the 4-turbo models, and seems *much* less lazy.
GPT-4o is also able to use aider's established "diff" edit format that uses
`SEARCH/REPLACE` blocks.
This diff format is used by all the other capable models, including Opus and
the original GPT-4 models
The GPT-4 Turbo models have all required the "udiff" edit format, due to their
tendancy to lazy coding.
## Code editing leaderboard
[Aider's code editing benchmark](/docs/benchmarks.html#the-benchmark) asks the LLM to edit python source files to complete 133 small coding exercises. This benchmark measures the LLM's coding ability, but also whether it can consistently emit code edits in the format specified in the system prompt.
<table style="width: 100%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1); font-size: 14px;">
<thead style="background-color: #f2f2f2;">
<tr>
<th style="padding: 8px; text-align: left;">Model</th>
<th style="padding: 8px; text-align: center;">Percent completed correctly</th>
<th style="padding: 8px; text-align: center;">Percent using correct edit format</th>
<th style="padding: 8px; text-align: left;">Command</th>
<th style="padding: 8px; text-align: center;">Edit format</th>
</tr>
</thead>
<tbody>
{% assign edit_sorted = site.data.edit_leaderboard | sort: 'pass_rate_2' | reverse %}
{% for row in edit_sorted %}
<tr style="border-bottom: 1px solid #ddd;">
<td style="padding: 8px;">{{ row.model }}</td>
<td style="padding: 8px; text-align: center;">{{ row.pass_rate_2 }}%</td>
<td style="padding: 8px; text-align: center;">{{ row.percent_cases_well_formed }}%</td>
<td style="padding: 8px;"><code>{{ row.command }}</code></td>
<td style="padding: 8px; text-align: center;">{{ row.edit_format }}</td>
</tr>
{% endfor %}
</tbody>
</table>
<canvas id="editChart" width="800" height="450" style="margin-top: 20px"></canvas>
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<script>
document.addEventListener('DOMContentLoaded', function () {
var ctx = document.getElementById('editChart').getContext('2d');
var leaderboardData = {
labels: [],
datasets: [{
label: 'Percent completed correctly',
data: [],
backgroundColor: 'rgba(54, 162, 235, 0.2)',
borderColor: 'rgba(54, 162, 235, 1)',
borderWidth: 1
},
{
label: 'Percent using correct edit format',
data: [],
backgroundColor: 'rgba(255, 99, 132, 0.2)',
borderColor: 'rgba(255, 99, 132, 1)',
borderWidth: 1
}]
};
{% for row in edit_sorted %}
leaderboardData.labels.push('{{ row.model }}');
leaderboardData.datasets[0].data.push({{ row.pass_rate_2 }});
leaderboardData.datasets[1].data.push({{ row.percent_cases_well_formed }});
{% endfor %}
var leaderboardChart = new Chart(ctx, {
type: 'bar',
data: leaderboardData,
options: {
scales: {
yAxes: [{
scaleLabel: {
display: true,
},
ticks: {
beginAtZero: true
}
}]
}
}
});
});
</script>
## Code refactoring leaderboard
[Aider's refactoring benchmark](https://github.com/paul-gauthier/refactor-benchmark) asks the LLM to refactor 89 large methods from large python classes. This is a more challenging benchmark, which tests the model's ability to output long chunks of code without skipping sections or making mistakes. It was developed to provoke and measure [GPT-4 Turbo's "lazy coding" habit](/2023/12/21/unified-diffs.html).
The refactoring benchmark requires a large context window to
work with large source files.
Therefore, results are available for fewer models.
<table style="width: 100%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1); font-size: 14px;">
<thead style="background-color: #f2f2f2;">
<tr>
<th style="padding: 8px; text-align: left;">Model</th>
<th style="padding: 8px; text-align: center;">Percent completed correctly</th>
<th style="padding: 8px; text-align: center;">Percent using correct edit format</th>
<th style="padding: 8px; text-align: left;">Command</th>
<th style="padding: 8px; text-align: center;">Edit format</th>
</tr>
</thead>
<tbody>
{% assign refac_sorted = site.data.refactor_leaderboard | sort: 'pass_rate_1' | reverse %}
{% for row in refac_sorted %}
<tr style="border-bottom: 1px solid #ddd;">
<td style="padding: 8px;">{{ row.model }}</td>
<td style="padding: 8px; text-align: center;">{{ row.pass_rate_1 }}%</td>
<td style="padding: 8px; text-align: center;">{{ row.percent_cases_well_formed }}%</td>
<td style="padding: 8px;"><code>{{ row.command }}</code></td>
<td style="padding: 8px; text-align: center;">{{ row.edit_format }}</td>
</tr>
{% endfor %}
</tbody>
</table>
<canvas id="refacChart" width="800" height="450" style="margin-top: 20px"></canvas>
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<script>
document.addEventListener('DOMContentLoaded', function () {
var ctx = document.getElementById('refacChart').getContext('2d');
var leaderboardData = {
labels: [],
datasets: [{
label: 'Percent completed correctly',
data: [],
backgroundColor: 'rgba(54, 162, 235, 0.2)',
borderColor: 'rgba(54, 162, 235, 1)',
borderWidth: 1
},
{
label: 'Percent using correct edit format',
data: [],
backgroundColor: 'rgba(255, 99, 132, 0.2)',
borderColor: 'rgba(255, 99, 132, 1)',
borderWidth: 1
}]
};
{% for row in refac_sorted %}
leaderboardData.labels.push('{{ row.model }}');
leaderboardData.datasets[0].data.push({{ row.pass_rate_1 }});
leaderboardData.datasets[1].data.push({{ row.percent_cases_well_formed }});
{% endfor %}
var leaderboardChart = new Chart(ctx, {
type: 'bar',
data: leaderboardData,
options: {
scales: {
yAxes: [{
scaleLabel: {
display: true,
},
ticks: {
beginAtZero: true
}
}]
}
}
});
});
</script>
## LLM code editing skill by model release date
[![connecting to many LLMs](/assets/models-over-time.svg)](https://aider.chat/assets/models-over-time.svg)
## Notes on benchmarking results
The key benchmarking results are:
- **Percent completed correctly** - Measures what percentage of the coding tasks that the LLM completed successfully. To complete a task, the LLM must solve the programming assignment *and* edit the code to implement that solution.
- **Percent using correct edit format** - Measures the percent of coding tasks where the LLM complied with the edit format specified in the system prompt. If the LLM makes edit mistakes, aider will give it feedback and ask for a fixed copy of the edit. The best models can reliably conform to the edit format, without making errors.
## Notes on the edit format
Aider uses different "edit formats" to collect code edits from different LLMs.
The "whole" format is the easiest for an LLM to use, but it uses a lot of tokens
and may limit how large a file can be edited.
Models which can use one of the diff formats are much more efficient,
using far fewer tokens.
Models that use a diff-like format are able to
edit larger files with less cost and without hitting token limits.
Aider is configured to use the best edit format for the popular OpenAI and Anthropic models
and the [other models recommended on the LLM page](/docs/llms.html).
For lesser known models aider will default to using the "whole" editing format
since it is the easiest format for an LLM to use.
## Contributing benchmark results
Contributions of benchmark results are welcome!
See the
[benchmark README](https://github.com/paul-gauthier/aider/blob/main/benchmark/README.md)
for information on running aider's code editing benchmarks.
Submit results by opening a PR with edits to the
[benchmark results data files](https://github.com/paul-gauthier/aider/blob/main/_data/).

460
website/docs/llms.md Normal file
View file

@ -0,0 +1,460 @@
---
title: Connecting to LLMs
nav_order: 70
---
# Aider can connect to most LLMs
[![connecting to many LLMs](/assets/llms.jpg)](https://aider.chat/assets/llms.jpg)
## Best models
**Aider works best with [GPT-4o](#openai) and [Claude 3 Opus](#anthropic),**
as they are the very best models for editing code.
## Free models
Aider works with a number of **free** API providers:
- Google's [Gemini 1.5 Pro](#gemini) is the most capable free model to use with aider, with
code editing capabilities similar to GPT-3.5.
- You can use [Llama 3 70B on Groq](#groq) which is comparable to GPT-3.5 in code editing performance.
- The [Deepseek Chat v2](#deepseek) model work well with aider, better than GPT-3.5. Deepseek currently offers 5M free tokens of API usage.
- Cohere also offers free API access to their [Command-R+ model](#cohere), which works with aider as a *very basic* coding assistant.
## Local models
Aider can work also with local models, for example using [Ollama](#ollama).
It can also access
local models that provide an
[Open AI compatible API](#openai-compatible-apis).
## Use a capable model
Check
[Aider's LLM leaderboards](https://aider.chat/docs/leaderboards/)
to see which models work best with aider.
Be aware that aider may not work well with less capable models.
If you see the model returning code, but aider isn't able to edit your files
and commit the changes...
this is usually because the model isn't capable of properly
returning "code edits".
Models weaker than GPT 3.5 may have problems working well with aider.
## Configuring models
- [OpenAI](#openai)
- [Anthropic](#anthropic)
- [Gemini](#gemini)
- [Groq & Llama3](#groq)
- [Cohere](#cohere)
- [Azure](#azure)
- [OpenRouter](#openrouter)
- [Ollama](#ollama)
- [Deepseek](#deepseek)
- [OpenAI compatible APIs](#openai-compatible-apis)
- [Other LLMs](#other-llms)
- [Model warnings](#model-warnings)
- [Editing format](#editing-format)
- [Using a .env file](#using-a-env-file)
Aider uses the LiteLLM package to connect to LLM providers.
The [LiteLLM provider docs](https://docs.litellm.ai/docs/providers)
contain more detail on all the supported providers,
their models and any required environment variables.
## OpenAI
To work with OpenAI's models, you need to provide your
[OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key)
either in the `OPENAI_API_KEY` environment variable or
via the `--openai-api-key` command line switch.
Aider has some built in shortcuts for the most popular OpenAI models and
has been tested and benchmarked to work well with them:
```
pip install aider-chat
export OPENAI_API_KEY=<key> # Mac/Linux
setx OPENAI_API_KEY <key> # Windows
# GPT-4o is the best model, used by default
aider
# GPT-4 Turbo (1106)
aider --4-turbo
# GPT-3.5 Turbo
aider --35-turbo
# List models available from OpenAI
aider --models openai/
```
You can use `aider --model <model-name>` to use any other OpenAI model.
For example, if you want to use a specific version of GPT-4 Turbo
you could do `aider --model gpt-4-0125-preview`.
## Anthropic
To work with Anthropic's models, you need to provide your
[Anthropic API key](https://docs.anthropic.com/claude/reference/getting-started-with-the-api)
either in the `ANTHROPIC_API_KEY` environment variable or
via the `--anthropic-api-key` command line switch.
Aider has some built in shortcuts for the most popular Anthropic models and
has been tested and benchmarked to work well with them:
```
pip install aider-chat
export ANTHROPIC_API_KEY=<key> # Mac/Linux
setx ANTHROPIC_API_KEY <key> # Windows
# Claude 3 Opus
aider --opus
# Claude 3 Sonnet
aider --sonnet
# List models available from Anthropic
aider --models anthropic/
```
You can use `aider --model <model-name>` to use any other Anthropic model.
For example, if you want to use a specific version of Opus
you could do `aider --model claude-3-opus-20240229`.
## Gemini
Google currently offers
[*free* API access to the Gemini 1.5 Pro model](https://ai.google.dev/pricing).
This is the most capable free model to use with aider,
with code editing capability that's comparable to GPT-3.5.
You'll need a [Gemini API key](https://aistudio.google.com/app/u/2/apikey).
```
pip install aider-chat
export GEMINI_API_KEY=<key> # Mac/Linux
setx GEMINI_API_KEY <key> # Windows
aider --model gemini/gemini-1.5-pro-latest
# List models available from Gemini
aider --models gemini/
```
## GROQ
Groq currently offers *free* API access to the models they host.
The Llama 3 70B model works
well with aider and is comparable to GPT-3.5 in code editing performance.
You'll need a [Groq API key](https://console.groq.com/keys).
To use **Llama3 70B**:
```
pip install aider-chat
export GROQ_API_KEY=<key> # Mac/Linux
setx GROQ_API_KEY <key> # Windows
aider --model groq/llama3-70b-8192
# List models available from Groq
aider --models groq/
```
## Cohere
Cohere offers *free* API access to their models.
Their Command-R+ model works well with aider
as a *very basic* coding assistant.
You'll need a [Cohere API key](https://dashboard.cohere.com/welcome/login).
To use **Command-R+**:
```
pip install aider-chat
export COHERE_API_KEY=<key> # Mac/Linux
setx COHERE_API_KEY <key> # Windows
aider --model command-r-plus
# List models available from Cohere
aider --models cohere_chat/
```
## Azure
Aider can connect to the OpenAI models on Azure.
```
pip install aider-chat
# Mac/Linux:
export AZURE_API_KEY=<key>
export AZURE_API_VERSION=2023-05-15
export AZURE_API_BASE=https://myendpt.openai.azure.com
# Windows:
setx AZURE_API_KEY <key>
setx AZURE_API_VERSION 2023-05-15
setx AZURE_API_BASE https://myendpt.openai.azure.com
aider --model azure/<your_deployment_name>
# List models available from Azure
aider --models azure/
```
## OpenRouter
Aider can connect to [models provided by OpenRouter](https://openrouter.ai/models?o=top-weekly):
You'll need an [OpenRouter API key](https://openrouter.ai/keys).
```
pip install aider-chat
export OPENROUTER_API_KEY=<key> # Mac/Linux
setx OPENROUTER_API_KEY <key> # Windows
# Or any other open router model
aider --model openrouter/<provider>/<model>
# List models available from OpenRouter
aider --models openrouter/
```
In particular, Llama3 70B works well with aider, at low cost:
```
pip install aider-chat
export OPENROUTER_API_KEY=<key> # Mac/Linux
setx OPENROUTER_API_KEY <key> # Windows
aider --model openrouter/meta-llama/llama-3-70b-instruct
```
## Ollama
Aider can connect to local Ollama models.
```
# Pull the model
ollama pull <model>
# Start your ollama server
ollama serve
# In another terminal window...
pip install aider-chat
export OLLAMA_API_BASE=http://127.0.0.1:11434 # Mac/Linux
setx OLLAMA_API_BASE http://127.0.0.1:11434 # Windows
aider --model ollama/<model>
```
In particular, `llama3:70b` works very well with aider:
```
ollama pull llama3:70b
ollama serve
# In another terminal window...
export OLLAMA_API_BASE=http://127.0.0.1:11434 # Mac/Linux
setx OLLAMA_API_BASE http://127.0.0.1:11434 # Windows
aider --model ollama/llama3:70b
```
Also see the [model warnings](#model-warnings)
section for information on warnings which will occur
when working with models that aider is not familiar with.
## Deepseek
Aider can connect to the Deepseek.com API.
Deepseek appears to grant 5M tokens of free API usage to new accounts.
```
pip install aider-chat
export DEEPSEEK_API_KEY=<key> # Mac/Linux
setx DEEPSEEK_API_KEY <key> # Windows
# Use Deepseek Chat v2
aider --model deepseek/deepseek-chat
```
See the [model warnings](#model-warnings)
section for information on warnings which will occur
when working with models that aider is not familiar with.
## OpenAI compatible APIs
Aider can connect to any LLM which is accessible via an OpenAI compatible API endpoint.
```
pip install aider-chat
# Mac/Linux:
export OPENAI_API_BASE=<endpoint>
export OPENAI_API_KEY=<key>
# Windows:
setx OPENAI_API_BASE <endpoint>
setx OPENAI_API_KEY <key>
# Prefix the model name with openai/
aider --model openai/<model-name>
```
See the [model warnings](#model-warnings)
section for information on warnings which will occur
when working with models that aider is not familiar with.
## Other LLMs
Aider uses the [litellm](https://docs.litellm.ai/docs/providers) package
to connect to hundreds of other models.
You can use `aider --model <model-name>` to use any supported model.
To explore the list of supported models you can run `aider --models <model-name>`
with a partial model name.
If the supplied name is not an exact match for a known model, aider will
return a list of possible matching models.
For example:
```
$ aider --models turbo
Aider v0.29.3-dev
Models which match "turbo":
- gpt-4-turbo-preview (openai/gpt-4-turbo-preview)
- gpt-4-turbo (openai/gpt-4-turbo)
- gpt-4-turbo-2024-04-09 (openai/gpt-4-turbo-2024-04-09)
- gpt-3.5-turbo (openai/gpt-3.5-turbo)
- ...
```
See the [list of providers supported by litellm](https://docs.litellm.ai/docs/providers)
for more details.
## Model warnings
Aider supports connecting to almost any LLM,
but it may not work well with less capable models.
If you see the model returning code, but aider isn't able to edit your files
and commit the changes...
this is usually because the model isn't capable of properly
returning "code edits".
Models weaker than GPT 3.5 may have problems working well with aider.
Aider tries to sanity check that it is configured correctly
to work with the specified model:
- It checks to see that all required environment variables are set for the model. These variables are required to configure things like API keys, API base URLs, etc.
- It checks a metadata database to look up the context window size and token costs for the model.
Sometimes one or both of these checks will fail, so aider will issue
some of the following warnings.
#### Missing environment variables
```
Model azure/gpt-4-turbo: Missing these environment variables:
- AZURE_API_BASE
- AZURE_API_VERSION
- AZURE_API_KEY
```
You need to set the listed environment variables.
Otherwise you will get error messages when you start chatting with the model.
#### Unknown which environment variables are required
```
Model gpt-5: Unknown which environment variables are required.
```
Aider is unable verify the environment because it doesn't know
which variables are required for the model.
If required variables are missing,
you may get errors when you attempt to chat with the model.
You can look in the
[litellm provider documentation](https://docs.litellm.ai/docs/providers)
to see if the required variables are listed there.
#### Unknown model, did you mean?
```
Model gpt-5: Unknown model, context window size and token costs unavailable.
Did you mean one of these?
- gpt-4
```
If you specify a model that aider has never heard of, you will get an
"unknown model" warning.
This means aider doesn't know the context window size and token costs
for that model.
Some minor functionality will be limited when using such models, but
it's not really a significant problem.
Aider will also try to suggest similarly named models,
in case you made a typo or mistake when specifying the model name.
## Editing format
Aider uses different "edit formats" to collect code edits from different LLMs.
The "whole" format is the easiest for an LLM to use, but it uses a lot of tokens
and may limit how large a file can be edited.
Models which can use one of the diff formats are much more efficient,
using far fewer tokens.
Models that use a diff-like format are able to
edit larger files with less cost and without hitting token limits.
Aider is configured to use the best edit format for the popular OpenAI and Anthropic models
and the [other models recommended on the LLM page](https://aider.chat/docs/llms.html).
For lesser known models aider will default to using the "whole" editing format
since it is the easiest format for an LLM to use.
If you would like to experiment with the more advanced formats, you can
use these switches: `--edit-format diff` or `--edit-format udiff`.
# Using a .env file
Aider will read environment variables from a `.env` file in
root of your git repo or in current directory.
You can give it an explicit file to load with the `--env-file <filename>` parameter.
You can use a `.env` file to store various keys and other settings for the
models you use with aider.
Here is an example `.env` file:
```
OPENAI_API_KEY=<key>
ANTHROPIC_API_KEY=<key>
GROQ_API_KEY=<key>
OPENROUTER_API_KEY=<key>
AZURE_API_KEY=<key>
AZURE_API_VERSION=2023-05-15
AZURE_API_BASE=https://example-endpoint.openai.azure.com
OLLAMA_API_BASE=http://127.0.0.1:11434
```

264
website/docs/repomap.md Normal file
View file

@ -0,0 +1,264 @@
---
title: Building a better repository map with tree sitter
excerpt: Tree-sitter allows aider to build a repo map that better summarizes large code bases.
highlight_image: /assets/robot-ast.png
nav_exclude: true
---
# Building a better repository map with tree sitter
![robot flowchat](/assets/robot-ast.png)
GPT-4 is extremely useful for "self-contained" coding tasks,
like generating or modifying a simple function
that has no dependencies. Tools like GitHub CoPilot serve
these simple coding tasks well.
But making complex changes in a larger, pre-existing codebase
is much more difficult, for both humans and AIs.
To do this successfully, you need to:
1. Find the code that needs to be changed.
2. Understand how that code relates to the rest of the codebase.
3. Make the correct code change to accomplish the task.
GPT-4 is actually great at making the code changes (3),
once you tell it which files need to be changed (1)
and show it how they fit into the rest of the codebase (2).
This article is going to focus on step (2), providing "code context":
- We need to help GPT understand the overall codebase.
- This will help it understand the code it needs to change, which may depend on other parts of the codebase.
- It will also help GPT write new code and modify the existing code in a way
that respects and utilizes existing libraries, modules and abstractions
found elsewhere in the codebase.
- We must convey all of this "code context" to GPT in an
efficient manner that fits within the limited context window.
To address these issues, aider
sends GPT a **concise map of your whole git repository**
that includes
the most important classes and functions along with their types and call signatures.
This **repository map** is now built automatically using
[tree-sitter](https://tree-sitter.github.io/tree-sitter/)
to extract symbol definitions from source files.
Tree-sitter is used by many IDEs, editors and LSP servers to
help humans search and navigate large codebases.
Aider now uses it to help GPT better comprehend, navigate
and edit code in larger repos.
*To code with GPT-4 using the techniques discussed here, just install [aider](https://aider.chat/docs/install.html).*
## The problem: code context
GPT-4 is great at "self contained" coding tasks, like writing or
modifying a pure function with no external dependencies.
GPT can easily handle requests like "write a
Fibonacci function" or "rewrite this loop using list
comprehensions", because they require no context beyond the code
being discussed.
Most real code is not pure and self-contained, it is intertwined with
and depends on code from many different files in a repo.
If you ask GPT to "switch all the print statements in class Foo to
use the BarLog logging system", it needs to see and
modify the code in the Foo class, but it also needs to understand
how to use
the project's BarLog
subsystem.
A simple solution is to **send the entire codebase** to GPT along with
each change request. Now GPT has all the context! But this won't work
for even moderately
sized repos, because they won't fit into the context window.
A better approach is to be selective,
and **hand pick which files to send**.
For the example above, you could send the file that
contains the Foo class
and the file that contains the BarLog logging subsystem.
This works pretty well, and is supported by aider -- you
can manually specify which files to "add to the chat" you are having with GPT.
But sending whole files is a bulky way to send code context,
wasting the precious context window.
GPT doesn't need to see the entire implementation of BarLog,
it just needs to understand it well enough to use it.
You may quickly run out of context window by sending
full files of code
just to convey context.
Aider also strives to reduce the manual work involved in
coding with AI.
So in an ideal world, we'd like aider to automatically
identify and provide the needed code context.
## Using a repo map to provide context
Aider sends a **repo map** to GPT along with
each request from the user to make a code change.
The map contains a list of the files in the
repo, along with the key symbols which are defined in each file.
It shows how each of these symbols are defined in the
source code, by including the critical lines of code for each definition.
Here's a
sample of the map of the aider repo, just showing the maps of
[base_coder.py](https://github.com/paul-gauthier/aider/blob/main/aider/coders/base_coder.py)
and
[commands.py](https://github.com/paul-gauthier/aider/blob/main/aider/commands.py)
:
```
aider/coders/base_coder.py:
⋮...
│class Coder:
│ abs_fnames = None
⋮...
│ @classmethod
│ def create(
│ self,
│ main_model,
│ edit_format,
│ io,
│ skip_model_availabily_check=False,
│ **kwargs,
⋮...
│ def abs_root_path(self, path):
⋮...
│ def run(self, with_message=None):
⋮...
aider/commands.py:
⋮...
│class Commands:
│ voice = None
│
⋮...
│ def get_commands(self):
⋮...
│ def get_command_completions(self, cmd_name, partial):
⋮...
│ def run(self, inp):
⋮...
```
Mapping out the repo like this provides some key benefits:
- GPT can see classes, methods and function signatures from everywhere in the repo. This alone may give it enough context to solve many tasks. For example, it can probably figure out how to use the API exported from a module just based on the details shown in the map.
- If it needs to see more code, GPT can use the map to figure out by itself which files it needs to look at in more detail. GPT will then ask to see these specific files, and aider will automatically add them to the chat context.
## Optimizing the map
Of course, for large repositories even just the repo map might be too large
for GPT's context window.
Aider solves this problem by sending just the **most relevant**
portions of the repo map.
It does this by analyzing the full repo map using
a graph ranking algorithm, computed on a graph
where each source file is a node and edges connect
files which have dependencies.
Aider optimizes the repo map by
selecting the most important parts of the codebase
which will
fit into the token budget assigned by the user
(via the `--map-tokens` switch, which defaults to 1k tokens).
The sample map shown above doesn't contain *every* class, method and function from those
files.
It only includes the most important identifiers,
the ones which are most often referenced by other portions of the code.
These are the key pieces of context that GPT needs to know to understand
the overall codebase.
## Using tree-sitter to make the map
Under the hood, aider uses
[tree sitter](https://tree-sitter.github.io/tree-sitter/)
to build the
map.
It specifically uses the
[py-tree-sitter-languages](https://github.com/grantjenks/py-tree-sitter-languages)
python module,
which provides simple, pip-installable binary wheels for
[most popular programming languages](https://github.com/paul-gauthier/grep-ast/blob/main/grep_ast/parsers.py).
Tree-sitter parses source code into an Abstract Syntax Tree (AST) based
on the syntax of the programming language.
Using the AST, we can identify where functions, classes, variables, types and
other definitions occur in the source code.
We can also identify where else in the code these things are used or referenced.
Aider uses all of these definitions and references to
determine which are the most important identifiers in the repository,
and to produce the repo map that shows just those key
lines from the codebase.
## What about ctags?
The tree-sitter repository map replaces the
[ctags based map](https://aider.chat/docs/ctags.html)
that aider originally used.
Switching from ctags to tree-sitter provides a bunch of benefits:
- The map is richer, showing full function call signatures and other details straight from the source files.
- Thanks to `py-tree-sitter-languages`, we get full support for many programming languages via a python package that's automatically installed as part of the normal `pip install aider-chat`.
- We remove the requirement for users to manually install `universal-ctags` via some external tool or package manager (brew, apt, choco, etc).
- Tree-sitter integration is a key enabler for future work and capabilities for aider.
## Future work
You'll recall that we identified the 3 key steps
required to use GPT
to complete a coding task within a large, pre-existing codebase:
1. Find the code that needs to be changed.
2. Understand how that code relates to the rest of the codebase.
3. Make the correct code change to accomplish the task.
We're now using tree-sitter to help solve the code context problem (2),
but it's also an important foundation
for future work on automatically finding all the code which
will need to be changed (1).
Right now, aider relies on the user to specify which source files
will need to be modified to complete their request.
Users manually "add files to the chat" using aider's `/add` command,
which makes those files available for GPT to modify.
This works well, but a key piece of future work is to harness the
power of GPT and tree-sitter to automatically identify
which parts of the code will need changes.
## Try it out
To code with GPT-4 using the techniques discussed here,
just install [aider](https://aider.chat/docs/install.html).
## Credits
Aider uses
[modified versions of the tags.scm files](https://github.com/paul-gauthier/aider/tree/main/aider/queries)
from these
open source tree-sitter language implementations:
* [https://github.com/tree-sitter/tree-sitter-c](https://github.com/tree-sitter/tree-sitter-c) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-c-sharp](https://github.com/tree-sitter/tree-sitter-c-sharp) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-cpp](https://github.com/tree-sitter/tree-sitter-cpp) — licensed under the MIT License.
* [https://github.com/Wilfred/tree-sitter-elisp](https://github.com/Wilfred/tree-sitter-elisp) — licensed under the MIT License.
* [https://github.com/elixir-lang/tree-sitter-elixir](https://github.com/elixir-lang/tree-sitter-elixir) — licensed under the Apache License, Version 2.0.
* [https://github.com/elm-tooling/tree-sitter-elm](https://github.com/elm-tooling/tree-sitter-elm) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-go](https://github.com/tree-sitter/tree-sitter-go) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-java](https://github.com/tree-sitter/tree-sitter-java) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-javascript](https://github.com/tree-sitter/tree-sitter-javascript) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-ocaml](https://github.com/tree-sitter/tree-sitter-ocaml) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-php](https://github.com/tree-sitter/tree-sitter-php) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-python](https://github.com/tree-sitter/tree-sitter-python) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-ql](https://github.com/tree-sitter/tree-sitter-ql) — licensed under the MIT License.
* [https://github.com/r-lib/tree-sitter-r](https://github.com/r-lib/tree-sitter-r) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-ruby](https://github.com/tree-sitter/tree-sitter-ruby) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-rust](https://github.com/tree-sitter/tree-sitter-rust) — licensed under the MIT License.
* [https://github.com/tree-sitter/tree-sitter-typescript](https://github.com/tree-sitter/tree-sitter-typescript) — licensed under the MIT License.

18
website/docs/tips.md Normal file
View file

@ -0,0 +1,18 @@
---
nav_order: 50
---
## Tips
* Think about which files need to be edited to make your change and add them to the chat.
Aider can help the LLM figure out which files to edit all by itself, but the most efficient approach is to add the needed files to the chat yourself.
* Don't add *everything* to the chat, just the files you think need to be edited.
Aider also sends the LLM a [map of your entire git repo](https://aider.chat/docs/repomap.html).
So the LLM can see all the other relevant parts of your code base.
* Large changes are best performed as a sequence of thoughtful bite sized steps, where you plan out the approach and overall design. Walk the LLM through changes like you might with a junior dev. Ask for a refactor to prepare, then ask for the actual change. Spend the time to ask for code quality/structure improvements.
* Use Control-C to safely interrupt the LLM if it isn't providing a useful response. The partial response remains in the conversation, so you can refer to it when you reply to the LLM with more information or direction.
* Use the `/run` and `/test` commands to run the code or tests and show the output to the LLM so it can fix any issues.
* Use Meta-ENTER (Esc+ENTER in some environments) to enter multiline chat messages. Or enter `{` alone on the first line to start a multiline message and `}` alone on the last line to end it.
* If your code is throwing an error, share the error output with the LLM using `/run` or by pasting it into the chat. Let the LLM figure out and fix the bug.
* LLMs know about a lot of standard tools and libraries, but may get some of the fine details wrong about APIs and function arguments. You can paste doc snippets into the chat to resolve these issues.

13
website/docs/tutorials.md Normal file
View file

@ -0,0 +1,13 @@
---
nav_order: 25
---
## Tutorial videos
Here are a few tutorial videos made by aider users:
- [Aider : the production ready AI coding assistant you've been waiting for](https://www.youtube.com/watch?v=zddJofosJuM) -- Learn Code With JV
- [Holy Grail: FREE Coding Assistant That Can Build From EXISTING CODE BASE](https://www.youtube.com/watch?v=df8afeb1FY8) -- Matthew Berman
- [Aider: This AI Coder Can Create AND Update Git Codebases](https://www.youtube.com/watch?v=EqLyFT78Sig) -- Ian Wootten

381
website/docs/unified-diffs.md Normal file
View file

@ -0,0 +1,381 @@
---
title: Unified diffs make GPT-4 Turbo 3X less lazy
excerpt: GPT-4 Turbo has a problem with lazy coding, which can be signiciantly improved by asking for code changes formatted as unified diffs.
highlight_image: /assets/benchmarks-udiff.jpg
nav_exclude: true
---
# Unified diffs make GPT-4 Turbo 3X less lazy
![robot flowchart](/assets/benchmarks-udiff.svg)
Aider now asks GPT-4 Turbo to use
[unified diffs](#choose-a-familiar-editing-format)
to edit your code.
This dramatically improves GPT-4 Turbo's performance on a
challenging
new benchmark
and significantly reduces its bad habit of "lazy" coding,
where it writes
code with comments
like "...add logic here...".
Aider's new "laziness" benchmark suite
is designed to both provoke and quantify lazy coding.
It consists of
89 python refactoring tasks
which tend to make GPT-4 Turbo write lazy comments like
"...include original method body...".
This new laziness benchmark produced the following results with `gpt-4-1106-preview`:
- **GPT-4 Turbo only scored 20% as a baseline** using aider's existing "SEARCH/REPLACE block" edit format. It outputs "lazy comments" on 12 of the tasks.
- **Aider's new unified diff edit format raised the score to 61%**. Using this format reduced laziness by 3X, with GPT-4 Turbo only using lazy comments on 4 of the tasks.
- **It's worse to add a prompt that says the user is blind, has no hands, will tip $2000 and fears truncated code trauma.** Widely circulated "emotional appeal" folk remedies
produced worse benchmark scores
for both the baseline SEARCH/REPLACE and new unified diff editing formats.
The older `gpt-4-0613` also did better on the laziness benchmark using unified diffs:
- **The June GPT-4's baseline was 26%** using aider's existing "SEARCH/REPLACE block" edit format.
- **Aider's new unified diff edit format raised June GPT-4's score to 59%**.
- The benchmark was designed to use large files, and
28% of them are too large to fit in June GPT-4's 8k context window.
This puts a hard ceiling of 72% on how well the June model could possibly score.
With unified diffs, GPT acts more like it's writing textual data intended to be read by a program,
not talking to a person.
Diffs are
usually
consumed by the
[patch](https://www.gnu.org/software/diffutils/manual/html_node/Merging-with-patch.html)
program, which is fairly rigid.
This seems to encourage rigor, making
GPT less likely to
leave informal editing instructions in comments
or be lazy about writing all the needed code.
Aider's new unified diff editing format
outperforms other solutions I evaluated by a wide margin.
I explored many other approaches including:
prompts about being tireless and diligent,
OpenAI's function/tool calling capabilities,
numerous variations on aider's existing editing formats,
line number based formats
and other diff-like formats.
The results shared here reflect
an extensive investigation and benchmark evaluations of many approaches.
The rest of this article will describe
aider's new editing format and refactoring benchmark.
It will highlight some key design decisions,
and evaluate their significance using ablation experiments.
## Unified diff editing format
The design and implementation of aider's new unified diff editing format
helped clarify some general principles
for GPT-4 code editing:
- FAMILIAR - Choose an edit format that GPT is already familiar with.
- SIMPLE - Choose a simple format that avoids escaping, syntactic overhead and brittle specifiers like line numbers or line counts.
- HIGH LEVEL - Encourage GPT to structure edits as new versions of substantive code blocks (functions, methods, etc), not as a series of surgical/minimal changes to individual lines of code.
- FLEXIBLE - Strive to be maximally flexible when interpreting GPT's edit instructions.
A helpful shortcut here is to have empathy for GPT, and imagine you
are the one being asked to specify code edits.
Would you want to hand type a properly escaped json data structure
to invoke surgical insert, delete, replace operations on specific code line numbers?
Do you want to use a brittle format, where any mistake
causes an error that discards all your work?
GPT is quantitatively better at code editing when you reduce the
burden of formatting edits by using a familiar, simple, high level
and flexible editing format.
### Choose a familiar editing format
Unified diffs are perhaps the most common way to show
code edits, because it's the
default output format of `git diff`:
```diff
--- a/greeting.py
+++ b/greeting.py
@@ -1,5 +1,5 @@
def main(args):
# show a greeting
- print("Hello!")
+ print("Goodbye!")
return
```
Choosing such a popular format means that GPT has
seen *many* examples in its training data.
It's been trained to generate
text that conforms to the unified diff syntax.
### Use a simple editing format
Aider's [previous benchmark results](https://aider.chat/docs/benchmarks.html) made
it clear that simple editing formats
work best.
Even though OpenAI provides extensive support for
structured formats like json and function calls,
GPT is worse at editing code if you use them.
I repeated these and other similar benchmarks against GPT-4 Turbo,
and again reached these same conclusions.
Informally, this is probably because stuffing *source code* into JSON is complicated
and error prone.
Wrapping the python code
`print("On Windows use \"C:\\\"")`
as valid json is pretty painful and error prone.
Due to escaping issues GPT's code is often syntactically incorrect when it's
unpacked from JSON,
or the JSON decode just fails entirely.
On the other hand, the core of the unified diff format is very simple.
You include a hunk of the file that needs to be changed,
with every line prefixed by a character
to indicate unchanged, new or deleted lines.
A unified diff looks pretty much like the code it is modifying.
The one complicated piece is the line numbers found at the start
of each hunk. They look something like this: `@@ -2,4 +3,5 @@`.
GPT is terrible at working with source code line numbers.
This is a general observation about *any* use of line
numbers in editing formats,
backed up by many quantitative benchmark experiments.
You've probably ignored the line numbers in every diff you've seen,
because the diffs usually still make sense without them.
Aider tells GPT not to include line numbers,
and just interprets each hunk from the unified diffs
as a search and replace operation:
This diff:
```diff
@@ ... @@
def main(args):
# show a greeting
- print("Hello!")
+ print("Goodbye!")
return
```
Means we need to search the file for the
*space* and *minus* `-` lines:
```python
def main(args):
# show a greeting
print("Hello!")
return
```
And replace them with the *space* and *plus* `+` lines:
```python
def main(args):
# show a greeting
print("Goodbye!")
return
```
Simple, right?
### Encourage high level edits
The example unified diffs we've seen so far have all been single line changes,
which makes them pretty easy to read and understand.
Consider this slightly more complex change, which renames the variable `n` to
`number`:
```diff
@@ ... @@
-def factorial(n):
+def factorial(number):
- if n == 0:
+ if number == 0:
return 1
else:
- return n * factorial(n-1)
+ return number * factorial(number-1)
```
The following "high level diff" of the same
change is not as succinct as the minimal diff above,
but it is much easier to see two different coherent versions of the
`factorial()` function.
```diff
@@ ... @@
-def factorial(n):
- if n == 0:
- return 1
- else:
- return n * factorial(n-1)
+def factorial(number):
+ if number == 0:
+ return 1
+ else:
+ return number * factorial(number-1)
```
Aider's system prompt encourages
GPT to produce these high level diffs.
This makes GPT better at producing correct diffs, which can be successfully
applied to the original file.
**Experiments without "high level diff" prompting
produce a 30-50% increase in editing errors,**
where diffs fail to apply or apply incorrectly and
produce invalid code.
When a patch fails, aider needs to ask GPT for a corrected version of the diff.
This takes time, costs tokens and sometimes fails to produce a successful edit
even after multiple retries.
There are probably a couple of reasons why high level diffs
help:
- It's easier to produce diffs that both correctly match the original code and correctly produce the intended new code. There is less risk of GPT getting confused, compared to generating a series of surgical edits that interleave lines of old and new code.
- High level hunks often contain more lines than a surgical hunk, so they are less likely to accidentally match unrelated parts of the code. This is helpful because GPT can't reliably give us line numbers to specify exactly where in the file to make changes.
### Be flexible when applying edits
GPT frequently makes imperfect diffs that won't apply cleanly.
They exhibit a variety of problems:
- GPT forgets things like comments, docstrings, blank lines, etc. Or it skips over some code that it doesn't intend to change.
- GPT forgets the leading *plus* `+` character to mark novel lines that it wants to add to the file. It incorrectly includes them with a leading *space* as if they were already there.
- GPT outdents all of the code, removing all the leading white space which is shared across the lines. So a chunk of deeply indented code is shown in a diff with only the leading white space that changes between the lines in the chunk.
- GPT jumps ahead to show edits to a different part of the file without starting a new hunk with a `@@ ... @@` divider.
As an example of the first issue, consider this source code:
```python
import sys
def main(args):
# show a greeting
print("Hello!")
return
main(sys.argv[1:])
```
**The diff below is missing the "show a greeting" comment line**,
and represents a common type of mistake GPT might make.
When we search for the *minus* `-` lines, we won't find them
in the original file
because of the missing comment.
```diff
@@ ... @@
-def main(args):
- print("Hello!")
- return
+def main(args):
+ print("Goodbye!")
+ return
```
Aider tries to be very flexible when applying diffs,
in order to handle defects.
If a hunk doesn't apply cleanly, aider uses a number of strategies:
- Normalize the hunk, by taking the *minus* `-` and *space* lines as one version of the hunk and the *space* and *plus* `+` lines as a second version and doing an actual unified diff on them.
- Try and discover new lines that GPT is trying to add but which it forgot to mark with *plus* `+` markers. This is done by diffing the *minus* `-` and *space* lines back against the original file.
- Try and apply the hunk using "relative leading white space", so we can match and patch correctly even if the hunk has been uniformly indented or outdented.
- Break a large hunk apart into an overlapping sequence of smaller hunks, which each contain only one contiguous run of *plus* `+` and *minus* `-` lines. Try and apply each of these sub-hunks independently.
- Vary the size and offset of the "context window" of *space* lines from the hunk that are used to localize the edit to a specific part of the file.
- Combine the above mechanisms to progressively become more permissive about how to apply the hunk.
These flexible patching strategies are critical, and
removing them
radically increases the number of hunks which fail to apply.
**Experiments where flexible patching is disabled show a 9X increase in editing errors** on aider's original Exercism benchmark.
## Refactoring benchmark
Aider has long used a
[benchmark suite based on 133 Exercism python exercises]().
But these are mostly small coding problems,
usually requiring only a few dozen lines of code.
GPT-4 Turbo is typically only lazy on 2-3 of these exercises:
the ones with the most code and which involve refactoring.
Based on this observation, I set out to build a benchmark based on refactoring
a non-trivial amount of code found in fairly large files.
To do this, I used python's `ast` module to analyze
[9 popular open source python repositories](https://github.com/paul-gauthier/refactor-benchmark)
to identify challenging refactoring tasks.
The goal was to find:
- Source files that contain classes with non-trivial methods, having 100-250+ AST nodes in their implementation.
- Focus on methods that are part of a larger class, which has at least twice as much code as the method itself.
- Select methods that don't use their `self` parameter, so they can be trivially refactored out of the class.
We can then turn each of these source files into a task for the benchmark,
where we ask GPT to do something like:
> Refactor the `_set_csrf_cookie` method in the `CsrfViewMiddleware` class to be a stand alone, top level function.
> Name the new function `_set_csrf_cookie`, exactly the same name as the existing method.
> Update any existing `self._set_csrf_cookie` calls to work with the new `_set_csrf_cookie` function.
A [simple python AST scanning script](https://github.com/paul-gauthier/aider/blob/main/benchmark/refactor_tools.py)
found 89 suitable files
and packaged them up as benchmark tasks.
Each task has a test
that checks if the refactor
was performed roughly correctly:
- The updated source file must parse as valid python, to detect misapplied edits which produce invalid code.
- The target method must now exist as a top-level function in the file.
- This new top-level function must contain approximately the same number of AST nodes as the original class method. This ensures that GPT didn't elide code and replace it with comments.
- The original class must still be present in the file, and it must be smaller by about the number of AST nodes in the method which was removed. This helps confirm that the method was removed from the class, without other significant modifications.
To be clear, this is not a rigorous test that the refactor was performed correctly.
But it does serve as a basic sanity check that the refactor was essentially done as a cut & paste, without eliding any code as comments.
And it correlates well with other laziness metrics
gathered during benchmarking like the
introduction of new comments that contain "...".
The result is a pragmatic
[benchmark suite that provokes, detects and quantifies GPT coding laziness](https://github.com/paul-gauthier/refactor-benchmark).
## Conclusions and future work
Based on the refactor benchmark results,
aider's new unified diff format seems
to dramatically increase GPT-4 Turbo's skill at more complex coding tasks.
It also seems very effective at reducing the lazy coding
which has been widely noted as a problem with GPT-4 Turbo.
Unified diffs was one of the very first edit formats I tried
when originally building aider.
I think a lot of other AI coding assistant projects have also
tried going down this path.
It seems like any naive or direct use of structured diff formats
is pretty much doomed to failure.
But the techniques described here and
incorporated into aider provide
a highly effective way to harness GPT's knowledge of unified diffs.
There could be significant benefits to
fine tuning models on
aider's simple, high level style of unified diffs.
Dropping line numbers from the hunk headers and focusing on diffs of
semantically coherent chunks of code
seems to be an important part of successful GPT code editing
(besides the relentless focus on flexibly applying edits).
Most LLMs will have already seen plenty of unified diffs
in their normal training data, and so should be
amenable to fining tuning towards this
particular diff style.

31
website/docs/usage.md Normal file
View file

@ -0,0 +1,31 @@
---
nav_order: 30
---
## Usage
Run `aider` with the source code files you want to edit.
These files will be "added to the chat session", so that the LLM can see their
contents and edit them according to your instructions.
```
aider <file1> <file2> ...
```
Be selective, and just add the files that the LLM will need to edit.
If you add a bunch of unrelated files, the LLM can get overwhelmed
and confused (and it costs more tokens).
Aider will automatically
share snippets from other, related files with the LLM so it can
[understand the rest of your code base](https://aider.chat/docs/repomap.html).
You can also just launch aider anywhere in a git repo without naming
files on the command line. It will discover all the files in the
repo. You can then add and remove individual files in the chat
session with the `/add` and `/drop` chat commands described below.
If you or the LLM mention any of the repo's filenames in the conversation,
aider will ask if you'd like to add them to the chat.
Aider also has many other options which can be set with
command line switches, environment variables or via a configuration file.
See `aider --help` for details.

119
website/docs/voice.md Normal file
View file

@ -0,0 +1,119 @@
---
nav_order: 100
---
# Voice-to-code with aider
Speak with GPT about your code! Request new features, test cases or bug fixes using your voice and let GPT do the work of editing the files in your local git repo. As with all of aider's capabilities, you can use voice-to-code with an existing repo or to start a new project.
Voice support fits quite naturally into aider's AI pair programming
chat interface. Now you can fluidly switch between voice and text chat
when you ask GPT to edit your code.
## How to use voice-to-code
Use the in-chat `/voice` command to start recording,
and press `ENTER` when you're done speaking.
Your voice coding instructions will be transcribed
and sent to GPT, as if you had typed them into
the aider chat session.
See the [installation instructions](https://aider.chat/docs/install.html#install-portaudio-optional) for
information on how to enable the `/voice` command.
<br/>
<div class="chat-transcript" markdown="1">
> Aider v0.11.2-dev
> Added app.py to the chat.
#### /voice
<blockquote>
<div class="recording-line">
<p style="display: inline;">Recording, press ENTER when done... <span id="timer">3.5</span>sec</p>
<div id="progress-bar">
<div id="progress"></div>
</div>
</div>
</blockquote>
<style>
.recording-line > * {
display: inline-block;
vertical-align: middle;
}
#progress-bar {
width: 100px;
height: 20px;
background-color: #333;
margin-left: 10px;
}
#progress {
height: 100%;
background-color: #f0f0f0;
animation: progress 3s infinite;
}
@keyframes progress {
0% { width: 10%; }
20% { width: 90%; }
50% { width: 30%; }
60% { width: 90%; }
80% { width: 20%; }
100% { width: 10%; }
}
</style>
<script>
window.onload = function() {
var counter = 0;
var timer = document.getElementById('timer');
setInterval(function() {
counter = (counter + 0.1) % 9;
timer.textContent = counter.toFixed(1);
}, 100);
}
</script>
<div style="display: inline-block; position: relative; width: 200px; background-color: #f0f0f0; border-radius: 10px; padding: 10px; color: #333; font-family: 'Comic Sans MS', 'Comic Sans', 'Marker Felt', sans-serif;">
<span style="position: absolute; top: 0; left: 50%; transform: translate(-50%, -100%); width: 0; height: 0; border-left: 10px solid transparent; border-right: 10px solid transparent; border-bottom: 10px solid #f0f0f0;"></span>
“&nbsp;add a factorial endpoint that uses math factorial&nbsp;”
</div>
#### Add a factorial endpoint that uses math.factorial.
To add a factorial endpoint to the Flask application, we need to modify the `app.py` file.
Here are the steps:
1. Add a new route `/fact/<int:x>` to the Flask application.
2. In the function for this route, call `math.factorial(x)` and return the result as a string.
Here is the *edit block* for these changes:
```python
app.py
<<<<<<< HEAD
if __name__ == '__main__':
print("Starting...")
app.run()
=======
@app.route('/fact/<int:x>')
def factorial(x):
result = math.factorial(x)
return str(result)
if __name__ == '__main__':
print("Starting...")
app.run()
>>>>>>> updated
```
> Applied edit to app.py
> Commit ef9e3e7 aider: Add a factorial endpoint that uses math.factorial.
</div>
<br/>
<br/>
<br/>