From 4b0dfc3d0582b51477954f28cd89052fa8373343 Mon Sep 17 00:00:00 2001
From: Paul Gauthier <aider@paulg.org>
Date: Fri, 3 May 2024 15:45:12 -0700
Subject: [PATCH] copy

---
 docs/leaderboard.md | 40 ++++++++++++++++++++++++++++++++++++++++
 docs/llms.md        |  3 ++-
 2 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/docs/leaderboard.md b/docs/leaderboard.md
index 09d97f20a..b60da73fb 100644
--- a/docs/leaderboard.md
+++ b/docs/leaderboard.md
@@ -1,6 +1,20 @@
 
 # Aider's LLM code editing leaderboard
 
+Aider works best with LLMs which are good at *editing* code, not just good at writing
+code.
+Aider works with the LLM to make changes to the existing code in your git repo,
+so the LLM needs to be able to specify how to edit that code.
+
+Aider uses a
+[code editing benchmark](https://aider.chat/docs/benchmarks.html#the-benchmark)
+to measure an LLM's code editing ability.
+This table reports the results from a number of popular LLMs,
+to help users select which models to use with aider.
+While [aider can connect to almost any LLM](https://aider.chat/docs/llms.html)
+it will work best with models that score well on the code editing benchmark.
+
+
 <table style="width: 90%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
   <thead style="background-color: #f2f2f2;">
     <tr>
@@ -21,3 +35,29 @@
     {% endfor %}
   </tbody>
 </table>
+
+
+# Edit format
+
+
+Aider uses different "edit formats" to collect code edits from different LLMs:
+
+- `whole` is a "whole file" editing format, where the model edits a file by returning a full new copy of the file with any changes included.
+- `diff` is a more efficient diff style format, where the model specifies blocks of code to search and replace in order to made changes to files.
+- `diff-fenced` is similar to diff, but fences the entire diff block including the filename.
+- `udiff` is the most efficient editing format, where the model returns unified diffs to apply changes to the file.
+
+Different models work best with different editing formats.
+Aider is configured to use the best edit format for the popular OpenAI and Anthropic models
+and the [other models recommended on the LLM page](https://aider.chat/docs/llms.html).
+
+For lesser known models aider will default to using the "whole" editing format
+since it is the easiest format for an LLM to use.
+But it is also the least efficient, requiring the model to output the
+entire source file for every set of changes.
+This can raise costs and cause errors with LLMs that have smaller
+context window sizes.
+
+All of the other diff-like edit formats are much more efficient with their use of tokens.
+
+
diff --git a/docs/llms.md b/docs/llms.md
index 28dc55ceb..ec5420191 100644
--- a/docs/llms.md
+++ b/docs/llms.md
@@ -415,10 +415,11 @@ in case you made a typo or mistake when specifying the model name.
 
 ## Editing format
 
-Aider uses 3 different "edit formats" to collect code edits from different LLMs:
+Aider uses different "edit formats" to collect code edits from different LLMs:
 
 - `whole` is a "whole file" editing format, where the model edits a file by returning a full new copy of the file with any changes included.
 - `diff` is a more efficient diff style format, where the model specifies blocks of code to search and replace in order to made changes to files.
+- `diff-fenced` is similar to diff, but fences the entire diff block including the filename.
 - `udiff` is the most efficient editing format, where the model returns unified diffs to apply changes to the file.
 
 Different models work best with different editing formats.