mirror of
https://github.com/Aider-AI/aider.git
synced 2025-05-30 17:24:59 +00:00
feat: Add documentation for infinite output
This commit is contained in:
parent
8ebd519615
commit
d9c159fe7a
1 changed files with 48 additions and 0 deletions
48
aider/website/docs/more/infinite-output.md
Normal file
48
aider/website/docs/more/infinite-output.md
Normal file
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
parent: More info
|
||||
nav_order: 920
|
||||
description: Aider can handle "infinite output" from models that support prefill.
|
||||
---
|
||||
|
||||
# Infinite output
|
||||
|
||||
LLM providers limit how much output a model can generate from a single request.
|
||||
This is usually called the output token limit.
|
||||
|
||||
Aider is able to work around this limit with models that support
|
||||
"prefilling" the assistant response.
|
||||
When you use aider with a model that supports prefill, you will see
|
||||
"infinite output" noted in the announcement lines displayed at launch:
|
||||
|
||||
```
|
||||
Aider v0.58.0
|
||||
Main model: claude-3-5-sonnet-20240620 with diff edit format, prompt cache, infinite output
|
||||
```
|
||||
|
||||
Models that support prefill can be primed to think they started their response
|
||||
with a specific piece of text.
|
||||
You can put words in their mouth, and they will continue generating
|
||||
text from that point forward.
|
||||
|
||||
When aider is collecting code edits from a model and
|
||||
it hits the output token limit,
|
||||
aider simply initiates another LLM request with the partial
|
||||
response prefilled.
|
||||
This prompts the model to continue where it left off,
|
||||
generating more of the desired response.
|
||||
This prefilling of the partially completed response can be repeated,
|
||||
allowing for very long outputs.
|
||||
Joining the text across these output limit boundaries
|
||||
requires some heuristics, but is typically fairly reliable.
|
||||
|
||||
Aider supports "infinite output" for models that support "prefill",
|
||||
such as:
|
||||
|
||||
<!--[[[cog
|
||||
# TODO: list the models
|
||||
cog.out(text)
|
||||
]]]-->
|
||||
|
||||
<!--[[[end]]]-->
|
||||
|
||||
|
Loading…
Add table
Add a link
Reference in a new issue