This commit is contained in:
Paul Gauthier 2024-09-12 14:27:35 -07:00
parent 1755d2e0f4
commit 09cb4c4b09
3 changed files with 13 additions and 5 deletions

View file

@ -1104,7 +1104,7 @@
indentation_errors: 0 indentation_errors: 0
exhausted_context_windows: 0 exhausted_context_windows: 0
test_timeouts: 1 test_timeouts: 1
command: (not currently supported) command: aider --model openai/o1-mini --no-stream
date: 2024-09-12 date: 2024-09-12
versions: 0.56.1.dev versions: 0.56.1.dev
seconds_per_case: 103.0 seconds_per_case: 103.0

View file

@ -87,7 +87,7 @@
indentation_errors: 0 indentation_errors: 0
exhausted_context_windows: 0 exhausted_context_windows: 0
test_timeouts: 1 test_timeouts: 1
command: (not currently supported) command: aider --model openai/o1-mini --no-stream
date: 2024-09-12 date: 2024-09-12
versions: 0.56.1.dev versions: 0.56.1.dev
seconds_per_case: 103.0 seconds_per_case: 103.0

View file

@ -27,9 +27,17 @@ efficiently edit the source code.
The whole format requires the o1-mini to return a fresh copy of the entire file, The whole format requires the o1-mini to return a fresh copy of the entire file,
increasing costs and latency. increasing costs and latency.
OpenAI o1-mini does not support system prompts or temperature settings, OpenAI o1-mini does not support system prompts, temperature settings or streaming.
so aider was modified to complete this benchmark. Aider has been updated to work within these constraints.
The released aider will not yet work with o1-mini. To use o1-mini with aider you need to run the latest version from the main branch:
```
aider --install-main-branch
# or...
python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git
aider --model openai/o1-mini --no-stream
```
{: .note } {: .note }
> These are *preliminiary* benchmark results, which will be updated as > These are *preliminiary* benchmark results, which will be updated as