dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-01 10:14:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

copy

Browse source
This commit is contained in:
Paul Gauthier 2023-07-01 15:21:21 -07:00
parent 2c87b5b9a1
commit 51bc71446e
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/benchmarks.md
View file

@ -43,7 +43,7 @@ The results were quite interesting:
- Using the new function calling API performed worse than the above whole file method for all models. GPT-3.5 especially produced inferior code and frequently mangled this output format. This was surprising, as the functions API was introduced to enhance the reliability of structured outputs. The results from these `func` edit methods are shown as patterned bars in the graph (both green and blue). - Using the new function calling API performed worse than the above whole file method for all models. GPT-3.5 especially produced inferior code and frequently mangled this output format. This was surprising, as the functions API was introduced to enhance the reliability of structured outputs. The results from these `func` edit methods are shown as patterned bars in the graph (both green and blue).
- As expected, the GPT-4 models outperformed the GPT-3.5 models in code editing. - As expected, the GPT-4 models outperformed the GPT-3.5 models in code editing.
The quantitative benchmark results align with my developing intuition The quantitative benchmark results align with my intuitions
about prompting GPT for complex tasks like coding. It's beneficial to about prompting GPT for complex tasks like coding. It's beneficial to
minimize the "cognitive overhead" of formatting the response, allowing minimize the "cognitive overhead" of formatting the response, allowing
GPT to concentrate on the task at hand. As an analogy, asking a junior GPT to concentrate on the task at hand. As an analogy, asking a junior