dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-31 01:35:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Update benchmarks-1106.md

Browse source
This commit is contained in:
paul-gauthier 2023-11-06 19:50:23 -08:00 • committed by GitHub
parent a2d52536a5
commit c8b95b486f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/benchmarks-1106.md
View file

@ -16,9 +16,9 @@ For example,
whenever I change aider's prompting or the backend which drives LLM conversations,
I run the benchmark to make sure these changes produce improvements (not regressions).
The benchmark asks GPT to complete the
[Exercism Python coding exercises](https://github.com/exercism/python).
Exercism provides a starting python file with stubs for the needed functions,
The benchmark asks GPT to complete
[133 Exercism Python coding exercises](https://github.com/exercism/python).
For each exercise, Exercism provides a starting python file with stubs for the needed functions,
a natural language description of the problem to solve
and a test suite to evaluate whether the coder has correctly solved the problem.