From 5cd76983b6aea9011926a72ac7bbb128614dd752 Mon Sep 17 00:00:00 2001 From: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Date: Fri, 30 Jun 2023 19:56:58 -0700 Subject: [PATCH 1/3] Update benchmarks.md --- docs/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 6f0523600..4ff4fe070 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -56,7 +56,7 @@ changes as `diff -c` formatted updates. Using more complex output formats seem to cause two problems: - It makes GPT write worse code. Keeping the output format simple seems to leave GPT with more attention to devote to the actual coding task. - - It makes GPT less likely to adhere to the output format. + - It makes GPT less likely to adhere to the output format. This makes it harder to correctly identify and apply the edits it is trying to make. I had hoped that the new function calling API would enable more reliable use of structured output formats, but it does not appear to be a panacea From 8dfb8d569ec4da8b9d8c16c0ea782ae1fb7462c1 Mon Sep 17 00:00:00 2001 From: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Date: Fri, 30 Jun 2023 19:58:48 -0700 Subject: [PATCH 2/3] Update benchmarks.md --- docs/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 4ff4fe070..79c0ec856 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -106,7 +106,7 @@ The tests are correct. Fix the code in to resolve the errors. ``` -Editing the implementation in response to test failures is +Editing the implementation file in response to test failures is another way that this benchmark measures how well GPT can perform code editing. This second chance is also important because many of the unit tests check for specifics that are not From 76ab693aac75af0f190140209e11278d7d2f5a53 Mon Sep 17 00:00:00 2001 From: paul-gauthier <69695708+paul-gauthier@users.noreply.github.com> Date: Fri, 30 Jun 2023 20:04:44 -0700 Subject: [PATCH 3/3] Update benchmarks.md --- docs/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 79c0ec856..a38abd78c 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -8,7 +8,7 @@ code in your local git repos. You can use aider to ask GPT to add features, write tests or make other changes and improvements to your code. -Having a reliable way for GPT to read/modify/write +Having a reliable way for GPT to edit local source code files is a critical component of this functionality. Making code editing more reliable often involves changing and experimenting with