dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-30 17:24:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

feat: Add code-in-json-benchmark.js file and update code-in-json.md post

Browse source
This commit is contained in:
Paul Gauthier 2024-08-15 10:50:51 -07:00 • committed by Paul Gauthier (aider)
parent 8d4d549a98
commit e90642295d
2 changed files with 7 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

0
aider/website/_includes/code-in-json-benchmark.js Normal file
View file

16
aider/website/_posts/2024-08-14-code-in-json.md
View file

@ -157,21 +157,19 @@ function createStripedCanvas(isStrict) {
## Abstract
The newest LLMs have support for returning properly formatted JSON responses,
making it easier for clients to parse complex responses.
This makes it tempting for AI coding applications to
use JSON replies to
receive code from LLMs.
Current LLMs have support for returning properly formatted JSON,
making it easier for clients to reliably parse complex responses.
It therefore seems attractive for
AI coding applications ask LLMs to return code in structure JSON replies.
Unfortunately,
LLMs write worse code when asked to wrap it in JSON, harming their ability
to correctly solve coding tasks.
On a variant of the aider code editing benchmark,
JSON-wrapping code
asking for JSON-wrapped code
often significantly harms coding
performance
compared to returning code as plain text.
performance.
This holds true across many top coding LLMs,
including OpenAI's new gpt-4o-2024-08-06
including OpenAI's latest model gpt-4o-2024-08-06
which has strong JSON support.
## Introduction