From 19073dd93904ee51b4a10d124d53b2a4432ddcc4 Mon Sep 17 00:00:00 2001 From: Paul Gauthier Date: Thu, 15 Aug 2024 10:08:13 -0700 Subject: [PATCH] feat: Add section on overall coding skill and syntax errors to blog post on code in JSON --- aider/website/_posts/2024-08-14-code-in-json.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/aider/website/_posts/2024-08-14-code-in-json.md b/aider/website/_posts/2024-08-14-code-in-json.md index 9f3345971..601fab485 100644 --- a/aider/website/_posts/2024-08-14-code-in-json.md +++ b/aider/website/_posts/2024-08-14-code-in-json.md @@ -270,6 +270,9 @@ model & strategy combination. ## Results + +## Overall coding skill + All of the models did worse on the benchmark when asked to return JSON-wrapped code in a tool function call. Most did significantly worse, performing far below @@ -285,6 +288,9 @@ as compared to non-strict JSON. Of course, both JSON results were well below the markdown result. - The results from Sonnet and DeepSeek Coder suffered the worst harm from JSON wrapping. +## Syntax errors + + ## Conclusions While the quantitative results differ from the similar