39 commits

Author	SHA1	Message	Date
Paul Gauthier (aider)	6badf5ea1d	feat: Add cumulative sum column to distribution table	2024-12-18 12:47:14 -08:00
Paul Gauthier	20f5f3da24	chore: Remove unused import	2024-12-18 12:47:13 -08:00
Paul Gauthier (aider)	8c1b147705	style: Fix string formatting in problem stats	2024-12-18 12:46:02 -08:00
Paul Gauthier (aider)	366155b828	fix: Correct language lookup in problem_stats.py	2024-12-18 12:45:58 -08:00
Paul Gauthier (aider)	2c7d1897eb	style: Fix linting issues in problem_stats.py	2024-12-18 12:43:08 -08:00
Paul Gauthier (aider)	26ccb23402	feat: Add table showing exercise solution distribution	2024-12-18 12:43:04 -08:00
Paul Gauthier	d9e2471fcd	refactor: Remove unused dump calls in benchmark script	2024-12-18 12:43:03 -08:00
Paul Gauthier (aider)	8302b351dd	style: Fix line length in problem_stats.py	2024-12-18 12:38:45 -08:00
Paul Gauthier (aider)	b8647c0481	feat: Show exercises solved by all and total breakdown	2024-12-18 12:38:41 -08:00
Paul Gauthier	a168403d68	fix: Correctly extract language and testcase from results	2024-12-18 12:38:40 -08:00
Paul Gauthier (aider)	42d8650058	fix: Remove "exercises/" prefix from testcase output	2024-12-18 12:33:41 -08:00
Paul Gauthier (aider)	7ad0d46c11	style: Format problem_stats.py with linter	2024-12-18 12:31:54 -08:00
Paul Gauthier (aider)	58812f7f1f	refactor: Sort all exercises by solve rate, not by language	2024-12-18 12:31:50 -08:00
Paul Gauthier (aider)	65133b2aef	fix: Handle missing testcase key in problem stats	2024-12-18 12:29:38 -08:00
Paul Gauthier (aider)	291d8cd335	style: Run linter	2024-12-18 12:29:20 -08:00
Paul Gauthier (aider)	7a27e2b94b	fix: Handle missing testcase key in problem stats	2024-12-18 12:29:15 -08:00
Paul Gauthier	57a8eab1c3	chore: Add dump for debugging	2024-12-18 12:29:14 -08:00
Paul Gauthier (aider)	236a7f68e9	fix: Handle empty results in problem_stats.py	2024-12-18 12:26:51 -08:00
Paul Gauthier (aider)	81d424f475	fix: Remove unused max_lang_len variable	2024-12-18 12:25:47 -08:00
Paul Gauthier (aider)	687ba8c9a2	style: Apply linter fixes	2024-12-18 12:25:39 -08:00
Paul Gauthier (aider)	6d74a564e6	feat: Handle new exercise dir layout and add language info	2024-12-18 12:25:33 -08:00
Paul Gauthier (aider)	0a3e0665ab	style: Apply linter formatting	2024-12-18 12:21:19 -08:00
Paul Gauthier (aider)	a19f1fbc67	feat: Allow specifying dirs on cmd line for problem_stats	2024-12-18 12:21:13 -08:00
Paul Gauthier (aider)	5c55453a0e	fix: Correctly calculate and display never solved exercises	2024-12-17 18:14:48 -08:00
Paul Gauthier (aider)	59308c20c6	feat: Number exercises in the table	2024-12-17 14:15:40 -08:00
Paul Gauthier (aider)	cac5d8e716	style: Apply linter formatting	2024-12-17 14:15:06 -08:00
Paul Gauthier (aider)	7f16757bbe	fix: Handle missing results in topn leaderboard calculation	2024-12-17 14:15:02 -08:00
Paul Gauthier (aider)	674e3846e2	fix: Correctly sort leaderboard by pass rate	2024-12-17 14:13:43 -08:00
Paul Gauthier (aider)	3a0be0cca9	style: Apply linter formatting	2024-12-17 14:13:19 -08:00
Paul Gauthier (aider)	00d7c3a05a	feat: Add --topn argument to limit models by pass rate	2024-12-17 14:13:16 -08:00
Paul Gauthier (aider)	91f5fca5e9	feat: Include never solved exercises in stats	2024-12-17 14:10:47 -08:00
Paul Gauthier (aider)	1d7cb0c119	feat: Format problem stats output as a table with percentages	2024-12-17 14:10:00 -08:00
Paul Gauthier (aider)	24599aa64f	style: Run linter on problem_stats.py	2024-12-17 14:09:20 -08:00
Paul Gauthier (aider)	54c1553892	refactor: Remove distribution of solutions table	2024-12-17 14:09:17 -08:00
Paul Gauthier (aider)	0ae53ce1a1	feat: Output per-exercise stats, sort by solvers	2024-12-17 14:08:47 -08:00
Paul Gauthier	c69ffe02f8	chore: Make problem_stats.py executable	2024-12-17 14:08:46 -08:00
Paul Gauthier (aider)	7bfc2e0e74	style: Run linter on benchmark script	2024-12-17 14:06:56 -08:00
Paul Gauthier (aider)	9cc674c283	feat: Add script to analyze exercise solution stats	2024-12-17 14:06:53 -08:00
Paul Gauthier	66e597a05c	feat: Add problem stats benchmark	2024-12-17 14:06:52 -08:00