Commit graph

571 commits

Author SHA1 Message Date
Paul Gauthier
6b6548bd37 Merge remote-tracking branch 'origin/main' 2024-05-05 08:00:53 -07:00
paul-gauthier
fab8b8ae40
Update index.md 2024-05-04 20:07:32 -07:00
Paul Gauthier
1b35ca25c4 copy 2024-05-04 17:45:08 -07:00
Paul Gauthier
2d91ee8dbb copy 2024-05-04 17:33:25 -07:00
Paul Gauthier
425cb2941b Merge remote-tracking branch 'origin/main' 2024-05-04 16:26:36 -07:00
Paul Gauthier
366743493c renamed leaderboards 2024-05-04 16:25:46 -07:00
Paul Gauthier
812a620711 copy 2024-05-04 16:25:22 -07:00
paul-gauthier
df94f70a57
Update leaderboard.md 2024-05-04 11:32:43 -07:00
Paul Gauthier
b74edcf350 copy 2024-05-04 11:19:32 -07:00
Paul Gauthier
fbb3749270 copy 2024-05-04 11:14:15 -07:00
Paul Gauthier
ec07b6e556 updated refac 2024-05-04 11:11:34 -07:00
Paul Gauthier
e524dd9203 added refac leaderboard 2024-05-04 11:05:32 -07:00
Paul Gauthier
b02320b08f layout 2024-05-04 10:35:49 -07:00
Paul Gauthier
d96dc9ef63 Revert "aider: Change the graph to a horizontal bar graph."
This reverts commit 753ea441ea.
2024-05-04 10:35:26 -07:00
Paul Gauthier
26856c729e horizontal 2024-05-04 10:23:18 -07:00
Paul Gauthier
753ea441ea aider: Change the graph to a horizontal bar graph.
# Aider chat conversation:

USER: make the graph a horizontal bar graph

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
 <script>
   document.addEventListener('DOMContentLoaded', function () {
     var ctx = document.getElementById('leaderboardChart').getContext('2d');
     var leaderboardData = {
       labels: [],
       datasets: [{
         label: 'Percent correct',
         data: [],
         backgroundColor: 'rgba(54, 162, 235, 0.2)',
         borderColor: 'rgba(54, 162, 235, 1)',
         borderWidth: 1
       }]
     };

     {% for row in sorted %}
       leaderboardData.labels.push('{{ row.model }}');
       leaderboardData.datasets[0].data.push({{ row.second }});
     {% endfor %}

-    var leaderboardChart = new Chart(ctx, {
-      type: 'bar',
+    var leaderboardChart = new Chart(ctx, {
+      type: 'horizontalBar',
       data: leaderboardData,
       options: {
         scales: {
-          yAxes: [{
+          xAxes: [{
             scaleLabel: {
               display: true,
             },
             ticks: {
               beginAtZero: true
             }
           }]
         }
       }
     });
   });
 </script>
```
2024-05-04 10:22:54 -07:00
Paul Gauthier
e9228e0ed8 Update leaderboard chart with new label and style. 2024-05-04 10:22:53 -07:00
Paul Gauthier
9b4e643436 graph 2024-05-04 10:20:56 -07:00
Paul Gauthier
bc582723e3 aider: Add y-axis label "Percent correct" to the leaderboard chart.
# Aider chat conversation:

USER: add y-axis label "Percent correct"

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
 var leaderboardChart = new Chart(ctx, {
   type: 'bar',
   data: leaderboardData,
   options: {
     scales: {
       yAxes: [{
+        scaleLabel: {
+          display: true,
+          labelString: 'Percent correct'
+        },
         ticks: {
           beginAtZero: true
         }
       }]
     }
   }
 });
```
2024-05-04 10:20:42 -07:00
Paul Gauthier
e8ae216bbd graph 2024-05-04 10:19:46 -07:00
Paul Gauthier
98c8395d48 aider: Added Chart.js library to the docs/leaderboard.md file.
# Aider chat conversation:

USER: ReferenceError: Can't find variable: Chart

ASSISTANT: The error message "ReferenceError: Can't find variable: Chart" suggests that the Chart.js library is not being loaded before the script that is trying to use it. To fix this, we need to ensure that the Chart.js library is included in the HTML document before the script that initializes the chart.

Here is the diff to add the Chart.js library to the `docs/leaderboard.md` file:

```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
 <canvas id="leaderboardChart" width="800" height="450"></canvas>
+<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
 <script>
   document.addEventListener('DOMContentLoaded', function () {
     var ctx = document.getElementById('leaderboardChart').getContext('2d');
     var leaderboardData = {
       labels: [],
       datasets: [{
         label: 'Percent correct',
         data: [],
         backgroundColor: 'rgba(54, 162, 235, 0.2)',
         borderColor: 'rgba(54, 162, 235, 1)',
         borderWidth: 1
       }]
     };
@@ ... @@
```

This diff adds the Chart.js library from a CDN before the script tag that contains the chart initialization code. This should resolve the `ReferenceError` by ensuring that the `Chart` variable is defined when the script runs.
2024-05-04 10:16:08 -07:00
Paul Gauthier
9e8e47f1ce aider: Add script to draw a bar graph using Chart.js with model names as labels and percentages as bar heights in the docs/leaderboard.md file.
# Aider chat conversation:

USER: use chart.js to fill draw a bar graph in the canvas. is liquid templating to pull the "model" as bar label and "second" as bar height

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
-<canvas id="leaderboardChart" width="800" height="450"></canvas>
+<canvas id="leaderboardChart" width="800" height="450"></canvas>
+<script>
+  document.addEventListener('DOMContentLoaded', function () {
+    var ctx = document.getElementById('leaderboardChart').getContext('2d');
+    var leaderboardData = {
+      labels: [],
+      datasets: [{
+        label: 'Percent correct',
+        data: [],
+        backgroundColor: 'rgba(54, 162, 235, 0.2)',
+        borderColor: 'rgba(54, 162, 235, 1)',
+        borderWidth: 1
+      }]
+    };
+
+    {% for row in site.data.leaderboard | sort: 'second' | reverse %}
+      leaderboardData.labels.push('{{ row.model }}');
+      leaderboardData.datasets[0].data.push({{ row.second }});
+    {% endfor %}
+
+    var leaderboardChart = new Chart(ctx, {
+      type: 'bar',
+      data: leaderboardData,
+      options: {
+        scales: {
+          yAxes: [{
+            ticks: {
+              beginAtZero: true
+            }
+          }]
+        }
+      }
+    });
+  });
+</script>
```

This diff adds a script tag to the `docs/leaderboard.md` file that uses Chart.js to draw a bar graph on the canvas element with the id `leaderboardChart`. The script waits for the DOM to be fully loaded before executing. It then initializes the chart with the model names as labels and the corresponding percentages as the bar heights, using Liquid templating to pull the data from the site's data files.
2024-05-04 10:14:12 -07:00
Paul Gauthier
61a4c726a5 Updated leaderboard chart to dynamically fetch and populate data from a CSV file using Chart.js. 2024-05-04 10:14:11 -07:00
Paul Gauthier
c28d22f9b1 fixup 2024-05-04 10:12:06 -07:00
Paul Gauthier
6d9043cf24 aider: Added a bar chart using Chart.js to the leaderboard.md file.
# Aider chat conversation:

USER: add a bar chart using chart.js

ASSISTANT: To add a bar chart using Chart.js to the `leaderboard.md` file, you will need to include the Chart.js library, prepare a canvas element where the chart will be rendered, and write the JavaScript code to create the bar chart. Here's the diff that adds these elements to the `leaderboard.md` file:

```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
 <table style="width: 90%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1); font-size: 14px;">
   <!-- Table content -->
 </table>
+
+<canvas id="leaderboardChart" width="800" height="450"></canvas>
+
+<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
+<script>
+document.addEventListener('DOMContentLoaded', function () {
+  var ctx = document.getElementById('leaderboardChart').getContext('2d');
+  var leaderboardData = {
+    labels: [], // Populate with model names
+    datasets: [{
+      label: 'Percent correct',
+      data: [], // Populate with percent correct values
+      backgroundColor: 'rgba(54, 162, 235, 0.2)',
+      borderColor: 'rgba(54, 162, 235, 1)',
+      borderWidth: 1
+    }]
+  };
+
+  // Fetch leaderboard data and populate the chart
+  fetch('/_data/leaderboard.csv')
+    .then(function (response) {
+      return response.text();
+    })
+    .then(function (text) {
+      var rows = text.split('\n');
+      rows.forEach(function (row) {
+        var columns = row.split(',');
+        if (columns.length === 4) {
+          leaderboardData.labels.push(columns[0]);
+          leaderboardData.datasets[0].data.push(parseFloat(columns[1]));
+        }
+      });
+      new Chart(ctx, {
+        type: 'bar',
+        data: leaderboardData,
+        options: {
+          scales: {
+            y: {
+              beginAtZero: true
+            }
+          }
+        }
+      });
+    });
+});
+</script>
```

This diff adds a canvas element for the chart, includes the Chart.js library from a CDN, and provides a script that initializes the bar chart with data from the leaderboard CSV file. The script fetches the CSV data, parses it, and populates the chart with the model names and their corresponding "Percent correct" values.
2024-05-04 10:11:25 -07:00
Paul Gauthier
c9dbca9d0e gpt-4-0314 with examples as sys 2024-05-04 07:52:22 -07:00
Paul Gauthier
f6580fff76 updated all openai models 2024-05-04 07:38:50 -07:00
paul-gauthier
a496abbfa4
Update leaderboard.md 2024-05-03 18:46:35 -07:00
Paul Gauthier
6eeeb68049 layout 2024-05-03 18:06:47 -07:00
Paul Gauthier
1d1f53601d aider: Centered the "Percent correct" column in the leaderboard table.
# Aider chat conversation:

USER: center align the percent correct column

ASSISTANT: Here is the *SEARCH/REPLACE* block to center align the "Percent correct" column:

docs/leaderboard.md
```markdown
<<<<<<< SEARCH
      <th style="padding: 8px; text-align: left;">Percent correct</th>
=======
      <th style="padding: 8px; text-align: center;">Percent correct</th>
>>>>>>> REPLACE
```
2024-05-03 18:06:22 -07:00
paul-gauthier
b21952a71c
Update leaderboard.md 2024-05-03 17:59:25 -07:00
Paul Gauthier
bf94746614 copy 2024-05-03 17:37:41 -07:00
Paul Gauthier
6f20edd730 layout 2024-05-03 17:30:59 -07:00
Paul Gauthier
a2d16208ab aider: Replaced <pre> tag with <code> tag in leaderboard.md to denote literal code without boxing it.
# Aider chat conversation:

USER: what do the styles on the pre tag do?

ASSISTANT: The `<pre>` tag in the Markdown table is styled with the following CSS:

```css
pre {
  overflow-x: auto;
  white-space: pre-wrap;
}
```

Here's what those styles do:

- `overflow-x: auto;` adds a horizontal scrollbar to the `<pre>` block if the content is wider than the available space. This allows the user to scroll to see long commands without them wrapping or overflowing the table cell.

- `white-space: pre-wrap;` allows the content inside the `<pre>` to wrap to new lines if needed. Normally `<pre>` elements only honor line breaks that are explicitly in the content. With this style, the browser will wrap lines as needed to fit the available width, while still preserving the original line breaks and other whitespace.

So in summary, these styles allow the command examples in the table to be scrolled horizontally if long, while wrapping to multiple lines if needed to fit the cell. This provides a good balance of preserving the original formatting while keeping the content readable and not breaking the table layout.

USER: the pre is putting the text in a pale blue box. what's another way to denote literal code without boxing it?

ASSISTANT: Here are a couple options to denote literal code without using a `<pre>` tag that puts it in a box:

1. Use inline code formatting with backticks, like `this`. This works well for short code snippets within a sentence.

2. Use a `<code>` tag instead of `<pre>`. This will format the text in a monospace font to denote code, but won't put it in a box. You can add CSS to style it further if desired.

Here's a *SEARCH/REPLACE* block to make that change:

docs/leaderboard.md
```markdown
<<<<<<< SEARCH
        <td style="padding: 4px;"><pre style="overflow-x: auto; white-space: pre-wrap;">{{ row.command }}</pre></td>
=======
        <td style="padding: 4px;"><code>{{ row.command }}</code></td>
>>>>>>> REPLACE
```

This replaces the `<pre>` tag with a `<code>` tag. The code will now render inline without the pale blue box, but will still be visibly denoted as code.

You can remove the `overflow-x` and `white-space` styles since they are no longer needed on the `<code>` tag. The browser will wrap the code naturally within the table cell.

Let me know if you would like me to make this change or if you have any other questions!
2024-05-03 17:28:35 -07:00
Paul Gauthier
e99eaf5e90 aider: Reduced font size of table in leaderboard.md
# Aider chat conversation:

USER: make the font smaller for the table

ASSISTANT: Here is the *SEARCH/REPLACE* block to make the font smaller in the table:

docs/leaderboard.md
```markdown
<<<<<<< SEARCH
<table style="width: 90%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
=======
<table style="width: 90%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1); font-size: 14px;">
>>>>>>> REPLACE
```
2024-05-03 17:23:47 -07:00
Paul Gauthier
01cf0c43a0 layout 2024-05-03 17:21:04 -07:00
Paul Gauthier
8aa1e10f2a copy 2024-05-03 17:14:56 -07:00
paul-gauthier
f1871d104a
Update leaderboard.md 2024-05-03 16:27:51 -07:00
paul-gauthier
6f002bf68f
Update leaderboard.md 2024-05-03 16:14:45 -07:00
Paul Gauthier
052df34300 copy 2024-05-03 15:48:43 -07:00
Paul Gauthier
4b0dfc3d05 copy 2024-05-03 15:45:12 -07:00
Paul Gauthier
f3d9d8f08f aider: Styled the table to be 90% wide with a max-width of 800px, centered it, added a box shadow, and made other visual improvements.
# Aider chat conversation:

USER: add some styling to the table. it's very plain right now. it also is a bit too wide for the page and is forcing the user to scroll left/right.

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
-<table>
+<table style="width: 100%; max-width: 960px; margin: auto; border-collapse: collapse;">
@@ ... @@
-  <thead>
+  <thead style="background-color: #f2f2f2;">
@@ ... @@
-    <tr>
+    <tr style="border-bottom: 1px solid #ddd;">
@@ ... @@
-      <th>Model</th>
-      <th>Percent correct</th>
-      <th>Command</th>
-      <th>Edit format</th>
+      <th style="padding: 8px; text-align: left;">Model</th>
+      <th style="padding: 8px; text-align: left;">Percent correct</th>
+      <th style="padding: 8px; text-align: left;">Command</th>
+      <th style="padding: 8px; text-align: left;">Edit format</th>
@@ ... @@
-      <tr>
+      <tr style="border-bottom: 1px solid #ddd;">
@@ ... @@
-        <td>{{ row.model }}</td>
-        <td>{{ row.second }}%</td>
-        <td><pre>{{ row.command }}</pre></td>
-        <td>{{ row.format }}</td>
+        <td style="padding: 8px;">{{ row.model }}</td>
+        <td style="padding: 8px;">{{ row.second }}%</td>
+        <td style="padding: 8px;"><pre style="overflow-x: auto; white-space: pre-wrap;">{{ row.command }}</pre></td>
+        <td style="padding: 8px;">{{ row.format }}</td>
```

This diff adds some basic styling to the table to make it look less plain and ensures that it fits within the page without causing horizontal scrolling. The `max-width` property is set to `960px`, which is a common width for content on web pages, and `margin: auto` centers the table on the page. The `border-collapse` property is used to make the borders between cells look cleaner. The `background-color` for the table header improves the visual separation of the header from the body. Padding is added to the cells for better readability, and the `overflow-x: auto; white-space: pre-wrap;` styles on the `<pre>` tag ensure that the command text does not force the table to be wider than the page.

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ ... @@
-<table>
+<table style="width: 100%; max-width: 960px; margin: auto; border-collapse: collapse;">
@@ ... @@
-  <thead>
+  <thead style="background-color: #f2f2f2;">
@@ ... @@
-    <tr>
+    <tr style="border-bottom: 1px solid #ddd;">
@@ ... @@
-      <th>Model</th>
-      <th>Percent correct</th>
-      <th>Command</th>
-      <th>Edit format</th>
+      <th style="padding: 8px; text-align: left;">Model</th>
+      <th style="padding: 8px; text-align: left;">Percent correct</th>
+      <th style="padding: 8px; text-align: left;">Command</th>
+      <th style="padding: 8px; text-align: left;">Edit format</th>
@@ ... @@
-      <tr>
+      <tr style="border-bottom: 1px solid #ddd;">
@@ ... @@
-        <td>{{ row.model }}</td>
-        <td>{{ row.second }}%</td>
-        <td><pre>{{ row.command }}</pre></td>
-        <td>{{ row.format }}</td>
+        <td style="padding: 8px;">{{ row.model }}</td>
+        <td style="padding: 8px;">{{ row.second }}%</td>
+        <td style="padding: 8px;"><pre style="overflow-x: auto; white-space: pre-wrap;">{{ row.command }}</pre></td>
+        <td style="padding: 8px;">{{ row.format }}</td>
```

This diff adds some basic styling to the table to make it look less plain and ensures that it fits within the page without causing horizontal scrolling. The `max-width` property is set to `960px`, which is a common width for content on web pages, and `margin: auto` centers the table on the page. The `border-collapse` property is used to make the borders between cells look cleaner. The `background-color` for the table header improves the visual separation of the header from the body. Padding is added to the cells for better readability, and the `overflow-x: auto; white-space: pre-wrap;` styles on the `<pre>` tag ensure that the command text does not force the table to be wider than the page.

USER: UnifiedDiffNoMatch: hunk failed to apply!

docs/leaderboard.md does not contain lines that match the diff you provided!
Try again.
DO NOT skip blank lines, comments, docstrings, etc!
The diff needs to apply cleanly to the lines in docs/leaderboard.md!

docs/leaderboard.md does not contain these 1 exact lines in a row:
```
    <tr>
```

ASSISTANT: ```diff
--- docs/leaderboard.md
+++ docs/leaderboard.md
@@ -1,4 +1,4 @@
-<table style="width: 100%; max-width: 960px; margin: auto; border-collapse: collapse;">
+<table style="width: 90%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
   <thead style="background-color: #f2f2f2;">
     <tr>
       <th style="padding: 8px; text-align: left;">Model</th>
```
2024-05-03 15:33:42 -07:00
Paul Gauthier
d7ddeab294 Updated leaderboard table styling for better readability and aesthetics. 2024-05-03 15:33:41 -07:00
Paul Gauthier
0e79b3dcc6 Update leaderboard to display command in a preformatted block. 2024-05-03 15:33:32 -07:00
Paul Gauthier
471d637694 updated llama3 2024-05-03 15:31:20 -07:00
Paul Gauthier
b476671399 copy 2024-05-03 15:08:38 -07:00
Paul Gauthier
cb42150bba added leaderboard 2024-05-03 14:52:21 -07:00
Paul Gauthier
4825808f85 aider: Added an HTML table to display leaderboard data from _data/leaderboard.csv in the Jekyll site.
# Aider chat conversation:

USER: make an html table for this jekyll site pulling data from `_data/leaderboard.csv`

ASSISTANT: Here is the *SEARCH/REPLACE* block to add an HTML table to `docs/leaderboard.md` using data from `_data/leaderboard.csv`:

docs/leaderboard.md
```markdown
<<<<<<< SEARCH
```
=======
<table>
  <thead>
    <tr>
      <th>Model</th>
      <th>Second</th>
      <th>First</th>
      <th>Format</th>
      <th>Command</th>
      <th>Version</th>
      <th>Commit</th>
      <th>Date</th>
    </tr>
  </thead>
  <tbody>
    {% for row in site.data.leaderboard %}
      <tr>
        <td>{{ row.model }}</td>
        <td>{{ row.second }}</td>
        <td>{{ row.first }}</td>
        <td>{{ row.format }}</td>
        <td>{{ row.command }}</td>
        <td>{{ row.version }}</td>
        <td>{{ row.commit }}</td>
        <td>{{ row.date }}</td>
      </tr>
    {% endfor %}
  </tbody>
</table>
>>>>>>> REPLACE
```

This will:

1. Add an HTML `<table>` element
2. Define the table header row with column names matching the CSV
3. Loop through `site.data.leaderboard` (loaded from the CSV file)
4. Output a `<tr>` table row for each data row
5. Output `<td>` table cells for each column value

When rendered by Jekyll, this will display the leaderboard data in an HTML table on the page.
2024-05-03 14:51:31 -07:00
Paul Gauthier
a34f1a6529 Added documentation for the leaderboard feature. 2024-05-03 14:51:30 -07:00
Paul Gauthier
3e5f0d0c75 copy 2024-05-02 11:34:38 -07:00