feat: Improve graph visualization and add debugging
The changes made in this commit include:
1. Adjusting the y-axis limit to 100 to accommodate the higher pass rate values.
2. Rotating the x-axis labels for better readability.
3. Adding debug print statements to track the progress of figure generation and display.
4. Increasing the figure size for better visibility.
5. Adding additional debugging to ensure the data is being plotted correctly.
These improvements should help with the visualization and debugging of the graph generation process.
# Aider chat conversation:
USER: add more white space below the graph, to give more room for the note
ASSISTANT: Here is the change to add more white space below the graph:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
plt.tight_layout(pad=3.0)
=======
plt.tight_layout(pad=3.0, rect=[0, 0.05, 1, 1])
>>>>>>> REPLACE
```
This adjusts the `tight_layout` call to reserve 5% of the figure height at the bottom for the note, using the `rect` parameter. The `rect` tuple specifies the left, bottom, right and top of the area the subplots should occupy as a fraction of figure width and height.
# Aider chat conversation:
USER: add some white space at the bottom to make more room for the note
ASSISTANT: To add more white space at the bottom of the plot to make room for the note, we can adjust the `plt.tight_layout` call to include more padding.
Here is the *SEARCH/REPLACE* block:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
plt.tight_layout(pad=3.0)
=======
plt.tight_layout(pad=5.0)
>>>>>>> REPLACE
```
# Aider chat conversation:
USER: no, add it to the bottom of the graph
ASSISTANT: To add a note at the bottom of the graph explaining that (570) and (2294) refer to the number of instances processed by the agent, we can use the `text` function from Matplotlib to place the note at the bottom of the plot.
Here is the *SEARCH/REPLACE* block to make this change:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
ax.xaxis.label.set_color(font_color)
=======
ax.xaxis.label.set_color(font_color)
# Add note at the bottom of the graph
note = "(570) and (2294) refer to the number of instances that were processed by the agent."
plt.figtext(0.5, 0.01, note, wrap=True, horizontalalignment='center', fontsize=10, color=font_color)
>>>>>>> REPLACE
```
# Aider chat conversation:
USER: add a note at the bottom that explains that (570) and (2294) refer to the number of instances that were processed by the agent
ASSISTANT: Here is the change to add a note at the bottom of the script explaining that (570) and (2294) refer to the number of instances processed by the agent.
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
ax.xaxis.label.set_color(font_color)
=======
ax.xaxis.label.set_color(font_color)
# Note: (570) and (2294) refer to the number of instances that were processed by the agent.
>>>>>>> REPLACE
```
# Aider chat conversation:
USER: reverse the order of the bars, so they go short->tall
ASSISTANT: To reverse the order of the bars from short to tall, we can reverse the order of the `models` and `pass_rates` lists before plotting:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
models = []
pass_rates = []
for line in lines:
if line.strip():
pass_rate, model = line.split("%")
model = model.strip()
model = model.replace("|", "\n")
models.append(model.strip())
pass_rates.append(float(pass_rate.strip()))
=======
models = []
pass_rates = []
for line in lines:
if line.strip():
pass_rate, model = line.split("%")
model = model.strip()
model = model.replace("|", "\n")
models.insert(0, model.strip())
pass_rates.insert(0, float(pass_rate.strip()))
>>>>>>> REPLACE
```
This change uses `insert(0, ...)` instead of `append()` to add each model and pass rate to the beginning of their respective lists. This will effectively reverse the order, so the bars will be plotted from shortest to tallest.
# Aider chat conversation:
USER: there's no visible difference bwteen the bold and light fonts?
ASSISTANT: To make the bold font more visible, we can increase the font weight. Here are the changes:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
fontweight = "bold" if "Aider" in model else "normal"
=======
fontweight = 800 if "Aider" in model else "normal"
>>>>>>> REPLACE
```
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
label.set_fontweight("bold")
=======
label.set_fontweight(800)
>>>>>>> REPLACE
```
This sets the font weight to 800 for the "Aider" labels, which should make them bolder and more visibly distinct from the other labels.
# Aider chat conversation:
USER: there's no visible difference bwteen the bold and light fonts?
ASSISTANT: To ensure that the bold and light fonts are visibly different, we can use "normal" instead of "light" for the non-Aider bars. This should provide a more noticeable contrast.
Here are the *SEARCH/REPLACE* blocks:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
fontweight = "bold" if "Aider" in model else "light"
=======
fontweight = "bold" if "Aider" in model else "normal"
>>>>>>> REPLACE
```
Let's make this change and see if it improves the visibility of the font differences.
# Aider chat conversation:
USER: make the percentages on top of aiders bars bold. make the aider x labels bold too
ASSISTANT: To make the percentages on top of Aider's bars bold and the Aider x labels bold, we need to modify the text properties in the plotting section.
Here are the *SEARCH/REPLACE* blocks:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
if "Aider" in model:
y = yval + 0.75
va = "bottom"
else:
y = yval - 1.25
va = "top"
ax.text(
bar.get_x() + bar.get_width() / 2,
y,
f"{yval}%",
ha="center",
va=va,
fontsize=14,
# alpha=0.75,
)
=======
y = yval + 0.75 if "Aider" in model else yval - 1.25
va = "bottom" if "Aider" in model else "top"
fontweight = "bold" if "Aider" in model else "normal"
ax.text(
bar.get_x() + bar.get_width() / 2,
y,
f"{yval}%",
ha="center",
va=va,
fontsize=14,
fontweight=fontweight,
)
>>>>>>> REPLACE
```
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
plt.xticks(
# rotation=45,
# ha="center",
fontsize=16,
color=font_color,
)
=======
xticks = plt.xticks(
fontsize=16,
color=font_color,
)
for label in xticks[1]:
if "Aider" in label.get_text():
label.set_fontweight("bold")
>>>>>>> REPLACE
```
# Aider chat conversation:
USER: can we use more alpha for the aider bars and less alpha for the other bars?
ASSISTANT: To adjust the alpha values for the bars, we need to modify the section where the bars are created and set their alpha values based on whether the model name contains "Aider".
Here is the *SEARCH/REPLACE* block:
benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
bars = ax.bar(models, pass_rates, color=colors, alpha=0.75, zorder=3)
=======
bars = []
for model, pass_rate, color in zip(models, pass_rates, colors):
alpha = 0.9 if "Aider" in model else 0.5
bar = ax.bar(model, pass_rate, color=color, alpha=alpha, zorder=3)
bars.append(bar[0])
>>>>>>> REPLACE
```
