Commit graph

4237 commits

Author SHA1 Message Date
Paul Gauthier
c9349d6e23 Updated SWE Bench Lite to display pass rates by model with adjusted colors and title. 2024-05-22 15:13:00 -07:00
Paul Gauthier
61019093a2 aider: Added code to plot a vertical bar graph of data from tmp.txt in swe_bench_lite.py.
# Aider chat conversation:

USER: plot a vertical bar graph of the data in tmp.txt. use similar fonts, colors and graph style to the one in over_time.py. put the new graph code in swe_bench_lite.py

ASSISTANT: Here is the code to plot a vertical bar graph of the data in `tmp.txt` using similar fonts, colors, and graph style to the one in `over_time.py`. This code will be placed in `swe_bench_lite.py`.

benchmark/swe_bench_lite.py
```python
<<<<<<< SEARCH
=======
import matplotlib.pyplot as plt
from matplotlib import rc

def plot_swe_bench_lite(data_file):
    with open(data_file, "r") as file:
        lines = file.readlines()

    models = []
    pass_rates = []

    for line in lines:
        if line.strip():
            pass_rate, model = line.split("%")
            models.append(model.strip())
            pass_rates.append(float(pass_rate.strip()))

    plt.rcParams["hatch.linewidth"] = 0.5
    plt.rcParams["hatch.color"] = "#444444"

    rc("font", **{"family": "sans-serif", "sans-serif": ["Helvetica"], "size": 10})

    fig, ax = plt.subplots(figsize=(10, 5))
    ax.grid(axis="y", zorder=0, lw=0.2)
    for spine in ax.spines.values():
        spine.set_edgecolor("#DDDDDD")
        spine.set_linewidth(0.5)

    colors = [
        "red" if "Aider" in model else "green" if "AutoCodeRover" in model else "blue" for model in models
    ]
    bars = ax.bar(models, pass_rates, color=colors, alpha=0.5, zorder=3)

    for bar in bars:
        yval = bar.get_height()
        ax.text(bar.get_x() + bar.get_width()/2, yval + 0.5, f'{yval}%', ha='center', va='bottom', fontsize=12, alpha=0.75)

    ax.set_xlabel("Models", fontsize=18)
    ax.set_ylabel("Pass Rate (%)", fontsize=18)
    ax.set_title("SWE Benchmark Pass Rates by Model", fontsize=20)
    plt.xticks(rotation=45, ha='right')
    plt.tight_layout()
    plt.savefig("swe_bench_lite.png")
    plt.savefig("swe_bench_lite.svg")
    plt.show()

# Example usage
plot_swe_bench_lite("benchmark/tmp.txt")
>>>>>>> REPLACE
```
2024-05-22 15:11:22 -07:00
Paul Gauthier
db30e5da2b Added draft post 2024-05-22 15:03:11 -07:00
Paul Gauthier
d94da4f809 draft posts 2024-05-22 14:34:12 -07:00
Paul Gauthier
81ec0515bb copy 2024-05-22 12:24:34 -07:00
Paul Gauthier
2374a2166f set version to 0.36.1-dev 2024-05-22 12:16:33 -07:00
Paul Gauthier
8868caf2b1 version bump to 0.36.0 2024-05-22 12:15:53 -07:00
Paul Gauthier
13ada38ecc copy 2024-05-22 12:15:40 -07:00
Paul Gauthier
589f1d510e copy 2024-05-22 12:12:30 -07:00
Paul Gauthier
c74c297aa2 copy 2024-05-22 12:09:35 -07:00
Paul Gauthier
29e92b2398 Updated HISTORY, added linting image 2024-05-22 12:08:50 -07:00
Paul Gauthier
210679f020 Use the test_cmd by default 2024-05-22 12:05:29 -07:00
Paul Gauthier
61ac9b7991 HISTORY 2024-05-22 11:55:14 -07:00
Paul Gauthier
47810310c9 Added linting article 2024-05-22 11:55:10 -07:00
Paul Gauthier
5032ce1a95 bump deps for +grep-ast==0.3.2 2024-05-20 09:19:02 -07:00
Paul Gauthier
57a2c61386 updated lint cmd descriptions 2024-05-20 09:01:22 -07:00
Paul Gauthier
ee516e3958 Added import statement for regular expressions in linter module. 2024-05-20 08:56:42 -07:00
Paul Gauthier
ce5dabd5f1 Removed unused import statement for 're' module. 2024-05-20 08:56:30 -07:00
Paul Gauthier
a115e594c0 Add import statement for 're' module and fix regex pattern in linter.py. 2024-05-20 08:56:21 -07:00
Paul Gauthier
9c28bbc98e Refactored linting process and error handling in commands. 2024-05-20 08:55:49 -07:00
Paul Gauthier
deb13c060c copy 2024-05-19 15:14:22 -07:00
Paul Gauthier
04539feccb bump deps to get grep-ast 0.3.1 2024-05-19 15:10:57 -07:00
Paul Gauthier
5745657991 ask to fix lint/test 2024-05-19 15:02:16 -07:00
Paul Gauthier
038806f694 max_reflex=3, auto fix lint/test 2024-05-19 14:24:06 -07:00
Paul Gauthier
642be2a8f0 S/R uniq prompt 2024-05-19 14:22:29 -07:00
Paul Gauthier
ca18220b77 num_with_malformed_responses 2024-05-19 14:19:06 -07:00
Paul Gauthier
5641483a3b S/R will replace all matches 2024-05-19 10:31:36 -07:00
Paul Gauthier
ec7af731e0 confirm auto lint and test 2024-05-19 10:26:06 -07:00
Paul Gauthier
cb6e7cabb6 Added lint/test_outcome 2024-05-19 08:13:20 -07:00
Paul Gauthier
41c678da09 +litellm.suppress_debug_info 2024-05-19 07:56:13 -07:00
Paul Gauthier
0b18f3b4ef Add check to skip setting up lint commands if none are provided. 2024-05-19 07:42:39 -07:00
Paul Gauthier
afd827b9d9 aider: Fixed f-string error and added missing blank line before the main function.
# Aider chat conversation:

USER: # Fix any errors below, if possible.

## Running: flake8 aider/main.py

aider/main.py:195:27: F541 f-string is missing placeholders
aider/main.py:202:1: E302 expected 2 blank lines, found 1

## See relevant lines below marked with █.

aider/main.py:
...⋮...
180│def parse_lint_cmds(lint_cmds, io):
181│    err = False
182│    res = dict()
183│    for lint_cmd in lint_cmds:
184│        pieces = lint_cmd.split(":")
...⋮...
191│        if lang and cmd:
192│            res[lang] = cmd
193│        else:
194│            io.tool_error(f'Unable to parse --lint-cmd "{lint_cmd}"')
195█            io.tool_error(f'The arg should be "language: cmd --args ..."')
196│            io.tool_error('For example: --lint-cmd "python: flake8 --select=E9"')
197│            err = True
198│    if err:
199│        return
200│    return res
201│
202█def main(argv=None, input=None, output=None, force_git_root=None, return_coder=False):
203│    if argv is None:
204│        argv = sys.argv[1:]
205│
...⋮...

ASSISTANT: Here are the necessary changes to fix the flake8 errors:

1. Correct the f-string to not use placeholders when not needed.
2. Add the missing blank line before the `main` function.

aider/main.py
```python
<<<<<<< SEARCH
            io.tool_error(f'The arg should be "language: cmd --args ..."')
=======
            io.tool_error('The arg should be "language: cmd --args ..."')
>>>>>>> REPLACE
```

aider/main.py
```python
<<<<<<< SEARCH
    return res
=======
    return res

>>>>>>> REPLACE
```
2024-05-19 07:40:54 -07:00
Paul Gauthier
25e44f584b aider: Fixed extra blank lines and added missing whitespace after a comma.
# Aider chat conversation:

USER: # Fix any errors below, if possible.

## Running: flake8 aider/coders/base_coder.py

aider/coders/base_coder.py:43:1: E303 too many blank lines (3)
aider/coders/base_coder.py:334:17: E231 missing whitespace after ','

## See relevant lines below marked with █.

aider/coders/base_coder.py:
...⋮...
 40│
 41│
 42│
 43█def wrap_fence(name):
 44│    return f"<{name}>", f"</{name}>"
 45│
 46│
 47│class Coder:
 48│    abs_fnames = None
...⋮...
331│                self.io.tool_output(json.dumps(self.functions, indent=4))
332│
333│    def setup_lint_cmds(self, lint_cmds):
334█        for lang,cmd in lint_cmds.items():
335│            self.linter.set_linter(lang, cmd)
336│
337│    def show_announcements(self):
...⋮...

ASSISTANT: Here are the necessary changes to fix the flake8 errors:

1. Remove the extra blank lines before the `wrap_fence` function.
2. Add a missing whitespace after the comma in the `setup_lint_cmds` function.

aider/coders/base_coder.py
```python
<<<<<<< SEARCH
class ExhaustedContextWindow(Exception):
    pass

def wrap_fence(name):
=======
class ExhaustedContextWindow(Exception):
    pass

def wrap_fence(name):
>>>>>>> REPLACE
```

aider/coders/base_coder.py
```python
<<<<<<< SEARCH
        for lang,cmd in lint_cmds.items():
=======
        for lang, cmd in lint_cmds.items():
>>>>>>> REPLACE
```
2024-05-19 07:40:48 -07:00
Paul Gauthier
9bd78d7ff3 Refactored lint command parsing and setup logic. 2024-05-19 07:40:41 -07:00
Paul Gauthier
990dda1122 Added empty lines for better readability. 2024-05-19 07:35:27 -07:00
Paul Gauthier
9ee332f5d9 Added options for automatic linting and testing after changes. 2024-05-19 07:34:19 -07:00
Paul Gauthier
398a1300dc aider: Added --auto-lint/--no-auto-lint switch to the parser in aider/args.py.
# Aider chat conversation:

USER: add a --auto-lint/--no-auto-lint switch

ASSISTANT: To add the `--auto-lint/--no-auto-lint` switch, we need to:

1. Add the new argument to the parser in `aider/args.py`.

Here is the *SEARCH/REPLACE* block:

aider/args.py
```python
<<<<<<< SEARCH
    )
=======
    )
    group.add_argument(
        "--auto-lint",
        action=argparse.BooleanOptionalAction,
        default=True,
        help="Enable/disable automatic linting after changes (default: True)",
    )
>>>>>>> REPLACE
```

USER:
# Fix any errors below

## Running: flake8 --select=E9,F821,F823,F831,F406,F407,F701,F702,F704,F706 --show-source aider/args.py

aider/args.py:17:5: F821 undefined name 'group'
    group.add_argument(
    ^

## See relevant line below marked with █.

/Users/gauthier/Projects/aider/aider/args.py:
...⋮...
  9│def get_parser(default_config_files, git_root):
 10│    parser = configargparse.ArgumentParser(
 11│        description="aider is GPT powered coding in your terminal",
 12│        add_config_file_help=True,
 13│        default_config_files=default_config_files,
 14│        config_file_parser_class=configargparse.YAMLConfigFileParser,
 15│        auto_env_var_prefix="AIDER_",
 16│    )
 17█    group.add_argument(
 18│        "--auto-lint",
 19│        action=argparse.BooleanOptionalAction,
 20│        default=True,
 21│        help="Enable/disable automatic linting after changes (default: True)",
...⋮...

ASSISTANT: To fix the error, we need to move the `--auto-lint` argument addition to the correct location within the `get_parser` function, after the `group` variable is defined.

Here is the corrected *SEARCH/REPLACE* block:

aider/args.py
```python
<<<<<<< SEARCH
    group.add_argument(
        "--auto-lint",
        action=argparse.BooleanOptionalAction,
        default=True,
        help="Enable/disable automatic linting after changes (default: True)",
    )

    ##########
=======
    ##########
    group.add_argument(
        "--auto-lint",
        action=argparse.BooleanOptionalAction,
        default=True,
        help="Enable/disable automatic linting after changes (default: True)",
    )
>>>>>>> REPLACE
```

USER:
# Fix any errors below

## Running: flake8 --select=E9,F821,F823,F831,F406,F407,F701,F702,F704,F706 --show-source aider/args.py

aider/args.py:18:5: F821 undefined name 'group'
    group.add_argument(
    ^

## See relevant line below marked with █.

/Users/gauthier/Projects/aider/aider/args.py:
...⋮...
  9│def get_parser(default_config_files, git_root):
 10│    parser = configargparse.ArgumentParser(
 11│        description="aider is GPT powered coding in your terminal",
 12│        add_config_file_help=True,
 13│        default_config_files=default_config_files,
 14│        config_file_parser_class=configargparse.YAMLConfigFileParser,
 15│        auto_env_var_prefix="AIDER_",
 16│    )
 17│    ##########
 18█    group.add_argument(
 19│        "--auto-lint",
 20│        action=argparse.BooleanOptionalAction,
 21│        default=True,
 22│        help="Enable/disable automatic linting after changes (default: True)",
...⋮...

ASSISTANT: The error is occurring because the `group` variable is not defined before it is used. We need to place the `--auto-lint` argument addition after the `group` variable is defined.

Here is the corrected *SEARCH/REPLACE* block:

aider/args.py
```python
<<<<<<< SEARCH
    ##########
    group.add_argument(
        "--auto-lint",
        action=argparse.BooleanOptionalAction,
        default=True,
        help="Enable/disable automatic linting after changes (default: True)",
    )
=======
    group = parser.add_argument_group("Main")
>>>>>>> REPLACE
```

aider/args.py
```python
<<<<<<< SEARCH
    group.add_argument(
=======
    group.add_argument(
        "--auto-lint",
        action=argparse.BooleanOptionalAction,
        default=True,
        help="Enable/disable automatic linting after changes (default: True)",
    )
    group.add_argument(
>>>>>>> REPLACE
```
2024-05-19 06:23:28 -07:00
Paul Gauthier
76038e69d7 Added an argument for enabling/disabling automatic linting after changes. 2024-05-19 06:23:27 -07:00
Paul Gauthier
73dd873f86 Add option to enable/disable automatic linting after changes. 2024-05-19 06:23:20 -07:00
Paul Gauthier
6474139d04 Lint files before fixing errors and commit changes if necessary. 2024-05-18 21:13:19 -07:00
Paul Gauthier
d79a629db0 Fix tool error message to display the correct error information. 2024-05-18 21:06:45 -07:00
Paul Gauthier
fec1a646d5 Refactored linting process to handle FileNotFoundError exceptions. 2024-05-18 21:06:30 -07:00
Paul Gauthier
38de35e924 Set linters based on provided commands in args.lint_cmd. 2024-05-18 21:04:47 -07:00
Paul Gauthier
5a6e72a64b Updated linting functionality to only run linting without committing. 2024-05-18 21:00:45 -07:00
Paul Gauthier
7391903759 Removed redundant lint command option and added support for specifying lint commands for different languages. 2024-05-18 20:57:01 -07:00
Paul Gauthier
06bda2d730 aider: Added a --lint-cmd argument to allow users to specify a lint command multiple times.
# Aider chat conversation:

USER: add a --lint-cmd arg that takes a string like "python python-linter --with --args". allow the user to specify this args multiple times

ASSISTANT: To add a `--lint-cmd` argument that takes a string and allows the user to specify this argument multiple times, we need to:

1. Add the new argument to the parser.
2. Ensure it can be specified multiple times.

Here is the *SEARCH/REPLACE* block for `aider/args.py`:

aider/args.py
```python
<<<<<<< SEARCH
    )
=======
    )

    ##########
    group = parser.add_argument_group("Lint Settings")
    group.add_argument(
        "--lint-cmd",
        action="append",
        metavar="LINT_CMD",
        help="Specify a lint command to run. Can be specified multiple times.",
    )
>>>>>>> REPLACE
```
2024-05-18 20:52:21 -07:00
Paul Gauthier
6fac50a3a1 aider: Fix undefined name 'Path' errors by importing the Path class from the pathlib module.
# Aider chat conversation:

USER: # Fix any errors below

## Running: flake8 --select=E9,F821,F823,F831,F406,F407,F701,F702,F704,F706 --show-source aider/coders/wholefile_coder.py

aider/coders/wholefile_coder.py:73:62: F821 undefined name 'Path'
                    if fname and fname not in chat_files and Path(fname).name in chat_files:
                                                             ^
aider/coders/wholefile_coder.py:74:33: F821 undefined name 'Path'
                        fname = Path(fname).name
                                ^
aider/coders/wholefile_coder.py:103:30: F821 undefined name 'Path'
                full_path = (Path(self.root) / fname).absolute()
                             ^
aider/coders/wholefile_coder.py:133:12: F821 undefined name 'Path'
        if Path(full_path).exists():
           ^

## See relevant lines below marked with █.

aider/coders/wholefile_coder.py:
...⋮...
  2│from aider import diffs
  3│
...⋮...
  9│class WholeFileCoder(Coder):
 10│    edit_format = "whole"
 11│
...⋮...
 30│    def get_edits(self, mode="update"):
 31│        content = self.partial_response_content
 32│
...⋮...
 44│        for i, line in enumerate(lines):
 45│            if line.startswith(self.fence[0]) or line.startswith(self.fence[1]):
 46│                if fname is not None:
 47│                    # ending an existing block
 48│                    saw_fname = None
 49│
 50│                    full_path = self.abs_root_path(fname)
 51│
 52│                    if mode == "diff":
 53│                        output += self.do_live_diff(full_path, new_lines, True)
 54│                    else:
 55│                        edits.append((fname, fname_source, new_lines))
 56│
...⋮...
 63│                if i > 0:
 64│                    fname_source = "block"
...⋮...
 70│                    # Did gpt prepend a bogus dir? It especially likes to
 71│                    # include the path/to prefix from the one-shot example in
 72│                    # the prompt.
 73█                    if fname and fname not in chat_files and Path(fname).name in chat_files:
 74█                        fname = Path(fname).name
 75│                if not fname:  # blank line? or ``` was on first line i==0
 76│                    if saw_fname:
 77│                        fname = saw_fname
...⋮...
100│        if mode == "diff":
101│            if fname is not None:
102│                # ending an existing block
103█                full_path = (Path(self.root) / fname).absolute()
104│                output += self.do_live_diff(full_path, new_lines, False)
105│            return "\n".join(output)
106│
...⋮...
130│            self.io.write_text(full_path, new_lines)
131│
132│    def do_live_diff(self, full_path, new_lines, final):
133█        if Path(full_path).exists():
134│            orig_lines = self.io.read_text(full_path).splitlines(keepends=True)
135│
136│            show_diff = diffs.diff_partial_update(
137│                orig_lines,
138│                new_lines,
139│                final=final,
140│            ).splitlines()
141│            output = show_diff
142│        else:
...⋮...

ASSISTANT: To fix the errors, we need to import the `Path` class from the `pathlib` module. Here are the necessary changes:

aider/coders/wholefile_coder.py
<source>python
<<<<<<< SEARCH
from aider import diffs
=======
from aider import diffs
from pathlib import Path
>>>>>>> REPLACE
</source>

aider/coders/wholefile_coder.py
<source>python
<<<<<<< SEARCH
                    if fname and fname not in chat_files and Path(fname).name in chat_files:
                        fname = Path(fname).name
=======
                    if fname and fname not in chat_files and Path(fname).name in chat_files:
                        fname = Path(fname).name
>>>>>>> REPLACE
</source>

aider/coders/wholefile_coder.py
<source>python
<<<<<<< SEARCH
                full_path = (Path(self.root) / fname).absolute()
=======
                full_path = (Path(self.root) / fname).absolute()
>>>>>>> REPLACE
</source>

aider/coders/wholefile_coder.py
<source>python
<<<<<<< SEARCH
        if Path(full_path).exists():
=======
        if Path(full_path).exists():
>>>>>>> REPLACE
</source>
2024-05-18 20:49:17 -07:00
Paul Gauthier
a50f01a004 Removed unnecessary import of Path from pathlib. 2024-05-18 20:49:10 -07:00
Paul Gauthier
828e14fd3d Updated Linter to return LintResult if text or lines are present. 2024-05-18 20:48:53 -07:00
Paul Gauthier
1a3ae1fb60 Refactored Linter class to only return LintResult if text is not empty. 2024-05-18 20:48:36 -07:00