Commit graph

2786 commits

Author SHA1 Message Date
Paul Gauthier
b600e17910 Added traceback information to the linting output. 2024-05-18 17:42:04 -07:00
Paul Gauthier
567c470c0b Refactored error message and added a todo comment for printing out the last call stack. 2024-05-18 17:41:58 -07:00
Paul Gauthier
f699241e12 aider: Change error message format to display the class name instead of the full type.
# Aider chat conversation:

USER:         res = f"{type(err)}: {err}\n" is producing: "<class 'IndentationError'>: unexpected indent (tmp.syntaxerror.py, line 4)" I want it to say "IndentationError: unexpected indent (tmp.syntaxerror.py, line 4)"

ASSISTANT: To change the error message format from `<class 'IndentationError'>: unexpected indent (tmp.syntaxerror.py, line 4)` to `IndentationError: unexpected indent (tmp.syntaxerror.py, line 4)`, we need to modify the line where the error message is being constructed.

Here is the *SEARCH/REPLACE* block:

aider/linter.py
```python
<<<<<<< SEARCH
        res = f"{type(err)}: {err}\n"
=======
        res = f"{type(err).__name__}: {err}\n"
>>>>>>> REPLACE
```
2024-05-18 17:38:26 -07:00
Paul Gauthier
30c9c4c319 Refactored linter to support linting Python code with py_compile. 2024-05-18 17:38:25 -07:00
Paul Gauthier
b26969ea90 Refactored linting to use absolute file path. 2024-05-17 19:22:17 -07:00
Paul Gauthier
0412b19f6b aider: Fix flake8 errors by removing unnecessary f-string prefixes in aider/coders/base_coder.py.
# Aider chat conversation:

USER: I ran this command:

 pre-commit run --files aider/coders/base_coder.py

And got this output:

isort....................................................................Passed
black....................................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

aider/coders/base_coder.py:1135:32: F541 f-string is missing placeholders
            self.io.tool_error(f"The LLM did not conform to the edit format.")
                               ^
aider/coders/base_coder.py:1148:32: F541 f-string is missing placeholders
            self.io.tool_error(f"Exception while updating files:")
                               ^

ASSISTANT: Please add `aider/coders/base_coder.py` to the chat so I can propose the necessary changes to fix the flake8 errors.

USER: I added these *read-write* files: aider/coders/base_coder.py

ASSISTANT: Here are the changes to fix the flake8 errors related to the f-strings missing placeholders in `aider/coders/base_coder.py`:

```diff
--- aider/coders/base_coder.py
+++ aider/coders/base_coder.py
@@ ... @@
-            self.io.tool_error(f"The LLM did not conform to the edit format.")
+            self.io.tool_error("The LLM did not conform to the edit format.")
@@ ... @@
-            self.io.tool_error(f"Exception while updating files:")
+            self.io.tool_error("Exception while updating files:")
```

These changes remove the unnecessary f-string prefixes since there are no variables or expressions to be evaluated inside the strings.
2024-05-17 19:11:49 -07:00
Paul Gauthier
8f194c8313 Add reflection tracking and limit to Coder class. 2024-05-17 19:11:48 -07:00
Paul Gauthier
4b0c38254e added lint reflection 2024-05-17 16:58:04 -07:00
Paul Gauthier
fe65b7d11f roughed in flake8 2024-05-17 16:41:36 -07:00
Paul Gauthier
c7196b8390 aider: Remove unused variable _output in the run_cmd method in aider/linter.py.
# Aider chat conversation:

USER: I ran this command:

 pre-commit run --files aider/linter.py

And got this output:

isort....................................................................Failed
- hook id: isort
- files were modified by this hook

Fixing /Users/gauthier/Projects/aider/aider/linter.py

black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted aider/linter.py

All done!  🍰 
1 file reformatted.

flake8...................................................................Failed
- hook id: flake8
- exit code: 1

aider/linter.py:37:13: F841 local variable '_output' is assigned to but never used
            _output = subprocess.check_output(cmd, cwd=self.root).decode()
            ^

ASSISTANT: Please add `aider/linter.py` to the chat so I can make the necessary changes to fix the flake8 issue.

USER: I added these files to the chat: aider/linter.py

ASSISTANT: To fix the flake8 issue, we need to remove the unused variable `_output` in the `run_cmd` method. Here is the *SEARCH/REPLACE* block:

aider/linter.py
```python
<<<<<<< SEARCH
            _output = subprocess.check_output(cmd, cwd=self.root).decode()
            return  # zero exit status
=======
            subprocess.check_output(cmd, cwd=self.root).decode()
            return  # zero exit status
>>>>>>> REPLACE
```

This change will remove the assignment to the unused variable `_output` while keeping the call to `subprocess.check_output`.
2024-05-17 15:37:06 -07:00
Paul Gauthier
4f81971a13 Add support for running custom linters and improve error handling. 2024-05-17 15:37:05 -07:00
Paul Gauthier
dc2850b53d Added a Linter class for linting files and refactored linting logic in the main function. 2024-05-17 15:21:09 -07:00
Paul Gauthier
778dbcabb0 aider: Remove the unused import get_language from aider/linter.py.
# Aider chat conversation:

USER: I ran this command:

 flake8 aider/linter.py

And got this output:

aider/linter.py:9:1: F401 'tree_sitter_languages.get_language' imported but unused

ASSISTANT: Please add the file `aider/linter.py` to the chat so I can make the necessary changes.

USER: I added these files to the chat: aider/linter.py

ASSISTANT: To resolve the flake8 warning, we need to remove the unused import `get_language` from `aider/linter.py`.

Here is the *SEARCH/REPLACE* block:

aider/linter.py
```python
<<<<<<< SEARCH
from tree_sitter_languages import get_language, get_parser  # noqa: E402
=======
from tree_sitter_languages import get_parser  # noqa: E402
>>>>>>> REPLACE
```
2024-05-17 14:30:58 -07:00
Paul Gauthier
a9fd02c74d aider: Remove unused imports and variable in aider/linter.py to pass pre-commit hooks.
# Aider chat conversation:

USER: I ran this command:

 .git/hooks/pre-commit

And got this output:

[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /Users/gauthier/.cache/pre-commit/patch1715981257-12950.
isort....................................................................Failed
- hook id: isort
- files were modified by this hook

Fixing /Users/gauthier/Projects/aider/aider/linter.py

black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted aider/linter.py

All done!  🍰 
1 file reformatted.

flake8...................................................................Failed
- hook id: flake8
- exit code: 1

aider/linter.py:1:1: F401 'os' imported but unused
import os
^
aider/linter.py:6:1: F401 'tree_sitter' imported but unused
import tree_sitter
^
aider/linter.py:9:1: F401 'aider.dump.dump' imported but unused
from aider.dump import dump
^
aider/linter.py:21:5: F841 local variable 'language' is assigned to but never used
    language = get_language(lang)
    ^

[WARNING] Stashed changes conflicted with hook auto-fixes... Rolling back fixes...
[INFO] Restored changes from /Users/gauthier/.cache/pre-commit/patch1715981257-12950.

ASSISTANT: Please add `aider/linter.py` to the chat so I can make the necessary changes to fix the issues reported by the pre-commit hooks.

USER: I added these files to the chat: aider/linter.py

ASSISTANT: To fix the issues reported by the pre-commit hooks, we need to:

1. Remove unused imports: `os`, `tree_sitter`, and `dump`.
2. Remove the unused variable `language`.

Here are the *SEARCH/REPLACE* blocks:

aider/linter.py
```python
<<<<<<< SEARCH
import os
import sys
import warnings
from pathlib import Path

import tree_sitter
from grep_ast import TreeContext, filename_to_lang

from aider.dump import dump
=======
import sys
import warnings
from pathlib import Path

from grep_ast import TreeContext, filename_to_lang
>>>>>>> REPLACE
```

aider/linter.py
```python
<<<<<<< SEARCH
    language = get_language(lang)
=======
>>>>>>> REPLACE
```

Now, the file should pass the pre-commit hooks.
2024-05-17 14:27:54 -07:00
Paul Gauthier
494aa0140b Refactored the linter script to handle language detection and error traversal more efficiently. 2024-05-17 14:27:53 -07:00
Paul Gauthier
cb8a487c89 Display syntax errors with tree context 2024-05-17 14:22:25 -07:00
Paul Gauthier
86fdeb0597 aider: Update traverse_tree to print error type, line number, and 3 lines of context before/after the error.
# Aider chat conversation:

USER: print out the line and the N=3 lines before/after it

ASSISTANT: To print out the line with the error and the 3 lines before and after it, we need to modify the `traverse_tree` function in `aider/linter.py`. Here are the necessary changes:

aider/linter.py
```python
<<<<<<< SEARCH
    # Traverse the tree to find errors
=======
    # Traverse the tree to find errors and print context
>>>>>>> REPLACE
```

aider/linter.py
```python
<<<<<<< SEARCH
            print(f"{error_type} at line: {node.start_point[0] + 1}")
=======
            start_line = max(0, node.start_point[0] - 3)
            end_line = node.end_point[0] + 3
            error_line = node.start_point[0] + 1

            with open(file_path, 'r') as file:
                lines = file.readlines()

            print(f"{error_type} at line: {error_line}")
            print("Context:")
            for i in range(start_line, min(end_line, len(lines))):
                line_number = i + 1
                prefix = ">> " if line_number == error_line else "   "
                print(f"{prefix}{line_number}: {lines[i].rstrip()}")
            print("\n")
>>>>>>> REPLACE
```

This will ensure that the linter prints the error type, the line number, and the context of 3 lines before and after the error.
2024-05-17 14:00:59 -07:00
Paul Gauthier
0493680d57 aider: Update the linter to check for both 'ERROR' and 'MISSING' nodes.
# Aider chat conversation:

USER: is node.type 'ERROR" a standard across all tree sitter languages?

ASSISTANT: The `node.type == 'ERROR'` check is specific to the Tree-sitter library and is used to identify syntax errors in the parsed code. This is a standard feature across all languages supported by Tree-sitter.

Would you like to proceed with any changes or enhancements to the `aider/linter.py` file? If so, please specify the changes you need.

USER: is MISSING ever used to indicate a parsing/syntax problem

ASSISTANT: Yes, in Tree-sitter, `MISSING` is also used to indicate a parsing or syntax problem. Specifically, it indicates that a required element is missing in the syntax tree.

Would you like to update the `aider/linter.py` file to also check for `MISSING` nodes? If so, I can provide the necessary changes.

USER: yes

ASSISTANT: To update the `aider/linter.py` file to also check for `MISSING` nodes, we need to modify the `traverse_tree` function. Here are the necessary changes:

aider/linter.py
```python
<<<<<<< SEARCH
        if node.type == 'ERROR':
            print(f"Syntax error at line: {node.start_point[0] + 1}")
=======
        if node.type == 'ERROR' or node.is_missing:
            error_type = 'Syntax error' if node.type == 'ERROR' else 'Missing element'
            print(f"{error_type} at line: {node.start_point[0] + 1}")
>>>>>>> REPLACE
```

This change will ensure that the linter checks for both `ERROR` and `MISSING` nodes and reports them accordingly.
2024-05-17 14:00:42 -07:00
Paul Gauthier
c26833419d Fixed tree-sitter FutureWarning and refactored parsing logic in linter. 2024-05-17 14:00:41 -07:00
Paul Gauthier
3cc0a3c263 aider: Implemented function to parse file with tree-sitter and print syntax error line numbers.
# Aider chat conversation:

USER: write me a function that takes a file path and uses tree-sitter to parse the AST and print out the line number of every syntax/parse error in the file

ASSISTANT: To implement the function that takes a file path, uses tree-sitter to parse the AST, and prints out the line number of every syntax/parse error in the file, we need to:

1. Install the `tree-sitter` library.
2. Write the function in `aider/linter.py`.

Here is the *SEARCH/REPLACE* block for `aider/linter.py`:

aider/linter.py
```python
<<<<<<< SEARCH
=======
import os
import tree_sitter

def parse_file_for_errors(file_path):
    """
    Parses the given file using tree-sitter and prints out the line number of every syntax/parse error.
    """
    # Load the language
    Language = tree_sitter.Language
    PARSER = tree_sitter.Parser()
    LANGUAGE = Language(os.path.join('build', 'my-languages.so'), 'python')
    PARSER.set_language(LANGUAGE)

    # Read the file content
    with open(file_path, 'r') as file:
        content = file.read()

    # Parse the content
    tree = PARSER.parse(bytes(content, "utf8"))

    # Traverse the tree to find errors
    def traverse_tree(node):
        if node.type == 'ERROR':
            print(f"Syntax error at line: {node.start_point[0] + 1}")
        for child in node.children:
            traverse_tree(child)

    traverse_tree(tree.root_node)
>>>>>>> REPLACE
```

This code will:
1. Load the `tree-sitter` library and set up the parser for Python.
2. Read the content of the file specified by `file_path`.
3. Parse the content and traverse the AST to find and print the line numbers of any syntax errors.

You can now run this function by calling `parse_file_for_errors('path/to/your/file.py')`.
2024-05-17 13:47:52 -07:00
Paul Gauthier
819fccc7a4 added temp param, prompt strong that files message is truth 2024-05-16 08:52:30 -07:00
Paul Gauthier
df84bcf38b Merge branch 'main' into swe-bench 2024-05-15 12:02:00 -07:00
Paul Gauthier
72613f3b27 switch naming from openai/gpt-4o to gpt-4o 2024-05-15 06:25:30 -07:00
Paul Gauthier
ecc46bd3e3 16x 2024-05-14 14:04:45 -07:00
Paul Gauthier
dea93bbfd9 mentioned_idents 2024-05-14 12:18:48 -07:00
Paul Gauthier
e5616c0247 mentioned_fnames 2024-05-14 11:52:32 -07:00
Paul Gauthier
73e6949287 refac 2024-05-14 11:12:36 -07:00
Paul Gauthier
851e98a398 added tree cache 2024-05-14 10:32:16 -07:00
Paul Gauthier
3251380364 refac 2024-05-14 10:23:53 -07:00
Paul Gauthier
370f668955 remove useless noqa 2024-05-14 06:02:40 -07:00
Paul Gauthier
6638efbee4 better prompting for LLM to suggest files 2024-05-14 06:02:16 -07:00
Paul Gauthier
e96420f999 fix to ignore long (!) lines in repomap 2024-05-14 06:01:24 -07:00
Paul Gauthier
292250db4f prompt and token adjustments when no files in chat with repo 2024-05-13 12:30:42 -07:00
Paul Gauthier
edc70bdf84 Handle multiline tool_error() better 2024-05-13 11:42:22 -07:00
Paul Gauthier
b2fbafe41b docs and --4-turbo updates 2024-05-13 11:24:52 -07:00
Paul Gauthier
e909207241 set version to 0.35.1-dev 2024-05-13 11:19:29 -07:00
Paul Gauthier
aaaef12cce version bump to 0.35.0 2024-05-13 11:18:51 -07:00
Paul Gauthier
ea05cf05e1 Added --restore-chat-history 2024-05-13 11:09:28 -07:00
Paul Gauthier
bfba56c5f1 added 4o as default 2024-05-13 10:57:26 -07:00
Paul Gauthier
b6cd852124 added openai/ prefix 2024-05-13 10:36:07 -07:00
Paul Gauthier
7b19f767fc Added gpt-4o 2024-05-13 10:33:08 -07:00
Paul Gauthier
eea8e50c60 ask llm are you sure? when REPLACE exists 2024-05-11 16:56:14 -07:00
Paul Gauthier
90c0a5b9a4 reflect multiple messages; do not strip edit error messages 2024-05-11 16:40:37 -07:00
Paul Gauthier
0f9e314bb2 emit edit errors as proper tool_error() markdown 2024-05-11 15:38:03 -07:00
Paul Gauthier
cd78080fa5 on search/replace error ask the LLM did you mean? 2024-05-11 15:02:22 -07:00
Paul Gauthier
4841f318c1 always retry httpx errors 2024-05-11 15:01:56 -07:00
Paul Gauthier
6c45b58698 aider: Implemented find_similar_lines to fuzzy search content and return closest matching contiguous chunk of lines.
# Aider chat conversation:

USER: implement `find_similar_lines` which will fuzzy search the content for the original lines and return the contiguous chunk of lines from content which are the closest match

ASSISTANT: Here is the implementation of `find_similar_lines` to fuzzy search the content and return the closest matching contiguous chunk of lines:

aider/coders/editblock_coder.py
<source>python
<<<<<<< SEARCH
    #did_you_mean = find_similar_lines(original, content)
=======
    did_you_mean = find_similar_lines(original, content)
>>>>>>> REPLACE
</source>

aider/coders/editblock_coder.py
<source>python
<<<<<<< SEARCH
=======
def find_similar_lines(search_lines, content_lines, threshold=0.8):
    search_lines = search_lines.splitlines()
    content_lines = content_lines.splitlines()

    best_ratio = 0
    best_match = None

    for i in range(len(content_lines) - len(search_lines) + 1):
        chunk = content_lines[i:i+len(search_lines)]
        ratio = SequenceMatcher(None, search_lines, chunk).ratio()
        if ratio > best_ratio:
            best_ratio = ratio
            best_match = chunk

    if best_ratio >= threshold:
        return "\n".join(best_match)
    else:
        return ""
>>>>>>> REPLACE
</source>

This adds the `find_similar_lines` function which:

1. Splits the search lines and content lines on newlines
2. Iterates through the content lines, considering each contiguous chunk the same length as the search lines
3. Calculates the SequenceMatcher ratio between the search lines and each chunk
4. Keeps track of the chunk with the highest ratio
5. If the best ratio is above the threshold (default 0.8), it returns the best matching chunk joined back into a string
6. Otherwise it returns an empty string to indicate no close enough match was found

It also uncomments the line to call `find_similar_lines` and store the result in `did_you_mean`.

Let me know if you have any other questions!
2024-05-11 10:49:34 -07:00
Paul Gauthier
056dae6530 Removed unused find_similar_lines function call from EditBlockCoder class. 2024-05-11 10:49:33 -07:00
Paul Gauthier
fd6f31f972 Replaced find_similar_text with find_similar_lines in editblock_coder.py. 2024-05-11 10:48:45 -07:00
Paul Gauthier
d02506db16 aider: Implemented find_similar_text function to fuzzy search the content for the closest match to the original text.
# Aider chat conversation:

USER: I did `git reset --hard HEAD~1` to discard the last edits. Please wait for further instructions before attempting that change again. Feel free to ask relevant questions about why the changes were reverted.

ASSISTANT: I understand you reverted the previous changes. Before I attempt to implement the `find_similar_lines` function again, I have a few questions to ensure I follow your guidelines correctly:

1. When proposing changes, should I only modify the `aider/coders/editblock_coder.py` file since that is the only one you have explicitly added to the chat
^C KeyboardInterrupt

USER: implement `find_similar_text` which will fuzzy search the content for the original text and return the part of content which is the closest match

ASSISTANT: Here is the implementation of `find_similar_text` to fuzzy search the content for the closest match to the original text:

aider/coders/editblock_coder.py
<source>python
<<<<<<< SEARCH
=======
from difflib import SequenceMatcher

def find_similar_text(original, content):
    content_len = len(content)
    original_len = len(original)

    best_ratio = 0
    best_match = None

    for i in range(content_len - original_len + 1):
        chunk = content[i:i+original_len]
        ratio = SequenceMatcher(None, original, chunk).ratio()

        if ratio > best_ratio:
            best_ratio = ratio
            best_match = chunk

    return best_match
>>>>>>> REPLACE
</source>

This function:

1. Iterates through the content, considering each chunk of text the same length as the original
2. Calculates the SequenceMatcher ratio between the original text and each chunk
3. Keeps track of the chunk with the highest ratio
4. Returns the chunk of content that best matches the original text

It will return the portion of `content` that is the closest fuzzy match to `original`.

Let me know if you have any other questions!
2024-05-11 10:43:40 -07:00