Commit graph

48 commits

Author SHA1 Message Date
Paul Gauthier
2c32fe5eb8 Adopt safe_abs_path 2023-07-07 17:17:33 -07:00
Paul Gauthier
9cab570f87 refactor to use io.read_text 2023-07-05 20:21:32 -07:00
Paul Gauthier
58e763cee7 roughed in diff-func-string 2023-06-29 15:10:33 -07:00
Paul Gauthier
6c87d5be1c better 2023-06-28 13:52:43 -07:00
Paul Gauthier
96fda2aa9f abandon n-backticks, try some html fences 2023-06-25 20:23:36 -07:00
Paul Gauthier
9151cf03a1 quote with num_ticks 2023-06-25 19:58:17 -07:00
Paul Gauthier
f005404ec0 Roughed in updated sys msg for whole format 2023-06-25 15:27:06 -07:00
Paul Gauthier
5e63ce3352 refac 2023-06-20 17:04:06 -07:00
Paul Gauthier
44198e8b3b Fix for issue #24, provided by @wwestenbrink 2023-06-20 06:57:19 -07:00
Paul Gauthier
806805c0b1 Ensure leading_whitespace is actually space 2023-06-10 06:16:22 -07:00
Paul Gauthier
8d408e1fc4 Try replace_part_with_missing_leading_whitespace first, so that single line edits get the benefit 2023-06-09 15:05:34 -07:00
Paul Gauthier
5612c8e13d Don't indent blank lines 2023-06-09 14:51:37 -07:00
Paul Gauthier
5a3fd35c16 If all lines in the part start with whitespace, then honor it. 2023-06-09 14:50:21 -07:00
Paul Gauthier
329baf3717 Fix the actual replace step 2023-06-09 14:46:43 -07:00
Paul Gauthier
1daf175c83 aider: Implemented replace_part_with_missing_leading_whitespace function to find and replace the portion of whole that matches part with the same leading whitespace and replace it with replace containing the same leading whitespace.
# Context:
USER: in `replace_part_with_missing_leading_whitespace`, we want to find the string part in whole and replace it with the string `replace`. but `part` may not exactly occur in whole. instead, every line of part appears in whole with a fixed amount of extra whitespace in front of it. so every line in part might appear in whole with 4 spaces prepended. or every line might have 2 tabs. always the exact same whitespaces. finish implementing this function so that it finds the portion of whole which matches part in this manner (if any) and replace it with `replace` where replace is updated to contain that repeated/omitted whitespace.
ASSISTANT: To implement the `replace_part_with_missing_leading_whitespace` function, follow these steps:

1. Split `whole` and `part` into lines.
2. Iterate through the lines of `whole` and find the first line that matches the first line of `part` with any leading whitespace.
3. Check if the following lines in `whole` also match the remaining lines in `part` with the same leading whitespace.
4. If all lines match, replace the matched lines in `whole` with the lines in `replace` with the same leading whitespace.
5. Return the modified `whole` or `None` if no match is found.

Here's the updated function:

```python
aider/utils.py
<<<<<<< ORIGINAL
def replace_part_with_missing_leading_whitespace(whole, part, replace):
    pass
=======
def replace_part_with_missing_leading_whitespace(whole, part, replace):
    whole_lines = whole.splitlines()
    part_lines = part.splitlines()
    replace_lines = replace.splitlines()

    for i in range(len(whole_lines) - len(part_lines) + 1):
        leading_whitespace = ""
        for j, c in enumerate(whole_lines[i]):
            if c == part_lines[0][0]:
                leading_whitespace = whole_lines[i][:j]
                break

        if not leading_whitespace:
            continue

        matched = all(
            whole_lines[i + k].startswith(leading_whitespace + part_lines[k])
            for k in range(len(part_lines))
        )

        if matched:
            for k in range(len(part_lines)):
                whole_lines[i + k] = leading_whitespace + replace_lines[k]
            return "\n".join(whole_lines)

    return None
>>>>>>> UPDATED
```

This implementation should find the portion of `whole` that matches `part` with the same leading whitespace and replace it with `replace` containing the same leading whitespace.
2023-06-09 14:37:08 -07:00
Paul Gauthier
490796de78 wip 2023-06-09 14:32:45 -07:00
Paul Gauthier
e4d9e71f5b simplified append/create 2023-06-09 14:27:17 -07:00
Paul Gauthier
9cef379abd refac 2023-06-07 12:28:45 -07:00
Paul Gauthier
95b32a74a9 Moved all model names into an enum 2023-06-05 09:19:29 -07:00
Paul Gauthier
c0bf25d05b undo trailing newline "fix" 2023-06-04 15:03:13 -07:00
Paul Gauthier
f47f611e65 Handle empty replacement text too 2023-06-04 14:30:22 -07:00
Paul Gauthier
fe1271114f Avoid stripping a newline when doing nearest search/replace 2023-06-04 14:28:03 -07:00
Paul Gauthier
ca74eee2f9 Allow GPT to omit the filename from edit blocks, if an earlier block provided it 2023-05-30 11:45:21 -07:00
Paul Gauthier
445c253abc fixed corner case bugs in find_original_update_blocks 2023-05-29 07:12:57 -07:00
Paul Gauthier
87c902b285 aider: Fixed IndexError in find_original_update_blocks function.
# Context:
USER: I ran this command:

 pytest tests/test_utils.py

And got this output:

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/gauthier/Projects/aider
collected 11 items

tests/test_utils.py F..........                                          [100%]

=================================== FAILURES ===================================
__________________ TestUtils.test_false_incomplete_edit_block __________________

content = '\nNo problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:\n\n`...h `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.\n'

    def find_original_update_blocks(content):
        # make sure we end with a newline, otherwise the regex will miss <<UPD on the last line
        if not content.endswith("\n"):
            content = content + "\n"

        pieces = re.split(split_re, content)

        pieces.reverse()
        processed = []

        try:
            while pieces:
                cur = pieces.pop()

                if cur in (DIVIDER, UPDATED):
                    processed.append(cur)
                    raise ValueError(f"Unexpected {cur}")

                if cur.strip() != ORIGINAL:
                    processed.append(cur)
                    continue

                processed.append(cur)  # original_marker

                filename = processed[-2].splitlines()[-1].strip()
                if not len(filename) or "`" in filename:
>                   filename = processed[-2].splitlines()[-2].strip()
E                   IndexError: list index out of range

aider/utils.py:234: IndexError

During handling of the above exception, another exception occurred:

self = <tests.test_utils.TestUtils testMethod=test_false_incomplete_edit_block>

        def test_false_incomplete_edit_block(self):
            edit = """
    No problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:

    ```python
    tests/test_repomap.py
    <<<<<<< ORIGINAL
        def test_check_for_ctags_failure(self):
            with patch("subprocess.run") as mock_run:
                mock_run.side_effect = Exception("ctags not found")
    =======
        def test_check_for_ctags_failure(self):
            with patch("subprocess.check_output") as mock_check_output:
                mock_check_output.side_effect = Exception("ctags not found")
    >>>>>>> UPDATED

    <<<<<<< ORIGINAL
        def test_check_for_ctags_success(self):
            with patch("subprocess.run") as mock_run:
                mock_run.return_value = CompletedProcess(args=["ctags", "--version"], returncode=0, stdout='''{
      "_type": "tag",
      "name": "status",
      "path": "aider/main.py",
      "pattern": "/^    status = main()$/",
      "kind": "variable"
    }''')
    =======
        def test_check_for_ctags_success(self):
            with patch("subprocess.check_output") as mock_check_output:
                mock_check_output.return_value = '''{
      "_type": "tag",
      "name": "status",
      "path": "aider/main.py",
      "pattern": "/^    status = main()$/",
      "kind": "variable"
    }'''
    >>>>>>> UPDATED
    ```

    These changes replace the `subprocess.run` patches with `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.
    """
            # Should not raise a ValueError
>           list(utils.find_original_update_blocks(edit))

tests/test_utils.py:199:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

content = '\nNo problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:\n\n`...h `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.\n'

    def find_original_update_blocks(content):
        # make sure we end with a newline, otherwise the regex will miss <<UPD on the last line
        if not content.endswith("\n"):
            content = content + "\n"

        pieces = re.split(split_re, content)

        pieces.reverse()
        processed = []

        try:
            while pieces:
                cur = pieces.pop()

                if cur in (DIVIDER, UPDATED):
                    processed.append(cur)
                    raise ValueError(f"Unexpected {cur}")

                if cur.strip() != ORIGINAL:
                    processed.append(cur)
                    continue

                processed.append(cur)  # original_marker

                filename = processed[-2].splitlines()[-1].strip()
                if not len(filename) or "`" in filename:
                    filename = processed[-2].splitlines()[-2].strip()
                    if not len(filename) or "`" in filename:
                        raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")

                original_text = pieces.pop()
                processed.append(original_text)

                divider_marker = pieces.pop()
                processed.append(divider_marker)
                if divider_marker.strip() != DIVIDER:
                    raise ValueError(f"Expected {DIVIDER}")

                updated_text = pieces.pop()

                updated_marker = pieces.pop()
                if updated_marker.strip() != UPDATED:
                    raise ValueError(f"Expected {UPDATED}")

                yield filename, original_text, updated_text
        except ValueError as e:
            processed = "".join(processed)
            err = e.args[0]
            raise ValueError(f"{processed}\n^^^ {err}")
        except IndexError:
            processed = "".join(processed)
>           raise ValueError(f"{processed}\n^^^ Incomplete ORIGINAL/UPDATED block.")
E           ValueError:
E           No problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:
E
E           ```python
E           tests/test_repomap.py
E           <<<<<<< ORIGINAL
E               def test_check_for_ctags_failure(self):
E                   with patch("subprocess.run") as mock_run:
E                       mock_run.side_effect = Exception("ctags not found")
E           =======
E
E           <<<<<<< ORIGINAL
E
E           ^^^ Incomplete ORIGINAL/UPDATED block.

aider/utils.py:259: ValueError
=========================== short test summary info ============================
FAILED tests/test_utils.py::TestUtils::test_false_incomplete_edit_block - Val...
========================= 1 failed, 10 passed in 0.04s =========================

ASSISTANT: It seems that the test `test_false_incomplete_edit_block` is failing due to an IndexError in the `find_original_update_blocks` function in `aider/utils.py`. I will fix the issue by handling the IndexError properly.

```python
aider/utils.py
<<<<<<< ORIGINAL
    if not len(filename) or "`" in filename:
        filename = processed[-2].splitlines()[-2].strip()
        if not len(filename) or "`" in filename:
            raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
=======
    try:
        if not len(filename) or "`" in filename:
            filename = processed[-2].splitlines()[-2].strip()
            if not len(filename) or "`" in filename:
                raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
    except IndexError:
        raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
>>>>>>> UPDATED
```

Now, the IndexError should be handled properly, and the test should pass. Please run the tests again to confirm the fix.
2023-05-29 07:09:09 -07:00
Paul Gauthier
92cd2f3ce6 isort 2023-05-28 22:47:38 -07:00
Paul Gauthier
7e6ea0660d Refactor show_messages function to accept an optional title parameter and remove unused code. 2023-05-19 10:57:24 -07:00
Paul Gauthier
07da14d3c7 make line numbers optional 2023-05-18 10:04:13 -07:00
Paul Gauthier
891d5b2263 aider: Added line numbers to the output of the get_quoted_file function.
# Context:
USER: git get_files_content prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT: To add line numbers to the `get_files_content` method, we will modify the method in the `aider/coder.py` file. We will read the file content line by line and add the line number with padding to each line.

Here's the change:

```python
aider/coder.py
<<<<<<< ORIGINAL
    def get_files_content(self, fnames=None):
        if not fnames:
            fnames = self.abs_fnames

        prompt = ""
        for fname in fnames:
            relative_fname = self.get_rel_fname(fname)
            prompt += utils.quoted_file(fname, relative_fname)
        return prompt
=======
    def get_files_content(self, fnames=None):

^C KeyboardInterrupt
USER: sorry, in get_quoted_file prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT:
^C KeyboardInterrupt
USER: Please note that I shared content of these additional files: aider/utils.py
ASSISTANT: To add line
^C KeyboardInterrupt
USER: in get_quoted_file prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT: To add line numbers to the `get_quoted_file` function in the `aider/utils.py` file, we will read the file content line by line and add the line number with padding to each line.

Here's the change:

```python
aider/utils.py
<<<<<<< ORIGINAL
def quoted_file(fname, display_fname):
    prompt = "\n"
    prompt += display_fname
    prompt += "\n```\n"
    prompt += Path(fname).read_text()
    prompt += "\n```\n"
    return prompt
=======
def quoted_file(fname, display_fname):
    prompt = "\n"
    prompt += display_fname
    prompt += "\n```\n"
    file_content = Path(fname).read_text()
    lines = file_content.splitlines()
    for i, line in enumerate(lines, start=1):
        prompt += f"{i:3d} {line}\n"
    prompt += "```\n"
    return prompt
>>>>>>> UPDATED
```

This change will add line numbers to the output of the `get_quoted_file` function, starting at 1 and padded out like '  1'.
2023-05-18 07:00:50 -07:00
Paul Gauthier
9a413d351d black formatting 2023-05-17 18:24:35 -07:00
Paul Gauthier
cc0ee034f4 aider: Added a check to ensure that part appears in whole exactly once in try_dotdotdots.
# Context:
USER: in try_dotdotdots, after checking that part is in whole add a check that part is in whole exactly 1 time
ASSISTANT: ```python
aider/utils.py
<<<<<<< ORIGINAL
        if part not in whole:
            raise ValueError("No perfect matching chunk in edit block with ...")

        whole = whole.replace(part, replace)
=======
        if whole.count(part) != 1:
            raise ValueError("No perfect matching chunk in edit block with ... or part appears more than once")

        whole = whole.replace(part, replace, 1)
>>>>>>> UPDATED
```
2023-05-17 16:15:09 -07:00
Paul Gauthier
0c05e3d957 Handle case where part="" and replace is non-empty 2023-05-14 11:07:22 -07:00
Paul Gauthier
e9debe71e8 Attempt to honot edit blocks which elide content using ... 2023-05-14 11:00:45 -07:00
Paul Gauthier
ab68022fb9 wip: Added a dry-run option to do_replace function. 2023-05-14 10:15:08 -07:00
Paul Gauthier
0b8690654e aider: Added dry_run parameter to do_replace function and passed it from Coder class.
# Context:
USER: pass dry run into do_replace and only write to the file if false
ASSISTANT: To implement this change, we need to modify the `do_replace` function in the `aider/utils.py` file. We will add a new parameter `dry_run` to the function and only write to the file if `dry_run` is `False`.

Here's the change:

```python
aider/utils.py
<<<<<<< ORIGINAL
def do_replace(fname, before_text, after_text):
=======
def do_replace(fname, before_text, after_text, dry_run=False):
>>>>>>> UPDATED
    before_text = strip_quoted_wrapping(before_text, fname)
    after_text = strip_quoted_wrapping(after_text, fname)
    fname = Path(fname)

    # does it want to make a new file?
    if not fname.exists() and not before_text.strip():
        fname.touch()

    content = fname.read_text()

    if not before_text.strip():
        if content:
            new_content = content + after_text
        else:
            # first populating an empty file
            new_content = after_text
    else:
        new_content = replace_most_similar_chunk(content, before_text, after_text)
        if not new_content:
            return

    if not dry_run:
        fname.write_text(new_content)
    return True
```

Now, you need to pass the `dry_run` parameter from the `Coder` class in the `aider/coder.py` file to the `do_replace` function. Here's the change:

```python
aider/coder.py
<<<<<<< ORIGINAL
if utils.do_replace(full_path, original, updated):
=======
if utils.do_replace(full_path, original, updated, self.dry_run):
>>>>>>> UPDATED
    self.io.tool(f"Applied edit to {path}")
else:
    self.io.tool_error(f"Failed to apply edit to {path}")
```

These changes will ensure that the file is only written to when `dry_run` is `False`.
2023-05-14 10:14:20 -07:00
Paul Gauthier
92e091f5a6 fix bug if ORIG/UPD block is missing trailing newline; added test 2023-05-12 13:30:39 -07:00
Paul Gauthier
d45b7ae752 handle filenames above the triple quotes 2023-05-11 22:59:02 -07:00
Paul Gauthier
6e4f17d377 cleanup 2023-05-11 22:06:32 -07:00
Paul Gauthier
684b0e4964 Provide structured errors to GPT if it generates malformed ORIG/UPD blocks 2023-05-11 22:06:02 -07:00
Paul Gauthier
ae41782cb4 wip 2023-05-11 21:05:15 -07:00
Paul Gauthier
797372c69e refactor orig/upd into utils 2023-05-11 20:57:27 -07:00
Paul Gauthier
6133eb61c1 fix tests and bug 2023-05-11 17:15:19 -07:00
Paul Gauthier
35a4a14a6a Avoid sending 2 copies of the added files 2023-05-10 19:13:33 -07:00
Paul Gauthier
b139dbf475 refactor 2023-05-10 18:41:36 -07:00
Paul Gauthier
79d38b5d68 handle the simple replace case quickly 2023-05-10 18:21:41 -07:00
Paul Gauthier
923e42c176 fixed off by 1 error in replace_most_similar_chunk() 2023-05-10 16:22:08 -07:00
Paul Gauthier
2339dd4803 aider: Added display_fname arg to quoted_file and updated call to quoted_file to pass relative file path.
# Context:
USER: add a display_fname arg to quoted_file. add it, not fname, into the prompt. change the call to quoted_file to pass along the fname relative to self.root
ASSISTANT: To add a `display_fname` argument to the `quoted_file` function and update the call to `quoted_file`, follow these changes:

1. Add the `display_fname` argument to the `quoted_file` function definition.
2. Replace `fname` with `display_fname` in the prompt string.
3. Update the call to `quoted_file` in the `get_files_content` method to pass the relative file path.

Here are the code changes:

```python
aider/utils.py
<<<<<<< ORIGINAL
def quoted_file(fname):
    prompt = "\n"
    prompt += fname
=======
def quoted_file(fname, display_fname):
    prompt = "\n"
    prompt += display_fname
>>>>>>> UPDATED
    prompt += "\n```\n"
    prompt += Path(fname).read_text()
    prompt += "\n```\n"
    return prompt
```

```python
aider/coder.py
<<<<<<< ORIGINAL
        for fname in self.fnames:
            prompt += utils.quoted_file(fname)
=======
        for fname in self.fnames:
            relative_fname = os.path.relpath(fname, self.root)
            prompt += utils.quoted_file(fname, relative_fname)
>>>>>>> UPDATED
        return prompt
```
2023-05-10 09:24:27 -07:00
Paul Gauthier
984e2f78ce moved into subdir 2023-05-09 10:52:18 -07:00
Renamed from utils.py (Browse further)