Commit graph

119 commits

Author SHA1 Message Date
Paul Gauthier (aider)
edf2b7fc13 Captured pip install output line by line, printing each line with a carriage return to overwrite, and displaying full output on failure. 2024-07-12 17:36:36 +01:00
Paul Gauthier
504a72b5cb handle failed install of hf-embed 2024-07-10 17:35:13 +01:00
Paul Gauthier
f688c18b65 offer to install aider[hf] 2024-07-10 16:20:06 +01:00
John-Mason P. Shackelford
dd6a7964b6
tempdirs in test_main now cleanup without windows errors 2024-06-18 11:57:53 -04:00
Daniel Vainsencher
abf8d5e60d aider: Refactored format_messages to use a new function format_content for line-by-line formatting.
# Aider chat conversation:

USER: Now refactor format_messages: extract a function that takes a role and content (where content is a string), and returns the line by line formatted string.

ASSISTANT: To refactor `format_messages`, we will:

1. Create a new function `format_content` that takes a role and content (where content is a string) and returns the line-by-line formatted string.
2. Update `format_messages` to use `format_content`.

Here are the *SEARCH/REPLACE* blocks:

aider/utils.py
```python
<<<<<<< SEARCH
def format_messages(messages, title=None):
=======
def format_content(role, content):
    formatted_lines = []
    for line in content.splitlines():
        formatted_lines.append(f"{role} {line}")
    return "\n".join(formatted_lines)

>>>>>>> REPLACE
```

aider/utils.py
```python
<<<<<<< SEARCH
        elif isinstance(content, str):  # Handle string content
            for line in content.splitlines():
                output.append(f"{role} {line}")
=======
        elif isinstance(content, str):  # Handle string content
            output.append(format_content(role, content))
>>>>>>> REPLACE
```

USER:
# Fix any errors below, if possible.

## Running: flake8 --select=E9,F821,F823,F831,F406,F407,F701,F702,F704,F706 --show-source --isolated aider/utils.py

aider/utils.py:94:8: F821 undefined name 'title'
    if title:
       ^
aider/utils.py:95:26: F821 undefined name 'title'
        output.append(f"{title.upper()} {'*' * 50}")
                         ^
aider/utils.py:97:16: F821 undefined name 'messages'
    for msg in messages:
               ^
aider/utils.py:115:24: F821 undefined name 'format_messages'
    formatted_output = format_messages(messages, title)
                       ^

## See relevant lines below marked with █.

aider/utils.py:
...⋮...
 87│def format_content(role, content):
 88│    formatted_lines = []
...⋮...
 91│    return "\n".join(formatted_lines)
 92│
 93│    output = []
 94█    if title:
 95█        output.append(f"{title.upper()} {'*' * 50}")
 96│
 97█    for msg in messages:
 98│        output.append("")
 99│        role = msg["role"].upper()
100│        content = msg.get("content")
...⋮...
112│
113│
114│def show_messages(messages, title=None, functions=None):
115█    formatted_output = format_messages(messages, title)
116│    print(formatted_output)
117│
118│    if functions:
...⋮...

ASSISTANT: To fix the errors, we need to ensure that the `format_messages` function is properly defined and that the misplaced code is correctly organized. Here are the corrected *SEARCH/REPLACE* blocks:

aider/utils.py
```python
<<<<<<< SEARCH
    return "\n".join(formatted_lines)

    output = []
    if title:
        output.append(f"{title.upper()} {'*' * 50}")

    for msg in messages:
        output.append("")
        role = msg["role"].upper()
        content = msg.get("content")
        if isinstance(content, list):  # Handle list content (e.g., image messages)
            for item in content:
                if isinstance(item, dict) and "image_url" in item:
                    output.append(f"{role} Image URL: {item['image_url']['url']}")
        elif isinstance(content, str):  # Handle string content
            output.append(format_content(role, content))
        content = msg.get("function_call")
        if content:
            output.append(f"{role} {content}")

    return "\n".join(output)
=======
    return "\n".join(formatted_lines)

def format_messages(messages, title=None):
    output = []
    if title:
        output.append(f"{title.upper()} {'*' * 50}")

    for msg in messages:
        output.append("")
        role = msg["role"].upper()
        content = msg.get("content")
        if isinstance(content, list):  # Handle list content (e.g., image messages)
            for item in content:
                if isinstance(item, dict) and "image_url" in item:
                    output.append(f"{role} Image URL: {item['image_url']['url']}")
        elif isinstance(content, str):  # Handle string content
            output.append(format_content(role, content))
        content = msg.get("function_call")
        if content:
            output.append(f"{role} {content}")

    return "\n".join(output)
>>>>>>> REPLACE
```
2024-06-07 09:39:03 -04:00
Daniel Vainsencher
a0b279f105 Refactored format_messages to use format_content for string content handling. 2024-06-07 09:39:03 -04:00
Daniel Vainsencher
69c27d6cd1 aider: Refactored show_messages to call format_messages for formatting title and messages.
# Aider chat conversation:

USER: Refactor show_messages in utils, so that the formatting of title and messages is done in a separate function which returns a string. show_messages should call it and print its output.

ASSISTANT: To refactor the `show_messages` function, we will:

1. Create a new function `format_messages` that formats the title and messages and returns a string.
2. Update `show_messages` to call `format_messages` and print its output.

Here are the *SEARCH/REPLACE* blocks:

aider/utils.py
```python
<<<<<<< SEARCH
def show_messages(messages, title=None, functions=None):
    if title:
        print(title.upper(), "*" * 50)

    for msg in messages:
        print()
        role = msg["role"].upper()
        content = msg.get("content")
        if isinstance(content, list):  # Handle list content (e.g., image messages)
            for item in content:
                if isinstance(item, dict) and "image_url" in item:
                    print(role, "Image URL:", item["image_url"]["url"])
        elif isinstance(content, str):  # Handle string content
            for line in content.splitlines():
                print(role, line)
        content = msg.get("function_call")
        if content:
            print(role, content)

    if functions:
        dump(functions)
=======
def format_messages(messages, title=None):
    output = []
    if title:
        output.append(f"{title.upper()} {'*' * 50}")

    for msg in messages:
        output.append("")
        role = msg["role"].upper()
        content = msg.get("content")
        if isinstance(content, list):  # Handle list content (e.g., image messages)
            for item in content:
                if isinstance(item, dict) and "image_url" in item:
                    output.append(f"{role} Image URL: {item['image_url']['url']}")
        elif isinstance(content, str):  # Handle string content
            for line in content.splitlines():
                output.append(f"{role} {line}")
        content = msg.get("function_call")
        if content:
            output.append(f"{role} {content}")

    return "\n".join(output)

def show_messages(messages, title=None, functions=None):
    formatted_output = format_messages(messages, title)
    print(formatted_output)

    if functions:
        dump(functions)
>>>>>>> REPLACE
```
2024-06-07 09:39:03 -04:00
Paul Gauthier
478b5c8e7c improved utils.split_chat_history_markdown 2024-05-11 08:06:16 -07:00
Paul Gauthier
602a0c7c31 refac utils.split_chat_history_markdown 2024-05-11 07:52:06 -07:00
Paul Gauthier
7fd3b8aeee Tell the AI we are switching code bases 2024-04-30 20:51:24 -07:00
Paul Gauthier
0da1b59901 Fixed up images in chat 2024-04-18 14:39:32 -07:00
Paul Gauthier
c770fc4380 cleaned up client refs 2024-04-17 15:47:07 -07:00
Paul Gauthier
5b78d929a6 Roughly working scraper 2024-02-08 11:44:55 -08:00
Paul Gauthier
1758937042 wip 2024-02-08 07:18:39 -08:00
Joshua Vial
179b648864 merging from upstream main 2023-12-19 22:04:39 +13:00
Paul Gauthier
7113a30271 unified diffs 2023-12-17 12:54:34 -08:00
Joshua Vial
90d5071709 fix failing tests 2023-12-11 22:37:23 +13:00
Joshua Vial
9ceaf97f08 making image code more robust 2023-12-11 22:25:35 +13:00
Joshua Vial
f9ba8e7b41 Remove unnecessary comment and method call in Commands class. 2023-12-11 22:24:02 +13:00
Joshua Vial
c919f9f0c6 handle switching to gpt4-vision-preview 2023-12-11 22:23:54 +13:00
Joshua Vial
d8f33a8124 Auto switch to gpt-4-vision-preview if image files added to context 2023-11-29 22:48:45 +13:00
Paul Gauthier
2c32fe5eb8 Adopt safe_abs_path 2023-07-07 17:17:33 -07:00
Paul Gauthier
9cab570f87 refactor to use io.read_text 2023-07-05 20:21:32 -07:00
Paul Gauthier
58e763cee7 roughed in diff-func-string 2023-06-29 15:10:33 -07:00
Paul Gauthier
6c87d5be1c better 2023-06-28 13:52:43 -07:00
Paul Gauthier
96fda2aa9f abandon n-backticks, try some html fences 2023-06-25 20:23:36 -07:00
Paul Gauthier
9151cf03a1 quote with num_ticks 2023-06-25 19:58:17 -07:00
Paul Gauthier
f005404ec0 Roughed in updated sys msg for whole format 2023-06-25 15:27:06 -07:00
Paul Gauthier
5e63ce3352 refac 2023-06-20 17:04:06 -07:00
Paul Gauthier
44198e8b3b Fix for issue #24, provided by @wwestenbrink 2023-06-20 06:57:19 -07:00
Paul Gauthier
806805c0b1 Ensure leading_whitespace is actually space 2023-06-10 06:16:22 -07:00
Paul Gauthier
8d408e1fc4 Try replace_part_with_missing_leading_whitespace first, so that single line edits get the benefit 2023-06-09 15:05:34 -07:00
Paul Gauthier
5612c8e13d Don't indent blank lines 2023-06-09 14:51:37 -07:00
Paul Gauthier
5a3fd35c16 If all lines in the part start with whitespace, then honor it. 2023-06-09 14:50:21 -07:00
Paul Gauthier
329baf3717 Fix the actual replace step 2023-06-09 14:46:43 -07:00
Paul Gauthier
1daf175c83 aider: Implemented replace_part_with_missing_leading_whitespace function to find and replace the portion of whole that matches part with the same leading whitespace and replace it with replace containing the same leading whitespace.
# Context:
USER: in `replace_part_with_missing_leading_whitespace`, we want to find the string part in whole and replace it with the string `replace`. but `part` may not exactly occur in whole. instead, every line of part appears in whole with a fixed amount of extra whitespace in front of it. so every line in part might appear in whole with 4 spaces prepended. or every line might have 2 tabs. always the exact same whitespaces. finish implementing this function so that it finds the portion of whole which matches part in this manner (if any) and replace it with `replace` where replace is updated to contain that repeated/omitted whitespace.
ASSISTANT: To implement the `replace_part_with_missing_leading_whitespace` function, follow these steps:

1. Split `whole` and `part` into lines.
2. Iterate through the lines of `whole` and find the first line that matches the first line of `part` with any leading whitespace.
3. Check if the following lines in `whole` also match the remaining lines in `part` with the same leading whitespace.
4. If all lines match, replace the matched lines in `whole` with the lines in `replace` with the same leading whitespace.
5. Return the modified `whole` or `None` if no match is found.

Here's the updated function:

```python
aider/utils.py
<<<<<<< ORIGINAL
def replace_part_with_missing_leading_whitespace(whole, part, replace):
    pass
=======
def replace_part_with_missing_leading_whitespace(whole, part, replace):
    whole_lines = whole.splitlines()
    part_lines = part.splitlines()
    replace_lines = replace.splitlines()

    for i in range(len(whole_lines) - len(part_lines) + 1):
        leading_whitespace = ""
        for j, c in enumerate(whole_lines[i]):
            if c == part_lines[0][0]:
                leading_whitespace = whole_lines[i][:j]
                break

        if not leading_whitespace:
            continue

        matched = all(
            whole_lines[i + k].startswith(leading_whitespace + part_lines[k])
            for k in range(len(part_lines))
        )

        if matched:
            for k in range(len(part_lines)):
                whole_lines[i + k] = leading_whitespace + replace_lines[k]
            return "\n".join(whole_lines)

    return None
>>>>>>> UPDATED
```

This implementation should find the portion of `whole` that matches `part` with the same leading whitespace and replace it with `replace` containing the same leading whitespace.
2023-06-09 14:37:08 -07:00
Paul Gauthier
490796de78 wip 2023-06-09 14:32:45 -07:00
Paul Gauthier
e4d9e71f5b simplified append/create 2023-06-09 14:27:17 -07:00
Paul Gauthier
9cef379abd refac 2023-06-07 12:28:45 -07:00
Paul Gauthier
95b32a74a9 Moved all model names into an enum 2023-06-05 09:19:29 -07:00
Paul Gauthier
c0bf25d05b undo trailing newline "fix" 2023-06-04 15:03:13 -07:00
Paul Gauthier
f47f611e65 Handle empty replacement text too 2023-06-04 14:30:22 -07:00
Paul Gauthier
fe1271114f Avoid stripping a newline when doing nearest search/replace 2023-06-04 14:28:03 -07:00
Paul Gauthier
ca74eee2f9 Allow GPT to omit the filename from edit blocks, if an earlier block provided it 2023-05-30 11:45:21 -07:00
Paul Gauthier
445c253abc fixed corner case bugs in find_original_update_blocks 2023-05-29 07:12:57 -07:00
Paul Gauthier
87c902b285 aider: Fixed IndexError in find_original_update_blocks function.
# Context:
USER: I ran this command:

 pytest tests/test_utils.py

And got this output:

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/gauthier/Projects/aider
collected 11 items

tests/test_utils.py F..........                                          [100%]

=================================== FAILURES ===================================
__________________ TestUtils.test_false_incomplete_edit_block __________________

content = '\nNo problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:\n\n`...h `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.\n'

    def find_original_update_blocks(content):
        # make sure we end with a newline, otherwise the regex will miss <<UPD on the last line
        if not content.endswith("\n"):
            content = content + "\n"

        pieces = re.split(split_re, content)

        pieces.reverse()
        processed = []

        try:
            while pieces:
                cur = pieces.pop()

                if cur in (DIVIDER, UPDATED):
                    processed.append(cur)
                    raise ValueError(f"Unexpected {cur}")

                if cur.strip() != ORIGINAL:
                    processed.append(cur)
                    continue

                processed.append(cur)  # original_marker

                filename = processed[-2].splitlines()[-1].strip()
                if not len(filename) or "`" in filename:
>                   filename = processed[-2].splitlines()[-2].strip()
E                   IndexError: list index out of range

aider/utils.py:234: IndexError

During handling of the above exception, another exception occurred:

self = <tests.test_utils.TestUtils testMethod=test_false_incomplete_edit_block>

        def test_false_incomplete_edit_block(self):
            edit = """
    No problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:

    ```python
    tests/test_repomap.py
    <<<<<<< ORIGINAL
        def test_check_for_ctags_failure(self):
            with patch("subprocess.run") as mock_run:
                mock_run.side_effect = Exception("ctags not found")
    =======
        def test_check_for_ctags_failure(self):
            with patch("subprocess.check_output") as mock_check_output:
                mock_check_output.side_effect = Exception("ctags not found")
    >>>>>>> UPDATED

    <<<<<<< ORIGINAL
        def test_check_for_ctags_success(self):
            with patch("subprocess.run") as mock_run:
                mock_run.return_value = CompletedProcess(args=["ctags", "--version"], returncode=0, stdout='''{
      "_type": "tag",
      "name": "status",
      "path": "aider/main.py",
      "pattern": "/^    status = main()$/",
      "kind": "variable"
    }''')
    =======
        def test_check_for_ctags_success(self):
            with patch("subprocess.check_output") as mock_check_output:
                mock_check_output.return_value = '''{
      "_type": "tag",
      "name": "status",
      "path": "aider/main.py",
      "pattern": "/^    status = main()$/",
      "kind": "variable"
    }'''
    >>>>>>> UPDATED
    ```

    These changes replace the `subprocess.run` patches with `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.
    """
            # Should not raise a ValueError
>           list(utils.find_original_update_blocks(edit))

tests/test_utils.py:199:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

content = '\nNo problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:\n\n`...h `subprocess.check_output` patches in both `test_check_for_ctags_failure` and `test_check_for_ctags_success` tests.\n'

    def find_original_update_blocks(content):
        # make sure we end with a newline, otherwise the regex will miss <<UPD on the last line
        if not content.endswith("\n"):
            content = content + "\n"

        pieces = re.split(split_re, content)

        pieces.reverse()
        processed = []

        try:
            while pieces:
                cur = pieces.pop()

                if cur in (DIVIDER, UPDATED):
                    processed.append(cur)
                    raise ValueError(f"Unexpected {cur}")

                if cur.strip() != ORIGINAL:
                    processed.append(cur)
                    continue

                processed.append(cur)  # original_marker

                filename = processed[-2].splitlines()[-1].strip()
                if not len(filename) or "`" in filename:
                    filename = processed[-2].splitlines()[-2].strip()
                    if not len(filename) or "`" in filename:
                        raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")

                original_text = pieces.pop()
                processed.append(original_text)

                divider_marker = pieces.pop()
                processed.append(divider_marker)
                if divider_marker.strip() != DIVIDER:
                    raise ValueError(f"Expected {DIVIDER}")

                updated_text = pieces.pop()

                updated_marker = pieces.pop()
                if updated_marker.strip() != UPDATED:
                    raise ValueError(f"Expected {UPDATED}")

                yield filename, original_text, updated_text
        except ValueError as e:
            processed = "".join(processed)
            err = e.args[0]
            raise ValueError(f"{processed}\n^^^ {err}")
        except IndexError:
            processed = "".join(processed)
>           raise ValueError(f"{processed}\n^^^ Incomplete ORIGINAL/UPDATED block.")
E           ValueError:
E           No problem! Here are the changes to patch `subprocess.check_output` instead of `subprocess.run` in both tests:
E
E           ```python
E           tests/test_repomap.py
E           <<<<<<< ORIGINAL
E               def test_check_for_ctags_failure(self):
E                   with patch("subprocess.run") as mock_run:
E                       mock_run.side_effect = Exception("ctags not found")
E           =======
E
E           <<<<<<< ORIGINAL
E
E           ^^^ Incomplete ORIGINAL/UPDATED block.

aider/utils.py:259: ValueError
=========================== short test summary info ============================
FAILED tests/test_utils.py::TestUtils::test_false_incomplete_edit_block - Val...
========================= 1 failed, 10 passed in 0.04s =========================

ASSISTANT: It seems that the test `test_false_incomplete_edit_block` is failing due to an IndexError in the `find_original_update_blocks` function in `aider/utils.py`. I will fix the issue by handling the IndexError properly.

```python
aider/utils.py
<<<<<<< ORIGINAL
    if not len(filename) or "`" in filename:
        filename = processed[-2].splitlines()[-2].strip()
        if not len(filename) or "`" in filename:
            raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
=======
    try:
        if not len(filename) or "`" in filename:
            filename = processed[-2].splitlines()[-2].strip()
            if not len(filename) or "`" in filename:
                raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
    except IndexError:
        raise ValueError(f"Bad/missing filename. It should go right above {ORIGINAL}")
>>>>>>> UPDATED
```

Now, the IndexError should be handled properly, and the test should pass. Please run the tests again to confirm the fix.
2023-05-29 07:09:09 -07:00
Paul Gauthier
92cd2f3ce6 isort 2023-05-28 22:47:38 -07:00
Paul Gauthier
7e6ea0660d Refactor show_messages function to accept an optional title parameter and remove unused code. 2023-05-19 10:57:24 -07:00
Paul Gauthier
07da14d3c7 make line numbers optional 2023-05-18 10:04:13 -07:00
Paul Gauthier
891d5b2263 aider: Added line numbers to the output of the get_quoted_file function.
# Context:
USER: git get_files_content prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT: To add line numbers to the `get_files_content` method, we will modify the method in the `aider/coder.py` file. We will read the file content line by line and add the line number with padding to each line.

Here's the change:

```python
aider/coder.py
<<<<<<< ORIGINAL
    def get_files_content(self, fnames=None):
        if not fnames:
            fnames = self.abs_fnames

        prompt = ""
        for fname in fnames:
            relative_fname = self.get_rel_fname(fname)
            prompt += utils.quoted_file(fname, relative_fname)
        return prompt
=======
    def get_files_content(self, fnames=None):

^C KeyboardInterrupt
USER: sorry, in get_quoted_file prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT:
^C KeyboardInterrupt
USER: Please note that I shared content of these additional files: aider/utils.py
ASSISTANT: To add line
^C KeyboardInterrupt
USER: in get_quoted_file prefix each line with a line number, starting at 1, padded out like '  1'
ASSISTANT: To add line numbers to the `get_quoted_file` function in the `aider/utils.py` file, we will read the file content line by line and add the line number with padding to each line.

Here's the change:

```python
aider/utils.py
<<<<<<< ORIGINAL
def quoted_file(fname, display_fname):
    prompt = "\n"
    prompt += display_fname
    prompt += "\n```\n"
    prompt += Path(fname).read_text()
    prompt += "\n```\n"
    return prompt
=======
def quoted_file(fname, display_fname):
    prompt = "\n"
    prompt += display_fname
    prompt += "\n```\n"
    file_content = Path(fname).read_text()
    lines = file_content.splitlines()
    for i, line in enumerate(lines, start=1):
        prompt += f"{i:3d} {line}\n"
    prompt += "```\n"
    return prompt
>>>>>>> UPDATED
```

This change will add line numbers to the output of the `get_quoted_file` function, starting at 1 and padded out like '  1'.
2023-05-18 07:00:50 -07:00