Commit graph

2344 commits

Author SHA1 Message Date
paul-gauthier
c5f8f564a4
Merge pull request #58 from paul-gauthier/encodings
Use an explicit encoding to read/write text files, utf-8 by default
2023-07-06 13:40:02 -07:00
Paul Gauthier
95a777a48c More accurate /tokens 2023-07-06 13:35:17 -07:00
Paul Gauthier
6cb4428cb7 Merge branch 'main' into encodings 2023-07-06 13:33:53 -07:00
Paul Gauthier
1291c2424a better message for no /ls output 2023-07-06 12:52:33 -07:00
Paul Gauthier
274caab6b1 Merge branch 'main' into glob-add 2023-07-06 12:43:53 -07:00
Paul Gauthier
09f79e5a42 fix PermErr in test on windows 2023-07-06 12:37:32 -07:00
Paul Gauthier
0f1af1d991 improve filehandle hygene 2023-07-06 12:29:49 -07:00
Paul Gauthier
656af5cd83 rename did not help 2023-07-06 12:26:29 -07:00
Paul Gauthier
2b69c11672 Try renaming to avoid windows PermErr 2023-07-06 12:08:07 -07:00
Paul Gauthier
4a15b6c010 test_run_with_file_utf_unicode_error 2023-07-06 11:57:44 -07:00
Paul Gauthier
7cadfd9b4c aider: Changed encoding to 'latin-1' and provided content that will cause an error when read with 'utf-8' encoding in the test_run_with_file_utf_unicode_error method.
# Context:
USER: do the TODOs
ASSISTANT: I see two TODOs in the provided code.

1. In the `test_run_with_file_utf_unicode_error` method, you need to specify an encoding other than 'utf-8'.
2. In the same method, you need to provide content that will cause an error when read with 'utf-8' encoding.

Here are the proposed changes:

```python
tests/test_coder.py
<<<<<<< ORIGINAL
        encoding = 'pick something other than utf-8' # TODO
=======
        encoding = 'latin-1'
>>>>>>> UPDATED
```

```python
tests/test_coder.py
<<<<<<< ORIGINAL
        some_content_which_will_error_if_read_with_encoding_utf8 = '' # TODO
=======
        some_content_which_will_error_if_read_with_encoding_utf8 = 'ÅÍÎÏ'.encode('utf-16')
>>>>>>> UPDATED
```

In the first change, I've chosen 'latin-1' as the encoding. It's a common encoding that's different from 'utf-8'.

In the second change, I've provided a string of characters that are not valid 'utf-8' when encoded with 'utf-16'. This will cause an error when trying to read the file with 'utf-8' encoding.
2023-07-06 11:56:15 -07:00
Paul Gauthier
5ec5b60660 better /tokens test 2023-07-06 11:50:13 -07:00
Paul Gauthier
eb6ac6841f aider: Redirected the standard output to an instance of io.StringIO and captured the console output of cmd_tokens.
# Context:
USER: how can i capture the console output of `cmd_tokens`?
ASSISTANT: To capture the console output of `cmd_tokens`, you can modify the `test_cmd_tokens` method in the `tests/test_commands.py` file. You can use the `io.StringIO` class to capture the standard output.

Here's how you can do it:

1. Import the `io` module at the top of the file.
2. In the `test_cmd_tokens` method, redirect the standard output to an instance of `io.StringIO` before calling `commands.cmd_tokens("")`.
3. After calling `commands.cmd_tokens("")`, you can get the console output from the `io.StringIO` instance.

Here is the *edit block* for the changes:

```python
tests/test_commands.py
<<<<<<< ORIGINAL
import codecs
import os
import shutil
import tempfile
from unittest import TestCase

from aider import models
from aider.commands import Commands
from aider.io import InputOutput
=======
import codecs
import os
import shutil
import tempfile
from unittest import TestCase
import io

from aider import models
from aider.commands import Commands
from aider.io import InputOutput
>>>>>>> UPDATED

tests/test_commands.py
<<<<<<< ORIGINAL
    def test_cmd_tokens(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        commands.cmd_add("foo.txt bar.txt")
        commands.cmd_tokens("")
=======
    def test_cmd_tokens(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        commands.cmd_add("foo.txt bar.txt")

        # Redirect the standard output to an instance of io.StringIO
        stdout = io.StringIO()
        sys.stdout = stdout

        commands.cmd_tokens("")

        # Reset the standard output
        sys.stdout = sys.__stdout__

        # Get the console output
        console_output = stdout.getvalue()
        print(console_output)
>>>>>>> UPDATED
```

Please note that this will only capture the output of `print` statements. If the `cmd_tokens` method writes to the console in a different way (for example, by writing directly to `sys.stdout`), this might not capture that output.
2023-07-06 11:46:20 -07:00
Paul Gauthier
089fa57ede full edit for wholefile 2023-07-06 11:43:20 -07:00
Paul Gauthier
7d3c40ea21 test dry run 2023-07-06 11:34:19 -07:00
Paul Gauthier
7bbf0c0870 test full editblock edit 2023-07-06 11:32:31 -07:00
Paul Gauthier
610dd223b5 added test for choose_fence 2023-07-06 11:08:13 -07:00
Paul Gauthier
b03f2e54cc Merge branch 'main' into encodings 2023-07-06 10:59:45 -07:00
Paul Gauthier
ffee2c9511 test for unicode errors 2023-07-06 10:59:25 -07:00
Paul Gauthier
5647240207 aider: Added a new test test_run_with_file_unicode_error that expects a UnicodeDecodeError to be raised when the run method of the Coder object is called with a file containing non-UTF8 text.
# Context:
USER: I did `git reset --hard HEAD~1` to discard the last edits.
ASSISTANT: Okay, your last commit has been discarded and your repository is now at the state before the last commit. If you need to make any further changes or need assistance with anything else, feel free to ask.
USER: try again, but don't elide code with ...
ASSISTANT: Understood. Here's the updated *edit block* without eliding any code:

```python
tests/test_coder.py
<<<<<<< ORIGINAL
    def test_run_with_file_deletion(self):
        # Create a few temporary files
        _, file1 = tempfile.mkstemp()
        _, file2 = tempfile.mkstemp()

        files = [file1, file2]

        # Initialize the Coder object with the mocked IO and mocked repo
        coder = Coder.create(
            models.GPT4, None, io=InputOutput(), openai_api_key="fake_key", fnames=files
        )

        def mock_send(*args, **kwargs):
            coder.partial_response_content = "ok"
            coder.partial_response_function_call = dict()

        coder.send = MagicMock(side_effect=mock_send)

        # Call the run method with a message
        coder.run(with_message="hi")
        self.assertEqual(len(coder.abs_fnames), 2)

        # Delete one of the files
        os.remove(file1)

        # Call the run method again with a message
        coder.run(with_message="hi")
        self.assertEqual(len(coder.abs_fnames), 1)

if __name__ == "__main__":
    unittest.main()
=======
    def test_run_with_file_deletion(self):
        # Create a few temporary files
        _, file1 = tempfile.mkstemp()
        _, file2 = tempfile.mkstemp()

        files = [file1, file2]

        # Initialize the Coder object with the mocked IO and mocked repo
        coder = Coder.create(
            models.GPT4, None, io=InputOutput(), openai_api_key="fake_key", fnames=files
        )

        def mock_send(*args, **kwargs):
            coder.partial_response_content = "ok"
            coder.partial_response_function_call = dict()

        coder.send = MagicMock(side_effect=mock_send)

        # Call the run method with a message
        coder.run(with_message="hi")
        self.assertEqual(len(coder.abs_fnames), 2)

        # Delete one of the files
        os.remove(file1)

        # Call the run method again with a message
        coder.run(with_message="hi")
        self.assertEqual(len(coder.abs_fnames), 1)

    def test_run_with_file_unicode_error(self):
        # Create a temporary file
        _, file = tempfile.mkstemp()

        # Write some non-UTF8 text into the file
        with open(file, "wb") as f:
            f.write(b"\x80abc")

        # Initialize the Coder object with the temporary file
        coder = Coder.create(
            models.GPT4, None, io=InputOutput(), openai_api_key="fake_key", fnames=[file]
        )

        # Expect a UnicodeDecodeError to be raised when the run method is called
        with self.assertRaises(UnicodeDecodeError):
            coder.run(with_message="hi")

    if __name__ == "__main__":
        unittest.main()
>>>>>>> UPDATED
```

This new test `test_run_with_file_unicode_error` will write non-UTF8 text into a temporary file and expect a `UnicodeDecodeError` to be raised when the `run` method of the `Coder` object is called.
2023-07-06 10:58:36 -07:00
Paul Gauthier
3c443d68d0 working test 2023-07-06 10:55:38 -07:00
Paul Gauthier
79828096d1 copy 2023-07-06 06:55:49 -07:00
Paul Gauthier
c2abffe003 copy 2023-07-05 22:04:35 -07:00
Paul Gauthier
0590846b29 copy 2023-07-05 21:59:05 -07:00
Paul Gauthier
4e84ca97ea copy 2023-07-05 21:58:53 -07:00
Paul Gauthier
ca7d43ac4d Updated HISTORY 2023-07-05 21:56:38 -07:00
Paul Gauthier
d2c13b2976 aider: Refactored cmd_drop method to match glob pattern against the filename only, not the full file path.
# Context:
USER: I ran this command:

 pytest tests/test_commands.py

And got this output:

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.2.0
rootdir: /Users/gauthier/Projects/aider
configfile: pytest.ini
collected 3 items

tests/test_commands.py ..F                                               [100%]

=================================== FAILURES ===================================
________________ TestCommands.test_cmd_drop_with_glob_patterns _________________

self = <tests.test_commands.TestCommands testMethod=test_cmd_drop_with_glob_patterns>

    def test_cmd_drop_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        with open("test1.py", "w") as f:
            f.write("print('test1')")
        with open("test2.py", "w") as f:
            f.write("print('test2')")

        # Add some files to the chat session
        commands.cmd_add("*.py")

        # Call the cmd_drop method with a glob pattern
        commands.cmd_drop("*2.py")

        # Check if the Python files have been removed from the chat session
>       self.assertIn(os.path.abspath("test1.py"), coder.abs_fnames)
E       AssertionError: '/private/var/folders/49/kxrdwwbx0h9bchx99397477c0000gn/T/tmpijmz7big/test1.py' not found in set()

tests/test_commands.py:81: AssertionError
----------------------------- Captured stdout call -----------------------------
Model: gpt-3.5-turbo
Git repo: none
Repo-map: disabled
Added test1.py to the chat
Added test2.py to the chat
Removed test2.py from the chat
Removed test1.py from the chat
=========================== short test summary info ============================
FAILED tests/test_commands.py::TestCommands::test_cmd_drop_with_glob_patterns
========================= 1 failed, 2 passed in 0.98s ==========================

ASSISTANT: It seems like the test `test_cmd_drop_with_glob_patterns` is failing because the `cmd_drop` method is removing all files that match the glob pattern "*2.py", which includes "test1.py" and "test2.py". This is because the glob pattern is being matched against the full file path, not just the filename.

To fix this, we need to modify the `cmd_drop` method in the `Commands` class in the `aider/commands.py` file. Specifically, we need to change the way we match the glob pattern against the file paths.

Here are the changes:

```python
aider/commands.py
<<<<<<< ORIGINAL
    def cmd_drop(self, args):
        "Remove matching files from the chat session"

        if not args.strip():
            self.io.tool_output("Dropping all files from the chat session.")
            self.coder.abs_fnames = set()

        for word in args.split():
            matched_files = [
                file
                for file in self.coder.abs_fnames
                if any(glob.fnmatch.fnmatch(os.path.relpath(file, self.coder.root), word) for file in self.coder.abs_fnames)
            ]
            if not matched_files:
                self.io.tool_error(f"No files matched '{word}'")

            for matched_file in matched_files:
                relative_fname = os.path.relpath(matched_file, self.coder.root)
                self.coder.abs_fnames.remove(matched_file)
                self.io.tool_output(f"Removed {relative_fname} from the chat")
=======
    def cmd_drop(self, args):
        "Remove matching files from the chat session"

        if not args.strip():
            self.io.tool_output("Dropping all files from the chat session.")
            self.coder.abs_fnames = set()

        for word in args.split():
            matched_files = [
                file
                for file in self.coder.abs_fnames
                if glob.fnmatch.fnmatch(os.path.basename(file), word)
            ]
            if not matched_files:
                self.io.tool_error(f"No files matched '{word}'")

            for matched_file in matched_files:
                relative_fname = os.path.relpath(matched_file, self.coder.root)
                self.coder.abs_fnames.remove(matched_file)
                self.io.tool_output(f"Removed {relative_fname} from the chat")
>>>>>>> UPDATED
```

This change will ensure that the glob pattern is matched against the filename only, not the full file path. This should fix the failing test.
2023-07-05 21:42:36 -07:00
Paul Gauthier
927b97bea1 Added test1.py and test2.py files, dropped *2.py files, and checked if test1.py file is in the abs_fnames list. 2023-07-05 21:41:31 -07:00
Paul Gauthier
7f76559e6c aider: Added a test for the cmd_drop method with glob patterns in the TestCommands class.
# Context:
USER: add a test for that
ASSISTANT: To add a test for the `cmd_drop` method with glob patterns, we need to modify the `tests/test_commands.py` file.

Here's the step-by-step process:

1. Add a new test method `test_cmd_drop_with_glob_patterns` in the `TestCommands` class.
2. In this method, initialize the `Commands` and `InputOutput` objects.
3. Add some files to the chat session.
4. Call the `cmd_drop` method with a glob pattern.
5. Check if the correct files have been removed from the chat session.

Here's the code change:

```python
tests/test_commands.py
<<<<<<< ORIGINAL
    def test_cmd_add_no_match(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Call the cmd_add method with a non-existent file pattern
        commands.cmd_add("*.nonexistent")

        # Check if no files have been added to the chat session
        self.assertEqual(len(coder.abs_fnames), 0)
=======
    def test_cmd_add_no_match(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Call the cmd_add method with a non-existent file pattern
        commands.cmd_add("*.nonexistent")

        # Check if no files have been added to the chat session
        self.assertEqual(len(coder.abs_fnames), 0)

    def test_cmd_drop_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Add some files to the chat session
        commands.cmd_add("*.py")

        # Call the cmd_drop method with a glob pattern
        commands.cmd_drop("*.py")

        # Check if the Python files have been removed from the chat session
        self.assertNotIn(os.path.abspath("test1.py"), coder.abs_fnames)
        self.assertNotIn(os.path.abspath("test2.py"), coder.abs_fnames)
>>>>>>> UPDATED
```
Please let me know if you need any further changes.
2023-07-05 21:40:01 -07:00
Paul Gauthier
ad5cdc463a aider: Refactored cmd_drop to use glob for matching files against the given word.
# Context:
USER: have `cmd_drop` also glob against the `files`
ASSISTANT: To implement this change, we need to modify the `aider/commands.py` file.

Here's the step-by-step process:

1. In the `cmd_drop` method, use the `glob.glob` function to match the word against the `files`.
2. If no files match, print an error message.
3. If files match, remove them from the chat session.

Here's the code change:

```python
aider/commands.py
<<<<<<< ORIGINAL
    def cmd_drop(self, args):
        "Remove matching files from the chat session"

        if not args.strip():
            self.io.tool_output("Dropping all files from the chat session.")
            self.coder.abs_fnames = set()

        for word in args.split():
            matched_files = [
                file
                for file in self.coder.abs_fnames
                if word.lower() in os.path.relpath(file, self.coder.root).lower()
            ]
            if not matched_files:
                self.io.tool_error(f"No files matched '{word}'")

            for matched_file in matched_files:
                relative_fname = os.path.relpath(matched_file, self.coder.root)
                self.coder.abs_fnames.remove(matched_file)
                self.io.tool_output(f"Removed {relative_fname} from the chat")
=======
    def cmd_drop(self, args):
        "Remove matching files from the chat session"

        if not args.strip():
            self.io.tool_output("Dropping all files from the chat session.")
            self.coder.abs_fnames = set()

        for word in args.split():
            matched_files = [
                file
                for file in self.coder.abs_fnames
                if any(glob.fnmatch.fnmatch(os.path.relpath(file, self.coder.root), word) for file in self.coder.abs_fnames)
            ]
            if not matched_files:
                self.io.tool_error(f"No files matched '{word}'")

            for matched_file in matched_files:
                relative_fname = os.path.relpath(matched_file, self.coder.root)
                self.coder.abs_fnames.remove(matched_file)
                self.io.tool_output(f"Removed {relative_fname} from the chat")
>>>>>>> UPDATED
```
Please let me know if you need any further changes.
2023-07-05 21:38:44 -07:00
Paul Gauthier
5cd7d9d743 aider: aider: Modified cmd_add to skip file creation if word contains glob characters.
# Context:
USER: if word doesn't match anything, don't ask to add it if it contains glob characters
ASSISTANT: To implement this change, we need to modify the `aider/commands.py` file.

Here's the step-by-step process:

1. In the `cmd_add` method, before asking to create a file, check if the word contains any glob characters (`*`, `?`, `[`, `]`).
2. If it does, skip the creation process and print an error message.

Here's the code change:

```python
aider/commands.py
<<<<<<< ORIGINAL
            if not matched_files:
                if self.coder.repo is not None:
                    create_file = self.io.confirm_ask(
                        (
                            f"No files matched '{word}'. Do you want to create the file and add it"
                            " to git?"
                        ),
                    )
                else:
                    create_file = self.io.confirm_ask(
                        f"No files matched '{word}'. Do you want to create the file?"
                    )

                if create_file:
                    with open(os.path.join(self.coder.root, word), "w"):
                        pass
                    matched_files = [word]
                    if self.coder.repo is not None:
                        self.coder.repo.git.add(os.path.join(self.coder.root, word))
                        commit_message = f"aider: Created and added {word} to git."
                        self.coder.repo.git.commit("-m", commit_message, "--no-verify")
                else:
                    self.io.tool_error(f"No files matched '{word}'")
=======
            if not matched_files:
                if any(char in word for char in "*?[]"):
                    self.io.tool_error(f"No files matched '{word}' and it contains glob characters")
                else:
                    if self.coder.repo is not None:
                        create_file = self.io.confirm_ask(
                            (
                                f"No files matched '{word}'. Do you want to create the file and add it"
                                " to git?"
                            ),
                        )
                    else:
                        create_file = self.io.confirm_ask(
                            f"No files matched '{word}'. Do you want to create the file?"
                        )

                    if create_file:
                        with open(os.path.join(self.coder.root, word), "w"):
                            pass
                        matched_files = [word]
                        if self.coder.repo is not None:
                            self.coder.repo.git.add(os.path.join(self.coder.root, word))
                            commit_message = f"aider: Created and added {word} to git."
                            self.coder.repo.git.commit("-m", commit_message, "--no-verify")
                    else:
                        self.io.tool_error(f"No files matched '{word}'")
>>>>>>> UPDATED
```
Please let me know if you need any further changes.
2023-07-05 21:37:22 -07:00
Paul Gauthier
2b07e2de30 aider: Add a test for cmd_add where nothing matches.
# Context:
USER: add a test for cmd_add where nothing matches
ASSISTANT: To add a test for the `cmd_add` method where nothing matches, we need to modify the `tests/test_commands.py` file.

Here's the step-by-step process:

1. Add a new test method `test_cmd_add_no_match` in the `TestCommands` class.
2. In this method, initialize the `Commands` and `InputOutput` objects.
3. Call the `cmd_add` method with a non-existent file pattern.
4. Check if no files have been added to the chat session.

Here's the code change:

```python
tests/test_commands.py
<<<<<<< ORIGINAL
    def test_cmd_add_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Create some test files
        with open("test1.py", "w") as f:
            f.write("print('test1')")
        with open("test2.py", "w") as f:
            f.write("print('test2')")
        with open("test.txt", "w") as f:
            f.write("test")

        # Call the cmd_add method with a glob pattern
        commands.cmd_add("*.py")

        # Check if the Python files have been added to the chat session
        self.assertIn(os.path.abspath("test1.py"), coder.abs_fnames)
        self.assertIn(os.path.abspath("test2.py"), coder.abs_fnames)

        # Check if the text file has not been added to the chat session
        self.assertNotIn(os.path.abspath("test.txt"), coder.abs_fnames)
=======
    def test_cmd_add_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Create some test files
        with open("test1.py", "w") as f:
            f.write("print('test1')")
        with open("test2.py", "w") as f:
            f.write("print('test2')")
        with open("test.txt", "w") as f:
            f.write("test")

        # Call the cmd_add method with a glob pattern
        commands.cmd_add("*.py")

        # Check if the Python files have been added to the chat session
        self.assertIn(os.path.abspath("test1.py"), coder.abs_fnames)
        self.assertIn(os.path.abspath("test2.py"), coder.abs_fnames)

        # Check if the text file has not been added to the chat session
        self.assertNotIn(os.path.abspath("test.txt"), coder.abs_fnames)

    def test_cmd_add_no_match(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Call the cmd_add method with a non-existent file pattern
        commands.cmd_add("*.nonexistent")

        # Check if no files have been added to the chat session
        self.assertEqual(len(coder.abs_fnames), 0)
>>>>>>> UPDATED
```
Please let me know if you need any further changes.
2023-07-05 21:35:09 -07:00
Paul Gauthier
09605788d5 cleanup 2023-07-05 21:33:59 -07:00
Paul Gauthier
d97bf17e19 aider: Moved the import statement for the glob module to the top of the aider/commands.py file.
# Context:
USER: I ran this command:

 pytest tests/test_commands.py

And got this output:

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.2.0
rootdir: /Users/gauthier/Projects/aider
configfile: pytest.ini
collected 1 item

tests/test_commands.py F                                                 [100%]

=================================== FAILURES ===================================
_________________ TestCommands.test_cmd_add_with_glob_patterns _________________

self = <tests.test_commands.TestCommands testMethod=test_cmd_add_with_glob_patterns>

    def test_cmd_add_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Create some test files
        with open("test1.py", "w") as f:
            f.write("print('test1')")
        with open("test2.py", "w") as f:
            f.write("print('test2')")
        with open("test.txt", "w") as f:
            f.write("test")

        # Call the cmd_add method with a glob pattern
>       commands.cmd_add("*.py")

tests/test_commands.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <aider.commands.Commands object at 0x1113b2250>, args = '*.py'

    def cmd_add(self, args):
        "Add matching files to the chat session using glob patterns"

        added_fnames = []
        for pattern in args.split():
>           matched_files = glob.glob(pattern, recursive=True)
E           NameError: name 'glob' is not defined

aider/commands.py:226: NameError
----------------------------- Captured stdout call -----------------------------
Model: gpt-3.5-turbo
Git repo: none
Repo-map: disabled
=========================== short test summary info ============================
FAILED tests/test_commands.py::TestCommands::test_cmd_add_with_glob_patterns
============================== 1 failed in 0.92s ===============================

ASSISTANT: The error message indicates that the `glob` module is not defined in the `cmd_add` method of the `Commands` class in the `aider/commands.py` file. This is because the `glob` module is imported inside the `Commands` class, but it should be imported at the top of the file.

Here are the changes you need to make:

```python
aider/commands.py
<<<<<<< ORIGINAL
import json
import os
import shlex
import subprocess
import sys

import git
import tiktoken
from prompt_toolkit.completion import Completion

from aider import prompts, utils

class Commands:
    import glob
=======
import json
import os
import shlex
import subprocess
import sys
import glob

import git
import tiktoken
from prompt_toolkit.completion import Completion

from aider import prompts, utils

class Commands:
>>>>>>> UPDATED
```

This change moves the `import glob` statement from inside the `Commands` class to the top of the file, which is the standard place for import statements in Python. This will make the `glob` module available to all methods in the `Commands` class.
2023-07-05 21:30:57 -07:00
Paul Gauthier
7b4088c507 aider: Added a new test method test_cmd_add_with_glob_patterns to the TestCommands class in tests/test_commands.py to test the /add command with glob patterns.
# Context:
USER: add some tests
ASSISTANT: To add tests for the new functionality, we need to modify the `tests/test_commands.py` file.

We will add a new test method `test_cmd_add_with_glob_patterns` to the `TestCommands` class. This method will test the `/add` command with glob patterns.

Here is the proposed change:

```python
tests/test_commands.py
<<<<<<< ORIGINAL
    def test_cmd_add(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Call the cmd_add method with 'foo.txt' and 'bar.txt' as a single string
        commands.cmd_add("foo.txt bar.txt")

        # Check if both files have been created in the temporary directory
        self.assertTrue(os.path.exists("foo.txt"))
        self.assertTrue(os.path.exists("bar.txt"))
=======
    def test_cmd_add_with_glob_patterns(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Create some test files
        with open("test1.py", "w") as f:
            f.write("print('test1')")
        with open("test2.py", "w") as f:
            f.write("print('test2')")
        with open("test.txt", "w") as f:
            f.write("test")

        # Call the cmd_add method with a glob pattern
        commands.cmd_add("*.py")

        # Check if the Python files have been added to the chat session
        self.assertIn(os.path.abspath("test1.py"), coder.abs_fnames)
        self.assertIn(os.path.abspath("test2.py"), coder.abs_fnames)

        # Check if the text file has not been added to the chat session
        self.assertNotIn(os.path.abspath("test.txt"), coder.abs_fnames)
>>>>>>> UPDATED
```

This test creates three files: `test1.py`, `test2.py`, and `test.txt`. It then calls the `/add` command with the glob pattern `*.py`. The test checks that the Python files have been added to the chat session and that the text file has not been added.
2023-07-05 21:29:53 -07:00
Paul Gauthier
874b5f8c7a aider: The cmd_add method in the Commands class has been modified to allow the /add command to accept glob patterns.
# Context:
USER: Feature Description
Currently, the /add function accepts directory paths to add the entire directory or spefic file paths to the program's context. A useful enhancement would be to support the use of regex or glob expressions with the /add function. This would give users more flexibility and control to specify exactly which files to include.

Proposed Solution
Allow the /add function to interpret glob or regex patterns. For example, a user could use /add src/*.py to include only Python files from the src directory or /add src/[A-C]* to add only files that start with A, B, or C.

Expected Behavior
When a user runs /add with a regex or glob pattern, the program should add to the context only the files that match the pattern. The program should handle non-matching files or non-existent paths gracefully, preferably by ignoring them and printing a warning message.

Alternatives Considered
An alternative could be providing an additional function, say /addPattern, that accepts regex/glob patterns. This approach would leave the existing /add functionality unchanged for users who don't need the enhanced functionality.

Additional Context
Adding this feature would boost the flexibility of the /add function, making the program more convenient to use for a variety of tasks. Users could, for instance, exclude certain types of files or select a subset of files based on their naming conventions.
ASSISTANT: To implement this feature, we need to modify the `cmd_add` method in the `Commands` class in the `aider/commands.py` file.

The changes will involve:
1. Interpreting the input argument as a glob pattern.
2. Using the `glob` module to find all files that match the pattern.
3. Adding the matched files to the chat session.

Here is the proposed change:

```python
aider/commands.py
<<<<<<< ORIGINAL
    def cmd_add(self, args):
        "Add matching files to the chat session"

        added_fnames = []
        files = self.coder.get_all_relative_files()
        for word in args.split():
            matched_files = [file for file in files if word in file]
=======
    import glob

    def cmd_add(self, args):
        "Add matching files to the chat session using glob patterns"

        added_fnames = []
        for pattern in args.split():
            matched_files = glob.glob(pattern, recursive=True)
>>>>>>> UPDATED
```

This change will allow the `/add` command to accept glob patterns. For example, `/add src/*.py` will add all Python files in the `src` directory to the chat session.
2023-07-05 21:28:53 -07:00
Paul Gauthier
391088fd14 add self.dry_run 2023-07-05 21:22:12 -07:00
Paul Gauthier
85c3bb8f12 rely on io.dry_run 2023-07-05 21:21:37 -07:00
Paul Gauthier
4ee5c49a04 have io.write_text handle dry_run 2023-07-05 21:20:22 -07:00
Paul Gauthier
b46dffbb20 refactor to use io.write_text 2023-07-05 21:18:19 -07:00
Paul Gauthier
0ea52b412f cleanup 2023-07-05 21:11:46 -07:00
Paul Gauthier
1808c47d96 aider: Added write_text function to InputOutput class in aider/io.py.
# Context:
USER: add a write_text func
ASSISTANT: To add a `write_text` function to the `InputOutput` class in the `aider/io.py` file, we need to modify the `aider/io.py` file.

Here is the change:

```python
aider/io.py
<<<<<<< ORIGINAL

    def read_text(self, filename):
        try:
            with open(filename, "r", encoding=self.encoding) as f:
                return f.read()
        except (FileNotFoundError, UnicodeError) as e:
            self.tool_error(f"{filename}: {e}")
            return
=======

    def read_text(self, filename):
        try:
            with open(filename, "r", encoding=self.encoding) as f:
                return f.read()
        except (FileNotFoundError, UnicodeError) as e:
            self.tool_error(f"{filename}: {e}")
            return

    def write_text(self, filename, content):
        try:
            with open(filename, "w", encoding=self.encoding) as f:
                f.write(content)
        except (FileNotFoundError, UnicodeError) as e:
            self.tool_error(f"{filename}: {e}")
            return
>>>>>>> UPDATED
```

This `write_text` function takes a filename and content as arguments, opens the file in write mode, and writes the content to the file. If there is a `FileNotFoundError` or `UnicodeError`, it calls the `tool_error` method with the error message.
2023-07-05 21:11:30 -07:00
Paul Gauthier
de02038a7e remove ref to quoted_file 2023-07-05 21:08:49 -07:00
Paul Gauthier
4126875425 added test_cmd_tokens 2023-07-05 21:08:12 -07:00
Paul Gauthier
7c56dcb16f noop 2023-07-05 21:02:40 -07:00
Paul Gauthier
3897d0a665 removed recheck_abs_fnames 2023-07-05 21:01:26 -07:00
Paul Gauthier
c2c80bbd3c do not /add files that fail to decode 2023-07-05 21:00:10 -07:00
Paul Gauthier
c6e71b443b move import 2023-07-05 20:57:59 -07:00
Paul Gauthier
388f603ef9 aider: Created a file 'foo.bad' with non-utf8 characters before calling the 'cmd_add' method.
# Context:
USER: do the todo
ASSISTANT: To complete the TODO, we need to create a file named 'foo.bad' with a non-utf8 encoding. We can use the Python 'codecs' module to create a file with a specific encoding. In this case, we'll use 'iso-8859-15' which is a common encoding that includes characters not present in utf-8.

Here's how to modify the 'test_cmd_add_bad_encoding' method in the 'tests/test_commands.py' file:

```python
tests/test_commands.py
<<<<<<< ORIGINAL
    def test_cmd_add_bad_encoding(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # TODO: make a new file foo.bad which will fail to decode as utf-8

        commands.cmd_add("foo.bad")

        self.assertEqual(coder.abs_fnames, set())
=======
    import codecs
    def test_cmd_add_bad_encoding(self):
        # Initialize the Commands and InputOutput objects
        io = InputOutput(pretty=False, yes=True)
        from aider.coders import Coder

        coder = Coder.create(models.GPT35, None, io, openai_api_key="deadbeef")
        commands = Commands(io, coder)

        # Create a new file foo.bad which will fail to decode as utf-8
        with codecs.open('foo.bad', 'w', encoding='iso-8859-15') as f:
            f.write('ÆØÅ')  # Characters not present in utf-8

        commands.cmd_add("foo.bad")

        self.assertEqual(coder.abs_fnames, set())
>>>>>>> UPDATED
```

This will create a file 'foo.bad' with non-utf8 characters before calling the 'cmd_add' method.
2023-07-05 20:57:21 -07:00
Paul Gauthier
ca45db2898 use utf-8 2023-07-05 20:51:35 -07:00