Commit graph

2042 commits

Author SHA1 Message Date
Paul Gauthier
0650db2ee7 print the benchmark dirname 2023-06-25 13:59:19 -07:00
Paul Gauthier
78bc901a04 noop 2023-06-25 13:30:38 -07:00
Paul Gauthier
0c3d4198f5 Show variants 2023-06-25 13:18:43 -07:00
Paul Gauthier
5b57024c84 cleanup --stats output 2023-06-25 08:42:06 -07:00
Paul Gauthier
fad464a987 version bump to 0.7.0 2023-06-25 08:34:47 -07:00
Paul Gauthier
c5a232eeaf Added --stats-only 2023-06-25 08:33:24 -07:00
Paul Gauthier
5eaa50fe6c added --num-tests 2023-06-25 08:30:14 -07:00
Paul Gauthier
e0e680e781 prettier test report 2023-06-24 20:54:47 -07:00
Paul Gauthier
436d0dd774 restore the original file, in case we interrupted a prev run 2023-06-24 20:44:37 -07:00
Paul Gauthier
5c3a635eed added --threads for speed 2023-06-24 20:41:35 -07:00
Paul Gauthier
5b4fbd9901 refac 2023-06-24 20:30:58 -07:00
Paul Gauthier
37c282c7f3 remove chdir 2023-06-24 20:28:59 -07:00
Paul Gauthier
959861ee2e refac 2023-06-24 20:22:10 -07:00
Paul Gauthier
6172a70ed3 undo refac 2023-06-24 20:10:26 -07:00
Paul Gauthier
fe1d7aa887 path/to not /path/to 2023-06-24 20:10:17 -07:00
Paul Gauthier
b1cdb9d488 handle /path/to 2023-06-24 20:05:35 -07:00
Paul Gauthier
fe3f1c194d Added APIError 2023-06-24 20:04:49 -07:00
Paul Gauthier
8dcf819cb5 added ServiceUnavailableError for retries 2023-06-24 19:54:18 -07:00
Paul Gauthier
1acc5e04d0 fix 2023-06-24 16:34:10 -07:00
Paul Gauthier
a5746d9e54 removed threading 2023-06-24 15:59:50 -07:00
Paul Gauthier
197d8fe057 Tell gpt to only use std libraries in the benchmark 2023-06-24 15:56:03 -07:00
Paul Gauthier
a1f4ae6a07 Stronger prompt 2023-06-24 15:49:26 -07:00
Paul Gauthier
ac560c2877 Handle missing filenames by scanning for mentions in the text 2023-06-24 15:32:35 -07:00
Paul Gauthier
f2a03e917d Handle missing filename when only 1 in session 2023-06-24 15:13:39 -07:00
Paul Gauthier
d8294cab44 simpler 2023-06-24 14:57:40 -07:00
Paul Gauthier
81b1916c6a added --verbose 2023-06-24 14:53:32 -07:00
Paul Gauthier
1754e574fd cleanup 2023-06-24 14:53:19 -07:00
Paul Gauthier
82e19fcf1d save the file even if tick fence wasn't closed properly 2023-06-24 14:47:29 -07:00
Paul Gauthier
eaf02da46b stronger wholefile prompt about file listings; adopt allowed_to_edit in wholefile; tests 2023-06-24 14:45:43 -07:00
Paul Gauthier
d00e7c39d1 working test 2023-06-24 14:37:25 -07:00
Paul Gauthier
b4f78547bc aider: Added setUp and tearDown methods to TestWholeFileCoder class to store and change the working directory back to the original one.
# Context:
USER: add setup and teardown which getcwd and chdir back
ASSISTANT: I will add `setUp` and `tearDown` methods to the `TestWholeFileCoder` class in the `tests/test_wholefile.py` file. The `setUp` method will store the current working directory, and the `tearDown` method will change the working directory back to the original one.

```python
tests/test_wholefile.py
<<<<<<< ORIGINAL
class TestWholeFileCoder(unittest.TestCase):
    def test_update_files(self):
=======
class TestWholeFileCoder(unittest.TestCase):
    def setUp(self):
        self.original_cwd = os.getcwd()

    def tearDown(self):
        os.chdir(self.original_cwd)

    def test_update_files(self):
>>>>>>> UPDATED
```

Let me know if you need any further changes.
2023-06-24 14:35:07 -07:00
Paul Gauthier
b4a92e312a aider: Added a test function to test the WholeFileCoder's update_files method.
# Context:
USER: make a test for `WholeFileCoder`.updated_files
ASSISTANT: To create a test for the `WholeFileCoder`'s `update_files` method, we will add a new test function in the `tests/test_wholefile.py` file. Here's the test function:

```python
tests/test_wholefile.py
<<<<<<< ORIGINAL

=======
import os
from pathlib import Path
import tempfile
import unittest

from aider.coders.wholefile_coder import WholeFileCoder
from aider.io import InputOutput

class TestWholeFileCoder(unittest.TestCase):
    def test_update_files(self):
        with tempfile.TemporaryDirectory() as temp_dir:
            # Create a sample file in the temporary directory
            sample_file = os.path.join(temp_dir, "sample.txt")
            with open(sample_file, "w") as f:
                f.write("Original content\n")

            # Initialize WholeFileCoder with the temporary directory
            io = InputOutput()
            coder = WholeFileCoder(root=temp_dir, io=io)

            # Set the partial response content with the updated content
            coder.partial_response_content = f"{sample_file}\n```\nUpdated content\n```"

            # Call update_files method
            edited_files = coder.update_files()

            # Check if the sample file was updated
            self.assertIn("sample.txt", edited_files)

            # Check if the content of the sample file was updated
            with open(sample_file, "r") as f:
                updated_content = f.read()
            self.assertEqual(updated_content, "Updated content\n")

if __name__ == "__main__":
    unittest.main()
>>>>>>> UPDATED
```

This test function creates a temporary directory with a sample file, initializes a `WholeFileCoder` instance with the temporary directory, sets the partial response content with the updated content, calls the `update_files` method, and checks if the sample file was updated correctly.
2023-06-24 14:33:20 -07:00
Paul Gauthier
b0d72cf296 Added new files to the git repo: tests/test_wholefile.py 2023-06-24 14:32:10 -07:00
Paul Gauthier
a521d4a3ce Renamed --cmd to --msg 2023-06-24 13:50:45 -07:00
Paul Gauthier
39c1aeac9e Renamed --cmd to --msg 2023-06-24 13:49:27 -07:00
Paul Gauthier
6590faa428 initial 2023-06-24 13:41:50 -07:00
Paul Gauthier
146efe3278 Allow yes to be False 2023-06-24 13:38:54 -07:00
Paul Gauthier
54cd797e64 More clear instructions 2023-06-24 10:29:47 -07:00
Paul Gauthier
64dcfefbec avoid div by zero 2023-06-24 10:29:36 -07:00
Paul Gauthier
467a9cb587 more verbose 2023-06-24 08:40:19 -07:00
Paul Gauthier
86457b8097 Limit retries on bad update replies 2023-06-24 08:38:52 -07:00
Paul Gauthier
a1909611d9 Raise ValueError if gpt makes an unknown function_call 2023-06-24 08:32:12 -07:00
Paul Gauthier
4eee71b956 print just the summary line from pytest 2023-06-24 08:30:33 -07:00
Paul Gauthier
cee0f528ab better 2023-06-24 07:24:41 -07:00
Paul Gauthier
7d2fc1b2d9 fixed bug in show_send_output(), more robust error handling/reporting 2023-06-24 07:24:04 -07:00
Paul Gauthier
29943b8c21 added --no-test 2023-06-24 06:57:14 -07:00
Paul Gauthier
6731fb0710 Increased timeout for coder_thread.join() in benchmark.py. 2023-06-23 21:18:13 -07:00
Paul Gauthier
be1cf5cc5d aider: Added a stop_event object to stop the thread if it's still alive after the timeout.
# Context:
USER: can we stop the thread if it is still alive?
ASSISTANT: Yes, we can stop the thread if it's still alive using the `threading.Event` mechanism. We'll create an event object and pass it to the `run_coder` function. If the thread is still alive after the timeout, we'll set the event, and the `run_coder` function will check for the event being set and exit gracefully.

Here are the changes needed in the `scripts/benchmark.py` file:

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        def run_coder():
            coder.run(with_message=instructions)
=======
        def run_coder(stop_event):
            try:
                coder.run(with_message=instructions)
            except Exception as e:
                if stop_event.is_set():
                    print("Thread stopped due to timeout.")
                else:
                    raise e
>>>>>>> UPDATED
```

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        coder_thread = threading.Thread(target=run_coder)
=======
        stop_event = threading.Event()
        coder_thread = threading.Thread(target=run_coder, args=(stop_event,))
>>>>>>> UPDATED
```

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        if coder_thread.is_alive():
            # Handle the case when the coder.run call takes longer than 60 seconds
            # You can raise an exception or handle it accordingly
            raise Exception("coder.run took longer than 60 seconds")
=======
        if coder_thread.is_alive():
            stop_event.set()
            coder_thread.join()  # Wait for the thread to exit gracefully
            print("coder.run took longer than 60 seconds and was stopped.")
            # Handle the case when the coder.run call takes longer than 60 seconds
            # You can raise an exception or handle it accordingly
            # raise Exception("coder.run took longer than 60 seconds")
>>>>>>> UPDATED
```

These changes create a `stop_event` object, pass it to the `run_coder` function, and set the event if the thread is still alive after the timeout. The `run_coder` function checks for the event being set and exits gracefully.
2023-06-23 21:14:54 -07:00
Paul Gauthier
e18ecf6320 aider: Added timeout of 60 seconds to coder.run call using threading module.
# Context:
USER: make sure the call into `coder.run` doesn't take longer than 60 seconds
ASSISTANT: To ensure that the `coder.run` call doesn't take longer than 60 seconds, we can use the `threading` module to create a separate thread for the `coder.run` call and use the `join` method with a timeout of 60 seconds. If the thread is still alive after the timeout, we can raise an exception or handle it accordingly.

Here are the changes needed in the `scripts/benchmark.py` file:

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
from json.decoder import JSONDecodeError
from pathlib import Path
=======
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
import threading
from json.decoder import JSONDecodeError
from pathlib import Path
>>>>>>> UPDATED
```

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        start = time.time()
        coder.run(with_message=instructions)
        dur += time.time() - start
=======
        def run_coder():
            coder.run(with_message=instructions)

        start = time.time()
        coder_thread = threading.Thread(target=run_coder)
        coder_thread.start()
        coder_thread.join(60)  # 60 seconds timeout

        if coder_thread.is_alive():
            # Handle the case when the coder.run call takes longer than 60 seconds
            # You can raise an exception or handle it accordingly
            raise Exception("coder.run took longer than 60 seconds")

        dur += time.time() - start
>>>>>>> UPDATED
```

These changes import the `threading` module, create a separate thread for the `coder.run` call, and set a timeout of 60 seconds. If the thread is still alive after the timeout, an exception is raised. You can modify the exception handling as needed.
2023-06-23 21:14:08 -07:00
Paul Gauthier
3abde2ffb2 more stats output 2023-06-23 21:12:53 -07:00