aider: Added timeout of 60 seconds to coder.run call using threading module.

# Context:
USER: make sure the call into `coder.run` doesn't take longer than 60 seconds
ASSISTANT: To ensure that the `coder.run` call doesn't take longer than 60 seconds, we can use the `threading` module to create a separate thread for the `coder.run` call and use the `join` method with a timeout of 60 seconds. If the thread is still alive after the timeout, we can raise an exception or handle it accordingly.

Here are the changes needed in the `scripts/benchmark.py` file:

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
from json.decoder import JSONDecodeError
from pathlib import Path
=======
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
import threading
from json.decoder import JSONDecodeError
from pathlib import Path
>>>>>>> UPDATED
```

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        start = time.time()
        coder.run(with_message=instructions)
        dur += time.time() - start
=======
        def run_coder():
            coder.run(with_message=instructions)

        start = time.time()
        coder_thread = threading.Thread(target=run_coder)
        coder_thread.start()
        coder_thread.join(60)  # 60 seconds timeout

        if coder_thread.is_alive():
            # Handle the case when the coder.run call takes longer than 60 seconds
            # You can raise an exception or handle it accordingly
            raise Exception("coder.run took longer than 60 seconds")

        dur += time.time() - start
>>>>>>> UPDATED
```

These changes import the `threading` module, create a separate thread for the `coder.run` call, and set a timeout of 60 seconds. If the thread is still alive after the timeout, an exception is raised. You can modify the exception handling as needed.
This commit is contained in:
Paul Gauthier 2023-06-23 21:14:08 -07:00
parent 3abde2ffb2
commit e18ecf6320

View file

@ -5,6 +5,7 @@ import os
import shutil
import subprocess
import time
import threading
from json.decoder import JSONDecodeError
from pathlib import Path
@ -172,8 +173,19 @@ def run_test(testdir, model_name, edit_format, retries):
dur = 0
test_outcomes = []
for i in range(retries):
start = time.time()
def run_coder():
coder.run(with_message=instructions)
start = time.time()
coder_thread = threading.Thread(target=run_coder)
coder_thread.start()
coder_thread.join(60) # 60 seconds timeout
if coder_thread.is_alive():
# Handle the case when the coder.run call takes longer than 60 seconds
# You can raise an exception or handle it accordingly
raise Exception("coder.run took longer than 60 seconds")
dur += time.time() - start
if coder.num_control_c: