aider: Added timeout of 60 seconds to coder.run call using threading module.

# Context:
USER: make sure the call into `coder.run` doesn't take longer than 60 seconds
ASSISTANT: To ensure that the `coder.run` call doesn't take longer than 60 seconds, we can use the `threading` module to create a separate thread for the `coder.run` call and use the `join` method with a timeout of 60 seconds. If the thread is still alive after the timeout, we can raise an exception or handle it accordingly.

Here are the changes needed in the `scripts/benchmark.py` file:

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
from json.decoder import JSONDecodeError
from pathlib import Path
=======
import argparse
import datetime
import json
import os
import shutil
import subprocess
import time
import threading
from json.decoder import JSONDecodeError
from pathlib import Path
>>>>>>> UPDATED
```

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
        start = time.time()
        coder.run(with_message=instructions)
        dur += time.time() - start
=======
        def run_coder():
            coder.run(with_message=instructions)

        start = time.time()
        coder_thread = threading.Thread(target=run_coder)
        coder_thread.start()
        coder_thread.join(60)  # 60 seconds timeout

        if coder_thread.is_alive():
            # Handle the case when the coder.run call takes longer than 60 seconds
            # You can raise an exception or handle it accordingly
            raise Exception("coder.run took longer than 60 seconds")

        dur += time.time() - start
>>>>>>> UPDATED
```

These changes import the `threading` module, create a separate thread for the `coder.run` call, and set a timeout of 60 seconds. If the thread is still alive after the timeout, an exception is raised. You can modify the exception handling as needed.
This commit is contained in:
Paul Gauthier 2023-06-23 21:14:08 -07:00
parent 3abde2ffb2
commit e18ecf6320

View file

@ -5,6 +5,7 @@ import os
import shutil import shutil
import subprocess import subprocess
import time import time
import threading
from json.decoder import JSONDecodeError from json.decoder import JSONDecodeError
from pathlib import Path from pathlib import Path
@ -172,8 +173,19 @@ def run_test(testdir, model_name, edit_format, retries):
dur = 0 dur = 0
test_outcomes = [] test_outcomes = []
for i in range(retries): for i in range(retries):
def run_coder():
coder.run(with_message=instructions)
start = time.time() start = time.time()
coder.run(with_message=instructions) coder_thread = threading.Thread(target=run_coder)
coder_thread.start()
coder_thread.join(60) # 60 seconds timeout
if coder_thread.is_alive():
# Handle the case when the coder.run call takes longer than 60 seconds
# You can raise an exception or handle it accordingly
raise Exception("coder.run took longer than 60 seconds")
dur += time.time() - start dur += time.time() - start
if coder.num_control_c: if coder.num_control_c: