Commit graph

2344 commits

Author SHA1 Message Date
Paul Gauthier
8999c3fd97 move the progress bar to the bottom 2023-06-27 14:58:15 -07:00
Paul Gauthier
9556313819 Added functionality to use pre-existing benchmark results if they exist. 2023-06-27 14:27:07 -07:00
Paul Gauthier
70050ca004 improved logic for creating benchmark dirs 2023-06-27 08:02:37 -07:00
Paul Gauthier
29d60b9ddb check for dirname on --clean 2023-06-27 07:52:48 -07:00
Paul Gauthier
7736289309 test error prompt 2023-06-27 07:49:47 -07:00
Paul Gauthier
39434289f5 updated docker image name 2023-06-26 20:52:02 -07:00
Paul Gauthier
a109ab1890 aider: Added check for existing Docker image before building.
# Context:
USER: in `build_docker` check if the image is already built and return
ASSISTANT: To check if the Docker image is already built and return if it exists, we can use the `docker images` command and parse its output. Here's how to modify the `build_docker` function:

```python
benchmark/benchmark.py
<<<<<<< ORIGINAL
def build_docker():
    command = [
        "docker",
        "build",
        "--quiet",
        "-t",
        "benchmark",
        "-f",
        "benchmark/Dockerfile",
        "/dev/null",
    ]
    print(" ".join(command))

    try:
        result = subprocess.run(
            command,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
        )
        res = result.stdout
        print(res)

    except subprocess.CalledProcessError as e:
        res = f"Failed to build Docker image: {e.output}"

    return res
=======
def build_docker():
    image_name = "benchmark"
    check_command = ["docker", "images", "-q", image_name]
    check_result = subprocess.run(check_command, stdout=subprocess.PIPE, text=True)
    if check_result.stdout.strip():
        print(f"Docker image '{image_name}' already exists, skipping build.")
        return

    command = [
        "docker",
        "build",
        "--quiet",
        "-t",
        image_name,
        "-f",
        "benchmark/Dockerfile",
        "/dev/null",
    ]
    print(" ".join(command))

    try:
        result = subprocess.run(
            command,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
        )
        res = result.stdout
        print(res)

    except subprocess.CalledProcessError as e:
        res = f"Failed to build Docker image: {e.output}"

    return res
>>>>>>> UPDATED
```

This modification adds a check for the existence of the Docker image before attempting to build it. If the image already exists, the function prints a message and returns early.
2023-06-26 20:48:55 -07:00
Paul Gauthier
5e2681ec9d call build_docker 2023-06-26 20:47:49 -07:00
Paul Gauthier
898d98fdb3 aider: Added a build_docker() function that builds a docker image called benchmark from the benchmark/Dockerfile with no other fs context.
# Context:
USER: add a build_docker() func that builds a docker image called `benchmark` from the benchmark/Dockerfile with no other fs context
2023-06-26 20:41:38 -07:00
Paul Gauthier
095f47593d run unit tests inside docker for safety 2023-06-26 20:37:55 -07:00
Paul Gauthier
d13ff918de aider: Refactor unit tests to run using docker python3.8 and mount only the file being run.
# Context:
USER: run the unit tests using docker python3.8 mounting just the file being run
2023-06-26 20:18:03 -07:00
Paul Gauthier
4d3c77332f ssh typer 2023-06-26 20:15:42 -07:00
Paul Gauthier
dd30cb34ed refac 2023-06-26 20:11:41 -07:00
Paul Gauthier
5c8546c140 sssh typer 2023-06-26 16:48:15 -07:00
Paul Gauthier
acc6be9e95 switched back to ``` as default fence 2023-06-26 16:42:29 -07:00
Paul Gauthier
307f3a4da4 Just HH-MM 2023-06-26 16:36:50 -07:00
Paul Gauthier
23f96a28a0 use <source> fence by default 2023-06-26 16:34:46 -07:00
Paul Gauthier
9e91a392c4 aider: Switched the script to use typer and returned a file listing.
# Context:
USER: switch this script to typer
USER: return a file listing
2023-06-26 16:33:55 -07:00
Paul Gauthier
3c557a00af refac 2023-06-26 16:28:33 -07:00
Paul Gauthier
38f7b36bb1 improved error/ask reporting 2023-06-26 11:11:12 -07:00
Paul Gauthier
651b1293c3 tweaks 2023-06-26 10:51:06 -07:00
Paul Gauthier
2c7960ae26 prefix date/time on benchmark dirs 2023-06-26 10:46:30 -07:00
Paul Gauthier
1370da14fb track and report errors/asks during benchmarking 2023-06-26 10:33:16 -07:00
Paul Gauthier
cbfda391bb version bump to 0.7.2 2023-06-26 10:24:14 -07:00
Paul Gauthier
079682b29e Omit the "No changes are needed" redaction from commit messages with wholefile coders 2023-06-26 10:21:27 -07:00
Paul Gauthier
33cc72a0df ssh 2023-06-25 21:40:08 -07:00
Paul Gauthier
819b8b2672 tolerate missing final newline 2023-06-25 21:27:18 -07:00
Paul Gauthier
dc535a2aa0 aider: Updated summarize_results function to display remaining time in MM:SS format.
# Context:
USER: show MM:SS remaining
ASSISTANT: To show the remaining time in MM:SS format, you need to modify the `summarize_results` function in the `scripts/benchmark.py` file. Update the `remaining` variable calculation and its print statement.

```python
scripts/benchmark.py
<<<<<<< ORIGINAL
    avg_duration = duration / completed_tests
    remaining = (total_tests - completed_tests) * avg_duration

    console.print(f"duration: {avg_duration:.1f} sec/test-case, {remaining:.0f} sec remaining")
=======
    avg_duration = duration / completed_tests
    remaining_seconds = (total_tests - completed_tests) * avg_duration
    remaining_minutes, remaining_seconds = divmod(remaining_seconds, 60)

    console.print(f"duration: {avg_duration:.1f} sec/test-case, {remaining_minutes:.0f}:{remaining_seconds:02.0f} remaining")
>>>>>>> UPDATED
```

This change will display the remaining time in MM:SS format.
2023-06-25 21:22:55 -07:00
Paul Gauthier
f836a4b760 show remaining 2023-06-25 20:59:03 -07:00
Paul Gauthier
58690da8b4 fences are obfuscated so aider can modify itself 2023-06-25 20:55:10 -07:00
Paul Gauthier
d4d327e6d9 added test for non-backtick fences 2023-06-25 20:47:17 -07:00
Paul Gauthier
0520981135 Attempt to quote diffs safely 2023-06-25 20:42:47 -07:00
Paul Gauthier
e5ad9d501f aider: Added extra backticks to the start and end of the show variable to ensure proper formatting.
# Context:
USER: do the todo
ASSISTANT: To complete the TODO task in the `diff_partial_update` function, I will find the longest string of backticks (`) that start a line in the `diff` variable. I will then add one more backtick to the start and end of the `show` variable to ensure proper formatting.

Here are the changes:

```python
aider/diffs.py
<<<<<<< ORIGINAL
    for line in diff:
        pass

    diff = "".join(diff)
    if not diff.endswith("\n"):
        diff += "\n"

    show = "```diff\n"
=======
    max_backticks = max(len(line) - len(line.lstrip("`")) for line in diff.splitlines())
    extra_backticks = "`" * (max_backticks + 1)

    diff = "".join(diff)
    if not diff.endswith("\n"):
        diff += "\n"

    show = f"{extra_backticks}diff\n"
>>>>>>> UPDATED
```

```python
aider/diffs.py
<<<<<<< ORIGINAL
    show += diff + "```\n\n"
=======
    show += diff + f"{extra_backticks}\n\n"
>>>>>>> UPDATED
```

These changes will find the longest string of backticks at the start of a line in the `diff` variable and add one more backtick to the start and end of the `show` variable.
2023-06-25 20:38:46 -07:00
Paul Gauthier
ef88cc6d55 refactor choose_fence 2023-06-25 20:34:32 -07:00
Paul Gauthier
96fda2aa9f abandon n-backticks, try some html fences 2023-06-25 20:23:36 -07:00
Paul Gauthier
9151cf03a1 quote with num_ticks 2023-06-25 19:58:17 -07:00
Paul Gauthier
a6d80e815e wip 2023-06-25 19:54:15 -07:00
Paul Gauthier
f00a1e827a Merge branch 'main' into triple-backticks 2023-06-25 19:45:03 -07:00
Paul Gauthier
52343b5340 More generic handling of GPT mangling of filenames 2023-06-25 19:40:12 -07:00
Paul Gauthier
2b8717bdb6 Merge branch 'main' into triple-backticks 2023-06-25 19:34:20 -07:00
Paul Gauthier
de2808f742 Updated benchmark instructions 2023-06-25 18:01:48 -07:00
Paul Gauthier
e8df4a55b9 version bump to 0.7.1 2023-06-25 17:55:18 -07:00
Paul Gauthier
e535f163a9 Updated benchmarking instructions 2023-06-25 17:54:41 -07:00
Paul Gauthier
3393f560fb make sure diffs display with newlines 2023-06-25 17:33:33 -07:00
Paul Gauthier
f625d52bec added self.fence_ticks 2023-06-25 17:03:20 -07:00
Paul Gauthier
59d99835b1 use a word for 3,4,5 backticks 2023-06-25 17:00:59 -07:00
Paul Gauthier
6b2024a58a ssh 2023-06-25 15:55:27 -07:00
Paul Gauthier
060cbeafff Added back "entire content" prompt" 2023-06-25 15:54:56 -07:00
Paul Gauthier
68640c9bb2 Record commit_hash 2023-06-25 15:46:15 -07:00
Paul Gauthier
f005404ec0 Roughed in updated sys msg for whole format 2023-06-25 15:27:06 -07:00