Commit graph

239 commits

Author SHA1 Message Date
Paul Gauthier
b6760e26c7 fix: Ensure token count calculation handles small text samples 2024-08-05 17:14:42 -03:00
Paul Gauthier
75c3c40354 Estimate tokenization to speed up repo map 2024-08-05 17:09:27 -03:00
Paul Gauthier (aider)
de26f8e5e0 style: format code with linter 2024-08-05 16:54:46 -03:00
Paul Gauthier (aider)
276b30506c fix: Add elapsed time logging to print statements 2024-08-05 16:54:42 -03:00
Paul Gauthier
042a4f14d2 fix: Update repomap.py to improve caching and logging 2024-08-05 16:52:22 -03:00
Paul Gauthier
3ba6e95d22 fix: Ensure token count is calculated only when verbose mode is enabled 2024-08-05 16:37:34 -03:00
Paul Gauthier
b654a16b48 Accept repo maps within 10% of token budget 2024-08-05 16:26:53 -03:00
Paul Gauthier (aider)
65247353bd fix: Remove unused linter variable in get_supported_languages_md function 2024-08-05 16:17:29 -03:00
Paul Gauthier (aider)
b86dde5f0d style: Apply linter edits to repomap.py 2024-08-05 16:17:07 -03:00
Paul Gauthier (aider)
ba1c8d1701 feat: Estimate initial middle value for binary search in get_ranked_tags_map 2024-08-05 16:17:03 -03:00
Paul Gauthier
b19795228f fix: Adjust the starting number for the binary search in the RepoMap class 2024-08-05 16:11:08 -03:00
Paul Gauthier
e2fb2466a9 Improve linter support and repo map coverage for various programming languages. 2024-07-30 12:30:29 -03:00
Paul Gauthier (aider)
74201dd733 Add two new columns to the get_supported_languages_md function: Repo map and Linter, and check the existence of the SCM file and linter support for each language. 2024-07-30 12:26:01 -03:00
Paul Gauthier
859a13eb70 Updated languages doc to correctly list repo map supported langs 2024-07-24 11:28:37 +02:00
Paul Gauthier
397c5d5921 added a test for /tokens #863 2024-07-18 16:54:26 +01:00
Paul Gauthier
4fba17c18b switched langs table from html -> md 2024-07-06 18:56:33 -03:00
Paul Gauthier
e4c637724f For /help, turn down the repo map 2024-07-06 17:11:24 -03:00
Paul Gauthier
d403f37469 cleanup 2024-07-03 17:06:10 -03:00
Paul Gauthier
b5cd5f0e23 Use a thread to import slow modules in the background 2024-07-03 13:55:29 -03:00
Paul Gauthier
5e0ff7627e Defer loading of networkx 2024-07-03 13:17:04 -03:00
Paul Gauthier
c020d94d5f 8x repomap when finding files; fix max_input_tokens 2024-06-26 04:14:51 +00:00
Paul Gauthier
965c35870c Notice if the user mentions the basename of a filename; fixes to repomap context 2024-06-24 15:12:58 -07:00
Paul Gauthier
0d9150c77b coq for languages.md 2024-06-05 20:24:19 -07:00
Paul Gauthier
a4a6066858 handle missing max_map_tokens/max_context_window 2024-05-23 13:46:08 -07:00
Paul Gauthier
6345cd3734 cleanup 2024-05-23 13:43:35 -07:00
Paul Gauthier
56f5e7e10c expand repomap even in smaller context windows 2024-05-21 11:04:18 -07:00
Paul Gauthier
1effd73e80 Only expand the repomap if we know it will fit into context window 2024-05-20 10:22:57 -07:00
Paul Gauthier
819fccc7a4 added temp param, prompt strong that files message is truth 2024-05-16 08:52:30 -07:00
Paul Gauthier
ecc46bd3e3 16x 2024-05-14 14:04:45 -07:00
Paul Gauthier
dea93bbfd9 mentioned_idents 2024-05-14 12:18:48 -07:00
Paul Gauthier
e5616c0247 mentioned_fnames 2024-05-14 11:52:32 -07:00
Paul Gauthier
73e6949287 refac 2024-05-14 11:12:36 -07:00
Paul Gauthier
851e98a398 added tree cache 2024-05-14 10:32:16 -07:00
Paul Gauthier
3251380364 refac 2024-05-14 10:23:53 -07:00
Paul Gauthier
370f668955 remove useless noqa 2024-05-14 06:02:40 -07:00
Paul Gauthier
e96420f999 fix to ignore long (!) lines in repomap 2024-05-14 06:01:24 -07:00
Paul Gauthier
292250db4f prompt and token adjustments when no files in chat with repo 2024-05-13 12:30:42 -07:00
Paul Gauthier
1d55f749fb cleanup 2024-05-08 13:56:40 -07:00
Paul Gauthier
9e7f2ec790 cleanup 2024-05-08 13:41:24 -07:00
Paul Gauthier
f256b43043 wip 2024-05-08 13:35:34 -07:00
Paul Gauthier
f0f0c330d7 cleanup 2024-04-19 12:14:58 -07:00
Paul Gauthier
547ae142ba refactor tokenizer 2024-04-19 12:08:35 -07:00
Paul Gauthier
3449f7da42 quiet tree-sitter warnings 2024-04-18 16:36:56 -07:00
Paul Gauthier
65400981c5 aider: Replace deprecated pkg_resources.resource_filename with importlib.resources.files for accessing package resources in Python 3.9+.
# Aider chat conversation:

USER: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

ASSISTANT: To address the `DeprecationWarning` regarding `pkg_resources`, we will replace its usage with the recommended `importlib.resources` or `importlib.metadata` from the Python standard library. The specific replacement depends on the original use case of `pkg_resources`. In the provided code, `pkg_resources` is used to access package data, so we will use `importlib.resources`.

Here are the diffs for the necessary changes:

```diff
--- aider/repomap.py
+++ aider/repomap.py
@@ ... @@
 import networkx as nx
-import pkg_resources
+import importlib.resources as pkg_resources
 from diskcache import Cache
 from grep_ast import TreeContext, filename_to_lang
 from pygments.lexers import guess_lexer_for_filename
 from pygments.token import Token
 from pygments.util import ClassNotFound
@@ ... @@
         # Load the tags queries
         try:
-            scm_fname = pkg_resources.resource_filename(
-                __name__, os.path.join("queries", f"tree-sitter-{lang}-tags.scm")
-            )
+            scm_fname = pkg_resources.files(__package__).joinpath(
+                "queries", f"tree-sitter-{lang}-tags.scm"
+            )
         except KeyError:
             return
-        query_scm = Path(scm_fname)
+        query_scm = scm_fname
         if not query_scm.exists():
             return
         query_scm = query_scm.read_text()
```

These changes replace the deprecated `pkg_resources.resource_filename` with `importlib.resources.files`, which is the recommended way to access package resources in Python 3.9 and later. The `files()` function returns a `Path`-like object that can be used to read the contents of the resource.
2024-04-18 16:30:10 -07:00
Paul Gauthier
c9bb22d6d5 roughed in tokenizer, dropped openai, openrouter 2024-04-17 15:22:35 -07:00
Paul Gauthier
8f98f3b430 Handle languages without tags.scm files #514 2024-03-22 10:14:22 -07:00
Paul Gauthier
f37488447f handle RecursionError 2024-01-02 10:12:44 -08:00
Paul Gauthier
d54f883698 do not include added files in the repo map #315 2023-11-15 13:06:22 -08:00
Paul Gauthier
f0c575e413 use io.read_text to avoid unicode errors in repomap #305 2023-11-02 09:30:59 -07:00
Paul Gauthier
5edf20915f updated sample map 2023-10-22 15:06:35 -07:00