mirror of
https://github.com/Aider-AI/aider.git
synced 2025-05-20 12:24:59 +00:00
31c4198cee
**tl;dr** Introduce a common umbrella constraints file (that works across requirement extras) to avoid package version conflicts and to reduce the need for manual pinning in `*.in` files. Previously, spurious package version conflicts could sometimes occur across requirements for `pip install -e .`, `pip install -e .[help]`, `pip install -e .[playwright]`, and so on. Here’s why: - There are five different requirement configs: the set of base requirements (`requirements.txt`) and four additional requirement sets\ (aka "extras"): `dev`, `help`, `browser`, and `playwright`. - Each of those five configurations spans its own tree of dependencies [1]. Those five trees can slightly overlap. (For example, `greenlet` is a transitive requirement for both the `help` and `playwright` trees, respectively.) - If you want to resolve those dependency trees so you get concrete version numbers, you can’t just look at each tree independently. This is because when trees overlap, they sometimes pull in the same package for different reasons, respectively, and maybe with different version constraints. For example, the `help` tree pulls in `greenlet`, because `sqlalchemy` requires it. At the same time, the `playwright` tree also pulls in `greenlet` because it’s needed by the `playwright` package. Resolving those constraints strictly individually (i.e., per tree) is usually a mistake. It may work for a while, but occasionally you’re going to end up with two conflicting versions of the same package. To prevent those version conflicts from occurring, the five `pip-compile` invocations were designed as a chain. The process starts at the smallest tree (i.e., the base `requirements.in` file). It calculates the version numbers for the tree, remembers the result, and feeds it into the calculation of the next tree. The chain design somewhat helped mitigate conflicts, but not always. The reason for that is that the chain works like a greedy algorithm: once a decision has been made for a given package in a tree, that decision is immediately final, and the compilation process isn’t allowed to go back and change that decision if it learns new information. New information comes in all the time, because larger trees usually have more complex constraints than smaller trees, and the process visits larger trees later, facing additional constraints as it hops from tree to tree. Sometimes it bumps into a new constraint against a package for which it has already made a decision earlier (i.e., it has pinned the concrete version number in the `requirements*.txt` file of an earlier tree). That’s why the greedy chain-based method, even though it mostly works just fine, can never avoid spurious conflicts entirely. To help mitigate those conflicts, pinning entries were manually added to `requirements.in` files on a case-by-case basis as conflicts occurred. Those entries can make the file difficult to reason about, and they must be kept in sync manually as packages get upgraded. That’s a maintenance burden. Turning the chain into an umbrella may help. Instead of hopping from tree to tree, look at the entire forest at once, calculate all the concrete version numbers for all trees in one fell swoop, and save the results in a common, all-encompassing umbrella file. Armed with the umbrella file (called `common-constraints.txt`), visit each tree (in any order – it no longer matters) and feed it just the umbrella file as a constraint, along with its own `*.in` file as the input. Chaining is no longer necessary, because the umbrella file already contains all version constraints for all the packages one tree could possibly need, and then some. This technique should reduce manual pinning inside `*.in` files, and makes sure that computed version numbers no longer contradict each other across trees. [1]: From a graph theory point of view, I’m being blatantly incorrect here; those dependency graphs are usually not trees, because they have cycles. I’m still going to call them "trees" for the sake of this discussion, because the word "tree" feels less abstract and intimidating and hopefully more relatable. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| blame.py | ||
| Dockerfile.jekyll | ||
| history_prompts.py | ||
| issues.py | ||
| jekyll_build.sh | ||
| jekyll_run.sh | ||
| my_models.py | ||
| pip-compile.sh | ||
| update-blame.sh | ||
| update-docs.sh | ||
| update-history.py | ||
| versionbump.py | ||
| yank-old-versions.py | ||