dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-31 17:55:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
aider/requirements/common-constraints.txt
Claudia Pellegrino 31c4198cee
fix: let fewer conflicts occur across requirements 
**tl;dr** Introduce a common umbrella constraints file (that works
across requirement extras) to avoid package version conflicts and
to reduce the need for manual pinning in `*.in` files.

Previously, spurious package version conflicts could sometimes occur
across requirements for `pip install -e .`, `pip install -e .[help]`,
`pip install -e .[playwright]`, and so on. Here’s why:

- There are five different requirement configs: the set of base
  requirements (`requirements.txt`) and four additional requirement sets\
  (aka "extras"): `dev`, `help`, `browser`, and `playwright`.

- Each of those five configurations spans its own tree of dependencies
  [1]. Those five trees can slightly overlap. (For example, `greenlet`
  is a transitive requirement for both the `help` and `playwright`
  trees, respectively.)

- If you want to resolve those dependency trees so you get concrete
  version numbers, you can’t just look at each tree independently.
  This is because when trees overlap, they sometimes pull in the same
  package for different reasons, respectively, and maybe with different
  version constraints.
  For example, the `help` tree pulls in `greenlet`, because `sqlalchemy`
  requires it. At the same time, the `playwright` tree also pulls in
  `greenlet` because it’s needed by the `playwright` package.
  Resolving those constraints strictly individually (i.e., per tree) is
  usually a mistake. It may work for a while, but occasionally you’re
  going to end up with two conflicting versions of the same package.

To prevent those version conflicts from occurring, the five
`pip-compile` invocations were designed as a chain.
The process starts at the smallest tree (i.e., the base
`requirements.in` file). It calculates the version numbers for the tree,
remembers the result, and feeds it into the calculation of the next
tree.

The chain design somewhat helped mitigate conflicts, but not always.
The reason for that is that the chain works like a greedy algorithm:
once a decision has been made for a given package in a tree, that
decision is immediately final, and the compilation process isn’t allowed
to go back and change that decision if it learns new information.
New information comes in all the time, because larger trees usually have
more complex constraints than smaller trees, and the process visits
larger trees later, facing additional constraints as it hops from tree
to tree. Sometimes it bumps into a new constraint against a package for
which it has already made a decision earlier (i.e., it has pinned the
concrete version number in the `requirements*.txt` file of an earlier
tree).

That’s why the greedy chain-based method, even though it mostly works
just fine, can never avoid spurious conflicts entirely.
To help mitigate those conflicts, pinning entries were manually added to
`requirements.in` files on a case-by-case basis as conflicts occurred.
Those entries can make the file difficult to reason about, and they must
be kept in sync manually as packages get upgraded. That’s a maintenance
burden.

Turning the chain into an umbrella may help. Instead of hopping from
tree to tree, look at the entire forest at once, calculate all the
concrete version numbers for all trees in one fell swoop, and save the
results in a common, all-encompassing umbrella file.

Armed with the umbrella file (called `common-constraints.txt`), visit
each tree (in any order – it no longer matters) and feed it just the
umbrella file as a constraint, along with its own `*.in` file as the
input.
Chaining is no longer necessary, because the umbrella file already
contains all version constraints for all the packages one tree could
possibly need, and then some.

This technique should reduce manual pinning inside `*.in` files, and
makes sure that computed version numbers no longer contradict each other
across trees.

[1]: From a graph theory point of view, I’m being blatantly incorrect
here; those dependency graphs are usually not trees, because they have
cycles. I’m still going to call them "trees" for the sake of this
discussion, because the word "tree" feels less abstract and intimidating
and hopefully more relatable.
2025-03-02 02:50:03 +01:00

557 lines
11 KiB
Text
Raw Blame History

#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --allow-unsafe --output-file=requirements/common-constraints.txt requirements/requirements-browser.in requirements/requirements-dev.in requirements/requirements-help.in requirements/requirements-playwright.in requirements/requirements.in
#
aiohappyeyeballs==2.4.6
# via aiohttp
aiohttp==3.11.13
# via
# huggingface-hub
# litellm
# llama-index-core
aiosignal==1.3.2
# via aiohttp
alabaster==1.0.0
# via sphinx
altair==5.5.0
# via streamlit
annotated-types==0.7.0
# via pydantic
anyio==4.8.0
# via
# httpx
# openai
# watchfiles
attrs==25.1.0
# via
# aiohttp
# jsonschema
# referencing
babel==2.17.0
# via sphinx
backoff==2.2.1
# via
# -r requirements/requirements.in
# posthog
beautifulsoup4==4.13.3
# via -r requirements/requirements.in
blinker==1.9.0
# via streamlit
build==1.2.2.post1
# via pip-tools
cachetools==5.5.2
# via streamlit
certifi==2025.1.31
# via
# httpcore
# httpx
# requests
cffi==1.17.1
# via
# sounddevice
# soundfile
cfgv==3.4.0
# via pre-commit
charset-normalizer==3.4.1
# via requests
click==8.1.8
# via
# litellm
# nltk
# pip-tools
# streamlit
# typer
codespell==2.4.1
# via -r requirements/requirements-dev.in
cogapp==3.4.1
# via -r requirements/requirements-dev.in
configargparse==1.7
# via -r requirements/requirements.in
contourpy==1.3.1
# via matplotlib
cycler==0.12.1
# via matplotlib
dataclasses-json==0.6.7
# via llama-index-core
deprecated==1.2.18
# via llama-index-core
diff-match-patch==20241021
# via -r requirements/requirements.in
dill==0.3.9
# via
# multiprocess
# pathos
dirtyjson==1.0.8
# via llama-index-core
diskcache==5.6.3
# via -r requirements/requirements.in
distlib==0.3.9
# via virtualenv
distro==1.9.0
# via
# openai
# posthog
docutils==0.21.2
# via
# sphinx
# sphinx-rtd-theme
filelock==3.17.0
# via
# huggingface-hub
# torch
# transformers
# virtualenv
filetype==1.2.0
# via llama-index-core
flake8==7.1.2
# via -r requirements/requirements.in
fonttools==4.56.0
# via matplotlib
frozenlist==1.5.0
# via
# aiohttp
# aiosignal
fsspec==2025.2.0
# via
# huggingface-hub
# llama-index-core
# torch
gitdb==4.0.12
# via gitpython
gitpython==3.1.44
# via
# -r requirements/requirements.in
# streamlit
greenlet==3.0.3
# via
# playwright
# sqlalchemy
grep-ast==0.6.1
# via -r requirements/requirements.in
h11==0.14.0
# via httpcore
httpcore==1.0.7
# via httpx
httpx==0.28.1
# via
# litellm
# llama-index-core
# openai
huggingface-hub[inference]==0.29.1
# via
# llama-index-embeddings-huggingface
# sentence-transformers
# tokenizers
# transformers
identify==2.6.8
# via pre-commit
idna==3.10
# via
# anyio
# httpx
# requests
# yarl
imagesize==1.4.1
# via sphinx
imgcat==0.6.0
# via -r requirements/requirements-dev.in
importlib-metadata==7.2.1
# via
# -r requirements/requirements.in
# litellm
importlib-resources==6.5.2
# via -r requirements/requirements.in
iniconfig==2.0.0
# via pytest
jinja2==3.1.5
# via
# altair
# litellm
# pydeck
# sphinx
# torch
jiter==0.8.2
# via openai
joblib==1.4.2
# via
# nltk
# scikit-learn
json5==0.10.0
# via -r requirements/requirements.in
jsonschema==4.23.0
# via
# -r requirements/requirements.in
# altair
# litellm
jsonschema-specifications==2024.10.1
# via jsonschema
kiwisolver==1.4.8
# via matplotlib
litellm==1.61.16
# via -r requirements/requirements.in
llama-index-core==0.12.20
# via
# -r requirements/requirements-help.in
# llama-index-embeddings-huggingface
llama-index-embeddings-huggingface==0.5.2
# via -r requirements/requirements-help.in
lox==0.12.0
# via -r requirements/requirements-dev.in
markdown-it-py==3.0.0
# via rich
markupsafe==3.0.2
# via jinja2
marshmallow==3.26.1
# via dataclasses-json
matplotlib==3.10.0
# via -r requirements/requirements-dev.in
mccabe==0.7.0
# via flake8
mdurl==0.1.2
# via markdown-it-py
mixpanel==4.10.1
# via -r requirements/requirements.in
monotonic==1.6
# via posthog
mpmath==1.3.0
# via sympy
multidict==6.1.0
# via
# aiohttp
# yarl
multiprocess==0.70.17
# via pathos
mypy-extensions==1.0.0
# via typing-inspect
narwhals==1.28.0
# via altair
nest-asyncio==1.6.0
# via llama-index-core
networkx==3.2.1
# via
# -r requirements/requirements.in
# llama-index-core
# torch
nltk==3.9.1
# via llama-index-core
nodeenv==1.9.1
# via pre-commit
numpy==1.26.4
# via
# -r requirements/requirements.in
# contourpy
# llama-index-core
# matplotlib
# pandas
# pydeck
# scikit-learn
# scipy
# soundfile
# streamlit
# transformers
openai==1.64.0
# via litellm
packaging==24.2
# via
# -r requirements/requirements.in
# altair
# build
# huggingface-hub
# marshmallow
# matplotlib
# pytest
# sphinx
# streamlit
# transformers
pandas==2.2.3
# via
# -r requirements/requirements-dev.in
# streamlit
pathos==0.3.3
# via lox
pathspec==0.12.1
# via
# -r requirements/requirements.in
# grep-ast
pexpect==4.9.0
# via -r requirements/requirements.in
pillow==10.4.0
# via
# -r requirements/requirements.in
# llama-index-core
# matplotlib
# sentence-transformers
# streamlit
pip-tools==7.4.1
# via -r requirements/requirements-dev.in
platformdirs==4.3.6
# via virtualenv
playwright==1.47.0
# via -r requirements/requirements-playwright.in
pluggy==1.5.0
# via pytest
posthog==3.16.0
# via -r requirements/requirements.in
pox==0.3.5
# via pathos
ppft==1.7.6.9
# via pathos
pre-commit==4.1.0
# via -r requirements/requirements-dev.in
prompt-toolkit==3.0.50
# via -r requirements/requirements.in
propcache==0.3.0
# via
# aiohttp
# yarl
protobuf==5.29.3
# via streamlit
psutil==7.0.0
# via -r requirements/requirements.in
ptyprocess==0.7.0
# via pexpect
pyarrow==19.0.1
# via streamlit
pycodestyle==2.12.1
# via flake8
pycparser==2.22
# via cffi
pydantic==2.10.6
# via
# litellm
# llama-index-core
# openai
pydantic-core==2.27.2
# via pydantic
pydeck==0.9.1
# via streamlit
pydub==0.25.1
# via -r requirements/requirements.in
pyee==12.0.0
# via playwright
pyflakes==3.2.0
# via flake8
pygments==2.19.1
# via
# rich
# sphinx
pypandoc==1.15
# via -r requirements/requirements.in
pyparsing==3.2.1
# via matplotlib
pyperclip==1.9.0
# via -r requirements/requirements.in
pyproject-hooks==1.2.0
# via
# build
# pip-tools
pytest==8.3.4
# via
# -r requirements/requirements-dev.in
# pytest-env
pytest-env==1.1.5
# via -r requirements/requirements-dev.in
python-dateutil==2.9.0.post0
# via
# matplotlib
# pandas
# posthog
python-dotenv==1.0.1
# via litellm
pytz==2025.1
# via pandas
pyyaml==6.0.2
# via
# -r requirements/requirements.in
# huggingface-hub
# llama-index-core
# pre-commit
# transformers
referencing==0.36.2
# via
# jsonschema
# jsonschema-specifications
regex==2024.11.6
# via
# nltk
# tiktoken
# transformers
requests==2.32.3
# via
# huggingface-hub
# llama-index-core
# mixpanel
# posthog
# sphinx
# streamlit
# tiktoken
# transformers
rich==13.9.4
# via
# -r requirements/requirements.in
# streamlit
# typer
roman-numerals-py==3.1.0
# via sphinx
rpds-py==0.23.1
# via
# jsonschema
# referencing
safetensors==0.5.3
# via transformers
scikit-learn==1.6.1
# via sentence-transformers
scipy==1.13.1
# via
# -r requirements/requirements.in
# scikit-learn
# sentence-transformers
semver==3.0.4
# via -r requirements/requirements-dev.in
sentence-transformers==3.4.1
# via llama-index-embeddings-huggingface
shellingham==1.5.4
# via typer
six==1.17.0
# via
# mixpanel
# posthog
# python-dateutil
smmap==5.0.2
# via gitdb
sniffio==1.3.1
# via
# anyio
# openai
snowballstemmer==2.2.0
# via sphinx
socksio==1.0.0
# via -r requirements/requirements.in
sounddevice==0.5.1
# via -r requirements/requirements.in
soundfile==0.13.1
# via -r requirements/requirements.in
soupsieve==2.6
# via beautifulsoup4
sphinx==8.2.1
# via
# sphinx-rtd-theme
# sphinxcontrib-jquery
sphinx-rtd-theme==3.0.2
# via lox
sphinxcontrib-applehelp==2.0.0
# via sphinx
sphinxcontrib-devhelp==2.0.0
# via sphinx
sphinxcontrib-htmlhelp==2.1.0
# via sphinx
sphinxcontrib-jquery==4.1
# via sphinx-rtd-theme
sphinxcontrib-jsmath==1.0.1
# via sphinx
sphinxcontrib-qthelp==2.0.0
# via sphinx
sphinxcontrib-serializinghtml==2.0.0
# via sphinx
sqlalchemy[asyncio]==2.0.38
# via
# llama-index-core
# sqlalchemy
streamlit==1.42.2
# via -r requirements/requirements-browser.in
sympy==1.13.3
# via torch
tenacity==9.0.0
# via
# llama-index-core
# streamlit
threadpoolctl==3.5.0
# via scikit-learn
tiktoken==0.9.0
# via
# litellm
# llama-index-core
tokenizers==0.19.1
# via
# litellm
# transformers
toml==0.10.2
# via streamlit
torch==2.2.2
# via sentence-transformers
tornado==6.4.2
# via streamlit
tqdm==4.67.1
# via
# huggingface-hub
# llama-index-core
# nltk
# openai
# sentence-transformers
# transformers
transformers==4.44.2
# via sentence-transformers
tree-sitter==0.21.3
# via
# -r requirements/requirements.in
# grep-ast
# tree-sitter-languages
tree-sitter-languages==1.10.2
# via grep-ast
typer==0.15.1
# via -r requirements/requirements-dev.in
typing-extensions==4.12.2
# via
# altair
# anyio
# beautifulsoup4
# huggingface-hub
# llama-index-core
# openai
# pydantic
# pydantic-core
# pyee
# referencing
# sqlalchemy
# streamlit
# torch
# typer
# typing-inspect
typing-inspect==0.9.0
# via
# dataclasses-json
# llama-index-core
tzdata==2025.1
# via pandas
urllib3==2.3.0
# via
# mixpanel
# requests
virtualenv==20.29.2
# via pre-commit
watchdog==4.0.2
# via streamlit
watchfiles==1.0.4
# via -r requirements/requirements.in
wcwidth==0.2.13
# via prompt-toolkit
wheel==0.45.1
# via pip-tools
wrapt==1.17.2
# via
# deprecated
# llama-index-core
yarl==1.18.3
# via aiohttp
zipp==3.21.0
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
pip==25.0.1
# via
# -r requirements/requirements.in
# pip-tools
setuptools==75.8.1
# via pip-tools
Reference in a new issue View git blame Copy permalink