Merge pull request #199 from joshuavial/openrouter

Openrouter compatibility
This commit is contained in:
paul-gauthier 2023-08-31 17:17:28 -07:00 committed by GitHub
commit 30a3cc0847
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
13 changed files with 163 additions and 42 deletions

View file

@ -4,6 +4,7 @@
- [How does aider use git?](#how-does-aider-use-git)
- [GPT-4 vs GPT-3.5](#gpt-4-vs-gpt-35)
- [Aider isn't editing my files?](#aider-isnt-editing-my-files)
- [Accessing other LLMs with OpenRouter](#accessing-other-llms-with-openrouter)
- [Can I use aider with other LLMs, local LLMs, etc?](#can-i-use-aider-with-other-llms-local-llms-etc)
- [Can I change the system prompts that aider uses?](#can-i-change-the-system-prompts-that-aider-uses)
- [Can I run aider in Google Colab?](#can-i-run-aider-in-google-colab)
@ -112,6 +113,19 @@ In these cases, here are some things you might try:
- Use `/drop` to remove files from the chat session which aren't needed for the task at hand. This will reduce distractions and may help GPT produce properly formatted edits.
- Use `/clear` to remove the conversation history, again to help GPT focus.
## Accessing other LLMs with OpenRouter
[OpenRouter](https://openrouter.ai) provide an interface to [many models](https://openrouter.ai/docs) which are not widely accessible, in particular gpt-4-32k and claude-2.
To access the openrouter models simply
- register for an account, purchase some credits and generate an api key
- set --openai-api-base to https://openrouter.ai/api/v1
- set --openai-api-key to your openrouter key
- set --model to the model of your choice (openai/gpt-4-32k, anthropic/claude-2 etc.)
Some of the models weren't very functional and each llm has its own quirks. The anthropic models work ok, but the llama-2 ones in particular will need more work to play friendly with aider.
## Can I use aider with other LLMs, local LLMs, etc?
Aider provides experimental support for LLMs other than OpenAI's GPT-3.5 and GPT-4. The support is currently only experimental for two reasons: