dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-06-01 18:25:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Merge ff3cfa816b into 3caab85931

Browse source
This commit is contained in:
Lin Sun 2025-05-13 16:48:46 -07:00 • committed by GitHub
commit c228695fcc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 22 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

22
aider/website/docs/llms/openai-compat.md
View file

@ -37,3 +37,25 @@ aider --model openai/<model-name>
See the [model warnings](warnings.html)
section for information on warnings which will occur
when working with models that aider is not familiar with.
Example:
Here's a simplified example demonstrating how to request LLM server deployed using SGLang via an OpenAI compatible API endpoint.
Step 1: Launch the SGLang server
In this example, we use the [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) model. For demonstration purposes, the SGLang server is launched locally. After installing SGLang, or within an [SGLang docker container](https://hub.docker.com/r/lmsysorg/sglang/tags), before starting the server, ensure that the model checkpoints are available in your huggingface cache directory, or that you have authenticated with huggingface tokens. Then you can start the server using the following command:
```
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
```
Step 2: modify API endpoint and run aider
```bash
# Mac/Linux:
Export OPENAI_API_BASE=http://localhost:30000/v1
Export OPENAI_API_KEY=None
# Change directory into your codebase
cd /to/your/project
aider --model openai/meta-llama/Llama-3.1-8B-Instruct
```