dearwolf/aider
1
0
Fork 0
mirror of https://github.com/Aider-AI/aider.git synced 2025-05-28 08:14:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

add example to OpenAI compatible APIs document

Browse source
This commit is contained in:
linsun12 2025-05-08 22:35:09 +00:00
parent 023e939798
commit ff3cfa816b
1 changed files with 22 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

22
aider/website/docs/llms/openai-compat.md
View file

@ -37,3 +37,25 @@ aider --model openai/<model-name>
See the [model warnings](warnings.html)
section for information on warnings which will occur
when working with models that aider is not familiar with.
Example:
Here's a simplified example demonstrating how to request LLM server deployed using SGLang via an OpenAI compatible API endpoint.
Step 1: Launch the SGLang server
In this example, we use the [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) model. For demonstration purposes, the SGLang server is launched locally. After installing SGLang, or within an [SGLang docker container](https://hub.docker.com/r/lmsysorg/sglang/tags), before starting the server, ensure that the model checkpoints are available in your huggingface cache directory, or that you have authenticated with huggingface tokens. Then you can start the server using the following command:
```
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
```
Step 2: modify API endpoint and run aider
```bash
# Mac/Linux:
Export OPENAI_API_BASE=http://localhost:30000/v1
Export OPENAI_API_KEY=None
# Change directory into your codebase
cd /to/your/project
aider --model openai/meta-llama/Llama-3.1-8B-Instruct
```