Enhance autogptq backend to support VL models (#1860)

* Enhance autogptq backend to support VL models

* update dependencies for autogptq

* remove redundant auto-gptq dependency

* Convert base64 to image_url for Qwen-VL model

* implemented model inference for qwen-vl

* remove user prompt from generated answer

* fixed write image error

---------

Co-authored-by: Binghua Wu <bingwu@estee.com>
This commit is contained in:
Sebastian.W 2024-03-27 01:48:14 +08:00 committed by GitHub
parent e58410fa99
commit b7ffe66219
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 75 additions and 18 deletions

View file

@ -24,9 +24,10 @@ dependencies:
- xz=5.4.2=h5eee18b_0
- zlib=1.2.13=h5eee18b_0
- pip:
- accelerate==0.23.0
- accelerate==0.27.0
- aiohttp==3.8.5
- aiosignal==1.3.1
- auto-gptq==0.7.1
- async-timeout==4.0.3
- attrs==23.1.0
- bark==0.1.5
@ -56,6 +57,7 @@ dependencies:
- multiprocess==0.70.15
- networkx
- numpy==1.26.0
- optimum==1.17.1
- packaging==23.2
- pandas
- peft==0.5.0
@ -74,13 +76,13 @@ dependencies:
- six==1.16.0
- sympy==1.12
- tokenizers
- torch==2.1.2
- torch==2.2.1
- torchvision==0.17.1
- torchaudio==2.1.2
- tqdm==4.66.1
- triton==2.1.0
- typing-extensions==4.8.0
- tzdata==2023.3
- auto-gptq==0.6.0
- urllib3==1.26.17
- xxhash==3.4.1
- yarl==1.9.2
@ -103,5 +105,6 @@ dependencies:
- vocos
- vllm==0.3.2
- transformers>=4.38.2 # Updated Version
- transformers_stream_generator==0.0.5
- xformers==0.0.23.post1
prefix: /opt/conda/envs/transformers