feat(vllm): Allow to set quantization (#1094)

Browse source

This particularly useful to set AWQ

**Description**

Follow up of #1015 

**Notes for Reviewers**


**[Signed
commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)**
- [ ] Yes, I signed my commits.
 

<!--
Thank you for contributing to LocalAI! 

Contributing Conventions:

1. Include descriptive PR titles with [<component-name>] prepended.
2. Build and test your changes before submitting a PR. 
3. Sign your commits

By following the community's contribution conventions upfront, the
review process will
be accelerated and your PR merged more quickly.
-->

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

...

This commit is contained in:

Ettore Di Giacinto 2023-09-22 15:52:38 +02:00 • committed by GitHub

parent 048b81373d

commit a28ab18987

No known key found for this signature in database

GPG key ID: 4AEE18F83AFDEB23

13 changed files with 357 additions and 332 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

56

extra/grpc/diffusers/backend_pb2.py

View file

File diff suppressed because one or more lines are too long