dearwolf/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-05-20 02:24:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

feat(api): allow to pass videos to backends (#3601)

Browse source 
This prepares the API to receive videos as well for video understanding.

It works similarly to images, where the request should be in the form:

{
 "type": "video_url",
 "video_url": { "url": "url or base64 data" }
}

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2024-09-19 11:21:59 +02:00 • committed by GitHub
parent c6a819e92f
commit fbb9facda4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 47 additions and 27 deletions
Download patch file Download diff file Expand all files Collapse all files

2
core/schema/openai.go
View file

@ -58,6 +58,7 @@ type Content struct {
Type string `json:"type" yaml:"type"`
Text string `json:"text" yaml:"text"`
ImageURL ContentURL `json:"image_url" yaml:"image_url"`
VideoURL ContentURL `json:"video_url" yaml:"video_url"`
}
type ContentURL struct {
@ -76,6 +77,7 @@ type Message struct {
StringContent string `json:"string_content,omitempty" yaml:"string_content,omitempty"`
StringImages []string `json:"string_images,omitempty" yaml:"string_images,omitempty"`
StringVideos []string `json:"string_videos,omitempty" yaml:"string_videos,omitempty"`
// A result of a function call
FunctionCall interface{} `json:"function_call,omitempty" yaml:"function_call,omitempty"`