ha-llmvision: Image error failure for video files
Bug Description
When I run a video action call to LLM Vision I get an error in the traces (and logs):
Stopped because an error was encountered at 29 October 2024 at 19:35:44 (runtime: 11.03 seconds)
No image input provided
I write my video file to /share/front-camera-clip.mp4 (and I can load that file and play it myself and it works), and it was working fine until recently. It makes no difference if I use /www/llmvision
It works in image mode if I point it at an image in /share
action: llmvision.image_analyzer
data:
provider: (redacted not sure if this is secret or a uid!)
message: tell me what you see
image_file: /share/front-snapshot.jpg
include_filename: false
max_tokens: 86
temperature: 0.4
It fails in video mode if I point it at a video in /share
action: llmvision.video_analyzer
data:
max_frames: 3
include_filename: false
target_width: 1280
detail: low
max_tokens: 100
temperature: 0.2
provider: (redacted not sure if this is secret or a uid!)
message: tell me what you see
video_file: /share/front-camera-clip.mp4
I have also seen the following errors shown underneath - note different folder names tmp_clips and tmp_frames. Is this relevant?
The automation trace is still giving the same error as before but is there an issue with inconsistent folder names?
Version: 1.3 (works in 1.1.x)
Logs
2024-11-04 15:59:15.979 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Processing videos: ['/config/www/llmvision/front-camera-clip.mp4']
2024-11-04 15:59:15.980 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Created /config/custom_components/llmvision/tmp_frames
2024-11-04 15:59:16.293 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Adding frame frame0001.jpg
2024-11-04 15:59:16.307 INFO (MainThread) [custom_components.llmvision.media_handlers] Failed to delete tmp folder: [Errno 2] No such file or directory: '/config/custom_components/llmvision/tmp_clips'
2024-11-04 15:59:16.308 INFO (MainThread) [custom_components.llmvision.media_handlers] Deleted tmp folder: /config/custom_components/llmvision/tmp_frames
2024-11-04 15:59:16.309 INFO (MainThread) [custom_components.llmvision.request_handlers] Provider from call: Google
Additional context
Discussions for this started in my feature request 75 here
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 16 (5 by maintainers)
Commits related to this issue
- Image error failure for video files #80 Potential fix, properly run blocking call in hass.loop — committed to valentinfrlch/ha-llmvision by valentinfrlch 2 years ago
- Image error failure for video files #80 properly await async calls — committed to valentinfrlch/ha-llmvision by valentinfrlch 2 years ago
- Image error failure for video files #80 pass arguments correctly — committed to valentinfrlch/ha-llmvision by valentinfrlch 2 years ago
- Image error failure for video files #80; Added check to ensure at least one frame is always analyzed — committed to valentinfrlch/ha-llmvision by valentinfrlch 2 years ago
I can’t see errors any more even at short durations. Looks fixed to me