ha-llmvision: Image error failure for video files

Bug Description

When I run a video action call to LLM Vision I get an error in the traces (and logs):


Stopped because an error was encountered at 29 October 2024 at 19:35:44 (runtime: 11.03 seconds)

No image input provided

I write my video file to /share/front-camera-clip.mp4 (and I can load that file and play it myself and it works), and it was working fine until recently. It makes no difference if I use /www/llmvision

It works in image mode if I point it at an image in /share


action: llmvision.image_analyzer
data:
  provider: (redacted not sure if this is secret or a uid!)
  message: tell me what you see
  image_file: /share/front-snapshot.jpg
  include_filename: false
  max_tokens: 86
  temperature: 0.4

It fails in video mode if I point it at a video in /share


action: llmvision.video_analyzer
data:
  max_frames: 3
  include_filename: false
  target_width: 1280
  detail: low
  max_tokens: 100
  temperature: 0.2
  provider: (redacted not sure if this is secret or a uid!)
  message: tell me what you see
  video_file: /share/front-camera-clip.mp4

I have also seen the following errors shown underneath - note different folder names tmp_clips and tmp_frames. Is this relevant?

The automation trace is still giving the same error as before but is there an issue with inconsistent folder names?

Version: 1.3 (works in 1.1.x)

Logs


2024-11-04 15:59:15.979 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Processing videos: ['/config/www/llmvision/front-camera-clip.mp4']
2024-11-04 15:59:15.980 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Created /config/custom_components/llmvision/tmp_frames
2024-11-04 15:59:16.293 DEBUG (MainThread) [custom_components.llmvision.media_handlers] Adding frame frame0001.jpg
2024-11-04 15:59:16.307 INFO (MainThread) [custom_components.llmvision.media_handlers] Failed to delete tmp folder: [Errno 2] No such file or directory: '/config/custom_components/llmvision/tmp_clips'
2024-11-04 15:59:16.308 INFO (MainThread) [custom_components.llmvision.media_handlers] Deleted tmp folder: /config/custom_components/llmvision/tmp_frames
2024-11-04 15:59:16.309 INFO (MainThread) [custom_components.llmvision.request_handlers] Provider from call: Google

Additional context

Discussions for this started in my feature request 75 here

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 16 (5 by maintainers)

Commits related to this issue

Most upvoted comments

I can’t see errors any more even at short durations. Looks fixed to me