frigate: Jetson Nano Unknown decoder 'h264_nvmpi'
Describe the bug Basically I am running this on a jetson nano, I have compiled ffmpeg with the jocover/jetson-ffmpeg package. I have also confirmed that it is properly showing by running ffmpeg -encoders |grep 264 and I see it there. Frigate say that it is an unknown decoder.
Version of frigate 0.8.4-5043040
Config file Include your full config file wrapped in triple back ticks.
detectors:
coral:
type: edgetpu
device: pci
ffmpeg:
global_args: -hide_banner -loglevel warning
hwaccel_args: []
input_args: -c:v h264_nvmpi -vsync drop
output_args:
detect: -f rawvideo -pix_fmt rgb24
mqtt:
host: 192.168.0.000
user: **********
password: *********
objects:
# Optional: list of objects to track from labelmap.txt (default: shown below)
track:
- person
- car
- truck
- dog
- horse
cameras:
front_left:
ffmpeg:
inputs:
- path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/101
roles:
- detect
- rtmp
- clips
width: 2688
height: 1520
fps: 1
clips:
enabled: true
motion:
mask:
- 0,0,690,0,499,35,0,479
- 2688,0,2688,277,2320,142,1623,0
#Removing timestamp
- 114,94,104,153,857,177,888,97
front_right:
ffmpeg:
inputs:
- path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/301
roles:
- detect
- rtmp
- clips
width: 2688
height: 1520
fps: 1
clips:
enabled: true
motion:
mask:
- 2688,0,2688,548,2504,316,2372,284,1810,156,1502,132,1179,118,482,135,146,180,0,243,0,0
#Removing timestamp
- 114,94,104,153,857,177,888,97
back_left:
ffmpeg:
inputs:
- path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/201
roles:
- detect
- rtmp
width: 2688
height: 1520
fps: 1
motion:
mask:
- 191,496,83,55,836,69,1044,205,538,1304
backyard:
ffmpeg:
inputs:
- path: rtsp://192.168.0.000:554/live0
roles:
- detect
- rtmp
width: 1920
height: 1080
fps: 1
garage:
ffmpeg:
inputs:
- path: rtsp://192.168.0.000:554/live0
roles:
- detect
- rtmp
width: 1920
height: 1080
fps: 1
Frigate container logs
frigate.video INFO : backyard: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video INFO : backyard: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video INFO : backyard: ffmpeg process is not running. exiting capture thread...
ffmpeg.garage.detect ERROR : Unknown decoder 'h264_nvmpi'
Frigate stats
{
"back_left": {
"camera_fps": 0.0,
"capture_pid": 63,
"detection_fps": 0.0,
"pid": 44,
"process_fps": 0.0,
"skipped_fps": 0.0
},
"backyard": {
"camera_fps": 0.0,
"capture_pid": 66,
"detection_fps": 0.0,
"pid": 55,
"process_fps": 0.0,
"skipped_fps": 0.0
},
"detection_fps": 0.0,
"detectors": {
"coral": {
"detection_start": 0.0,
"inference_speed": 10.0,
"pid": 36
}
},
"front_left": {
"camera_fps": 0.0,
"capture_pid": 57,
"detection_fps": 0.0,
"pid": 39,
"process_fps": 0.0,
"skipped_fps": 0.0
},
"front_right": {
"camera_fps": 0.0,
"capture_pid": 58,
"detection_fps": 0.0,
"pid": 41,
"process_fps": 0.0,
"skipped_fps": 0.0
},
"garage": {
"camera_fps": 0.0,
"capture_pid": 70,
"detection_fps": 0.0,
"pid": 56,
"process_fps": 0.0,
"skipped_fps": 0.0
},
"service": {
"storage": {
"/dev/shm": {
"free": 1047.9,
"mount_type": "tmpfs",
"total": 1048.6,
"used": 0.7
},
"/media/frigate/clips": {
"free": 35464.0,
"mount_type": "ext4",
"total": 62716.1,
"used": 24542.0
},
"/media/frigate/recordings": {
"free": 35464.0,
"mount_type": "ext4",
"total": 62716.1,
"used": 24542.0
},
"/tmp/cache": {
"free": 35464.0,
"mount_type": "ext4",
"total": 62716.1,
"used": 24542.0
}
},
"uptime": 572,
"version": "0.8.4-5043040"
}
}
FFprobe from your camera
Run the following command and paste output below
I don't think this is useful as my streams show fine without hwaccell
Screenshots Video is all green with this option on
Computer Hardware
- OS: JetPack Ubuntu
- Install method: Docker Compose
- Virtualization: N/A
- Coral Version: m2 ekey
- Network Setup: Wired
Camera Info: Not relevant
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 66 (10 by maintainers)
https://github.com/e1z0/jetson-frigate i’ve do some research on it also.
I’m still picking away at this to see if this makes sense before submitting a pull request. This nvidia hardware tends to have a long life so I think once it’s figured out it shouldn’t be too bad to maintain. I’m still using my shield from 2016 for example.
I tried to make jetson-ffmpeg work with no luck, this likely to my limited experience building docker images–good opportunity to learn I suppose. I believe I found the compiled image mentioned earlier in this thread from the jetson-frigate project here: https://hub.docker.com/r/nulldevil/frigate/tags
I also noticed the same very limited GPU use as @KillahB33. More like flickering really. However I noticed zero NVDEC use which is really what we’re after. So based on that I wouldn’t write this off just yet.
If we are mostly concerned with NVDEC (hardware decoding) and can forgo hardware encoding a potentially simpler direction could be to use the Nvidia ffmpeg and just install it with apt.
If the correct way to do this would be to create a Dockerfile.ffmpeg.aarch64jetson. The Nvidia l4t base image could be used. You have to align the l4t image with the jetpack cuda version so for me it would be: R32.6.1 here: nvcr.io/nvidia/l4t-base:r32.6.1 I think you could also expose cuda that way but not sure if that’s valuable.
I did some limited testing with my camera streams and nvidia ffmpeg using that in the Nvidia l4t base image and it works great. There is also a version with apt sources enabled which may be easier to incorporate: https://hub.docker.com/r/mdegans/l4t-base
Next I think I’ll try to incorporate the Nvidia FFMPEG directly into the base like this:
Anyways, let me know I’m barking up the wrong tree.
@e1z0 @KillahB33 Is the built image hosted somewhere? I’m struggling to build jetson-frigate as well.
@blakeblackshear Do you think an official frigate aarch64nvidia for the Jetson Nano is in the cards? I have a m.2 Coral TPU on the way but if the NVEC, NVENC and potentially even the GPU can take care of decoding, the Nano + Coral could be a powerful hardware option.
Hey Guys, Any Progress with having a Frigate Version for Jetson Nano?
If it requires a separate based image, I would rather have a separate image like we do for amd64nvidia.
I managed to get FFMPEG working for me. Basically I used https://github.com/Metric-Void/jetson-ffmpeg-docker for FFMPEG, but I had to modify it at bit to get the entrypoints and install locations correct. I also had to modify Frigate a bit so that it was able to work with Python 3.6 (as the Nvidia docker container is still based on Ubuntu 18.04, which doesn’t really support Python 3.8). I’m just in the process of cleaning up some of the code, but I should have some stuff pushed up soon. But I doubt I’ll be able to incorporate it as a pull request, as it modifies too many core files. Hopefully once Nvidia releases a Jetpack image based on 20.04 (hopefully next year), it should be a lot simpler.
I’m getting pretty consistent use of NVDEC, with sporadic NVENC usage according to jtop:
I’ve got a pretty good setup now for testing as well. Just setup Linux server in ESXi 7 and was able to pass through a GeForce 790. Inference speeds for 5 streams are under 100ms but nvidia-smi doesn’t show it utilizing it I don’t think. I’m also waiting to hear back from the Double Take and DeepStack devs as my DeepStack install on my Jetson Nano quit being recognized by Double Take. Although it still is able to process commands through its API so I’m pretty sure it’s on the Double Take side of things. But those 4 pieces of software together are so very powerful, that is including Home Assistant.
On Dec 25, 2021, at 2:51 AM, toz @.***> wrote:
8 cams seems possible with good performance… wb666greene/AI-Person-Detector#11 (comment)https://github.com/wb666greene/AI-Person-Detector/issues/11#issuecomment-1000903998
— Reply to this email directly, view it on GitHubhttps://github.com/blakeblackshear/frigate/issues/1175#issuecomment-1000993775, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACGXNJBLMVTC6TGKCH4STWLUSWAXNANCNFSM46DAEPAQ. You are receiving this because you commented.Message ID: @.***>
I haven’t really stressed it too much. I’ve just got 6 cameras, and don’t use RTMP (although it was enabled). All seemed to be running fine, with no artifacts or anything.
the small jetson nano device is comparable cheap and AI powerfull on its own. Its desinged for that. Additonalle Edge TPU would enhance this. But it surely needs time and effort to make the frigate details stable and efficient on that device. even in a docker-ish way. maybe i jump into this game, but i wished others had already spent this time… 😄
I would put it in the docker folder.
So finally got back on this, hoping I’ll be able to close this out thanks to e1z0’s help. Basically I built his image, the base image he is using is pretty huge in comparison so trying get the best of both and get this working.
I’m afraid that, as is, the repo does build the ffmpeg binary, but doesn’t work on the jetson nano as per this issue.
I wouldn’t remove anything.