frigate: [Camera Support]: OOM killed ffmpeg process when rtsp-restream enabled
Describe the problem you are having
I am having an oom kill of ffmpeg process when I have preset-rtsp-restrem in the input_args, removing it ffmpeg: inputs: - path: rtsp://127.0.0.1:8554/front_door input_args: preset-rtsp-restream
Version
0.12.0-rc2
Frigate config file
detectors:
coral:
type: edgetpu
device: usb
mqtt:
host: 10.8.8.1
user: xxx
password: xxx
ffmpeg:
hwaccel_args: preset-vaapi
birdseye:
enabled: True
mode: motion
go2rtc:
streams:
entryway:
- rtsp://wyze1.gmg.lan/live
frontdoor:
- rtsp://wyze2.gmg.lan/live
record:
enabled: True
retain:
days: 7
mode: motion
events:
retain:
default: 14
mode: active_objects
cameras:
frontdoor:
detect:
width: 1280
height: 720
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/frontdoor
roles:
- detect
- record
entryway:
detect:
width: 1280
height: 720
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/entryway
roles:
- detect
- record
Relevant log output
2023-04-06 19:31:14.585492245 [2023-04-06 19:31:14] watchdog.frontdoor INFO : frontdoor exceeded fps limit. Exiting ffmpeg...
2023-04-06 19:31:14.585718067 [2023-04-06 19:31:14] watchdog.frontdoor INFO : Waiting for ffmpeg to exit gracefully...
2023-04-06 19:31:20.466185139 172.20.0.2 - - [06/Apr/2023:19:31:20 -0700] "GET /api/stats HTTP/1.1" 200 2527 "-" "HomeAssistant/2023.4.0 aiohttp/3.8.4 Python/3.10" "-"
^[[C2023-04-06 19:31:29.466407224 172.20.0.2 - - [06/Apr/2023:19:31:29 -0700] "GET /api/stats HTTP/1.1" 200 2526 "-" "HomeAssistant/2023.4.0 aiohttp/3.8.4 Python/3.10" "-"
2023-04-06 19:31:30.631212197 19:31:30.631 WRN github.com/AlexxIT/go2rtc/cmd/streams/producer.go:133 > error="read tcp 172.20.0.4:40882->172.18.9.91:554: i/o timeout" url=rtsp://Frontent:Char16ly@wyze1.gmg.lan/live
2023-04-06 19:31:34.595539190 [2023-04-06 19:31:34] watchdog.entryway INFO : entryway exceeded fps limit. Exiting ffmpeg...
2023-04-06 19:31:34.595717644 [2023-04-06 19:31:34] watchdog.entryway INFO : Waiting for ffmpeg to exit gracefully...
2023-04-06 19:31:38.474349320 172.20.0.2 - - [06/Apr/2023:19:31:38 -0700] "GET /api/stats HTTP/1.1" 200 2530 "-" "HomeAssistant/2023.4.0 aiohttp/3.8.4 Python/3.10" "-"
2023-04-06 19:31:45.833307431 [2023-04-06 19:31:45] frigate.record WARNING : Unable to find file from recordings database: /media/frigate/recordings/2023-03-31/02/front_door/44.51.mp4
2023-04-06 19:31:45.962100239 fatal error: concurrent map iteration and map write
2023-04-06 19:31:45.967621354
2023-04-06 19:31:45.967717053 goroutine 86 [running]:
2023-04-06 19:31:45.980792898 github.com/AlexxIT/go2rtc/pkg/streamer.(*Track).WriteRTP(0xc000290540, 0xc000290570?)
2023-04-06 19:31:45.980799100 github.com/AlexxIT/go2rtc/pkg/streamer/track.go:48 +0x8d
2023-04-06 19:31:45.981739410 github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).Handle(0xc0001fe1e0)
2023-04-06 19:31:45.981744299 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:688 +0x97b
2023-04-06 19:31:45.981745742 github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).Start(0xc0001fe1e0?)
2023-04-06 19:31:45.981751252 github.com/AlexxIT/go2rtc/pkg/rtsp/producer.go:64 +0x94
2023-04-06 19:31:45.982581917 github.com/AlexxIT/go2rtc/cmd/streams.(*Producer).worker(0xc000188360, {0xcc2920?, 0xc0001fe1e0?}, 0x1)
2023-04-06 19:31:45.982586736 github.com/AlexxIT/go2rtc/cmd/streams/producer.go:124 +0x35
2023-04-06 19:31:45.983809575 created by github.com/AlexxIT/go2rtc/cmd/streams.(*Producer).reconnect
2023-04-06 19:31:45.983868355 github.com/AlexxIT/go2rtc/cmd/streams/producer.go:180 +0x42c
2023-04-06 19:31:45.983870740
2023-04-06 19:31:45.984754214 goroutine 1 [chan receive]:
2023-04-06 19:31:45.988393145 main.main()
2023-04-06 19:31:45.988398415 github.com/AlexxIT/go2rtc/main.go:61 +0x133
2023-04-06 19:31:45.995873106
2023-04-06 19:31:45.995878356 goroutine 6 [IO wait]:
2023-04-06 19:31:45.995894977 internal/poll.runtime_pollWait(0x7f3724802d28, 0x72)
2023-04-06 19:31:45.995896570 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:45.996069003 internal/poll.(*pollDesc).wait(0xc0000b3680?, 0x6?, 0x0)
2023-04-06 19:31:45.996081867 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:45.996789091 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:45.996810040 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:45.996811994 internal/poll.(*FD).Accept(0xc0000b3680)
2023-04-06 19:31:45.996813727 internal/poll/fd_unix.go:614 +0x234
2023-04-06 19:31:45.996907092 net.(*netFD).accept(0xc0000b3680)
2023-04-06 19:31:45.996915337 net/fd_unix.go:172 +0x35
2023-04-06 19:31:45.997026876 net.(*TCPListener).accept(0xc0000108d0)
2023-04-06 19:31:45.997029030 net/tcpsock_posix.go:142 +0x28
2023-04-06 19:31:45.997037305 net.(*TCPListener).Accept(0xc0000108d0)
2023-04-06 19:31:45.997040862 net/tcpsock.go:288 +0x3d
2023-04-06 19:31:45.997465477 net/http.(*Server).Serve(0xc0001fe000, {0xcc0e78, 0xc0000108d0})
2023-04-06 19:31:45.997486226 net/http/server.go:3070 +0x385
2023-04-06 19:31:45.997487368 github.com/AlexxIT/go2rtc/cmd/api.Init.func1()
2023-04-06 19:31:45.997488290 github.com/AlexxIT/go2rtc/cmd/api/api.go:73 +0x32
2023-04-06 19:31:45.997613494 created by github.com/AlexxIT/go2rtc/cmd/api.Init
2023-04-06 19:31:45.997621800 github.com/AlexxIT/go2rtc/cmd/api/api.go:72 +0x47d
2023-04-06 19:31:45.997623232
2023-04-06 19:31:45.997637719 goroutine 7 [IO wait]:
2023-04-06 19:31:45.997660322 internal/poll.runtime_pollWait(0x7f3724802c38, 0x72)
2023-04-06 19:31:45.997671603 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:45.997692782 internal/poll.(*pollDesc).wait(0xc0000b3800?, 0x6?, 0x0)
2023-04-06 19:31:45.997694706 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:45.997708792 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:45.997710315 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:45.997717509 internal/poll.(*FD).Accept(0xc0000b3800)
2023-04-06 19:31:45.997718761 internal/poll/fd_unix.go:614 +0x234
2023-04-06 19:31:45.997724732 net.(*netFD).accept(0xc0000b3800)
2023-04-06 19:31:45.997726045 net/fd_unix.go:172 +0x35
2023-04-06 19:31:45.997732046 net.(*TCPListener).accept(0xc000010a68)
2023-04-06 19:31:45.997738508 net/tcpsock_posix.go:142 +0x28
2023-04-06 19:31:45.997743618 net.(*TCPListener).Accept(0xc000010a68)
2023-04-06 19:31:45.997748827 net/tcpsock.go:288 +0x3d
2023-04-06 19:31:45.997750450 github.com/AlexxIT/go2rtc/cmd/rtsp.Init.func1()
2023-04-06 19:31:45.997756181 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:62 +0x58
2023-04-06 19:31:45.997761912 created by github.com/AlexxIT/go2rtc/cmd/rtsp.Init
2023-04-06 19:31:45.997767582 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:60 +0x413
2023-04-06 19:31:45.997913025
2023-04-06 19:31:45.997915780 goroutine 8 [IO wait]:
2023-04-06 19:31:45.997931820 internal/poll.runtime_pollWait(0x7f3724802b48, 0x72)
2023-04-06 19:31:45.997934635 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:45.997936228 internal/poll.(*pollDesc).wait(0xc0000b3a80?, 0xc000126000?, 0x0)
2023-04-06 19:31:45.997937821 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:45.998048658 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:45.998050782 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:45.998159035 internal/poll.(*FD).ReadFromInet6(0xc0000b3a80, {0xc000126000, 0x800, 0x800}, 0x7f3724ca5fff?)
2023-04-06 19:31:45.998165728 internal/poll/fd_unix.go:277 +0x1e5
2023-04-06 19:31:45.998266146 net.(*netFD).readFromInet6(0xc0000b3a80, {0xc000126000?, 0x0?, 0x417c0b?}, 0x4?)
2023-04-06 19:31:45.998276996 net/fd_posix.go:72 +0x29
2023-04-06 19:31:45.998285161 net.(*UDPConn).readFrom(0xc000100000?, {0xc000126000?, 0x0?, 0x1010000000000?}, 0xc0001160f0)
2023-04-06 19:31:45.998286554 net/udpsock_posix.go:59 +0x85
2023-04-06 19:31:45.998295611 net.(*UDPConn).readFromUDP(0xc000012970, {0xc000126000?, 0x44f352?, 0x800?}, 0xab98a0?)
2023-04-06 19:31:45.998302103 net/udpsock.go:149 +0x31
2023-04-06 19:31:45.998309196 net.(*UDPConn).ReadFrom(0x18?, {0xc000126000, 0x800, 0x800})
2023-04-06 19:31:45.998310459 net/udpsock.go:158 +0x50
2023-04-06 19:31:45.998547152 github.com/AlexxIT/go2rtc/pkg/srtp.(*Server).Serve(0xc000124000, {0xcc4e60?, 0xc000012970})
2023-04-06 19:31:45.998549967 github.com/AlexxIT/go2rtc/pkg/srtp/server.go:41 +0x82
2023-04-06 19:31:45.998668008 github.com/AlexxIT/go2rtc/cmd/srtp.Init.func1()
2023-04-06 19:31:45.998670492 github.com/AlexxIT/go2rtc/cmd/srtp/srtp.go:39 +0x85
2023-04-06 19:31:45.998671574 created by github.com/AlexxIT/go2rtc/cmd/srtp.Init
2023-04-06 19:31:45.998673268 github.com/AlexxIT/go2rtc/cmd/srtp/srtp.go:37 +0x28f
2023-04-06 19:31:46.003519739
2023-04-06 19:31:46.003525590 goroutine 9 [IO wait]:
2023-04-06 19:31:46.003526913 internal/poll.runtime_pollWait(0x7f3724802a58, 0x72)
2023-04-06 19:31:46.003527835 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:46.003537974 internal/poll.(*pollDesc).wait(0xc0000b3c80?, 0xc000128000?, 0x0)
2023-04-06 19:31:46.003541691 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:46.003545097 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:46.003580744 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:46.003584681 internal/poll.(*FD).ReadFromInet6(0xc0000b3c80, {0xc000128000, 0x2000, 0x2000}, 0x7f3724ca5fff?)
2023-04-06 19:31:46.003588398 internal/poll/fd_unix.go:277 +0x1e5
2023-04-06 19:31:46.003591153 net.(*netFD).readFromInet6(0xc0000b3c80, {0xc000128000?, 0x0?, 0x0?}, 0x1?)
2023-04-06 19:31:46.003594049 net/fd_posix.go:72 +0x29
2023-04-06 19:31:46.003596653 net.(*UDPConn).readFrom(0xc000100000?, {0xc000128000?, 0x0?, 0x1010000000000?}, 0xc000116120)
2023-04-06 19:31:46.003632941 net/udpsock_posix.go:59 +0x85
2023-04-06 19:31:46.003635667 net.(*UDPConn).readFromUDP(0xc0000129e0, {0xc000128000?, 0x44f352?, 0x2000?}, 0xab98a0?)
2023-04-06 19:31:46.003656185 net/udpsock.go:149 +0x31
2023-04-06 19:31:46.003657768 net.(*UDPConn).ReadFrom(0x0?, {0xc000128000, 0x2000, 0x2000})
2023-04-06 19:31:46.003658900 net/udpsock.go:158 +0x50
2023-04-06 19:31:46.007714061 github.com/pion/ice/v2.(*UDPMuxDefault).connWorker(0xc0000b3d00)
2023-04-06 19:31:46.007719711 github.com/pion/ice/v2@v2.2.6/udp_mux.go:239 +0xdb
2023-04-06 19:31:46.007721605 created by github.com/pion/ice/v2.NewUDPMuxDefault
2023-04-06 19:31:46.007737765 github.com/pion/ice/v2@v2.2.6/udp_mux.go:68 +0x1e5
2023-04-06 19:31:46.007738637
2023-04-06 19:31:46.007739438 goroutine 10 [IO wait]:
2023-04-06 19:31:46.007740380 internal/poll.runtime_pollWait(0x7f3724802968, 0x72)
2023-04-06 19:31:46.007741292 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:46.007742985 internal/poll.(*pollDesc).wait(0xc0000b3e00?, 0x0?, 0x0)
2023-04-06 19:31:46.007749567 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:46.007750780 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:46.007751902 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:46.007757893 internal/poll.(*FD).Accept(0xc0000b3e00)
2023-04-06 19:31:46.007759135 internal/poll/fd_unix.go:614 +0x234
2023-04-06 19:31:46.007760207 net.(*netFD).accept(0xc0000b3e00)
2023-04-06 19:31:46.007765868 net/fd_unix.go:172 +0x35
2023-04-06 19:31:46.007767020 net.(*TCPListener).accept(0xc000010c48)
2023-04-06 19:31:46.007772460 net/tcpsock_posix.go:142 +0x28
2023-04-06 19:31:46.007773672 net.(*TCPListener).Accept(0xc000010c48)
2023-04-06 19:31:46.007779944 net/tcpsock.go:288 +0x3d
2023-04-06 19:31:46.007781187 github.com/pion/ice/v2.(*TCPMuxDefault).start(0xc0001c4d40)
2023-04-06 19:31:46.007782359 github.com/pion/ice/v2@v2.2.6/tcp_mux.go:91 +0xdd
2023-04-06 19:31:46.007788059 github.com/pion/ice/v2.NewTCPMuxDefault.func1()
2023-04-06 19:31:46.007789242 github.com/pion/ice/v2@v2.2.6/tcp_mux.go:82 +0x5a
2023-04-06 19:31:46.007790474 created by github.com/pion/ice/v2.NewTCPMuxDefault
2023-04-06 19:31:46.007796104 github.com/pion/ice/v2@v2.2.6/tcp_mux.go:80 +0x1a5
2023-04-06 19:31:46.008988536
2023-04-06 19:31:46.008993616 goroutine 17 [syscall]:
2023-04-06 19:31:46.008994878 os/signal.signal_recv()
2023-04-06 19:31:46.008996251 runtime/sigqueue.go:152 +0x2f
2023-04-06 19:31:46.008997373 os/signal.loop()
2023-04-06 19:31:46.009013764 os/signal/signal_unix.go:23 +0x19
2023-04-06 19:31:46.009015026 created by os/signal.Notify.func1.1
2023-04-06 19:31:46.013969250 os/signal/signal.go:151 +0x2a
2023-04-06 19:31:46.013974740
2023-04-06 19:31:46.014025775 goroutine 33 [runnable]:
2023-04-06 19:31:46.016642673 github.com/AlexxIT/go2rtc/cmd/streams.(*Producer).stop(0xc000188360)
2023-04-06 19:31:46.016647533 github.com/AlexxIT/go2rtc/cmd/streams/producer.go:200 +0x127
2023-04-06 19:31:46.016649226 github.com/AlexxIT/go2rtc/cmd/streams.(*Stream).stopProducers(0xc0001c4580)
2023-04-06 19:31:46.016650678 github.com/AlexxIT/go2rtc/cmd/streams/stream.go:177 +0xb2
2023-04-06 19:31:46.016652312 github.com/AlexxIT/go2rtc/cmd/streams.(*Stream).RemoveConsumer(0xc0001c4580, {0xcc01b8?, 0xc000294000})
2023-04-06 19:31:46.016653724 github.com/AlexxIT/go2rtc/cmd/streams/stream.go:147 +0x13a
2023-04-06 19:31:46.017586771 github.com/AlexxIT/go2rtc/cmd/rtsp.tcpHandler.func1.1()
2023-04-06 19:31:46.017591940 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:191 +0x28
2023-04-06 19:31:46.017593123 github.com/AlexxIT/go2rtc/cmd/rtsp.tcpHandler(0xc000294000)
2023-04-06 19:31:46.017594235 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:240 +0x323
2023-04-06 19:31:46.017611066 created by github.com/AlexxIT/go2rtc/cmd/rtsp.Init.func1
2023-04-06 19:31:46.017612168 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:72 +0x48
2023-04-06 19:31:46.017612920
2023-04-06 19:31:46.017613751 goroutine 34 [IO wait]:
2023-04-06 19:31:46.017614713 internal/poll.runtime_pollWait(0x7f3724802788, 0x72)
2023-04-06 19:31:46.017615625 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:46.017629310 internal/poll.(*pollDesc).wait(0xc00028e080?, 0xc000297000?, 0x0)
2023-04-06 19:31:46.017630312 internal/poll/fd_poll_runtime.go:84 +0x32
2023-04-06 19:31:46.018766840 internal/poll.(*pollDesc).waitRead(...)
2023-04-06 19:31:46.018771949 internal/poll/fd_poll_runtime.go:89
2023-04-06 19:31:46.018773572 internal/poll.(*FD).Read(0xc00028e080, {0xc000297000, 0x1000, 0x1000})
2023-04-06 19:31:46.018774955 internal/poll/fd_unix.go:167 +0x25a
2023-04-06 19:31:46.018789873 net.(*netFD).Read(0xc00028e080, {0xc000297000?, 0x0?, 0x4d29e0?})
2023-04-06 19:31:46.018791175 net/fd_posix.go:55 +0x29
2023-04-06 19:31:46.018792648 net.(*conn).Read(0xc000292008, {0xc000297000?, 0xc00028e080?, 0x0?})
2023-04-06 19:31:46.018793900 net/net.go:183 +0x45
2023-04-06 19:31:46.023067430 bufio.(*Reader).fill(0xc0002820c0)
2023-04-06 19:31:46.023072669 bufio/bufio.go:106 +0xff
2023-04-06 19:31:46.027952033 bufio.(*Reader).Peek(0xc0002820c0, 0x4)
2023-04-06 19:31:46.027957683 bufio/bufio.go:144 +0x5d
2023-04-06 19:31:46.027959777 github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).Handle(0xc0002940f0)
2023-04-06 19:31:46.027961721 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:598 +0x3cf
2023-04-06 19:31:46.027964316 github.com/AlexxIT/go2rtc/cmd/rtsp.tcpHandler(0xc0002940f0)
2023-04-06 19:31:46.027966530 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:236 +0x2a7
2023-04-06 19:31:46.027968624 created by github.com/AlexxIT/go2rtc/cmd/rtsp.Init.func1
2023-04-06 19:31:46.027970317 github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:72 +0x48
2023-04-06 19:31:46.027971740
2023-04-06 19:31:46.027973693 goroutine 87 [sleep]:
2023-04-06 19:31:46.027976108 time.Sleep(0x5d21dba00)
2023-04-06 19:31:46.027977721 runtime/time.go:195 +0x135
2023-04-06 19:31:46.030542512 github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).keepalive(0xc0001fe1e0)
2023-04-06 19:31:46.030547441 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:713 +0x70
2023-04-06 19:31:46.030548743 created by github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).Handle
2023-04-06 19:31:46.030549825 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:562 +0x23c
2023-04-06 19:31:46.030550707
2023-04-06 19:31:46.030551569 goroutine 98 [sleep]:
2023-04-06 19:31:46.030552390 time.Sleep(0x5d21dba00)
2023-04-06 19:31:46.030553252 runtime/time.go:195 +0x135
2023-04-06 19:31:46.030554244 github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).keepalive(0xc0001fe2d0)
2023-04-06 19:31:46.030555235 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:713 +0x70
2023-04-06 19:31:46.030556227 created by github.com/AlexxIT/go2rtc/pkg/rtsp.(*Conn).Handle
2023-04-06 19:31:46.030557159 github.com/AlexxIT/go2rtc/pkg/rtsp/client.go:562 +0x23c
2023-04-06 19:31:46.030557910
2023-04-06 19:31:46.030558712 goroutine 97 [IO wait]:
2023-04-06 19:31:46.030559644 internal/poll.runtime_pollWait(0x7f3724802698, 0x72)
2023-04-06 19:31:46.030560505 runtime/netpoll.go:305 +0x89
2023-04-06 19:31:46.030561477 internal/poll
FFprobe output from your camera
no output
Frigate stats
No response
Operating system
Other Linux
Install method
Docker CLI
Coral version
USB
Network connection
Wired
Camera make and model
wyze
Any other information that may be helpful
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 21 (1 by maintainers)
It has already been updated in the dev branch, will be part of 0.13
yes, this is a known issue with the currently included go2rtc 1.2.0. You can update the internal go2rtc to a newer version which many users have said fixes the issue https://docs.frigate.video/configuration/advanced#custom-go2rtc-version