dendrite: dendrite keeps crashing

We are running dendrite monolith from official docker repo with version 0.13.1+49d75d3 with PG. We suddenly are experiencing server crashes with the following in our logs:

dendrite-dendrite-1  | time="2023-07-10T08:36:42.877813732Z" level=info msg="Server is ready" component=jetstream
dendrite-dendrite-1  | time="2023-07-10T08:36:43.106458129Z" level=debug msg="Updating server ACLs for \"!RcWPWcZrMeBxOGaalX:matrix.org\"" allow_ip_literals=false num_allowed=1 num_denied=623
dendrite-dendrite-1  | time="2023-07-10T08:36:43.179897070Z" level=debug msg="Setting DB connection limits" conn_max_lifetime=-1s data_source_name="postgresql://postgres/dendrite?sslmode=disable" max_idle_conns=5 max_open_conns=90
dendrite-dendrite-1  | time="2023-07-10T08:36:43.437272507Z" level=panic msg="roomserver output log: write room event failure" add="[$FqTXDcxpzANTjqqfdBMPfyWoZqAQkZKA39prqIHaaQQ $drRD4AYpfNyDjygDKQV3VNqcTwjLhCiqAJxz4h3ANls $fsBrKinoE0Q-9ZJvIvOInCJKM7yy2xcJ7TNw3_iKqrk $JA_gYx7b_mdv1nXl5lkSrpCi5QT481e2BfhA34TXbRs $jAk4qdFKOr9c4C5m0ltrHas7UupI_cJh6unsUi8h9lU $8SBbgT78yrC0IGCyk_dUYwqjuAM052UKnqzh0uNqIak $vj7I9TT5N6_Bre6aTLFJThymBJtSD0JY2Ix9zYpcFFw $CL0ULuBK5JobhkcCcRimilY-02oSTjWyn-v4he0CY-M $JnRiQ33ZTD53x2KmphyF1e3yG31nq1ighga27IizLxo]" del="[$F1mZXPwlz9CTNxSjgQXLLOCdL3KJejJjAgKcrmwZW9s $EZM5oaY963eFLV4Gm-5yuH6Skvn8qXl0sFYErlWqUa4 $drRD4AYpfNyDjygDKQV3VNqcTwjLhCiqAJxz4h3ANls $fsBrKinoE0Q-9ZJvIvOInCJKM7yy2xcJ7TNw3_iKqrk $JA_gYx7b_mdv1nXl5lkSrpCi5QT481e2BfhA34TXbRs $jAk4qdFKOr9c4C5m0ltrHas7UupI_cJh6unsUi8h9lU $F1mZXPwlz9CTNxSjgQXLLOCdL3KJejJjAgKcrmwZW9s $FO8sjr_wqlvLo0ijO2TqBBUej3lg5HhvRab8OeOerKo $J9qnDRKI2ghzzFjIIgdePHur88BuqehK-2R4jxIvYbg $rBVR-cSFwWE7j-Le1ZMAyniV3RheQCFdQyKRDyMzbfc $f1ai0vfQcvjwZqllXK98-_qPquysxXcDnNM0NuZZ-x4 $FqTXDcxpzANTjqqfdBMPfyWoZqAQkZKA39prqIHaaQQ]" error="missing state events" event="{\"auth_events\":[\"$F35rpro7JOOszmY1uvC02D9wbRoJknNdvZw-da2ZPfE\",\"$cUryPGSGk6SNtI1Ni6hyeufOUqcXkHlMwkwYjGMt4DA\",\"$CMCptS0SN1RcCvjNLgwG_8pK-3JLWANxkdSqQUhLgAU\"],\"content\":{\"body\":\"\\u003e \\u003c@fireglow488:matrix.org\\u003e Hi! I've tried to join this room from my homeserver (matrix.firc.de), but element said it was banned from this room. I'm not sure if I ever joined this room before from my server. What to do?\\n\\nhttps://federationtester.matrix.org/#matrix.firc.de\",\"format\":\"org.matrix.custom.html\",\"formatted_body\":\"\\u003cmx-reply\\u003e\\u003cblockquote\\u003e\\u003ca href=\\\"https://matrix.to/#/!RcWPWcZrMeBxOGaalX:matrix.org/$xvsSJKijHLKG6DzLBWLgffyQ-3_M7i_IElfWL2YnYn8?via=matrix.org\\u0026via=envs.net\\u0026via=dendrite.matrix.org\\\"\\u003eIn reply to\\u003c/a\\u003e \\u003ca href=\\\"https://matrix.to/#/@fireglow488:matrix.org\\\"\\u003e@fireglow488:matrix.org\\u003c/a\\u003e\\u003cbr\\u003eHi! I\\u0026#39;ve tried to join this room from my homeserver (matrix.firc.de), but element said it was banned from this room. I\\u0026#39;m not sure if I ever joined this room before from my server. What to do? \\u003c/blockquote\\u003e\\u003c/mx-reply\\u003ehttps://federationtester.matrix.org/#matrix.firc.de\",\"m.relates_to\":{\"m.in_reply_to\":{\"event_id\":\"$xvsSJKijHLKG6DzLBWLgffyQ-3_M7i_IElfWL2YnYn8\"}},\"msgtype\":\"m.text\"},\"depth\":32215,\"hashes\":{\"sha256\":\"sUUOrAa4+gQSjyFxe0oErRDV9CHCYch0xq89Yq6QE4E\"},\"origin\":\"aguiarvieira.pt\",\"origin_server_ts\":1688819576127,\"prev_events\":[\"$wd1L92k-1ILw5bViz2NaA-2LepYFnLhRoqNEd7itPus\"],\"room_id\":\"!RcWPWcZrMeBxOGaalX:matrix.org\",\"sender\":\"@daedric:aguiarvieira.pt\",\"signatures\":{\"aguiarvieira.pt\":{\"ed25519:a_Yofy\":\"iOSgP/dAoqKWzsS7hVHhIKpAo2iazt87uIahrh7E/FB98kMl8C2Xm3+hF8ZeHNbYsebSWb2Y+5EZcVbrjfZ0BA\"}},\"type\":\"m.room.message\"}" event_id="$BIy1wVT--iTI-suA2aLU827hFwy9VToQ_D4hBM0egDU"
dendrite-dendrite-1  | panic: (*logrus.Entry) 0xc00049ce70
dendrite-dendrite-1  |
dendrite-dendrite-1  | goroutine 2804 [running]:
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).log(0xc00049ce00, 0x0, {0xc003e7c150, 0x2f})
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:260 +0x4d6
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).Log(0xc00049ce00, 0x0, {0xc00011ce08?, 0xe54679?, 0x0?})
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:304 +0x4f
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).Panic(...)
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:342
dendrite-dendrite-1  | github.com/MFAshby/stdemuxerhook.(*StdDemuxerHook).Fire(0x0?, 0xc00049ccb0)
dendrite-dendrite-1  |  github.com/MFAshby/stdemuxerhook@v1.0.0/stdemuxerhook.go:58 +0x19e
dendrite-dendrite-1  | github.com/sirupsen/logrus.LevelHooks.Fire(0xc00011cf48?, 0x11cf18?, 0x5?)
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/hooks.go:28 +0x7a
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).fireHooks(0xc00049ccb0)
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:280 +0x1f0
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).log(0xc00049cc40, 0x0, {0xc003e7c120, 0x2f})
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:242 +0x3c5
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).Log(0xc00049cc40, 0x0, {0xc00011d208?, 0x0?, 0x0?})
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:304 +0x4f
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).Logf(0xc00049cc40, 0x0, {0x185d4b4?, 0x5?}, {0x0?, 0xc005fbee10?, 0x9?})
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:349 +0x85
dendrite-dendrite-1  | github.com/sirupsen/logrus.(*Entry).Panicf(...)
dendrite-dendrite-1  |  github.com/sirupsen/logrus@v1.9.3/entry.go:387
dendrite-dendrite-1  | github.com/matrix-org/dendrite/federationapi/consumers.(*OutputRoomEventConsumer).onMessage(0xc0061275f0, {0x1a9b468, 0xc0062b6de0}, {0xc0062bc2a8?, 0xc005ea5ca0?, 0xc00599ba10?})
dendrite-dendrite-1  |  github.com/matrix-org/dendrite/federationapi/consumers/roomserver.go:121 +0x915
dendrite-dendrite-1  | github.com/matrix-org/dendrite/setup/jetstream.JetStreamConsumer.func2()
dendrite-dendrite-1  |  github.com/matrix-org/dendrite/setup/jetstream/helpers.go:100 +0x454
dendrite-dendrite-1  | created by github.com/matrix-org/dendrite/setup/jetstream.JetStreamConsumer
dendrite-dendrite-1  |  github.com/matrix-org/dendrite/setup/jetstream/helpers.go:44 +0x31e
dendrite-dendrite-1 exited with code 2

The server keeps crashing and we are unable to start it anymore.