electrs: Bug: Too many open files (os error 24)

On a nodl OG, after upgrading to electrs 9.1. I guess electrs is reindexing. The nodl frontend shows the block height increasing. After a few hours of running, electrs terminates early and logs the following:

[2021-11-01T11:21:06.033Z ERROR tiny_http] Error accepting new client: Too many open files (os error 24)          [0/20]
[2021-11-01T11:21:26.099Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:21:32.620Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:21:39.146Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:21:45.542Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:21:52.025Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:21:58.476Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:22:04.994Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:22:11.414Z WARN  electrs::thread] recv_loop thread failed: Too many open files (os error 24)             
[2021-11-01T11:22:17.730Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel             
[2021-11-01T11:22:24.569Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel             
[2021-11-01T11:22:31.132Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel            
 [2021-11-01T11:22:37.483Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel             
[2021-11-01T11:22:43.813Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel             
[2021-11-01T11:22:50.210Z WARN  electrs::thread] recv_loop thread failed: sending on a disconnected channel             

Error: electrs failed

Caused by:
    0: sync failed
    1: failed to get DB size
    2: Too many open files (os error 24)

Restarting continues at the block height it was at before. I’ve been through this dance for a few days and am now at height 660k - almost synced.

Electrs version 0.9.1

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 22 (14 by maintainers)

Most upvoted comments

I couldn’t yet reproduce this issue on one of my local devices but once the ulimit was effectively increased (to 65k, finally found why my new limit wasn’t taken into account), the sync finished and electrs is working properly. Still no clue why and which files it opens to exceed the default 1024 limit.

ketominer on Nov 9, 2021

it seems that nodl had imposed some ulimits and has now increased them in the latest update specifically for electrs. So it appears this was more of a nodl problem than an electrs problem.

stefanwouldgo on Nov 4, 2021