FlareSolverr: too many open files ?
Have you checked our README?
- I have checked the README
Have you followed our Troubleshooting?
- I have followed your Troubleshooting
Is there already an issue for your problem?
- I have checked older issues, open and closed
Have you checked the discussions?
- I have read the Discussions
Environment
- FlareSolverr version: 3.3.9
- Last working FlareSolverr version: 3.3.9
- Operating system: Debian 12
- Are you using Docker: [yes/no] no
- FlareSolverr User-Agent (see log traces or / endpoint): Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
- Are you using a VPN: [yes/no] no
- Are you using a Proxy: [yes/no] no
- Are you using Captcha Solver: [yes/no] no
- If using captcha solver, which one:
- URL to test this issue:
Description
after a few days of correct functionning, I get the error “internal server error” with “Too many open files”
Logged Error Messages
Nov 20 13:27:50 prowlarr flaresolverr[315582]: 2023-11-20 13:27:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 13:40:58 prowlarr flaresolverr[315582]: 2023-11-20 13:40:58 INFO Incoming request => POST /v1 body: {'maxTimeout': 60000, 'cmd': 'request.get', 'url': 'https://www
3.yggtorrent.wtf/engine/search?category=all&do=search&order=desc&sort=publish_date', 'proxy': {}}
Nov 20 13:40:58 prowlarr flaresolverr[315582]: 2023-11-20 13:40:58 ERROR Error: Error solving the challenge. [Errno 24] Too many open files
Nov 20 13:40:58 prowlarr flaresolverr[315582]: 2023-11-20 13:40:58 INFO Response in 0.033 s
Nov 20 13:40:58 prowlarr flaresolverr[315582]: 2023-11-20 13:40:58 INFO 127.0.0.1 POST http://localhost:8191/v1 500 Internal Server Error
Nov 20 13:42:50 prowlarr flaresolverr[315582]: 2023-11-20 13:42:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 13:56:26 prowlarr flaresolverr[315582]: 2023-11-20 13:56:26 INFO Incoming request => POST /v1 body: {'maxTimeout': 60000, 'cmd': 'request.get', 'url': 'https://www
3.yggtorrent.wtf/engine/search?category=all&do=search&order=desc&sort=publish_date', 'proxy': {}}
Nov 20 13:56:26 prowlarr flaresolverr[315582]: 2023-11-20 13:56:26 ERROR Error: Error solving the challenge. [Errno 24] Too many open files
Nov 20 13:56:26 prowlarr flaresolverr[315582]: 2023-11-20 13:56:26 INFO Response in 0.021 s
Nov 20 13:56:26 prowlarr flaresolverr[315582]: 2023-11-20 13:56:26 INFO 127.0.0.1 POST http://localhost:8191/v1 500 Internal Server Error
Nov 20 13:57:50 prowlarr flaresolverr[315582]: 2023-11-20 13:57:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 14:12:50 prowlarr flaresolverr[315582]: 2023-11-20 14:12:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 14:27:26 prowlarr flaresolverr[315582]: 2023-11-20 14:27:26 INFO Incoming request => POST /v1 body: {'maxTimeout': 60000, 'cmd': 'request.get', 'url': 'https://www
3.yggtorrent.wtf/engine/search?category=all&do=search&order=desc&sort=publish_date', 'proxy': {}}
Nov 20 14:27:26 prowlarr flaresolverr[315582]: 2023-11-20 14:27:26 ERROR Error: Error solving the challenge. [Errno 24] Too many open files
Nov 20 14:27:26 prowlarr flaresolverr[315582]: 2023-11-20 14:27:26 INFO Response in 0.022 s
Nov 20 14:27:26 prowlarr flaresolverr[315582]: 2023-11-20 14:27:26 INFO 127.0.0.1 POST http://localhost:8191/v1 500 Internal Server Error
Nov 20 14:27:50 prowlarr flaresolverr[315582]: 2023-11-20 14:27:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 14:42:50 prowlarr flaresolverr[315582]: 2023-11-20 14:42:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 14:57:50 prowlarr flaresolverr[315582]: 2023-11-20 14:57:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 15:12:50 prowlarr flaresolverr[315582]: 2023-11-20 15:12:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 15:27:50 prowlarr flaresolverr[315582]: 2023-11-20 15:27:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 15:42:50 prowlarr flaresolverr[315582]: 2023-11-20 15:42:50 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 15:44:58 prowlarr flaresolverr[315582]: 2023-11-20 15:44:58 INFO Incoming request => POST /v1 body: {'maxTimeout': 60000, 'cmd': 'request.get', 'url': 'https://www
3.yggtorrent.wtf/engine/search?category=all&do=search&order=desc&sort=publish_date', 'proxy': {}}
Nov 20 15:44:58 prowlarr flaresolverr[315582]: 2023-11-20 15:44:58 ERROR Error: Error solving the challenge. [Errno 24] Too many open files
Nov 20 15:44:58 prowlarr flaresolverr[315582]: 2023-11-20 15:44:58 INFO Response in 0.026 s
Nov 20 15:44:58 prowlarr flaresolverr[315582]: 2023-11-20 15:44:58 INFO 127.0.0.1 POST http://localhost:8191/v1 500 Internal Server Error
Nov 20 15:57:51 prowlarr flaresolverr[315582]: 2023-11-20 15:57:51 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 16:12:51 prowlarr flaresolverr[315582]: 2023-11-20 16:12:51 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Nov 20 16:27:51 prowlarr flaresolverr[315582]: 2023-11-20 16:27:51 INFO 192.168.1.111 GET http://192.168.1.207:8191/ 200 OK
Screenshots
No response
About this issue
- Original URL
- State: open
- Created 7 months ago
- Reactions: 1
- Comments: 44 (13 by maintainers)
Commits related to this issue
- Fix too many open files error. resolves #983 (#1033) — committed to luisriverag/FlareSolverr by ilike2burnthing 6 months ago
After 6 days, not a single failure 😃. It usually started failing after 2, 3 days for me.
I adapted the code as suggested. As this problem only occurs after a few days I’ll give you a feedback when relevant (did the change yesterday).
May have something to do with the usage of kill while multiprocessing was starter so they become orphan
I still have the “too many open files” issue even with the suggested fix.
This is the last time that I received a challenge from Yggtorrent.
2023-12-23 22:54:39 INFO Incoming request => POST /v1 body: {‘maxTimeout’: 120000, ‘cmd’: ‘request.get’, ‘url’: ‘https://www3.yggtorrent.qa/’, ‘proxy’: {}} version_main cannot be converted to an integer 2023-12-23 22:54:42 INFO Challenge detected. Title found: Just a moment… 2023-12-23 22:54:51 INFO Challenge solved! 2023-12-23 22:54:51 INFO Response in 11.945 s
After that I have no mentions of YggTorrent in my logs except those from my manual testing of today (the logs I posted in my previous message). But I explicitily mentioned the Yggtorrent URL so that makes sense.
This is weird as I know for sure that I downloaded content from Ygg the last few days (qbittorrent shows Ygg as a source for one of my file downloaded on the 26th).
Anyway I guess we can simplify the testing. Since the UC will be used even if there is no challenge I can just point it to my prowlarr ping url and do the call 2000 times (my max open file is back to 1024) times and see if I have any problem.
The test is running, I’ll post an update when it’s done.
@chnob hopefully in the next few days if the users above report back that it’s working for them.
If you want to make changes to the container, you can use
docker exec -it --user root flaresolverr /bin/bashor in Portainer click the Exec Console button>_for the FlareSolverr container, enter the user asroot, and connect.You can then enter
vim undetected_chromedriver/dprocess.py, pressi, add a new line afterreader.closeon line 43, addprocess.close(), and pressEsc. Finally, you can type:wqto write (save) and quit, then just disconnect.Restarting the container will now run FlareSolverr with that change.
I also change the code as suggested déc 22 on version 3.3.12 normaly the error occur 4/5 days after container start and never reach 1 week of uptime so i will update you next week if it solve
I updated my open files limit with
Will give feedback in a few days if the problem happens again.