bitmagnet: bitmagnet Web UI slow to return results (approx 2 mins)
- I have checked the existing issues to avoid duplicates
- I have redacted any info hashes and content metadata from any logs or screenshots attached to this issue
Describe the bug
bitmagnet Web UI is slow to display results, i am seeing around 2 minutes waiting times before results are displayed in the Web UI, if i attempt to retrieve results via the bitmagnet torznab api i can do so with results showing very quickly (less than a second) so i do not believe the issue is PostgreSQL related (even though the logs show ‘SLOW SQL’ - see below).
Things i have tried so far:-
- Changed the bitmagnet config.yml to specify a
processor_concurrencyof3, also tried other numbers from 1-3 - this was based off this post. - Used this website to create a optimization file for PostgreSQL based on my hardware specification.
- Tried different browsers (Brave and Chrome), also tried incognito mode to rule out any extensions causing issues.
- Reindex database
- Performed Vacuum on database
Logs are showing the following warnings:-
2025-03-13 09:15:27,406 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120013.445885, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''comic''', 5000)", "rows": -1}
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120010.48843, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''game''', 5000)", "rows": -1}
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/query/query.go:176", "slowLog": "SLOW SQL >= 30s", "elapsed": 120013.426855, "sql": "SELECT * FROM \"content\" WHERE (\"content\".\"type\" = 'movie' AND \"content\".\"source\" = 'tmdb' AND \"content\".\"id\" IN ('1064486','385332','498210','296098')) OR (\"content\".\"type\" = 'tv_show' AND \"content\".\"source\" = 'tmdb' AND \"content\".\"id\" IN ('58188','131033','81241','93290','154827','205242')) ", "rows": 10}
2025-03-13 09:15:27,406 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120009.807258, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" IS NULL', 5000)", "rows": -1}
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120013.326415, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''xxx''', 5000)", "rows": -1}
2025-03-13 09:15:27,415 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120026.100527, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''movie''', 5000)", "rows": -1}
2025-03-13 09:15:27,416 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120026.939889, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''ebook''', 5000)", "rows": -1}
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120026.980549, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''tv_show''', 5000)", "rows": -1}
2025-03-13 09:15:27,420 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120030.819835, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''software''', 5000)", "rows": -1}
2025-03-13 09:15:27,600 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120008.797867, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" IS NULL', 5000)", "rows": -1}
2025-03-13 09:15:27,601 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120010.047139, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''xxx''', 5000)", "rows": -1}
2025-03-13 09:15:27,603 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120011.836532, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''audiobook''', 5000)", "rows": -1}
2025-03-13 09:15:27,603 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120012.683674, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''game''', 5000)", "rows": -1}
2025-03-13 09:15:27,604 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120012.990585, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''comic''', 5000)", "rows": -1}
2025-03-13 09:15:27,605 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/query/query.go:176", "slowLog": "SLOW SQL >= 30s", "elapsed": 120012.899384, "sql": "SELECT *, \"torrent_contents\".\"published_at\" AS _order_0, \"torrent_contents\".\"info_hash\" AS _order_1 FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = 'movie' ORDER BY \"_order_0\" DESC,\"_order_1\" DESC LIMIT 21", "rows": 21}
2025-03-13 09:15:27,619 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120028.276321, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''movie''', 5000)", "rows": -1}
2025-03-13 09:15:27,620 DEBG 'start' stdout output:
[33mWARN[0m gorm logger/logger.go:76 gorm trace {"location": "/tmp/bitmagnet/internal/database/dao/budgeted_count.go:24", "slowLog": "SLOW SQL >= 30s", "elapsed": 120029.050612, "sql": "SELECT count, cost, budget_exceeded from budgeted_count('SELECT * FROM \"torrent_contents\" WHERE \"torrent_contents\".\"content_type\" = ''music''', 5000)", "rows": -1}
To Reproduce
Steps to reproduce the behavior:
- Ensure you have at least 5 million torrents in your database.
- Start bitmagnet and attempt to go to the web ui - notice the web ui does not load and takes around 2 minutes before results are returned.
- Notice warning messages in the logs.
Expected behavior
- Ensure you have at least 5 million torrents in your database.
- Start bitmagnet and the web ui returns results within a few seconds.
- Check log and see no warning messages.
Environment Information (Required)
- Bitmagnet version:
0.10.0 - OS and version:
Linux - Slackware running Dockerversion 27.0.3, build 7d4bcd8` - Browser and version (if issue is with WebUI):
Brave [Version 1.76.74 Chromium: 134.0.6998.89] - Please specify any config values for which you have overridden the defaults:
processor.concurrency - tried values from 1-3
Additional context
This is a Docker container running bitmagnet with the SQLPostgres instance running inside of the same container, i can see from the warnings in the logs that its pointing at the database being the issue, but as mentioned earlier i can query bitmagnet via the torznab api and get results straight away, even if the Web UI is in a waiting to return results state.
One other important thing to note is that once the Web UI has loaded and results are shown if i press the refresh button on the browser the page loads instantly, and i can also then change ‘Content Type’ to another type then again the results are loaded nearly instantly, if i then wait a period of time (not sure how long exactly, say 1 hour?) and then navigate to the Web UI again and his refresh then we are back to the long wait before anything is returned.
One last thing to mention is startup times, bitmagnet Web UI takes around 3 minutes before i can access the Web UI on a fresh startup.
Spec of my machine:- CPU:- AMD 3900X (12 core) RAM:- 2 x Corsair CMK32GX4M2A2666C16 (64GB) Storage:- Samsung 970 EVO Plus 1 TB (NVME) (database stored on this)
Possibly related?:- https://github.com/bitmagnet-io/bitmagnet/issues/390#issuecomment-2676369406
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18
OK due to other issues i am seeing with UNRAID v7.0.1 i decided to downgrade back to 6.12.15, this instantly fixes the issue with slow Web UI for bitmagnet (and other issues i am seeing), so as this issue is not due to bitmagnet changes i am closing the issue, i shall take my findings and post a bug report on UNRAID forum.