kaminari: paginate calls slow COUNT query for large sets (when it might not need to)
I’m experiencing slow performance issues on my app using the kaminari paginate command, and wondering if there might be a way to avoid it.
Here’s the relevant trace:
/kaminari-0.15.1/lib/kaminari/models/active_record_relation_methods.rb:24:in `total_count'
/kaminari-0.15.1/lib/kaminari/models/page_scope_methods.rb:22:in `total_pages'
/kaminari-0.15.1/lib/kaminari/helpers/action_view_extension.rb:18:in `paginate'
And here’s the SQL query, which in my case is related to a Spree backend view:
(209.9ms) SELECT DISTINCT COUNT(DISTINCT `spree_orders`.`id`) FROM `spree_orders` WHERE (`spree_orders`.`completed_at` IS NOT NULL)
It seems that kaminari needs to know the full count from this table (which in my case is circa 250,000) in order to generate the “LAST” link.
What I’m thinking is, perhaps there’s a way to pass ?page=last into kaminari and have the last page’s number computed only then. If that were the case, we would only need to know whether the count exceeded the number of items per page * the number of pages. Presumably that would be a much faster SQL query than getting the total number each time we run the query.
I’d be interested to know what the community thinks. This would be a huge performance boost for my application, and likely for anyone else working with large data sets.
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 28 (5 by maintainers)
Commits related to this issue
- Enable lazy evaluation of paginate options ref: https://github.com/amatsuda/kaminari/issues/545#issuecomment-40663427 — committed to mizoR/kaminari by deleted user 10 years ago
Solution!
Pagination without count is now available: https://github.com/amatsuda/kaminari/pull/681#issuecomment-258727907 Give it a shot!
@sg552 repo moved New URL should be https://github.com/kaminari/kaminari/pull/681#issuecomment-258727907
There is solution/workaround suggested in another issue: https://github.com/amatsuda/kaminari/pull/818#issuecomment-252788488
Optimize paginate COUNT query for large sets
the example code as below:
Find
Search
+1 or just saying “it doesn’t work” don’t solve the issue. Any contribution is appreciated.
+1 for this, it doesn’t seem to work