scout: Meilisearch engine pagination total broken

  • Scout Version: 9.4.6
  • Scout Driver: MeiliSearch
  • Laravel Version: 8.83.6
  • PHP Version: 8.0.16
  • Database Driver & Version: Mysql 8.0.28-0ubuntu0.20.04.3
  • SDK Version (If using a third-party service): 0.23.1

Description:

This issue is related with https://github.com/laravel/scout/issues/535 and map id with key name, the solution was https://github.com/laravel/scout/commit/8a7782c5ce72b9407a3cb2e29a4865cf4efc964b creating mapIdsFrom method

The same issue is occurring with the method keys in Engine, does not take into account mapIdsFrom and call directly mapIds, which breaks the pagination count.

A solution is create keysFrom method:

/**
     * Get the results of the query as a Collection of primary keys.
     *
     * @param  \Laravel\Scout\Builder  $builder
     * @param  string  $key
     * @return \Illuminate\Support\Collection
     */
    public function keysFrom(Builder $builder, $key)
    {
        return $this->mapIdsFrom($this->search($builder), $key);
    }

and modify the getTotalCount method:

/**
     * Get the total number of results from the Scout engine, or fallback to query builder.
     *
     * @param  mixed  $results
     * @return int
     */
    protected function getTotalCount($results)
    {
        $engine = $this->engine();

        $totalCount = $engine->getTotalCount($results);

        if (is_null($this->queryCallback)) {
            return $totalCount;
        }

        $ids = $engine->mapIdsFrom(
            $results,
            $key = Str::afterLast($this->model->getScoutKeyName(), '.')
        )->all();

        if (count($ids) < $totalCount) {
            $ids = $engine->keysFrom(tap(clone $this, function ($builder) use ($totalCount) {
                $builder->take(
                    is_null($this->limit) ? $totalCount : min($this->limit, $totalCount)
                );
            }), $key)->all();
        }

        return $this->model->queryScoutModelsByIds(
            $this, $ids
        )->toBase()->getCountForPagination();
    }

Changing $key = Str::afterLast($this->model->getScoutKeyName(), '.') and calling $engine->keysFrom reusing the $key variable

Steps To Reproduce:

Using query() reproduce the bug

return Post::search($search)->query(function($query) { $query->with(‘image’); })->paginate(5);

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (7 by maintainers)

Most upvoted comments

hi @driesvints this is very serious bug in meilisearch driver and could be fix easily.

cc @MatusBoa

Hello @slovenianGooner This is something related to the Meilisearch behavior, and not related to laravel/scout if I’m correct. The Meilisearch support is available to help you on our side 😄 We’ve opened a discussion to explain the behavior of nbHits in more detail: https://github.com/meilisearch/product/discussions/467 Otherwise, for any other questions related to Meilisearch you can open a discussion here or join our slack.

Hope it helps 😄

I am having a similar issue:

// URL: ?page=1
Document::search("e")->paginate(15);
// nbHits = 15

But if I load page 2, the pagination is correct.

// URL: ?page=2
Document::search("e")->paginate(15);
// nbHits = 893

The only difference in the params is that with page 1, we sent “offset” of 0, and with page 2, the “offset” is 15.

@SadeghPM

Found same problem with this. Hours of debugging and the whole problem was that primaryKey was not on the first place in hits (result from meilisearch).

This could be solved somehow with fetching primaryKey from index and using it instead of using first key in the array.

@driesvints

problem still exists. Another problem for pagination count comes from mapIds in MeiliSearchEngine.php. In functions comment the problem described!

    /**
     * Pluck and return the primary keys of the given results.
     *
     * This expects the first item of each search item array to be the primary key.
     */
    public function mapIds($results)
    {

First item of each search result is not primary key.