scout-extended: Aggregators broken since laravel/scout was updated to `v9.1.0` when deleting Models became queueable

After a few hours of investigation, we finally figured out that this PR over at laravel/scout, broke our production setup when trying to remove Models from Algolia using an Aggregator.

The following exception is thrown…

[Algolia\ScoutExtended\Exceptions\ModelNotDefinedInAggregatorException] Model not defined in aggregator.

Here’s the stack trace…

#0 /home/vendor/algolia/scout-extended/src/Searchable/ObjectIdEncrypter.php(42): Algolia\ScoutExtended\Searchable\Aggregator->getScoutKey()
#1 /home/vendor/algolia/scout-extended/src/Jobs/DeleteJob.php(58): Algolia\ScoutExtended\Searchable\ObjectIdEncrypter::encrypt(Object(App\Search\KeywordSearch))
#2 [internal function]: Algolia\ScoutExtended\Jobs\DeleteJob->Algolia\ScoutExtended\Jobs\{closure}(Object(App\Search\KeywordSearch), 0)
#3 /home/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(642): array_map(Object(Closure), Array, Array)
#4 /home/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Collection.php(348): Illuminate\Support\Collection->map(Object(Closure))
#5 /home/vendor/algolia/scout-extended/src/Jobs/DeleteJob.php(59): Illuminate\Database\Eloquent\Collection->map(Object(Closure))
#6 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Algolia\ScoutExtended\Jobs\DeleteJob->handle(Object(Algolia\AlgoliaSearch\SearchClient))
#7 /home/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#8 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#9 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#10 /home/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#11 /home/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call(Array)
#12 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(Algolia\ScoutExtended\Jobs\DeleteJob))
#13 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Algolia\ScoutExtended\Jobs\DeleteJob))
#14 /home/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#15 /home/vendor/laravel/framework/src/Illuminate/Foundation/helpers.php(421): Illuminate\Bus\Dispatcher->dispatchNow(Object(Algolia\ScoutExtended\Jobs\DeleteJob), false)
#16 /home/vendor/algolia/scout-extended/src/Engines/AlgoliaEngine.php(78): dispatch_now(Object(Algolia\ScoutExtended\Jobs\DeleteJob))
#17 /home/vendor/laravel/scout/src/Jobs/RemoveFromSearch.php(41): Algolia\ScoutExtended\Engines\AlgoliaEngine->delete(Object(Illuminate\Database\Eloquent\Collection))
#18 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Laravel\Scout\Jobs\RemoveFromSearch->handle()
#19 /home/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#20 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#21 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#22 /home/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#23 /home/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call(Array)
#24 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(Laravel\Scout\Jobs\RemoveFromSearch))
#25 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Laravel\Scout\Jobs\RemoveFromSearch))
#26 /home/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#27 /home/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(120): Illuminate\Bus\Dispatcher->dispatchNow(Object(Laravel\Scout\Jobs\RemoveFromSearch), false)
#28 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}(Object(Laravel\Scout\Jobs\RemoveFromSearch))
#29 /home/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Laravel\Scout\Jobs\RemoveFromSearch))
#30 /home/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#31 /home/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\Queue\Jobs\RedisJob), Object(Laravel\Scout\Jobs\RemoveFromSearch))
#32 /home/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(98): Illuminate\Queue\CallQueuedHandler->call(Object(Illuminate\Queue\Jobs\RedisJob), Array)
#33 /home/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(428): Illuminate\Queue\Jobs\Job->fire()
#34 /home/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(378): Illuminate\Queue\Worker->process('redis', Object(Illuminate\Queue\Jobs\RedisJob), Object(Illuminate\Queue\WorkerOptions))
#35 /home/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(172): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\RedisJob), 'redis', Object(Illuminate\Queue\WorkerOptions))
#36 /home/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(117): Illuminate\Queue\Worker->daemon('redis', 'redis-low', Object(Illuminate\Queue\WorkerOptions))
#37 /home/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(101): Illuminate\Queue\Console\WorkCommand->runWorker('redis', 'redis-low')
#38 /home/vendor/laravel/horizon/src/Console/WorkCommand.php(51): Illuminate\Queue\Console\WorkCommand->handle()
#39 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Laravel\Horizon\Console\WorkCommand->handle()
#40 /home/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#41 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#42 /home/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#43 /home/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#44 /home/vendor/laravel/framework/src/Illuminate/Console/Command.php(136): Illuminate\Container\Container->call(Array)
#45 /home/vendor/symfony/console/Command/Command.php(288): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#46 /home/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#47 /home/vendor/symfony/console/Application.php(974): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#48 /home/vendor/symfony/console/Application.php(291): Symfony\Component\Console\Application->doRunCommand(Object(Laravel\Horizon\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#49 /home/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#50 /home/vendor/laravel/framework/src/Illuminate/Console/Application.php(92): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#51 /home/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#52 /home/artisan(37): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#53 {main}

If you need any more information to figure this out, let me know.

Thanks

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 16 (6 by maintainers)

Most upvoted comments

We did dig into this, but ultimately hit a brick wall coming up with a solution.

I will try and put together a repo that recreates the error this week if I get a chance.

👍

GC-Mark on Jul 12, 2021

@GC-Mark Depending on your use case, you can just disable queuing for scout. The PR disables queuing for delete anyway. Alternatively it should also work if you just copy and paste the method from the PR into your searchable classes if you prefer to keep queueing for updates.

    public function queueRemoveFromSearch($models)
    {
        if ($models->isEmpty()) {
            return;
        }

        $models->first()->searchableUsing()->delete($models);
    }

I think there may be a better solution than the proposed PR. We could set the Scout::$removeFromSearchJob to a custom Job which can detect whether it’s been handed a normal searchable or an aggregate and behave accordingly. Or continue with overriding the queueRemoveFromSearch method and dispatch to a removeAggregateFromSearchJob. I haven’t looked in detail but the Laravel\Scout\Jobs\MakeSearchable seems to work with Aggregates just fine, so it shouldn’t be too hard.

goldmerc on Feb 11, 2022