meilisearch: Meilisearch intermittently becomes incapable of handling new requests after index creation timeout

Describe the bug I have an initialization service that creates indexes and updates documents. Sometimes this hangs when creating an index after which point all subsequent requests fail.

To Reproduce Create multiple indexes while updating bulk documents

Expected behavior All requests succeed

Additional context Index has ranking rules:

[
      "typo",
      "words",
      "proximity",
      "attribute",
      "wordsPosition",
      "exactness",
      "desc(score)"
]

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 2
Comments: 20 (9 by maintainers)

Most upvoted comments

Hello @Tarpsvo, @ssankko, @roelvanmeer, @JayDijkstra, @Bajix and everyone following the issue!

The first RC of MeiliSearch v0.21.0 is out. We did our best to fix indexation and crash issues. We succeeded to improve it, but not to totally fix them.

You can test this new release by downloading the binaries available in this release. Or you can use it with docker:

docker run -p 7700:7700 getmeili/meilisearch:v0.21.0rc1 ./meilisearch

We will still improve this after the release of the v0.21.0. We would rather release a non-completely optimized version rather than delay it and, at the same time, delay the release of new features. Be sure we are doing our best to always improve these indexation issues.

As a reminder:

despite the improvements, we always recommend pushing your documents to MeiliSearch by batch and not one by one. The maximum payload size of MeiliSearch is 100Mb and can be optionally increased. It means most of the dataset can be pushed with one call to the API.
If you still have a memory crash, it means the RAM of your machine might not be adapted to your dataset size. We recommend increasing the RAM of your machine.

Thanks for your patience and your help with this! ❤️

curquiza on Jul 1, 2021

Hello @curquiza! I’m sorry, I can’t provide a dataset for a reason. We were testing different search engines to replace our dependency on Algolia. So far we used our production dataset which is private.

Though, we conducted some extensive testing to find the cause. Looks like the engine works alright (without deadlocks) if you don’t create an index while another one is being created. Basically, if meilisearch is indexing documents, we can’t create another index, or system will block itself. This is very unfortunate tbh, so I’m looking forward to see what kind of a technical miracle the new engine will become with the next release.

Also, to further aid the investigation, I’ll add some details about a machine we are running meilisearch on. We are using the Compute Engine by Google Cloud. Machine is categorized by Google as n1-standard-2 with 2 vCPUs and 7.5 Gb of RAM. Machine is running Container Optimized OS with meilisearch image from Docker Hub. As for storage, there is one persistent hard drive with 100 Gb capacity. It’s not a SSD, just the “Standard” one Google provides.

ssankko on May 13, 2021

Here’s the log. I’m unsure if it’s any use, though. Meilisearch works, and then just stops. There are no additional messages in the log when this happens, but the request hangs. Using strace on the meilisearch process shows it is waiting for epoll_pwait() to finish. If I can provide you with anything else, please let me know, thanks!

meilisearch.log

roelvanmeer on Aug 6, 2020