meilisearch: Cannot allocate memory when creating indexes

I’m trying to create 10000+ indexes with 500+ documents and I get this error on a machine with 16GB of ram:

MeiliSearch ApiException: Http Status: 500 - Message: Cannot allocate memory (os error 12) - Error code: internal - Error type: internal_error - Error link: https://docs.meilisearch.com/errors#internal

MeiliSearch version: 0.23.1

Additional context Amazon Linux 2 - x86_64 - 16GB ram - With 10GB of ram free

1,1G data.ms/

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 18 (9 by maintainers)

Commits related to this issue

Merge #3331 3331: Limit the number of concurrently opened indexes r=dureuill a=dureuill # Pull Request ## Related issue Relevant to #1841, fixes #3382 ## What does this PR do? ### User sta... — committed to meilisearch/meilisearch by bors[bot] a year ago

Most upvoted comments

FYI @meilisearch/product-team about this feature request: https://github.com/meilisearch/product/discussions/284

curquiza on Oct 26, 2021

@curquiza I think you mean issue #2616 👍

dmitriid on Jul 18, 2022

Hello @neylsongularte,

A solution you could use is to use only one index where you just add a path field to every document/block of text you have and simply filter for the file you want i.e. path = document_name_1.pdf. This way you will not have this issue anymore as you will have only one index.

Thank you very much for reporting this issue 😃

Kerollmops on Oct 25, 2021