meilisearch: Cannot build MeiliSearch v0.19.0 as NixOS package

Describe the bug I tried to use MeiliSearch on my NixOS server today, but noticed the official repos only have v0.9, so I tried to update it. I took the build script from the official repository and upped the version and fixed the hash of the source code. The build process stops with an error as seen below.

To Reproduce Steps to reproduce the behavior:

  1. Run NixOS 20.09
  2. Download this Nix expression: https://gist.github.com/michcioperz/4ecdeaf7921d63784eedf7197eafcb71
  3. Run nix-build meilisearch.nix (the filename you saved to)
  4. An error like this should show up during the vendoring phase:
error: failed to sync

Caused by:
  found duplicate version of package `pest v2.1.3` vendored from two sources:

        source 1: https://github.com/pest-parser/pest.git?rev=51fd1d49f1041f7839975664ef71fe15c7dcaf67#51fd1d49
        source 2: registry `https://github.com/rust-lang/crates.io-index`
Traceback (most recent call last):
  File "/nix/store/m6x1vskdlgsray23j2m2k5j7xzgcjhry-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 42, in <module>
    main()
  File "/nix/store/m6x1vskdlgsray23j2m2k5j7xzgcjhry-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 17, in main
    assert list(data.keys()) == ["source"]
AssertionError
builder for '/nix/store/vm7f4lngx8ar6qbh0n1snrwrscldwab8-meilisearch-0.19.0-vendor.tar.gz.drv' failed with exit code 1
cannot build derivation '/nix/store/cjxkk5frkqdikx6rzvyjlbviz2mlqd99-meilisearch-0.19.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cjxkk5frkqdikx6rzvyjlbviz2mlqd99-meilisearch-0.19.0.drv' failed

Expected behavior The vendoring process should finish and Nix should complain that the hash of the vendor directory is different from what I told it to expect.

Desktop:

  • OS: NixOS 20.09

Additional context NixOS is really into reproducible builds and pinning hashes of all things that are downloaded, so the first part of the process of building a package of something in Rust is vendoring the dependencies and checking that the hash of vendor directory matches what was given in the build script. Vendoring doesn’t complete successfully, because there are two colliding definitions of pest 2.1.3 package: a crates.io version depended on by pest_derive and a few other pest_* crates, and a git version depended on by meilisearch directly. If I understand correctly the problem would go away if pest had a new release on crates.io so you wouldn’t have to use a git version, but its maintainer is busy as seen in https://github.com/pest-parser/pest/issues/491 and https://github.com/pest-parser/pest/pull/454

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 19 (14 by maintainers)

Commits related to this issue

Most upvoted comments

The invitation to the Pest Discord in the issue about the Pest working group has expired, so I can’t see if there have been any updates on the Pest situation.

But I just realized this could be worked around a bit differently https://github.com/michcioperz/MeiliSearch/commit/68e4a52e2ffe3a01c2628f41c9ef04c8c31fcd9e Instead of only changing the version of pest used by meilisearch-core, I’m overriding the version of pest used in the whole dependency tree. This includes the version used by pest_derive.

I tried running a build now locally to see if it solves my issue, but I ran out of disk space at the very end, so I’m gonna try again on a different machine in the morning (it’s pretty late now and I don’t wanna wake my neighbours up with a rack server, haha). At the very least, vendoring runs successfully with this patch, and the build process continues.

I’ve just made a PR to try to integrate the fix you proposed https://github.com/meilisearch/MeiliSearch/pull/1659

meilisearch has been integrated into nixpkgs, and after the proposed fixes have been merged to meilisearch, I’ve reverted the build system to a more traditional one. (see PR that also updates to 0.23.1)

I think we can safely close this issue. Let me know if anything doesn’t make sense.