tinysearch: Error: NotEnoughSpace

This bug should now be fixed in master. I ported tinysearch over to Xor filters.

• 20-25% smaller wasm size: The test index is 99kB now, 49kB gzipped, 40kB brotli. (Keep in mind that this was already highly optimized before.)
• Faster initial compilation time thanks to fewer dependencies.
• No more TINY_MAGIC. The Xor filter doesn’t run into this problem.

Along the way I cleand up the custom serialization code, removed the forked dependencies, which were no longer needed and updated the docs. Oh and I replaced lazy_static! with once_cell! and include_bytes! with the new include_str. All around good update. 😄

If anyone wants to give it a shot. I’ll close this to clean up the issue tracker, but feel free to play with the master branch and report back here.

Thanks for the great work @fluential

Was able to build a search index using magic number 64, end result is

-rw-r--r-- 1 q q 6,7M Jun 11 22:46 index.json
-rw-r--r-- 1 q q 2,7M Jun 19 20:38 tinysearch_engine_bg.wasm
-rw-r--r-- 1 q q 1,1M Jun 19 20:38 tinysearch_engine_bg.wasm.gz

This is including all the content, could cut it down if I used the title only.

Probably good enough for me as I’m building media heavy website.

@quangloc1993 Better create a new issue regarding that

This project is mostly about generating the search database using rust/wasm, there’s only a very basic example of usage.

If you need supporting older browsers, you may need to investigate it yourself…

But let’s start with making new issue 👍

Actually, I think we can use #15 for that.

Sure! Will create a new issue for that.

Going from 26 to 64 the produced files go from 780KB to 868KB.

Hm… 🤔 So these files were generated:

-rw-r--r--  1 kvz kvz      1 Jun 16 16:11 .gitignore
-rw-r--r--  1 kvz kvz    477 Jun 16 16:11 package.json
-rw-r--r--  1 kvz kvz    279 Jun 16 16:11 tinysearch_engine_bg.d.ts
-rw-r--r--  1 kvz kvz 171772 Jun 16 16:11 tinysearch_engine_bg.wasm
-rw-r--r--  1 kvz kvz    889 Jun 16 16:11 tinysearch_engine.d.ts
-rw-r--r--  1 kvz kvz   4560 Jun 16 16:11 tinysearch_engine.js

But not a demo.html so that i could access http://0.0.0.0:8000/demo.html as explained in the README:

Edit: But when I copy that file manually from https://raw.githubusercontent.com/mre/tinysearch/master/bin/assets/demo.html, searches seem to work 💯

I wonder if it’s a good idea to make that value configurable still. Apparently there’s a trade-off for high values, so maybe only people with large indices running into errors should be paying that, while you have a low default that caters to most use cases.

@mre Actually I’ve checked and for nginx to serve gzipped wasm requires extra config for mime types echo 'application/wasm wasm;' >> /etc/nginx/mime.types and adding to gzip_types

via https://rustwasm.github.io/book/reference/deploying-to-production.html

Thanks for hint!