mkdocs-material: Search freeze on first search
Description
When ever we open the page first time and click on search box, it freezes for almost 10 seconds.
Expected behavior
Should be instant like in other themes OR show what it is doing.
Actual behavior
Freezing the whole window
Steps to reproduce the bug
- I’ve around 20 MD files with 5000 lines of text.
- Open main page and click search box
- Wait and enjoy the slow speed.
In this screencast, i pressed button instantly but results were way too late. https://i.imgur.com/KZU3G66.gifv
I changed theme to readthedocs, and tried as well which does not freeze. Let me know what i am missing.
Package versions
- Python:
3.7 - MkDocs:
1.0.4 - Material:
3.0.4
Project configuration
site_name: ................
site_description: .......... Documentation
# site_url:
site_author: '..............'
site_url: 'https://........../docs'
# Copyright
copyright: 'Copyright © 2010 - 2018'
nav:
- Getting Started:
- 'Setup' : 'index.md'
- 'Creating' : 'creating.md'
- Customization:
- 'Selecting Columns' : 'selecting-columns.md'
- 'Column Options' : 'column-options.md'
- 'Options' : 'options.md'
- Features:
- 'Master Detail' : 'master-detail.md'
- 'Sub' : 'sub.md'
- 'Exporting Data' : 'exporting-data.md'
- 'Importing Data' : 'importing-data.md'
- 'Conditional Formatting' : 'conditional-formatting.md'
- 'Grouping' : 'grouping.md'
- 'Grouping Headers' : 'grouping-headers.md'
- 'Search' : 'search.md'
- 'Events' : 'events.md'
- 'Validation' : 'validation.md'
- 'Upload Files' : 'upload-files.md'
- 'Summary' : 'summary.md'
- 'Autocomplete' : 'autocomplete.md'
- Layout & UI:
- 'Responsive Design' : 'responsive-design.md'
- 'Localization' : 'localization.md'
- 'Themes' : 'themes.md'
- Databases:
- 'Connecting with SQL Server' : 'connecting-with-sql-server.md'
- 'Connecting with Postgres' : 'connecting-with-postgres.md'
- 'Connecting with Oracle' : 'connecting-with-oracle.md'
- 'Debug Mode' : 'debug-mode.md'
- FAQs:
- 'General' : 'faq-general.md'
- 'Appearance' : 'faq-appearance.md'
- 'Column Appearance' : 'faq-column-appearance.md'
- 'Column Settings' : 'faq-column-settings.md'
- 'Dialog Customization' : 'faq-dialog-customization.md'
- 'Javascript API' : 'faq-javascript-api.md'
- 'Databases' : 'faq-databases.md'
- 'Master Detail' : 'faq-master-detail.md'
- 'Export' : 'faq-export.md'
- 'Misc' : 'faq-misc.md'
theme:
logo: 'https://.............png'
name: material
favicon: 'css/favicon.gif'
# Customization
extra:
feature:
tabs: true
search:
# tokenizer: '[\s\-\.]+'
language: 'en'
# prebuild_index: true
site_dir: ../docs
docs_dir: md
extra_css:
- extra/extra.css
extra_javascript:
- extra/extra.js
System information
- OS: Win10
- Browser: Chrome Version 70.0.3538.67 (Official Build) (64-bit)
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 27 (23 by maintainers)
This problem has been known for some time and it is, of course, annoying. The delay is actually caused by three factors: (1) the download of the
search_index.json, (2) pre-processing the index data and (3) the actual indexing. Let me elaborate on how we could cut time and what it would cost, annotating the separate steps with the times I measured for the official docs:Process
1. Download (74ms)
Downloading the
search_index.jsonis only done when the search is first focused. The download is network time, so it depends on two main factors, namely the connection quality and the size of the index. The index contains all documents two times: the whole document, as well as all separate subsections of a document. I don’t know why the authors choosed to do that, probably to allow whole-page or sub-section indexing.Material takes advantage of the presence of both, as it groups sub-section results by page, which is a major UX improvement over the default MkDocs search, especially if you have a lot of pages with the same sub-section names (e.g. try to search the term
installationin the official docs). Therefore, size may be a critical factor that impacts search performance. It can be improved with appropriate caching headers and gzip encoding.2. Preprocessing (5ms)
The pre-processing of the index is a Material-specific feature. The Markdown is filtered for better readability within the search results and sections are annotated with their respective page, see:
https://github.com/squidfunk/mkdocs-material/blob/fa8d490b16155003386224cc82e71aafb471c828/src/assets/javascripts/components/Material/Search/Result.jsx#L137-L164
This seems to take only very little time as it is contained in the block prior to the anonymous function passed to
lunr.js.3. Indexing (150ms)
This takes the most time, as
lunr.jshas to tokenize the words and do a lot of string processing operations. Afterlunr.jsis finished, the search is operational.Mitigation
The problem is actually two problems: (1) the building of the index takes too long and (2) it freezes the UI. Both could be mitigated, but it comes at a cost:
1. Improving on indexing time
The indexing time could be improved by pre-building the index which is already possible with
lunr.js. However, as Material does pre-processing for an optimal delivery of search results, this pre-processing must also be carried out during the prebuild. It’s all JavaScript, so it is easy to be done with Node.js.However, this would mean that Material would need to supply custom scripts which must be executed during the actual build of the documentation project which would in turn add a dependency on a functioning Node.js environment and some additional setup. This is something that I really don’t want to do, because it would make things more complicated than just spinning up MkDocs with the packaged Material theme for every user.
Frankly, I don’t really know how we could come up with a way that doesn’t complicated the setup for delivering a prebuilt index. If somebody has an idea, feel free to discuss.
2. Moving search into a separate thread
We could (and should!) leverage web workers to make the search index processing asynchronous. The UI wouldn’t freeze, so the UX would be much better. However, this would probably increase the index building time (as it’s done with sub-priority) which means that we wouldn’t gain much here, either. Neverheless, if somebody wants to go ahead on web worker support, let’s discuss and PR.
Conclusion
As said, it’s a know problem, but I haven’t come up with a one-fits-all solution yet. Note that I elaborated on the way search is carried out for future reference in the form of similar questions.
Thank you! 😃
No, that shouldn’t be. You may open an issue with instructions to reproduce the problem, so we can solve it!
I don’t get the concept of
prebuild_indexoption anyway. If I enable it, the file size of mysearch_index.jsonincreases from 6 MB to 17 MB. So while it may be nice to have the JSON preloaded, if the download size triples, the whole thing doesn’t make much sense to me.So yeah, making the process asynchronous or at least tell the user what’s going on, while keeping the search index file as small as possible, sounds better.
My understanding on this issue (and I may well be wrong 😃 ) The search file is prebuilt. You can see how it works by loading https://squidfunk.github.io/mkdocs-material/ - then opening the Network panel in Chrome Dev Tools, disabling cache and then clicking on the Search input after the page has finished loading. You’ll see that it then downloads: https://squidfunk.github.io/mkdocs-material/search/search_index.json. That particular file is 39KB (gzipped) so is quite fast, but on larger sites it means you need to wait for that file to download - causing a delay. I think the solution lies in that file should ‘preload’ somehow once the page has finished loading (so it doesn’t delay the page rendering). I took a look at trying to resolve this a while ago by adding a preload script but it didn’t work as expected (it preloaded the file as expected/desired but the fetch on clicking the search meant that it downloaded again)
I love the user interface of the MkDocs-Material search with the grouped search results, real-time underlining of the search term in the results and displaying the number of matching results - it’s beautiful and elegant. I just wish that the
search_index.jsonfile could be preloaded somehow (but not sure how to fix).UPDATE: oops… It’d been a while since I looked at this thread and had forgotten that squidfunk has already outlined this issue above.