mkdocs-material: Optimize plugin fails with "No such file or directory" in .cache directory

Context

I recently tried adding the optimize plugin to work with a variety of PNGs I use on https://distant.dev/. It appears that I am hitting the issue described in #5098, so taking a stab at reporting it.

I’m using commit fa5bd3f45024328d973977db80cccd5b1df89228 of Insiders.

Bug description

Running mkdocs build or mkdocs serve with the optimize plugin configured within mkdocs.yml results in a FileNotFoundError.

❯ mkdocs build
INFO     -  [macros] - Macros arguments: {'module_name': 'macros', 'modules': [], 'render_by_default': True, 'include_dir': '',
            'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '',
            'on_undefined': 'strict', 'on_error_fail': True, 'verbose': False}
INFO     -  [macros] - Found local Python module 'macros' in: /Users/senkwich/projects/distant.dev
INFO     -  [macros] - Found external Python module 'macros' in: /Users/senkwich/projects/distant.dev
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /Users/senkwich/projects/distant.dev/site

  Optimizations:
Traceback (most recent call last):
  File "/Users/senkwich/Library/Python/3.9/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/mkdocs/__main__.py", line 250, in build_command
    build.build(cfg, dirty=not clean)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/mkdocs/commands/build.py", line 332, in build
    config.plugins.run_event('post_build', config=config)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/mkdocs/plugins.py", line 522, in run_event
    result = method(**kwargs)
  File "/Users/senkwich/Library/Python/3.9/lib/python/site-packages/material/plugins/optimize/plugin.py", line 113, in on_post_build
    size_opt += os.path.getsize(target)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '.cache/plugin/optimize/assets/favicon.png'

Related links

Reproduction

example.zip

I’ve removed everything but a blank index.md page as it appears unrelated to which images are actual referenced. Additionally, I cleared out the majority of images and other assets alongside all css and javascript.

mkdocs.yml

nav:
  - 'index.md'

theme:
  name: material
  favicon: assets/favicon.png
  logo: assets/logo.svg

plugins:
  - optimize

Steps to reproduce

  1. Create a couple of PNGs in the assets directory
    • For me, a minimal layout is
      assets/
          favicon.png
          logo.svg
          images/
              architecture/
                  picture.png
      
  2. Attempt to run mkdocs build or mkdocs serve
  3. Observe error about a file not existing in the .cache directory
    • In my case, there is a path to .cache/plugin/optimize/assets/images/architecture without anything inside, nor anything else in the parent directories.

Browser

No response

Before submitting

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 25 (14 by maintainers)

Most upvoted comments

@squidfunk Can confirm this looks fixed for me. Added a .jpeg file back to the project, pulling from latest insiders, and it is now correctly optimized and successfully builds. Thank you for the super fast response as always!

Released as part of 9.1.18+insiders-4.37.0 – I’m sufficiently confident that this should fix all reported problems, i.e., instruct the user to install pngquant and pillow, if they’re not available.

Perfect, thanks for testing! We use pngquant directly, because pip install pngquant will basically only install a thin Python wrapper that doesn’t add much value, and you would need to install pngquant on your system anyway. Thus, we decided to use subprocess to run it directly. Let’s leave this open until it is released.

I didn’t install your latest, but I did add pngquant via brew to the pipeline and successfully built.

Thanks for testing! Perfect, sounds like we found the issue! The latest commit only improves error reporting and checks if pngquant is installed, the logic itself is not changed (only refactored). Thus, I think, we can consider this resolved.

My gut tells me it might be a race condition. Does the error occur when you set concurrency to 1?

plugins:
 - optimize:
     concurrency: 1

Additionally, please check if https://github.com/squidfunk/mkdocs-material/issues/5098#issuecomment-1493485296 fixes it.

@squidfunk setting concurrency: 1 for optimize did not seem to impact the issue, but print_gain_summary: false enabled it to succeed. I’m not seeing any actual images in .cache though. Is that to be expected?

If it is a race condition, my machine hits it consistently.


I also tried https://github.com/squidfunk/mkdocs-material-insiders/commit/647614b2710e544009555fa3a7c358aa712bf5c4 and using concurrency: 1 yielded same error and print_gain_summary: false continued to succeed with nothing in the cache.