mkdocs: Error reporting from plugins, and log.error

I’ve been wondering how to properly report warnings and errors that happen during operation of my plugin.

I found (not through documentation) that perhaps it should be this:

https://github.com/mkdocs/mkdocs/blob/44f3ae212d85a60750769df0e9fa8022274166c8/mkdocs/structure/files.py#L7-L11 https://github.com/mkdocs/mkdocs/blob/44f3ae212d85a60750769df0e9fa8022274166c8/mkdocs/structure/files.py#L241

(with the very subtle issue that mkdocs configures only mkdocs.* loggers but the plugin would not fall under that and would hence not follow the --verbose option and such.)

What’s nice is that warnings automatically cause the generation to fail in MkDocs strict mode.

But I was surprised that log.error does not cause a failure at all even in strict mode (so I guess it looks for warning specifically), though I could argue that it should cause a failure even in non-strict mode. But I guess it’s controversial to change the default behavior, and there’s something to be said about errors causing an immediate crash versus being able to wait until the end and then cause a failure.

MkDocs itself seems to always employ a combination of log.error + raise (sometimes raise SystemExit). But I don’t think a plugin can just raise anything like that, it’ll cause a stack trace to show up, which MkDocs makes sure to avoid in its own handling, but a plugin can’t prevent that, other than intentionally raising SystemExit, which is not great. It’s also impossible to implement an error that should not be allowed to go unnoticed but also doesn’t have to cause an immediate crash (config validation errors are the one example where that does work).

So to sum up, the use case of “log an error now and make sure to fail at the end” seems impossible to do from a plugin, and the use case of “log an error now and fail now without a stack trace” is kinda ugly. And either way, error reporting for plugins is under-documented. All of the nuance above is certainly not obvious.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 2
Comments: 27 (27 by maintainers)

Commits related to this issue

Add plugin error exception and event Related to #2082. — committed to mkdocs/mkdocs by pawamoy 4 years ago
Use the logging library This makes the plugin use python's standard logging system and mkdocs' configuration of that, as done in mkdocs/mkdocs#2082, fralau/mkdocs-mermaid2-plugin#18 and fralau/mkdocs... — committed to joelhed/mkdocs-autolinks-plugin by joelhed 3 years ago
Refactorings and optimizations (#16) * Rename variables The variables that were named *_url did not actually refer to values that were URLs (Universal Resource Locator, ie. "http://example.com/")... — committed to zachhannum/mkdocs-autolinks-plugin by joelhed 3 years ago

Most upvoted comments

A PluginError might make more sense. That way it would be an error unique to plugins. After all, MkDocs itself doesn’t need any special exception types (except for the existing config error) and the whole purpose is to address the needs of plugins specifically. And the reraise attribute gives the plugin dev some control. I’m thinking the default for reraise should be False though. I realize you make prefer True but most plugins don’t need that. At least there hasn’t been a high demand for it.

I’d be happy to review a PR.

waylan on May 9, 2020

but not at the cost of debugging

Note that I’m not suggesting that MkDocs itself use that error. It is exactly designed for this particular use-case: a plugin developer wants to abort the build without printing the traceback, because the traceback is not useful. There’s not even debugging to do: the error was handled properly.

Also, only BuildError is caught in this snippet. Every other exception bubbles up and is caught, logged, and re-raised as usual.

pawamoy on Apr 29, 2020

Alright, here’s my concrete example 😁

My plugin renders Jinja templates. Users can override these templates with custom ones. If they try to {% include a template that does not exist, Jinja will raise a jinja2.exceptions.TemplateNotFound. while rendering them. In that case, I catch the exception and log an error. I don’t let the exception bubble up because its stack trace is not useful (it does not contain the directory listing of the Jinja environment loader), instead I log a single error line. Like you said @waylan, there’s no point in continuing to build: it should abort right now and fail. But for this, the only way is to raise an error, which I avoided in the first place.

My plugin also uses its own Markdown extension. The extension injects the HTML rendered by Jinja back into the XML tree with xml.etree.ElementTree.XML. If the HTML cannot be parsed correctly, XML raises a ParseError. If it happens, I catch the exception and log an error. Just like before, I don’t let it bubble up, because the stack trace is not useful: it doesn’t contain the text that XML failed to parse. In the error I log, I show the line and character that caused a parse error. It actually happened a few days ago: markdown-katex was not properly closing a link tag, causing XML to fail.

I admit these are cosmetic details: I want the build to fail and I don’t want to print a traceback for it. But it also happens that raising an exception will prevent MkDocs from running my on_post_build hook, which I need in any case to terminate opened background processes and invalidate caches. This is especially required when serving the site with livereload and watched directories.

To summarize, the current behavior is:

log.warning: build continues ✓ on_post_build runs ✓ build fails ✓
log.error without raise: build doesn’t stop ✗ on_post_build runs ✓ build doesn’t fail ✗
log.error with raise: build stops ✓ on_post_build doesn’t run ✗ build fails ✓ (cosmetic details: extra error line and noisy traceback ✗)

Of course, I’m not saying MkDocs should never catch and re-raise exceptions: it is extremely useful for unexpected errors.

What I need 🙏 :

log.warning: build continues ✓ on_post_build runs ✓ build fails ✓
(log.error: build continues ✓ on_post_build runs ✓ build fails ✓)
current behavior + error_filter
log.error + way to abort the build: build stops ✓ on_post_build runs ✓ build fails ✓

Finally, I realize I’m asking for several things that should maybe be discussed separately, though I think they are closely related. Also, @oprypin, @waylan, I hope I’m not hijacking this issue. If you feel so, I’ll gladly move this comment to the one I opened a few days ago, or open a new issue on your advice @waylan 🙂

pawamoy on Apr 27, 2020

Currently, to get an error you raise one. This means that the error happens immediately

Yes but then a stack trace output is unavoidable if coming from a plugin, while MkDocs has its private ways to prevent a stack trace.

Can we introduce an exception type that plugins would use and MkDocs would catch and silence?

oprypin on Apr 27, 2020

Yes, because mkdocs is being configured in main and that affects everything under it. https://github.com/mkdocs/mkdocs/blob/3bada392d2430e31edd55d67b59d60899c22ae11/mkdocs/__main__.py#L25-L26

To clarify, these namespaces should represent the actual module name, and someone’s plugin is not actually part of mkdocs namespace, so this is not a best practice still.

oprypin on Apr 26, 2020