watchdog: watchmedo does not support ignoring monitoring specific folder and its sub-folders

From the usage of watchmedo auto-restart, we can see the only possible way we can use to exclude a specific folder is using --ignore-pattern parameter, as following shows.

sh-4.4$ watchmedo auto-restart --help
usage: watchmedo auto-restart [-h] [-d directory] [-p PATTERNS] [-i IGNORE_PATTERNS] [-D] [-R] [--interval TIMEOUT] [--signal SIGNAL]
                              [--debug-force-polling] [--kill-after KILL_AFTER]
                              command [arg [arg ...]]

    Subcommand to start a long-running subprocess and restart it
    on matched events.

    :param args:
        Command line argument options.
    

positional arguments:
  command               Long-running command to run in a subprocess.
  arg                   Command arguments. Note: Use -- before the command arguments, otherwise watchmedo will try to interpret them. (default:
                        -)

optional arguments:
  -h, --help            show this help message and exit
  -d directory, --directory directory
                        Directory to watch. Use another -d or --directory option for each directory. (default: -)
  -p PATTERNS, --pattern PATTERNS, --patterns PATTERNS
                        matches event paths with these patterns (separated by ;). (default: '*')
  -i IGNORE_PATTERNS, --ignore-pattern IGNORE_PATTERNS, --ignore-patterns IGNORE_PATTERNS
                        ignores event paths with these patterns (separated by ;). (default: '')
  -D, --ignore-directories
                        ignores events for directories (default: False)
  -R, --recursive       monitors the directories recursively (default: False)
  --interval TIMEOUT, --timeout TIMEOUT
                        use this as the polling interval/blocking timeout (default: 1.0)
  --signal SIGNAL       stop the subprocess with this signal (default SIGINT) (default: 'SIGINT')
  --debug-force-polling
                        [debug] forces polling (default: False)
  --kill-after KILL_AFTER
                        when stopping, kill the subprocess after the specified timeout (default 10) (default: 10.0)

Checking the source code of watchmedo, we can find that watchmedo uses PurePosixPath to match the path and patterns (including ignore-patterns), and it is NOT supporting regex at all, so we cannot use regex to match the pattern.

in the doc for match method of PurePosixPath, can find that it does NOT support recursively searching and matching files in a specific folder and its sub-folders.

Is it possible for watchmedo to support such feature that can ignore any changes for a specific folder and its subfolders recursively? Thanks

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 7
  • Comments: 15 (6 by maintainers)

Most upvoted comments

Thanks for everyone’s help here!

The pathlib module recently introduced the ability to match a path against a provided glob-style pattern through a new method, PurePath.full_match. The only limitation is that this feature is available only in Python 3.13 and later.

Personally, I would prefer leveraging this built-in functionality rather than introducing an additional dependency to the library. One of the strengths of watchdog in its current form is its self-contained nature, free of third-party dependencies. This simplicity is particularly valuable in tightly regulated environments, where every dependency must be carefully reviewed and mirrored.

Would it make sense to dynamically check at runtime if full_match is available and fall back to match() to ensure backward compatibility with Python versions before 3.13 while leveraging the new functionality when possible? Something like this:

def path_matches(pattern, path):
    if hasattr(PurePath, 'full_match'):
        return PurePath(path).full_match(pattern)
    else:
        return PurePath(path).match(pattern)

Once Python 3.13 becomes the minimum required version in a couple of years, this runtime check can be removed for simplicity

@taleinat

It seems that watchmedo auto-restart currently doesn’t support recursive patterns. I believe that for pathlib.PurePath.match(), path/**/* is equivalent to path/*/*, which only takes into account 2 levels of nesting. This is also what happens in my testing: changes deeper than 2 layers are not taken into account by either --patterns or --ignore-patterns. You can confirm this behavior in the python shell:

$ python3
>>> from pathlib import PurePath
>>> PurePath("a/b/c/d.txt").match("b/**/*")
True
>>> PurePath("a/b/c/d/e.txt").match("b/**/*")
False

This means that there is no reliable way to, for example, watch for recursive changes in directory my_project while recursively ignoring anything that happens in my_project/build. My current workaround is to construct ignore patterns programmatically with many layers of nesting i.e. build;build/*;build/*/*;build/*/*/*;[...], which, as you can imagine, not ideal.

I believe this is a real functionality limitation and I’m wondering if it’s worth introducing additional parameters, such as --glob-pattern(s) and --ignore-glob-pattern(s), that would use more of a glob-style pattern instead?

Appreciate your help on this!

For future reference, if someone does implement something like this, it seems it would be better to use gitignore-style path specifications (e.g. via the pathspec library), since fnmatch is aimed at matching file names rather than paths, and has some drawbacks for this use case due to that.

Also, to allow providing multiple path specs via cmdline, it would probably be better to allow specifying --pattern(s) and/or --ignore-pattern(s) multiple times rather than using a separator character as is currently done.