prefect: GitHub storage block downloads read-only `.git` files, causing permission errors for `shutil.copytree`

First check

I added a descriptive title to this issue.
I used the GitHub search to find a similar issue and didn’t find it.
I searched the Prefect documentation for this issue.
I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

Running a workflow from GitHub storage changes the working directory permissions of the flow from rwxrwxr-x to rwx------. When the flow is executed a second time the code can not be copied into this directory anymore.

Reproduction

Configure a GitHub storage block and a Process infrastructure block with a configured working_dir.

Deploy a flow and run it twice. After the first run the permissions of the working directory are changed. The second flow run will fail during flow retrieval from GitHub storage.

Error

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/engine.py", line 255, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/deployments.py", line 163, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/filesystems.py", line 917, in get_directory
    copytree(src=content_source, dst=content_destination, dirs_exist_ok=True)
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/shutil.py", line 568, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/shutil.py", line 522, in _copytree
    raise Error(errors)
shutil.Error: [('/tmp/tmpjf2bug7lprefect/.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', "[Errno 13] Permission denied: './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', "[Errno 13] Permission denied: './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', "[Errno 13] Permission denied: './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', "[Errno 13] Permission denied: './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', "[Errno 13] Permission denied: './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', "[Errno 13] Permission denied: './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', "[Errno 13] Permission denied: './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/9d/10168d3633499e25522a57c8109767f0819735', './.git/objects/9d/10168d3633499e25522a57c8109767f0819735', "[Errno 13] Permission denied: './.git/objects/9d/10168d3633499e25522a57c8109767f0819735'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', "[Errno 13] Permission denied: './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', "[Errno 13] Permission denied: './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', "[Errno 13] Permission denied: './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c', './.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c', "[Errno 13] Permission denied: './.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f', './.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f', "[Errno 13] Permission denied: './.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', "[Errno 13] Permission denied: './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', "[Errno 13] Permission denied: './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', "[Errno 13] Permission denied: './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/be/e211a193292471d39e45698d228ee54039d802', './.git/objects/be/e211a193292471d39e45698d228ee54039d802', "[Errno 13] Permission denied: './.git/objects/be/e211a193292471d39e45698d228ee54039d802'")]

Versions

Python 3.9.13 and Prefect 2.6.9.

Additional context

This error happens since #7441. Changing this import to from distutils.dir_util import copy_tree as copytree and changing L917 to copytree(src=content_source, dst=content_destination) fixes the issue.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 15 (9 by maintainers)

Most upvoted comments

@peytonrunyan I was able to test this and it works like a charm 🎆

tibuch on Jan 19, 2023

Alright @tibuch I’m back with instructions. The solution that we have adds a new parameter to the GitHub block: include_git_objects. You’ll want to create a new GitHub block with include_git_objects=False.

IMPORTANT: When you go to register a block with a new schema, it will force your UI to show the latest version of the block. It’s a pain to undo. I’m still testing this code and would hate for you to have an experimental version of it stuck on your production UI if for whatever reason we change it. So the best way to test this is going to be to use a local orion instance with the sqlite database for testing, not your cloud account. That way there’s no harm if you delete the sqlite database after testing.

Once you have that setup, try registering a new GitHub block and using that in your deployment

g = GitHub(
    repository=<your_repo>,
    ...
    include_git_objects=False,
)

I believe that this should work.

peytonrunyan on Jan 18, 2023