aws-elastic-beanstalk-cli: .ebignore includes symlinked files in ignored directories

Description

Having specified .ebignore to ignore a whole directory, if that directory contains a symlink, that symlink is not ignored and included in the source bundle that eb cli creates.

Steps to reproduce

With an eb application and environment set up - where the project contains a directory called ignore and, within that, one standard directory dir and one symlinked directory symlink; and the project’s .ebignore file contains an entry to ignore that whole directory from the created artifact - run eb deploy env-name.

.ebignore file:

/bin/ignore/

Observed result

The artifact created and uploaded to S3 will contain symlinked directory ignore/symlink, but not ignore/dir.

Structure of example project eb-bug:

Actual (partial) output from eb deploy env-name --debug shows no entry at all for ./bin/ignore/symlink:

...
2021-03-21 08:28:27,522 (INFO) eb : Found .ebignore, using system zip.
2021-03-21 08:28:27,522 (DEBUG) ebcli.core.fileoperations : Project root found at: /path/2/eb-bug
2021-03-21 08:28:27,522 (INFO) eb : Zipping up folder at location: /path/2/eb-bug
2021-03-21 08:28:27,523 (INFO) eb :  +adding: .//
2021-03-21 08:28:27,523 (INFO) eb :   +adding: ./.envrc
2021-03-21 08:28:27,524 (INFO) eb :   +adding: ./Issue.md
2021-03-21 08:28:27,524 (INFO) eb :   +adding: ./.gitignore
2021-03-21 08:28:27,524 (INFO) eb :   -skipping: ./.ebignore
2021-03-21 08:28:27,524 (INFO) eb :  +adding: ./bin/
2021-03-21 08:28:27,525 (INFO) eb :   +adding: ./bin/cli
2021-03-21 08:28:27,525 (INFO) eb :   -skipping: ./bin/ignore/dir/file
2021-03-21 08:28:27,525 (INFO) eb :   -skipping: ./.elasticbeanstalk
2021-03-21 08:28:27,525 (INFO) eb :   -skipping: ./.elasticbeanstalk/app_versions
2021-03-21 08:28:27,525 (INFO) eb :  +adding: ./.git/
...

Expected result

The artifact created and uploaded to S3 should not contain directory ignore.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macos 11.2.3
EBCLI version: EB CLI 3.19.3 (Python 3.9.2)

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 10
Comments: 21 (5 by maintainers)

Commits related to this issue

Fixed Issue69 and a Unit test bug(#69) Fixed bug where a folder mentioned in .ebignore file was being included in the final code.zip on running eb deploy. This was due to the presence of a symlink wi... — committed to aws/aws-elastic-beanstalk-cli by yohanthshetty 2 years ago

Most upvoted comments

@tomchkk thanks for reporting this issue. We will look into it and post any updates here.

Palakpatel67 on Mar 22, 2021

I have fixed the symlink issue. The symlinks are being treated like just another file/directory. The symlinks aren’t lost either.

yohanthshetty on Sep 28, 2022

any updates on this ?

itelescu on Jun 4, 2023

For anyone facing this issue, I created a branch that does NOT zip symlinks (even when they aren’t ignored). It is a fork from the version 3.20.3.

If you have awsebcli in your requirements file, you can download de patched version directly from git:

(requirements.txt)
...
git+https://github.com/llorenspujol/aws-elastic-beanstalk-cli.git@zip-no-symlinks-v3.20.3#egg=awsebcli
...

Hope this helps!

llorenspujol on Apr 25, 2022

I’ve opened a PR with the fix for that.

vince1995 on Sep 13, 2021

It seems like it wasn’t fixed yet. Contents of .ebignore:

# ignore everything
*
# Don't ignore docker-compose.yml
!./docker-compose.yml

Error when running eb deploy:

ERROR: FileNotFoundError - [Errno 2] No such file or directory: './env/bin/python'

version is latest:

$ eb --version
EB CLI 3.20.6 (Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0])

The culprit:

$ ls-la env/bin/python
lrwxr-xr-x 1 root root 7 Apr 12 13:16 env/bin/python -> python3

andraspatka on Apr 28, 2023

Yes, that’s something I thought of as well, but I wanted to create a solution that could be merged into the master branch because some developers may explicitly want the symlinks in the zip bundle.

vince1995 on Apr 26, 2022

Any updates on this issue? IMO It should be a high priority issue. If one folder is ignored, it should be ignored entirely, included its symlinks to folders.

Without this fix, Elastic Beanstalk app.zip can end with a bunch of symlinks that point to undefined locations. For example, I can have this error on startup:

2022/04/21 09:11:08.715481 [ERROR] An error occurred during execution of command [app-deploy] - [StageApplication]. Stop running the command. Error: chown /var/app/staging/venv/dev/lib/python3.8/collections: no such file or directory

Because ‘collections’ is a symlink to a folder that do not exist in AL2.

All workarounds I am studying are quite painful… so it would be great to fix it.

llorenspujol on Apr 21, 2022

I’ve looked into the code and noticed that here the symlinks are written into the zip without checking if they are in the .ebignore file.

vince1995 on Sep 13, 2021