dvc: unexpected error for aws training machine

Bug Report

dvc pull and reproduce raise a unexpected error on aws gpu training machine

I use gitlab on aws g3xlarge and try to dvc pull data from gdrive

$ dvc version 1.10.0 $ git version 2.17.1

$ dvc repro
ERROR: unexpected error - expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/main.py", line 89, in main
    cmd = args.func(args)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/command/base.py", line 40, in __init__
    self.repo = Repo(uninitialized=self.UNINITIALIZED)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 177, in __init__
    self._ignore()
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 256, in _ignore
    self.scm.ignore_list(flist)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/base.py", line 81, in ignore_list
    return [self.ignore(path) for path in p_list]
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/base.py", line 81, in <listcomp>
    return [self.ignore(path) for path in p_list]
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/git.py", line 195, in ignore
    entry, gitignore = self._get_gitignore(path)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/git.py", line 181, in _get_gitignore
    if not path_isin(os.path.realpath(gitignore), self.root_dir):
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/utils/fs.py", line 156, in path_isin
    parent = os.path.join(normalize_path(parent), "")
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/utils/fs.py", line 154, in normalize_path
    return os.path.normpath(path)
  File "/home/ubuntu/anaconda3/lib/python3.7/posixpath.py", line 340, in normpath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/bin/dvc", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/main.py", line 119, in main
    dvc_info = get_dvc_info()
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/info.py", line 37, in get_dvc_info
    repo = Repo()
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 177, in __init__
    self._ignore()
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 256, in _ignore
    self.scm.ignore_list(flist)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/base.py", line 81, in ignore_list
    return [self.ignore(path) for path in p_list]
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/base.py", line 81, in <listcomp>
    return [self.ignore(path) for path in p_list]
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/git.py", line 195, in ignore
    entry, gitignore = self._get_gitignore(path)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/scm/git.py", line 181, in _get_gitignore
    if not path_isin(os.path.realpath(gitignore), self.root_dir):
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/utils/fs.py", line 156, in path_isin
    parent = os.path.join(normalize_path(parent), "")
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/dvc/utils/fs.py", line 154, in normalize_path
    return os.path.normpath(path)
  File "/home/ubuntu/anaconda3/lib/python3.7/posixpath.py", line 340, in normpath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

the error raised before any verbose output

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 21 (13 by maintainers)

Most upvoted comments

@marksein07 it looks like there is something strange going on in your git repo/environment. git rev-parse --git-dir should return the path to a .git directory, and git rev-parse --show-toplevel should return the root of your git repo. If --show-toplevel isn’t returning anything it means git does not think it is inside a git working directory, which in turn will break DVC.

Are you inside a bare git repository (a server-side repo that you normally clone/push/pull to and not a working git clone’d repo)?

Alternatively, do you have $GIT_DIR set in your environment? If so try unsetting it and then retrying those commands?

pmrowla on Dec 24, 2020

@marksein07 can you run these commands on your aws machine and post the output (from the same directory you are trying to init DVC in):

$ git rev-parse --git-dir
$ git rev-parse --show-toplevel

pmrowla on Dec 22, 2020

Another thing to point out is that _root_dir and gitpython’s repo are in different path, /sa vs / whereas it should have been /sa and /sa/.git or (/ vs /.git).

skshetry on Dec 21, 2020

@marksein07, could you please try 1.11.6 version? We have made some changes there, so maybe that’ll work? Although the chances are slim. Thanks.

skshetry on Dec 15, 2020

I tried 1.11.2, and It doesn’t work.

marksein07 on Dec 4, 2020