dvc: Unexpected error when cache and artifacts are not on the same partition on SSH
Setup
pip
installed, version 0.66.1.
Platform: Ubuntu 18.04
Steps to reproduce
- Create SSH remote
cache
, and configure it as SSH cache. - Create directory on the same computer, but on other partition. Add some
test.txt
to this dir. - Add this remote directory as SSH remote called
workspace
. - Run
dvc add remote://workspace/test.txt -f test.dvc
.
Bash script reproducing error:
# arrange
workspace=$(mktemp -d)
cache=/tmp/ram
mkdir -p /tmp/ram
sudo mount -t tmpfs -o size=512M tmpfs /tmp/ram # creating distinct partition for cache
cd $(mktemp -d)
dvc init --no-scm
dvc remote add workspace ssh://localhost/${workspace}
dvc remote add cache ssh://localhost/${cache}
dvc config cache.ssh cache
echo "foo" > ${workspace}/foo
# act
dvc add remote://workspace/foo
# cleanup
sudo umount /tmp/ram
rmdir /tmp/ram
Observed behavior
100%|██████████|Add 1/1 [00:00<00:00, 1.02file/s]
ERROR: unexpected error - Failure
Expected behavior
I believe that by design DVC should raise error in this case - in order to avoid implicit movement of large files between partitions.
However, the error message should suggest solution:
ERROR: File move to cache failed.
Ensure that file remote://workspace/test.txt is on the same partition as SSH cache.
Some people might by mistake put file to different partition (it happened to me! - such message could save me a bit of time 😉).
Because I had already debugged/studied --verbose
DVC to understand the source of the problem, I could try to implement such message through Pull Request.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 15 (6 by maintainers)
@shcheklein , yes 😃