WSL: rsync hangs on particular file

  • Your Windows build number: Microsoft Windows [Version 10.0.15063] + BOW 16.04 (reinstalled today)

  • What you’re doing and what’s happening: I am trying to rsync folders between two Windows (both BashOnWindows). I am using following commands:

rsync --archive --stats --progress --delete /mnt/u/1-folder /mnt/u/2nd-folder some_name@192.168.99.17:/mnt/u/2-folder/
rsync -rl --progress --delete ...
rsync -rl --delete ...
  • What’s wrong / what should be happening instead: It works almost flawlessly. But sometimes (at least 5 different cases) it stucks on particular file. When stopped (Ctrl+C) and resumed, it always hangs on the same file. Usually it hangs only on .pch and .pdb files (about 20MB). See the example below. I stop it with Ctrl+C (see the ^C).
        560,502 100%    1.63MB/s    0:00:00 (xfr#78, ir-chk=1056/39982)
some-path/more-path/cl.command.1.tlog
          6,060 100%   17.83kB/s    0:00:00 (xfr#79, ir-chk=1055/39982)
some-path/more-path/some-file.Build.CppClean.log
          1,817 100%    5.33kB/s    0:00:00 (xfr#80, ir-chk=1054/39982)
some-path/more-path/some-file.pch
         32,768   0%   95.81kB/s    0:00:46  ^C
rsync error: unexplained error (code 130) at rsync.c(632) [sender=3.1.0]
[sender] io timeout after 60 seconds -- exiting

I tried to wait for more than 24h… it stays on the one file.

Also, strange thing is:

  • it does not fail, if I move files around to different folders
  • as a workaround I can move offending files one by one with rsync (then rsync works again)
  • Strace of the failing command, if applicable: Ohhh… Sorry. Tried to reproduce. But I can’t. Because I moved files around. If you can not resolve this without strace, then I will wait till the next time and will provide strace here in comments. (If <cmd> is failing, then run strace -o strace.txt -ff <cmd>, and post the strace.txt output here)

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Reactions: 29
  • Comments: 117 (7 by maintainers)

Commits related to this issue

Most upvoted comments

I might have some info to add.

From PowerShell, I run wsl. I run rsync and it freezes. However, if I resize the PowerShell window, it continues… I’ve been able to complete an rsync by constantly resizing the window.

By the way, for everyone else hitting this – there is a really kludgy workaround: when it hangs (probably multiple times during a run), send one of the ssh/rsync processes a SIGCHLD. It ignores the signal, but this causes the blocked syscall to return and generally un-wedges everything.

Or just sit in a loop sending the signal until it completes, like so:

while killall -CHLD ssh; do sleep 0.1; done

I have the same problem, and indeed, a console resize causes it to continue.

are you folks on 1809 (aka “RS5”) or an earlier version? I’m pretty sure this was fixed as #3100 – I haven’t seen it since upgrading to 1809.

i have this issue since today. I am on 1809 and last month the rsync works great. But today i can only use it when i use

while killall -CHLD ssh; do sleep 0.1; done

that is realy strange?!

Possible workaround for freezes on partial transfers: use the -W (–whole-file) option to force complete re-transfer of the problematic files. This avoids the freezing for me, at the cost of additional bandwidth for files that changed since last run.

(I’ve found that I see regular freezes like this rsyncing files in the 1-2 gig range when I’m doing a second pass after some files have changed: file is transferred at size X, then later transferred again after it’s grown to size Y. This is rsyncing from WSL on Win10 Creator’s Update to macOS.)

This issue has been automatically closed since it has not had any activity for the past year. If you’re still experiencing this issue please re-file this as a new issue or feature request.

Thank you!

I guess all tickets are being closed automatically now without a week of response from the ticket opener.

I have re-opened this issue at #11157 in case anyone wants to follow-on the journey of this clown car.

Using the workaround mentioned in this bug report, I am using this to run my regular backup. Works for me.

#!/bin/bash
IGN_WIN='--exclude Cache --exclude .cache --exclude cache --exclude cache2 --exclude NTUSER.DAT --exclude ntuser.dat.LOG1 --exclude ntuser.dat.LOG2'

/usr/bin/rsync -vxrlHpEogDth -W --delete-after --delete-excluded --ignore-errors $IGN_WIN  -e 'ssh -i /root/.ssh/id_rsa.winbackup' /mnt/c/Users/ winbackup@xxx:/backup 1> /root/winbackup.1.log 2> /root/winbackup.2.log &
pid=$!

while /bin/kill -0 $pid 2>/dev/null; do
    /usr/bin/killall -CHLD ssh
    /bin/sleep 0.1
done

I have the same problem.

Why on earth has this not been fixed???

Please at least have the decency to close this as WONTFIX instead of lying about this being completed!

Thank you for this. Swapping the -P flag for the -W flag works for me as well, both on HDD and SSD. I’ve been trying to resolve this for 2 years, so much appreciated!

this issue has never been addressed, it should not be closed.

It never will be. WSL1 is long dead, and this can’t be fixed by the community.

As a confirmation, just ran into this issue on fresh W10 install, where rsync of large files (win->lin) would hang. Spend few hours trying to debug it myself, until I noticed that my WSL was created as WSL1 instance, instead of WSL2. Things seem to work fine in WSL2.

As a possible reason as to why signals help - strace(ing) some of the hanged rsync instances were stuck in “select” calls. Assuming there is some race condition somewhere in select & pipe combo, where write to pipe won’t wake up the select on another side, this would explain why resizing terminal (or sending SIGCHLD) helps - signal will interrupt the select, and subsequent call into it will notice data in the pipe and resume copying.

Adding a -W to the rsync command line had fixed this problem for my rsync based backups for quite some time. But it recently got stuck again. I updated from Windows from 1803 to 1903, updated WSL from 16.04 to 18.04 but nothing helped. So this definitely has not been fixed.

Running a “while killall -CHLD ssh; do sleep 0.1; done” in the background made the sync work again. It is really disappointing that after more than two years Microsoft has shown no interest in fixing really major bugs affecting basic linux tools like rsync.

I’m having massive issues with this right now, but neither --whole-file nor the SIGCHLD hack appear to be working for me. :-\

Can confirm that resizing the PowerShell window or sending SIGWINCH signals works for me as well, both on my legacy 14.04 installation and a brand-new 18.04 one.

@0xabu 's workaround might be a bit clunky but gets the job done…

while killall -CHLD ssh; do sleep 0.1; done

I think I’ll be using rclone for now.