WSL: Symlinks on the mounted Windows directories are not compatible with native.

On build 14341,

I created a symbolic link to a file on a mounted Windows directory in BASH.

tinysun@TEST-PC:/mnt/c/Temp$ ln -s origfile.txt symlink.txt
tinysun@TEST-PC:/mnt/c/Temp$ ls -l
total 0
-rwxrwxrwx 1 root root 39 May  4 01:48 origfile.txt
lrwxrwxrwx 1 root root 12 May 13 02:58 symlink.txt -> origfile.txt
tinysun@TEST-PC:/mnt/c/Temp$

In the command prompt CMD.exe, the file appeared as a JUNCTION and can not be accessed.

C:\Temp>dir
 Volume in drive C has no label.
 Volume Serial Number is EAE0-0FD9

 Directory of C:\Temp

05/13/2016  11:58 AM    <DIR>          .
05/13/2016  11:58 AM    <DIR>          ..
05/04/2016  10:48 AM                39 origfile.txt
05/13/2016  11:58 AM    <JUNCTION>     symlink.txt [...]
               2 File(s)             51 bytes
               2 Dir(s)  194,170,494,976 bytes free

C:\Temp>type symlink.txt
The file cannot be accessed by the system.

C:\Temp>

It should be displayed as follows.

05/13/2016  11:58 AM    <SYMLINK>     symlink.txt [origfile.txt]

I think the ‘JUNCTION’ can be used when the link references a directory not a file.

About this issue

  • Original URL
  • State: closed
  • Created 11 years ago
  • Reactions: 32
  • Comments: 71 (11 by maintainers)

Most upvoted comments

@jordwalke - @SvenGroot is working on unifying the symlinks story between WSL and Windows so that WSL will create symlinks in a way that Windows can understand them.

Would also like to see this. I’ve tried it both ways (link and target both on Windows file system):

  1. Creating a symlink via ln -s from Bash/WSL.
  2. Creating a symlink via mklink in Windows cmd.

With approach 1 I observe the behaviour mentioned here. Also the link will simply not appear in my editor on the Windows side (VS Code). In Windows Explorer it will appear but give an error upon trying to open it. However it works with everything on the WSL side.

With approach 2 everything works on the Windows side (as you would expect), but the symlink isn’t visible and doesn’t work on the WSL side. However, subsequently trying approach 1 results in this error: ln: failed to create symbolic link 'link.txt': File exists, and doing an rm will actually remove it, so it does exist at some level in WSL.

So to sum things up, a symlink created on either side cannot be effectively utilised by the other side, kinda breaking the file system interoperability promise. I would expect DriveFS to correctly handle symlinks created with mklink at least.

Thanks for your consideration.

Yes, this a big mess especially when git gets involved…

Personally, coming from Linux and OSX the entire SeCreateSymbolicLinkPrivilege situation is one that has repeatedly caused me headache and grief. I was hoping against hope that BoW would finally fix the issue but, alas, it seems instead of making symlinks work properly in user space, MS has decided to just make a new standard? I can’t follow any of the symlinks created in the linux subsystem even in windows exporer, much less in my IDE or other windows host apps. This makes me sad. 😦

Update for anyone following this issue: I installed the Fall Creator’s Update, and it looks like symlinks created from the Windows side (e.g. using mklink) now work properly in WSL.

Links created on the WSL side (e.g. using ln -s) still aren’t properly recognized by Windows.

If you have stuff that relies on symlinks, it may be worth creating them or “initializing” things (e.g. git cloning) on the Windows side if possible (if you want interoperability).

ln -s somedir someotherdir now generates a file again instead of a symlink. That file is not accessible in explorer (or in VSCode, for that matter). I’m running Windows 10 Home, version 1903, build 19002.1002, and WSL v1.

I would also like this. Currently on the Bash side we’re using Yarn for dependency management. We’re sharing files between Windows and Bash by using /mnt/c/… . Yarn ends up using symlinks and works well if you’re entirely in the Bash side. However, these symlinks are not seen on the windows side properly, which breaks development tools over there.

Note: if i manually make a junction or symlink on the Windows side (i use the Link Shell extension here: http://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html) then the Bash side sees things fine. So there does appear to be interoperable symlinks, it’s just that “ln -s” doesn’t make them inside Bash. Fixing this would be a godsend. Thanks!

Note: I’m on 16251.

Windows Insider Build 17046 contains some improvements for symlink compatibility. See the full description in the release notes

Right, it looks like the fix for WSL to read directory junctions and symbolic links created in windows is no longer present in Windows 1809. image

I have a mess of utilities installed redundantly on Windows and Ubuntu due to this issue. Hopefully it will get resolved at some point.

Repost of the suggestion I just posted on Issue #1475:

lxss symlinks also break my win32 backup tools.
+1 for the urgent need to unify lxss symlinks and win32 symlinks.

The tools on both sides of the programming world should be able to view and copy directory trees created on the other side, with all links in the copy usable by both sides as well.
=> The win32 and Linux subsystems must hide to user-mode programs what target path is really written on disk, and instead return to them converted targets they can understand.
For relative links this is pretty straightforward: Simply change all slahes to backslashes for the Win32 SS, or forward slahes for the Linux SS.
For absolute links, the problem is that the targets don’t refer to the same root. Yet that root can be identified by the kind of slash written by the link creator:

  • If the link contains back slashes, it was created by a Win32 process relative to a Windows volume.
  • If it contains forward slashes, it was created by a Linux process, relative to the Linux FS root.

Then the subsystems can convert the link targets to absolute paths formatted as their respective user-mode programs expect.

I’m the man created this issue two years ago. Now, in ver. 1803 (OS Build 17134.48), I got the expected result after performing the test scenario which I wrote in the first post.

05/18/2018  05:48 PM    <SYMLINK>     symlink.txt [origfile.txt]

I think that the issue of absolute paths or empty target is out of the scope of this issue.

This issue is certainly closed. Thanks for WSL team, WSL is the wonderful software.

Right now there isn’t a really good way to do this. The best option would be to leave a bash window open when you already have the drive mounted. The new bash window will use the same instance and will be able to resolve the working directory.

We’re working on providing a solution for this, so stay tuned!

We are working on better integration with Windows symlinks (i.e. the ability to create them from WSL), though there will always be limits to what we can do due to difference in semantics between Windows and Linux symlinks.

The biggest issue is that Windows doesn’t have a symlink file type. Instead, a symlink is a reparse point that can be applied to either a directory or regular file, and the type used for the symlink must match the type of the target. If it doesn’t, the link does not work.

In Windows, all the mechanisms used to create symlinks require you to explicitly specify whether you want a file or directory symlink. CreateSymbolicLinkW, mklink, Powershell’s New-Item, they all require the user to make this distinction.

The Linux “symlink” API, and as a result the “ln” tool as well, does not have such an option. We therefore need to be able to determine what type of link to create before we can create it.

This means that we can only create Windows symlinks if all of the following are true:

  1. The target is reachable from Windows (which means the link is relative and does not cross any mount points).
  2. The target exists (so we can determine if it’s a file or a directory).
  3. The user has sufficient privileges to create symlinks (which currently requires elevation unless Developer Mode is turned on).

In other situations, we will still be forced to create a WSL symlink.

If you mount using a UNC path, DrvFs goes through SMB rather than directly to NTFS. This is both a lot slower and lacks quite a bit of functionality, including metadata, case sensitivity, and indeed symlinks. It is strongly not recommended to mount DrvFs this way for local filesystems.

To achieve the effect you want, use bind mounts instead. Put the following in your /etc/fstab:

C:              /mnt/c  drvfs   rw,noatime,metadata,uid=1000,gid=1000,umask=022,fmask=011
/mnt/c/Windows  /mnt/h  none    bind

Note you have to explicitly mount C: before creating the bind mount because we process /etc/fstab before automounting DrvFs volumes.

Removing the “matching file type” restriction on existing symlinks, or introducing a new symlink type that doesn’t have that restriction, is definitely something we looked at.

Unfortunately, the assumption that the symlink has the same type as what you’ll eventually open is baked into the system at a bunch of places, and done in such a way that it would be hard to change without likely breaking something. Backwards compatibility can be a pain sometimes. While it wouldn’t be impossible to pull this off, it would definitely not be trivial.

I’m running version 1803 (build 17134.191), but symlinks to files created on WSL are still being treated as directory junctions, making it inaccessible.

File system performance is always a priority for us, and we are working on several ways to improve this. We’ve already made significant strides in RS3 (the upcoming Fall Creator’s Update), and are continuing to work to improve this further.

There are limits to what we can do due to requirements of application and file system filter compatibility, but we’re very much aware that file system performance is a pain point for WSL (and Windows as a whole) and are working to improve the situation.

@ned14 Sure, I can add that privilege, but as you’ve pointed out, I then lose the ability for that user to escalate privileges. That doesn’t seem a fair trade. 😉

@russalex Can it not be the case that if the user running WSL has the SeCreateSymbolicLinkPrivilege that WSL creates NTFS format symbolic links instead of a custom link type?

Do you have a pointer to a description what exactly goes wrong?

It was some time ago I last wrestled with this issue, so my memory may be faulty. The principle cause are repos which use libtool, and they ship in the repo prebuilt binaries linked by libtool. That said, I’ve occasionally bumped into repos which are Unix-only, and thus they think it’s fine to commit symlinks.

There are two separate problems:

  1. drvfs appears to git that it doesn’t support symlinks, when in fact WSL can create WSL-only symlinks just fine. Rather than git reporting this, or warning about it, instead it silently creates filesystem entities at those locations which don’t work, but in a highly non-obvious way. I’ve no idea if the cause is git, or drvfs, or something else here.

  2. I can’t see why relative symlinks from “the other side” can’t work. I mean, if you get a Windows symlink which says “…\foo\bar”, that clearly maps into “…/foo/bar” in WSL. Or vice versa.

Now I appreciate that relative vs absolute symlinks on Windows is a bit fickle. Plenty of absolute symlinks ought to be relative. But it would be a start if Windows symlinks with the SYMLINK_FLAG_RELATIVE flag set would parse within WSL.

Outside WSL, it’s much harder. Symlinks have their own reparse type. Me personally, I would have WSL create Windows symlinks all the time if the symlink is a relative path, but with the PrintName being the POSIX side of things, and SubstituteName being the Windows side of things. That dispenses with the problem entirely for many use cases.

Obviously absolute path symlinks are much harder to solve. But relative path symlinks I think are very solvable, and without needing to invent new reparse point types, or break anything existing.

Firstly, Corinna I’d like to thank you so much for all the material on the internal workings of the NT kernel API which you have built up over many years on various mailing lists. I found them invaluable during the development of LLFIO, the reference library implementation for https://wg21.link/P1031 Low level file i/o library. Thank you so much for that.

I’d like to raise the point that fixing Windows symlinks to work well between WSL and native would help solve a long standing bug when using git on Windows, or git within WSL on a mounted Windows drive, where somebody has committed a symbolic link to the repo. Right now it doesn’t work most of the time, and moreover, it doesn’t work in a particularly non-obvious and hard to detect way.

If Windows symlinks understood both Windows and POSIX semantics, then git-created symlinks - whether created by a git inside WSL or outside - would “just work”. This ought to be not hard to fix - git-created symlinks are always internal to the repo, referencing one part of the git repo from another part of the git repo. They should “just work”.

I really wonder why there isn’t an easy way out: Instead of the current Windows symlink reparse point, invent a new symlink reparse point which is type agnostic (not file or dir, but “just a symlink”). Create symlinks using this new type on both sides of the fence (Windows/WSL). Support the old symlink reparse point for backward compat only.

This new symlink reparse point could be defined to contain a Windows path as well as a POSIX path, so it could simply work for files and dirs both, and it would finally be possible to create working symlinks with no target on the Windows side as well.

What is so complicated about this that this hasn’t already been done?

Corinna

Can this issue be reopened?

Thanks for the work you put in the symlinks, @SvenGroot . I do actually have a question about the requirements you posted, in order for WSL to make a Windows symlink:

The target is reachable from Windows (which means the link is relative and does not cross any mount points). The target exists (so we can determine if it’s a file or a directory). The user has sufficient privileges to create symlinks (which currently requires elevation unless Developer Mode is turned on).

Today I noticed a strange effect with VirtualBox and its shared folders. I can allow VB to create symlinks as well, for example by running it elevated (or enabling developer mode). Then VB does allow me to create any symlink I want inside the Linux guest, also to non-Windows locations. For example: VB: ln -s /proc test Then in VB it shows up as a normal symlink (lrwxrwxrwx 1 vagrant vagrant 0 Nov 21 16:26 test -> /proc), and it even maps to Windows without problems (21-11-2018 17:26 <SYMLINK> test [\proc]), to my great surprise. Of course I cannot open the symlink or anything like that, but it is apparently stored in Windows. Consequently, this symlink works without any problems in WSL where it actually maps to /proc.

I think in practice this behaviour (making symlinks even though the requirements are not met) has the same effect as making the symlink windows-incompatible as is done in WSL now: Windows cannot do anything with it, but VB/WSL can. On the other hand, if I were to create this symlink in WSL currently, I cannot access it in VirtualBox, which has actually broken some things for me (the links were incorrectly read by VB).

So my question then is: Is VirtualBox doing something tricky here? I would assume the Windows API would also enforce these requirements, but apparently not? Is this behaviour desirable or potentially harmful? Would it be an idea to adopt this behaviour in WSL as well, to provide compatibility between VB and WSL? (or is it better/easier to fix VB to support the reparse tag used by WSL?) Or for that matter, not just VB but I can also think of NFS shares and many other things.

A possible alternative to condition #3 is:

3bis) If the target exists and is a directory, then create a junction instead of a symlink. This does not not require any special privilege.

----- Mail original -----

De: “Sven Groot” notifications@github.com À: “Microsoft/WSL” WSL@noreply.github.com Cc: “JFLarvoire” jf.larvoire@free.fr, “Comment” comment@noreply.github.com Envoyé: Mardi 31 Octobre 2017 22:30:50 Objet: Re: [Microsoft/WSL] Symlinks on the mounted Windows directories are not compatible with native. (#353)

We are working on better integration with Windows symlinks (i.e. the ability to create them from WSL), though there will always be limits to what we can do due to difference in semantics between Windows and Linux symlinks. The biggest issue is that Windows doesn’t have a symlink file type. Instead, a symlink is a reparse point that can be applied to either a directory or regular file, and the type used for the symlink must match the type of the target. If it doesn’t, the link does not work. In Windows, all the mechanisms used to create symlinks require you to explicitly specify whether you want a file or directory symlink. CreateSymbolicLinkW, mklink, Powershell’s New-Item, they all require the user to make this distinction. The Linux “symlink” API, and as a result the “ln” tool as well, does not have such an option. We therefore need to be able to determine what type of link to create before we can create it. This means that we can only create Windows symlinks if all of the following are true:

1. The target is reachable from Windows (which means the link is relative and does not cross any mount points). 
2. The target exists (so we can determine if it's a file or a directory). 
3. The user has sufficient privileges to create symlinks (which currently requires elevation unless Developer Mode is turned on). 

In other situations, we will still be forced to create a WSL symlink. — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or mute the thread .

I would want ln to use mklink so that when I use WSL to create git checkouts, those git checkouts can be understood by windows GUI editors.

@dreamcat4 (FYI rm from WSL works to delete NTFS junction.)

@131 / @noinkling Could/should someone make a version of ln (in WSL) that uses mklink under the hood? Wouldn’t that achieve what this original issue requests?