react-native: [Packager] doesn't resolve modules that are symlinks
As described in the browserify handbook you can use symlinks in the node_modules
folder to reference modules. This is mainly useful for preventing ../../../../
requires.
With a symlink setup from the project root setup like:
ln -s src/app node_modules/app
in any file in the project you can:
var app = require('app');
var somethingElse = require('app/lib/something-else');
However, the packager can’t resolve them: Requiring unkonwn module "app"
.
Edit: Check out this repo to easily reproduce the error above:
https://github.com/JaapRood/react-native-packager-symlink-bug
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 83
- Comments: 114 (49 by maintainers)
Commits related to this issue
- bump version to publish npm package without tests — committed to morenoh149/react-native-contacts by morenoh149 8 years ago
- add development helper — committed to joenoon/react-native-router-flux by joenoon 8 years ago
Hi all, being tired of how screwed up this situation is for us I wrote a simple tool that listens to changes in one folder and copies them to another folder. It’s called WML (Watchman-Links, since it’s based on watchman). Check it out here.
It’s very simple to use:
Just ran into this trying to develop a npm library locally using
npm link
Any ideas when this would be resolved?
This is really frustrating 😕
I’ve just upgraded to React Native 0.35 which includes #9792 and thus should fix this issue.
I’ve created a
node_modules/src
symlink tosrc
but when I include a component (import Button from 'src/components/Button/'
) it says that I’m requiring an unknown module. With plain React (created withcreate-react-app
) under Firefox it works.I’ve tried to rerun both
react-native run-android
andreact-native start
but it doesn’t change anythingFigured out a little more about the cause – the issue goes away when switching to the Node-based watcher instead of the Watchman-based one. The watchman homepage says, “Watchman does not follow symlinks. It knows they exist, but they show up the same as any other file in its reporting.” so this could be the root case.
Support for symlinks seemed to get pretty strong pushback in https://github.com/facebook/watchman/issues/105 so perhaps the packager could use a combination of Watchman for most files and the Node watcher for symlinked packages. npm linked packages reside in the first
node_modules
directory (you’d have to go out of your way to end up withnode_modules/pkg1/node_modules/<linkedpkg>
), so that might simplify things a little. I tried adding a--noWatchman
option to the packager but ran into fd limit issues unfortunately.The npm link workflow is really important for anyone developing a React Native module so I do think this is a priority for the external RN community.
This is not a good practice. We need to get support for symlinks easily! Come on!
I hope it is annoying enough so they finally fix a 2 year old issue.
Someone should really find a proper solution for this thing. developing without npm link is kinda foolish. copying the dependency into node_modules, is also strange, what do you do on deployment?
I have posted the solution in http://stackoverflow.com/a/38976236/3360028
This is hindering me from making really (or relatively) awesome RN modules 😦
+1
@lacker I think we’re close.
I just tried now on using
react-native@0.42.0-rc2
andlerna@2.0.0-beta.37
in https://github.com/infinitered/reactotron (specifically https://github.com/infinitered/reactotron/tree/master/packages/demo-react-native)I tried a few different things.
Thing 1
If we just try to do a symlink, and the target library has it’s own symlink it, then the packager can’t resolve it.
Thing 2
If i symlink a react-native-based dependency, the packager detects it just fine 🎉 , but if that dependency has it’s own
react-native
in itsnode_modules
, then all hell breaks loose. You get duplicate@providesModules
if they’re the same RN, and conflicted@providesModules
if they’re different.Lerna Almost Saves The Day
Lerna can now deal with this by hoisting that dependency up to the root of the repo. 🎉
But if you do that for both the host app, then the cli doesn’t work (because the host app doesn’t have
node_modules/react-native/*
.So, if we manually symlink that to ‘…/…/node_modules/react-native’, the package goes off on a spiritual journey rounding up everything in the mono repo, eventually dying because of conflicts. I understand that. The packager was designed to be at the root level of an app, not a monorepo.
The idea experience for me
Up until 40, I’ve been just developing my 3rd party stuff right in the
node_modules
. I’d then manually copy & paste the stuff out. It felt bad (like editing files in production) and I cried myself to sleep, but you know what? It worked. Live reloading worked too. But now, the packager has to be restarted.It’d be great if we could use
npm link
to drive this. It’d feel like normal node module development.I haven’t explored
rn-cli.config.js
blacklisting & package roots yet.I also volunteer as tribute if you need people to test stuff.
@mkonicek I think there are 2 issues. Consider a repo layout like the following and running the RN packager from
./example
./lib
won’t be noticed by the RN packager../node_modules
which will typically (for e.g. testing purposes) contain all of the same dependencies as exist in./example/node_modules
, which leads to the RN packager complaining about duplicate source files.Hi guys, any update on this ?
My only way to develop a react native library is to git clone my repo inside my node_modules, isn’t it ?
Perhaps related, relative requiring a module above the iOS/React Native directory fails the same way:
Error: Requiring unknown module “…/common/auth”. If you are sure the module is there, try restarting the packager.
I tried symlinking that higher-level directory within the React Native project to fix, but it failed as well, leading me to this issue.
We definitely need to find way how to run React Native Packager with
--preserve-symlinks
option.Looks like the pain will be over soon 😃 https://github.com/facebook/react-native/pull/9009
OK, so here’s the repro steps:
react-native init AwesomeProject
mkdir test
cd test
npm init (all defaults)
Make an index.js, set it to:
npm link
cd …/AwesomeProject
npm link test
Add this to your index.android.js :
brew uninstall watchman
react-native run-android
Open chrome debugger to check the console.log hey
Change console.log(‘hey’) to console.log(‘ho’)
Reload the javascript in the android emulator
Still says ‘hey’
Environment:
Mac OS X 10.11.3 react-native 0.20.0 react-native-cli 0.1.10 node 5.5.0 npm 3.7.2 Genymotion Galaxy S6 API 23
Thank you 😄
Here’s a crude workaround:
Create file watchAndCopy.js in your react-native project root:
Replace my-package with your package and make sure the path is right. Make sure rimraf is installed globally (
npm install rimraf -g
). Runnode watchAndCopy.js
and you’re good to go.Plus side is it works with watchman so it’s super fast 😃
The RN packager was built for Facebook’s needs; which unfortunately (for us) didn’t include symlinks.
But… packager will be getting some love shortly. Hoping to see some news this week.
In the meantime:
react-native
dependency, this works really well. (if it does you can delete the files temporarily).node_modules
again with live reloading. Yes, it’s pretty derpy, but pair it with a file watcher and it’s better than nothing.I couldn’t use a second project root, because the RN packager would find duplicate files in my package’s
node_modules
dir. I ended up syncing the package’s changed files into the consuming app’snode_modules
dir: https://github.com/artsy/emission/commit/1049aa79c8117b0490f4ce2565ad22d628e90154#diff-b9cfc7f2cdf78a7f4b91a753d10865a2R12@nikki93 investigated using bindfs, which lets you mount a directory elsewhere in your filesystem, and found that it partly works – the packager/watchman can see the files within the mounted directory, but it doesn’t pick up changes so it’s not 100% useful for development & iteration. It seems that bindfs doesn’t propagate whatever mechanism watchman uses to detect changes.
While there’s progress being made here, we should improve the error message and throw an exception if there’s a symlink pointing to this issue/product pains. Otherwise people will keep getting confused until they figure out that react native doesn’t support symlinks
I’d also love to see this fixed! In the meantime it would great if there was something written up on the best workflow to develop third-party components, specifically when it comes to testing them in an example app (which is where this problem comes into play).
Can you move this to https://github.com/facebook/metro-bundler now that the packager has been moved out of React Native?
This wasn’t in the original post, but please note that this isn’t some obscure feature request.
npm link
, for example, uses symbolic links behind the scenes, so it breaks with that too. I can confirm that copying, rather then symlinking, does work, however that becomes impractical when working on different npm packages locally.Also, I tried it on macOS and linux and same issue occurs.
Not sure but probably node option
--preserve-symlinks
can help you somehow. https://nodejs.org/dist/latest-v6.x/docs/api/cli.html#cli_preserve_symlinksI had similar problem with symlinked files/modules (but on regular node project) and this option resolves problems for me.
Hey guys, I got tired with dealing with the symlink handling so I forked react-native and patched the packager to fix the issues I was having. This might not fix everyones issues but it has certainly fixed mine. The issue that I was having (and have had since the dawn of time in using react-native) was caused by the packagers inability of discovering peer dependencies on symlinked modules (npm link or yarn link). I’ve updated the resolver for the packager to be aware of the root paths of the project and automatically add these paths to the search query used to resolve modules. The prior implementation would try to be smart about resolving the paths based on the modules real path which breaks on symlinked modules.
Take it or leave it, but you are all welcome to use it. I’ve patched both the 0.42-stable branch and the 0.43-stable branch but haven’t really tested 0.43 since it relies on an alpha version of react.
Github repo
To use 0.42-stable update package.json react-native dependency to:
"react-native": "https://github.com/braunsquared/react-native/tarball/b576d07039dfaa0045fcff8b463f7490a86a7465"
For 0.43-stable use:
"react-native": "https://github.com/braunsquared/react-native/tarball/843be3173f617de0e13fc219f1aa2a809b94d1a3"
Any ideas when this would be resolved?
Yeah, I think symlinks are still busted. @ericvicenti is looking into the general problem of “how does one develop a React Native library with best practices” so 1/ people should throw their random ideas at him and 2/ I am hopeful this will get fixed en route.
I’m having issues with this as well. I can pass in the project roots and the packer will pick it up but it will only watch for file changes in the root. Here’s what I’ve seen so far with
/mysymlink
/mysymlink
into projectRootsChanges to the files in that folder root are picked up, but it does not recurse at all, so /mysymlink/index.js works but /mysymlink/foo/bar.js does not.
Debugging this a little bit it seems like:
The packager server uses a filewatcher https://github.com/facebook/react-native/blob/master/packager/react-packager/src/Server/index.js#L13
The filewatcher comes from node-haste https://github.com/facebook/node-haste/blob/master/src/FileWatcher/index.js#L80 Which uses sane
Sane uses walker to walk the tree https://github.com/amasad/sane/blob/master/src/node_watcher.js#L329 (note the dir callback)
and finally the walker https://github.com/daaku/nodejs-walker/blob/master/lib/walker.js#L76 doesn’t recurse down symbolic links (whereas it does if isDirectory() is true)
So somewhere in this whole chain it needs to recurse for this solution to actually work
I don’t know the advantages of Watchman but like others have pointed out, a fix in the meantime is just uninstalling Watchman:
According to https://github.com/facebook/react-native/blob/master/local-cli/util/Config.js, you can put custom configuration for the packager. It should be named rn-cli.config.js.
Here is an example configuration: https://gist.github.com/andon/bdf061e607f68def26fa#file-rn-cli-config-js
Declaring my local dependency as follow :
dependencies: { my_local_module: "file:../../path/to/local/module" }
thennpm install my_local_module
seems to work for me.@tuckerconnelly if you can come up with an isolated reproduction steps I’ll happily fix it (or accept patches). The underlying watching library is Sane.
Also cc @kittens who was looking into making node the default watcher
It worked after uninstalling watchman! Hooray for fallbacks 😃
However when I update my symlinked package, the packager doesn’t pick up on it and re-build the app. Any way to fix this?
How will we get better development, modules, tools, if developing modules that use other modules is a complete pain?
Sorry for the delay, was out for a while.
What happened is that we started using watchman for the filesystem crawling. My understanding is that watchman doesn’t follow symlinks but should treat them like any other file. Which in theory should still work. I can try to debug this soon. Alternatively we can add an option to force not using watchman for filesystem crawling.
I’ve found that this can be worked around by uninstalling watchman and therefore forcing react native to use the default node file watcher, however this does have speed drawbacks, and you can come up against issues with the number of files being accessed depending on the project.
-------- Original message -------- From: Braden Simpson notifications@github.com Date: 14/08/2015 20:25 (GMT+00:00) To: facebook/react-native react-native@noreply.github.com Cc: Gethin Webster gethin.webster@agilityworks.co.uk Subject: Re: [react-native] [Packager] doesn’t resolve modules that are symlinks (#637)
Is there any good workaround here?
Reply to this email directly or view it on GitHubhttps://github.com/facebook/react-native/issues/637#issuecomment-131214239.
👍 Same problem here. Symlinks are very useful, especially when I don’t want to make a node package just to share a component between two related React apps.
I have an internal fix for this. Aiming to get it out next sync 😃
is it a bug or is it by design? symlinks would be really handy actually 😦