webpacker: yarn_install task is not installing devDependencies
Due to the recent changes that have been made, webpacker:yarn_install
task is not installing devDependencies
. This is breaking my CI builds, since I relied on webpacker to do the install of packages for my test environment on the CI server.
Change that brought this in the webpacker gem https://github.com/rails/webpacker/commit/d5fe0fe8e492d1511c6f31648f8fe861504c111e#diff-5a0f30530a2f1282593526bb85d65029 Change that brought this in the Rails gem https://github.com/rails/rails/commit/2118c98b98c4984069e070aa0d5c398de618d02a#diff-dcd44149ba1cd25c12956a3ebecc32a3
I’m just wondering if we could introduce an optional --production=true|false
flag in the rake task that does yarn install
?
Or create a NODE_ENV
dependant flag for the rake task?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 32 (14 by maintainers)
@leikind They are not
devDependencies
really because you are using them on Heroku as well, which is a production environment. If however, you compile your assets locally and then push them to Heroku then they can safely live under dev dependencies, consideringassets:precompile
is never run on Heroku.With webpacker, we recommend putting all build and app related dependencies under dependencies section in package.json (except test deps). I understand the confusion but you see they are not app dependencies i.e. it won’t be included in your pack but they are needed to produce a pack or build - either on Heroku or on your local machine. Does this make sense?
IMHO lots of people install
webpack-*
andbabel-*
in devDependencies since they are for building instead of serving things.yarn install --production
can cause devDependencies being used to build removed. I always performyarn
before doingassets:precompile
but now I get mywebpack-chunk-hash
in devDependencies missing afterwebpacker:yarn_install
after upgrading webpacker. I don’t think it is a good idea forcingyarn install --production
in webpacker rake task.We just don’t use devDependencies. Yeah extra packages might get downloaded in production but it shouldn’t be that big of a deal. To me it’s just a paper cut, but the few hours I spent debugging I’ll never get back.
I am also having issues with
--production
being hardcoded. I have a lot of things indevDependencies
that don’t belong in production and I am more likely to remove webpacker than I am to change this. I also have more than just production and development environments and would like more flexibility in passing options to NODE_ENV. It’s fine if webpacker makes certain assumptions for me but it shouldn’t force me to do things a certain way.I don’t think you’d like the the voice of my documentation on this one. 😉
It’s just frustrating to see a lack of attention to his, especially when dev team has come up against it themselves and there’s been an unaddressed PR about it for nine months. Even closing something as a #wontfix is fine if that’s what the maintainers decide.
Many teams run assets:precompile on heroku. In fact, it is a standard part of the rails build pack…
It does not. Webpack, babel, etc are dev ops tools. They are not application code. Heroku installs devDeps, so why force everyone to make a bad organization decision and move those to deps?
This should be addressed by the Webpacker team…
@csalvato Don’t feel bad, people have been confusing this for over ~a year~ 2 years:
I’m having issues with this change as well. In particular,
yarn install --production
seems to be removing the babel-plugin-relay, but it’s required as part of the asset precompilation… which is now failing due to being unable to find the plugin.Neat. Awesome. I’m on board as long as it’s in the docs.
Yes, and that’s how I’d prefer it to stay. Using the
--production
flag bypasses that. And the test guy is someone else, but I do agree with you on that.The dev team has come up against it? Pretty sure the resolution was the opposite of what you portray:
👆That’s bikeshedding if I ever saw it. The WebpackER team is aligning themselves with how Webpack is used by the Node.js community. This is smart, it will do a lot to reduce friction between Rails & JS pipelines.
Remember that
package.json
&devDependencies
are all constructs from the Node.js world. Nothing is being imposed, it’s just being aligned with existing usage/documentation.Like I said here, those instructions aren’t always meant for end users that are not creating a package themselves (like any Webpacker app). The 2 links below are useful for explaining why this is: https://yarnpkg.com/blog/2018/04/18/dependencies-done-right/ https://stackoverflow.com/a/22004559/6091685
Rails is a development tool too 🙃. Did you know that both Webpack and Babel are set up to behave differently depending on which flag is set? (NODE_ENV) I personally would want to run tests against production bundles.
The webpacker team is just trying to keep you on the most pain-free path. Just chuck your
devDependencies
independencies
and be done with it…none of this will matter soon anyway because the rules are changing ¯\(ツ)/¯ .IMO there are legitimate use cases for
devDependencies
in thepackage.json
of a Rails app - say you want to have JS testing libraries like Karma, Jasmine etc installed viayarn
on CI but don’t want these when in the production environment (see https://github.com/rails/webpacker/issues/1330 for further context).Webpacker hardcoding
--production
into a Rake task is disruptive of otheryarn
uses which don’t relate to asset compilation.The app is indeed built on Heroku,
assets:precompile
is executed.You see, if my app were a Node.js app using only the node buildpack, I’d have no issues building my app on Heroku, as the node buildpack would install all packages first, run build scripts (webpack for instance), and then strip devDependencies. So, the node team in Heroku does invite its customers to build their node apps on Heroku.
I see no argument against building my app on Heroku except for the webpacker team making it harder.
Here’s my current hack to solve my problem:
We’re still having production flag hardcoded y’all and I still don’t understand why 😄
Thanks, everyone and sorry about the confusion 🙏 @jakeNiemiec really like the way you have explained it ❤️
@trent-boyd I understand this could be confusing but even if we remove production flag, which I think we should because it hardcodes the environment. Anyway, Yarn would use
NODE_ENV
to infer the current env and install dependencies accordingly. That’s why I suggested putting all build dependencies in the dependencies section because they are needed to build the webpack bundle, unless you are building locally. Of course, you are free to exclude test and linter modules because you won’t be using them on production (up to you).@jakeNiemiec is there some confusion here about what
devDependencies
vs production dependencies means? I feel like many believe that thedependencies
include something in a production bundle and therefore increase the package size? (if this question/comment is just chatter, thumbs down and I’ll delete)I have a Rails 5.1.6 app with webpacker successfully deployed on Heroku app with 2 buildpacks: (1)
nodejs
and (2)ruby
.Upgrading to 5.2.0 breaks the Heroku build for the reasons @nikolalsvk listed above. The build fails because there is a number of
devDependencies
not installed afterrake yarn:install
.Moving
devDependencies
todependencies
feels very very wrong.What is interesting is Heroku’s approach to handling
devDependencies
anddependencies
in theirnodejs
buildpack:( https://devcenter.heroku.com/articles/nodejs-support#package-installation )
It seems like a more reasonable approach than just excluding all
devDependencies
.Unfortunately, the rails buildpack is executed after the nodejs buildpack, and
webpacker:yarn_install
makes sure none of mydevDependencies
are present.I am wondering what the correct solution should be.
Maybe I’m doing something wrong, but this change broke my production deploys. I too had all my compilation related packages in devDependencies, so the production server couldn’t build the assets correctly.
@trent-boyd Mr Boyd, ~please feel free~ you should ~to~ send a PR
Thank you @gauravtiwari for your time contributing to the rails/webpacker project. Webpacker has been a delight and it saves me so much time integrating Vue into Rails ❤️
@gauravtiwari
That’s an example of bikeshedding if I ever saw one. Further, this is listed nowhere in the documentation. I should not have to crawl the Webpacker source or track down this exact issue just to find out that your team is imposing their own view on how
devDependencies
should work.Hundreds of other JS-based open source projects intentionally instruct you to use the
--dev
flag during installation. Webpacker is a development tool for bundling up code intended to be run in a browser. It would make sense to me that other development tools (like Babel, its plugins, and other Webpack loaders) would also be installed as development dependencies.And besides that, what is the
--production
flag really helping anyway? If I want to really streamline my NPM install times or cut down on disk usage, that should be something I opt-in to.You never need to worry about downstream
dependencies
in a Rails app, if you come across a--save-dev
in a README, that was not put there for you (the same problem exists in Node.js-land). @nikolalsvk, What is the purpose for a CI test? To simulate a production environment and programmatically test a bunch of edge cases. Wouldn’t you want to test against minified/transpiled production bundles?Are you using webpack to bundle Karma or Jasmine? I don’t understand why you would do this.
Less “confusion” and more like ritual inclusion of programing patterns that subvert their original purpose (e.g.: running CI tests in development mode). All it does is tell yarn/npm what libraries to put in
node_modules
. If you include Karma independencies
, you will just additionally have Karma and it’sdependencies
in yournode_modules
. At worse you have an extra 8 seconds to wait when you install yourdependencies
.TL;DR: You should follow what @gauravtiwari said, there are mechanical differences between the
production
anddevelopment
flags, they are more than just labels:devDependencies
has more usefulness in the context of a JS library.Let’s take
webpack
as an example. Notice thatreact
is installed underdevDependencies
.webpack
library locally, it would installreact
.webpack
as a dependency, it would NOT installreact
. This is because webpack does not needreact
to function, therefore it does not need to pass the lib downstream.This is what @gauravtiwari was getting at. Your rails app is “the end of the JS line”, you don’t need to worry about downstream users. I am aware that many READMEs will tell you to use
--save-dev
, butdevDependencies
offer no practical use. (esp. in the context of webpacker)TL;DR: If you need something to build you production bundle, it cannot be placed in
devDependencies
. (including: webpack, typescript, node-sass, sass-loader)@gauravtiwari I agree this makes sense if you look at the
devDependencies
/dependencies
distinction as code that’s used only locally during the dev workflow (like Prettier, ESLint, etc) vs code that’s executed on production. I think what you’re hearing from other people (and please count myself in that list) is that they all have different understandings of the term and some of those understandings have been implemented into widely used tools (like Heroku).I ran into the same problem myself when deploying this via the
opsworks_ruby
cookbook to AWS. My solution was to move thedevDependencies
intodependencies
, but I’d like to see it install dependencies preferably by default, or as an option.