polymer: Yarn Flat Mode Attribute in Package.json breaks projects

hi~

we’ve got a thread going over at https://github.com/package-community/discussions/issues/2 about this issue.

For those who don’t want to read through the Dostoyevsky book that thread’s become, here’s the tl;dr of what my team’s planning on implementing:

1. a new type of dependency, called an asset dependency, which will install into an assets/ directory, distinct from node_modules
2. assets are guaranteed maximally flat, and existing separate means node tooling isn’t broken by spurious forced flattening that it was never developed to support in the first place
3. we’re going to port Bundler’s tree-flattening algorithm, which is pretty good at finding flat trees without requiring manual resolution. This is probably going to be much better than bower’s algorithm (and, tbh, probably Yarn’s) at finding flat trees without user involvement
4. when all else fails, we’re going to have a bower-style (but not bower-identical) resolutions definition in package.json so you can force a package into a version
5. assets will be additionally written into package-lock.json, so you’ll have reproducible builds
6. all of this will be built in to npm itself, with the same stuff that’s shipped out by default with node.js (once it lands)

tl;dr:

$ npm i --save-asset @polymer/example
$ echo '<script type="module" src="/assets/@polymer/example/example.js"></script><example></example>' \
  > index.html
$ npx serve
...(open http://localhost:5000/index.html and play with <example>)

Note: I’ve popped into the polymer slack if anyone wants more back-and-forth questions, and we’ve also got conversations going on about this in the Discord for https://package.community. I probably won’t say much more in this thread, specifically, but I figured this’d be useful. ❤️

But aren’t peer dependencies essentially what Polymer 3 elements are trying to accomplish in the npm ecosystem? Elements don’t import '@polymer/paper-button', they import '../@polymer/paper-button'

To me that’s the element asserting that it’s requesting a dependency that it does not have control of. It’s expecting a sibling dependency and not a child dependency right?

For all front-end polymer elements, yes their dependencies should be peer dependencies since the convention is to require by path up to a sibling folder. However, this doesn’t mean that all dependencies in the node_modules tree have to be peer. We can still get the benefit of nested dependency trees for other modules that are not importing each other by path.

Say for example, we have build tool that uses moment for dates, and our element is also using moment, but a different version. The packages would be

build-tool package.json

{
  "name": "build-tool",
  "dependencies": {
    "moment": "^1.0.0"
  }
}

build-tool.js

const moment = require('moment'); // Version 1

my-element package.json

{
  "name": "my-element",
  "peerDependencies": {
    "@polymer/polymer": "^3.0.0",
    "moment": "^2.0.0"
  }
}

my-element.js

import '../@polymer/polymer.html';
import '../moment/moment.min.js'; // Version 2

The node_modules tree would be

├── @polymer/poylmer@3.0.0
├── moment@2.0.0
├── my-element@1.0.0
├── build-tool@1.0.0
  └── moment@1.0.0

Using yarn with --flat is essentially forcing the entire node_modules tree to be “peer dependencies” instead of nested dependencies, so what’s the real difference?

@FredKSchott could you give an update about the status of yarn --flat? I found https://github.com/Polymer/polymer-modulizer/issues/254 but not sure what the latest status is.

I’m all for discussions on this type of front, but in the short term the flat attribute still needs removed from any package designed to be a dependency. It’s breaking yarn today.