berry: [Bug] Package hoisting in workspaces are broken since v2.4.x
- I’d be willing to implement a fix
Describe the bug
Before Yarn v2.4.0 all packages in the workspaces were properly hoisted to the root node_modules
folder with node_linker
.
However after updating to any version that is >= v2.4.0 hoisting is no longer working and every package in the workspace receives it’s own copies of all dependencies despite being compatible with others.
To Reproduce
Reproduction
// Sherlock reproduction
const fs = require('fs')
// Enable `node-modules` nodeLinker
fs.writeFileSync('.yarnrc.yml', 'nodeLinker: node-modules');
// Setup Workspaces
await packageJson({
name: 'root', workspaces: ['packages/*'],
dependencies: {
"@webcomponents/webcomponentsjs": "~2.4.0",
"rxjs": "~6.6.0",
"zone.js": "~0.10.3"
},
devDependencies: {
"@angular/core": "~9.1.12",
"tslib": "~1.11.1",
"typescript": "~3.8.3"
},
resolutions: {
"typescript": "3.8.3"
}
})
await packageJson({
name: 'dashboard',
dependencies: {
"@angular/core": "^9.1.12",
"@spryker/button": "^0.2.0-next.9",
"rxjs": "^6.5.4",
"tslib": "^1.10.0"
}
}, { cwd: 'packages/dashboard' })
await packageJson({
name: 'security',
dependencies: {
"@angular/core": "^9.1.12",
"@spryker/card": "^0.2.0-next.2",
"rxjs": "^6.5.4",
"tslib": "^1.10.0"
}
}, { cwd: 'packages/security' })
await packageJson({
name: 'zed-ui',
dependencies: {
"@angular/core": "^9.1.12",
"@spryker/button": "^0.2.0-next.9",
"@spryker/web-components": "^0.2.1-next.5",
"@webcomponents/webcomponentsjs": "^2.4.1",
"rxjs": "^6.5.4",
"tslib": "^1.10.0"
}
}, { cwd: 'packages/zed-ui' })
// Perform install
await yarn('install')
// Check that all packages were hoisted in root
expect(fs.existsSync('node_modules/@angular/core')).toBe(true)
expect(fs.existsSync('node_modules/@spryker/button')).toBe(true)
expect(fs.existsSync('node_modules/@spryker/card')).toBe(true)
expect(fs.existsSync('node_modules/@spryker/web-components')).toBe(true)
expect(fs.existsSync('node_modules/@webcomponents/webcomponentsjs')).toBe(true)
expect(fs.existsSync('node_modules/rxjs')).toBe(true)
expect(fs.existsSync('node_modules/tslib')).toBe(true)
expect(fs.existsSync('node_modules/zone.js')).toBe(true)
// Check that there are no packages in workspaces packages
expect(fs.existsSync('packages/dashboard/node_modules')).toBe(false)
expect(fs.existsSync('packages/product-list/node_modulesr')).toBe(false)
expect(fs.existsSync('packages/product-offer/node_modules')).toBe(false)
expect(fs.existsSync('packages/profile/node_modules')).toBe(false)
expect(fs.existsSync('packages/sales-orders/node_modules')).toBe(false)
expect(fs.existsSync('packages/security/node_modules')).toBe(false)
expect(fs.existsSync('packages/zed-ui/node_modules')).toBe(false)
Also repro in separate repo:
- Clone the following repository for repro: https://github.com/gund/yarn-worksapces-bug
- Execute Yarn install:
yarn install
- Inspect root
node_modules/@spryker
folder - it only contains a few packages that are directly referenced in rootpackage.json
but it should contain a few dozen other packages from the nested workspace packages (located underpackages/*/package.json
)
Actual content of the root folder `node_modules/@spryker`:
➜ suite git:(demo-yarn-latest-bug) ls node_modules/@spryker
frontend-config.eslint frontend-config.stylelint jquery-query-builder oryx-for-zed sql-parser-mistic
frontend-config.prettier frontend-config.tslint nestable sass-resources-loader
Expected content of the root folder `node_modules/@spryker`:
➜ suite git:(demo-yarn-latest-bug) ✗ ls node_modules/@spryker
ajax-action interception table.datasource.http
ajax-action.post-action.close-overlay jquery-query-builder table.datasource.inline
ajax-action.post-action.redirect label table.feature.batch-actions
ajax-action.post-action.refresh-drawer layout table.feature.editable
ajax-action.post-action.refresh-parent-table locale table.feature.filters
ajax-action.post-action.refresh-table logo table.feature.pagination
ajax-form modal table.feature.row-actions
button navigation table.feature.search
card nestable table.feature.selectable
checkbox notification table.feature.settings
chips oryx-for-zed table.feature.sync-state
collapsible pagination table.feature.title
data-serializer popover table.feature.total
date-picker sass-resources-loader table.filter.date-range
drawer select table.filter.select
dropdown sidebar table.filter.tree-select
form-item sql-parser-mistic tabs
frontend-config.eslint styles textarea
frontend-config.prettier table toggle
frontend-config.stylelint table.action-handler.form-overlay tree-select
frontend-config.tslint table.action-handler.html-overlay unsaved-changes
grid table.action-handler.url unsaved-changes.guard.browser
header table.column.chip unsaved-changes.guard.drawer
headline table.column.date unsaved-changes.guard.navigation
html-renderer table.column.image unsaved-changes.monitor.form
icon table.column.input utils
input table.column.select web-components
input.password table.column.text
Screenshots
Missing dependencies install via Yarn v2.4.0:
All dependencies installed via Yarn v2.3.1:
Environment if relevant (please complete the following information):
- OS: WSL Ubuntu on Windows 10 x64
- Node version v14.15.0
- Yarn version >=v2.4.0
Thanks!
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 5
- Comments: 20 (10 by maintainers)
Thank you for the feedback! I’ve created another issue so this one can be closed.
Let’s continue in #2727.
@gund It seems to me that another issue should be opened to discuss your concerns. The title of the issue is misleading. You are discussing the resolution algorithm and changes to the resolution algorithm. You are not proposing to change the hoisting algorithm in any way, the current hoisting algorithm will satisfy your needs and need not be changed.
The resolution algorithm is how package ranges are being resolved. The hoisting algorithm is how dependency graph with already resolved dependencies is being flattened on the file system.
I feel the discussion will be more constructive if it will be focused. Since no changes need to be done to hoisting algorithm. Its better focus on resolution algorithm only.
@gund Thanks, pulling out
package.json
into a separate repro would be helpful. Also, yes, Sherlock supports workspaces, for example see the Sherlock repro in the issue below: https://github.com/yarnpkg/berry/issues/1928