components: Update Schematics to version 8 fails with Out-of-Memory exception
Reproduction
Steps to reproduce: 0. Start with a “large” Angular v7 workspace
- Update
@angular/core
etc to v8 - Run
ng update @angular/material
“Large” is a relative term, of course. We’re using a monorepo style workspace (using the Nrwl’s NX tools) for our Angular Applications. We have 390 *.component.ts
and *.directive.ts
files.
Expected Behavior
The update to successfully run the upgrade schematics.
Actual Behavior
After it installs the @angular/material
and @angular/cdk
packages this is displayed to the terminal:
** Executing migrations for package '@angular/cdk' **
<--- Last few GCs --->
[15728:0000017D08BC7A90] 978327 ms: Mark-sweep 2016.3 (2069.9) -> 2000.2 (2069.9) MB, 772.3 / 0.0 ms (average mu = 0.273, current mu = 0.240) allocation failure scavenge might not succeed
[15728:0000017D08BC7A90] 979306 ms: Mark-sweep 2016.2 (2069.9) -> 2000.4 (2070.4) MB, 720.2 / 0.0 ms (average mu = 0.269, current mu = 0.264) allocation failure scavenge might not succeed
<--- JS stacktrace --->
==== JS stack trace =========================================
0: ExitFrame [pc: 00007FF600E82C16]
Security context: 0x03f554a193e1 <JSObject>
1: createScanner [000003B48610A701] [C:\git\Daktronics-Web-Platform\node_modules\typescript\lib\typescript.js:~7066] [pc=000002E34F358EE6](this=0x01dc41080509 <Object map = 0000028D62B90EE9>,1,0x02b136100709 <false>,0,0x03ce79b0e971 <String[425]\: `.control-container {\r\n background: linear-gradient(var(--background-standard), var(--background-standa...
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Writing Node.js report to file: report.20190529.132332.15728.0.001.json
Node.js report completed
1: 00007FF60033674F napi_wrap+120927
2: 00007FF6002DFE26 uv_loop_fork+44550
3: 00007FF6002E0CC6 uv_loop_fork+48294
4: 00007FF600618BCE private: void __cdecl v8::internal::Parser::ReportMessageAt(struct v8::internal::Scanner::Location,enum v8::internal::MessageTemplate,class v8::internal::AstRawString const * __ptr64,enum v8::internal::ParseErrorType) __ptr64+206
5: 00007FF6005ECC4A public: void __cdecl v8::internal::wasm::StreamingDecoder::NotifyCompilationEnded(void) __ptr64+666
6: 00007FF60068309C public: class v8::internal::HeapObject __cdecl v8::internal::Heap::CreateFillerObjectAt(unsigned __int64,int,enum v8::internal::ClearRecordedSlots,enum v8::internal::ClearFreedMemoryMode) __ptr64+2972
7: 00007FF60068AE0F public: void __cdecl v8::internal::LocalEmbedderHeapTracer::NotifyV8MarkingWorklistWasEmpty(void) __ptr64+5375
8: 00007FF600681483 public: bool __cdecl v8::internal::Heap::CollectGarbage(enum v8::internal::AllocationSpace,enum v8::internal::GarbageCollectionReason,enum v8::GCCallbackFlags) __ptr64+1235
9: 00007FF60067FB84 public: void __cdecl v8::internal::GCTracer::AddScopeSample(enum v8::internal::GCTracer::Scope::ScopeId,double) __ptr64+1316
10: 00007FF6006C07B3 public: class v8::internal::Handle<class v8::internal::HeapObject> __cdecl v8::internal::Factory::NewFillerObject(int,bool,enum v8::internal::AllocationSpace) __ptr64+67
11: 00007FF6009E76E6 public: void __cdecl v8::internal::OptimizingCompileDispatcher::Unblock(void) __ptr64+116966
12: 00007FF600E82C16 public: static class v8::internal::Vector<char const > __cdecl v8::internal::NativesCollection<0>::GetScriptsSource(void)+662454
I uploaded the report.json to a Gist here.
This ends up running for a long time. The NodeJS process continually increases memory usage until it crosses the 2GB threshold and then fails out. Unfortunately I can’t share link to a reproducing repository.
I also tried running this on an old Mac laptop we have in the office (late 2011), and it failed with a similar Out of Memory error.
Since I can’t upload a reproducing repo, please let me know if there’s any other information that can help!
Environment
Angular CLI: 8.0.0 Node: 12.3.1 OS: win32 x64 Angular: 8.0.0 … animations, cdk, cli, common, compiler, compiler-cli, core … forms, language-service, material, platform-browser … platform-browser-dynamic, router
Package Version
@angular-devkit/architect 0.800.0 @angular-devkit/build-angular 0.800.0 @angular-devkit/build-ng-packagr 0.800.0 @angular-devkit/build-optimizer 0.800.0 @angular-devkit/build-webpack 0.800.0 @angular-devkit/core 8.0.0 @angular-devkit/schematics 8.0.0 @ngtools/json-schema 1.1.0 @ngtools/webpack 8.0.0 @schematics/angular 8.0.0 @schematics/update 0.800.0 ng-packagr 5.2.0 rxjs 6.5.2 typescript 3.4.5 webpack 4.30.0
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 20 (8 by maintainers)
Closing as we reworked ng-update to no longer use
tslint
. I would be interested in hearing how the update now goes for big applications. Thanks!@devversion Thank you for prioritizing this! I ended up writing my own migration script that I was able to use for both @angular/material and ngrx-data.
Also as mentioned in previous posts. TypeScript
v3.4
had significant performance issues and upgrading toTypeScript v3.5
before running the migrations will also help with the OOM exceptions.Increasing the memory limit let the schematics finish out.
To save people a click, I ran this command:
I watched it run for about 1.5 hours before I had to leave at the end of the day. The node.exe process topped out around 5.7GB at one point, and finished with 145 files modified.
Since my issue is solved I’m fine closing out this bug - seems like this is more of an issue with angular/cli?
Here is the PR that reworks
ng update
to no longer usetslint
. https://github.com/angular/components/pull/16433@specnazzz Yeah there is still a massive overhead with tslint as it re-runs all enabled rules once replacements for a source file have been applied. This is still a significant issue if you have ~6000 typescript files…
We actually also decided to prioritize this issue and we’ll be working on switching away from
tslint
as soon as possible. The goal is to make the upgrade as seamless as possible. Especially for big projects!@devversion the first time around I didn’t properly upgraded to
typescript 3.5
so it was still usingtypescript 3.4
and was running for 14 hours and was still stuck on the line** Executing migrations for package '@angular/cdk' **
. The second time I properly upgraded totypescript 3.5
and it finally upgraded @angular/material “only” after 4.5 hours. Both times I removed theupdateProgram
function call, so I think it’s safe to say that removing this function didn’t help. If the issue istslint
then,tslint
is adding a massive amount of overhead for changing just imports.I don’t know if my issue is related to this issue, because I don’t get the OOM exception (maybe because I already increased the
--max_old_space_size
option before running it), but when runningng update @angular/material
I’m always stuck on the** Executing migrations for package '@angular/cdk' **
line and was stopping it after about an hour. Based on @devversion comments, maybe it’s just slow and I should have left it running longer. We have around 6000 typescript files.@devversion I did the changes you proposed (e.g. upgrade to
typescript 3.5
and removed theupdateProgram
call fromlinter.js
) and run theng update @angular/material
again. Will let it run through the night to see if it completes. I’m running it on a Windows machine.The workaround is to increase the memory https://github.com/angular/components/issues/16154#issuecomment-497335439 unless there are some other known issues with this? I brought this up multiple times in our team meeting but we decided that it’s not a priority for us to switch away from TSLint right now as the current focus is other stuff (see
README.md
). I wish we could switch away from TSLint as soon as possible but it just involves a lot of work and we decided to not prioritize this right now since there is a workaround.There are a couple of issues open in @angular/cli regarding memory issues. Try to increase the memory limit for node like here angular/cli#14543