Repository navigation
TSC Regression causing crash (Heap OOM) between version 4.8.4 & 4.9.3 in large project #53087
Description
Activity
FYI: You can try https://github.com/microsoft/TypeScript/wiki/Performance-Tracing, it helped us locate the problem in #50515.
Reacted by Noman GulFYI: You can try https://github.com/microsoft/TypeScript/wiki/Performance-Tracing, it helped us locate the problem in #50515.
I will attempt this as soon as i have recruited someone with enough ram to run it without crashing :)
4.8.4 Diagnostic information:
Files: 1023 Lines of Library: 28146 Lines of Definitions: 218302 Lines of TypeScript: 25936 Lines of JavaScript: 0 Lines of JSON: 0 Lines of Other: 0 Nodes of Library: 118224 Nodes of Definitions: 564026 Nodes of TypeScript: 115815 Nodes of JavaScript: 0 Nodes of JSON: 0 Nodes of Other: 0 Identifiers: 301992 Symbols: 865373 Types: 208799 Instantiations: 365698 Memory used: 909729K Assignability cache size: 27600 Identity cache size: 4422 Subtype cache size: 7237 Strict subtype cache size: 9428 Tracing time: 0.17s I/O Read time: 0.06s Parse time: 0.96s ResolveModule time: 0.08s ResolveTypeReference time: 0.00s Program time: 1.27s Bind time: 0.62s Check time: 2.85s printTime time: 0.00s Emit time: 0.00s Dump types time: 9.03s Total time: 4.74s5.1.0-dev.20230305 Diagnostic information:
Files: 1029 Lines of Library: 29397 Lines of Definitions: 218302 Lines of TypeScript: 25916 Lines of JavaScript: 0 Lines of JSON: 0 Lines of Other: 0 Identifiers: 302941 Symbols: 6242486 Types: 702542 Instantiations: 417693 Memory used: 5496499K Assignability cache size: 31947 Identity cache size: 4714 Subtype cache size: 5307 Strict subtype cache size: 7579 Tracing time: 0.08s I/O Read time: 0.04s Parse time: 0.94s ResolveModule time: 0.09s ResolveTypeReference time: 0.00s Program time: 1.19s Bind time: 0.49s Check time: 57.68s printTime time: 0.00s Emit time: 0.00s Dump types time: 7.27s Total time: 59.37sNotably there is an about 5x increase in memory usage, as well as a >5x increase in the "Types" and "Symbols".
Now that I can actually build on v10 there several new errors, most interesting among them the following:
src/module/item/license-sheet.ts:37:23 - error TS2590: Expression produces a union type that is too complex to represent.The line is ```TS
for (let d of docs as LancerItem[])// Find the assoc frame for (let et of [EntryType.FRAME, EntryType.MECH_SYSTEM, EntryType.MECH_WEAPON, EntryType.WEAPON_MOD]) { let pack = game.packs.get(`world.${et}`); if (pack) { let docs = await pack.getDocuments({ "system.license": license.system.key }); for (let d of docs as LancerItem[]) { let rank = (d as any).system.license_level as number; while (unlocks.length <= rank) { unlocks.push([]); } unlocks[rank].push(d); } } }Where
docsis of type:
docs: StoredDocument<LancerActor | Scene | LancerItem | JournalEntry | Macro | Playlist | RollTable>[].
Here's where "it's a big project" sort of bites us, in that I can't really minimally represent these types here in this comment. Each is a unique class, all with a shared parent class.Maybe this is indicative of large union types being the root of our problem?
...
Observing the trace, the file
src/module/comp-builder.tsis responsible for an entire 53 SECONDS of the total 57 second Check Time. It produces no errors.
Unsure how to proceed from here.
i had a quite similar issue recently after upgrading typescript. i can't remember the details, but it seems the problem was that under some circumstances tsc started creating a lot of types internally when asked to infer the type of a (typewise) complicated function. in the end, writing
const foo:FunctionType = (it) => ...instead ofconst foo = (it:FunctionArgs):FunctionReturn => ...solved the issue. maybe something similar is happening here?Ran into this issue myself today on a large project.
Similar regression on 5.0.2. For me, 4.8.3 is ok.
thanks for the exceptional bug report Jade Henry (@whitespine)!
Reacted by Melroy van den BergI was able to solve the OOM issue for myself with
NODE_OPTIONS=--max-old-space-size=8192 tsc
Reacted by Karl Horky, Dawson Botsford, Bertrand Bordage and earonestyReacted by Karl Horky and Melroy van den BergSince Jake Verbaten (@Raynos) solution is really just a stop gap. What is needed to further this ticket. We also have a large project that works fine on
4.8.4but cant upgrade to 4.9 or 5.0 because tsc gets an OOM errorHere is what I have gathered from running some test.
Test Ran with
NODE_OPTIONS=--max-old-space-size=8192 tsc --noEmit --diagnostics --extendedDiagnostics --incremental false
Here is 4.8Files: 2979 Lines of Library: 33311 Lines of Definitions: 233669 Lines of TypeScript: 59133 Lines of JavaScript: 0 Lines of JSON: 943 Lines of Other: 0 Nodes of Library: 148246 Nodes of Definitions: 723896 Nodes of TypeScript: 250983 Nodes of JavaScript: 0 Nodes of JSON: 2256 Nodes of Other: 0 Identifiers: 382980 Symbols: 597111 Types: 216450 Instantiations: 7252666 Memory used: 684086K Assignability cache size: 202376 Identity cache size: 3213 Subtype cache size: 12303 Strict subtype cache size: 4719 I/O Read time: 0.11s Parse time: 1.95s ResolveModule time: 0.58s ResolveTypeReference time: 0.01s Program time: 3.04s Bind time: 1.00s Check time: 16.47s printTime time: 0.00s Emit time: 0.00s Total time: 20.50sHere is 4.9
Files: 2980 Lines of Library: 33853 Lines of Definitions: 233669 Lines of TypeScript: 59133 Lines of JavaScript: 0 Lines of JSON: 943 Lines of Other: 0 Identifiers: 384065 Symbols: 571900 Types: 18112640 Instantiations: 7055141 Memory used: 5726426K Assignability cache size: 10914208 Identity cache size: 3215 Subtype cache size: 11958 Strict subtype cache size: 4719 I/O Read time: 0.15s Parse time: 1.72s ResolveModule time: 0.47s ResolveTypeReference time: 0.01s Program time: 2.65s Bind time: 0.79s Check time: 199.89s printTime time: 0.00s Emit time: 0.00s Total time: 203.33sI also generated a trace
From looking into the trace. It seems to be stuck in a loop related to the
i18next&react-i18nextThis is basically the part that loops (if I followed the trace right)
import type { Namespace, TFuncKey } from 'i18next'; import { i18n } from './i18n'; export interface I18nText<N extends Namespace = Namespace> { namespace: N; keyName: TFuncKey<N>; // <-- TFuncKey seems to be the culprit }
That lib is for sure complicated typewise. But it worked as it should in 4.8 and breaks compilation in 4.9 and 5.0
Anything else I can provide? Jake Bailey (@jakebailey)
Reacted by Dean KertaiWe've made some improvements recently that feel like this issue; can you try the nightly build?
Reacted by Jarek Toro29 remaining items
Dominik Urban (@ashylen) unfortunately the latest versions of i18next didn't resolve the issue for me. See reproducible example here i18next/i18next#1921 (comment)
Still getting this issue. I'm getting the following error:
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memorywhen running
tsc --pretty --noEmiteven if I set the space size like the following:cross-env NODE_OPTIONS=--max-old-space-size=10240 tsc --pretty --noEmitReacted by Brian CooperTypescript compiler
4.8.4works, I downgraded to build my projectReacted by Melroy van den BergThis thread has lots and lots of people with likely different issues; I've recently published a tool which makes it easy to bisect issues in TypeScript without knowing how to build TypeScript itself: https://www.npmjs.com/package/every-ts#bisecting
It'd be extremely helpful for those who know which versions do and don't work to use this tool to identify which change in particular broke things; doing so will allow us to categorize these issues.
We have the same issue as original and it is also with
i18next, it's been blocking us from upgrading the library for almost a year :/Reacted by earonestyDo you have a repo that can be cloned and tested consistently?
Hi, I didn't manage to create a repo reproducing the error but I have found a fix for the issue in i18next, I will report back there with a PR in 2-3 weeks if everything has worked as expected in the meantime, and link it here so that you can analyse wether it's something ts could get better at.
Basically the issue has to do with values and inferences within conditional types being computed when they shouldn't (moving a ternary up the type stack and doing some type caching with const fixed the issue (not sure what I say strictly makes sense as I'm a noob in typing jargon but you get the idea)).Reacted by Karl HorkyReacted by Karl HorkyIn the meantime here is what seems to do the trick, in case someone wants to have a look i18next/i18next@master...GreenGoTech:i18next:master
Reacted by Karl HorkyFYI: You can try https://github.com/microsoft/TypeScript/wiki/Performance-Tracing, it helped us locate the problem in #50515.
doing this helped me trace the actual problem. I had some recursive types in my code that were causing this issue.
Bug Report
When updating typescript within our project from version 4.7.3 to a more recent version, our project began experiencing OOM issues.
Specifically, it is completely unable to build via
tscor eventsc --noEmit.Ours is a somewhat large project, so I couldn't point to a minimal reproduction (sorry!).
I have 16GB of RAM so I doubt its a "classic" memory error, seems more likely something started infinitely using memory during a version upgrade.
🔎 Search Terms
tsc 4.9 4.9.5 Out of Memory Heap Out of Memory Crash Killed Silent Error
(Am i doing this right?)
🕗 Version & Regression Information
Error began occuring when updating from typescript 4.7.3 to any 4.9.3^ version.
Issue persists into Version 5.1.0-dev.20230303 (nightly at time of testing).
Issue is NOT present in v4.8.4, or any prior version we tried (that satisfied our dependencies, anyways.)
💻 Code
Branch state is: https://github.com/Eranziel/foundryvtt-lancer/tree/oom
Was unable to produce a minimally viable reproduction, as the OOM error gives next to no context as to where the issue occurs or why. Sorry. More than willing to provide more investigative information, but I don't really know what there is to provide.
🙁 Actual behavior
It crashes on more recent versions for no clear reason.
🙂 Expected behavior
I'd rather it didn't.