ext.js + ext-debug.js causes Tern to consume all CPU

[dangoor]

This is a followup to comments in #185.

I have created a gist containing just the two Ext files that, when combined, cause Tern to go out of control.

If I paste one or the other into a new file in the Tern demo page, everything works fine. The files are big, but Tern handles them well. If I add both to the page, Tern consumes all CPU and memory grows as well.

[marijnh]

Well, yes, Tern is not a 'throw code at it and it'll be fine' kind of tool. You loaded the equivalent of 46 000 lines of unminified code into it. That's beyond what it is expected to handle.

[dangoor]

I'm not actually sure that the size is the issue. I will see if I can identify a smaller case (we've had a number of reports of Brackets sprint 27 freezing up on javascript).

[dangoor]

I paused execution a few times and looked at where things were. I consistently saw Ext.apply as the scope, so I:

• formatted the minified ext.js so that I could reasonably monkey with it
• deleted most of the code in both ext-debug.js and ext.js

The problem is still reproducible in about 1500 lines of un-minified JS which I have placed in the gist.

Both files are using Ext.apply to modify the Ext object (putting the same functions on it, even).

[marijnh]

Thanks for narrowing this down. I can't currently access the gist for some reason, but I'll look into it once I can.

[dangoor]

I just tried it afresh and did some bisecting... and now I'm not seeing the behavior I saw before. Right now what I'm seeing is that it hits 100% CPU for a few seconds on my machine (a fairly recent macbook air), but then it finishes normally. So, maybe this code is hitting a corner case in performance but is not representative of an "out of control" situation after all.

[marijnh]

It can be further reduced to the following five lines:

window.clone = function(item) {
  for (var key in item) {};
  window.clone(item[i]);
  window.clone(item[i]);
};

Loading that twice takes a few seconds. If you duplicate the statements in the body a few times, you'll get an apparently exponential blowup of runtime. For some reason, using window.clone is important -- simply saying clone doesn't trigger the problem. Still digging to figure out what causes this. It most likely has to do with instantiated functions somehow being run recursively (though that's supposed to be impossible).

[marijnh]

Re your last message: since this looks exponential, I can easily imagine how it'd get into a situation where it appears to hang forever with the full codebase.

[dangoor]

Huh... that's an interesting reduction and seems generic enough to be a problem for other scripts. Do you have to have two files with that construction in it in order to hit the exponential runtime? Or is it just that because it's exponential, having two files with a similar construction causes a runtime blowup? The behavior definitely seemed to be tied to having both files present.

I'm working on reproducing another hanging case right now, and I'll keep your analysis mind if I'm able to successfully cause Brackets to hang with that case.

[marijnh]

Attached patch fixes the tiny case I reduced above, but not the full files. Trying to do another reduction.

[marijnh]

Reduced the new hang to 10 lines, but unfortunately 10 very weird lines.

window.extend = function(subclass, superclass) {
  superclass = subclass;
  subclass = function() { superclass.apply(this, arguments); };

  var F = function() {};
  F.prototype = superclass.prototype;
  subclass.prototype = new F();

  subclass.extend = function(o) { return window.extend(subclass, o); };
};

[marijnh]

The problem is a loop in the graph that goes through an IsProto constraint, causing an infinite expansion of instances that only terminates when it runs into the airbrake in withWorkList. It does this repeatedly, consuming a lot of memory and time in the process.

It isn't clear to me yet how to catch this case. Maybe keeping a history of propagations that caused the current addType call will be necessary -- such a thing would be useful for other things as well, and help debugging, but it'd be seriously expensive.

Will think about it some more.

[marijnh]

Okay, I now have it so that your last test file runs well. Please test!

[dangoor]

Works great! I pasted in all 33,000 lines (unminified) of the original ext.js and ext-debug.js files and Tern handled it well indeed.

Thanks for digging in to this and coming up with a solution so quickly!