polymer: Performance: dom-repeat does not perform on large datasets

When using dom-repeat with large datasets the browser becomes unresponsive for a period of time while the render task is being completed. The <iron-list> component I believe was designed to solve this. However there are use cases that <iron-list> does not support (a grid view until that is implemented, or any other UI where repeated item position is calculated/randomized, etc).

I have thrown together a demo on github (was going to do a fiddle but the dummy api I am calling causes the request to throw a NET::ERR_CERT_COMMON_NAME_INVALID error)

http://jshcrowthe.github.io/dom-repeat-perf/non-chunked.html

In the following timeline you can see that there is an long, event-loop blocking, render task that takes place immediately after the request completes

NOTE: My machine is a 2013 MacBook Pro 2.3 GHz i7 with 16 GB RAM

If you repeatedly click on the counter button while the request is being made you will notice that, up until the XHR request completes, the UI is responsive. However once the render task for the dom-repeat begins the subsequent click events are queued up and are unable to process until the full render has completed.

I hacked around with chunking the rendering task (utilizing setTimeout(function() {...}, 0)) and was able to create a functional proof of concept that illustrates a potential solution. A demo can be found at the following URL:

http://jshcrowthe.github.io/dom-repeat-perf/chunked.html

Before I go too much further with this I wanted to get feedback to see if this was a good potential solution or if I am altogether going about this wrong. You can see the actual change I made here:

https://github.com/jshcrowthe/polymer/commit/a5d5edd930db3dba873056261e0234acb08f9ce9

NOTE: I have only modified the _applyFullRefresh render method for the sake of demo. If this is a good solution all 3 of the render types would probably want to have this type of logic. In addition to that a way to abort an “in progress” render would need to be implemented to ensure that multiple calls to _applyFullRefresh don’t result in duplication. You could also implement this type of chunking with DOM removals if it proves to be similarly slow.

If you compare the timeline for the chunked render you can see a drastic change in the timeline. The UI is not rendered all at one time however the render task never takes so long (at least in this simple instance) where it blocks user interaction.