polymerfire: firebase-query is very slow when loading an array with more than just a few items
Description
I am loading a firebase array with around 12000 items, using a Polymer element like:
<link rel="import" href="/bower_components/polymer/polymer.html">
<link rel="import" href="/bower_components/polymerfire/firebase-query.html">
<dom-module id="loader">
<template>
<firebase-query id="query"
path="/data"
data="{{data}}">
</firebase-query>
</template>
<script>
(function () {
'use strict';
Polymer ({
is: 'loader',
properties: {
data: {
type: Object,
value: [],
notify: true
}
},
ready: function () {
var self = this;
setTimeout (function () {
console.log ('Data load took ' + (self.$.query.__loadEnd - self.$.query.__loadStart) + 'ms');
}, 60000);
}
});
}) ();
</script>
</dom-module>
The setTimeout () callback is for my instrumentation, which in the current version of firebase-query looks like:
--- firebase-query.html.orig 2017-04-28 12:13:59.221771214 +0100
+++ firebase-query.html.instr 2017-04-28 14:31:49.199146378 +0100
@@ -308,6 +308,10 @@
},
__onFirebaseChildAdded: function(snapshot, previousChildKey) {
+ if (!this.__loadStart) {
+ this.__loadStart = performance.now ();
+ }
+
var key = snapshot.key;
var value = snapshot.val();
var previousChildIndex = this.__indexFromKey(previousChildKey);
@@ -318,6 +322,7 @@
this.__map[key] = value;
this.splice('data', previousChildIndex + 1, 0, value);
+ this.__loadEnd = performance.now ();
},
__onFirebaseChildRemoved: function(snapshot) {
Expected outcome
My app should load the data quickly, without interrupting the user experience too severely.
Actual outcome
My instrumentation timings for several loads are (times in ms):
37324.61 37408.37 36633.60 37129.21 38887.08
Yes, that is nearly 40 seconds to load just 12000 items! Worse, the browser is completely frozen while this is happening.
Steps to reproduce
- Have a decent-sized array in Firebase
- Load it as above
- Try to interact with page elements while the load happens
Browsers Affected
- Chrome
- Firefox
- Safari 9
- Safari 8
- Safari 7
- Edge
- IE 11
- IE 10
I’ve tested with Chrome, but I doubt this is browser specific.
About this issue
- Original URL
- State: open
- Created 7 years ago
- Reactions: 1
- Comments: 34 (15 by maintainers)
The Firebase SDK definitely caches locally in-memory, so an on(‘value’) will start a listener and you can separately do on(‘child_added’). This will not incur double network cost.
On Fri, May 5, 2017 at 11:55 AM Toni-Jan Keith Monserrat < notifications@github.com> wrote:
It would be nice to see the limitations documented then, before anyone else runs into this.
@mikelehen thank you for the clarification regarding the
valueevent. In that case, I think that some minor changes to @dickp 's design could be sufficient to improve performance in a general way.Also the child-added handler invokes Polymer data-binding path notifications for the array each time.
I agree with @mbleigh 's assessment that the number of items being considered is unrealistic. Even with the speed up, there will be problems rendering this much data in the client for most non-trivial usages.
That said, seems 40s is pretty bad. It seems likely that the problem is at a higher level than the Firebase client library. However, it would be nice to confirm that with profiling.
The timings I posted at the top of this thread would beg to differ. 40 seconds using child_added on its own, including 10s of seconds of frozen browser UI, vs with value it’s 40 milliseconds to the last child_added event even if the data is loaded twice.
Whatever. The patch works for me, take it or leave it. If you don’t want a 1000x speed improvement it makes no difference to me.