keystone-classic: Keystone is crazy slow to start

This is an umbrella issue to track startup performance. It should be possible to get startup perf <100ms because there just isn’t that much code!

console.time('keystone');
var keystone = require('keystone');
console.timeEnd('keystone');

$ node server.js 
keystone: 3297.995ms

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 35 (25 by maintainers)

Most upvoted comments

Without knowledge of keystone’s internals, some obvious candidates for lazy / optional module loading:

lib/email (670ms)
browserify (612ms). The keystone admin UI javascript should be compiled in an npm postinstall script.

Alone those two changes would decrease startup time by >50% (to about 1 second on my mba).

josephg on May 12, 2016

Problem: Loading the Admin UI for the first time (after the server restarts) takes way too long (1min), and unfortunately slows down my productivity a lot.

Cause: fields.js and admin.js take > 1 min to be generated

Proof: Proof

Details: Lately, more than ever, this is something I have been struggling with. I don’t want to be picky about it because having an auto-generated admin UI, even if slowly generated at first, is far better than not having one and it’s the first reason for me to use KeystoneJS. But unfortunately, since I’me developing my biggest project so far, I have to take a lot of consideration into how the Admin UI displays the data and enables any user to manage it = which translates in restarting the server all the time and checking the changes.

Is it something I’m doing wrong? Any tips on how to decrease this, or is it already fixed and it’s waiting for a PR? I suppose it has to do with reading all the Models structure and translating to React.js components/etc (I have no experience with reactjs yet).

ghuroo on Jan 18, 2017

I’ve just deferred loading browserify and babelify until the first Admin UI script; that drops the startup time down another half a second on my machine. Getting there.

JedWatson on May 19, 2016

it should be enabled in 4.0.0-beta-5, so if you upgrade your keystone 4 app it should just work out of the box (it’s enabled by default). The first time you build it will still be slow, but it should cache the build between server restarts after that.

bassjacob on Jan 30, 2017

@mxstbr that #3035 issue will in fact be helped by a pure-data-schema driven UI approach, since all you need to do is amend the schema and the admin UI would follow suit.

wmertens on Jul 13, 2016

Oh, there’s also https://www.npmjs.com/package/postinstall-build obviating the need for checked-in builds for git installs.

wmertens on Jul 13, 2016

@wmertens @josephg a lot of the time spent here is actually babel, not browserify. Not to detract from the fact that I’d like to switch over to webpack, if we were to pre-transpile the fields and Admin UI it would:

make npm install lighter, babel and friends would move to devDependencies
reduce the standard keystone install size
make startup time faster
reduce memory usage
make the Admin UI load much faster on first use

… which is a lot of win, and a much easier target.

When we get 0.4 published to npm, we can use prepublish to do the build, and if you’re running linked from a git checkout we can have an npm script that does the build (including watchify for dev)

Until then though, when testing the alpha with your package.json pointed at git, to continue supporting that we’d need to have transpiled versions in our main repo, which is sucky. I’m not sure it’s more sucky than our current startup times, but it’ll introduce a lot of churn (we currently do this in the sense that I pre-build all our common packages and check that in when they’re updated in keystone’s package.json, but that happens a lot less frequently than keystone’s Admin UI is updated)

Basically this is a trade-off between speed to boot and “it works like you expect, first time, every time”.

Thoughts?

JedWatson on Jul 13, 2016

@wmertens 👍 Sounds great. Ideally I’d like to build that bundle in an npm prepublish script and then we can remove the dependancies on webpack / etc from keystone itself.

josephg on Jul 13, 2016

I should be able to trim at least 30% off of Juice, still working on it. When I eventually get to a release I’ll open a PR to Keystone.

jrit on May 26, 2016

@jrit We have to go deeper:

      ┃  ┣ juice (414ms)
      ┃  ┗ []
      ┃     ┣ ./lib/utils (143ms)
      ┃     ┣ []
      ┃     ┃  ┗ cheerio (124ms)
      ┃     ┗ web-resource-inliner (269ms)

I don’t know anything about juice or those libraries. If one (or both) is optional, it might be worth lazy-loading them.

There’s a pretty high timing variance between runs (sometimes its 250ms). YMMV.

(So to read that, almost all the time spent loading juice is spent in turn in juice’s dependancies.)

josephg on May 19, 2016