chimp: Extreme slowdown over time

Expected behaviour

Tests should execute at a uniform pace in a reasonable period of time.

Actual behaviour

Tests begin running quickly, but slow down to several seconds per step towards the end of the test run.

Example of slow step:

this.Then(/^The page is (.+)$/i, function (arg1) {
        browser.waitForExist('.navbar', 5000);
        var path;
        if (/t(est)?\s?a(pp)?1/i.test(arg1)) path = /\/ta1/;
        if (/t(es(t)?)?\s?a(pp)?2/i.test(arg1)) path = /\/ta2/;
        if (/(login)/i.test(arg1)) path = /\/login/;
        expect(browser.getUrl()).toMatch(path);
    });

56 scenarios (56 passed)
836 steps (836 passed)
26m21.872s

Exact steps to reproduce / repository that demonstrates the problem

Please put long code snips in a Gist and provide a link here.

Version & tools:

Chimp command line used: ./node_modules/.bin/chimp
Chimp config file: None.
Chimp:

./node_modules/.bin/chimp -v
0.41.2

Node.js: 6.9.1
Java:

java -version
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Operation system:

uname -v
Darwin Kernel Version 16.1.0: Thu Oct 13 21:26:57 PDT 2016; root:xnu-3789.21.3~60/RELEASE_X86_64

OS X Sierra

Console / Log Output

Nothing except for passing steps. it is simply extremely slow.

Join our Slack xolv.io/community #chimp channel, where you can find help and help others.

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 1
Comments: 43 (14 by maintainers)

Most upvoted comments

It’s great to hear this issue seems to be resolved! As the maintainer of the meteor-promise package, I would humbly

apologize for the unfortunate interaction between that package and Node 6+, and
mention that the latest version of meteor-promise (0.8.4) not only fixes these problems itself but also eliminates the problem from any other (older) copies of meteor-promise that might be accidentally in use.

benjamn on May 16, 2017

Hey guys, I believe this is finally fixed with the version I just released 0.49.0 . I tested it on v7.9.0 node. Switching back to 0.48.0 makes it slow again. Please update and confirm.

The meteor-promise turned out to indeed be the culprit, I had to update cucumber-js and sync-webdriverio accordingly.

Thanks @corwin-of-amber @benjamn @Sanjo @And-re @lucetius @andrialexandrou @samhatoum for help with debugging, and @tiagolr @schovi @viditm @woutervandergraaf @timcosta @workflow for reporting and providing details. ( I hope I haven’t missed anyone ) Phew! This took us long enough!

lgandecki on May 12, 2017

@lgandecki

Node 7.10 and still gets one TypeError: Cannot read property 'trim' of undefined error
About speed it is awesome! It took 1/4 time than before. Totally progress! 😃

schovi on May 12, 2017

Morning all,

I am getting into similar issues, I’ve not used chimp.js before but is creating a negative perception for the business. Our test suite takes over an hour to execute. any help would be appreciated. 64 Scenario took 1h 5m 12s 065ms to complete. Thank you

code-plumber on Dec 2, 2016

@lgandecki any luck on this issue yet?

timcosta on Jan 8, 2017

93ae4baa31e98be52e9f2b325a3e19b3753e7ffdfc5517bcb7264659435119e0

workflow on Dec 9, 2016

Thanks a lot! I will have a pair working on this starting today… we will give you update by the end of the day

lgandecki on Dec 8, 2016

Guys, any update on this? This is making chimp unusable, and increasing the number of flickers we’re seeing exponentially.

Thank you!

workflow on Dec 7, 2016

So far we know that the version 0.31.0 works fast if you disable the screenshots. We’ve also tried updating meteor/promise in cucumberjs and sync-webdriverio to the newest stable version - according to the article @Sanjo suggested: https://sandstorm.io/news/2016-09-30-fiber-bomb-debugging-story?utm_source=nodeweekly&utm_medium=email

But it didn’t really change anything (or the change was small but didn’t fix the main issue). Will keep investigating tomorrow if time allows.

lgandecki on Nov 8, 2016

@samhatoum @tjsail33 I managed to reproduce that problem in this repo: https://github.com/And-re/chimp-issue

And-re on Nov 8, 2016