servo: New test failures in test-perf

We started getting more test failures in test-perf, starting 2018-01-20. https://datastudio.google.com/u/0/reporting/1b16G5pQNp2lE-1nNcPlOure9G0eR36cq/page/heOK

screenshot-2018-2-26 servo test-perf

An example that used to pass is http:/localhost:8123/page_load_test/tp5n/en.wikipedia.org/en.wikipedia.org/wiki/Rorschach_test.html. To replicate this

  1. Grab the nightly builds http://servo-builds.s3.amazonaws.com/nightly/linux/2018-01-19T01:01:57Z-servo-tech-demo.tar.gz and http://servo-builds.s3.amazonaws.com/nightly/linux/2018-01-20T00:56:21Z-servo-tech-demo.tar.gz.
  2. Start a web server inside the etc/ci/performance directory, e.g. python -m SimpleHTTPServer 8123
  3. Point servo at http:/localhost:8123/page_load_test/tp5n/en.wikipedia.org/en.wikipedia.org/wiki/Rorschach_test.html.

The test fails nondeterministically, sometimes rendering and sometimes just producing a blank page.

There are no obvious culprits in the commits after 6fc71a7644bd7afcccf83f67cfdaf01897e1a3bc and before b62b51ab4b373b6b972d2fef11d0d314c11de362, which are the ones for Jan 19.

My suspicion is that this is a timing-sensitive deadlock, and that some commit on Jan 19 happened to change the timing such that the deadlock happens on CI.

My attempts to diagnose this haven’t borne any fruit. I did once get a panic:

$ ../../../target/release/servo-2018-01-20 http://localhost:8123/page_load_test/tp5n/en.wikipedia.org/en.wikipedia.org/wiki/Rorschach_test.html
assertion failed: *body == ResponseBody::Empty (thread fetch thread for http://localhost:8123/page_load_test/tp5n/en.wikipedia.org/bits.wikimedia.org/en.wikipedia.org/load.php, at components/net/fetch/methods.rs:438)
ERROR:servo: assertion failed: *body == ResponseBody::Empty
called `Result::unwrap()` on an `Err` value: "PoisonError { inner: .. }" (thread fetch worker thread, at libcore/result.rs:916)
ERROR:servo: called `Result::unwrap()` on an `Err` value: "PoisonError { inner: .. }"

which seems to be the same panic as #19707 but I don’t know if this is related. Most runs deadlock, they don’t panic.

IRC conversation with @jdm at https://mozilla.logbot.info/servo/20180226#c14363739

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 19 (19 by maintainers)

Most upvoted comments

Good news… on my laptop in the office, I can trigger this with a debug build! Yay, less stumbling around in the dark! A sample run ends with:

...
INFO:net::http_loader: response for http://localhost:8123/page_load_test/tp5n/en.wikipedia.org/upload.wikimedia.org/wikipedia/commons/thumb/4/4a/Commons-logo.svg/30px-Commons-logo.svg.png
INFO:net::http_loader:  - Server: SimpleHTTP/0.6 Python/2.7.12
INFO:net::http_loader:  - Date: Tue, 27 Feb 2018 20:02:32 GMT
INFO:net::http_loader:  - Content-type: image/png
INFO:net::http_loader:  - Content-Length: 1593
INFO:net::http_loader:  - Last-Modified: Thu, 14 Sep 2017 22:01:44 GMT
DEBUG:script::dom::window: script: performing reflow for reason RefreshTick
DEBUG:script::dom::window: script: layout forked
INFO:script::dom::window: script: waiting on layout

So deadlock is looking quite likely, Script is waiting on layout which never completes. I wonder if script is holding a lock while doing this?