servo: "Too many open files" crash after loading many small test cases via webdriver
Built from git: 7adf022 on 20190724. Running on Ubuntu 16.04.2 LTS with Radeon HD 7770.
I have a test system (for my PhD dissertation) that loads a bunch of smallish test cases repeatedly via webdriver on multiple browsers (currently Chrome, Firefox and Servo). I can trigger a “Too many open files” panic in Servo after enough test cases have been loaded. I’m pretty sure I can reproduce this fairly easily by running long enough (I believe this is third time I’ve hit it today). I’m including a crash dump below but is anything additional I can do that would be particularly helpful for debugging this?
Here is an example that crashed after the 587th test case was loaded:
servo-20190724.git $ ./mach run --release -z --webdriver=7002 --resolution=400x300
VMware, Inc.
softpipe
3.3 (Core Profile) Mesa 18.3.0-devel
[2019-08-01T03:03:00Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/3.html:1:2 fields are not currently supported
[2019-08-01T03:03:03Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/4.html:1:2 fields are not currently supported
[2019-08-01T03:03:07Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/5.html:1:2 fields are not currently supported
[2019-08-01T03:03:17Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/8.html:1:2 fields are not currently supported
[2019-08-01T03:03:20Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/9.html:1:2 fields are not currently supported
[2019-08-01T03:03:33Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/13.html:1:2 fields are not currently supported
[2019-08-01T03:03:43Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/16.html:1:2 fields are not currently supported
[2019-08-01T03:03:52Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/19.html:1:2 fields are not currently supported
[2019-08-01T03:04:02Z ERROR script::dom::bindings::error] Error at http://127.0.0.1:3000/gen/19183-2-34/22.html:1:2 fields are not currently supported
Could not stop player Backend("Missing dependency: playbin")
called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: "Too many open files" } (thread ScriptThread PipelineId { namespace_id: PipelineNamespaceId(1), index: PipelineIndex(2) }, at src/libcore/result.rs:1051)
stack backtrace:
0: servo::main::{{closure}}::h1a74bd812d203c72 (0x55f41009633f)
1: std::panicking::rust_panic_with_hook::h0529069ab88f357a (0x55f413330105)
at src/libstd/panicking.rs:481
2: std::panicking::continue_panic_fmt::h6a820a3cd2914e74 (0x55f41332fba1)
at src/libstd/panicking.rs:384
3: rust_begin_unwind (0x55f41332fa85)
at src/libstd/panicking.rs:311
4: core::panicking::panic_fmt::he00cfaca5555542a (0x55f4133527ec)
at src/libcore/panicking.rs:85
5: core::result::unwrap_failed::h4239b9d80132a0db (0x55f4133528e6)
at src/libcore/result.rs:1051
6: script::stylesheet_loader::StylesheetLoader::load::he3c6d2f4f6b2d86d (0x55f410cc94b4)
7: script::dom::htmllinkelement::HTMLLinkElement::handle_stylesheet_url::h08d096fd325f038f (0x55f410da0119)
8: <script::dom::htmllinkelement::HTMLLinkElement as script::dom::virtualmethods::VirtualMethods>::bind_to_tree::h4b49aadc206a8bb4 (0x55f410d9f9d5)
9: script::dom::node::Node::insert::h5f83017d8cd2b0c4 (0x55f410e5f494)
10: script::dom::node::Node::pre_insert::hc92be38a4a5c65ff (0x55f410e5e7d2)
11: script::dom::servoparser::insert::hc2bdfe4c53d2659e (0x55f410c611de)
12: html5ever::tree_builder::TreeBuilder<Handle,Sink>::insert_element::h483a7708bc71a57c (0x55f410fe59cd)
13: html5ever::tree_builder::TreeBuilder<Handle,Sink>::step::h53264c73d52d73b7 (0x55f410ffaed7)
14: <html5ever::tree_builder::TreeBuilder<Handle,Sink> as html5ever::tokenizer::interface::TokenSink>::process_token::h8d8307514ab0ae38 (0x55f410fb0653)
15: _ZN9html5ever9tokenizer21Tokenizer$LT$Sink$GT$13process_token17h75019972a696a3eeE.llvm.7363365923994120909 (0x55f41062d408)
16: html5ever::tokenizer::Tokenizer<Sink>::emit_current_tag::h45460aa41ef12f15 (0x55f41062daf1)
17: html5ever::tokenizer::Tokenizer<Sink>::step::h866e9b63f9f503ff (0x55f41063dcf1)
18: _ZN9html5ever9tokenizer21Tokenizer$LT$Sink$GT$3run17h8ce04a69e91aa671E.llvm.7363365923994120909 (0x55f41063194a)
19: script::dom::servoparser::html::Tokenizer::feed::h9fdf5468e43321e7 (0x55f41065532a)
20: script::dom::servoparser::ServoParser::do_parse_sync::hed9498713ee0e8aa (0x55f410c5cb3e)
21: profile_traits::time::profile::h4e1d5a89a249ef96 (0x55f410e1b2c6)
22: _ZN6script3dom11servoparser11ServoParser10parse_sync17h88e07764d212f834E.llvm.530561247764639767 (0x55f410c5c72f)
23: <script::dom::servoparser::ParserContext as net_traits::FetchResponseListener>::process_response_chunk::h61914a25d0137f92 (0x55f410c60885)
24: script::script_thread::ScriptThread::handle_msg_from_constellation::h600baf3383c41b0e (0x55f410caa068)
25: _ZN6script13script_thread12ScriptThread11handle_msgs17haf4ee7b3baaa2b69E.llvm.530561247764639767 (0x55f410ca488d)
26: profile_traits::mem::ProfilerChan::run_with_memory_reporting::hce1ba89d5eb53e5e (0x55f410e1a1e7)
27: std::sys_common::backtrace::__rust_begin_short_backtrace::hc93943e85aa6fa26 (0x55f411198e52)
28: _ZN3std9panicking3try7do_call17hb93b842dfb1f8343E.llvm.11073267061158468562 (0x55f411245103)
29: __rust_maybe_catch_panic (0x55f413339db9)
at src/libpanic_unwind/lib.rs:82
30: core::ops::function::FnOnce::call_once{{vtable.shim}}::h7217876d16e085d4 (0x55f410a082b2)
31: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h10faab75e4737451 (0x55f41331e8ee)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
32: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1aad66e3be56d28c (0x55f4133390df)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
std::sys_common::thread::start_thread::h9a8131af389e9a10
at src/libstd/sys_common/thread.rs:13
std::sys::unix::thread::Thread::new::thread_start::h5c5575a5d923977b
at src/libstd/sys/unix/thread.rs:79
33: start_thread (0x7fab3e4cc6b9)
34: clone (0x7fab3cd6841c)
35: <unknown> (0x0)
[2019-08-01T04:24:22Z ERROR servo] called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: "Too many open files" }
Pipeline script content process shutdown chan: Os { code: 24, kind: Other, message: "Too many open files" } (thread Constellation, at src/libcore/result.rs:1051)
stack backtrace:
0: servo::main::{{closure}}::h1a74bd812d203c72 (0x55f41009633f)
1: std::panicking::rust_panic_with_hook::h0529069ab88f357a (0x55f413330105)
at src/libstd/panicking.rs:481
2: std::panicking::continue_panic_fmt::h6a820a3cd2914e74 (0x55f41332fba1)
at src/libstd/panicking.rs:384
3: rust_begin_unwind (0x55f41332fa85)
at src/libstd/panicking.rs:311
4: core::panicking::panic_fmt::he00cfaca5555542a (0x55f4133527ec)
at src/libcore/panicking.rs:85
5: core::result::unwrap_failed::h4239b9d80132a0db (0x55f4133528e6)
at src/libcore/result.rs:1051
6: constellation::pipeline::Pipeline::spawn::hc4537e426638cac3 (0x55f4101814af)
7: constellation::constellation::Constellation<Message,LTF,STF>::new_pipeline::h03e845e5b2cd3635 (0x55f410196bdf)
8: constellation::constellation::Constellation<Message,LTF,STF>::handle_panic::h9c59071776586bee (0x55f41019574b)
9: constellation::constellation::Constellation<Message,LTF,STF>::handle_log_entry::hff41c6262b0a65ca (0x55f410198de1)
10: constellation::constellation::Constellation<Message,LTF,STF>::handle_request_from_compositor::he6733721600a19c9 (0x55f4101ae38d)
11: constellation::constellation::Constellation<Message,LTF,STF>::run::hc89c0bcdfed723c1 (0x55f4101b64d9)
12: std::sys_common::backtrace::__rust_begin_short_backtrace::he8170c3b0c1833bc (0x55f41020014f)
13: _ZN3std9panicking3try7do_call17h6580004dd9fdb22cE.llvm.4941401487006519239 (0x55f4100db513)
14: __rust_maybe_catch_panic (0x55f413339db9)
at src/libpanic_unwind/lib.rs:82
15: core::ops::function::FnOnce::call_once{{vtable.shim}}::hd78ccb91aec8cf3a (0x55f4100efe12)
16: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h10faab75e4737451 (0x55f41331e8ee)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
17: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1aad66e3be56d28c (0x55f4133390df)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
std::sys_common::thread::start_thread::h9a8131af389e9a10
at src/libstd/sys_common/thread.rs:13
std::sys::unix::thread::Thread::new::thread_start::h5c5575a5d923977b
at src/libstd/sys/unix/thread.rs:79
18: start_thread (0x7fab3e4cc6b9)
19: clone (0x7fab3cd6841c)
20: <unknown> (0x0)
[2019-08-01T04:24:23Z ERROR servo] Pipeline script content process shutdown chan: Os { code: 24, kind: Other, message: "Too many open files" }
called `Result::unwrap()` on an `Err` value: RecvError (thread LayoutThread PipelineId { namespace_id: PipelineNamespaceId(1), index: PipelineIndex(695) }, at src/libcore/result.rs:1051)
stack backtrace:
0: servo::main::{{closure}}::h1a74bd812d203c72 (0x55f41009633f)
1: std::panicking::rust_panic_with_hook::h0529069ab88f357a (0x55f413330105)
at src/libstd/panicking.rs:481
2: std::panicking::continue_panic_fmt::h6a820a3cd2914e74 (0x55f41332fba1)
at src/libstd/panicking.rs:384
3: rust_begin_unwind (0x55f41332fa85)
at src/libstd/panicking.rs:311
4: core::panicking::panic_fmt::he00cfaca5555542a (0x55f4133527ec)
at src/libcore/panicking.rs:85
5: core::result::unwrap_failed::h4239b9d80132a0db (0x55f4133528e6)
at src/libcore/result.rs:1051
6: layout_thread::LayoutThread::start::hb675360933e398d4 (0x55f41047e063)
7: profile_traits::mem::ProfilerChan::run_with_memory_reporting::h8a534464fbe2083e (0x55f4104f3d23)
8: std::sys_common::backtrace::__rust_begin_short_backtrace::hab512255cc628b7e (0x55f4105a9144)
9: _ZN3std9panicking3try7do_call17h15789f162d15d069E.llvm.197603739779113467 (0x55f410513fd3)
10: __rust_maybe_catch_panic (0x55f413339db9)
at src/libpanic_unwind/lib.rs:82
11: core::ops::function::FnOnce::call_once{{vtable.shim}}::h1b24f55261127c94 (0x55f41053f242)
12: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h10faab75e4737451 (0x55f41331e8ee)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
13: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1aad66e3be56d28c (0x55f4133390df)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
std::sys_common::thread::start_thread::h9a8131af389e9a10
at src/libstd/sys_common/thread.rs:13
std::sys::unix::thread::Thread::new::thread_start::h5c5575a5d923977b
at src/libstd/sys/unix/thread.rs:79
14: start_thread (0x7fab3e4cc6b9)
15: clone (0x7fab3cd6841c)
16: <unknown> (0x0)
[2019-08-01T04:24:23Z ERROR servo] called `Result::unwrap()` on an `Err` value: RecvError [151/362]
called `Option::unwrap()` on a `None` value (thread ScriptThread PipelineId { namespace_id: PipelineNamespaceId(1), index: PipelineIndex(2) }, at src/libcore/option.rs:378)
stack backtrace:
0: servo::main::{{closure}}::h1a74bd812d203c72 (0x55f41009633f)
1: std::panicking::rust_panic_with_hook::h0529069ab88f357a (0x55f413330105)
at src/libstd/panicking.rs:481
2: std::panicking::continue_panic_fmt::h6a820a3cd2914e74 (0x55f41332fba1)
at src/libstd/panicking.rs:384
3: rust_begin_unwind (0x55f41332fa85)
at src/libstd/panicking.rs:311
4: core::panicking::panic_fmt::he00cfaca5555542a (0x55f4133527ec)
at src/libcore/panicking.rs:85
5: core::panicking::panic::h73f4a74a29ff704a (0x55f41335272b)
at src/libcore/panicking.rs:49
6: <script::dom::window::Window as script::dom::bindings::codegen::Bindings::WindowBinding::WindowBinding::WindowMethods>::Frames::h70e9762199220474 (0x55f410ab3346)
7: <script::dom::htmlmediaelement::HTMLMediaElement as core::ops::drop::Drop>::drop::h82f851c26fc8d1fe (0x55f4107912f6)
8: _ZN4core3ptr18real_drop_in_place17hb15edba9f2627c0eE.llvm.9711593096204109679 (0x55f410a27005)
9: _ZN3std9panicking3try7do_call17hf7bcf8d4fe880bf2E.llvm.11073267061158468562 (0x55f411258803)
10: __rust_maybe_catch_panic (0x55f413339db9)
at src/libpanic_unwind/lib.rs:82
11: script::dom::bindings::codegen::Bindings::HTMLAudioElementBinding::HTMLAudioElementBinding::_finalize::h510de5481af6c7d4 (0x55f410e9bcc0)
12: _ZNK2js5Class10doFinalizeEPNS_6FreeOpEP8JSObject (0x55f411880a25)
at /data/joelm/personal/UTA/dissertation/servo/servo-20190724.git/target/release/build/mozjs_sys-0c8fde13422c462f/out/dist/include/js/Class.h:872
_ZN8JSObject8finalizeEPN2js6FreeOpE
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/vm/JSObject-inl.h:83
_ZN2js2gc5Arena8finalizeI8JSObjectEEmPNS_6FreeOpENS0_9AllocKindEm
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:591
13: _ZL19FinalizeTypedArenasI8JSObjectEbPN2js6FreeOpEPPNS1_2gc5ArenaERNS4_15SortedArenaListENS4_9AllocKindERNS1_11SliceBudgetENS4_10ArenaLists14KeepArenasEnumE (0x55f411880872)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:651
14: _ZL14FinalizeArenasPN2js6FreeOpEPPNS_2gc5ArenaERNS2_15SortedArenaListENS2_9AllocKindERNS_11SliceBudgetENS2_10ArenaLists14KeepArenasEnumE (0x55f411861c3b)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:683
15: _ZN2js2gc10ArenaLists18foregroundFinalizeEPNS_6FreeOpENS0_9AllocKindERNS_11SliceBudgetERNS0_15SortedArenaListE (0x55f41186e169)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:5820
16: _ZN2js2gc9GCRuntime17finalizeAllocKindEPNS_6FreeOpERNS_11SliceBudgetEPN2JS4ZoneENS0_9AllocKindE (0x55f41186f0fa)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6110
17: _ZN11sweepaction18SweepActionForEachI13ContainerIterIN7mozilla7EnumSetIN2js2gc9AllocKindEjEEES7_JPNS5_9GCRuntimeEPNS4_6FreeOpERNS4_11SliceBudgetEPN2JS4ZoneEEE3runESA_SC_SE_SH_ (0x55f4118891ab)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6327
18: _ZN11sweepaction19SweepActionSequenceIJPN2js2gc9GCRuntimeEPNS1_6FreeOpERNS1_11SliceBudgetEPN2JS4ZoneEEE3runES4_S6_S8_SB_ (0x55f4118893d4)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6296
19: _ZN11sweepaction18SweepActionForEachIN2js2gc19SweepGroupZonesIterEP9JSRuntimeJPNS2_9GCRuntimeEPNS1_6FreeOpERNS1_11SliceBudgetEEE3runES7_S9_SB_ (0x55f4118898b1)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6327
20: _ZN11sweepaction19SweepActionSequenceIJPN2js2gc9GCRuntimeEPNS1_6FreeOpERNS1_11SliceBudgetEEE3runES4_S6_S8_ (0x55f411889af1)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6296
21: _ZN11sweepaction20SweepActionRepeatForIN2js2gc15SweepGroupsIterEP9JSRuntimeJPNS2_9GCRuntimeEPNS1_6FreeOpERNS1_11SliceBudgetEEE3runES7_S9_SB_ (0x55f411889fc5)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6356
22: _ZN2js2gc9GCRuntime19performSweepActionsERNS_11SliceBudgetE (0x55f41186f5dd)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:6528
23: _ZN2js2gc9GCRuntime16incrementalSliceERNS_11SliceBudgetEN2JS8GCReasonERNS0_13AutoGCSessionE (0x55f411871550)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:7049
24: _ZN2js2gc9GCRuntime7gcCycleEbNS_11SliceBudgetEN2JS8GCReasonE (0x55f4118727d8)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:7398
25: _ZN2js2gc9GCRuntime7collectEbNS_11SliceBudgetEN2JS8GCReasonE (0x55f41187335f)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:7569
26: _ZN2js2gc9GCRuntime2gcE18JSGCInvocationKindN2JS8GCReasonE (0x55f41185640c)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/gc/GC.cpp:7657
27: _ZN9JSRuntime14destroyRuntimeEv (0x55f41162eaa3)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/vm/Runtime.cpp:284
28: _ZN2js14DestroyContextEP9JSContext (0x55f4115c25d1)
at /home/joelm/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/b2f8393/mozjs/js/src/vm/JSContext.cpp:197
29: <mozjs::rust::Runtime as core::ops::drop::Drop>::drop::hed132fc3d12a947c (0x55f4114bfe75)
30: _ZN4core3ptr18real_drop_in_place17h22626fd093207c2fE.llvm.9711593096204109679 (0x55f410a0f6dc)
31: <alloc::rc::Rc<T> as core::ops::drop::Drop>::drop::h025c8fd65330ce43 (0x55f410a3c0e9)
32: core::ptr::real_drop_in_place::hfb055ca4c92ce1d8 (0x55f4111a72b4)
33: std::sys_common::backtrace::__rust_begin_short_backtrace::hc93943e85aa6fa26 (0x55f41119907d)
34: _ZN3std9panicking3try7do_call17hb93b842dfb1f8343E.llvm.11073267061158468562 (0x55f411245103)
35: __rust_maybe_catch_panic (0x55f413339db9)
at src/libpanic_unwind/lib.rs:82
36: core::ops::function::FnOnce::call_once{{vtable.shim}}::h7217876d16e085d4 (0x55f410a082b2)
37: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h10faab75e4737451 (0x55f41331e8ee)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
38: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1aad66e3be56d28c (0x55f4133390df)
at /rustc/273f42b5964c29dda2c5a349dd4655529767b07f/src/liballoc/boxed.rs:766
std::sys_common::thread::start_thread::h9a8131af389e9a10
at src/libstd/sys_common/thread.rs:13
std::sys::unix::thread::Thread::new::thread_start::h5c5575a5d923977b
at src/libstd/sys/unix/thread.rs:79
39: start_thread (0x7fab3e4cc6b9)
40: clone (0x7fab3cd6841c)
41: <unknown> (0x0)
[2019-08-01T04:24:23Z ERROR servo] called `Option::unwrap()` on a `None` value
thread panicked while panicking. aborting.
Servo exited with return value -4
About this issue
- Original URL
- State: open
- Created 7 years ago
- Comments: 138 (137 by maintainers)
Commits related to this issue
- Auto merge of #24007 - gterzian:use_router_in_webdriver, r=asajeffrey Use ipc router in webdriver <!-- Please describe your changes on the following line: --> Fix for https://github.com/servo/servo... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
- Auto merge of #24047 - gterzian:ensure_documents_and_fetch_cancellers_drop, r=<try> Ensure documents and fetch cancellers drop <!-- Please describe your changes on the following line: --> Related t... — committed to servo/servo by deleted user 7 years ago
Things are MUCH improved with this build!
I did a test run for about 2000 loads with no crash/panic. There was a still a slight increase in fds over time, but it’s very slow and certainly not happening with every page load. The fds started at about 220 and increased to 263 on the 2000th load. The number of fds wobbles up and down some (perhaps an artifact of how I’m counting them via /proc) but on average has a slow increase.
Here are the first few fd counts and then every 100 page loads after that:
So it’s not the settings stack after all, in fact the tracing of
IpcSenderdidn’t occur as part oftrace_rust_roots. It turns out it was theFetchCanceller(altough I don’t fully understand it didn’t show up as part of the tracing intrace_rust_roots).I don’t have a trace handy now, however I remember it was always as part of
clear_js_runtime, which calls intoGC. So essentially they were kept alive, and traced each time a new pipeline was closed andclear_js_runtimewas called, but they weren’t dropped ever.So I was able to stop the
FecthCanceller, or any tracedIpcSenderfor that matter, from growing steadily, and I was also able to get the docs/windows/globalscope to actually drop when we close a pipeline, I’ll have a PR ready with those changes soon(it does not involve any changes to tracing).However, this still doesn’t fix the mach ports count from going up for each load. So I assume some ipc sender/receivers are being leaked somewhere outside of script. Actually I haven’t checked whether
IpcReceiverare traced, I should.Hmm, so something is keeping a live reference to the document, oh that’s annoying.!
I wonder if there’s some way we can find out what’s keeping the document alive? Can we temporarily add an
assert!to the implementation ofDocument::trace()which panics if we end up tracing a document that’s been discarded? The stack trace might help figure out what’s keeping the document alive.On Wed, Aug 21, 2019 at 8:38 AM Gregory Terzian notifications@github.com wrote:
I have developed a very small script that can crash servo:
The HTML and CSS file that I used, along with the script the commands I used to start the various pieces is captured at the following gist: https://gist.github.com/kanaka/119f5ed9841e23e35d07e8944cca6aa7
I built servo just now from d2856ce8aeca and then ran the test against it. I got a crash on the 819th page load. The crash is not a “too many open files” crash, but I’m posting it here because I suspect it may be related; it’s simplified version of the process I used earlier and it crashed after a similar order of page loads. I’ve included the crash below. If you determine that this is likely a different test case I’m happy to file a new ticket.
Please see https://github.com/servo/ipc-channel/issues/396 for a prototype of a proposed solution to https://github.com/servo/ipc-channel/issues/240. I am looking for Servo developers to review the prototype and perhaps take it for a spin.
Ok so those big chunks of documents with the same pipeline id dropping, they were created as part of https://github.com/servo/servo/blob/174bcc443435100da31b20db5ab684f4f3c8255b/components/script/dom/servoparser/mod.rs#L162
Basically, if a pages uses an API like
SetInnerHTML, Servo creates a new document for each fragment that is parsed. There are few other places where new documents are created, like when usingHTMLTemplateElementorCreateHTMLDocument.Those document do not have a browsing context, and they share the window of the document in which they are created(so they share the pipeline).
On another note, there is a PR at https://github.com/servo/servo/pull/24047 which fixes the issues noted with the
FetchCancellerand windows not dropping as part ofclear_js_runtime.However, the mach ports count still steadily goes up on each load.
And, I’m not so sure anymore if it really has to do with leaking ipc-channels as part of the load workflow, because I can get the count to steadily go up also just by scrolling the page.
So it looks like the mach ports leak is somewhere in the glutin/compositing/rendering part. Also, I’m not completely sure “mach ports” equals icp-sender/recevier, since from sampling I can see quite a lot of
mach_msgcalls that are unrelated to icp-channel, for example as part of running the gluting event-loop, handling input events, swapping buffers as part of the call toself.window.present();in the compositor, and so on…Anyway, when https://github.com/servo/servo/pull/24047 merges, @kanaka it might be worth another try using the master branch, since we were leaking ipc-channels along with the DOM objects, and the mach ports count I see on Mac might not actaully related to the fd related problem on linux.
Perhaps we should have:
This would avoid introducing bugs when forgetting to clear fields of Window in the future.
By the way, to narrow things down a bit, it seems we are after all leaking ipc-senders somewhere in a refcounted object, see:
Does the drop on no trace indicate that what is keeping the window alive is a child holding a reference to the window?
A couple of other questions:
Dom<Window>and so on? Logically, if there was aDom<Window>stored somewhere, would the window still drop if we didn’t trace it?trace_rust_rootscalled? Is it some sort of interupt? I’m asking because I got a few double borrow errors on aDomRefCellwhich seems to only be possible if the refcell was traced when the script-thread was interrupted while borrowing it.Gosh, not tracing a Window is dangerous! It’ll end up gc’ing the window’s children, but not the window itself. We have to be very confident that the window is unreachable for this to be safe!
The good news is there are only 20 places where an ipc-sender is used is stored in something traceable, so it must be one of them that is leaking! yeah!
Ok so I checked again, and while we are dropping documents once trimming the session history kicks in, we’re not dropping windows(unless the script-thread shuts down).
So it looks like if we load, say, 1000 documents in a script-thread, we’re actually keeping 1000 windows somewhere.
I can’t tell what is keeping a reference to them, could it be some Js engine thing? Maybe the windowproxy needs to unset it’s window? Currently
unset_currently_activesets the window to a dissimilar origin one(which clones all the ipc senders), and I can’t see any code actually setting the window to nothing, or the equivalent(altough each new load willset_windowto a new one, and I’m not sure what that does to the previous one).Note that each new window does a
WindowBinding::Wrap(JSContext::from_ptr(runtime.cx()), win), maybe we need to do “the opposite” when shutting down that pipeline? It seems to create a global object, and I can imagine we need to explicitly undo that?Well, it does appear to have done something positive. The test ran for 428 loads and then the screenshot command hung. I manually loaded a page again and it seems to have unstuck it. So I’ll add a little more intelligent error handling to the script so that it will keep going in that case (there was no error from servo itself). But that will have to wait for tomorrow because it’s 1:45am in the morning here.
BTW, for anybody curious the system I’m building as part of my research is at https://github.com/kanaka/bartender. I presented an early version of it at COMPSAC last year (“Property-Based Testing of Browser Rendering Engines with a Consensus Oracle”). The code base is still really rough and lacking in friendly documentation and likely to remain so for a while so caveat emptor.
@gterzian no problem. One of my original inspirations for this dissertation topic was discovering Servo and realizing that there was an interesting gap in the space of “automated testing of software with multiple implementations”. I do hope that I can contribute to the quality of Servo’s rendering engine (and browser rendering standardization in general). So I’m fairly motivated to include Servo in my results/data but depending on timing I may have to make due with comparing Chrome and Firefox for my actual dissertation results. On the other hand, since I’m spending a lot of time writing, it’s pretty easy for me to do builds/tests in the background.
I can also see https://github.com/servo/servo/blob/45af8a34fe6cc6f304f935ababdbc90fb4c50ede/components/constellation/pipeline.rs#L279
So the question is whether this is because there are too many stylesheet loaders creating channels, or if it’s more like webdriver is creating too many pipelines in general, and any creation of an ipc-chan will eventually fail.