onnxscript: Test crashes sometimes for `scaled_dot_product_attention_bool_mask`

E.g. https://github.com/microsoft/onnx-script/actions/runs/4539041271/jobs/7998517646?pr=566#step:5:3923

worker 'gw1' crashed while running 'onnxscript/tests/function_libs/torch_lib/ops_correctness_test.py::TestOutputConsistencyFullGraphCPU::test_output_match_opinfo__nn_functional_scaled_dot_product_attention_bool_mask_cpu_float32'

About this issue

Original URL
State: closed
Created a year ago
Comments: 16 (16 by maintainers)

Commits related to this issue

Disable tests for scaled_dot_product_attention | ci(atenlib) (#633) Temporarily disable tests for scaled_dot_product_attention because ORT segfaults on linux. #567 — committed to microsoft/onnxscript by justinchuby a year ago

Most upvoted comments

-inf is a constant in our function definition. I am still not sure why it is happening randomly

justinchuby on Apr 7, 2023

onnx doesn’t parse when the float constant is -inf (https://github.com/onnx/onnx/issues/5102). This could be causing the instability
dropout as an attribute is supplied 0 when it should have been 0.0. ORT doesn’t complain but it could be another thing.

I changed -inf to -1000000000000000000000000000.0 and dropout to 0.0 and ran the session 100000 times. Everything looks fine

justinchuby on Apr 7, 2023