charon: Tracking issue for not crashing on the rustc test suite
In https://github.com/AeneasVerif/charon/pull/136/ we added a script that runs charon on the rustc test suite and displays the results. As we’d like to support a large portion of the rust language, we track in this issue our results on these tests.
Our first aim is to never panic. Initial results:
5269 expected-failure
2585 failure
3105 panic
4207 success
27 timeout
(Please don’t post on this conversation, it’s mean for automated posting)
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 89 (12 by maintainers)
Summary of the results: 4673 expected-failure 2065 failure 1731 panic 3546 success 30 timeout 3148 unsupported-build-settings
Big change this week, in https://github.com/AeneasVerif/charon/pull/536 I was finally able to distinguish rustc errors that indicate incorrect code from charon panics, thanks to catch_fatal_errors.
Summary of the results:
❌ expected: success; got: failure in rustc
test-results/proc-macro/auxiliary/issue-107113.rs
test-results/proc-macro/auxiliary/parse-invis-delim-issue-128895.rs
test-results/proc-macro/crate-attrs-multiple.rs
test-results/attributes/main-removed-2/auxiliary/tokyo.rs
❌ expected: success; got: failure in hax
test-results/macros/std-2024-macros.rs
❌ expected: success; got: stack overflow
test-results/impl-trait/issue-36792.rs
test-results/associated-type-bounds/struct-bounds.rs
test-results/associated-type-bounds/enum-bounds.rs
test-results/associated-type-bounds/union-bounds.rs
test-results/nll/ty-outlives/issue-53789-2.rs
test-results/nll/ty-outlives/issue-53789-1.rs
test-results/associated-types/issue-71113.rs
test-results/self/self-in-typedefs.rs
test-results/issues/issue-19601.rs
test-results/derives/derive-hygiene.rs
test-results/generic-associated-types/issue-87750.rs
No panics left!
It’s a case where charon successfully compiled the test despite the fact that the test was supposed to be a failing test. This is most likely because we don’t parse most of the test options in files.
I just noticed we almost have 7.2k success now, to be compared with the 4.2k success from when we started tracking this issue 😃