platformio-core: Native: PlatformIO reports test as "Failed" even when all passed
What kind of issue is this?
-
Question. This issue tracker is not the place for questions. If you want to ask how to do something, or to understand why something isn’t working the way you expect it to, use Community Forums or Premium Support
-
PlatformIO IDE. All issues related to PlatformIO IDE should be reported to appropriate repository: PlatformIO IDE for Atom or PlatformIO IDE for VSCode
-
Development Platform or Board. All issues (building, uploading, adding new boards, etc.) related to PlatformIO development platforms should be reported to appropriate repository related to your hardware https://github.com/topics/platformio-platform
-
Feature Request. Start by telling us what problem you’re trying to solve. Often a solution already exists! Don’t send pull requests to implement new features without first getting our support. Sometimes we leave features out on purpose to keep the project small.
-
PlatformIO Core. If you’ve found a bug, please provide an information below.
You can erase any parts of this template not applicable to your Issue.
Configuration
Operating system: Ubuntu 20.04.4 LTS
PlatformIO Version (platformio --version): 5.2.5
Description of problem
When running tests on native platform, PlatformIO marks some tests as “Failed” even when unity reports that all tests have been passed successfully and prints final message “OK”. This appears only on native platform. The interesting thing about it is if test-run order is shuffled then everything is ok.

Steps to Reproduce
- Clone repo: https://github.com/Di-Strix/PIOTestFailDemo
- Run test natively using
pio test -e native - Get “OK” from unity and “Fail” from PlatformIO
- Swap test-run order so that ‘test_injection_empty_file’ runs first and then ‘test_injection_url’
- Run test again using
pio test -e native - Get “OK” from unity and “Passed” from PlatformIO
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 25 (12 by maintainers)
Finally I resolved this issue. That was completely my fault. Inside tests I create local server instances and provide them to
MaterializeLayout. The problem is thatMaterializeLayoutis global and it is re-initialized before each test. TheMaterializeLayoutdestructor tried to remove all handlers from the provided server but it was removed since test-function is completed. So destructor tried to access deleted local server instance. So there weren’t any issues with PlatformIO core.Thanks for help
That was the bug in PlatformIO Core 5.0, see https://github.com/platformio/platformio-core/issues/2910
PlatformIO Core MUST resolve all dependencies.