Ceedling: ceedling exit code is 0 on unit test failure
When running ceedling test:all with failing unit tests, the process returns a 0 status code. It should return 1 as the status code so CI tools can detect the failure.
I am using the following versions of the farmework:
Ceedling:: 0.27.0 CException:: 1.3.1.18 CMock:: 2.4.4.215 Unity:: 2.4.1.120
This issue was reported in #117 and was reported to be fixed in #130; however, I seem to still experience this problem.
I have setup a repository that reproduces this problem. It can be found at: https://github.com/nmisoft/ceedling_exit_code_bug
Interestingly using rake test:all will correctly return 1 as the status code.
I have reproduced this problem both on Windows and Linux.
Windows
OS Version: 10.0.14393 N/A Build 14393
Using ceedling
>ceedling test:all
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.
ERROR: Aborted with Exit Code 1
>echo %ERRORLEVEL%
0
Using rake
>rake test:all
Test 'test_demo.c'
------------------
Running test_demo.out...
-------------------
FAILED TEST SUMMARY
-------------------
[test_demo.c]
Test: test_demo_fail
At line (16): "Expected TRUE Was FALSE"
--------------------
OVERALL TEST SUMMARY
--------------------
TESTED: 1
PASSED: 0
FAILED: 1
IGNORED: 0
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.
>echo %ERRORLEVEL%
1
Linux
Linux 4.4.0-78-generic #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017 x86_64 GNU/Linux
Using ceedling
$ ceedling test:all
Test 'test_demo.c'
------------------
Running test_demo.out...
-------------------
FAILED TEST SUMMARY
-------------------
[test_demo.c]
Test: test_demo_fail
At line (16): "Expected TRUE Was FALSE"
--------------------
OVERALL TEST SUMMARY
--------------------
TESTED: 1
PASSED: 0
FAILED: 1
IGNORED: 0
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.
$ echo $?
0
Using rake
$ rake test:all
Test 'test_demo.c'
------------------
Running test_demo.out...
-------------------
FAILED TEST SUMMARY
-------------------
[test_demo.c]
Test: test_demo_fail
At line (16): "Expected TRUE Was FALSE"
--------------------
OVERALL TEST SUMMARY
--------------------
TESTED: 1
PASSED: 0
FAILED: 1
IGNORED: 0
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.
$ echo $?
1
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 1
- Comments: 26 (12 by maintainers)
@mvandervoord I have tested this on my Linux Mint setup and Windows 10 setup and it works in both cases!
@mvandervoord it works fine on Ubuntu