if: 👍🏼 Update IF to dump logs and errors to yaml file
What
Update IF so that errors and logs are saved to file as well as being displayed in the console.
Why
As a core developer I want to be able to run negative tests, which requires errors and logs to be captured in a file we can snapshot against. As a user I want to be able to provide a log file as part of a bug report to help the core team guide me through debugging.
Context
Testing negative flows is important for us to make sure IF behaves as intended across a large set of scenarios. It is hard to do this without dumping errors and logs to files because the alternative is for IF to panic without any persistent result to test against. This can be resolved by capturing the errors and logs to a file.
SoW
- This output manifest format can be used to test negative test cases as per #615
- Documentation has been added to the if-docs repo to explain this change.
Acceptance criteria
- Manifests that causes critical exceptions (causing the tool to fail) output status and error detail in an
executionnode.
GIVEN the user is in a folder WHEN they run this manifest file:
name: basic-error-demo
description:
tags:
initialize:
plugins:
teads-curve:
path: '@grnsft/if-unofficial-plugins'
method: TeadsCurve
global-config:
interpolation: spline
outputs:
- yaml
tree:
children:
child-0:
defaults:
cpu/thermal-design-power: 100
pipeline:
- teads-curve
inputs:
- timestamp: 2023-07-06T00:00
cpu/utilization: 20
THEN the tool logs the error, exits and outputs a manifest file which contains these additional nodes:
executionstatus: (success/fail)error: The exception that caused the failure and contents of the exception (to string())
name: basic-error-demo
description:
tags:
initialize:
plugins:
teads-curve:
path: '@grnsft/if-unofficial-plugins'
method: TeadsCurve
global-config:
interpolation: spline
outputs:
- yaml
execution:
status: fail
error: 'InputValidationError: "duration" parameter is required. Error code: invalid_type'.
tree:
children:
child-0:
defaults:
cpu/thermal-design-power: 100
pipeline:
- teads-curve
inputs:
- timestamp: 2023-07-06T00:00
cpu/utilization: 20
The errors and logs should still be displayed in the console as usual, because the version int he manifest file will not contain the full stack trace.
About this issue
- Original URL
- State: open
- Created 3 months ago
- Reactions: 1
- Comments: 16 (13 by maintainers)
@zanete FYI I untagged @MariamKhalatova and added a failing manifest and expected output manifest to the ticket description
@jmcook1186 this one is good to go from me.