data-prepper: [BUG] Flaky tests high-level tracking
Describe the bug Running the data-prepper build and github actions frequently has failing flaky tests, and often requires retrying the tests multiple times before it succeeds. This issue is meant to allow easy tracking of these flaky tests, by adding an item to the checklist when a flaky test is seen.
Flaky tests
-
> Task :data-prepper-core:integrationTest PipelinesWithAcksIT > simple_pipeline_with_multiple_records() FAILED java.lang.NullPointerException at PipelinesWithAcksIT.java:85 -
testcase name="three_pipelines_with_route_and_multiple_records()" classname="org.opensearch.dataprepper.integration.PipelinesWithAcksIT" time="11.003"> <failure message="java.lang.NullPointerException" type="java.lang.NullPointerException">java.lang.NullPointerException at org.opensearch.dataprepper.integration.PipelinesWithAcksIT.three_pipelines_with_route_and_multiple_records(PipelinesWithAcksIT.java:130) - #3470
- #1374
- Router_ThreeRoutesIT https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1793501480
- AwsCloudMapPeerListProviderTest.java:345 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1795279643
- OTelTraceRawProcessorTest.java:270 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1796339462
- OTelTraceRawProcessorTest.java:193 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1806598375
- #4298
- HistogramAggregateActionTests.java:370 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2207244965
- LambdaSinkServiceTest.java:316 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2221378060; fixed by #4723
- RDS ExportSchedulerTest - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2393877779
- RDS StreamScheduler - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2394058183. (added sleep time in #5016, will check again see if this helps)
- Truncate processor - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2405626969
- RDS DataFileSchedulerTest - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2581524346
- PipelinesWithAcksIT simple_pipeline_with_single_record - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2581526898
- GrpcRetryInfoCalculatorTest - testResetAfterDelayWearsOff - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2581528755
- ExportSchedulerTest test_export_run - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2581533044
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 25 (24 by maintainers)
RDS now has a flaky test:
https://github.com/opensearch-project/data-prepper/actions/runs/11172396206/job/31058770072?pr=5021
@oeyh , Can you look at this one?
https://github.com/opensearch-project/data-prepper/actions/runs/12371897788/job/35401071596?pr=5268
@oeyh , Here is another failure in RDS.