MachineLearningNotebooks: Pipeline step allow_reuse flag is set to True but steps are not being reused
I am creating a multistep pipeline in azureml and all of the steps have the allow_reuse flag set to true. However, none of the steps are being reused.
Included are two runs that have no changes between them but there is no reuse: Run ID: 9aa9bee5-d27e-48b3-b03e-72703f0e4c10 Step ID: f3fe9958-a650-44c2-8ece-6e42f9d287dc
Run ID: d45f5f9d-df6b-449e-8855-1e340043eb32 Step ID: 7207f375-d908-4562-94bf-02f022f28047
Specifically, I am looking at the 1st step in the pipeline as I am still working on the subsequent steps. Does the entire pipeline have to complete successfully in order for the steps to be eligible for reuse?
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 4
- Comments: 25
I see in the new portal that it shows as reused but the duration indicates that it was not reused and i am seeing the output being re-generated in blob storage each time. the step is definitely NOT getting reused but I see that the new portal says that is is being reused.
Fix has been ready. We expect to deliver the fix on next release.
We are looking at a reuse bug that got introduced recently. Please stay tuned.
@sanpil I know that is not getting reused because I see it recomputing as I monitor the run and the
reusecolumn in the portal indicates it is not being reused. the step takes a similar amount of time to complete as it does when there are changes.@swanderz produced a really clean repro for this issue in #740
see original comment for the Run ID and Step ID of both runs
@clmccart Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.