kpack: Churn bug when clusterstack run image digest is out of date with tagged run image

Update

Hi Guys, based on what we discussed last Friday, I just created a Draft PR with some changes that I tested locally. I ran my test scenario and the bug was also solved with the changes describe in the Pull Request and here are a couple of screenshots with the pod execution data.

Build-2

Nothing new, the manual build triggered just right after the run-image was updated.

Build-3

New stack build executed just right after the build-2 finished. Here we can see the --run-image flag with the data took it from the new field added to BuildSpec and we can also see the new and old pointers are switched

Build-4

New stack build executed just right after the build-3 finished. We can see similar data to Build-3 but now the new and old references points to the correct values

Thanks @zhoufenqin,

I managed to reproduce it, triggering the new build right after I pushed my run-image. I will try to check the code tomorrow!

➜  kpack kubectl get build -l image.kpack.io/image=tutorial-image-641 --sort-by='{.metadata.creationTimestamp}' -o go-template='{{range .items}} {{.metadata.creationTimestamp}} {{.metadata.name}} {{.metadata.resourceVersion}} {{index .metadata.annotations "image.kpack.io/reason"}} {{index .metadata.labels "image.kpack.io/imageGeneration"}} &{{end}}' | tr '\&' '\n'

 2022-07-06T21:32:50Z tutorial-image-641-build-1 21375 CONFIG 1
 2022-07-06T21:51:18Z tutorial-image-641-build-2 21996 STACK 1
 2022-07-06T21:53:21Z tutorial-image-641-build-3 22288 TRIGGER 1
 2022-07-06T22:00:46Z tutorial-image-641-build-4 43907 STACK 1
 2022-07-07T01:48:01Z tutorial-image-641-build-5 44168 TRIGGER 1 <--- I triggered this one
 2022-07-07T01:49:06Z tutorial-image-641-build-6 44242 STACK 1    <--- Loop begins
 2022-07-07T01:49:23Z tutorial-image-641-build-7 44321 STACK 1
 2022-07-07T01:49:40Z tutorial-image-641-build-8 44397 STACK 1
 2022-07-07T01:49:57Z tutorial-image-641-build-9 44476 STACK 1
 2022-07-07T01:50:16Z tutorial-image-641-build-10 44488 STACK 1

Hi @zhoufenqin I wasn’t able to work on this last week, but, my plan is to do it this week. I will keep everybody up to date.

Hi @jjbustamante , any update on this issue? Which kpack version will fix this?

not sure if I have all the knowledge but I can give it a shot.

Thank you @jjbustamante 😃 I added some details, once the cluster runImage is updated and the builder configured stack runImage hasn’t been updated, multiple builds will be triggered. The image.kpack.io/reason is STACK or CONFIG, STACK

> k get build -l image.kpack.io/image=default.fenzho-0523 --sort-by='{.metadata.creationTimestamp}' -o go-template='{{range .items}} {{.metadata.creationTimestamp}} {{.metadata.name}} {{.metadata.resourceVersion}} {{index .metadata.annotations "image.kpack.io/reason"}} {{index .metadata.labels "image.kpack.io/imageGeneration"}} &{{end}}' | tr '\&' '\n'
 2022-05-25T03:16:03Z default.fenzho-0523-build-1 4203472 CONFIG 1
 2022-05-25T03:17:27Z default.fenzho-0523-build-2 4203527 STACK 1
 2022-05-25T03:17:33Z default.fenzho-0523-build-3 4203575 STACK 1
 2022-05-25T03:17:37Z default.fenzho-0523-build-4 4203617 STACK 1
 2022-05-25T03:17:41Z default.fenzho-0523-build-5 4203662 STACK 1
 2022-05-25T03:17:44Z default.fenzho-0523-build-6 4203707 STACK 1
......
......