argo-workflows: HTTP Template hangs indefinitely
Pre-requisites
- I have double-checked my configuration
- I can confirm the issues exists when I tested with
:latest - I’d like to contribute the fix myself (see contributing guide)
What happened/what you expected to happen?
What happened:
- I ran
argo submit -n argo --watch httptest.yamlthe content from https://argoproj.github.io/argo-workflows/http-template/ . The workflow keeps in ‘Running’ state forever ( more than 5 hours ).
Expected:
- The workflow should succeed in a couple of minutes.
Environment:
% kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:57:26Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:49:09Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/arm64"}
% argo version
argo: v3.4.3
BuildDate: 2022-10-31T09:13:13Z
GitCommit: eddb1b78407adc72c08b4ed6be8f52f2a1f1316a
GitTreeState: clean
GitTag: v3.4.3
GoVersion: go1.18.7
Compiler: gc
Platform: darwin/amd64
minikube v1.28.0 on Darwin 13.0 (arm64)
Version
v3.4.3
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don’t enter a workflows that uses private images.
https://argoproj.github.io/argo-workflows/http-template/
Logs from the workflow controller
kubectl logs -n argo deploy/workflow-controller | grep http-template-8879r
% kubectl logs -n argo deploy/workflow-controller | grep http-template-8879r
time="2022-11-08T02:00:29.184Z" level=info msg="Processing workflow" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.189Z" level=info msg="Updated phase -> Running" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.189Z" level=info msg="Steps node http-template-8879r initialized Running" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.189Z" level=info msg="StepGroup node http-template-8879r-3317262273 initialized Running" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.190Z" level=info msg="HTTP node http-template-8879r-1743288300 initialized Pending" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.190Z" level=info msg="Workflow step group node http-template-8879r-3317262273 not yet completed" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.190Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.190Z" level=info msg="Creating TaskSet" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.194Z" level=info msg=reconcileAgentPod namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.207Z" level=info msg="Created Agent pod" namespace=argo podName=http-template-8879r-1340600742-agent workflow=http-template-8879r
time="2022-11-08T02:00:29.207Z" level=info msg=updateAgentPodStatus namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:29.207Z" level=info msg=assessAgentPodStatus namespace=argo podName=http-template-8879r-1340600742-agent
time="2022-11-08T02:00:29.213Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=7906 workflow=http-template-8879r
time="2022-11-08T02:00:39.209Z" level=info msg="Processing workflow" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:39.210Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=http-template-8879r
time="2022-11-08T02:00:39.210Z" level=info msg=updateAgentPodStatus namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:39.210Z" level=info msg=assessAgentPodStatus namespace=argo podName=http-template-8879r-1340600742-agent
time="2022-11-08T02:00:39.211Z" level=info msg="Workflow step group node http-template-8879r-3317262273 not yet completed" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:39.211Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:00:39.211Z" level=info msg=reconcileAgentPod namespace=argo workflow=http-template-8879r
time="2022-11-08T02:05:36.440Z" level=info msg="Processing workflow" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:05:36.441Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=http-template-8879r
time="2022-11-08T02:05:36.441Z" level=info msg=updateAgentPodStatus namespace=argo workflow=http-template-8879r
time="2022-11-08T02:05:36.441Z" level=info msg=assessAgentPodStatus namespace=argo podName=http-template-8879r-1340600742-agent
time="2022-11-08T02:05:36.442Z" level=info msg="Workflow step group node http-template-8879r-3317262273 not yet completed" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:05:36.442Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=http-template-8879r
time="2022-11-08T02:05:36.442Z" level=info msg=reconcileAgentPod namespace=argo workflow=http-template-8879r
Logs from in your workflow’s wait container
% kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=http-template-8879r,workflow.argoproj.io/phase!=Succeeded
error: container wait is not valid for pod http-template-8879r-1340600742-agent
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 1
- Comments: 21 (9 by maintainers)
@mcgrawia The HTTP Template is the main reason I’m still considering Argo workflows. I was looking for a way to “chain” various REST calls, or at least call those in sequence. I was expecting the template to make external a REST call and add the result to its output. It would be nice to have a way for the output of one template to become the input of the next.
@gcsfred2 can you provide the pod events and pod spec for the agent pod? looks like the agent is not coming up.
Can you paste the log from your agent pod (
http-template-8879r-1340600742-agent)?