argo-workflows: Workflow with Http call stuck on Pending State
Checklist
- Double-checked my configuration.
- Tested using the latest version.
- Used the Emissary executor.
Summary
I was trying to do an Http Call from Workflow. But workflow got stuck in Pending State. I noticed this bug - https://github.com/argoproj/argo-workflows/pull/7084 Thought this is fixed. But I am seeing that now also. I am on v3.3.0
What version are you running? v3.3.0
Diagnostics
Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.
metadata:
name: delightful-python
namespace: default
labels:
example: 'true'
spec:
entrypoint: main
templates:
- name: main
steps:
- - name: get-google-homepage
template: http
arguments:
parameters: [{name: url, value: "https://www.google.com"}]
- name: http
inputs:
parameters:
- name: url
http:
timeoutSeconds: 20 # Default 30
url: "{{inputs.parameters.url}}"
method: "GET"
headers:
- name: "x-header-name"
value: "test-value"
successCondition: "response.body contains \"google\""
body: "test body"
# Logs from the workflow controller:
kubectl logs -n argo deploy/workflow-controller | grep ${workflow}
time="2022-04-01T09:29:28.761Z" level=info msg="Processing workflow" namespace=default workflow=delightful-python
time="2022-04-01T09:29:28.762Z" level=info msg=updateAgentPodStatus namespace=default workflow=delightful-python
time="2022-04-01T09:29:28.763Z" level=info msg=assessAgentPodStatus namespace=default podName=delightful-python-1340600742-agent
time="2022-04-01T09:29:28.764Z" level=info msg="Workflow step group node delightful-python-4228632240 not yet completed" namespace=default workflow=delightful-python
time="2022-04-01T09:29:28.764Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=delightful-python
time="2022-04-01T09:29:28.764Z" level=info msg=reconcileAgentPod namespace=default workflow=delightful-python
# If the workflow's pods have not been created, you can skip the rest of the diagnostics.
# The workflow's pods that are problematic:
kubectl get pod -o yaml -l workflows.argoproj.io/workflow=${delightful-python},workflow.argoproj.io/phase!=Succeeded
# Logs from in your workflow's wait container, something like:
kubectl logs -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded
Message from the maintainers:
Impacted by this bug? Give it a đź‘Ť. We prioritise the issues with the most đź‘Ť.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 7
- Comments: 15 (4 by maintainers)
@clementlecorre @yeongu-dev I use the Helm chart to deploy ArgoWorkflows and serviceaccounts. I wanted to use the http template but got
"Watch workflowtasksets 403"
error in the pod too. I noticed there thedefault
serviceaccount was used. Specifying the right serviceaccount solves the issue in my case :argo submit --serviceaccount argo-workflow
@clementlecorre did you make any progress on this?
I have created the
agent
role and added arolebinding
for my service account and I still see always-pending HTTP templates with no errors logged.My agent pod appears to be in permanent “init” state:
EDIT: after using
kubectl describe pod http-condition-zrw6k-1340600742-agent
to inspect the pod status I saw the following eventI followed this process (with added
-n argo
argument) to create a token secret and then the HTTP template was able to execute as expected. I did not see this step described in any of the docs linked from the HTTP template docs.See https://github.com/argoproj/argo-workflows/issues/9195#issuecomment-1191573224