amazon-ecs-plugin: [BUG] Declarative pipeline overriding Fargate definition doesn't work
Overriding attributes from an inherited template works when creating dynamic agents via Declarative pipeline ONLY when the original template uses EC2 as launch mode.
So this WORKS:
agent {
ecs {
inheritFrom 'ec2-template'
cpu 1024
memory 4096
}
}
this DOES NOT WORK:
agent {
ecs {
inheritFrom 'fargate-template'
cpu 1024
memory 4096
}
}
Here are the relevant logs for a task inherited from fargate-template
:
Dec 19, 2019 11:28:52 AM INFO com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSDeclarativeAgent getAsArgs
In getAsArgs. label: ecs-agent-test-10-zl3f1
Dec 19, 2019 11:28:52 AM INFO com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSDeclarativeAgent getAsArgs
In getAsArgs. argMap: {assignPublicIp=false, cpu=1024, inheritFrom=fargate-template, label=ecs-agent-test-10-zl3f1, memory=8192, name=ecs-agent-test-10-zl3f1, overrides=[cpu, inheritFrom, memory, label], privileged=false}
Dec 19, 2019 11:28:52 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStep
In ECSTaskTemplateStep start. label: ecs-agent-test-10-zl3f1
Dec 19, 2019 11:28:52 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStep
In ECSTaskTemplateStep start. cloud: cloud-default
Dec 19, 2019 11:28:52 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
In ECSTaskTemplateExecution run()
Dec 19, 2019 11:28:52 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
cloud: cloud-default
Dec 19, 2019 11:28:52 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
label: ecs-agent-test-10-zl3f1
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] ecsTaskTemplate
[Pipeline] // ecsTaskTemplate
[Pipeline] End of Pipeline
ERROR: Cloud does not exist: cloud-default
Finished: FAILURE
And here’s what happens when I run the exact same thing but inheriting from ec2-template
:
Dec 19, 2019 11:31:46 AM INFO com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSDeclarativeAgent getAsArgs
In getAsArgs. label: ecs-agent-test-11-krf00
Dec 19, 2019 11:31:46 AM INFO com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSDeclarativeAgent getAsArgs
In getAsArgs. argMap: {assignPublicIp=false, cpu=1024, inheritFrom=ec2-template, label=ecs-agent-test-11-krf00, memory=8192, name=ecs-agent-test-11-krf00, overrides=[cpu, inheritFrom, memory, label], privileged=false}
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStep
In ECSTaskTemplateStep start. label: ecs-agent-test-11-krf00
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStep
In ECSTaskTemplateStep start. cloud: cloud-default
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
In ECSTaskTemplateExecution run()
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
cloud: cloud-default
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
label: ecs-agent-test-11-krf00
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Cloud maxCpu: 0
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Cloud maxMemory: 0
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Cloud maxMemoryReservation: 0
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Step cpu: 1,024
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Step memory: 8,192
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Step memoryReservation: 0
Dec 19, 2019 11:31:46 AM FINE com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution
Step shareMemorySize: 0
Dec 19, 2019 11:31:46 AM INFO com.cloudbees.jenkins.plugins.amazonecs.pipeline.ECSTaskTemplateStepExecution start
Registering task template with name ecs-agent-test-11-krf00-88vps
Dec 19, 2019 11:31:48 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
Asked to provision 1 agent(s) for: ecs-agent-test-11-krf00
Dec 19, 2019 11:31:48 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
In provisioning : []
Dec 19, 2019 11:31:48 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
Excess workload after pending ECS agents: 1
Dec 19, 2019 11:31:48 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
Will provision ECS Agent ecs-agent-test-11-krf00, for label: ecs-agent-test-11-krf00
Dec 19, 2019 11:31:58 AM FINE com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher
ECS: Launching agent
Dec 19, 2019 11:31:58 AM FINE com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher
[Dev-cluster-wsvqv]: Creating Task in cluster null
Dec 19, 2019 11:31:58 AM FINE com.cloudbees.jenkins.plugins.amazonecs.ECSCloud
Selected Region: us-east-1
Dec 19, 2019 11:31:58 AM FINE com.cloudbees.jenkins.plugins.amazonecs.ECSCloud
Selected Region: us-east-1
Dec 19, 2019 11:31:58 AM FINE com.cloudbees.jenkins.plugins.amazonecs.ECSCloud
No existing task definition found for family or ARN: Dev-cluster-ecs-agent-test-11-krf00-88vps
[..]
[.. Task succesfully completes at the end. I've truncated output as it's not super relevant anymore ..]
I have thus far:
- Allowed
all
overrides in the settings - Confirmed that this DOES NOT happen to ANY task that is inheriting from an EC2 launch type templates
- Confirmed that specifying
cloud
doesn’t change anything. - Confirmed that NOT specifying
cloud
for an ec2 based template doesn’t break anything (at least on my setup with just a single cloud) - Made sure that the templates themselves work, if we don’t involve any overriding.
It feels to me that the issue lies here - https://github.com/jenkinsci/amazon-ecs-plugin/blob/a1c2e74fe8df5729e3410ddffb3407ac35e83d96/src/main/java/com/cloudbees/jenkins/plugins/amazonecs/pipeline/ECSTaskTemplateStepExecution.java#L52
That for some reason the Fargate based templates do not return as provisonable
, but I don’t have the environment setup to actually do any development or testing this.
P.S. A similar issue was described in https://github.com/jenkinsci/amazon-ecs-plugin/issues/116, abut then closed without a proper resolution.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 34 (18 by maintainers)
Commits related to this issue
- #139 - Fargate pipeline fix. Also, fix several NPEs, add more test coverage, remove the dependency on needing to integrate with AWS — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - updates based on integration testing — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 Fix fargate run/merge issue, add tests, merge in pipeline scripts config (#164) * #139 - Fargate pipeline fix. Also, fix several NPEs, add more test coverage, remove the dependency on needing to... — committed to jenkinsci/amazon-ecs-plugin by chb0github 4 years ago
- #139 - fix NPE in hashcode compute. Eventhough JSR-301 Annotations are present, they clearly aren't being applied. Everything but the templateName can now be null — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Merge in from origin/master — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Remove JSR annotations that didn't seem to be applied and assert Nonnull as code in the constructor — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Fix javadoc comment — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- Fix NPE in hash/equals code (#169) * #139 - Fargate pipeline fix. Also, fix several NPEs, add more test coverage, remove the dependency on needing to integrate with AWS * #139 - updates based on i... — committed to jenkinsci/amazon-ecs-plugin by chb0github 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways — committed to chb0github/amazon-ecs-plugin by deleted user 4 years ago
- Roll back the amazon-ecs plugin Roll back from version 1.32 to 1.31, because version 1.32 introduces a bug which causes builds like the frontend build to fail with the error: ``` java.lang.IllegalAr... — committed to nationalarchives/tdr-jenkins by suzannehamilton 4 years ago
- Roll back the amazon-ecs plugin Roll back from version 1.32 to 1.31, because version 1.32 introduces a bug which causes builds like the frontend build to fail with the error: ``` java.lang.IllegalAr... — committed to nationalarchives/tdr-jenkins by suzannehamilton 4 years ago
- #139 - Remove jealous null checks as the annotations they replaced never checked for it anyways (#171) — committed to jenkinsci/amazon-ecs-plugin by chb0github 4 years ago
- Roll back the amazon-ecs plugin Roll back from version 1.32 to 1.31, because version 1.32 introduces a bug which causes builds like the frontend build to fail with the error: ``` java.lang.IllegalAr... — committed to nationalarchives/tdr-jenkins by suzannehamilton 4 years ago
Actually… I don’t think a ticket is necessary - but, I am a little surprised. @webratz someone added a
@Nonnull
to this field and a few others - So, the JSR validation process clearly hasn’t occured. Regardless, it’s an easy fix.