spring-cloud-dataflow: Unable to set composed-task to boot3 by default

Description:

Note: IMO, My issue #5538 is very similar to #5519.

I run composed-task, but failed with following error messages. Error messages in composed-task:

2023-11-07 09:45:43.394  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2023-11-07 09:45:44.102  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2023-11-07 09:45:44.808  INFO 1 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-11-07 09:45:45.115  INFO 1 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.6.15.Final
2023-11-07 09:45:45.915  INFO 1 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-11-07 09:45:46.607  INFO 1 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.MariaDB53Dialect
2023-11-07 09:45:47.994  INFO 1 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-11-07 09:45:48.007  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-11-07 09:45:48.811  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [a1-integration-task-batch, integration-task-batch, a1-task-alarm-composed, task-alarm-composed]
2023-11-07 09:45:48.910  INFO 1 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: MYSQL
2023-11-07 09:45:49.099  INFO 1 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: MYSQL
2023-11-07 09:45:49.117  INFO 1 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
2023-11-07 09:45:51.207  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-11-07 09:45:58.496  WARN 1 --- [           main] i.m.p.rsocket.PrometheusRSocketClient    : Creating the connection and receiving the key timed out!
2023-11-07 09:45:58.698  WARN 1 --- [           main] onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobLauncherApplicationRunner': Unsatisfied dependency expressed through method 'setJobs' parameter 0; n
2023-11-07 09:45:58.701  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-11-07 09:45:58.705  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2023-11-07 09:45:58.804  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
2023-11-07 09:45:58.810 ERROR 1 --- [           main] o.s.c.t.listener.TaskLifecycleListener   : An event to end a task has been received for a task that has not yet started.
2023-11-07 09:45:58.893  INFO 1 --- [           main] ConditionEvaluationReportLoggingListener :

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-11-07 09:45:58.913 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobLauncherApplicationRunner': Unsatisfied dependency expressed through method 'setJobs' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'composedTaskJob': FactoryBean threw exception on object creation; nested
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:824) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:777) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:408) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921) ~[spring-context-5.3.30.jar!/:5.3.30]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.30.jar!/:5.3.30]
    at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'composedTaskJob': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.I
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1609) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1573) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1462) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1349) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.30.jar!/:5.3.30]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213) ~[spring-beans-5.3.30.jar!/:5.3.30]
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1166) ~[spring-context-5.3.30.jar!/:5.3.30]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getTaskAppFlow(ComposedRunnerJobFactory.java:359) ~[classes!/:na]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.createFlow(ComposedRunnerJobFactory.java:145) ~[classes!/:na]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getObject(ComposedRunnerJobFactory.java:112) ~[classes!/:na]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getObject(ComposedRunnerJobFactory.java:55) ~[classes!/:na]
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) ~[spring-beans-5.3.30.jar!/:5.3.30]
    ... 40 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
    at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135) ~[na:na]
    at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182) ~[na:na]
    at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) ~[na:na]
    at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850) ~[na:na]
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na]
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na]
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[na:na]
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[na:na]
    at org.springframework.cloud.dataflow.core.Base64Utils.decodeMap(Base64Utils.java:62) ~[spring-cloud-dataflow-core-2.11.1.jar!/:na]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunnerStepFactory.getObject(ComposedTaskRunnerStepFactory.java:157) ~[classes!/:na]
    at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunnerStepFactory.getObject(ComposedTaskRunnerStepFactory.java:59) ~[classes!/:na]
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) ~[spring-beans-5.3.30.jar!/:5.3.30]
    ... 51 common frames omitted

Note: Execute each of the two individual tasks that make up the composed-task, they work well without any problems.

Application properties in k8s ConfigMap:

Data
====
application.yaml:
----
spring:
  security:
    oauth2:
      client:
        registration:
          google:
            client-id: <REDACTED>
            client-secret: <REDACTED>
            scope:
            - profile
            - email
            redirect-uri: <REDACTED>
            authorization-grant-type: authorization_code
        provider:
          google:
            user-info-uri: <REDACTED>
  cloud:
    deployer:
      # [1]
      bootVersion: 3
    dataflow:
      security:
        authorization:
          provider-role-mappings:
            google:
              map-oauth-scopes: false
      task:
        platform:
          kubernetes:
            accounts:
              default:
                environmentVariables:
                  - <REDACTED>=dev
                limits:
                  cpu: 500m
                  memory: 1024Mi
                readinessProbeDelay: 120
                livenessProbeDelay: 90
                podSecurityContext:
                  runAsUser: 1001
                entryPointStyle: exec
    task:
      # [2]
      tablePrefix: BOOT3_TASK_
      # [3]
      schemaTarget: boot3
      closecontextEnabled: true
  datasource:
    url: '...'
    driverClassName: org.mariadb.jdbc.Driver
    username: DataFlowServer

    password: ${DATABASE_PASSWORD}
    testOnBorrow: true
    validationQuery: "SELECT 1"
  flyway:
    enabled: true

I set up Application Properties by referring to the two articles (#5470 #5482).

To change the boot version related arguments to boot3 in composed-task, I try to add the three parameters below referring this comment

• spring.cloud.task.schemaTarget = boot3
• spring.cloud.task.tablePrefix = BOOT3_TASK_
• spring.cloud.deployer.bootVersion = 3

Release versions:

• EKS v1.24
• x86_64 (AMD64), EC2 Worker nodes
• SCDF v2.11.1 (Installed by bitnami helm chart)
• A composed task consisting of two boot3 tasks.

Screenshots:

Screenshot moved after pressing the Launch Task button:

Failed composed-task parameter screenshot:

List of task execution:

Schema Target value is boot2, instead of boot3 in the Task executions menu.

Why is it output as boot2 even though Application Properties is set through ConfigMap? Is there a way to set it up or documents?