aws-cdk: (aws-stepfunctions-tasks): GlueStartJobRunProps missing support of AllocatedCapacity or MaxCapacity

Is there any update for this issue? Here is a workaround we are using now.

export interface GlueStartJobRunProps extends tasks.GlueStartJobRunProps {
  readonly numberOfWorkers?: number;
  readonly workerType?: WorkerType;
}

export class GlueStartJobRun extends tasks.GlueStartJobRun {
  constructor(scope: Construct, id: string, private readonly enhancedProps: GlueStartJobRunProps) {
    super(scope, id, enhancedProps);
  }

  protected override _renderTask(): any {
    const state = super._renderTask();
    if (this.enhancedProps.workerType) {
      state.Parameters.WorkerType = this.enhancedProps.workerType.name;
    }

    if (this.enhancedProps.numberOfWorkers) {
      state.Parameters.NumberOfWorkers = this.enhancedProps.numberOfWorkers;
    }

    return state;
  }
}

Do we have any updates on this?

update: checking with the Step Functions team whether they’re aware of these changes given that it’s still not documented

Remaining

• model the NumberOfWorkers and WorkerType properties
  • one thing to consider is whether they’re supported as dynamic properties (can be supplied through JSON Path) which will determine what types they should be
• expose properties through GlueStartJobRunProps
• get Step Functions documentation updated

@shivlaks I think you meant @johnhill2424

Hi I’m running into this as well; I really need to increase the number of workers for certain jobs.

As far as I can tell, there is no way to drop down to the Cfn* resource types for just this one action in the state machine. Is that accurate? If so, that means that I have to either port my entire state machine definition OFF of the high-level stepfunctions/stepfunctions-tasks CDK APIs and down to the low-level, or else it’s just not possible to override the number of workers when starting the glue job?