arcade: Why does execute-sdl.yml download all artifacts? Fails in Core-Setup due to disk space

https://github.com/dotnet/arcade/blob/ab99af9da2605b56f8720278a889bb0d2d711645/eng/common/templates/job/execute-sdl.yml#L21-L27

Why does this download everything, rather than the set of “final” artifacts Arcade uses in other situations? (PackageArtifacts/BlobArtifacts?)

The above along with the (I think relatively small) hosted agent that’s it’s hard-coded to use causes failures for Core-Setup:

https://github.com/dotnet/arcade/blob/ab99af9da2605b56f8720278a889bb0d2d711645/eng/common/templates/job/execute-sdl.yml#L16-L17

Core-Setup publishes a lot of artifacts to the pipeline that aren’t signed yet and/or won’t be published, so that other stages in the pipeline can work with them. This creates a large enough download that the SDL job fails due to lack of disk space in this prototype build:

https://dev.azure.com/dnceng/internal/_build/results?buildId=402553

@sunandabalu @dotnet/dnceng

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 25 (24 by maintainers)

Most upvoted comments

Sure thing

michellemcdaniel on Nov 13, 2019

I don’t see why not, @adiaaida could you send out a PR to merge this in 3.x branch for approval?

sunandabalu on Nov 13, 2019

This should already be mostly plumbed through - https://github.com/dotnet/arcade/blob/master/Documentation/AzureDevOps/TemplateSchema.md

Is work being tracked to have the SDL template follow this schema, or whatever it is that we need to plumb in the change above? (Not completely clear to me how the schema would be applied to this case, maybe just the DownloadArtifact schema as a param for execute-sdl.yml?)

Should someone be assigned to this issue?

We need this to turn on automatic SDL validation and satisfy 3.1 compliance requirements.

dagood on Oct 31, 2019