runner: Environment Secrets are not available on Reusable Workflow / Workflow Templates
Describe the bug
obs: this feature works as designed, but I believe it could be improved.
Problem: Passing an environment containing secrets to a reusable workflow is not enough to have the environment secrets avaiable.
Example: In a repository, there is an environment called “myenv”, which contains a single secret called “MY_SECRET”. In this repository, there is also a workflow calling a reusable workflow.
This is the reusable workflow
name: Reusable Workflow
on:
workflow_call:
inputs:
ENVIRONMENT:
required: true
type: string
jobs:
some-job:
runs-on: ubuntu-latest
environment: ${{ inputs.ENVIRONMENT }}
steps:
- run: env
And this is the workflow
name: Workflow Caller
on:
push:
branches:
- **
jobs:
some-job:
uses: Org/repo/.github/workflows/reusable-workflow.yml
with:
ENVIRONMENT: myenv
When running this workflow, MY_SECRET
isn’t available. I see something like this in the logs:
...
MY_SECRET:
...
instead of this
...
MY_SECRET:***
...
In order to make MY_SECRET
available in the reusable workflow, I must explicitly write it in the workflow caller, like so:
name: Workflow Caller
on:
push:
branches:
- **
jobs:
some-job:
uses: Org/repo/.github/workflows/reusable-workflow.yml
with:
ENVIRONMENT: myenv
secrets:
MY_SECRET: ${{ secrets.MY_SECRET }}
Why can’t the reusable workflow load all of the environment secrets automatically using just the environment’s name? Is there a reason for not doing it?
In this repo you can find all of my experiments: https://github.com/AllanOricil/workflow-template-bug
Expected behavior “Deployment Environment” secrets should be available in reusable workflows
What’s not working?
“Deployment Environment” secrets are not available in reusable workflows
Job Log Output
...
MY_SECRET:
...
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 41
- Comments: 41 (2 by maintainers)
Commits related to this issue
- https://github.com/actions/runner/issues/1490 — committed to slitviachenko/aio-template-submission by slitviachenko 2 years ago
- https://github.com/actions/runner/issues/1490#issuecomment-1203004481 — committed to achingono/app-templates-microservices-integration by achingono 2 years ago
- [GITHUB][ACTION] Add some secrets Reference: https://github.com/actions/runner/issues/1490 — committed to zuston/lizi by zuston 2 years ago
- [GITHUB][ACTION] Add some secrets Reference: https://github.com/actions/runner/issues/1490 — committed to zuston/lizi by zuston 2 years ago
- [GITHUB][ACTION] Add some secrets Reference: https://github.com/actions/runner/issues/1490 — committed to zuston/lizi by zuston 2 years ago
- Decoupling publishing from release This is done, as re-usable workflows do honer the approvals but do not have access to the secrets of that environment. See <https://github.com/actions/runner/issues... — committed to Shemnei/punktf by Shemnei 2 years ago
- fix(ci): remove env vars from npm build - inject env vars only at runtime - move discord webhook to main workflow https://github.com/actions/runner/issues/1490 — committed to shuttleday/shuttleday by PScoriae a year ago
It would be great if GitHub allows us to use environment with reusable workflows in an intuitive way with good documentation.
Can this please be re-opened? This is just extremely inefficient and resulted in me having to move our deployment steps into the release steps workflow file which is unpleasant to look at and couples our entire release pipeline too closely together from a structural point of view.
How Github thought this was acceptable is beyond me.
When I say “Environment Variables” I mean this feature:
“Environment Variables” are loaded without passing them in the caller workflow.
Different behaviour for the same thing is kind weird. For example:
Considering:
myenv
with a Variable calledMY_VARIABLE
with value equal totest
This is the output:
As you can see, I did not pass Environment “Variables” (vars) from the workflow caller and yet they were loaded in the reusable workflow. Why can’t the Environment “Secrets” (secrets) behave the same way?
That’s not adding anything to the conversation, and quite frankly, is not helpful in the slightest. That’s the point of opening Issues in the first place. @austinsasko
It’s already established that GitHub implemented that by design, as that’s what our quarrel is with. Thus why we opened this Issue in the first place.
We opened the issue because:
As a result, you are left with the only option of including each deployment job within the first workflow file which:
For TL;DR: We need to protect workflows, and their respective secrets while allowing for templates to avoid copy pasta level engineering practices.
I don’t see it mentioned anywhere, so to add context on this, GitHub implemented this by design and this is identical to the behavior for actions. The called workflow/action can never access environmental things like secrets because actions and reusable workflows are intended to be decoupled from the execution environment. It must be passed into the workflow/action from the callee workflow environment
Although, I do agree it would be much more ideal and developer-centric if this capability was an option rather than a mandate, I wanted to add context in case it helped anyone understand the reasoning for that.
See here: https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow
@samoht9277
Not sure if this is too late, but may be useful for anyone else struggling.
I’ve run into this and using
secrets: inherit
as mentioned above sort of solves the issue. You can’t defineenvironment
on the caller workflow as such but you can pass it down as aninput
.The reusable workflow needs to have an input to receive the environment. Then on your job, you can use
environment: ${{ inputs.env }}
.On the caller workflow, pass down the
environment
and declaresecrets: inherits
like this:The only thing I’m not clear on is if this passes the secrets down further into third-party workflows so treat with caution.
https://github.com/orgs/community/discussions/25238#discussioncomment-3247035 shows a simplified example that works.
Honestly I thought GitHub actions would offer every useful features from Azure Pipelines and more.
This issue is especially funny when the reusable workflow is in the same repo as the caller workflow 😃
This was not only very unpleasant to come across, it is something practically undiscoverable when going through the docs!
It is closed because their suggestion works. Check this repo I created that show common mistakes people will make because of the way they implemented it. https://github.com/AllanOricil/workflow-template-bug
I closed this because their design makes sense. Developers must choose which secrets they want to pass down to reusable workflows since there could exist 3rd party workflows that could send our secrets to services we are not aware of.
@AllanOricil Shouldn’t be closed. This is a horrible design. Completely restrictive, especially in 2023 when we want to implement basic DRY principles into our DevOps lifecycle. That’s what approvals are for, if anyone’s concerned about using 3rd party workflows with their secrets.
@AllanOricil the way you’ve described it is exactly how it should work. This is a completely absurd limitation in Actions that entirely defeats the purpose of reusable workflows. If a reusable workflow is coupled to identifiers in the calling workflow, why bother writing reusable workflows?
Github, please fix this. If I can opt into passing secrets to a downstream workflow, there is no security issue.
I am very surprised this is not a thing, can we get some updates to this?
I also think It is kind weird to bind an expression that isn’t resolved in the caller workflow in order to make the secret available in the reusable workflow. So, I would prefer to do something like shown below in order to “opt in”.
@brandongallagher1999 I completely agree. I have a workflow that calls another one, and I need to pass a string, that the called workflow then resolves as the
environment: ${{ inputs.ENVIRONMENT
, but when I try to access to environment specific secrets, they don’t appear.@AllanOricil I think the reason is security related. If Opt-out was the behavior, most people would leave the defaults which would open the footprint for malicious 3rd party actions to steal secrets. Manually requiring secret selection to pass requires more of a manual effort to shoot your own foot.
Does
secrets: inherit
solve this issue? Works for me but I’m not using it with Environments.https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsecretsinherit
I think this issue should be reopened. We are hitting the same issue which prevents us from using the “reusable workflows”.
In our case, the problem we have is that the keyword
environment
underjobs.job-name
is not accepted when using theuses:
key. So, our Github configured secrets (in different environments) are not visible to the called yml workflow. We need either a way to define anenvironment
(where secrets are pulled from ) at the workflow level , or to be able to use theenvironment
key within a job that calls a sub-workflow witht heuses:
keyword.@xeroc - Can you please elaborate how you are feeding secrets into
${{secrets*}}
in the reusable workflow? Thanks.This one weirdly resolved it for me despite using environments. So the workflow has no knowledge over the environment I use (other than passing a variable), and the re-usable puts and loads the environment, and apparently feeds the correct secrets into
${{secrets*}}
.@AllanOricil Your solution worked for me but I agree it is very not intuitive. Thanks for finding workaround.
@bswmservices I suggest combining all jobs on a workflow that require approval into 1 massive workflow file. It’s the only solution, unfortunately.
No I tried to use that and I get
But if I’m not requiring that secret I can use it in the resuable workflow just fine.
Hi @nopol10 ,
The problem still exists that we can’t use environments to protect workflows with approval. That was the main issue that we are concerned about.
Ideally, I’d like to call a reusable workflow and also couple it with workflow approval. So that only certain org/repo members can trigger staging/prod deployment workflows.
Shouldn’t this line in your code actually point to
workflow-template-fix-without-required-secret.yml
?Anyway, your fix is what I’ve done and it does not work. I still don’t get any secrets from the environment.
Also: required secret does not throw error for me.
And I have tried multiple cases:
env_name
in my case replaced withENVIRONMENT
input nameon
expression (because others should be - and are - available anyway)Nothing helps at this point for me. The only solution I see is to copy-paste the workflow code instead of using reusable workflows.
EDIT:
OK, it works, I had a bug in my code:
instead of:
This is still a bizarre solution to use. I agree this should work out of the box only by passing the env information and setting it in the reused workflow.
The funny thing here is that the protection rule works, but the environment variables for the given environment are not available 😕