salt: [BUG] An extra return was detected from minion

Description Master(s) receives duplicate events from minion when running Salt scheduler task under the following configuration, it shows this error on the master log

[ERROR   ] An extra return was detected from minion minion-win-1, please verify the minion, this could be a replay attack

Minion configured to connect to 2 masters (both masters servers alive and servicing ok) Having configured additional returner on the minion config

master: 
  - 172.21.0.10
  - 172.21.0.11

rawfile_json.filename: 'C:\ProgramData\Salt Project\Salt\var\log\salt\jobs.log'
return:
  - rawfile_json

The Salt scheduler task is set by a state file to use the rawfile_json returner this way

set_salt_sched_task:
  schedule.present:
    - function: state.sls
    - job_args:
      - win_tasks.2_win_task_sched
    - seconds: 3600
    - return_job: True
    - returner: rawfile_json 



salt-call schedule.list
local:
    schedule:
      set_salt_sched_task:
        args:
        - win_tasks.2_win_task_sched
        enabled: true
        function: state.sls
        jid_include: True
        kwargs:
          queue: true
        maxrunning: 1
        name: set_salt_sched_task
        return_job: True
        returner: rawfile_json
        saved: true
        seconds: 3600

Setup Windows Minion 3006.1 Salt Master 3006.1

Steps to Reproduce the behavior Create the schedule task and look for errors in the master log, reduce to seconds: 10 to get more frequent errors

[ERROR   ] An extra return was detected from minion minion-win-1, please verify the minion, this could be a replay attack

It does not happen if minion not set to multimaster configuration or if the option return_job is set to False in the salt scheduler task

Expected behavior ERROR

Screenshots If applicable, add screenshots to help explain your problem.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

The default is 3…and wouldn’t you want a minion to retry if it couldn’t send a return to the master? Adjusting it to 0 seems sorta like a work-around, but with a bad side effect. No?

Happening under 3007.1 as well.

Can you reproduce in 3006.3?

We are also seeing this on 3006.7 in a single master configuration.

3006.8 and 3006.6 are also affected.

We found this problem at our booth. We have up to 10,000 minions on masters.

@all please check your minion’s return_retry_tries config setting. If it is anything other than 0 you are likely seeing this log message because the minion timed out while trying to send the return to the master.

I still encounter An extra return was detected from minion minion_id please verify the minion, this could be a replay attack, server is running single master 3006.7 and minion is mixed of 3006.x still see even some minion is running on 3006.7

[salt.loaded.int.returner.local_cache][ERROR ] An extra return was detected from minion xxx please verify the minion, this could be a replay attack