salt: archive.extracted/file.managed source_hash format is invalid

Description of Issue/Question

When I try to extract archive or use file.managed with source_hash url, state fails. Error returns:

ID: archieve
    Function: archive.extracted
        Name: /tmp/
      Result: False
     Comment: Source hash http://192.168.1.5/storage/test/archive.md5 format is invalid. The supported formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) either a path to a local file containing hashes, or a URI of a remote hash file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift, s3. The hash may also not be of a valid length, the following are supported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (96), sha512 (128).
     Started: 02:29:24.927676
    Duration: 197.943 ms
     Changes:   

Error occurs on linux, on windows works fine

Steps to Reproduce Issue

test_hash.sls:

archieve:
  archive.extracted:
    - name: /tmp/
    - if_missing: /tmp/test
    - source: http://192.168.1.5/storage/test/archive.tar.gz
    - source_hash: http://192.168.1.5/storage/test/archive.md5
    - archive_format: tar
    - options: zx
    - enforce_toplevel: False

/tmp/zaq.conf:
  file.managed:
    - source: http://192.168.1.5/storage/test/zaq.conf
    - source_hash: http://192.168.1.5/storage/test/zaq.md5
    - backup: minion

Create simple archive (archive.tar.gz) and file zaq.conf Create md5sum using:

md5sum archive.tar.gz > archive.md5
md5sum zaq.conf > zaq.md5

Versions Report

Salt Version:
           Salt: 2017.7.2
 
Dependency Versions:
           cffi: 1.5.2
       cherrypy: 3.5.0
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-83-generic
         system: Linux
        version: Ubuntu 16.04 xenial

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 1
  • Comments: 15 (7 by maintainers)

Most upvoted comments

I can reproduce this in 2017.7.2, but not on the head of the 2017.7 branch (which will become 2017.7.3 once we are ready to release it).

It seems that there was a regression, but it has since been fixed.

(git) d6ccf4bb  2017.7 [upstream/2017.7]
erik ~/git/salt/main                                                    [21:29:41]
(⌐■_■) % git checkout v2017.7.2
Note: checking out 'v2017.7.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 80d90307b0... Don't allow path separators in minion ID
(git) 80d90307  v2017.7.2
erik ~/git/salt/main                                                    [21:29:49]
(⌐■_■) % sudo find . -type f -name \*.pyc -delete
(git) 80d90307  v2017.7.2
erik ~/git/salt/main                                                    [21:29:52]
(⌐■_■) % docker run --rm -v $PWD:/testing terminalmage/issues:44086 salt-call state.apply test
[ERROR   ] Source hash http://archive.ubuntu.com/ubuntu/dists/trusty/main/installer-amd64/current/images/SHA256SUMS format is invalid. The supported formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) either a path to a local file containing hashes, or a URI of a remote hash file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift, s3. The hash may also not be of a valid length, the following are supported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (96), sha512 (128).
local:
----------
          ID: /tmp/initrd.gz
    Function: file.managed
      Result: False
     Comment: Source hash http://archive.ubuntu.com/ubuntu/dists/trusty/main/installer-amd64/current/images/SHA256SUMS format is invalid. The supported formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) either a path to a local file containing hashes, or a URI of a remote hash file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift, s3. The hash may also not be of a valid length, the following are supported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (96), sha512 (128).
     Started: 03:30:28.616681
    Duration: 467.443 ms
     Changes:

Summary for local
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time: 467.443 ms
(git) 80d90307  v2017.7.2
erik ~/git/salt/main                                                    [21:30:30]
(⌐■_■) % git checkout 2017.7
Previous HEAD position was 80d90307b0... Don't allow path separators in minion ID
Switched to branch '2017.7'
Your branch is up to date with 'upstream/2017.7'.
(git) d6ccf4bb  2017.7 [upstream/2017.7]
erik ~/git/salt/main                                                    [21:31:09]
(⌐■_■) % docker run --rm -v $PWD:/testing terminalmage/issues:44086 salt-call state.apply test
local:
----------
          ID: /tmp/initrd.gz
    Function: file.managed
      Result: True
     Comment: File /tmp/initrd.gz updated
     Started: 03:31:22.809000
    Duration: 12570.065 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  12.570 s

I am unable to replicate this.

[root@salt html]# dd if=/dev/random of=/usr/share/nginx/html/this.txt bs=4 count=8
8+0 records in
8+0 records out
32 bytes (32 B) copied, 0.000185613 s, 172 kB/s
[root@salt html]# md5sum /usr/share/nginx/html/this.txt > /usr/share/nginx/html/this.md5
[root@salt html]# cat /srv/salt/test.sls
/tmp/this.txt:
  file.managed:
    - source: http://localhost/this.txt
    - source_hash: http://localhost/this.md5
    - backup: minion

[root@salt html]# salt-call --local state.apply test
local:
----------
          ID: /tmp/this.txt
    Function: file.managed
      Result: True
     Comment: File /tmp/this.txt updated
     Started: 15:47:02.221272
    Duration: 27.743 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  27.743 ms

The debug log would be helpful.

Thanks, Daniel