salt: Intermittent failure from apt-get locking on 16.04 minion

Description of Issue/Question

All my pkg related states on Ubuntu 16.04 keep failing because of a lock.

Part of my states are to ensure unattended upgrades and other apt services are disabled, but these aren’t working.

Setup

Building an AMI using Packer with a Salt Masterless provisioner (i.e. salt-call)

[snip]
    amazon-ebs:     Function: service.dead
    amazon-ebs:         Name: unattended_upgrades
    amazon-ebs:       Result: True
    amazon-ebs:      Comment: The named service unattended_upgrades is not available
    amazon-ebs:      Started: 15:04:59.987639
    amazon-ebs:     Duration: 14.204 ms
    amazon-ebs:      Changes:
    amazon-ebs: ----------
    amazon-ebs:           ID: unattended_disabled
    amazon-ebs:     Function: service.masked
    amazon-ebs:         Name: unattended_upgrades
    amazon-ebs:       Result: True
    amazon-ebs:      Comment: Service unattended_upgrades was runtime masked
    amazon-ebs:      Started: 15:05:00.002455
    amazon-ebs:     Duration: 82.046 ms
    amazon-ebs:      Changes:
    amazon-ebs:               ----------
    amazon-ebs:               runtime masked:
    amazon-ebs:                   ----------
    amazon-ebs:                   new:
    amazon-ebs:                       True
    amazon-ebs:                   old:
    amazon-ebs:                       False
    amazon-ebs: ----------
    amazon-ebs:           ID: apt_daily_timer_dead
    amazon-ebs:     Function: service.dead
    amazon-ebs:       Result: True
    amazon-ebs:      Comment: The named service apt_daily_timer_dead is not available
    amazon-ebs:      Started: 15:05:00.084770
    amazon-ebs:     Duration: 11.926 ms
    amazon-ebs:      Changes:
    amazon-ebs: ----------
    amazon-ebs:           ID: uptodate
    amazon-ebs:     Function: pkg.uptodate
    amazon-ebs:       Result: False
    amazon-ebs:      Comment: Problem encountered upgrading packages. Additional info follows:
    amazon-ebs:
    amazon-ebs:               result:
    amazon-ebs:                   ----------
    amazon-ebs:                   pid:
    amazon-ebs:                       3344
    amazon-ebs:                   retcode:
    amazon-ebs:                       100
    amazon-ebs:                   stderr:
    amazon-ebs:                       Running scope as unit run-r78484d7012594493b803086a08f96279.scope.
    amazon-ebs:                       E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
    amazon-ebs:                       E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
    amazon-ebs:                   stdout:
    amazon-ebs:      Started: 15:05:00.107993
    amazon-ebs:     Duration: 66362.892 ms
    amazon-ebs:      Changes:
[snip]

Steps to Reproduce Issue

Not clear how to reproduce perfectly, but running my packer AMI it does it fairly consistently now.

Versions Report

    amazon-ebs: Salt Version:
    amazon-ebs:            Salt: 2018.3.0
    amazon-ebs:
    amazon-ebs: Dependency Versions:
    amazon-ebs:            cffi: Not Installed
    amazon-ebs:        cherrypy: Not Installed
    amazon-ebs:        dateutil: 2.4.2
    amazon-ebs:       docker-py: Not Installed
    amazon-ebs:           gitdb: Not Installed
    amazon-ebs:       gitpython: Not Installed
    amazon-ebs:           ioflo: Not Installed
    amazon-ebs:          Jinja2: 2.8
    amazon-ebs:         libgit2: Not Installed
    amazon-ebs:         libnacl: Not Installed
    amazon-ebs:        M2Crypto: Not Installed
    amazon-ebs:            Mako: 1.0.3
    amazon-ebs:    msgpack-pure: Not Installed
    amazon-ebs:  msgpack-python: 0.4.6
    amazon-ebs:    mysql-python: Not Installed
    amazon-ebs:       pycparser: Not Installed
    amazon-ebs:        pycrypto: 2.6.1
    amazon-ebs:    pycryptodome: Not Installed
    amazon-ebs:          pygit2: Not Installed
    amazon-ebs:          Python: 2.7.14 (default, Sep 18 2017, 11:31:52)
    amazon-ebs:    python-gnupg: Not Installed
    amazon-ebs:          PyYAML: 3.11
    amazon-ebs:           PyZMQ: 15.2.0
    amazon-ebs:            RAET: Not Installed
    amazon-ebs:           smmap: Not Installed
    amazon-ebs:         timelib: Not Installed
    amazon-ebs:         Tornado: 4.2.1
    amazon-ebs:             ZMQ: 4.1.4
    amazon-ebs:
    amazon-ebs: System Versions:
    amazon-ebs:            dist: Ubuntu 16.04 xenial
    amazon-ebs:          locale: UTF-8
    amazon-ebs:         machine: x86_64
    amazon-ebs:         release: 4.4.0-1043-aws
    amazon-ebs:          system: Linux
    amazon-ebs:         version: Ubuntu 16.04 xenial

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 29 (10 by maintainers)

Most upvoted comments

I found this thread on Unix StackExchange which I’m going to try to implement by stopping/disabling/killing the apt-daily-. systemd units in the user_data script of the packer VM.

I still think it’d be nicer if we could wait for the locks to release before continuing.

https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image#315517