semaphore: Problem: Semaphore doesn't reads `ansible.cfg`
Issue
Here is the error I get (log from web ui):
ERROR! the role 'ping' was not found in /tmp/semaphore/repository_1_1/playbooks/all/roles:/tmp/semaphore/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/tmp/semaphore/repository_1_1/playbooks/all
Here is the ansible.cfg in the root of my ansible repository:
[defaults]
host_key_checking = False
roles_path = roles/all:roles/linux:roles/vmware:roles/windows
[inventory]
enable_plugins = yaml, vmware_vm_inventory
So why Ansible isn’t looking in the ./roles directory ?
It works fine on my local computer with a fresh Ansible setup, but not on my Semaphore server.
I thought it could be because of /tmp/semaphore permissions, as Ansible doesn’t reads config files that are in world writable directories.
Unfortunatly that’s probaly not the cause of this issue:
e825aa11cc8d:~$ ls -l /tmp/
total 4
drwxr-xr-x 1 semaphor root 4096 Jun 3 08:19 semaphore
Impact
Ansible (task execution)
Installation method
Docker
Database
MySQL
Browser
No response
Semaphore Version
develop-80e2c8e-1717270566
Ansible Version
No response
Logs & errors
root@semaphore:~# docker compose logs semaphore
semaphore-1 | Attempting to connect to database semaphore on semaphore-mariadb:3306 with user semaphore ...
semaphore-1 | No additional system dependencies to install
semaphore-1 | Installing additional python dependencies
semaphore-1 | ..Collecting git+https://github.com/vmware/vsphere-automation-sdk-python.git (from -r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Cloning https://github.com/vmware/vsphere-automation-sdk-python.git to /tmp/pip-req-build-qkf8dkrl
semaphore-1 | Running command git clone --filter=blob:none --quiet https://github.com/vmware/vsphere-automation-sdk-python.git /tmp/pip-req-build-qkf8dkrl
semaphore-1 | Resolved https://github.com/vmware/vsphere-automation-sdk-python.git to commit 99ee630cb819495f572e9775b48ce0ac1b0263de
semaphore-1 | Installing build dependencies: started
semaphore-1 | Installing build dependencies: finished with status 'done'
semaphore-1 | Getting requirements to build wheel: started
semaphore-1 | Getting requirements to build wheel: finished with status 'done'
semaphore-1 | Installing backend dependencies: started
semaphore-1 | Installing backend dependencies: finished with status 'done'
semaphore-1 | Preparing metadata (pyproject.toml): started
semaphore-1 | Preparing metadata (pyproject.toml): finished with status 'done'
semaphore-1 | Collecting pyvmomi (from -r /etc/semaphore/requirements.txt (line 1))
semaphore-1 | Downloading pyvmomi-8.0.2.0.1.tar.gz (852 kB)
semaphore-1 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 852.2/852.2 kB 7.9 MB/s eta 0:00:00
semaphore-1 | Installing build dependencies: started
semaphore-1 | Installing build dependencies: finished with status 'done'
semaphore-1 | Getting requirements to build wheel: started
semaphore-1 | Getting requirements to build wheel: finished with status 'done'
semaphore-1 | Installing backend dependencies: started
semaphore-1 | Installing backend dependencies: finished with status 'done'
semaphore-1 | Preparing metadata (pyproject.toml): started
semaphore-1 | Preparing metadata (pyproject.toml): finished with status 'done'
semaphore-1 | Collecting pywinrm[credssp] (from -r /etc/semaphore/requirements.txt (line 2))
semaphore-1 | Downloading pywinrm-0.4.3-py2.py3-none-any.whl.metadata (11 kB)
semaphore-1 | Requirement already satisfied: six>=1.7.3 in /opt/semaphore/venv/lib/python3.11/site-packages (from pyvmomi->-r /etc/semaphore/requirements.txt (line 1)) (1.16.0)
semaphore-1 | Collecting xmltodict (from pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2))
semaphore-1 | Downloading xmltodict-0.13.0-py2.py3-none-any.whl.metadata (7.7 kB)
semaphore-1 | Requirement already satisfied: requests>=2.9.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (2.32.3)
semaphore-1 | Collecting requests-ntlm>=1.1.0 (from pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2))
semaphore-1 | Downloading requests_ntlm-1.2.0-py3-none-any.whl.metadata (2.4 kB)
semaphore-1 | Collecting requests-credssp>=1.0.0 (from pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2))
semaphore-1 | Downloading requests_credssp-2.0.0-py2.py3-none-any.whl.metadata (10 kB)
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/vapi-runtime/vapi_runtime-2.44.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/vcenter-bindings/vcenter_bindings-4.2.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/vapi-common-client/vapi_common_client-2.44.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/vmwarecloud-aws/vmwarecloud_aws-1.64.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/nsx-python-sdk/nsx_python_sdk-4.1.2.0.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/nsx-policy-python-sdk/nsx_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/nsx-vmc-policy-python-sdk/nsx_vmc_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/nsx-vmc-aws-integration-python-sdk/nsx_vmc_aws_integration_python_sdk-4.1.2.0.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Processing //tmp/pip-req-build-qkf8dkrl/lib/vmwarecloud-draas/vmwarecloud_draas-1.23.0-py2.py3-none-any.whl (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Collecting lxml>=4.3.0 (from vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Downloading lxml-5.2.2-cp311-cp311-musllinux_1_2_x86_64.whl.metadata (3.4 kB)
semaphore-1 | Requirement already satisfied: charset-normalizer<4,>=2 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests>=2.9.1->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (3.3.2)
semaphore-1 | Requirement already satisfied: idna<4,>=2.5 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests>=2.9.1->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (3.7)
semaphore-1 | Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests>=2.9.1->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (2.2.1)
semaphore-1 | Requirement already satisfied: certifi>=2017.4.17 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests>=2.9.1->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (2024.2.2)
semaphore-1 | Requirement already satisfied: cryptography in /opt/semaphore/venv/lib/python3.11/site-packages (from requests-credssp>=1.0.0->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (42.0.7)
semaphore-1 | Collecting pyspnego>=0.5.0 (from requests-credssp>=1.0.0->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2))
semaphore-1 | Downloading pyspnego-0.10.2-py3-none-any.whl.metadata (5.4 kB)
semaphore-1 | Collecting pyOpenSSL (from vapi-runtime@ file://localhost//tmp/pip-req-build-qkf8dkrl/lib/vapi-runtime/vapi_runtime-2.44.0-py2.py3-none-any.whl->vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3))
semaphore-1 | Downloading pyOpenSSL-24.1.0-py3-none-any.whl.metadata (12 kB)
semaphore-1 | Requirement already satisfied: setuptools in /opt/semaphore/venv/lib/python3.11/site-packages (from vcenter-bindings@ file://localhost//tmp/pip-req-build-qkf8dkrl/lib/vcenter-bindings/vcenter_bindings-4.2.0-py2.py3-none-any.whl->vsphere-automation-sdk==1.86.0->-r /etc/semaphore/requirements.txt (line 3)) (65.5.0)
semaphore-1 | Requirement already satisfied: cffi>=1.12 in /opt/semaphore/venv/lib/python3.11/site-packages (from cryptography->requests-credssp>=1.0.0->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (1.16.0)
semaphore-1 | Requirement already satisfied: pycparser in /opt/semaphore/venv/lib/python3.11/site-packages (from cffi>=1.12->cryptography->requests-credssp>=1.0.0->pywinrm[credssp]->-r /etc/semaphore/requirements.txt (line 2)) (2.22)
semaphore-1 | Downloading lxml-5.2.2-cp311-cp311-musllinux_1_2_x86_64.whl (5.0 MB)
semaphore-1 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 56.7 MB/s eta 0:00:00
semaphore-1 | Downloading requests_credssp-2.0.0-py2.py3-none-any.whl (10 kB)
semaphore-1 | Downloading requests_ntlm-1.2.0-py3-none-any.whl (6.0 kB)
semaphore-1 | Downloading pywinrm-0.4.3-py2.py3-none-any.whl (44 kB)
semaphore-1 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.1/44.1 kB 7.5 MB/s eta 0:00:00
semaphore-1 | Downloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)
semaphore-1 | Downloading pyspnego-0.10.2-py3-none-any.whl (129 kB)
semaphore-1 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.9/129.9 kB 23.7 MB/s eta 0:00:00
semaphore-1 | Downloading pyOpenSSL-24.1.0-py3-none-any.whl (56 kB)
semaphore-1 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.9/56.9 kB 10.1 MB/s eta 0:00:00
semaphore-1 | Building wheels for collected packages: pyvmomi, vsphere-automation-sdk
semaphore-1 | Building wheel for pyvmomi (pyproject.toml): started
semaphore-1 | Building wheel for pyvmomi (pyproject.toml): finished with status 'done'
semaphore-1 | Created wheel for pyvmomi: filename=pyvmomi-8.0.2.0.1-py2.py3-none-any.whl size=527673 sha256=26909db37b6590ec724f35810362a9349bce46f701ece146fd4788ca0f9f837f
semaphore-1 | Stored in directory: /home/semaphore/.cache/pip/wheels/58/00/35/035f172707b539c0edd6e086053a4f177a424c0ab807799d1f
semaphore-1 | Building wheel for vsphere-automation-sdk (pyproject.toml): started
semaphore-1 | Building wheel for vsphere-automation-sdk (pyproject.toml): finished with status 'done'
semaphore-1 | Created wheel for vsphere-automation-sdk: filename=vsphere_automation_sdk-1.86.0-py3-none-any.whl size=2324 sha256=991228ee7d54835b18dcec5d2b702f9dcfc79df8efbcd609e2b75dda30bc37d5
semaphore-1 | Stored in directory: /tmp/pip-ephem-wheel-cache-8fn28ahv/wheels/32/3f/75/62d9c317cd99d4f8b7abfe4e2ac50658e57b7cc1984bb0bc28
semaphore-1 | Successfully built pyvmomi vsphere-automation-sdk
semaphore-1 | Installing collected packages: xmltodict, pyvmomi, lxml, pyspnego, pyOpenSSL, vapi-runtime, requests-ntlm, requests-credssp, vcenter-bindings, vapi-common-client, pywinrm, vmwarecloud-draas, vmwarecloud-aws, nsx-vmc-policy-python-sdk, nsx-vmc-aws-integration-python-sdk, nsx-python-sdk, nsx-policy-python-sdk, vsphere-automation-sdk
semaphore-1 | WARNING: The script pyspnego-parse is installed in '/home/semaphore/.local/bin' which is not on PATH.
semaphore-1 | Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
semaphore-1 | WARNING: The script vapi-server is installed in '/home/semaphore/.local/bin' which is not on PATH.
semaphore-1 | Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
semaphore-1 | Successfully installed lxml-5.2.2 nsx-policy-python-sdk-4.1.2.0.0 nsx-python-sdk-4.1.2.0.0 nsx-vmc-aws-integration-python-sdk-4.1.2.0.0 nsx-vmc-policy-python-sdk-4.1.2.0.0 pyOpenSSL-24.1.0 pyspnego-0.10.2 pyvmomi-8.0.2.0.1 pywinrm-0.4.3 requests-credssp-2.0.0 requests-ntlm-1.2.0 vapi-common-client-2.44.0 vapi-runtime-2.44.0 vcenter-bindings-4.2.0 vmwarecloud-aws-1.64.0 vmwarecloud-draas-1.23.0 vsphere-automation-sdk-1.86.0 xmltodict-0.13.0
semaphore-1 | Starting semaphore server
semaphore-1 | Loading config
semaphore-1 | Validating config
semaphore-1 | MySQL semaphore@semaphore-mariadb:3306 semaphore
semaphore-1 | Tmp Path (projects home) /tmp/semaphore
semaphore-1 | Semaphore develop-80e2c8e-1717270566
semaphore-1 | Interface
semaphore-1 | Port 3000
semaphore-1 | Server is running
semaphore-1 | time="2024-06-03T08:19:47Z" level=info msg="Task 14 added to queue"
semaphore-1 | time="2024-06-03T08:19:49Z" level=info msg="Set resource locker with TaskRunner 14"
semaphore-1 | time="2024-06-03T08:19:49Z" level=info msg="Task 14 removed from queue"
semaphore-1 | time="2024-06-03T08:19:52Z" level=error msg="remove /tmp/semaphore/inventory_14: directory not empty"
semaphore-1 | time="2024-06-03T08:19:52Z" level=info msg="Stopped running TaskRunner 14"
semaphore-1 | time="2024-06-03T08:19:52Z" level=info msg="Release resource locker with TaskRunner 14"
Manual installation - system information
No response
Configuration
compose.yml:
---
services:
# Semaphore
semaphore-mariadb:
restart: unless-stopped
image: mariadb:latest
hostname: mariadb
volumes:
- /srv/semaphore/data:/var/lib/mysql
expose:
- 3306
environment:
MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
MYSQL_DATABASE: 'semaphore'
MYSQL_USER: 'semaphore'
MYSQL_PASSWORD: '<redacted>'
semaphore:
image: semaphoreui/semaphore:develop
restart: unless-stopped
environment:
SEMAPHORE_DB_USER: 'semaphore'
SEMAPHORE_DB_PASS: '<redacted>'
SEMAPHORE_DB_HOST: 'semaphore-mariadb'
SEMAPHORE_DB_PORT: 3306
SEMAPHORE_DB_DIALECT: 'mysql'
SEMAPHORE_DB: 'semaphore'
depends_on:
- semaphore-mariadb
volumes:
- ./config.json:/etc/semaphore/config.json:ro
- ./requirements.txt:/etc/semaphore/requirements.txt:ro
expose:
- 3000
# Caddy reverse proxy
caddy:
image: 'caddy:latest'
restart: unless-stopped
ports:
- 0.0.0.0:80:80
- 0.0.0.0:443:443
- 0.0.0.0:443:443/udp
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- /srv/caddy/data:/data
- /srv/caddy/config:/config
- ./certs:/certs:ro
...
config.json:
{
"port": "3000",
"web_host": "https://semaphore.example.org",
"tmp_path": "/tmp/semaphore/",
"access_key_encryption": "<redacted>",
"cookie_hash": "<redacted>",
"cookie_encryption": "<redacted>",
"ldap_enable": true,
"ldap_needtls": false,
"ldap_binddn": "<redacted>",
"ldap_bindpassword": "<redacted>",
"ldap_server": "<redacted>:389",
"ldap_searchdn": "<redacted>",
"ldap_searchfilter": "(&(mail=%s)(memberOf:1.2.840.113556.1.4.1941:=<redacted>)(objectClass=user))",
"ldap_mappings": {
"dn": "dn",
"mail": "mail",
"uid": "mail",
"cn": "cn"
}
}
requirements.txt:
pyvmomi
pywinrm[credssp]
git+https://github.com/vmware/vsphere-automation-sdk-python.git
Additional information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18
I just updated my SemaphoreUI running as a podman Quadlet, to version 2.10.7 ( https://github.com/semaphoreui/semaphore/releases/tag/v2.10.7 ) and this issue seems to be fixed. I reads my ansible.cfg again. Thank you for fixing this
I’d prefer Semaphore to read the settings from the file so I don’t have to maintain them in one more place.
Also I have far more settings in there, those are just the most important.
I think the solution to
cdsomewhere first would make most sense, maybe a config option likeworking directoryor something? Has been suggested in #1984, too.I am also setting up Semaphore for the first time and testing it out, I am running into the same issue. I have a folder structure where separate internal and external roles and this is set in the ansible.cfg
But each project + playbook is creating new folder in which ansible.cfg is stored. For this solution to work i would need to make new enviroment for each playbook in project and set its path to correct one.
Solution i found so far is to create enviromental variable ANSIBLE_CONFIG = /tmp/semaphore/repository_3_* [SEMAPHORE_PLAYBOOK_PATH] + ‘repository_’ +[REPOSITORY NUMBER] + ‘_*’ to get config from any repository that is already checked out.
But this works until you edit something in config - you would need to clear all checked out repositories to make sure it loads config from last commit
I attempted this, but it still ignores it. I Use exec’ed in to the container and checked the contents of /tmp/semaphore/ansible.cfg after volume mounting it. I see my ansible.cfg file there, but Semaphore still ignores it.
Hey guys thank’s for your interest in this issue, I confirm as I said in my first post that my
ansible.cfgis also in the root of my repository, and that it works fine on my laptop.@tboerger thank for your advice, I’m gonna try this as a workaround !
However I agree with @davidwinterstein I think it would be great to manage general Ansible config in a file in the repo, that would be easier to port and maintain in my opinion.
Not directly a solution for loading the ansible.cfg, but all mentioned configs you guys try to apply you should be able to change via environment variables.