sensu-puppet: Could not execute posix command: No such file or directory -

Description of problem

when puppet runs in background it throws an error in the syslog, when it’s being run from the CLI this error doesn’t show.

This is the error that I have observed the logs:

Could not execute posix command: No such file or directory -

The error appears on a number of machines (not all) and I’m not able to state the difference between these VMs. I am not using agent_entity_config_provider, in any case (this is important, see below) hence, it should default to API.
I enabled trace and debug to and I found out that the problem was caused by a provider in the Sensu module, attempting to run sensuctl. The sensu provider is this one:

#  ll /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensuctl.rb
-rw-r--r-- 1 root root 6204 Jan 28 00:02 /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensuctl.rb

Trace output:

Feb 05 17:55:22 repositories02 puppet-agent[4673]: Could not execute posix command: No such file or directory -
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:360:in `exec'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:360:in `block (2 levels) in execute_posix'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:128:in `withenv'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:359:in `block in execute_posix'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:527:in `block in safe_posix_fork'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:508:in `fork'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:508:in `safe_posix_fork'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:323:in `execute_posix'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:217:in `execute'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider.rb:106:in `execute'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensuctl.rb:74:in `sensuctl'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensuctl.rb:94:in `sensuctl_list'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensu_namespace/sensuctl.rb:13:in `instances'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/cache/lib/puppet/provider/sensu_namespace/sensuctl.rb:25:in `prefetch'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:378:in `prefetch'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:260:in `prefetch_if_necessary'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:115:in `block in evaluate'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:120:in `traverse'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:178:in `evaluate'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `block (2 levels) in apply'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:237:in `block in apply'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:161:in `with_destination'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:236:in `apply'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:193:in `block (2 levels) in apply_catalog'
Feb 05 17:55:22 repositories02 puppet-agent[4673]: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 15 (9 by maintainers)

Most upvoted comments

Those thousands of resources would all be defined on the backend, and the way this module works is the backend hosts always have sensuctl. The slow down with namespace validation is when a single catalog contains thousands of resources, not when thousand of agents each validate a namespace for a resource. The way this module is designed is most sites should be defining all resources on the backend host, and agents only define sensu_agent_entity_config resources.

treydock on Feb 7, 2021