logstash-output-amazon_es: When run the logstash using the service it fail to send data to aws elasticsearch

Hello

When I use the following command to start logstash and send data to AWS elasticsearch it works:

sudo /opt/logstash/bin/logstash -f /etc/logstash/conf.d/01-logstash.conf

however when I use the following command:

sudo service logstash start

I got some errors like:

{:timestamp=>“2016-06-14T14:17:35.040000+0000”, :message=>“Attempted to send a bulk request to Elasticsearch configured at ‘["https://mhxqojiy.us-west.es.amazonaws.com:443"]’, but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?”, :client_config=>{:hosts=>[“https://jiy.us-west.es.amazonaws.com:443”], :region=>“us-west-1”, :aws_access_key_id=>nil, :aws_secret_access_key=>nil, :transport_options=>{:request=>{:open_timeout=>0, :timeout=>60}, :proxy=>nil}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::AWS, :logger=>nil, :tracer=>nil, :reload_connections=>false, :retry_on_failure=>false, :reload_on_failure=>false, :randomize_hosts=>false, :http=>{:scheme=>“https”, :user=>nil, :password=>nil, :port=>443}}, :error_message=>“undefined method credentials' for nil:NilClass", :error_class=>"NoMethodError", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.1.36/lib/aws-sdk-core/signers/v4.rb:24:ininitialize’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_v4_signer_impl.rb:36:in signer'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_v4_signer_impl.rb:48:incall’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_transport.rb:49:in perform_request'", "org/jruby/RubyProc.java:281:incall’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.17/lib/elasticsearch/transport/transport/base.rb:257:in perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_transport.rb:45:inperform_request’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.17/lib/elasticsearch/transport/client.rb:128:in perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.17/lib/elasticsearch/api/actions/bulk.rb:88:inbulk’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/http_client.rb:53:in bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:322:insubmit’”, “org/jruby/ext/thread/Mutex.java:149:in synchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:319:insubmit’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:352:in flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:219:inbuffer_flush’”, “org/jruby/RubyHash.java:1342:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:216:inbuffer_flush’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:159:in buffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:312:inreceive’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/outputs/base.rb:83:in multi_receive'", "org/jruby/RubyArray.java:1613:ineach’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/outputs/base.rb:83:in multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/output_delegator.rb:130:inworker_multi_receive’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/output_delegator.rb:114:in multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:301:inoutput_batch’”, “org/jruby/RubyHash.java:1342:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:301:inoutput_batch’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:232:in worker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:201:instart_workers’”], :level=>:error}

Version: logstash 2.3.2

Can you guys faced it? thanks!

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 28 (5 by maintainers)

Most upvoted comments

I’m having the same problem using latest Logstash 6.2.4 running on Amazon Linux. When I run logstash from command line it’s working fine. When I start logstash as a service using sudo initctl logstash start logs say service started successfully and no errors are seen but no data is sent to elasticsearch

I had a similar issue. Check the logs from the start to see if there were issues when loading the template. Mine was caused due to the machine clock drifting, You’ll get errors like “Signature not yet current”…