logstash-output-amazon_es: When run the logstash using the service it fail to send data to aws elasticsearch
Hello
When I use the following command to start logstash and send data to AWS elasticsearch it works:
sudo /opt/logstash/bin/logstash -f /etc/logstash/conf.d/01-logstash.conf
however when I use the following command:
sudo service logstash start
I got some errors like:
{:timestamp=>“2016-06-14T14:17:35.040000+0000”, :message=>“Attempted to send a bulk request to Elasticsearch configured at ‘["https://mhxqojiy.us-west.es.amazonaws.com:443"]’, but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?”, :client_config=>{:hosts=>[“https://jiy.us-west.es.amazonaws.com:443”], :region=>“us-west-1”, :aws_access_key_id=>nil, :aws_secret_access_key=>nil, :transport_options=>{:request=>{:open_timeout=>0, :timeout=>60}, :proxy=>nil}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::AWS, :logger=>nil, :tracer=>nil, :reload_connections=>false, :retry_on_failure=>false, :reload_on_failure=>false, :randomize_hosts=>false, :http=>{:scheme=>“https”, :user=>nil, :password=>nil, :port=>443}}, :error_message=>“undefined method
credentials' for nil:NilClass", :error_class=>"NoMethodError", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.1.36/lib/aws-sdk-core/signers/v4.rb:24:ininitialize’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_v4_signer_impl.rb:36:insigner'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_v4_signer_impl.rb:48:incall’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:inbuild_response'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_transport.rb:49:inperform_request'", "org/jruby/RubyProc.java:281:incall’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.17/lib/elasticsearch/transport/transport/base.rb:257:inperform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/aws_transport.rb:45:inperform_request’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.17/lib/elasticsearch/transport/client.rb:128:inperform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.17/lib/elasticsearch/api/actions/bulk.rb:88:inbulk’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es/http_client.rb:53:inbulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:322:insubmit’”, “org/jruby/ext/thread/Mutex.java:149:insynchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:319:insubmit’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:352:inflush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:219:inbuffer_flush’”, “org/jruby/RubyHash.java:1342:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:216:inbuffer_flush’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:159:inbuffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-amazon_es-0.3-java/lib/logstash/outputs/amazon_es.rb:312:inreceive’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/outputs/base.rb:83:inmulti_receive'", "org/jruby/RubyArray.java:1613:ineach’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/outputs/base.rb:83:inmulti_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/output_delegator.rb:130:inworker_multi_receive’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/output_delegator.rb:114:inmulti_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:301:inoutput_batch’”, “org/jruby/RubyHash.java:1342:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:301:inoutput_batch’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:232:inworker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:201:instart_workers’”], :level=>:error}
Version: logstash 2.3.2
Can you guys faced it? thanks!
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 28 (5 by maintainers)
I’m having the same problem using latest Logstash 6.2.4 running on Amazon Linux. When I run logstash from command line it’s working fine. When I start logstash as a service using
sudo initctl logstash startlogs say service started successfully and no errors are seen but no data is sent to elasticsearchI had a similar issue. Check the logs from the start to see if there were issues when loading the template. Mine was caused due to the machine clock drifting, You’ll get errors like “Signature not yet current”…