logstash-input-jdbc: Errors when using jdbc_default_timezone with logstash 6.6.0
- Version: 6.6.0
- Operating System: CentOS Linux release 7.6.1810 (official docker image)
- Description:
When using jdbc_default_timezone
with logstash-input-jdbc in logstash 6.6.0 an error occurs originating from the sequel
(version 5.16.0) library. In 6.5.4 this error is not present with the same config which uses sequel==5.14.0
. This library is not currently pinned to a specific version in the gemspec. I’m currently trying to figure out how to test downgrading this gem to see if the issue goes away.
- Config File:
input {
jdbc {
jdbc_driver_library => "./mysql-connector-java-5.1.36.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:32771/test"
jdbc_user => "root"
jdbc_password => "my-secret-pw"
schedule => "* * * * *"
statement => 'SELECT t1 FROM test_timestamp'
jdbc_default_timezone => "Etc/UTC"
}
}
output {
stdout {}
}
- Steps to Reproduce:
Start a mysql docker container
docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -p 3306 -d mysql:5.7
Attach into the mysql docker container
docker exec -ti some-mysql sh
Connect to mysql and add some test data with a timestamp
mysql -pmy-secret-pw
create database test;
use test;
CREATE TABLE IF NOT EXISTS test_timestamp (
t1 TIMESTAMP
);
SET time_zone='+00:00';
INSERT INTO test_timestamp
VALUES('2008-01-01 00:00:01');
SELECT t1 FROM test_timestamp;
Run logstash 6.6.0 with config. jdbc_default_timezone
needs to be set to trigger the error.
input {
jdbc {
jdbc_driver_library => "./mysql-connector-java-5.1.36.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:32771/test"
jdbc_user => "root"
jdbc_password => "my-secret-pw"
schedule => "* * * * *"
statement => 'SELECT t1 FROM test_timestamp'
jdbc_default_timezone => "Etc/UTC"
}
}
output {
stdout {}
}
The error:
[2019-01-31T14:53:00,295][INFO ][logstash.inputs.jdbc ] (0.002092s) SELECT t1 FROM test_timestamp
{ 2032 rufus-scheduler intercepted an error:
2032 job:
2032 Rufus::Scheduler::CronJob "* * * * *" {}
2032 error:
2032 2032
2032 Sequel::InvalidValue
2032 NoMethodError: undefined method `utc_total_offset_rational' for #<TZInfo::OffsetTimezonePeriod:0x55b909f8>
Did you mean? utc_total_offset
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/extensions/named_timezones.rb:71:in `convert_input_datetime_other'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/timezones.rb:119:in `convert_input_datetime_no_offset'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/timezones.rb:156:in `convert_input_timestamp'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/timezones.rb:77:in `convert_timestamp'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/database/misc.rb:298:in `to_application_timestamp'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:686:in `timestamp_convert'
2032 org/jruby/RubyMethod.java:119:in `call'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:793:in `block in process_result_set'
2032 org/jruby/RubyArray.java:1734:in `each'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:792:in `process_result_set'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:726:in `block in fetch_rows'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:244:in `block in execute'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:675:in `statement'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:239:in `block in execute'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/connection_pool/threaded.rb:92:in `hold'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/database/connecting.rb:270:in `synchronize'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:238:in `execute'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/dataset/actions.rb:1083:in `execute'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/adapters/jdbc.rb:726:in `fetch_rows'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sequel-5.16.0/lib/sequel/dataset/actions.rb:151:in `each'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:257:in `perform_query'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:229:in `execute_statement'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:277:in `execute_query'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:258:in `block in run'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:234:in `do_call'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:258:in `do_trigger'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:300:in `block in start_work_thread'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:299:in `block in start_work_thread'
2032 org/jruby/RubyKernel.java:1292:in `loop'
2032 /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:289:in `block in start_work_thread'
2032 tz:
2032 ENV['TZ']:
2032 Time.now: 2019-01-31 14:53:00 UTC
2032 scheduler:
2032 object_id: 2012
2032 opts:
2032 {:max_work_threads=>1}
2032 frequency: 0.3
2032 scheduler_lock: #<Rufus::Scheduler::NullLock:0xfbb6a5b>
2032 trigger_lock: #<Rufus::Scheduler::NullLock:0x1100c8>
2032 uptime: 560.731469 (9m20s731)
2032 down?: false
2032 threads: 2
2032 thread: #<Thread:0x16aa34ac>
2032 thread_key: rufus_scheduler_2012
2032 work_threads: 1
2032 active: 1
2032 vacant: 0
2032 max_work_threads: 1
2032 mutexes: {}
2032 jobs: 1
2032 at_jobs: 0
2032 in_jobs: 0
2032 every_jobs: 0
2032 interval_jobs: 0
2032 cron_jobs: 1
2032 running_jobs: 1
2032 work_queue: 0
} 2032 .
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 15 (3 by maintainers)
Just upgraded to 6.7.0 and the error described in the original post is no longer present. The output also seems to be correct for me.