datahub: oracle etl job fetch a error
backend service logs
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:19000
2018-05-03 18:34:20 INFO application:55 - Creating Pool for datasource 'wherehows'
2018-05-03 18:34:21 INFO p.a.d.DefaultDBApi:45 - Database [wherehows] connected at jdbc:mysql://192.168.241.17:3306/wherehows
2018-05-03 18:34:22 INFO application:34 - on start---===
2018-05-03 18:34:22 INFO p.a.Play:97 - Application started (Dev)
2018-05-03 18:34:22 INFO application:58 - Kafka jobs: []
2018-05-03 18:34:22 INFO application:61 - Enabled jobs: [ORACLE_METADATA_ETL]
2018-05-03 18:34:22 INFO application:146 - Terminating KafkaConsumerMaster...
2018-05-03 18:34:22 INFO application:64 - Scheduled jobs: {}
2018-05-03 18:39:22 INFO application:61 - Enabled jobs: [ORACLE_METADATA_ETL]
2018-05-03 18:39:22 INFO application:64 - Scheduled jobs: {ORACLE_METADATA_ETL=1525343700}
2018-05-03 18:39:22 INFO application:82 - Running job: ORACLE_METADATA_ETL
2018-05-03 18:39:22 INFO application:91 - Current running jobs :
2018-05-03 18:39:22 INFO application:98 - Send message : (jobName:ORACLE_METADATA_ETL, whEtlExecId:4)
2018-05-03 18:39:22 DEBUG application:72 - run command : [java, -cp, , -Dconfig=/root/data/wherehows/exec/4.properties, -DCONTEXT=ORACLE_METADATA_ETL, -Dlogback.configurationFile=etl_logback.xml, -DLOG_DIR=/root/data/wherehows, metadata.etl.Launcher] ; timeout: 600
2018-05-03 18:39:22 ERROR application:121 - *** Process + 22569 failed, status: 1
2018-05-03 18:39:22 ERROR application:122 - Error Details:
java.lang.Exception: Process + 22569 failed
at actors.EtlJobActor.onReceive(EtlJobActor.java:123)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
2018-05-03 18:39:22 ERROR application:138 - ETL job (jobName:ORACLE_METADATA_ETL, whEtlExecId:4) got a problem
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
ORACLE_METADATA_ETL.job
# Common ETL configs
job.class=metadata.etl.dataset.oracle.OracleMetadataEtl
job.cron.expr=0 0/5 * * * ? *
job.timeout=600
#job.cmd.params=
#job.disabled=1
job.ref.id=1892
# Oracle JDBC driver
oracle.db.driver=oracle.jdbc.OracleDriver
# Oracle DB name
oracle.db.name=HR
# Oracle username
oracle.db.username=system
# Oracle password
oracle.db.password=oracle
# Oracle JDBC URL
oracle.db.jdbc.url=jdbc:oracle:thin:@192.168.241.21:1521:xe
# Place to store datasets metadata csv file
oracle.metadata=oracle.csv
# Place to store dataset fields csv file
oracle.field_metadata=oracle_field.csv
# Place to store the sample data csv file
oracle.sample_data=oracle_sample.csv
# Whether to get sample data (True/False)
oracle.load_sample=False
# List of excluded databases in oracle
oracle.exclude_db=
# innodb_lock_wait_timeout when accessing MySQL Db
innodb_lock_wait_timeout=1500
wherehows/ORACLE_METADATA_ETL.stderr
Error: Could not find or load main class metadata.etl.Launcher
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 19
@xkcoding I have created a detailed document as to how you can use the docker option end to end. I have it tested. Let me know if you need it.
@ankurgadgilwar Oh, Genius! I so appreciate what you’ve done. The steps are very detailed. But I use Apache Atlas to store metadata and use JDBC way to collect metadata. After I finished this project, I will try your solution by using wherehows. Thanks again!
@ankurgadgilwar if I find a resolution, I will update this issue. 😄