camel-kafka-connector: Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.camel.kafkaconnector.CamelSinkConnectorConfig

I am trying to run camel-kafka-connector in my minikube and Strimzi but getting Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.camel.kafkaconnector.CamelSinkConnectorConfig

Here is my image Docker file

FROM strimzi/kafka:0.18.0-kafka-2.5.0
USER root:root
COPY ./my-plugins/ /opt/kafka/plugins/
USER 1001

Here is my plugins folder contains the following jars Screen Shot 2020-05-27 at 10 25 32 PM

When I see the pod logs, this is what I see

2020-05-28 16:17:11,313 INFO Registered loader: PluginClassLoader{pluginLocation=file:/opt/kafka/plugins/camel-kafka-connector-0.2.0.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
2020-05-28 16:17:11,313 INFO Added plugin 'org.apache.camel.kafkaconnector.CamelSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
2020-05-28 16:17:11,313 INFO Added plugin 'org.apache.camel.kafkaconnector.CamelSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
2020-05-28 16:17:11,313 INFO Added plugin 'org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
2020-05-28 16:17:11,313 INFO Added plugin 'org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]

I dont see org.apache.camel.kafkaconnector.CamelSinkConnectorConfig getting loaded from camel-kafka-connector-0.2.0.jar Which I feel would have caused the following error

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.camel.kafkaconnector.CamelSinkConnectorConfig
	at org.apache.camel.kafkaconnector.CamelSinkConnector.config(CamelSinkConnector.java:67)
	at org.apache.kafka.connect.connector.Connector.validate(Connector.java:129)
...

**Full log from connector pod ** logs-from-my-connect-cluster-connect-in-my-connect-cluster-connect-8866c5d89-rd4zs.txt

Any suggestions??

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 30 (16 by maintainers)

Most upvoted comments

@oscard Looking for option to reduce number of files generated in s3 like grouping all messages for a timeframe (say 5 seconds) before writing and generate one s3 file for that timeframe. When we add a hive schema on top of s3 location, query performance reduces with large number is small files.

One more question, is there a way to autocreate a folder in s3 using simple (yymmdd) given in camel.sink.url?

camel.sink.url: aws-s3://selumalai-kafka-s3?keyName=${date:now:yyyyMMdd}/${exchangeId}

On Sat, Jun 13, 2020, 12:37 AM Andrea Cosentino notifications@github.com wrote:

I don’t see how it would be useful. In camel, when you send a message to s3 through a producer it will be written as s3 object directly. In S3 there is no append operation. So I really don’t see why changing the behavior. Also batch operation in what sense? Writing multiple lines on a same file? Or write multiple file in one shot? There is no batch support in S3 sdk v1 as far as I know.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/camel-kafka-connector/issues/251#issuecomment-643573596, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEXOLXRBTDT662O4VHWC5ADRWMGCFANCNFSM4NNIPOPA .

saranyaeu2987 on Jun 13, 2020

We are working on enabling the dynamic resolver with #252

oscerd on May 29, 2020

@saranyaeu2987 can you please try with this Docker file:

FROM strimzi/kafka:0.18.0-kafka-2.5.0
USER root:root
COPY ./my-plugins/ /opt/kafka/plugins/camel-aws-s3/
USER 1001

valdar on May 28, 2020