quarkus: Funqy unable to get event data
Describe the bug
I’m using import io.quarkus.funqy.Funq to create a Serverless function for use with Knative and a Knative Kafka Source. The Funq processes JSON objects.
When I send a request via HTTP POST it works as expected. Example cURL:
curl -X POST -H "Content-Type: application/json; charset=utf-8" -H "ce-id: 03ce2ea0-c3a7-499c-8edd-ac4e31c6536a" -H "ce-type: shots" -H "ce-source: localhost" -H "ce-specversion: 1.0" -d @ce.body.json $URL_ENDPOINT
When the Kafka Source sends the data to the same Funq this error is printed. Could it be that some header is not set correctly by the Kafka source HTTP request?
ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to / failed, error id: 03b90aeb-11c9-41cd-8e01-b9407e83d38f-6: java.lang.RuntimeException: Don't know how to get event data (dataContentType: 'null', javaType: 'functions.Shot').
at io.quarkus.funqy.runtime.bindings.knative.events.HeaderCloudEventImpl.data(HeaderCloudEventImpl.java:185)
at io.quarkus.funqy.runtime.bindings.knative.events.VertxRequestHandler.lambda$processCloudEvent$5(VertxRequestHandler.java:232)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$13.runWith(VertxCoreRecorder.java:543)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
What makes this issue more interesting is that it only started occurring after I updated from Quarkus version 1.8.1.Final to version 2.2.3.Final. The issue is also happening with the latest Quarkus release. If I use Quarkus 1.8.1.Final the Funq is able to deserialise the incoming JSON event, but then I lose access to the CloudEventBuilder.
Expected behavior
Events sent from Kafka Source should be correctly deserialised and processed by my Funq.
Actual behavior
The stack trace above is printed.
How to Reproduce?
- Obtain an OpenShift/Kube cluster with Serverless/Knative installed.
- Enable Eventing and Serving per the docs, making sure to enable the KafkaSource option for Eventing.
Deploy a Knative function:
- kn func create -l quarkus
- kn func build
- kn service create my-service --image $IMAGE_FROM_BUILD
Create a Kafka Source that sinks events to my-service. Pod logs for my-service show the stack trace above.
Output of uname -a or ver
Darwin MacBook-Pro 20.6.0 Darwin Kernel Version 20.6.0
Output of java -version
AdoptOpenJDK-11.0.11
GraalVM version (if different from Java)
N/A
Quarkus version or git rev
2.2.3
Build tool (ie. output of mvnw --version or gradlew --version)
Knative Version: v0.25.1
Additional information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 35 (33 by maintainers)
@matejvasek yes, that header in the Kafka payload does the trick.
@lance I agree with this. It’s a downstream issue, and Quarkus 2.x is taking the right approach by not making an assumption. Perhaps the error message could be improved and the Kafka Source doc could be improved?
@patriot1burke the proper thing to do here, in my opinion, is to make no assumptions about content type. If the CloudEvent has a
datacontenttypeattribute, obviously, that should be used, but if there is nocontent-typeheader and noce-datacontenttypeheader for the CE code to parse, then the CE data should simply be abyte[].@evanshortiss so you were able to fix the component that is pushing events into kafka?
I mean we should change defaulting in Quarkus code not an app. Until then the string/byte[] workaround might work, I didn’t try it however.
@patriot1burke do you think that defaulting to JSON if Content-Type is not present is all right?
with it we should see headers/body and then hopefully reproduce the issue locally
@evanshortiss I would be useful to see what request form kafka looks like. I can create image logging this. Would you mind deploying it into your cluster and used is as a sink?