druid: KafkaEmitter for request emitting does not emit "queryType" for native queries
Affected Version
The Druid version where the problem was encountered: 0.22.x, 0.23.x
Description
When setting up a request emitter to use KafkaEmitter the queryType property is not serialized to a JSON. The problem is located in the method: public void emit(final Event event) of KafkaEmitter.java.
The problem is that the rendering of the JSON string is done on the Map<String, Object>. The Jackson ignores the polymorphic annotation on the Query class with queryType if the query is serialized as a part of the map (see: DefaultRequestLogEvent for explanation). If we serialize the query independently then the polymorphic annotation is respected.
Proposed solution
The simples solution is to serialize an object to JsonNode first. If the query is present, then serialize it independently and replace the non-polymorphic version in the JsonNode.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (17 by maintainers)
Commits related to this issue
- #12912 Fix KafkaEmitter not emitting queryType for a native query — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Fix KafkaEmitter not emitting queryType for a native query — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Replace bare Map with EventMap in Event#toMap — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Use @Test instead of TestCase — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Refactor: move test to correct modules, modify EventMap with builder — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Refactor: rename unit test classes — committed to deep-bi/druid by BartMiki 2 years ago
- #12912 Fix KafkaEmitter not emitting queryType for a native query (#12915) Fixes KafkaEmitter not emitting queryType for a native query. The Event to JSON serialization was extracted to the external ... — committed to apache/druid by BartMiki 2 years ago
@rohangarg Thank you for the response! I’m interested in contributing, actually, I already fix the issue on my fork. I will link the PR to this Issue once I finish the unit tests. I like your idea to parse each KV pair individually, I’ll do that.