[GitHub] [incubator-druid] satybald opened a new issue #8097: Custom Json extractors for Kafka lookup module

[GitBox <gi...@apache.org>]

satybald opened a new issue #8097: Custom Json extractors for Kafka lookup module
URL: https://github.com/apache/incubator-druid/issues/8097
 
 
   **Description**
   Extend [Kafka lookup](https://druid.apache.org/docs/latest/development/extensions-core/kafka-extraction-namespace.html) functionality to use Kafka JSON payloads to extract key/values pair and do simple Jq expression transformation.
   
   **Motivation**
   Kafka log compacted topics are a perfect feature for a lookup data source(aka facts tables). However, the current implementation of Kafka Lookups uses only topic message payload as values and topic key as the key for druid lookup map. This a rigid setup, requires a team to transform existing topics to plain key/value topic.
   
   For teams using Kafka as a data source and druid as an analytics engine, it's a nessary feature to extract values and keys from the message payload as [Global Cached Lookups](https://druid.apache.org/docs/latest/development/extensions-core/lookups-cached-global.html).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org