[GitHub] [druid] zrn6640 commented on issue #10252: Kafka indexing from multiple topics into single datasource

[GitBox <gi...@apache.org>]

zrn6640 commented on issue #10252:
URL: https://github.com/apache/druid/issues/10252#issuecomment-671210687


   Background: We need to perform statistical analysis on multiple edge data, and then display it on a large screen (the default is that one edge corresponds to a kafka topic, using druid 0.12 version, so there is no join function), so we want to implement druid The join function of (druid splits the data of multiple kafka topics through indicators (using the extension written by itself to split inside druid), and puts part of the data into a new druid dataSource, but it will also store it in their respective druid dataSource in)
   Some of the fields that need to be parsed on the edge (common in kafka topic data) have specific encoding rules, and then split them into new fields and put them in the public druid dataSource
   This is actually: multiple kafka topics correspond to multiple druid datasources, and now one kafka topic to multiple druid datasources have been implemented, and multiple kafka topics to one druid datasource are now discussed.
   At present, the company hopes to solve it in druid. If we use your solution, it will cause Kafka to store too much data.
   If you have any good ideas, you can bring them up and discuss them together


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org