[GitHub] [incubator-doris] morningman opened a new pull request #1041: Support kafka routine load

[GitBox <gi...@apache.org>]

morningman opened a new pull request #1041: Support kafka routine load
URL: https://github.com/apache/incubator-doris/pull/1041
 
 
   The background of this pr:
   There are many streaming datasource which were expected to load data into Doris. However, doris only support the stream load at once. User need to develop a external program which s responsible for continually load data from a streaming datasource. 
   
   This pr support the routine load for kafka. User only need to declaim their kafka properties when creating routine load job. The job will continually and automatically load data from kafka. The function is following:
   Step1: Create a routine load job.
   Step2: Pull data from streaming app such as kafka
   Step3: Push data into itself.
   Step4: Loop executes the step2 and 3.
   
   Please read the detail design from [here](https://github.com/apache/incubator-doris/issues/353).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
For additional commands, e-mail: dev-help@doris.apache.org