[GitHub] [incubator-doris] stalary commented on issue #4917: Routine load丢失数据

[GitBox <gi...@apache.org>]

stalary commented on issue #4917:
URL: https://github.com/apache/incubator-doris/issues/4917#issuecomment-737182844


   > 如果你用unique 模型的话可能是因为不保证顺序，导致数据被replace 了。
   > routine load如果并发的话不同任务之间的执行是无序的，也就是说你kafka 中的 offset 靠前的数据并不一定先导入。
   > 又由于是unique 模型，后面导入成功的数据会覆盖前面的数据，所以会产生你说的丢数据的错觉。
   > 
   > If you use the unique model, the data may be replaced because the order is not guaranteed.
   > If routine load is concurrent, the execution of different tasks is disordered, which means that the data with the first offset in your Kafka is not necessarily imported first.
   > Also, because it is a unique model, the data that is successfully imported later will overwrite the previous data, so the illusion of data loss will occur.
   
   如果一个routineload的话能保证分区顺序性吗？


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org