[GitHub] [incubator-doris] JNSimba edited a comment on pull request #7210: [feat](flink-connector) add BATCH DELETE support

[GitBox <gi...@apache.org>]

JNSimba edited a comment on pull request #7210:
URL: https://github.com/apache/incubator-doris/pull/7210#issuecomment-979966681


   > @morningman @hf200012 首先非常感谢之前的各种反馈.
   > 
   > 在学习理解doris-flink的过程中, 产生了一些不太成熟的想法, 希望能得到两位的意见和建议, 再次表示感谢.
   > 
   > 想法如下:
   > 
   > 1. Maven ArtifactId: 是否可以修改为flink-connector-doris, 符合Flink社区的命名风格
   > 2. Sink: StreamLoad目前支持CSV和JSON两种数据格式, Sink已经实现了针对Flink数据类型RowData的处理,转换为(1)JSON或者(2)\t列分割\n行分割两种数据格式
   >    为了能提供更加易于使用和理解的API以及后续代码逻辑的维护, 是否可以调整为针对RowData数据类型的转换只支持一种(1)或者(2)[可扩展性建议(1), 性能建议(2)]
   >    调整以后的API不再需要定义数据格式, Sink内部访问StreamLoad提交数据的时候也不再需要根据定义的数据格式构造请求内容
   > 3. 批量删除: SteamLoad是否有直接置位__DELETE_SIGN__的方式, 通过其他字段标识的方式可能需要使用者额外的增加一个软删除数据列
   >    如果可以直接置位, 针对Flink的Changelog(RowData.RowKind), Sink可以方便实现数据的增加,更新和删除
   > 4. DorisStreamLoad和RestService的关系是什么, 是否有可能将两者合并, 将网络请求和请求响应解析封装.
   
   Thank you so much for your suggestion
   1. The doris-flink naming method refers to the previous doris-spark naming
   2. I'm sorry, I don't understand what you mean very much, do you want to only support json format import?
   3. Batch delete only supports unique model. When merge_type: DELETE, data with the same key will be deleted automatically. For changelog data, adding this request header should be fine.
   4. RestService is a package class for requesting FE, such as reading the execution plan, obtaining the BE list, etc. DorisStreamLoad is the package class used by StreamLoad when writing Doris. I personally think that the distinction between meetings is clearer


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org