[GitHub] [inlong] yunqingmoswu commented on a diff in pull request #6383: [INLONG-6382][Sort] Iceberg misses data when the source table has no primary key in multiple sink scenes

[GitBox <gi...@apache.org>]

yunqingmoswu commented on code in PR #6383:
URL: https://github.com/apache/inlong/pull/6383#discussion_r1013660379


##########
inlong-sort/sort-connectors/iceberg/src/main/java/org/apache/inlong/sort/iceberg/sink/multiple/IcebergMultipleStreamWriter.java:
##########
@@ -130,12 +129,6 @@ public void processElement(RecordWithSchema recordWithSchema) throws Exception {
             List<Integer> equalityFieldIds = recordWithSchema.getPrimaryKeys().stream()
                     .map(pk -> recordWithSchema.getSchema().findField(pk).fieldId())
                     .collect(Collectors.toList());
-            // if physical primary key not exist, put all field to logical primary key
-            if (equalityFieldIds.isEmpty()) {

Review Comment:
   It is recommend to add a option 'sink.multiple.handle-strategy-without-pk'(The value maybe ['append'|'all-columns']) to decide what to do, if it is 'append',  all the records will be write use append mode, else use upsert mode by all columns.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org