[GitHub] [inlong] thesumery commented on a diff in pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

[GitBox <gi...@apache.org>]

thesumery commented on code in PR #6308:
URL: https://github.com/apache/inlong/pull/6308#discussion_r1006592055


##########
inlong-sort/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/table/DorisDynamicSchemaOutputFormat.java:
##########
@@ -130,26 +133,28 @@ public void open(int taskNumber, int numTasks) throws IOException {
             this.scheduledFuture = this.scheduler.scheduleWithFixedDelay(() -> {
                 synchronized (DorisDynamicSchemaOutputFormat.this) {
                     if (!closed) {
-                        try {
-                            flush();
-                        } catch (Exception e) {
-                            flushException = e;
-                        }
+                        flush();
                     }
                 }
             }, executionOptions.getBatchIntervalMs(), executionOptions.getBatchIntervalMs(), TimeUnit.MILLISECONDS);
         }
     }
 
-    private void checkFlushException() {
-        if (flushException != null) {
-            throw new RuntimeException("Writing records to streamload failed.", flushException);
+    private boolean checkFlushException(String tableIdentifier) {
+        Exception flushException = flushExceptionMap.get(tableIdentifier);
+        if (flushException == null) {
+            return false;
         }
+        if (!ignoreSingleTableErrors) {
+            throw new RuntimeException(

Review Comment:
   does here need a blacklist mechanism？



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org