[GitHub] [flink-ml] lindong28 commented on a diff in pull request #234: [FLINK-31887] Upgrade Flink version of Flink ML to 1.16.1

["lindong28 (via GitHub)" <gi...@apache.org>]

lindong28 commented on code in PR #234:
URL: https://github.com/apache/flink-ml/pull/234#discussion_r1174844283


##########
flink-ml-iteration/src/test/java/org/apache/flink/iteration/operator/HeadOperatorTest.java:
##########
@@ -318,6 +322,10 @@ public void testPostponeGloballyAlignedEventsAfterSnapshot() throws Exception {
                                                     OperatorUtils.getUniqueSenderId(operatorId, 0)),
                                             0)),
                             new ArrayList<>(harness.getOutput()));
+                    // TODO There might be a potential bug makes the testing hang, which could be

Review Comment:
   The root cause is that `RecordingHeadOperatorFactory.latestHeadOperator` is not properly closed. The issue can be fixed by having the following code in `HeadOperatorTest#createHarnessAndRun`:
   
   ```
   try {
       return runnable.apply(harness);
   } finally {
       RecordingHeadOperatorFactory.latestHeadOperator.close();
   }
   ```
   
   
   



##########
flink-ml-iteration/src/main/java/org/apache/flink/iteration/operator/OperatorUtils.java:
##########
@@ -136,31 +135,20 @@ public static StreamConfig createWrappedOperatorConfig(StreamConfig config, Clas
         wrappedConfig.setTypeSerializerOut(
                 ((IterationRecordSerializer<?>) typeSerializerOut).getInnerSerializer());
 
-        Stream.concat(
-                        config.getChainedOutputs(cl).stream(),
-                        config.getNonChainedOutputs(cl).stream())
+        config.getChainedOutputs(cl)
                 .forEach(
-                        edge -> {
-                            OutputTag<?> outputTag = edge.getOutputTag();
-                            if (outputTag != null) {
-                                TypeSerializer<?> typeSerializerSideOut =
-                                        config.getTypeSerializerSideOut(outputTag, cl);
-                                checkState(
-                                        typeSerializerSideOut instanceof IterationRecordSerializer,
-                                        "The serializer of side output with tag[%s] should be IterationRecordSerializer but it is %s.",
-                                        outputTag,
-                                        typeSerializerSideOut);
-                                wrappedConfig.setTypeSerializerSideOut(
-                                        new OutputTag<>(
-                                                outputTag.getId(),
-                                                ((IterationRecordTypeInfo<?>)
-                                                                outputTag.getTypeInfo())
-                                                        .getInnerTypeInfo()),
-                                        ((IterationRecordSerializer) typeSerializerSideOut)
-                                                .getInnerSerializer());
-                            }
+                        chainedOutput -> {
+                            OutputTag<?> outputTag = chainedOutput.getOutputTag();
+                            setTypeSerializerSideOut(outputTag, config, wrappedConfig, cl);
+                        });
+        config.getOperatorNonChainedOutputs(cl)

Review Comment:
   Would it be simpler to keep the previous style and do something like this:
   
   ```
   Stream.concat(
           config.getChainedOutputs(cl).stream(),
           config.getOperatorNonChainedOutputs(cl).stream())
           .forEach(
                   output -> {
                       OutputTag<?> outputTag = output.getOutputTag();
                       setTypeSerializerSideOut(outputTag, config, wrappedConfig, cl);
                   }
           );
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org