[GitHub] [flink] fapaul edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation

[GitBox <gi...@apache.org>]

fapaul edited a comment on pull request #18428:
URL: https://github.com/apache/flink/pull/18428#issuecomment-1025891089


   > For the `execute` method, sorry I has some misunderstandings initially that the clear happens before compile happens, but it should indeed be happen after the generate() is called. However, there seems still cases that the transformation is not cleared, like users call `StreamExecutionEnvironment#getExecutionPlan()` ?
   
   Afaict `getExecutionPlan` is only used within tests and it feels to me like it is some part of Flink's legacy and is not used anymore. I will double-check that.
   
   Edit: I found a way to still support the use case. I remove the `SinkTransformation` after the expansion in the translation from the transformation list. This prevents double expansions.
   
   > For the Batch exchange, I think perhaps with branches (namely pass the runtime mode to SinkExpander and avoid create the failover region for streaming mode), we could avoid creating the batch exchange in advance for streaming mode, thus we won't need to rely on the behavior of the blocking edge under streaming mode~?
   
   Your suggestion is definitely possible but it does not work for exchanges in the sink topology provided by the user. In this case, I would need to move the current logic from `PartitionTransformationTranslator` to `SinkTransformationTranslator`. Perhaps it is a more general issue and the fix should reside in `PartitionTransformationTranslator` rather than only in the sink translation.
   Is there a downside when adding it to the `PartitionTransformationTranslator`?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org