You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/08/28 15:23:15 UTC

[GitHub] [beam] lukecwik commented on a change in pull request #12708: [BEAM-10670] Update Flink to be opt-out for SplittableDoFn powering the Read transform.

lukecwik commented on a change in pull request #12708:
URL: https://github.com/apache/beam/pull/12708#discussion_r479375163



##########
File path: runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPipelineTranslator.java
##########
@@ -99,6 +99,13 @@ public CompositeBehavior enterCompositeTransform(TransformHierarchy.Node node) {
 
     PTransform<?, ?> transform = node.getTransform();
     if (transform != null) {
+      // TODO(BEAM-10670): Remove this and the ReadTranslator once the "use_deprecated_read"
+      // experiment is removed. Don't translate composite Read transforms since we expect the
+      // primitive expansion containing an SDF to be used.
+      if (PTransformTranslation.READ_TRANSFORM_URN.equals(
+          PTransformTranslation.urnForTransformOrNull(transform))) {
+        return CompositeBehavior.ENTER_TRANSFORM;
+      }

Review comment:
       The non SDF version of Read is always a primitive transform so it gets replaced down below in `visitPrimitiveTransform`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org