You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/29 17:11:30 UTC
[GitHub] [beam] robertwb commented on a change in pull request #11452: [BEAM-9692] Move apply_Read to PTransformOverride
robertwb commented on a change in pull request #11452:
URL: https://github.com/apache/beam/pull/11452#discussion_r417471423
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -117,13 +117,15 @@ class DataflowRunner(PipelineRunner):
# TODO: Remove the apache_beam.pipeline dependency in CreatePTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import CombineValuesPTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import CreatePTransformOverride
- from apache_beam.runners.dataflow.ptransform_overrides import ReadPTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import JrhReadPTransformOverride
+ from apache_beam.runners.dataflow.ptransform_overrides import ReadPTransformOverride
+ from apache_beam.runners.dataflow.ptransform_overrides import NativeReadPTransformOverride
- # Thesse overrides should be applied before the proto representation of the
+ # These overrides should be applied before the proto representation of the
# graph is created.
_PTRANSFORM_OVERRIDES = [
- CombineValuesPTransformOverride()
+ CombineValuesPTransformOverride(),
Review comment:
@ananvay we should look at how these interact with auto-opt-in at the service.
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
##########
@@ -594,6 +594,121 @@ def test_combine_values_translation(self):
self.assertIn(
u'CombineValues', set(step[u'kind'] for step in job_dict[u'steps']))
+ def test_read_create_translation(self):
Review comment:
These three feel very redundant; can a common helper method be used here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org