[GitHub] [beam] chamikaramj commented on a change in pull request #11466: [BEAM-9787] Clear error message on UW + BQSource

[GitBox <gi...@apache.org>]

chamikaramj commented on a change in pull request #11466:
URL: https://github.com/apache/beam/pull/11466#discussion_r411627605



##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -91,6 +91,12 @@
 _LOGGER = logging.getLogger(__name__)
 
 
+BQ_SOURCE_UW_ERROR = (
+    'The Read(BigQuerySource(...)) transform is not supported with newer stack '
+    'features (Fn API, Unified Worker, etc). Please use the transform '

Review comment:
       s/Unified Worker/Dataflow runner V2 I believe.
   
   cc: @robertwb @ananvay for the correct wording here.

##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -1182,6 +1188,11 @@ def run_Read(self, transform_node, options):
         raise ValueError(
             'BigQuery source is not currently available for use '
             'in streaming pipelines.')
+      debug_options = options.view_as(DebugOptions)
+      use_fn_api = (debug_options.experiments and
+                    'beam_fn_api' in debug_options.experiments)
+      if use_fn_api:

Review comment:
       Should we also check for experiment use_runner_v2 or is this adequate ?

##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
##########
@@ -258,6 +259,18 @@ def test_biqquery_read_streaming_fail(self):
                     PipelineOptions(self.default_properties)) as p:
         _ = p | beam.io.Read(beam.io.BigQuerySource('some.table'))
 
+  def test_biqquery_read_uw_fail(self):

Review comment:
       fnapi_fail (to be consistent with the above check).




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org