You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/28 18:51:51 UTC

[GitHub] [beam] davidcavazos commented on a change in pull request #11851: [BEAM-10144] Update PipelineOptions snippets for best practices

davidcavazos commented on a change in pull request #11851:
URL: https://github.com/apache/beam/pull/11851#discussion_r432049685



##########
File path: sdks/python/apache_beam/examples/snippets/snippets.py
##########
@@ -226,35 +227,33 @@ def _add_argparse_args(cls, parser):
 
   # [END pipeline_options_define_custom]
 
-  from apache_beam.options.pipeline_options import GoogleCloudOptions
-  from apache_beam.options.pipeline_options import StandardOptions
-
   # [START pipeline_options_dataflow_service]
-  # Create and set your PipelineOptions.
-  options = PipelineOptions(flags=argv)
+  import apache_beam as beam
+  from apache_beam.options.pipeline_options import PipelineOptions
 
+  # Create and set your PipelineOptions.
   # For Cloud execution, specify DataflowRunner and set the Cloud Platform
-  # project, job name, staging file location, temp file location, and region.
-  options.view_as(StandardOptions).runner = 'DataflowRunner'
-  google_cloud_options = options.view_as(GoogleCloudOptions)
-  google_cloud_options.project = 'my-project-id'
-  google_cloud_options.job_name = 'myjob'
-  google_cloud_options.staging_location = 'gs://my-bucket/binaries'
-  google_cloud_options.temp_location = 'gs://my-bucket/temp'
-  google_cloud_options.region = 'us-central1'
+  # project, job name, temporary files location, and region.
+  # For more information about regions, check:
+  # https://cloud.google.com/dataflow/docs/concepts/regional-endpoints
+  options = PipelineOptions(
+      flags=argv,
+      runner='DataflowRunner',
+      project='my-project-id',
+      job_name='unique-job-name',
+      temp_location='gs://my-bucket/temp',
+      region='us-central1')
 
   # Create the Pipeline with the specified options.
-  p = Pipeline(options=options)
+  # with beam.Pipeline(options=options) as pipeline:

Review comment:
       This is commented out because if we leave it uncommented, even if it doesn't do anything, it makes the test fail with an error.
   
   ```
   subprocess.CalledProcessError: Command '['/Users/dcavazos/src/beam/env/bin/python', '-m', 'pip', 'download', '--dest', '/var/folders/z2/zp_k4l5n2cq84fsn4y633mg400dsyy/T/tmpdv09ddqk', 'apache-beam==2.22.0.dev0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1.
   
   Pip install failed for package: apache-beam==2.22.0.dev0           
   Output from execution of subprocess: b''
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org