[GitHub] [beam] VictorPlusC commented on a change in pull request #17127: [BEAM-14130] Implement Jupyterlab extension for managing Dataproc clusters

[GitBox <gi...@apache.org>]

VictorPlusC commented on a change in pull request #17127:
URL: https://github.com/apache/beam/pull/17127#discussion_r836852439



##########
File path: sdks/python/apache_beam/runners/interactive/interactive_runner.py
##########
@@ -267,9 +267,12 @@ def _get_dataproc_cluster_master_url_if_applicable(
               category=DeprecationWarning)
         project_id = (user_pipeline.options.view_as(GoogleCloudOptions).project)
         region = (user_pipeline.options.view_as(GoogleCloudOptions).region)
-        cluster_name = ie.current_env().clusters.default_cluster_name
-        cluster_metadata = MasterURLIdentifier(
-            project_id=project_id, region=region, cluster_name=cluster_name)
+        if not project_id:
+          cluster_metadata = ie.current_env().clusters.default_cluster_metadata
+        else:
+          cluster_name = ie.current_env().clusters.default_cluster_name
+          cluster_metadata = MasterURLIdentifier(
+              project_id=project_id, region=region, cluster_name=cluster_name)

Review comment:
       This case was added in the scenario the user first uses the JupyterLab extension to select a default cluster, at this point, we update ib.clusters.default_cluster_metadata to be the MasterURLIdentifier that is used by the selected cluster. Afterwards, if they create a new InteractiveRunner(underlying_runner=FlinkRunner()) pipeline that does not have a Flink master_url and also does not have a Google Cloud project specified, we just assign the default Dataproc cluster to that pipeline.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org