[jira] [Updated] (BEAM-13812) Integrate DataprocClusterManager into Interactive Environment

["Victor Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Victor Chen updated BEAM-13812:
-------------------------------
    Fix Version/s: 2.38.0
                       (was: Not applicable)

> Integrate DataprocClusterManager into Interactive Environment
> -------------------------------------------------------------
>
>                 Key: BEAM-13812
>                 URL: https://issues.apache.org/jira/browse/BEAM-13812
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-py-interactive
>            Reporter: Victor Chen
>            Assignee: Victor Chen
>            Priority: P2
>             Fix For: 2.38.0
>
>          Time Spent: 29h 50m
>  Remaining Estimate: 0h
>
> * Integrates the DataprocClusterManager previously introduced in:
>  ** BEAM-13799 Create an Interactive Beam Dataproc package for users to manage clusters with
>  * Adds code to automatically create a Dataproc cluster configured with Docker and Flink if a user instantiates an InteractiveRunner using underlying_runner='FlinkRunner'
>  * By default, the created Dataproc cluster will be called 'interactive-beam-cluster' and will be cleaned up when the Interactive Environment is interrupted
>  * Introduces the Clusters class in interactive_beam.py, which acts as a user interface that provides the user with the following capabilities:
>  ** Changing the default name of the clusters created by Interactive Beam
>  ** Cleaning up a cluster by providing a corresponding pipeline, an option to force the deletion is provided, in the case where more than one pipeline use the same cluster
>  ** Getting all pipelines that use the provided master_url
>  ** Receiving a description back for either the cluster corresponding to a specific pipeline, or the descriptions of all pipelines



--
This message was sent by Atlassian Jira
(v8.20.1#820001)