You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/08/12 05:14:00 UTC

[jira] [Commented] (AIRFLOW-5165) Make Dataproc highly available

    [ https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904851#comment-16904851 ] 

ASF GitHub Bot commented on AIRFLOW-5165:
-----------------------------------------

mauliksoneji commented on pull request #5781: [AIRFLOW-5165] make number of dataproc masters configurable
URL: https://github.com/apache/airflow/pull/5781
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   https://issues.apache.org/jira/browse/AIRFLOW-5165
     
   
   ### Description
   
   Configure the number of dataproc masters in DataprocCreateOperator
   
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Make Dataproc highly available
> ------------------------------
>
>                 Key: AIRFLOW-5165
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5165
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: plugins
>    Affects Versions: 1.10.1, 1.10.2, 1.10.3, 1.10.4
>            Reporter: Maulik Soneji
>            Assignee: Maulik Soneji
>            Priority: Trivial
>
> Currently the number of master nodes in dataproc create operator is set to 1.
> There is no way to increase the number of master nodes in dataproc.
> Inorder to have HA dataproc, Google Cloud suggests to have 3 master nodes
> [https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability|https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability]
> The number of masters should be a config in DataprocCreateOperator so that the created dataproc is HA



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)