You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Kamil Bregula (JIRA)" <ji...@apache.org> on 2019/06/11 17:03:00 UTC
[jira] [Updated] (AIRFLOW-4771) Initialization of hook in the GCP
operator constructors
[ https://issues.apache.org/jira/browse/AIRFLOW-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kamil Bregula updated AIRFLOW-4771:
-----------------------------------
Description:
Hello,
This affects following operators:
* BigtableInstanceCreateOperator
* BigtableInstanceDeleteOperator
* BigtableTableCreateOperator
* BigtableTableDeleteOperator
* BigtableClusterUpdateOperator
* BigtableTableWaitForReplicationSensor
* GceBaseOperator
* GcfFunctionDeployOperator
* GcfFunctionDeleteOperator
* CloudSpannerInstanceDeployOperator
* CloudSpannerInstanceDeleteOperator
* CloudSpannerInstanceDatabaseQueryOperator
* CloudSpannerInstanceDatabaseDeployOperator
* CloudSpannerInstanceDatabaseUpdateOperator
* CloudSpannerInstanceDatabaseDeleteOperator
* CloudSqlBaseOperator
* CloudVisionProductSetCreateOperator
* and more...
This is not good practice for two reasons:
* The gcp_conn_id parameter can not be a Jinja template. Templates are resolved before calling the execute method, but after constrcutor.
* Hook initialization calls the database connection, because In the GoogleCloudBaseHook class, we have a code snippet:
{code:java}
def __init__(self, gcp_conn_id='google_cloud_default', delegate_to=None):
self.gcp_conn_id = gcp_conn_id
self.delegate_to = delegate_to
self.extras = self.get_connection(self.gcp_conn_id).extra_dejson
{code}
self.get_connection triggers a database query.
was:
Hello,
This affects following operators:
* BigtableInstanceCreateOperator
* BigtableInstanceDeleteOperator
* BigtableTableCreateOperator
* BigtableTableDeleteOperator
* BigtableClusterUpdateOperator
* BigtableTableWaitForReplicationSensor
* GceBaseOperator
* GcfFunctionDeployOperator
* GcfFunctionDeleteOperator
* CloudSpannerInstanceDeployOperator
* CloudSpannerInstanceDeleteOperator
* CloudSpannerInstanceDatabaseQueryOperator
* CloudSpannerInstanceDatabaseDeployOperator
* CloudSpannerInstanceDatabaseUpdateOperator
* CloudSpannerInstanceDatabaseDeleteOperator
* CloudSqlBaseOperator
* CloudVisionProductSetCreateOperator
* and more...
This is not good practice for two reasons:
* The gcp_conn_id parameter can not be a Jinja template. Templates are resolved before calling the execute method.
* Hook initialization calls the database connection, because In the GoogleCloudBaseHook class, we have a code snippet:
{code:java}
def __init__(self, gcp_conn_id='google_cloud_default', delegate_to=None):
self.gcp_conn_id = gcp_conn_id
self.delegate_to = delegate_to
self.extras = self.get_connection(self.gcp_conn_id).extra_dejson
{code}
self.get_connection triggers a database query.
> Initialization of hook in the GCP operator constructors
> -------------------------------------------------------
>
> Key: AIRFLOW-4771
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4771
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
> Affects Versions: 1.10.3
> Reporter: Kamil Bregula
> Priority: Major
>
> Hello,
> This affects following operators:
> * BigtableInstanceCreateOperator
> * BigtableInstanceDeleteOperator
> * BigtableTableCreateOperator
> * BigtableTableDeleteOperator
> * BigtableClusterUpdateOperator
> * BigtableTableWaitForReplicationSensor
> * GceBaseOperator
> * GcfFunctionDeployOperator
> * GcfFunctionDeleteOperator
> * CloudSpannerInstanceDeployOperator
> * CloudSpannerInstanceDeleteOperator
> * CloudSpannerInstanceDatabaseQueryOperator
> * CloudSpannerInstanceDatabaseDeployOperator
> * CloudSpannerInstanceDatabaseUpdateOperator
> * CloudSpannerInstanceDatabaseDeleteOperator
> * CloudSqlBaseOperator
> * CloudVisionProductSetCreateOperator
> * and more...
> This is not good practice for two reasons:
> * The gcp_conn_id parameter can not be a Jinja template. Templates are resolved before calling the execute method, but after constrcutor.
> * Hook initialization calls the database connection, because In the GoogleCloudBaseHook class, we have a code snippet:
> {code:java}
> def __init__(self, gcp_conn_id='google_cloud_default', delegate_to=None):
> self.gcp_conn_id = gcp_conn_id
> self.delegate_to = delegate_to
> self.extras = self.get_connection(self.gcp_conn_id).extra_dejson
> {code}
> self.get_connection triggers a database query.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)