You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "César Reyes (JIRA)" <ji...@apache.org> on 2019/06/25 14:04:00 UTC
[jira] [Updated] (AIRFLOW-4849) Add gcp_conn_id parameter to
CloudSqlDatabaseHook class to avaid CloudSqlProxyRunner use the gcp_conn_id
by default
[ https://issues.apache.org/jira/browse/AIRFLOW-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
César Reyes updated AIRFLOW-4849:
---------------------------------
Description:
CloudSqlQueryOperator() class has gcp_conn_id, gcp_cloudsql_conn_id attributes and a object attribute of the CloudSqlDatabaseHook class to manage cloud_sql_proxy but CloudSqlDatabaseHook doesn't passing gcp_conn_id as argument, only use gcp_cloudsql_conn_id. So when the get_sqlproxy_runner() method in CloudSqlDatabaseHook call CloudSqlProxyRunner() class, CloudSqlProxyRunner try to get gcp credential from gcp_cloudsql_conn_id but it hasn't credential parameter and Airflow send the following message:
{{ }}
{{[2019-06-25 09:31:55,381] \{logging_mixin.py:95} INFO - [2019-06-25 09:31:55,381] \{gcp_sql_hook.py:508} INFO - The credentials are not supplied by neither key_path nor keyfile_dict of the gcp connection google_cloud_proxy_conn_XXX. Falling back to default activated account}}
This fix add gcp_coon_id attribute to CloudSqlDatabaseHook class, change parameter passed to CloudSqlProxyRunner from gcp_cloudsql_conn_id to gcp_conn_id and add gcp_conn_id parameter to CloudSqlDatabaseHook when it is called from CloudSqlQueryOperator.
There was other issue about the problem (https://issues.apache.org/jira/browse/AIRFLOW-4557?jql=text%20~%20%22AIRFLOW-4557%22) but the solution was not resolve the error because CloudSqlProxyRunner() need a google cloud platform connection not a google cloud sql connection.
was:
CloudSqlQueryOperator() class has gcp_conn_id, gcp_cloudsql_conn_id attributes and a object attribute of the CloudSqlDatabaseHook class to manage cloud_sql_proxy but CloudSqlDatabaseHook doesn't passing gcp_conn_id as argument, only use gcp_cloudsql_conn_id. So when the get_sqlproxy_runner() method in CloudSqlDatabaseHook call CloudSqlProxyRunner() class, CloudSqlProxyRunner try to get gcp credential from gcp_cloudsql_conn_id but it hasn't credential parameter and Airflow send the following message:
{{ }}
{{[2019-06-25 09:31:55,381] \{logging_mixin.py:95} INFO - [2019-06-25 09:31:55,381] \{gcp_sql_hook.py:508} INFO - The credentials are not supplied by neither key_path nor keyfile_dict of the gcp connection google_cloud_proxy_conn_XXX. Falling back to default activated account}}
This fix add gcp_coon_id attribute to CloudSqlDatabaseHook class, change parameter passed to CloudSqlProxyRunner from gcp_cloudsql_conn_id to gcp_conn_id and add gcp_conn_id parameter to CloudSqlDatabaseHook when it is called from CloudSqlQueryOperator
> Add gcp_conn_id parameter to CloudSqlDatabaseHook class to avaid CloudSqlProxyRunner use the gcp_conn_id by default
> -------------------------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-4849
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4849
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, gcp, hooks, operators
> Affects Versions: 1.10.3
> Reporter: César Reyes
> Priority: Major
> Labels: GCP, contrib, hooks, operators
> Fix For: 1.10.4
>
>
> CloudSqlQueryOperator() class has gcp_conn_id, gcp_cloudsql_conn_id attributes and a object attribute of the CloudSqlDatabaseHook class to manage cloud_sql_proxy but CloudSqlDatabaseHook doesn't passing gcp_conn_id as argument, only use gcp_cloudsql_conn_id. So when the get_sqlproxy_runner() method in CloudSqlDatabaseHook call CloudSqlProxyRunner() class, CloudSqlProxyRunner try to get gcp credential from gcp_cloudsql_conn_id but it hasn't credential parameter and Airflow send the following message:
> {{ }}
> {{[2019-06-25 09:31:55,381] \{logging_mixin.py:95} INFO - [2019-06-25 09:31:55,381] \{gcp_sql_hook.py:508} INFO - The credentials are not supplied by neither key_path nor keyfile_dict of the gcp connection google_cloud_proxy_conn_XXX. Falling back to default activated account}}
>
> This fix add gcp_coon_id attribute to CloudSqlDatabaseHook class, change parameter passed to CloudSqlProxyRunner from gcp_cloudsql_conn_id to gcp_conn_id and add gcp_conn_id parameter to CloudSqlDatabaseHook when it is called from CloudSqlQueryOperator.
>
> There was other issue about the problem (https://issues.apache.org/jira/browse/AIRFLOW-4557?jql=text%20~%20%22AIRFLOW-4557%22) but the solution was not resolve the error because CloudSqlProxyRunner() need a google cloud platform connection not a google cloud sql connection.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)