You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org> on 2019/05/18 02:46:01 UTC
[jira] [Updated] (AIRFLOW-3385) Airflow DAGs in Kubernetes ask
OAUTH Google Cloud Platform when access PostgreSQL database
[ https://issues.apache.org/jira/browse/AIRFLOW-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aizhamal Nurmamat kyzy updated AIRFLOW-3385:
--------------------------------------------
Labels: kubernetes (was: )
Component/s: (was: kubernetes)
(was: authentication)
operators
Moving to operators component, and labeling with kubernetes as part of the component refactor.
> Airflow DAGs in Kubernetes ask OAUTH Google Cloud Platform when access PostgreSQL database
> ------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3385
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3385
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.8.1
> Reporter: RIo
> Priority: Major
> Labels: kubernetes
>
> I have set and up Airflow in Kubernetes. This Airflow using postgresql database in other pods in same cluster.
> When I try to run some dags that connect to same postgresql with configured option, the connection to postgresql pods ask for OAUTH authorize
> {code:java}
> Reading local file: /root/airflow/logs/anu_funnel_analysis_v2/task_4_get_bigquery_pandas/2018-06-29T02:30:00+00:00/1.log
> [2018-11-23 15:56:22,796] {models.py:1335} INFO - Dependencies all met for <TaskInstance: anu_funnel_analysis_v2.task_4_get_bigquery_pandas 2018-06-29T02:30:00+00:00 [queued]>
> [2018-11-23 15:56:22,820] {models.py:1335} INFO - Dependencies all met for <TaskInstance: anu_funnel_analysis_v2.task_4_get_bigquery_pandas 2018-06-29T02:30:00+00:00 [queued]>
> [2018-11-23 15:56:22,821] {models.py:1547} INFO -
> --------------------------------------------------------------------------------
> Starting attempt 1 of 6
> --------------------------------------------------------------------------------
> [2018-11-23 15:56:22,838] {models.py:1569} INFO - Executing <Task(PythonOperator): task_4_get_bigquery_pandas> on 2018-06-29T02:30:00+00:00
> [2018-11-23 15:56:22,839] {base_task_runner.py:124} INFO - Running: ['bash', '-c', u'airflow run anu_funnel_analysis_v2 task_4_get_bigquery_pandas 2018-06-29T02:30:00+00:00 --job_id 82 --raw -sd DAGS_FOLDER/bigquery_for_funnel.py --cfg_path /tmp/tmpJ1Pe0O']
> [2018-11-23 15:56:23,656] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas [2018-11-23 15:56:23,655] {settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800
> [2018-11-23 15:56:23,817] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas [2018-11-23 15:56:23,816] {__init__.py:51} INFO - Using executor SequentialExecutor
> [2018-11-23 15:56:23,952] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas [2018-11-23 15:56:23,950] {models.py:258} INFO - Filling up the DagBag from /root/airflow/dags/bigquery_for_funnel.py
> [2018-11-23 15:56:24,737] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas /usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/bigquery_operator.py:148: DeprecationWarning: Deprecated parameter `bql` used in Task id: task_2_bq_read_new_table_funnel. Use `sql` parameter instead to pass the sql to be executed. `bql` parameter is deprecated and will be removed in a future version of Airflow.
> [2018-11-23 15:56:24,738] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas category=DeprecationWarning)
> [2018-11-23 15:56:24,778] {base_task_runner.py:107} INFO - Job 82: Subtask task_4_get_bigquery_pandas [2018-11-23 15:56:24,777] {cli.py:492} INFO - Running <TaskInstance: anu_funnel_analysis_v2.task_4_get_bigquery_pandas 2018-06-29T02:30:00+00:00 [running]> on host airflow-5d87576f5b-v5qmg
> [2018-11-23 15:56:24,837] {logging_mixin.py:95} INFO - [2018-11-23 15:56:24,837] {base_hook.py:83} INFO - Using connection to: anudata-postgresql-service
> [2018-11-23 15:56:25,335] {logging_mixin.py:95} INFO - Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=495642085510-k0tmvj2m941jhre2nbqka17vqpjfddtd.apps.googleusercontent.com&xxxxxxxxxxx
> {code}
>
> Why this happen? in the same pods, one ask for OAUTH and other not?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)