You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/24 12:22:55 UTC
[GitHub] [airflow] BhuviTheDataGuy opened a new issue #11822: Python Operator - Get xcom_pull from a different task
BhuviTheDataGuy opened a new issue #11822:
URL: https://github.com/apache/airflow/issues/11822
I have a SQL file like below.
```sql
select * from {{params.table_name }} where data > {{ params.maxdt }}
```
And Im calling a function from python operator. So I need to pass maxdt value while calling that python operator.
```py
class SQLTemplatedPythonOperator(PythonOperator):
template_ext = ('.sql',)
table_list = [public.t1]
for table_name in table_list:
pull = SQLTemplatedPythonOperator(
task_id='export_{}'.format(table_name),dag=dag,
templates_dict={'query': 'public.t1.sql'},
params = {'table_name': table_name, 'maxdt': {{ti.xcom_pull(task_ids='push_result_{}')}}.format(table_name)},
op_kwargs={'tablename':table_name},
python_callable=pgexport,
provide_context=True,
)
```
There was a task called `push_result_{}.format(table_name)` do some process and push a data value to **xcom**. Now here I need to get that value and pass it inside my pull task. So my SQL templated query will get the value and pass it to the pgexport function.
**pgexport** - it will use postgrestogcs operator to push the results of the templated SQL query.
Unfortunately, the syntax I used is not working.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] BhuviTheDataGuy closed issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
BhuviTheDataGuy closed issue #11822:
URL: https://github.com/apache/airflow/issues/11822
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #11822:
URL: https://github.com/apache/airflow/issues/11822#issuecomment-715908327
```diff
- params = {'table_name': table_name, 'maxdt': "{{ti.xcom_pull(task_ids='push_result_{}')}}.format(table_name)}",
+ params = {'table_name': table_name, 'maxdt': "{{ti.xcom_pull(task_ids='push_result_{}')}}".format(table_name)},
```
That should fix the problem. In the future, please try to fix the problem yourself or with the help of a #troubleshooting
channel ([](https://s.apache.org/airflow-slack)) before reporting a bug. Github Issues is not intended to teach new people how to use Airflow and share knowledge, but it is used to resolve bugs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] BhuviTheDataGuy commented on issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
BhuviTheDataGuy commented on issue #11822:
URL: https://github.com/apache/airflow/issues/11822#issuecomment-715910204
Hey @mik-laj Sure,
Seems it's not resolving the template instead its passing the string as it is.
```sql
select * from public.m1 where timestamp> { task_instance.xcom_pull(task_ids='push_result_public.m1') }
```
I'll try to get help from the slack.
Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] iam-mhaseeb commented on issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
iam-mhaseeb commented on issue #11822:
URL: https://github.com/apache/airflow/issues/11822#issuecomment-716425594
@BhuviTheDataGuy It should, you can take a look at this too (https://stackoverflow.com/a/46061162/6042903).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #11822:
URL: https://github.com/apache/airflow/issues/11822#issuecomment-716741770
AFAIK `params` is not a template_field so you can't have jinja templated inside `params` argument
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] BhuviTheDataGuy commented on issue #11822: Python Operator - Get xcom_pull from a different task
Posted by GitBox <gi...@apache.org>.
BhuviTheDataGuy commented on issue #11822:
URL: https://github.com/apache/airflow/issues/11822#issuecomment-716914390
Yeah got it. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org