You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Song Liu <so...@outlook.com> on 2018/05/09 07:22:00 UTC
About the difference between xcom and variable
Hi,
For the cross-task communication, there are two options:
- xcom
- variable
if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
Any information is appreciated.
Thanks,
Song
Re: About the difference between xcom and variable
Posted by "刘松 (Brain++组)" <li...@megvii.com>.
So if without task_id(s), is it equal with variable ?
On 09/05/2018, 6:26 PM, "Beau Barker" <be...@gmail.com> wrote:
You can still specify task_ids, it's optional.
> On 9 May 2018, at 20:15, 刘松(Brain++组) <li...@megvii.com> wrote:
>
> Since that it might have multiple upstream tasks, so which task to specify? Or you change the xcom syntax to be global?
>
>
>
>
> On Wed, May 9, 2018 at 6:13 PM +0800, "刘松(Brain++组)" <li...@megvii.com>> wrote:
>
> What is the magic to remove the task id ?
>
>
>
>
> On Wed, May 9, 2018 at 3:38 PM +0800, "Beau Barker" <be...@gmail.com>> wrote:
>
>
> Use xcom for communication between tasks.
>
> In the upcoming version 1.10 there's no need to specify task_ids for xcom.
>
> You can simply say:
> self.xcom_pull(context, key='name')
>
>
>> On 9 May 2018, at 17:22, Song Liu wrote:
>>
>> Hi,
>>
>> For the cross-task communication, there are two options:
>> - xcom
>> - variable
>>
>> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>>
>> Any information is appreciated.
>>
>> Thanks,
>> Song
>
Re: About the difference between xcom and variable
Posted by Beau Barker <be...@gmail.com>.
You can still specify task_ids, it's optional.
> On 9 May 2018, at 20:15, 刘松(Brain++组) <li...@megvii.com> wrote:
>
> Since that it might have multiple upstream tasks, so which task to specify? Or you change the xcom syntax to be global?
>
>
>
>
> On Wed, May 9, 2018 at 6:13 PM +0800, "刘松(Brain++组)" <li...@megvii.com>> wrote:
>
> What is the magic to remove the task id ?
>
>
>
>
> On Wed, May 9, 2018 at 3:38 PM +0800, "Beau Barker" <be...@gmail.com>> wrote:
>
>
> Use xcom for communication between tasks.
>
> In the upcoming version 1.10 there's no need to specify task_ids for xcom.
>
> You can simply say:
> self.xcom_pull(context, key='name')
>
>
>> On 9 May 2018, at 17:22, Song Liu wrote:
>>
>> Hi,
>>
>> For the cross-task communication, there are two options:
>> - xcom
>> - variable
>>
>> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>>
>> Any information is appreciated.
>>
>> Thanks,
>> Song
>
Re: About the difference between xcom and variable
Posted by "刘松 (Brain++组)" <li...@megvii.com>.
Since that it might have multiple upstream tasks, so which task to specify? Or you change the xcom syntax to be global?
On Wed, May 9, 2018 at 6:13 PM +0800, "刘松(Brain++组)" <li...@megvii.com>> wrote:
What is the magic to remove the task id ?
On Wed, May 9, 2018 at 3:38 PM +0800, "Beau Barker" <be...@gmail.com>> wrote:
Use xcom for communication between tasks.
In the upcoming version 1.10 there's no need to specify task_ids for xcom.
You can simply say:
self.xcom_pull(context, key='name')
> On 9 May 2018, at 17:22, Song Liu wrote:
>
> Hi,
>
> For the cross-task communication, there are two options:
> - xcom
> - variable
>
> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>
> Any information is appreciated.
>
> Thanks,
> Song
Re: About the difference between xcom and variable
Posted by "刘松 (Brain++组)" <li...@megvii.com>.
What is the magic to remove the task id ?
On Wed, May 9, 2018 at 3:38 PM +0800, "Beau Barker" <be...@gmail.com>> wrote:
Use xcom for communication between tasks.
In the upcoming version 1.10 there's no need to specify task_ids for xcom.
You can simply say:
self.xcom_pull(context, key='name')
> On 9 May 2018, at 17:22, Song Liu wrote:
>
> Hi,
>
> For the cross-task communication, there are two options:
> - xcom
> - variable
>
> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>
> Any information is appreciated.
>
> Thanks,
> Song
Re: About the difference between xcom and variable
Posted by Victor Noagbodji <vn...@amplify-nation.com>.
That's awesome.
> On May 9, 2018, at 3:38 AM, Beau Barker <be...@gmail.com> wrote:
>
> Use xcom for communication between tasks.
>
> In the upcoming version 1.10 there's no need to specify task_ids for xcom.
>
> You can simply say:
> self.xcom_pull(context, key='name')
>
>
>> On 9 May 2018, at 17:22, Song Liu <so...@outlook.com> wrote:
>>
>> Hi,
>>
>> For the cross-task communication, there are two options:
>> - xcom
>> - variable
>>
>> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>>
>> Any information is appreciated.
>>
>> Thanks,
>> Song
Re: About the difference between xcom and variable
Posted by Beau Barker <be...@gmail.com>.
Use xcom for communication between tasks.
In the upcoming version 1.10 there's no need to specify task_ids for xcom.
You can simply say:
self.xcom_pull(context, key='name')
> On 9 May 2018, at 17:22, Song Liu <so...@outlook.com> wrote:
>
> Hi,
>
> For the cross-task communication, there are two options:
> - xcom
> - variable
>
> if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
>
> Any information is appreciated.
>
> Thanks,
> Song
答复: About the difference between xcom and variable
Posted by Song Liu <so...@outlook.com>.
It seems that variable doesn't remove the duplication if have.
Anyway XCom.get / XCom.set looks like more flexible and it meets the requirement.
Thanks,
Song
________________________________
发件人: Song Liu <so...@outlook.com>
发送时间: 2018年5月9日 7:22
收件人: dev@airflow.incubator.apache.org
主题: About the difference between xcom and variable
Hi,
For the cross-task communication, there are two options:
- xcom
- variable
if I have make sure the key of variable is global unique (such uuid), is there any other limitations or cons for using variable instead of xcom ? since that xcom should specify the task_id which is used not very convenient compared with variable.
Any information is appreciated.
Thanks,
Song