You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Kim Caffeinism (Jira)" <ji...@apache.org> on 2019/10/14 07:25:00 UTC
[jira] [Updated] (AIRFLOW-5654) Make 'OperatorSequence' to flexible operator deps define

     [ https://issues.apache.org/jira/browse/AIRFLOW-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kim Caffeinism updated AIRFLOW-5654:
------------------------------------
    Description: 
USAGE

x, y, z, a is some operators

 

seq1 = x >> y

seq2 = z >> seq1 >> a

 

now deps defined like this

[x, z] >> y >> a

But this definition seems more intuitive

z >> x >> y >> a

 

code will be like this

 

class OperatorSequence:
 ....def __init__(self, head, tail):
 ........self.head = head
 ........self.tail = tail

....def __rshift__(self, other):
 ........self >> other
 ........return sequence_operator(self.head, other)

....def __lshift__(self, other):
 ........self << other
 ........return sequence_operator(other, self.tail)

....def __rrshift__(self, other):
 ........other >> self
 ........return sequence_operator(other, self.tail)

....def __rlshift__(self, other):
 ........other << self
 ........return sequence_operator(self.head, other)

 

  was:
USAGE

x, y, z, a is some operators

 

seq1 = x >> y

seq2 = z >> seq1 >> a

 

now deps defined 

[x, z] -> y -> a

But this definition seems more intuitive

z >> x >> y >> a

 

code will be like this

 

class OperatorSequence:
 ....def __init__(self, head, tail):
 ........self.head = head
 ........self.tail = tail
 
 ....def __rshift__(self, other):
 ........self >> other
 ........return sequence_operator(self.head, other)
 
 ....def __lshift__(self, other):
 ........self << other
 ........return sequence_operator(other, self.tail)
 
 ....def __rrshift__(self, other):
 ........other >> self
 ........return sequence_operator(other, self.tail)
 
 ....def __rlshift__(self, other):
 ........other << self
........return sequence_operator(self.head, other)

 


> Make 'OperatorSequence' to flexible operator deps define
> --------------------------------------------------------
>
>                 Key: AIRFLOW-5654
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5654
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: operators
>    Affects Versions: 1.10.5
>            Reporter: Kim Caffeinism
>            Priority: Minor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> USAGE
> x, y, z, a is some operators
>  
> seq1 = x >> y
> seq2 = z >> seq1 >> a
>  
> now deps defined like this
> [x, z] >> y >> a
> But this definition seems more intuitive
> z >> x >> y >> a
>  
> code will be like this
>  
> class OperatorSequence:
>  ....def __init__(self, head, tail):
>  ........self.head = head
>  ........self.tail = tail
> ....def __rshift__(self, other):
>  ........self >> other
>  ........return sequence_operator(self.head, other)
> ....def __lshift__(self, other):
>  ........self << other
>  ........return sequence_operator(other, self.tail)
> ....def __rrshift__(self, other):
>  ........other >> self
>  ........return sequence_operator(other, self.tail)
> ....def __rlshift__(self, other):
>  ........other << self
>  ........return sequence_operator(self.head, other)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)