You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/02 02:17:02 UTC

[GitHub] [airflow] casassg opened a new issue #8052: [AIP-31] Create XComArg model

casassg opened a new issue #8052: [AIP-31] Create XComArg model
URL: https://github.com/apache/airflow/issues/8052
 
 
   **Description**
   
   XComArg is a class that references an XCom entry from an operator. Can be used to explicitly set XCom dependencies and explictly pass results between operators.
   
   _Class attributes:_
   - `operator: BaseOperator`: Origin operator instance.
   - `key: str`: Stores key for XCom value. Defaults to `airflow.models.xcom. XCOM_RETURN_KEY`
   
   _Class methods_
   - `get(context: dict)-> Any`: Resolves XCom value given `operator` and `key`.
   - `__getitem__(key, str) -> XComArg`: Easy method to create new XComArg using the same operator but a different key.
   - `add_downstream_dependent(operator: BaseOperator)->None`: Add an operator as a downstream dependency of `operator`. [Optional] Makes this task simpler.
   
   Proposed implementation: https://github.com/casassg/corrent/blob/master/corrent/xcom_arg.py
   
   **Use case / motivation**
   
   - Explicit way to pass around results between operators. 
   - This object is a reference to an XCom value that has not been created and will need to be resolved in the future. 
   - It can generate new XComArgs that point to the same operator but has a different key by using the key accessor. E.g: output['train'].
   - To do so, it includes as class attributes both the originating operator instance and a key (both needed to pull the XCom).
   - Has a property op  that allows access to the origin operator.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8052: [AIP-31] Create XComArg model

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8052: [AIP-31] Create XComArg model
URL: https://github.com/apache/airflow/issues/8052#issuecomment-607582555
 
 
   Thanks for opening your first issue here! Be sure to follow the issue template!
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] casassg commented on issue #8052: [AIP-31] Create XComArg model

Posted by GitBox <gi...@apache.org>.
casassg commented on issue #8052: [AIP-31] Create XComArg model
URL: https://github.com/apache/airflow/issues/8052#issuecomment-615531173
 
 
   I've been a bit overwhelmed with the work from home situation. Will try to get to this on the weekend or next week. Otherwise, please feel free to take it from me 😄 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] turbaszek commented on issue #8052: [AIP-31] Create XComArg model

Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8052: [AIP-31] Create XComArg model
URL: https://github.com/apache/airflow/issues/8052#issuecomment-615206932
 
 
   Hey @casassg any update on this issue? :)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services